Applied Computational
Economics Workshop
Overview
Introduction
Gaussian elimination
Rounding error
Introduction
An n-dimensional linear equation takes the form
where
is a known
matrix
is a known
vector
Introduction
Many models in economics are naturally linear, e.g. linear
multi-commodity market equilibrium models, ordinary
least squares, etc.
However, we are primarily interested in linear equations
here because the methods we will employ to solve the
functional equations underlying dynamic models will
require us to solve very large linear equations repeatedly
Execution speed, storage requirements, and rounding
error associated with solving linear equations are thus
important practical issues
Gaussian Elimination
A linear equation may be solved using Gaussian
elimination
Gaussian elimination employs elementary row operations
Interchange two rows
Multiply a row by a nonzero constant
Add a nonzero multiple of one row to another
Elementary row operations alter the form of a linear
equation without changing its solution
Gaussian Elimination
Gaussian elimination is the most efficient known method for
solving a general -dimensional linear equation
=
For large , Gaussian elimination requires about
/3 +
multiplication/division operations
Explicitly computing
requires about
+
operations
Cramer’s rule requires ( + 1)! operations
For = 10, the number of operations are
Gaussian Elimination
430
Explicit Inverse
1,100
Gaussian Elimination
The MATLAB “backslash” operator
\
uses Gaussian
elimination to solve linear equations
For example, to solve the linear equation
1 1 2
3 4 8
2 1 1
=
18
5
6
one may execute the MATLAB code
A = [1 1 2; 3 4 8; 2 1 1];
b = [5; 18; 6];
Gaussian Elimination
These operations return
x =
2.0000
1.0000
1.0000
The backslash operator is the most frequently-used operator
in MATLAB, apart from routine arithmetic operators
The backslash operator detects special structures, such as
positive definiteness of A, and employs faster algorithms, such
as Cholesky factorization, whenever possible
Rounding Error
A computer has finite storage and can represent
only finitely many numbers exactly
Thus, exact arithmetic and computer arithmetic
do not always give the same results
If you attempt to compute a number that can’t
be represented exactly on a computer, the result
will be rounded to the nearest representable
Rounding Error
In particular, when adding or subtracting two numbers of extremely different magnitudes, the smaller number is effectively ignored
For example, in exact arithmetic
( + 1) − 1 = + (1 − 1) =
However, in MATLAB computer arithmetic
e = 1e–20; x = (e+1)–1; y = e+(1–1); will return x = 0 y = 1.0000e–020
Rounding Error
Rounding error can cause problems when solving linear equations
Consider the linear equation 1
1 1 = 12 where = 10
One can easily verify that the exact solution is
= which is slightly more than 1 = which is slightly less than 1
Rounding Error
To solve the linear equation using Gaussian elimination,
add
times the first row to the second row
then solve recursively
/ /
Rounding Error
If you compute
and
in this manner in MATLAB,
e = 1e–17;
x2 = (2–1/e)/(1–1/e)
x1 = (1–x2)/e
the operations return
x2 = 1
x1 = 0
The computed value for
is grossly inaccurate
Rounding Error
In the first step of Gaussian elimination, we computed
= 2 − 1/ 1 − 1/
However, since 1/ is very large compared to 1 or 2, rounding error was introduced, and the computer actually calculated
= −1/ −1/
which evaluated to exactly 1
The computer then calculated
= 1 −
Rounding Error
Now solve the linear equation again by Gaussian elimination, but
first interchange the two rows, which will not affect the solution
1 1
1
= 2
1
Now add − times the first row to the second row
1
1
0 1 −
=
1 − 2
2
then solve recursively
=
Rounding Error
If you compute and in this manner in MATLAB,
e = 1e–17;
x2 = (1–2*e)/(1–e) x1 = 2–x2
the operations return
x2 = 1 x1 = 1
The computed values for and are a little off, but are much more accurate than the first values we computed
Why did interchanging the two rows improve the accuracy of the computed solution?
Rounding Error
The inaccuracy of the first solution was due to rounding error caused by the very small magnitude of the diagonal element
By interchanging the two rows first, we brought a number of much larger magnitude into the diagonal, which reduced rounding error in subsequent computations
Interchanging rows to make the magnitude of the diagonal element as large as possible is called pivoting
Pivoting substantially enhances the computational accuracy of Gaussian elimination
All good linear solvers, including the MATLAB backslash operator, employ pivoting
Ill-Conditioning
Consider the n-dimensional linear equation =
A perturbation in will induce a change in solution
The sensitivity of the solution to perturbations in is measured by the condition number of matrix
The condition number of is the ratio its largest and smallest singular values — too technical
Rule of Thumb
Computed value of loses one significant digit per power of 10 of the condition number of
Ill-Conditioning
If the condition number of is large, small perturbations in
can lead to disproportionately large changes in
When this is the case, we say that is ill-conditioned or nearly
singular
When is ill-conditioned, unavoidable rounding errors in the
representation of make it impossible to compute an
accurate solution to
=
Ill-conditioning is endemic to the matrix and cannot be
corrected with simple tricks such as pivoting
Ill-Conditioning
Consider the notorious Vandermonde matrices
The
Vandermonde matrix has a typical element
Ill-Conditioning
Let us solve the linear equation
where is the
Vandermonde matrix and is the
row-sum of , that is, the
vector with a typical
element
By construction, the exact solution to this linear equation
Ill-Conditioning
In MATLAB, we can solve the linear equation for
as
follows
n = 4;
A = vander(1:n)
b = A*ones(n,1)
x = A\b
error = max(abs(x–1))
Here, we compute the matrix using the special MATLAB
function
vander
and we compute the maximum error
Ill-Conditioning
With
, executing this code returns, as expected,
x = 1
1 1 1 error = 0
With
, however, executing the code returns
Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 3.4e-032. error = 5.4e+011
