Spring Semester ’12-’13 Akila Weerapana
Supplement to Lecture 3: Basics of Linear Algebra
I. INTRODUCTION
• The tools of matrix algebra play a really important role in solving the model and in doing comparative statics. As you will see today, matrices present us with elegant methods of depicting, solving and analyzing linear models. These techniques are so powerful that, as we will see in a later class, we often resort to linear approximations of non-linear models just so we can use the tools of matrix algebra.
• In today’s lecture, we will do a very quick introduction to basic matrix operations such as addition, multiplication, inversion etc. If you have taken Math 206 already, this material will be extremely basic. Even if you have not, I am confident that we can move through the material fairly quickly and get to how the model can be used to solve the IS-MP model.
II. WHAT IS A MATRIX
Matrix
• A matrix is a rectangular array containing numbers or variables. Although hard to describe in words, a matrix can be visualized extremely easily. The following are all matrices.
4 5 6 8 1 3 5 7
4 5 1 3 5 7
x1 x2 x3 x4
x y z
1 4 3
• A matrix is typically described in terms of its dimensions. An m × n (pronounced m by n) matrix is a matrix with mn entries represented in m rows and n columns. So the 5 matrices above are 2 × 4, 3 × 2, 2 × 2, 1 × 3 and 3 × 1 matrices respectively.
• The entries of a matrix are usually referenced according to their location within the matrix.
So in general if we have an m × n matrix known as X, we will refer to the element in row i and column j of X as Xij . The elements that lie on the diagonal line between the first and the last entries, i.e. those elements for which i = j are known as the diagonal elements and the others as the off-diagonal elements.
• A matrix in which the number of rows equals the number of columns is called a square matrix.
Identity Matrix
• The identity matrix, denoted I, is a special matrix. It is a square matrix with with each diagonal entry being 1 and each non-diagonal entry being 0.
• We use the notation Into refer to the dimension of the matrix (since number of rows= number of columns in a square matrix, we only need 1 number to know its dimensions.
• Example
I3=
1 0 0 0 1 0 0 0 1
I2=1 0 0 1
Symmetric Matrix
• A symmetric matrix is another special matrix. A matrix A is said to be symmetric if Aij = Aji ∀i, j
• Given the definition we can see that a matrix that is symmetric must also be square. The converse is not true, not all square matrices are symmetric.
• Example
A =
1 0 3 0 7 5 3 5 8
is symmetric, but B =0 7 3 5
is not
Vector
• A matrix with a single row or column is sometimes referred to as a vector. A matrix with a single column is called a column vector and a matrix with a single row is called a row vector. A 1 × 1 matrix, i.e. a matrix with only 1 entry is called a scalar.
III. BASIC MATRIX OPERATIONS
Transposition
• The transpose of an m × n matrix is an n × m matrix constructed by switching rows and columns of the original matrix. That is to say, elements that were on the first column of the original matrix now make up the first row of the transposed matrix. The transpose of a given matrix X is denoted as X0. We can now simplify the definition of a symmetric matrix A as A is symmetric iff A = A0
4 5 6 8 1 3 5 7
0
=
4 1 5 3 6 5 8 7
4 5 1 3 5 7
0
=4 1 5 5 3 7
x1 x2
x3 x4
0
=x1 x3
x2 x4
x y z
0
=x y z 1 4 30 =
1 4 3
Matrix Addition
• Matrix addition is a simple operation. The result of adding two matrices together is a third matrix whose elements in any given row and column are obtained by adding elements in that same row and column in the first matrix to the elements in the same row and column of the second matrix.
• As a result, matrix addition is legal only when performed with two matrices of identical dimensions, i.e. a 5 × 6 matrix can only be added to another 5 × 6 matrix.
• In short, The result of A + B where A and B are both m × n matrices is another m × n matrix C where cij = aij+ bij.
• Example
4 1 5 1 3 7
+1 1 2 0 3 −5
=5 2 7 1 6 2
• Matrix subtraction is very similar. The result of A - B where A and B are both m×n matrices is another m × n matrix C where cij = aij− bij.
• Example
4 1 5 1 3 7
−1 1 2 0 3 −5
=3 0 3 1 0 12
• Rules of Addition
A + B = B + A (A + B) + C = A + (B + C) Scalar Multiplication
• Any matrix can be multiplied by a scalar. If A is an m×n matrix and q is a scalar, the matrix that results from multiplying A by the scalar q is a m × n matrix C, where each element is obtained by multiplying the corresponding element A by q, i.e. Cij = q × Aij.
• Example
54 1 5 1 3 7
=20 5 25 5 15 35
Matrix Multiplication
• Matrix multiplication is a more complicated basic operation. When we multiply matrix A by matrix B (denoted AB) to obtain matrix C, a given element cij is the sum-product of the ith row of A and the jth column of B. Therefore, multiplication of two matrices is legal only when the ith row of the first matrix has the same number of elements as the jth column of the second matrix: in other words the number of columns in A has to equal the number of rows in B.
• Thus an m × n matrix A can be multiplied by an n × p matrix B where p is any positive integer. The resulting matrix C is of dimension m × p with elements defined as follows:
Cij =
n
X
k=1
(Aik× Bkj) for i = 1 · · · m, j = 1 · · · p
• Suppose A =
1 1 1 3 2 −5
and B =4 1 5 1 3 7
• Examples
AB =
1 1
1 3
2 −5
4 1 5 1 3 7
=
5 4 12
7 10 26
3 −13 −25
BA =4 1 5 1 3 7
1 1
1 3
2 −5
=15 −18 18 −25
• Rules of Multiplication (where A is m × n, B is n × n, C is n × n, D is n × p and E is p × q)
BC 6= CB (AD)E = A(DE)
AIn = A ImA = A (AD)0 = D0A0 Determinant
• The determinant of a square matrix is a unique number associated with that matrix. The determinant of a matrix is usually denoted by writing the name of the matrix surrounded by two vertical bars: e.g. the determinant of A is |A|.
• How is the determinant of a matrix calculated. For a 1 × 1 matrix, a 2 × 2 matrix or a 3 × 3 matrix, the process is fairly simple. For higher dimension matrices, the calculation is more involved.
• The determinant of a 1 × 1 matrix (a scalar) is the scalar itself.
• The determinant of a 2 × 2 matrix of the form A =a b c d
is |A| = (ad − bc)
• The determinant of a 3 × 3 matrix of the form A =
a b c d e f g h i
is
|A| = (aei + bf g + cdh) − (ceg + bdi + af h)
Although this seems hard to remember, there is a simple trick that enables us to calculate the determinant of a 3 × 3 matrix. The trick requires you to write down two copies of the matrix side by side, then add the products of the three leftmost diagonals and subtract the products of the three rightmost diagonals.
• You should not try to find the determinant of a matrix with dimensions greater than n = 3 by hand. The procedure is complicated, and even if you remember the procedure, you will end up making an algebra error. Next week I will show you how to do this using a software program like MATLAB or MATHEMATICA.
• So the method below is purely for information sake. If you are ever tempted to use it, do so only if you are stuck on a desert island without MATHEMATICA and need to invert a matrix to escape.
Finding Determinants Using Laplace Expansion
• The determinant of an n×n matrix is found using a procedure known as a ‘Laplace expansion’.
The Laplace expansion is a recursive process: i.e. the determinant of an n × n matrix is expressed as a function of determinants of several (n − 1) × (n − 1)) matrices, each of which in turn are expressed as functions of several (n − 2) × (n − 2) matrices etc. Since we know how to calculate the determinant of a 3 × 3 matrix or a 2 × 2 matrix we can always build back up to the top.
• In order to write down the Laplace expansion formula, we need to define a few other terms.
The minor of a matrix is the determinant of a submatrix formed from a given matrix. In particular, the (i,j)th minor of a matrix, denoted as |Mij| is the determinant of a matrix obtained by omitting the ith row and jth column of the original matrix.
• Consider the following matrix A =
2 7 0 1
5 6 4 8
0 0 9 0
1 −3 1 4
. Some of the minors of this matrix are
|M11| =
6 4 8
0 9 0
−3 1 4
= 432, |M23| =
2 7 1
0 0 0
1 −3 4
= 0 and |M41| =
7 0 1 6 4 8 0 9 0
= −450.
• The cofactor of a matrix is another determinant of a submatrix formed from the original.
The (i,j)th cofactor of a matrix is defined as |Cij| = (−1)i+j|Mij|, that is to say the co-factor is similar to the minor, except that it may have a different sign. If the sum of i and j is an even number then the sign is positive, if the sum is an odd number, the sign is negative. So
|C11| = |M11|, |C12| = − |M12|, |C23| = − |M23|, |C24| = |M24| etc.
• We are finally ready to write down the formula for the Laplace expansion of a matrix A. If A is an n × n matrix then
|A| =
n
X
j=1
aij|Cij| for any i = 1 · · · n
• In other words, we can find the determinant of a matrix by picking any row, then multiplying each element of the row by the appropriate cofactor.
• Which should you choose? Well its always easier to pick a row with a lot of zeros, since that minimizes the number of calculations one has to do. So in our case, the best thing to do would be to use the expansion of the 3rd row.
|A| = 0 |C31| + 0 |C32| + 9 |C33| + 0 |C34|
= 9 |C33| = 9 |M33|
= 9
2 7 1
5 6 8
1 −3 4
|A| = −81
• As you can see this is a very tedious calculation even for a 4 × 4 matrix. This is why we will use computers or other short-cuts to preserve our sanity.
• Properties of the Determinant
There are several important properties that relate to determinants.
– The determinant of A is equal to the determinant of its transpose |A| = |A0|
– The determinant of a diagonal matrix (i.e. one whose off-diagonal elements are ALL zero) is equal to the product of the element of the diagonals.
– A is invertible iff |A| 6= 0
– If any row of a matrix is a linear combination of one or more of the other rows of the matrix then the matrix has a determinant of zero. The same holds true for columns. So matrices with linearly dependent rows or columns are not invertible.
– If any row of a matrix consists only of zeros then the matrix has a determinant of zero.
IV. CALCULATING THE INVERSE OF A MATRIX
• The inverse of a n-dimensional square matrix A is denoted A−1, if it exists. The inverse matrix has the property that AA−1= In and A−1A = In.
• Any square matrix that has an inverse is said to be a non-singular matrix. A matrix that has no inverse is said to be a singular matrix.
• If a matrix has an inverse matrix, then the matrix must be a square matrix, i.e. only square matrices have inverses. This does not mean that ALL square matrices have inverses, it just means that matrices that are not square do not have an inverse matrix.
• Consider the matrix A=4 5 1 3
, the inverse of this matrix is A−1 = 3/7 −5/7
−1/7 4/7
since AA−1= A−1A = A =1 0
0 1
• Properties of the Inverse
– If A−1 is the inverse of a matrix A, then A−1−1
= A – If A−1 is the inverse of a matrix A, then (A0)−1= (A−1)0
– If A and B are both square matrices with inverse matrices A−1 and B−1. Then the matrix C = AB has an inverse defined as C−1= B−1A−1
• The inverse of a 2×2 matrix is easy to remember and requires 3 steps: a) switch the diagonals;
b) change the sign of the off-diagonals; scalar-divide by the determinant.
• Going back to A=4 5 1 3
with
A =
4 5 1 3
= 7, we can apply the three step process.
• Step 1: Switch the diagonals 3 5 1 4
• Step 2: Change sign of off-diagonals 3 −5
−1 4
• Step 3: Scalar-divide by determinant 3/7 −5/7
−1/7 4/7
to obtain A−1= 3/7 −5/7
−1/7 4/7
• You can verify that AA−1= A−1A = A =1 0 0 1
• Calculating the inverse of a matrix of n > 2 is a chore. Again we will typically use MATLAB or MATHEMATICA to do this inversion. The information below is for reference, not for everyday use.
Finding The Inverse of a Matrix
• First, we have to define something called the adjoint of the matrix.
• The adjoint of an n × n matrix A is another n × n matrix defined as
adj(A) = C0 where C =
|C11| |C12| · · · |C1n|
|C21| |C22| · · · |C2n| ... ... · · · ...
|Cn1| |Cn2| · · · |Cnn|
• In other words the adjoint of an n × n matrix A is the transpose of an n × n matrix C whose (i,j)th element is the (i,j)th cofactor of A.
• Given the adjoint of an n × n matrix A matrix A that is invertible, we can then calculate the inverse as
A−1 = 1
|A|adj(A) Example
• Suppose A =
2 7 1
5 6 8
1 −3 4
. We can calculate |A| = −9 so we know that the inverse exists.
• The matrix of cofactors of the matrix A can be calculated as
|C11| =
6 8
−3 4
= 48, |C12| = −
5 8 1 4
= −12, |C13| =
5 6
1 −3
= −21
|C21| = −
7 1
−3 4
= −31, |C22| =
2 1 1 4
= 7, |C23| = −
2 7
1 −3
= 13
|C31| =
7 1 6 8
= 50, |C32| = −
2 1 5 8
= −11, |C33| =
2 7 5 6
= −23
• We can then write down the adjoint of A as the transpose of the above matrix of cofactors
adj(A) =
48 −12 −21
−31 7 13
50 −11 −23
0
=
48 −31 50
−12 7 −11
−21 13 −23
.
• Therefore
A−1 = 1
|A|adj(A) = 1
−9
48 −31 50
−12 7 −11
−21 13 −23
=
−48/9 31/9 −50/9 12/9 −7/9 11/9 21/9 −13/9 23/9
• I’ll leave it up to you verify that this is right. In general calculating the inverse of a 3 × 3 matrix is a lot of work and a 4 × 4 matrix is a monumental task to invert. Fortunately, you will rarely have to this by hand. There are plenty of excellent software packages like Mathematica and Matlab, that will calculate the determinant for you, even if the matrix consists of symbolic expressions instead of numbers.
