Fall Semester ’07-’08 Akila Weerapana
LECTURE 6: MATRIX ALGEBRA
I. INTRODUCTION
• The first few lectures in the class reviewed material that you had already covered, even though the applications may have been new. Starting with this lecture, we start covering new material, at least for some of you, drawn from linear algebra and differential equations.
• Since Math 206 is at least a co-requisite for the course you should already have started covering basic matrix operations such as addition, multiplication, inversion etc, so we will move quickly through that section. If you have taken Math 206 already, this material will be extremely basic.
• Today’s class will cover the most basic features of linear algebra: what matrices are, elemen- tary operations such as transposition, addition, multiplication and determinants of matrices.
We will also define the inverse of a matrix and how to calculate it.
II. WHAT IS A MATRIX
Matrix
• Most mathematics textbooks define a matrix as a “rectangular array” containing numbers or variables. Although hard to describe in words, a matrix can be visualized extremely easily.
The following are all matrices.
4 5 6 8 1 3 5 7
4 5 1 3 5 7
x1 x2 x3 x4
1 4 3
x y z
• A matrix is typically described in terms of its dimensions. An m × n (pronounced m by n) matrix is a matrix with m × n entries represented in m rows and n columns. So the 5 matrices above are 2 × 4, 3 × 2, 2 × 2, 1 × 3 and 3 × 1 matrices respectively.
• The entries of a matrix are usually referenced according to their location within the matrix.
So in general if we have an m × n matrix known as X, we will refer to the element in row i and column j of X as Xij . The elements that lie on the diagonal line between the first and the last entries, i.e. those elements for which i = j are known as the diagonal elements and the others as the off-diagonal elements.
• A matrix in which the number of rows equals the number of columns is called a square matrix.
Identity Matrix
• The identity matrix, denoted I, is a special matrix. It is a square matrix with with each diagonal entry being 1 and each non-diagonal entry being 0.
• We use the notation Into refer to the dimension of the matrix (since number of rows= number of columns in a square matrix, we only need 1 number to know its dimensions.
Example
I3=
1 0 0 0 1 0 0 0 1
I2=
1 0 0 1
Symmetric Matrix
• A symmetric matrix is another special matrix. A matrix A is said to be symmetric if Aij = Aji ∀i, j In other words when we transpose the rows and columns of A, the resulting matrix, denoted A0 is equivalent to A.
• Given the definition we can see that a matrix that is symmetric must also be square. The converse is not true, not all square matrices are symmetric.
Example
A =
1 0 3 0 7 5 3 5 8
is symmetric since A0=
1 0 3 0 7 5 3 5 8
but B =
0 7 3 5
is NOT symmetric since B0=
0 3 7 5
Vector
• A matrix with a single row or column is sometimes referred to as a vector. A matrix with a single column is called a column vector and a matrix with a single row is called a row vector. A 1 × 1 matrix, i.e. a matrix with only 1 entry is called a scalar.
III. BASIC MATRIX OPERATIONS
Transposition
• The transpose of an m × n matrix is an n × m matrix constructed by switching rows and columns of the original matrix. That is to say, elements that were on the first column of the original matrix now make up the first row of the transposed matrix. The transpose of a given matrix X is denoted as X0.
Example
• The transposes of the 5 matrices given in the above section are
4 1 5 3 6 5 8 7
4 1 5 1 3 7
x1 x3 x2 x4
1 4 3
x y z
• We can now simplify the definition of a symmetric matrix A as A is symmetric iff A = A0 Matrix Addition
• Matrix addition is a simple operation. The result of adding two matrices together is a third matrix whose elements in any given row and column are obtained by adding elements in that same row and column in the first matrix to the elements in the same row and column of the second matrix.
• As a result, matrix addition is legal only when performed with two matrices of identical dimensions, i.e. a 5 × 6 matrix can only be added to another 5 × 6 matrix.
Example
4 1 5 1 3 7
+
1 1 2 0 3 −5
=
5 2 7 1 6 2
• Matrix subtraction is very similar. The result of A - B where A and B are both m×n matrices is another m × n matrix C where cij = aij− bij.
Example
4 1 5 1 3 7
-
1 1 2 0 3 −5
=
3 0 3 1 0 12
• Rules of Addition
A + B = B + A (A + B) + C = A + (B + C) Scalar Multiplication
• Any matrix can be multiplied by a scalar. If A is an m×n matrix and q is a scalar, the matrix that results from multiplying A by the scalar q is a m × n matrix C, where each element is obtained by multiplying the corresponding element A by q, i.e. Cij = q × Aij.
Example
5
4 1 5 1 3 7
=
20 5 25 5 15 35
Matrix Multiplication
• Matrix multiplication is by far the most complicated of the basic operations. When we multiply matrix A by matrix B (denoted AB) to obtain matrix C, a given element is obtained by multiplying the ith row of A by the jth column of B. Therefore, multiplication of two matrices is legal only when the ith row of the first matrix has the same number of elements as the jth column of the second matrix: in other words the number of columns in A has to equal the number of rows in B.
• More generally an m × n matrix A can be multiplied by an n × p matrix B where p is any positive integer. The resulting matrix C is of dimension m × p with elements defined as follows: Cij =Pn
k=1(Aik× Bkj) for i = 1 · · · m, j = 1 · · · p.
• A simple example illustrates the procedure much more clearly than the notation. Suppose we have a 3×2 matrix, A, and a 2×3 matrix, B. Note that according to the rules of multiplication we can do either AB (which will be a 3 × 3 matrix) or BA (which will be a 2 × 2 matrix) in this case. (Note: If B was a 2 × 5 matrix, for example, only AB would be defined. BA would not be defined since the number of columns in B (5) is greater than the number of rows in A (3)).
Example
Suppose A=
1 1 1 3 2 −5
and B =
4 1 5 1 3 7
. Then
AB =
1 1 1 3 2 −5
4 1 5 1 3 7
=
5 4 12
7 10 26 3 −13 −25
BA =
4 1 5 1 3 7
1 1 1 3 2 −5
=
15 −18 18 −25
• Rules of Multiplication (where A is m × n, D is n × p and E is p × q) AB 6= BA
(AD)E = A(DE) AIn = A ImA = A (AD)0 = D0A0
• Rules of Vector Multiplication If x is an n × 1 column vector then a) x0x is a scalar equal Pn
k=1x2k1
b) xx0 is a symmetric matrix of dimension n xx0 =
x211 x12x21 · · · x1nxn1 x21x12 x222 · · · x2nxn2
... ... · · · ... xn1x1n xn2x2n · · · xnnn
Determinant
• The determinant of a square matrix is a unique number associated with that matrix. The determinant of a matrix is usually denoted by writing the name of the matrix surrounded by two vertical bars: e.g. the determinant of A is |A|.
• How is the determinant of a matrix calculated. For a 1 × 1 matrix, a 2 × 2 matrix or a 3 × 3 matrix, the process is fairly simple. For higher dimension matrices, the calculation is more involved.
• The determinant of a 1 × 1 matrix (a scalar) is the scalar itself.
• The determinant of a 2 × 2 matrix of the form A =
a b c d
is |A| = (ad − bc)
• The determinant of a 3 × 3 matrix of the form A =
a b c d e f g h i
is
|A| = (aei + bf g + cdh) − (ceg + bdi + af h)
Although this seems hard to remember, there is a simple trick that enables us to calculate the determinant of a 3 × 3 matrix. The trick requires you to write down two copies of the matrix side by side, then add the products of the three leftmost diagonals and subtract the products of the three rightmost diagonals.
• The determinant of an n×n matrix is found using a procedure known as a ‘Laplace expansion’.
The Laplace expansion is a recursive process: i.e. the determinant of an n × n matrix is expressed as a function of determinants of several (n − 1) × (n − 1)) matrices, each of which in turn are expressed as functions of several (n − 2) × (n − 2) matrices etc. Since we know how to calculate the determinant of a 3 × 3 matrix or a 2 × 2 matrix we can always build back up to the top.
• In order to write down the Laplace expansion formula, we need to define a few other terms.
The minor of a matrix is the determinant of a submatrix formed from a given matrix. In particular, the (i,j)th minor of a matrix, denoted as |Mij| is the determinant of a matrix obtained by omitting the ith row and jth column of the original matrix.
• Consider the following matrix A =
2 7 0 1 5 6 4 8 0 0 9 0 1 −3 1 4
. Some of the minors of this matrix are
|M11| =
6 4 8 0 9 0
−3 1 4
= 432, |M23| =
2 7 1 0 0 0 1 −3 4
= 0 and |M41| =
7 0 1 6 4 8 0 9 0
= −450.
• The cofactor of a matrix is another determinant of a submatrix formed from the original.
The (i,j)th cofactor of a matrix is defined as |Cij| = (−1)i+j|Mij|, that is to say the co-factor is similar to the minor, except that it may have a different sign. If the sum of i and j is an even number then the sign is positive, if the sum is an odd number, the sign is negative. So
|C11| = |M11|, |C12| = − |M12|, |C23| = − |M23|, |C24| = |M24| etc.
• We are finally ready to write down the formula for the Laplace expansion of a matrix A. If A is an n × n matrix then
|A| =
n
X
j=1
aij|Cij| for any i = 1 · · · n
• In other words, we can find the determinant of a matrix by picking any row, then multiplying each element of the row by the appropriate cofactor.
• Which should you choose? Well its always easier to pick a row with a lot of zeros, since that minimizes the number of calculations one has to do. So in our case, the best thing to do would be to use the expansion of the 3rd row.
|A| = 0 |C31| + 0 |C32| + 9 |C33| + 0 |C34|
= 9 |C33| = 9 |M33|
= 9
2 7 1 5 6 8 1 −3 4
|A| = −81
• As you can see this is a very tedious calculation even for a 4 × 4 matrix. Fortunately, we will rarely have to calculate determinants by hand, using computers for numerical matrices and other analytical short cuts for algebraic matrices. Nevertheless, you should have some idea about how determinants are calculated using co-factors.
• Properties of the Determinant
There are several important properties that relate to determinants.
– The determinant of A is equal to the determinant of its transpose |A| = |A0|
– The determinant of a diagonal matrix (i.e. one whose off-diagonal elements are ALL zero) is equal to the product of the element of the diagonals.
– A is invertible iff |A| 6= 0
– If any row of a matrix is a linear combination of one or more of the other rows of the matrix then the matrix has a determinant of zero. The same holds true for columns. So matrices with linearly dependent rows or columns are not invertible.
– If any row of a matrix consists only of zeros then the matrix has a determinant of zero.
IV. CALCULATING THE INVERSE OF A MATRIX
• Any square matrix that has an inverse is said to be a non-singular matrix. A matrix that has no inverse is said to be a singular matrix. The inverse of a n-dimensional square matrix A is denoted A−1, if it exists. The inverse matrix has the property that AA−1 = In
and A−1A = In.
• If a matrix has an inverse matrix, then the matrix must be a square matrix, i.e. only square matrices have inverses. This does not mean that ALL square matrices have inverses, it just means that matrices that are not square do not have an inverse matrix.
• Consider the matrix A=
4 5 1 3
, the inverse of this matrix is A−1 =
3/7 −5/7
−1/7 4/7
since AA−1= A−1A = A =
1 0 0 1
• Properties of the Inverse
– If A−1 is the inverse of a matrix A, then A−1−1
= A – If A−1 is the inverse of a matrix A, then (A0)−1= (A−1)0
– If A and B are both square matrices with inverse matrices A−1 and B−1. Then the matrix C = AB has an inverse defined as C−1= B−1A−1
• Calculating the inverse of a matrix is a chore. First, we have to define something called the adjoint of the matrix.
• The adjoint of an n × n matrix A is another n × n matrix defined as
adj(A) = C0 where C =
|C11| |C12| · · · |C1n|
|C21| |C22| · · · |C2n| ... ... · · · ...
|Cn1| |Cn2| · · · |Cnn|
• In other words the adjoint of an n × n matrix A is the transpose of an n × n matrix C whose (i,j)th element is the (i,j)th cofactor of A.
• Given the adjoint of an n × n matrix A matrix A that is invertible, we can then calculate the inverse as
A−1 = 1
|A|adj(A) Example
• Suppose A =
2 7 1 5 6 8 1 −3 4
. We can calculate |A| = −9 so we know that the inverse exists.
• The matrix of cofactors of the matrix A can be calculated as
|C11| =
6 8
−3 4
= 48, |C12| = −
5 8 1 4
= −12, |C13| =
5 6 1 −3
= −21
|C21| = −
7 1
−3 4
= −31, |C22| =
2 1 1 4
= 7, |C23| = −
2 7 1 −3
= 13
|C31| =
7 1 6 8
= 50, |C32| = −
2 1 5 8
= −11, |C33| =
2 7 5 6
= −23
• We can then write down the adjoint of A as the transpose of the above matrix of cofactors
adj(A) =
48 −12 −21
−31 7 13
50 −11 −23
0
=
48 −31 50
−12 7 −11
−21 13 −23
.
• Therefore
A−1= 1
|A|adj(A) = 1
−9
48 −31 50
−12 7 −11
−21 13 −23
=
−48/9 31/9 −50/9 12/9 −7/9 11/9 21/9 −13/9 23/9
• I’ll leave it up to you verify that this is right. In general calculating the inverse of a 3 × 3 matrix is a lot of work and a 4 × 4 matrix is a monumental task to invert. Fortunately, you will rarely have to this by hand. There are plenty of excellent software packages like Mathematica and Matlab, that will calculate the determinant for you, even if the matrix consists of symbolic expressions instead of numbers.
