Elementary Matrices

In Sections 3.1 and 3.5, we saw some connections between matrix-vector multiplica­

tion and systems of linear equations. In Section 3.2, we observed the connection be­

tween linear mappings and matrix-vector multiplication. Since matrix multiplication is a direct extension of matrix-vector multiplication, it should not be surprising that there is a connection between matrix multiplication, systems of linear equations, and linear mappings. We examine this connection through the use of elementary matrices.

A matrix that can be obtained from the identity matrix by a single elementary row operation is called an elementary matrix.

Note that it follows from the definition that an elementary matrix must be square.

E 1 =

[ ^� �]

is the elementary matrix obtained from I2 by adding the product oft times the second row to the first-a single elementary row operation. Observe that £1 is the matrix of a shear in the x1 -direction by a factor oft.

dard matrix of a shear, a stretch, or a reflection. The following theorem tells us that ele­

mentary matrices also represent elementary row operations. Hence, performing shears, stretches, and reflections; multiplying by elementary matrices; and using elementary row operations all accomplish the same thing.

If A is an n x n matrix and Eis the elementary matrix obtained from In by a certain elementary row operation, then the product EA is the matrix obtained from A by performing the same elementary row operation.

It would be tedious to write the proof in the general nxn case. Instead, we illustrate why this works by verifying the conclusion for some simple cases for 3 x 3 matrices.

EXERCISE 1

Theorem 2

EXAMPLE2

Section 3.6 Elementary Matrices 177

Case 2. Consider the elementary row operation of swapping row 2 _{with row} 3, _which Again, the conclusion of Theorem 1 is verified.

Verify that Theorem 1 also holds for the elementary row operation of multiplying the

tary row operations to bring A into its reduced row echelon form. Call the elementary matrix corresponding to the first operation E1, the elementary matrix corresponding to the second operation E2, and so on, until the final elementary row operation cor­

responds to Ek. Then, by Theorem 1, E1A is the matrix obtained by performing the first elementary row operation on A, E2E1A is the matrix obtained by performing the second elementary row operation on E1A (that is, performing the first two elementary row operations on A), and Ek··· E2E1A is the matrix obtained after performing all of the elementary row operations on A, in the specified order. •

Let A =

[ � � �].

Find a sequence of elementary matrices E1, .. ., Ek such that Ek··· E1A is the reduced row echelon form of A.

Solution: We row reduce A keeping track of our elementary row operations:

[

^{1 2 1}

] [

^{1 2 1}

] [

^{1 2 1}

]

^{R1 - Rz}

[

^{1 2 0}

2 4 4 R2 - 2R1 - 0 0 2

�

R2 - 0 0 1 - 0 0 The first elementary row operation is Rz - 2R 1, so E 1 =

[

^_

� �].

The second elementary row operation is

�

^{R2, so E2 =}

[�

¹

^�

²

]

^.

EXAMPLE2

(continued)

EXERCISE 2

Theorem 3

The third elementary row operation is R 1 - R2, so E3 =

[ _{� -} �].

Thus,E3E2E1A =

[� -�][� 1�2][_� �][� _� !]

=

[� _� �l

Remark

We know that the elementary matrices in Example

2

must be 2 x

2

for two reasons.

First, we had only two rows in A to perform elementary row operations on, so this must be the same with the corresponding elementary matrices. Second, for the matrix multiplication E1A to be defined, we know that the number of columns in E1 must be equal to the number of rows in A. Also, E1 is square since it is elementary.

Let A

= [� H

Find a sequence of elementacy matrices £1,. • • , E, such that Ek··· E1A is the reduced row echelon form of A.

In the special case where A is an invertible square matrix, the reduced row eche­

lon form of A is I. Hence, by Theorem

2,

there exists a sequence of elementary row operations such that Ek··· E1A = I. Thus, the matrix B ⁼ Ek··· E1 satisfies BA = I, so B is the inverse of A. Observe that this result corresponds exactly to two facts we observed in Section 3.5. First, it demonstrates our procedure for finding the inverse of a matrix by row reducing

[

^A

^j

^I

)

. Second, it shows us that solving a system Ax =

b

by row reducing or by computing x = A-'

b

yields the same result.

Finally, observe that elementary row operations are invertible since they are re­

versible, and thus reflections, shears, and stretches are invertible. Moreover, since the reverse operation of an elementary row operation is another elementary row operation, the inverse of an elementary matrix is another elementary matrix. We use this to prove the following important result.

If an n x n matrix A has rank n, then it may be represented as a product of elementary matrices.

Proof: By Theorem 2, there exists a sequence of elementary row operations such that Ek··· E1A = I. Since Ek is invertible, we can multiply both sides on the left by (Ekt' to get

(Ek)-1 EkEk-l · · · E1A = (Ek)-11 or Ek-I · · · E1A = E"k1 Next, we multiply both sides by E;!, to get

We continue to multiply by the inverse of the elementary matrix on the left until the equation becomes

EXAMPLE3

Section 3.6 Exercises 179

Thus, since the inverse of an elementary matrix is elementary, we have written A ^{as a}

product of elementary matrices. •

Remark

Observe that writing A as a product of simpler matrices is kind of like factoring a polynomial (although it is definitely not the same). This is an example of a matrix de­

composition. There are many very important matrix decompositions in linear algebra.

We will look at a useful matrix decomposition in the next section and a couple more of them later in the book.

Let A =

[� �l

^{Write A and A-1} as a product of elementary matrices.

Solution: We row reduce A ^to I, keeping track of the elementary row operations used:

[

^{0 2}

] [

^{1 1}

]

¹

-[

^{1 1}

]

^{R1 - R1}

-[

^{1 0}

]

1 1 R2 ! Ri

-

^{0 2 2R2 0 1 0 1}

Hence, we have

Thus,

and

A= e-lE-lE-1 = _{1 2 3} 1 00 20 1

[o

¹

] [1 OJ [

^{1 1}

]

PROBLEMS 3.6

Practice Problems

Al Write a 3 x 3 elementary matrix that corresponds to each of the following elementary row opera­

tions. Multiply each of the elementary matrices by A =

[- ^{� �} !]

and verify that the product EA is

4 2 0

the matrix obtained from A by the elementary row operation.

(a) Add (-5) times the second row to the first row.

(b) Swap the second and third rows.

(c) Multiply the third row by (-1).

(d) Multiply the second row by 6.

(e) Add 4 times the first row to the third.

A2 Write a 4 x 4 elementary matrix that corresponds to each of the following elementary row operations.

(a) Add (-3) times the third row to the fourth row.

(b) Swap the second and fourth rows.

(c) Multiply the third row by (-3).

(d) Add

2

times the first row to the third row.

(e) Multiply the first row by 3.

(f) Swap the first and third rows.

A3 For each of the following matrices, either state that it is an elementary matrix and state the correspond­

ing elementary row operation or explain why it is

tions. Multiply each of the elementary matrices by A =

[� � -�i

and verify that the product EA is each of the following elementary row operations.

(a) Add 6 times the fourth row to the second row.

A4 For each of the following matrices:

(i) Find a sequence of elementary matrices Eb ... , E1 such that Ek··· E1A =I.

(ii) Determine A-1 by computing Ek··· E1.

(iii) Write A as a product of elementary matrices.

(a) A =

[i ^� �]

it is an elementary matrix and state the correspond­

ing elementary row operation or explain why it is not elementary.

(a)[�� �i ^{0 1 0}

^(b)

[ ^{1 � �} o o�l

(c)

[-� � �i _{0 1 0}

^(d)

[� _{1 0 1} ⁰ �i

(e)

[� � �i _{0 0 1}

^cn

[

^-

l ! �]

B4 For each of the following matrices:

(i) Find a sequence of elementary matrices Eb ... , E1 such that Ek··· E1A = I.

(ii) Determine A-1 by computingEk···E1.

(iii) Write A as a product of elementary matrices.

(a) A =

[ ^{� � -} _{2 4 0} �i

(b)

A= [ ^{� �}

_{1 4 1}

�1

(c)

A= H _; -�]

Conceptual Problems

Dl (a) Let L ^:

IR.2

-t

IR.2

be the invertible linear op­

erator with standard matrix

A = [� =�l

^By

writing

A

as a product of elementary matrices, show that L can be written as a composition of shears, stretches, and reflections.

(b) Explain how we know that every invertible lin­

ear operator L ^: JRll ^-t

IR.11

can be written as a composition of shears, stretches, and reflec­

tions.

D2 For 2 x 2 matrices, verify that Theorem 1 holds for the elementary row operations add t times row 1 to row 2 and multiply row 2 by a factor of t t 0.

E2E1 b.

Instead of using matrix multiplication, calculate the solution 1 in the following way.

First, compute

E 1 b

by performing the elemen­

tary row operation associated with

£1

_{on the}

matrix

b.

Then compute

x = E2E 1 b

_{by per­}

forming the elementary row operation associ­

ated with

£2

on the result for

E

₁

b

.

(c) Solve the system

Ax = b

by row reducing

[ ^A I ^b ].

Observe the operations that you use on the augmented patt of the system and com­

pare with part (b).

3.7 LU-Decomposition

Definition

Upper Triangular Lower Triangular

One of the most basic and useful ideas in mathematics is the concept of a factorization of an object. You have already seen that it can be very useful to factor a number into primes or to factor a polynomial. Similarly, in many applications of linear algebra, we may want to decompose a matrix into factors that have certain properties.

In applied linear algebra, we often need to quickly solve multiple systems

Ax = b,

where the coefficient matrix

A

remains the same but the vector

b

changes. The goal of this section is to derive a matrix factorization called the LU-decomposition, which is commonly used in computer algorithms to solve such problems.

We now start our look at the LU-decomposition by recalling the definition of upper-triangular and lower-triangular matrices.

An n ^x n matrix U is said to be upper triangular if the entries beneath the main diagonal are all zero-that is, (U)ij

=

₀ whenever i > }. Ann x n matrix L is said to be lower triangular if the entries above the main diagonal are all zero-in particular, (L)iJ

=

₀ whenever i <

}.

Theorem 1

Definition

LU-Decomposition

Remark

By definition, a matrix in row echelon form is upper triangular. This fact is very im­

portant for the LU-decomposition.

Observe that for each such system Ax ⁼

b,

we can use the same row operations to row reduce

[

^A

I b J

to row echelon form and then solve the system using back­

substitution. The only difference between the systems will then be the effect of the row operations on

b.

In particular, we see that the two important pieces of information we require are the row echelon form of A and the elementary row operation used.

For our purposes, we will assume that our n x n coefficient matrix A can be brought into row echelon form using only elementary row operations of the form add a mul­

tiple of one row to another. Since we can row reduce a matrix to a row echelon form without multiplying a row by a non-zero constant, omitting this row operation is not a problem. However, omitting row interchanges may seem rather serious: without row interchanges, we cannot bring a matrix such as

[� �]

into row echelon form. How­

ever, we only omit row interchanges because it is difficult to keep track of them by hand. A computer can keep track of row interchanges without physically moving en­

tries from one location to another. At the end of the section, we will comment on the case where swapping rows is required.

Thus, for such a matrix A, to row reduce A to a row echelon form, we will only use row operations of the form R; + sR j• where i > j. Each such row operation will have a corresponding elementary matrix that is lower triangular and has ls along the main diagonal. So, under our assumption, there are elementary matrices E 1, ^{. • .} , Ek that are all lower triangular such that

where U is a row echelon form of A. Since Ek··· E1 is invertible, we can write A (Ek··· E1)-1 U and define

Since the inverse of a lower-triangular elementary matrix is lower triangular, and a product of lower-t1iangular matrices is lower triangular, Lis lower triangular. (You are asked to prove this in Problem Dl .) Therefore, this gives us the matrix decomposition A = LU, where U is upper triangular and Lis lower triangular. Moreover, L contains the information about the row operations used to bring A to U.

If A is an n x n matrix that can be row reduced to row echelon form without swap­

ping rows, then there exists an upper triangular matrix U and lower triangular matrix L such that A =LU.

Writing an n x n matrix A as a product LU, where Lis lower triangular and U is upper triangular, is called an LU-decomposition of A.

Our derivation has given an algorithm for finding the LU-decomposition of such a matrix.

EXAMPLE 1

EXAMPLE2

Section 3. 7 LU-Decomposition 183

[ 2 -1 4]

Find an LU-decomposition of

A = -1 -1 3 4 -1

^{6 .}

Solution: Row-reducing and keeping track of our row-operations gives

4 -1

⁶

^{R2 -} 2 ^{R 1} - 0 1 -2

[ 2 -1 4 l -1 -1 3 ^R3+�R1 [ 2 -1 4 l 0 -3/2

⁵

^{R3 + �R2} [ 2 -1 4 l

- 0 1 -2 0 0 2 ⁼

^u

E1 = [-201 ^� 0 1 1/2 0 1 0 3/2 �],

^{£2 =}

[ ^{� �} �],

^{£3 =}

[� ^� �ll

Hence, we let

L

= E; 1Ei1E;1 = [ �

= [� 0

�] [_lj 1 0

And we get

A=

^LU.

1 0 0 Ol [ 1 -1/2 0 1 0 0 0 1 0 0

^I

0 l 1 0 - 2

1 0 -3/2 1 0 0 Ol [

^I

-3/2 1 -1/2 0 01 -3/2 1 �I

Observe from this example that the entries in Lare just the negative of the multi­

pliers used to put a zero in the corresponding entry. To see why this is the case, observe that if

Ek··· E1A =

^{U , then}

Hence, the same row operations that reduce

A

to U will reduce L to /.

This makes the LU-decomposition extremely easy to find.

[ 2 1 -1]

Find an LU -decomposition of B ⁼

-4 3 3 .

^{6 8}

-3

EXAMPLE2

(continued)

EXAMPLE3

EXERCISE 1

Solution: By row-reducing, we get

Therefore. we have

B ^{= LU =}

H

⁰

�m i �: i

Find an LU-decomposition of C =

[ �

_-4

Solution: By row-reducing, we get

[ � ;

-4 -2

[

⁰⁰¹

_; -� l -[

^{6 -11 R3 + 3R2}

^�

⁰

Therefore, we have

2 -3

]

2 3.

-2 1 -3

i

-1

�

_

;

₀

-� l

₁₆

Find an LU-decomposition of A=

1-�

_{-3 -3} ^{-1 -}

^�

¹

]

^.

L =

H '. �]

L=

H

⁰

�]

L =

[ �

_-4

^�

*

�i

1 L=

[ �

_{-4 -3}

^� �ll

EXAMPLE4

Section 3.7 LU-Decomposition 185

In document Introduction to Linear Algebra for Science and Engineering 1st Ed (Page 191-200)