Elementary Matrices and The LU Factorization
Definition: Any matrix obtained by performing a single elementary row operation (ERO) on the identity (unit) matrix is called an elementary matrix.
There are three elementary operations:
1. Permute rows i and j
2. Multiply row i by a non-zero scalar k 3. Add k times row i to row j
Corresponding to the three ERO, we have then three elementary matrices:
Type 1: Pij - permute rows i and j in In.
Type 2: Mi
( )
k - multiply row i of In by a non-zero scalar k Type 3: Aij( )
k - Add k times row i of In to row jAll three types of 3×3elementary matrices are shown below:
Permutation matrix:
=
1 0 0
0 0 1
0 1 0
P12
Scaling matrix:
( )
=
1 0 0
0 0
0 0 1
2 k k
M
Row combination:
( )
=
1 0 0
0 1
0 0 1
12 k k
A
Pre-multiplying an n×pmatrix A by an n×nelementary matrix E has the effect of performing the corresponding ERO on A.
Example:
−
= 4 7 5
1 3 A 2
We can multiply the First row of the matrix A by 3 (an elementary row operation). The resulting matrix will become
−
5 7 4
3 9 6
We can achieve the same result by pre-multiplying A by the corresponding 2
2× elementary matrix.
( )
−
=
−
=
5 7 4
3 9 6 5 7 4
1 3 2 1 0
0 3 3
1 A
M
An ERO can be performed on a matrix by pre-multiplying the matrix by a corresponding elementary matrix. Therefore, we can show that any matrix A can be reduced to a row echelon form (REF) by multiplication by a sequence of elementary matrices. Therefore, we can write
R A E E
E1 2m k = (1) where R denotes an REF of A.
Consider a nonsingular n×nmatrix A. Since the unique reduced row echelon form (RREF) of such a matrix is the identity matrix I , it follows that there exists elementary n matrices (i.e. there exists elementary row operations) E , 1 E , ..., 2 E such that k
n
kA I
E E
E1 2m = (2) But we know that A−1A=In and this implies from Eqn. (2) that A E1E2mEk
1 =
− or
equivalently A E1E2mEkIn
1 =
− (3)
This shows that A can be obtained by applying to −1 I the same sequence of ERO that n reduces Ato the identity matrix. This is what we do to find A using the Gauss-Jordan −1 method.
LU decomposition of a nonsingular matrix
A nonsingular matrix can be reduced to an upper triangular matrix using elementary row operations of Type 3 only. The elementary matrices corresponding to Type 3 EROs are unit lower triangular matrices. We can write
U A E E
E1 2 k = (4)
where E , 1 E , ..., 2 E are unit lower triangular Type 3 elementary matrices and U is an k upper triangular matrix. Since each elementary matrix is nonsingular (meaning their inverse exist) we can write from Eqn. (4) that
U E E E
E
A= k−1 k−1−1m 2−1 1−1 (5) We know that the product of two lower triangular matrices is also a lower triangular matrix. Therefore Eqn. (5) can be written as
LU A=
where L=Ek−1Ek−−11E2−1E1−1 (6)
Of course we need to know the inverses of the Type 3 elementary matrices. Inverses of the three n×nelementary matrices are:
( )
k M( )
kMi −1 = i 1 , Pij−1 =Pij and Aij
( )
k −1 =Aij(−k)Example: Determine the LU factorization of the matrix
−
−
=
1 2 1
2 1 3
3 5 2 A
First, let us do the EROs to reduce A into an upper triangular matrix in the following manner.
(((( )))) (((( ))))
−−−−
−−−−
⇒
⇒
⇒
−−−− ⇒
⇒
⇒⇒
⇒
−−−−
−−−−
2 5 2 9 0
2 13 2 13 0
3 5
2 2 1 2
3 1
2 1
2 1 3
3 5 2
13
12 ,A
A ⇒⇒⇒⇒ A23
((((
9 13))))
−−−−
−−−−
−−−−
⇒⇒⇒
⇒
2 0
0
2 13 2 13 0
3 5
2
These EROs can be written in terms of their equivalent elementary matrices as
−−−−
−−−−
−−−−
====
2 0
0
2 13 2 13 0
3 5
2
3 2
1E E A
E (7)
where
((((
9 13))))
2 13(((( ))))
1 2 3 12((((
3 2))))
23
1 ====A , E ==== A , E ==== A −−−−
E
Note the order of multiplication in Eqn. (7).
−−−−
−−−−
−−−−
====
2 0
0
2 13 2 13 0
3 5
2
U and L====E3−−−−1E2−−−−1E1−−−−1
We can compute the inverses of the elementary matrices very easily.
((((
913)))) ((((
1 2))))
12(((( ))))
3 21 3 13
1 2 23
1
1 A , E A , E A
E −−−− ==== −−−− −−−− ==== −−−− −−−− ====
Therefore,
−−−−
−−−−
====
1 13 9 0
0 1 0
0 0 1
1 0 2 1
0 1 0
0 0 1
1 0 0
0 1 2 3
0 0 1 L
−−−−
−−−−
====
1 13 9 2 1
0 1 2
3
0 0 1
L
Therefore A can be written as
−−−−
−−−−
−−−−
−−−−
−−−−
====
2 0
0
2 13 2 13 0
3 5
2
1 13 9 2 1
0 1 2 3
0 0 1
A
We can construct the lower triangular matrix L without multiplying the elementary matrices if we utilize the multipliers obtained while we converted matrix A into an upper triangular matrix. But, what exactly are those multipliers?
Definition: When using ERO of Type 3, the multiple of a specific row i that is subtracted from row j to put a zero in the ji position is called a multiplier, and is denoted as mji . In our example we have three multipliers:
13 9 2
1 2
3 31 32
21 ==== , m ====−−−− , m ====−−−−
m
If we notice the unit lower triangular matrix L carefully, we see that the elements beneath the leading diagonal are just the corresponding multipliers. This relationship holds in general. Therefore, we can do elementary row operations of Type 3 to reduce A to upper triangular form and then utilize the corresponding multipliers to write L directly.
Example: Determine the LU factors for the matrix
A 3 4 5
−2 2
−1 2 1
1 3 4 3
−2 1 6 2
:=
Type 3 EROs to reduce A to the upper triangular matrix can be achieved by pre-
multiplying A by the corresponding elementary matrices. The elementary matrices are listed in the order they are multiplied.
E12 1
4
−3 0 0
0 1 0 0
0 0 1 0
0 0 0 1
:= E13
1 0 5
−3 0
0 1 0 0
0 0 1 0
0 0 0 1
:= E14
1 0 0 2 3
0 1 0 0
0 0 1 0
0 0 0 1
:=
E23 1 0 0 0
0 1 4
−11 0
0 0 1 0
0 0 0 1
:= E24
1 0 0 0
0 1 0 7 11
0 0 1 0
0 0 0 1
:=
E34
1 0 0 0
0 1 0 0
0 0 1 52
−19 0 0 0 1
:=
E34⋅E24⋅E23⋅E14⋅E13⋅E12⋅A 3 0 0 0
2 3.667
− 0 0
1 1.667 1.727
0
−2 3.667
8 18.895
−
=
If you like the fractional form then
U 3 0
0
0 2 11
− 3 0
0 1 5 3 19 11 0
−2 11
3 8 359
− 19
:=
The lower triangular matrix L can be found by the following.
L:=E12− 1⋅E13− 1⋅E14− 1⋅E23−1⋅E24− 1⋅E34− 1
L
1 1.333 1.667 0.667
−
0 1 0.364
0.636
−
0 0 1 2.737
0 0 0 1
=
If you like the fractional form then
L 1 4 3 5 3 2
−3 0 1 4 11
7
−11 0 0
1 52 19
0 0
0
1
:=
Note that the multipliers corresponding to the EROs are:
m21 4
:= 3 m31 5
:= 3 m41 2
−3
:= m32 4
:= 11 m42 7
−11
:= m43 52
:= 19 In a unit lower triangular case, the matrix L can be constructed directly by utilizing the multipliers.
To verify multiply L and U.
L U⋅ 3 4 5
−2 2
−1 2 1
1 3 4 3
−2 1 6 2
=
THE END
