CHAPTER 1
Linear Algebra
Linear algebra comprises of the theory and applications of linear system of equations, linear transformations and Eigen-value problems.
Matrix
DefinitionA system of “m n” numbers arranged along m rows and n columns Conventionally, single capital letter is used to denote matrices Thus, A = [ a a a a a a a a a a a a a ] a ith row, jth column
Types of Matrices
1. Row and Column matrices
Row Matrices [ 2, 7, 8, 9] single row ( or row vector)
Column Matrices [
] single column (or column vector)
2. Square matrix
Same number of rows and columns.
Order of Square matrix no. of rows or columns e.g. A = [
] ; order of this matrix is 3
Principal Diagonal (or main diagonal or leading diagonal)
The diagonal of a square matrix (from the top left to the bottom right) is called as principal diagonal.
Trace of the Matrix
The sum of the diagonal elements of a square matrix. - tr (λ A) = λ tr(A) , λ is scalar-
- tr ( A+B) = tr (A) + tr (B) - tr (AB) = tr (BA)
Symmetric Matrix 𝐀
𝐓= A
Skew Symmetric Matrix 𝐀
𝐓= - A
3. Rectangular Matrix
Number of rows Number of columns 4. Diagonal Matrix
A Square matrix in which all the elements except those in leading diagonal are zero. e.g. [ ] 5. Scalar Matrix
A Diagonal matrix in which all the leading diagonal elements are same. e.g [
]
6. Unit Matrix (or Identity Matrix)
A Diagonal matrix in which all the leading diagonal elements are ‘ ’.
e.g. = [
]
7. Null Matrix (or Zero Matrix)
A matrix is said to be Null Matrix if all the elements are zero. e.g. 0
1
8. Symmetric and Skew symmetric matrices
* Symmetric, when a = +a for all i and j. In other words = A
Note :- Diagonal elements can be anything.
* Skew symmetric, when a = - a In other words = - A
Note :- All the diagonal elements must be zero.
Symmetric Skew symmetric [ a h g h b f g f c ] [ h g h f g f ] 9. Triangular matrix
A matrix is said to be “upper triangular” if all the elements below its principal diagonal are zeros.
A matrix is said to be “lower triangular” if all the elements above its principal diagonal are zeros. [ a h g b f c ] [ a g b f h c ]
Upper triangular matrix Lower triangular matrix
11. Singular matrix: If |A| = 0, then A is called a singular matrix. 12. Unitary matrix
If we define, A = (A̅) = transpose of a conjugate of matrix A Then the matrix is unitary if A . A =
For example
A = [
] , A = [
] A. A = , and Hence it’s a Unitary Matrix
13. Hermitian matrix
It is a square matrix with complex entries which is equal to its own conjugate transpose. A = A or a = a̅
For example: 0 i
i 1
Note: In Hermitian matrix , diagonal elements always real
14. Skew Hermitian matrix : It is a square matrix with complex entries which is equal to the negative of conjugate transpose.
A = A or a = a̅̅̅
For example = 0 i
i 1
Note: In Skew-Hermitian matrix , diagonal elements either zero or Pure Imaginary
15. Idempotent Matrix : If A = A, then the matrix A is called idempotent matrix.
16. Nilpotent Matrix : If A = 0 (null matrix), then A is called Nilpotent matrix (where K is a +ve integer).
17. Periodic Matrix : If A = A (where k is a +ve integer), then A is called Periodic matrix. If k =1 , then it is an idempotent matrix.
18. Proper Matrix : If |A| = 1, matrix A is called Proper Matrix.
Equality of matrices
Two matrices can be equal if they are of (a) Same order
Addition and Subtraction of matrices [a b c d ] [ a b c d ] = [ a a b b c c d d ] Rules
1. Matrices of same order can be added 2. Addition is commutative A+B = B+A
3. Addition is associative (A+B) +C = A+ (B+C) = B + (C+A) Multiplication of matrix by a Scalar
Every element of the matrix gets multiplied by that scalar.
Multiplication of matrices
Condition: Two matrices can be multiplied only when number of columns of the first matrix is equal to the number of rows of the second matrix. Multiplication of (m n) and (n
p) matrices results in matrix of (m p)dimension 0 m n n p = m p1.
To find Rank always remember to make matrix a triangular matrix (Upper triangular) or (lower triangular) [ ] or [ ]
Try to make I, zero then II and then III in any Matrix to find Rank of Matrix, for easy execution of question to find rank.
Determinant
An n order determinant is an expression associated with n n square matrix.
If A = [a ] , Element a with ith row, jth column.
For n = 2 , D = det A = |aa a a | = (a a - a a ) Determinant of “order n” D = |A| = det A = || a a a a a a a a a | |
Minors & Co-factors
The minor of an element in a determinant is the determinant obtained by deleting the row and the column which intersect that element.
D = |
a a a b b b c c c | Minor of a = |bc bc |
Cofactor is the minor with “proper sign”. The sign is given by (-1) (where the element belongs to ith row, jth column).
A2 = Cofactor of = (-1) |bc bc |
Cofactor matrix can be formed as |
A A A C C C | In general = = j = 0 if i j * , , , , . + Drill Problem
Can the determinant be expanded about the diagonal?
Properties of Determinants
1. A determinant remains unaltered by changing its rows into columns and columns into rows. | a b c a b c a b c | = | a a a b b b c c c |
2. If two parallel lines of a determinant are inter-changed, the determinant retains it numerical values but changes in sign. (In a general manner, a row or column is referred as line). | a b c a b c a b c | = | a c b a c b a c b | = | c a b c a b c a b | 3. Determinant vanishes if two parallel lines are identical.
4. If each element of a line be multiplied by the same factor, the whole determinant is multiplied by that factor. [Note the difference with matrix].
| a b c a b c a b c | =| a b c a b c a b c |
5. If each element of a line consists of the m terms, then determinant can be expressed as sum of the m determinants.
| a b c d e a b c d e a b c d e | = | a b c a b c a b c | + | a b d a b d a b d | | a b e a b e a b e | 6. If each element of a line be added equi-multiple of the corresponding elements of one or
more parallel lines, determinant is unaffected.
e.g. By the operation, + p +q , determinant is unaffected.
7. Determinant of an upper triangular/ lower triangular/diagonal/scalar matrix is equal to the product of the leading diagonal elements of the matrix.
8. If A & B are square matrix of the same order, then |AB|=|BA|=|A||B|. 9. If A is non singular matrix, then |A |=
| | (as a result of previous).
10. Determinant of a skew symmetric matrix (i.e. A =-A) of odd order is zero. 11. If A is a unitary matrix or orthogonal matrix (i.e. A = A ) then |A|= ±1.
12. If A is a square matrix of order n then |k A| = k |A|. 13. | | = 1 ( is the identity matrix of order n).
Multiplication of determinants
The product of two determinants of same order is itself a determinant of that order.
In determinants we multiply row to row (instead of row to column which is done for matrix).
Comparison of Determinants & Matrices
Although looks similar, but actually determinant and matrix is totally different thing and its technically unfair to even compare them. However just for reader’s convenience, following comparative table has been prepared.
Determinant Matrix
No of rows and columns are always equal No of rows and column need not be same (square/rectangle)
Scalar multiplication: elements of one line (i.e. one row and column) is multiplied by the constant
Scalar multiplication: all elements of matrix is multiplied by the constant
Can be reduced to one number Can’t be reduced to one number Interchanging rows and column has no
effect
Interchanging rows and columns changes the meaning all together
Multiplication of 2 determinants is done by multiplying rows of first matrix & rows of second matrix
Multiplication of the 2 matrices is done by multiplying rows of first matrix & column of second matrix
Transpose of Matrix
Matrix formed by interchanging rows & columns is called the transpose of a matrix and denoted by A .
Example: A = [
] Transpose of A= Trans (A)= A ’= A = 0 1
Note
A = (A + A ) + (A - A ) = symmetric matrix + skew-symmetric matrix
If A & B are symmetric, then AB+BA is symmetric and AB-BA is skew symmetric.
If A is symmetric, then An is symmetric (n= , , …….).
If A is skew-symmetric, then An is symmetric when n is even and skew symmetric when n is
odd.
Adjoint of a matrix
Adjoint of A is defined as the transposed matrix of the cofactors of A. In other words, Adj (A)= Trans (cofactor matrix)
Determinant of the square matrix A = [
a b c a b c a b c ] is = | a b c a b c a b c | The matrix formed by the cofactors of the elements in is
[
A C A C A C
] Also called as cofactor matrix
Then transpose of this matrix is [
A A A C C C
] --- This is Adj (A)
Inverse of a matrix
A =
| |
|A| must be non-zero (i.e. A must be non-singular).
Inverse of a matrix, if exists, is always unique.
If it is a 2x2 matrix 0 1 , its inverse will be
0
1
Drill Problem: Prove (A ) = A
Proof
RHS = ( A )
Pre-multiplying the RHS by AB, (A B) ( A ) = A (B. ) A = I
Similarly, Post-multiplying the RHS by AB, ( A ) (A B) = (A A ) B= B = Hence, AB & A are inverse to each other
Important Points
1. A = A = A, (Here A is square matrix of the same order as that of ) 2. 0 A = A 0 = 0, (0 is null matrix)
3. If AB = 0, then it is not necessarily that A or B is null matrix. Also at doesn’t mean A =
Example AB = 0
1 .0
1 = 0
4. If the product of two non-zero square matrix A & B is a zero matrix, then A & B are singular matrix.
5. If A is non-singular matrix and A.B=0, then B is null matrix. 6. AB BA (in general) commutative property is not applicable 7. A(BC) = (A B)C Associative property holds
8. A(B+C) = AB AC Distributive property holds 9. AC = AD , doesn’t imply C = D ,even when A -.
10. If A, C, D be nxn matrix, then If r(A)=n and AC=AD, then C=D. (Prove it. Hint: pre-multiply both sides by A )
11. (A+B)T = A +
12. (AB)T = . A
13. (AB)-1 = . A
14. A A = A A =
15. (kA)T = k.A (k is scalar, A is vector)
16. (kA)-1 = k . A (k is scalar , A is vector)
17. (A ) = (A )
18. ( A̅̅̅̅) = (A̅) (Conjugate of a transpose of matrix = Transpose of conjugate of matrix) 19. If a non-singular matrix A is symmetric, then A is also symmetric.
20. f A is a orthogonal matrix , then A and A are also orthogonal.
21. If A is a square matrix of order n then (i) |adj A|=|A|
(ii) |adj (adj A)|=|A|( )
Example
Demonstrate by example that AB BA Solution Suppose, A = 0 1, B= 0 1 AB = 0 1 , BA = 0 1 Example
Demonstrate by example that AB = 0 A = 0 or B = 0 or BA = 0 Solution AB = 0 1 . 0 1 = 0 1 BA = 0 1 Example
Demonstrate that AC = AD C = D (even when A 0) Solution AC = 0 1 . 0 1 = 0 1 AD = 0 1 . 0 1 = 0 1 Although AC = AD, but C D
Example
Write the following matrix A as a sum of symmetric and skew symmetric matrix A = [ ] Solution Symmetric matrix = (A +A ) = {[ ] [ ]} = [ ] = [ ]
Skew symmetric matrix = (A -A ) = [
Example
Check, if the following matrix A is orthogonal. A = [ ] Solution A = A = [ ] A . A = A . A = [ ] = [ ] = , Hence A is orthogonal matrix
Elementary transformation of matrix 1. Interchange of any 2 lines
2. Multiplication of a line by a constant (e.g. k )
3. Addition of constant multiplication of any line to the another line (e. g. + p )
Note
Elementary transformations don’t change the rank of the matrix. However it changes the Eigen value of the matrix.
We call a linear system S1 “ ow Equivalent” to linear system S2, if S1 can be obtained from S2 by finite number of elementary row operations.
Gauss-Jordan method of finding Inverse
Elementary row transformations which reduces a given square matrix A to the unit matrix, when applied to unit matrix , gives the inverse of A
Example
Find the inverse of [
] Solution
Write in the form, [
] Operate + , - [ ]
Operate - [ ] , /2 , /5 [ ] , + [ ] + [ ] Rank of matrix
If we select any r rows and r columns from any matrix A, deleting all other rows and columns, then the determinant formed by these r r elements is called minor of A of order r.
Definition: A matrix is said to be of rank r when, i) It has at least one non-zero minor of order r. ii) Every minor of order higher than r vanishes.
Other definition: The rank is also defined as maximum number of linearly independent row vectors.
Special case: Rank of Square matrix
Rank = Number of non-zero row in upper triangular matrix using elementary transformation.
Note
1. r(A.B) min { r(A), r (B)} 2. r(A+B) r(A) + r (B) 3. r(A-B) r(A) - r (B)
4. The rank of a diagonal matrix is simply the number of non-zero elements in principal diagonal.
5. A system of homogeneous equations such that the number of unknown variable exceeds the number of equations, necessarily has non-zero solutions.
6. If A is a non-singular matrix, then all the row/column vectors are independent. 7. If A is a singular matrix, then vectors of A are linearly dependent.
Example Find rank of [ ] Solution , , , , [ ] [ ] , - [ ] [ ]
Hence Rank = 3 (Number of Non zero rows)
Example
The rank of a diagonal matrix [ ] (A) 1 (B) 2 (C) 3 (D) 4 Solution
Option (C) is correct as the number of non-zero elements in the diagonal matrix gives the rank.
Example Rank of matrix [ ] is (A) 3 (B) 2 (C) 0 (D) 1 Solution Option (D) is correct By doing , we get [ ]
Example The rank of [ ] is 2 , value of µ =? (A) 3 (B) 2 (C) 1 (D) 0 Solution Option (C) is correct
(Hint: Just by equating the determinant to zero)
Vector space
It is a set V of non-empty vectors such that with any two vectors “a and b” in V, all their linear combinations ( , are real numbers) are elements of V.
Dimension
The maximum number of linearly independent vectors in V is called the dimension of V (and denoted as dim V).
Basis
A linearly independent set in V consisting of a maximum possible number of vectors in V is called a basis for V. Thus the number of vectors of a basis for V is equal to dim V.
Span
The set of all linear combinations of given vector ( ), ( ), ……… ( ) with same number
of components is called the span of these vectors. Obviously, a span is a vector space.
Solution of linear System of Equation For the following system of equations
a x + a x + - - - a x = k
a x + a x + - - - a x = k
- - - -
- - -
a x + a x + - - - a x = k
In matrix form, it can be written as A X = B Where,
Overlap 4 Parallel 4 2 8 = [ a a a a a a a a ] , = [ x x x ] , = [ k k k ] A= Coefficient Matrix
C = (A, B) = Augmented Matrix
r = rank (A), r = rank (C), n = Number of unknown variables (x , x , - - - x )
Meaning of consistency, inconsistency of Linear Equation
Consistent mean: one or more solution (i.e. unique or infinite solution) Consistent a) Unique solution x+2y = 4 3x +2y = 2 b) Infinite solution x+2y = 3x +6y = 12 Inconsistent No solution x+2y = 4 x +2y = 8
Consistency of a system of equations
For non-homogenous equations (A X = B)
i) If r r , the equations are inconsistent i.e. there is no solution.
ii) If r = r = n, the equations are consistent and there is a unique solution.
iii) If r = r < n, the equations are consistent and there are infinite number of solutions. For homogenous equations (A X = 0)
i) If r =n, the equations have only a trivial zero solution (i.e. x = x = - - - x = 0). ii) If r<n, then (n-r) linearly independent solution ( i.e. infinite non-trivial solutions).
Cramer’s ule
Let the following two equations be there
a x + a x = b ---(i) a x + a x = b ---(ii) D = |ab a b | D = |bb aa | D = |aa b b |
Solution using Cramer’s rule: x = and x =
In the above method, it is assumed that 1. No of equation = no of unknown 2. D 0
In general, for non-homogenous Equations D 0 single solution (non trivial) D = 0 infinite solution
For homogenous Equations
D 0 trivial solutions ( x = x =………x = 0) D = 0 non- trivial solution (or infinite solution)
Example
Solve the following Simultaneous Equation: x 2x x = 1 x 2x x = 2 7x 2x x = 5 Solution [ ] [ x x x ] = [ ] 3 , – ( )
[ ] [ x x x ] = [ ] x 2x x = 1 ---(i) 8x x = -1---(ii) Assume, x = k---(iii) From (ii), x = (5k+1) From (i), x = 1- (5k+1)+k = (3-k)
Has infinite solution (For every of k, there will be a solution set)
Example
For the given simultaneous equation, written in matrix form [ ] [ x y z ] = [
], determine the value of & µ for the following cases:
(A) No solution (B) Unique solution (C) Infinite solution Solution [ ] [ x y z ] = [ ] A X = B C = (A, B) = [ : : : ]
{Trick: Try to bring maximum zeros in last row through elementary transformation} - , - [ ] [ ]
= 3, 10 ii) For unique solution: R (A) = R (C) = 3
-3 0, may be anything 3, may be anything iii) Infinite solution: R (A) = R (C) = 2
-3 = 0, -10 = 0 = 3, = 10
Characteristic equation and Eigen Values
f A is a matrix, equation formed by ‘determinant of A λ equated to zero’ is called characteristic equation. i.e. | A λ |= 0
The roots of this equation are called the characteristic roots / latent roots / Eigen values of the matrix A.
Eigen vectors
[A λ ] X = 0For each Eigen value λ, solving for X gives the Eigen vectors. Clearly, zero vector X = 0 is one of the solutions. (But it is of no practical interest).
Note
1. The set of Eigen values is called SPECTRUM of A.
2. The largest of the absolute value of Eigen values is called spectral radius of A. 3. Multiplying the Eigen vector by a scalar (λ) and matrix (A) gives the same result. 4. For a given Eigen value, there can be different Eigen vectors, but for same Eigen vector,
there can’t be different Eigen values.
Properties of Eigen value
1. The sum of the Eigen values of a matrix is equal to the sum of its principal diagonal. 2. The product of the Eigen values of a matrix is equal to its determinant.
3. The largest Eigen values of a matrix is always greater than or equal to any of the diagonal elements of the matrix.
4. If λ is an Eigen value of orthogonal matrix, then 1/ λ is also its Eigen value. 5. If A is real, then its Eigen value is real or complex conjugate pair.
6. Matrix A and its transpose A has same characteristic root (Eigen values).
7. The Eigen values of triangular matrix are just the diagonal elements of the matrix. 8. Zero is the Eigen value of the matrix if and only if the matrix is singular.
9. Eigen values of a unitary matrix or orthogonal matrix has absolute value ‘ ’. 10. Eigen values of Hermitian or symmetric matrix are purely real.
12. | | is an Eigen value of adj A (because adj A = | |. ).
13. If λ is an Eigen value of the matrix then , i) Eigen value of A is 1/λ
ii) Eigen value of A is λ
iii) Eigen value of kA are λ (k is scalar) iv) Eigen value of A k are λ + k v) Eigen value of (A k )2 are ( k)
Properties of Eigen Vector
1) Eigen vector X of matrix A is not unique.
Let is Eigen vector, then C is also Eigen vector (C = scalar constant). 2) If , , . . . . . are distinct, then , . . . are linearly independent .
3) If two or more Eigen values are equal, it may or may not be possible to get linearly independent Eigen vectors corresponding to equal roots.
4) Two Eigen vectors are called orthogonal vectors if T∙ = 0.
( , are column vector)
(Note:for a single vector to be orthogonal , A = A or, A. A = A. A = )
5) Eigen vectors of a symmetric matrix corresponding to different Eigen values are orthogonal.
Example
Find the Eigen value and the Eigen vector of A = 0 1 Solution |A - λ | = 0 | λ λ | = 0 λ λ = λ = 6, 1
By definition of Eigen vector, [A - λ ] X = 0 Hence, [ λ λ ] 0xx 1 = 0
For λ = 6 , 0
1 0 x x 1 = 0 Which gives following two equation
-x + 4x = 0 ---(i) x - 4x = 0 ---(ii) However only one equation is independent
= giving the Eigen vector (4, 1) For λ = 1, 0 1 0 x x 1 = 0 4x + 4x = 0 , x + x = 0 =
giving the Eigen vector (1, -1)
Cayley Hamilton Theorem
Every square matrix satisfies its own characteristic equation. Linear Dependence of Vectors
Vector: Any quantity having n components is called a vector of order n.
If one vector can be written as linear combination of others, the vector is linearly dependent.
Linearly Independent Vectors
If no vectors can be written as a linear combination of others, then they are linearly independent.
Suppose the vectors are x , x , x , x
Its linear combination is λ x + λ x + λ x + λ x = 0
If λ , λ , λ , λ are not “all zero” they are linearly dependent.
If all λ are zero they are linearly independent.
Example If A = [
] , find the value of A A Solution
|A - | = 0 λ λ λ
= (-λ λ ) (λ+1) = 0 λ λ =0 or λ = 0
From Cayley Hamilton Theorem, A = 0 or A+I=0 As A
Example
For matrix A =[
], find Eigen vector of 3A A A . Solution |A |= 0 | | = 0 (1- ) (3 - ) (-2- ) = 0 = 1, = 3, = -2 Eigen value of A = 1, 3, -2 Eigen value of A = 1, 27, -8 Eigen value of A = 1, 9, 4 Eigen value of = 1, 1, 1
First Eigen value of 3A A A = 3 = 4
Second Eigen value of 3A A A = 3 (27)+5(9)–6(3)+2=81+45–18+2 = 110 Third Eigen value of 3A A A =3(-8)+5(4)–6(-2)+2= -24 + 20 + 12 + 2 = 10
Example
Find Eigen values of matrix A = [ a a a a a a a a a a ] Solution |A- | = | a a a a a a a a a a | = 0 Expanding, (a ) (a ) (a ) (a ) = 0
= a , a , a , a which are just the diagonal elements
{Note: recall the property of Eigen value, “The Eigen value of triangular matrix are just the diagonal elements of the matrix”+
