Rank plus nullity is the dimension of the domain.
Theorem 1 (Rank-Nullity) If L : V → W is a linear transformation, then the sum of the dimension of the kernel of L and the dimension of the image of L is the dimension of V .
The dimension of the kernel is sometimes called the “nullity” of L, and the dimension of the image is sometimes called the “rank” of L.
Hence the name “rank-nullity” theorem.
Prove this theorem Warning 2 This is hard!
Let v1, v2, v3, ..., vm be a basis of ker(L). We can extend this to a basis of V , v1, v2, ..., vn, u1, u2, ..., uk. We need to show that We will be done if we can show that L(u1), L(u2), ..., L(uk) form a basis of Im(L).
Let w ∈ Im(L). Then w = L(v) for some v ∈ V . Since v1, v2, ..., vn, u1, u2, ..., uk is a basis of V , we can write
w = L(a1v1+ a2v2+ ... + anvn+ b1u1+ ... + bkuk)
= a1L(v1) + ... + anL(vn) + b1L(u1) + ... + bkL(uk)
= b1L(u1) + ... + bkL(uk)
So Im(L) is spanned by the L(ui). Now we need to see that the L(ui) are linearly independent.
Assume b1L(u1) + b2L(u2) + ... + bkL(uk) = 0. Then L(b1u1+ ... + bkuk) = 0.
Then b1u1+ ... + bkuk would be in the null space of L. But the ui were chosen specifically to be linearly independent of all of the vectors in the null space. So b1= b2= ... = bk= 0. Thus the L(ui) are linearly independent and we are done.
58 Eigenvectors
Eigenvectors are mapped to multiples of themselves.
Definition 1 Let L : V → V be a linear map. A vector ~v ∈ V is called an eigenvector of L if L(~v) = λ~v for some λ ∈ R.
A constant λ ∈ R is called an eigenvalue of L if there is a nonzero eigenvector
~v with L(~v) = λ~v.
Geometrically, eigenvectors of L are those vectors whose direction is not changed (or at worst, negated!) when they are transformed by L.
Let’s try some examples.
Question 2 Suppose L : R2→ R2 is the linear map represented by the matrix
3 2 4 1
. Which of these vectors is an eigenvector of L?
Solution
Hint: We want to compute3 2 4 1
Consequently, 1
−1
is not an eigenvector. The eigenvector must be1 1
That’s right! Note that L1 1
is an eigenvector.
58 Eigenvectors
Solution
Hint: Try computing L 1
−2
.
Hint: In this case, L 1
−2
=−1 2
.
Hint: Question 4 Find λ ∈ R so that 1
−2
= λ ·−1 2
. Solution
Hint: The sign is opposite on both sides of the equation.
Hint: So try λ = −1.
λ = −1
And so −1 is an eigenvalue, with eigenvector 1
−2
.
Which of the following is another eigenvector?
(a) 1
−2
X
(b) 2
−1
Rock on! We check that L 1
−2
=−1 2
= −1 · 1
−2
and so 1
−2
is an eigenvector with eigenvalue −1.
59 Eigenvalues
Eigenvalues measure how much eigenvectors are scaled.
Definition 1 Let L : V → V be a linear operator (NB: linear maps with the same domain and codomain are called linear operators). The set of all eigenvalues of L is the spectrum of L.
Let’s try finding the spectrum.
Question 2 Let L : R2 → R2 be the linear map whose matrix is 1 2 2 1
with respect to the standard basis. L has two different eigenvalues. What are they?
Give your answer in the form of a matrixλ1 λ2
, where λ1≤ λ2.
Solution
Hint: For lambda to be an eigenvalue we need1 2 2 1
x y
= λx y
Hint: This is the same as
(x + 2y = λx 2x + y = λy or(
(1 − λ)x + 2y = 0 2x + (1 − λ)y = 0
Hint: These are two lines passing through the origin. To have more than just the origin as a solution, we need that the slope of the two lines is the same. So
1 − λ
2 = 2
1 − λ
Hint:
1 − λ
2 = 2
1 − λ (1 − λ)2= 4
1 − λ = ±2 lambda = −1 or 3
Hint: Let us now check that these really are eigenvalues:
If we let λ = −1, we have the equation 2x + 2y = 0. Check that 1
−1
is an eigenvector with eigenvalue −1
If we let λ = 3, we have the equation 2x − 2y = 0. Check that 1 1
is an eigenvector with eigenvalue 3
59 Eigenvalues
Question 3 Let’s try another example. Suppose F : R2 → R2 is the linear map represented by the matrix
0 −1
1 0
. Which of these numbers is an eigenvalue of F ? Solution
Hint: Let’s suppose thatx y
is an eigenvector.
Hint: Then there is some λ ∈ R so that0 −1
1 0
x y
= λx y
.
Hint: But0 −1
1 0
x y
=−y x
.
Hint: And so−y x
= λx y
.
Hint: This means that −y = λx and x = λy.
Hint: Putting this together, −y = λ2y and x = −λ2x.
Hint: Since we are looking for a nonzero eigenvector (in order to have an eigenvalue), we must have that either x 6= 0 or y 6= 0.
Hint: Consequently, λ2= −1.
Hint: But there is no real number λ ∈ R so that λ2 = −1, since the square of any real number is nonnegative.
Hint: Therefore, there is no real eigenvalue.
(a) There is no real eigenvalue. X (b) −1
(c) √ 2 (d) 1
Perhaps surprisingly, not every linear operator from Rn to Rn has any real eigenvalues.
60 Eigenspace
An eigenspace collects together all the eigenvectors for a given eigenvalue.
Theorem 1 Let λ be an eigenvalue of a linear operator L : V → V . Then the set Eλ(L) = {v ∈ V : L(v) = λv} of all (including zero) eigenvectors with eigenvalue λ forms a subspace of V .
This subspace is the eigenspace associated to the eigenvalue λ.
Prove this theorem. We need to check that Eλ(L) is closed under scalar multi-plication and vector addition
If v ∈ Eλ(L), and c ∈ R, then L(cv) = cL(v) = cλv = λ(cv), so cv is also an eigenvector of L.
If v1, v2∈ Eλ(L), then L(v1+ v2) = λv1+ λv2= λ(v1+ v2), so v1+ v2 is also an eigenvector of L.
The kernel of L is the eigenspace of the eigenvalue 0.
61 Eigenbasis
An eigenbasis is a basis of eigenvectors.
Observation 1 If (v1, v2, ..., vn) is a basis of eigenvectors of a linear operator L, then the matrix of L with respect to that basis is diagonal, with the eigenvalues of L appearing along the diagonal.
Theorem 2 Let L : V → W be a linear map. If v1, v2, ..., vn are nonzero eigen-vectors of L with distinct eigenvalues λ1, λ2, ...λn, then (v1, v2, ..., vn) are linearly independent.
Prove this theorem.
Assume to the contrary that the list is linearly dependent. Let vk be the first vector in the list which is in the span of the preceding vectors, so that the vectors (v1, v2, ..., vk−1) are linearly independent. Let a1v1+ a2v2+ ... + ak−1vk−1 = vk. Then applying L to both sides of this equation we have a1λ1v1+ a2λ2v2+ ... + ak−1λk−1vk−1= λkvk. If we multiply the first equation by λkwe also have a1λ1v1+ a2λ1v2+ ... + ak−1λ1vk−1= λ1vk. Subtracting these two equations we have
a1(λk− λ1)v1+ a2(λk− λ2)v2+ ... + a3(λk− λk−1)vk= 0.
Since the vectors (v1, v2, ..., vk−1) are linearly independent, we must have that ai(λk− λi) = 0. But λk 6= λi, so ai= 0 for each i. Looking back at where the ai
came from, we see that this implies that vk = 0. This contradicts the assumption that the vj were all nonzero.
So our assumption that the list was linearly dependent was absurd, hence the list is linearly independent.
A corollary of this theorem is that if V is n dimensional and L : V → V has n distinct eigenvalues, then the eigenvectors of L form a basis of V . The matrix of the operator with respect to this basis is diagonal.
62 Python
We can find eigenvectors in Python.
Let’s suppose I have an n × n matrix M , expressed in Python as a list of lists. For example, suppose
M =
6 4 3 4 5 2 3 2 7
= [[6,4,3],[4,5,2],[3,2,7]].
Further suppose that the matrix M = (mij) is symmetric, meaning that mij = mji. I’d like to compute an eigenvector of M quickly.
Question 1 Here’s a procedure that I’d like you to code in Python:
(a) Start with some vector ~v.
(b) Replace ~v with the result of applying the linear map LM to the vector ~v.
(c) Normalize ~v so that it has unit length.
(d) Repeat many times.
You can try print eigenvector([[6,4,3],[4,5,2],[3,2,7]]) to see what happens in the case of the matrix above.
Solution
Python
1 def eigenvector(M):
2 # start with a random vector v
3 v = [1] * len(M[0])
4 # for many, many times
5 # replace v with Mv
6 # normalize v
7 # return v
8
9 def validator():
10 v = eigenvector([[6, 5, 5], [5, 2, 3],[5, 3, 8]])
11 if abs((v[1] / v[0]) - 0.6514182851) > 0.01:
12 return False
13 if abs((v[2] / v[0]) - 1.0603152077) > 0.01:
14 return False
15 return True
Can you use your program to find, numerically, an eigenvector of the matrix M ?