Linear maps respect addition and scalar multiplication.
We begin by defining linear maps.
Definition 1 A function L : Rn → Rm is called a linear map if it “respects addition and scalar multiplication.”
Symbolically, for a map to be linear, we must have that L(~v + ~w) = L(~v) + L( ~w) for all ~v, ~w ∈ Rn and also L(a~v) = aL(~v) for all a ∈ R and ~v ∈ Rn.
Definition 2 Linear Algebra is the branch of mathematics concerning vector spaces and linear mappings between such spaces.
Question 3 Which of the following functions are linear?
Solution
Hint: For a function to be linear, it must respect scalar multiplication. Let’s see how f
51
1
compares to 5f1 1
compares to 5h1 1
.
Question 4 Solution
Hint: Remember f is defined by fx y
Hint: Remember f is defined by fx y
Solution Is f
12 Linear maps
Hint: Remember h is defined by hx y
Solution Is h
Great! So h is not linear: by looking at this particular example, we can see that h does not always respect scalar multiplication. So h is not linear.
Since we know one of the two functions is linear, we can already answer the question:
The answer is f . To be thorough, lets check that f really is linear.
First we check that f really does respect scalar multiplication:
Let a ∈ R be an arbitrary scalar andx y
∈ R2 be an arbitrary vector. Then
f
Now we check that f really does respect vector addition:
Letx1
be arbitrary vectors in R2. Then
fx1
This proves that f is linear!
(a) f : R2→ R1 defined by fx
What about these two functions? Which of them is a linear map?
Solution
12 Linear maps
Hint: For a function to be linear, it must respect scalar addition. Let’s see how h(5+2) compares to h(5)+h(2) and also how g
Question 5 Solution
Hint: Remember h is defined by h(x) =
Hint: Remember h is defined by h(x) =
Great! So h has a chance of being linear, since it is respecting vector addition in this case. What about g?
Solution
Hint: Remember g is defined by g
12 Linear maps
Solution Is g
Great! So g is not linear: by looking at this particular example, we can see that g does not always respect vector addition. So g is not linear.
Since we know one of the two functions is linear, we can already answer the question:
The answer is h. To be thorough, lets check that h really is linear.
First we check that h really does respect scalar multiplication:
Let a ∈ R be an arbitrary scalar and x ∈ R be an arbitrary vector. Then
h (ax) =
Now we check that h really does respect vector addition:
Let x and y be arbitrary vectors in R1. Then This proves that h is linear!
(a) g : R3→ R2 defined by g
12 Linear maps
And finally, which of the following functions are linear?
Solution
Hint: For a function to be linear, it must respect scalar multiplication. Let’s see how
A
22
3
compares to 2A2 3
compares to 2G
Question 6 Solution
Hint: Remember A is defined by Ax y
Hint: Remember A is defined by Ax y
Solution Is A
Great! So A has a chance of being linear, since it is respecting vector addition in this case. What about G?
Solution
Hint: Remember G is defined by G
12 Linear maps
Hint: Remember G is defined by G
Solution Is G
Great! So G is not linear: by looking at this particular example, we can see that G does not always respect scalar multiplication. So G is not linear.
Since we know one of the two functions is linear, we can already answer the question:
The answer is A. To be thorough, lets check that A really is linear.
First we check that A really does respect scalar multiplication:
Let c ∈ R be an arbitrary scalar andx y
∈ R2 be an arbitrary vector. Then
A
Now we check that A really does respect vector addition:
Letx1
be arbitrary vectors in R2. Then
12 Linear maps
This proves that A is linear!
(a) G : R4→ R3 defined by G
Warning 7 Note that the function which sends every vector to the zero vector is linear.
Hint: The only thing we know about linear maps is that they respect scalar multi-plication and vector addition. So we need to somehow rewrite the vector
12 Linear maps
Hint: Consider the coefficient on
Now using the linearity of L, we can see that
L
Can you finish off the computation?
Hint:
Can you generalize this?
Solution
12 Linear maps
Hint: The only thing we know about linear maps is that they respect scalar multi-plication and vector addition. So we need to somehow rewrite the vector
of the vectors
, scalar multiplication, and vector addition, to exploit what we know about L.
Question 10 Can you rewrite
Hint: Now using the linearity of L, we can see that
L Can you finish off the computation?
Hint:
12 Linear maps
As you have already discovered a linear map L : Rn→ Rmis fully determined
by its action on the “standard basis vectors” e1=
compute L(~v). So L is completely determined once I know what it does to each of the standard basis vectors.
1
1YouTube link: http://www.youtube.com/watch?v=8BFsz1FCdxM
13 Matrices
Matrices are a way to represent linear maps.
To make writing a linear map a little less cumbersome, we will develop a com-pact notation for linear maps using our previous observation that a linear map is determined by its action on the standard basis vectors.
Definition 1 An m × n matrix is an array of numbers which has m rows and n columns. The numbers in a matrix are called entries.
When A is a matrix, we write A = (aij), meaning that ai,j is the entry in the ith row and jth column of the matrix. Note: We start counting with 1 not 0. So the upper lefthand entry of the matrix is a1,1.
Question 2 The matrix A =
1 −1
2 4
3 −5
is an n × m matrix.
Solution
Hint: Note that this is n × m whereas the definition above used m × n.
Hint: n is the number of rows, and m is the number of columns
Hint: n = 3 and m = 2 In this case, n is 3.
Solution And m is 2.
Remember, we write ai,j for the entry in the ith row and jth column of the matrix.
Solution
Hint: a3,2is the entry in the 3rd row and the 2ndcolumn.
Hint: a3,2= −5
13 Matrices
called the matrix of the linear map with respect to the standard coordinates. It is defined by setting ai,j to be the ith component of L(ej). In other words, the jth column of the matrix AL is the vector L(ej).
Going the other way, we likewise associate to each matrix m × n matrix M a linear map LM : Rn → Rmby requiring that L(ej) be the jthcolumn of the matrix
Hint: Remember that, by definition, the first column of this matrix should be L1 0
and the second column should be L0 1
.
Hint: The matrix of L is
Let’s do another example.
Question 6 Suppose L is a linear map represented by the matrix A =
Hint: A should have one column for each basis vector of the domain.
Hint: A has 2 columns, so the dimension of the domain is 2.
The dimension of the domain of L is 2.
Solution
Hint: Each column of A is the image of a basis vector under the action of L
13 Matrices
Hint: Since the columns are of length 3, that means L is spitting out vectors of length 3.
Hint: The codomain of L is R3which is 3 dimensional.
The dimension of the codomain of L is 3.
Suppose ~v = L0 1
. What is ~v?
Solution
Hint: Remember that, by definition, the ith column of A is L(~ei).
Hint: So, by definition, L0 1
is the second column of the matrix A.
Hint: So L0
Hint: By definition of the matrix associated to a linear map, we know that L1 0
Hint: Can you rewrite4 5
so that you can use the linearity of L to compute L4
13 Matrices
Hint: By definition of the matrix associated to a linear map, we know that L1 0
Hint: Can you rewritex y
so that you can use the linearity of L to compute L4
As an antidote to the abstraction, let’s take a look at a simplistic “real world”
example.
Question 7 In the local barter economy, there is an exchange where you can
• trade 1 spoon for 2 apples and 1 orange,
• trade 1 knife for 2 oranges, and
• trade 1 fork for 3 apples and 4 oranges.
Model this as a linear map from L : R3 → R2, where the coordinates on R3 are
and the coordinates on R2 are apples oranges
.
13 Matrices
Solution
Hint: Remember the matrix of a linear map is defined by the fact the the kth column of the matrix is the image of the kth standard basis vector.
Hint:
represents one spoon in the codomain. Its image under this linear map is 2 apples and 1 orange, which is represented by the vector2
1
in the codomain. So the first column of the matrix should be2
1
Hint: The full matrix is
2 0 3 1 2 4
What is the matrix of the linear map L?
Solution So you would be able to get 18 apples and 19 oranges.
13 Matrices
Prove the following statement: if S : Rn → Rm and T : Rn → Rm are both linear maps, then the map (S + T ) : Rn → Rmdefined by (S + T )(~v) = S(~v) + T (~v) is also linear.
We need to check that (S + T ) respects both scalar multiplication and vector addition.
Scalar multiplication:
Choose and arbitrary scalar c ∈ R and an arbitrary vector ~v ∈ Rn. Then (S + T )(c~v) = S(c~v) + T (c~v) by definition of (S + T )
= cS(~v) + cT (~v) by the linearity of S and T
= c (S(~v) + T (~v)) by the distributivity of scalar multiplication over addition in Rm
= c(S + T )(~v) by definition of (S + T )
Vector addition: Choose two arbitrary vectors ~v and ~w in Rn. Then
(S + T )(~v + ~w) = S(~v + ~w) + T (~v + ~w) by definition of S + T
= S(~v) + S( ~w) + T (~v) + T ( ~w) by the linearity of S and T
= S(~v) + T (~v) + S( ~w) + T ( ~w) by the commutativity of vector addition in Rm
= (S + T )(~v) + (S + T )( ~w) by the definition of S + T.
Prove that if T : Rn→ Rm is a linear map and c ∈ R is a scalar, then the map cT : Rn→ Rm, defined by
(cT )(~v) = cT (~v) is also a linear map.
We need to check that cT respects both scalar multiplication and vector addi-tion.
Scalar multiplication:
Choose and arbitrary scalar a ∈ R and an arbitrary vector ~v ∈ Rn. Then (cT )(a~v) = cT (a~v)
= acT (~v)
= a(cT )(~v)
Vector addition: Choose two arbitrary vectors ~v and ~w in Rn. Then
(cT )(~v + ~w) = cT (~v + ~w)
= c (T (~v) + T ( ~w))
= cT (~v) + cT ( ~w)
= (cT )(~v) + (cT )( ~w)
Observation 8 The last two exercises show that we have a nice way to both add linear maps and multiply linear maps by scalars. So linear maps themselves
“feel” a bit like vectors. You do not have to worry about this now, but we will see that the linear maps from Rn→ Rmform an “abstract vector space.” Much of the power of linear algebra is that we can apply linear algebra to spaces of linear maps!
14 Composition
The composition of linear maps can be computed with matrices.
Prove that if S : Rn → Rmis a linear map, and T : Rm→ Rk is a linear map, then the composite function T ◦ S : Rn→ Rk is also linear.
We need to show that T ◦ S respects scalar multiplication and vector addition:
Scalar multiplication: For every scalar a ∈ R and every vector ~v ∈ Rn, we have:
(T ◦ S)(a~v) = T (S(a~v))
= T (aS(~v)) because S respects scalar multiplication
= aT (S(~v)) because T respects scalar multiplication
= a(T ◦ S)(~v)
Vector addition: For every two vectors ~v, ~w ∈ Rn, we have:
(T ◦ S)(~v + ~w) = T (S(~v + ~w))
= T (S(~v + S( ~w)))because S respects vector addition
= T (S(~v)) + T (S( ~w))because T respects vector addition
= (T ◦ S)(~v) + (T ◦ S)( ~w)
Question 1 Suppose the matrix of S is MS = 2 0 −1
−1 1 1
and the matrix of
T is MT =
−1 −1
0 2
−1 1
.
Solution
Hint: Remember that the matrix for S ◦ T will have columns given by (S ◦ T )1 0
and (S ◦ T )0 1
Hint: Question 2 Solution Hint:
14 Composition
What is (S ◦ T )1 0
?
Question 3 Solution Hint:
because by definition, T0 1
is the second column of the matrix of T
= −1S
Hint: Remember that the matrix for T ◦S will have columns given by (T ◦S)
Hint: Question 4 Solution Hint:
because by definition, S
is the first column of the matrix of S
= 2T1
by the linearity of T
= 2
14 Composition
Question 5 Solution Hint:
because by definition, S
is the first column of the matrix of S
=
we got lucky: by definition T0 1
is the second column of the matrix of T
What is (T ◦ S)
Question 6 Solution Hint:
because by definition, S
is the third column of the matrix of S
= −1T1
by the linearity of T
= −1
14 Composition
Definition 7 If M is a m × n matrix and N is a k × m matrix, then the product N M of the matrices is defined as the matrix of the composition of the linear maps defined by M and N .
In other words, N M is the matrix of LN◦ LM.
Warning 8 You may have seen another definition for matrix multiplication in the past. That definition could be seen as a shortcut for how to compute the product, but it is usually presented devoid of mathematical meaning.
Hopefully our definition seems properly motivated: matrix multiplication is just what you do to compose linear maps. We suggest working out the problems here using our definition: you will develop your own efficient shortcuts in time.
You have already multiplied two matrices, even though you didn’t know it, above. Take some time now to get a whole lot of practice. You do not need us to prompt you: invent your own matrices and try to multiply them, on paper.
What condition is needed on the rows and columns of the two matrices for matrix multiplication to even make sense? You can check your work using a computer algebra system, like SAGE1or you can use a free web hosted app like Reshih2. Use our definition, and think through it each time. Try to get faster and more efficient.
Eventually you should be able to do this quite rapidly.
Question 9 Suppose B =1 2 3 4
. Find a 2 × 2 matrix A so that AB 6= BA. Play around! Can you find more than one?
Solution
Hint: There is no systematic way to answer this question: you just have to play around, and see what you discover!
Hint: Question 10 Solution Hint: 1 2
3 4
1 0 0 0
=1 0 3 0
What is1 2 3 4
1 0 0 0
?
Question 11 Solution Hint: 1 0
0 0
1 2 3 4
=1 2 0 0
What is1 0 0 0
1 2 3 4
?
A matrix that doesn’t commute with B is
1http://www.sagemath.org/
2http://matrix.reshish.com/
14 Composition
Question 12 Solution
Hint: Try some simple matrices. Maybe limit yourself to 2 × 2 matrices?
Hint: One simple linear map which would work is Lx y
=y 0
. Applying this twice to any vector would give you the zero vector. This linear map is great for cooking up counterexamples to all sorts of naive things you might think about matrices! See this Mathoverflow answer3 (you will understand more and more of these terms as the course progresses).
Question 13 Hint: The matrix of L is0 1 0 0
What is the matrix of the example linear map L?
Find A 6= 0 with AA = 0. (Note: such a matrix is called “nilpotent”)
Question 14 If A =2 8 3 12
, find v 6= 0 with Av = ~0.
Solution
Hint: Let ~v =x y
, and solve a system of equations
Hint:
A(~v) = ~0
2 8 3 12
x y
=0 0
2x + 8y 3x + 12y
=0 0
Hint: Both of these conditions (2x + 8y = 0 and 3x + 12y = 0) are saying the same
14 Composition
Hint: Let ~v =x y
and solve a system of equations.
Hint:
A~v =0 8
1 3 2 4
x y
=0 8
x + 3y 2x + 4y
=0 8
Hint:
(x + 3y = 0 2x + 4y = 8 (x + 3y = 0
x + 2y = 4 (x + 3y = 0
y = −4 (x = 12
y = −4
In the last two exercises, you found that solving matrix equations is equivalent to solving systems of linear equations.
Question 16 Rewrite
(4x + 7y + z = 3
−x + 8y − z = 2 as A
x y z
=3 2
.
Solution
Hint: A = 4 7 1
−1 8 −1
15 Python
Build up some linear algebra in python.
Exercise 1 We will store a vector as a list. So the vector
1 2 3
will be stored as [1,2,3]. Let’s try to write some Python code for working with lists as if they were vectors.
Solution
Hint: This was discussed on http: // stackoverflow. com/ questions/ 14050824/ add-sum-of-values-of-two-lists-into-new-list StackOverflow.
Write a “vector add” function. Your function may assume that the two vectors have the same number of entries.
Python
1 # write a function vector_sum(v,w) which takes two vectors v and w,
2 # and returns the sum v + w.
3 #
4 # For example, vector_sum([1,2], [4,1]) equals [5,3]
5 #
6
7 def vector_sum(v,w):
8 # your code here
9 return # the sum v+w
10
11 def validator():
12 # It would be better to try more cases
13 if vector_sum([-5,23],[10,2])[0] != 5:
14 return False
15 if vector_sum([1,5,6],[2,3,6])[1] != 8:
16 return False
17 return True
18
Solution
Hint: Try a Python “list comprehension”
15 Python
9
10 def validator():
11 # It would be better to try more cases
12 if scale_vector(-3,[2,3,10])[1] != -9:
13 return False
14 if scale_vector(10,[4,3,2,1])[2] != 20:
15 return False
16 return True
17
Let’s write a dot product function.
Solution
Python
1 # Write a function dot_product(v,w) which takes two vectors v and w,
2 # and returns the dot product of v and w.
3 #
4 # For example, dot_product([1,2],[0,3]) is 6.
5
6 def dot_product(v,w):
7 # your code here
8 return # the dot product "v dot w"
9
10 def validator():
11 if dot_product([1,2],[-3,5]) != 7:
12 return False
13 if dot_product([0,4,2],[2,3,-7]) != -2:
14 return False
15 return True
And we will store a matrix as a list of lists. For example the list [[1,3,5],[2,4,6]]
will represent the matrix
1 3 5 2 4 6
.
Note that there are two different conventions that we could have chosen: the in-nermost lists could be the rows, or the columns. There are good reasons to have chosen the opposite convention: after all, when thinking of a matrix as a linear map, we should be paying attention to the columns, since the ith column tells us what the corresponding linear map does when applied to ~ei.
Nevertheless, the innermost lists are rows in our chosen representation.
This way, to talk about the entry mij, we write m[i][j]. Had we made the other choice, the mijentry would have been accessed by writing j and i in the other order.
This is also the same convention used by the computer algebra system, Sage.
Exercise 2 Write a “matrix multiplication” function.
Solution
15 Python
Python
1 # write a function multiply(A,B) which takes two matrices A and B stored in the above format,
2 # and returns the matrix of their product
3
4 def multiply(A,B):
5 # your code here
6 return # the product AB
7
8 def validator():
9 # It would be better to try more cases
10 a = [[-2, 0], [-2, -3], [-1, 3]]
11 b = [[-3, 2, -1, -2], [3, 2, 1, 3]]
12 result = multiply(a,b)
13 if (len(result) != 3):
14 return False
15 if (len(result[0]) != 4):
16 return False
17 if (result[2][1] != 4):
18 return False
19 return True Fantastic!
Next, let’s think more about how matrices and linear maps are related.
Solution Hint:
Warning 3 This is a function whose output is a function.
Hint: Try using lambda.
Write a function matrix_to_function which takes a matrix MLrepresenting the linear map L, and returns a Python function. The returned Python function should take a vector
~
v and send it to L(~v).
Python
1 # For example, if M = [[1,2],[3,4]], then matrix_to_function(M)([0,1]) should be [2,4]
2
3 def matrix_to_function(M):
15 Python
Solution Now let’s go the other way. Write a function function_to_matrix which takes a Python function f—assumed to be a linear map from R2 to R2—and returns the 2 × 2 matrix representing that linear map.
Python
1 # For example if you had defined
2 #
3 # def L(v):
4 # return [2*v[0]+3*v[1], -4*v[0]]
5 #
6 # Then function_to_matrix(L) is
7
8 # You may assume that L takes [x,y] to another list with two entries
9 # and you may assume that L is linear
10
11 def function_to_matrix(L):
12 #your code here
13 return # the matrix
14
15 def validator():
16 M = function_to_matrix( lambda v: [3*v[0]+5*v[1], -2*v[0] + 4*v[1]] )
17 if (M[0][0] != 3):
18 return False
19 M = function_to_matrix( lambda v: [2*v[0]-3*v[1], -7*v[0] - 5*v[1]] )
20 if (M[1][0] != -7):
21 return False
22 M = function_to_matrix( lambda v: [v[0]+7*v[1], 3*v[0] - 2*v[1]] )
23 if (M[1][1] != -2):
24 return False
25 return True
Great work! If you like, you can try to compute function_to_matrix(matrix_to_function(M)).
You should get back M .