The related notions of a basis and of linear independence are funda- mental in linear algebra.
3.4.1
Linear Independence and Bases
As we shall see shortly, the dimension of a finite-dimensional vector space will be defined as the number of members of any basis. Obviously, we first need to see what a basis is, and then to convince ourselves that if a vector spaceV has a basis withN members, then every basis forV has
N members.
Definition 3.4 Thespanof a collection of vectors{u1, ..., uN}inV is the set of all vectors x that can be written as linear combinations of the un; that is, for which there are scalars c1, ..., cN, such that
x=c1u1+...+cNuN. (3.6) Definition 3.5 A collection of vectors{w1, ..., wN} inV is called aspan-
ning setfor a subspace W if the set W is their span.
Definition 3.6 A subspaceW of a vector spaceV is called finite dimen- sionalif it is the span of a finite set of vectors fromV. The whole spaceV
The assertion in the following proposition may seem obvious, but the proof, which the reader is asked to supply as Exercise 3.12, is surprisingly subtle.
Proposition 3.3 LetV be a finite dimensional vector space andW a sub- space ofV. ThenW is also finite dimensional.
Ex. 3.12 Prove Proposition 3.3.
This definition tells us what it means to be finite dimensional, but does not tell us whatdimensionmeans, nor what the actual dimension of a finite dimensional subset is; for that we need the notions oflinear independence
andbasis.
Definition 3.7 A collection of vectors U ={u1, ..., uN} in V is linearly
independent if there is no choice of scalars α1, ..., αN, not all zero, such that
0 =α1u1+...+αNuN. (3.7) Ex. 3.13 Show that the following are equivalent:
• 1. the set U ={u1, ..., uN} is linearly independent;
• 2.u16= 0and noun is a linear combination of the members ofU that precede it in the list;
• 3. noun is a linear combination of the other members of U.
Definition 3.8 A collection of vectors U = {u1, ..., uN} in V is called a
basis for a subspace W if the collection is linearly independent and W is their span.
Ex. 3.14 Show that
• 1. if U ={u1, ..., uN} is a spanning set forW, thenU is a basis for
W if and only if, after the removal of any one member,U is no longer a spanning set for W; and
• 2. if U ={u1, ..., uN} is a linearly independent set in W, then U is a basis for W if and only if, after including in U any new member from W,U is no longer linearly independent.
Ex. 3.15 Prove that every finite dimensional vector space that is not just the zero vector has a basis.
3.4.2
Dimension
We turn now to the task of showing that every basis for a finite dimen- sional vector space has the same number of members. That number will then be used to define the dimension of that space.
Suppose thatW is a subspace ofV, thatW={w1, ..., wN}is a spanning
set for W, and U = {u1, ..., uM} is a linearly independent subset of W.
Beginning with w1, we augment the set {u1, ..., uM} with wj if wj is not
in the span of theum and the wk previously included. At the end of this
process, we have a linearly independent spanning set, and therefore, a basis, for W (Why?). Similarly, beginning withw1, we remove wj from the set
{w1, ..., wN} if wj is a linear combination of the wk, k = 1, ..., j−1. In
this way we obtain a linearly independent set that spansW, hence another basis forW. The following lemma will allow us to prove that all bases for a subspaceW have the same number of elements.
Lemma 3.1 Let W={w1, ..., wN} be a spanning set for a subspaceW of
V, andU ={u1, ..., uM}a linearly independent subset ofW. ThenM ≤N. Proof:Suppose thatM > N. LetB0=W={w1, ..., wN}. To obtain the
setB1, form the setC1={u1, w1, ..., wN}and remove the first member of
C1 that is a linear combination of members ofC1 that occur to its left in
the listing; sinceu1 has no members to its left, it is not removed. SinceW
is a spanning set,u16= 0 is a linear combination of the members ofW, so
that some member ofW is a linear combination ofu1and the members of
W to the left of it in the list; remove the first member ofW for which this is true.
We note that the set B1 is a spanning set forW and hasN members.
Having obtained the spanning setBk, withN members and whose first k
members are uk, ..., u1, we form the set C
k+1 =Bk∪ {uk+1}, listing the
members so that the firstk+ 1 of them are{uk+1, uk, ..., u1}. To get the set
Bk+1 we remove the first member ofCk+1 that is a linear combination of
the members to its left; there must be one, sinceBk is a spanning set, and
so uk+1 is a linear combination of the members of Bk. Since the set U is
linearly independent, the member removed is from the setW. Continuing in this fashion, we obtain a sequence of spanning setsB1, ..., BN, each with
N members. The set BN is BN = {uN, ..., u1} and uN+1 must then be
a linear combination of the members ofBN, which contradicts the linear
independence ofU.
Corollary 3.1 Every basis for a subspaceW has the same number of ele- ments.
Definition 3.9 Thedimension of a subspace W, denoted dim(W), is the number of elements in any basis.
Ex. 3.16 Let V be a finite dimensional vector space and W any subspace ofV. Show that dim(W)cannot exceed dim(V).
3.4.3
Rank of a Matrix
We rely on the following lemma to define the rank of a matrix.
Lemma 3.2 For any matrixA, the maximum number of linearly indepen- dent rows equals the maximum number of linearly independent columns.
Proof: Suppose that A is an M by N matrix, and that K ≤ N is the maximum number of linearly independent columns ofA. SelectKlinearly independent columns ofAand use them as theK columns of anM byK
matrixU. Since every column ofAmust be a linear combination of these
K selected ones, there is aK byN matrixB such thatA=U B; see the discussion that follows Exercise 3.5. From A† = B†U† we conclude that every column ofA† is a linear combination of theKcolumns of the matrix
B†. Therefore, there can be at mostKlinearly independent columns ofA†.
Definition 3.10 TherankofA, written rank(A), is the maximum number of linearly independent rows or of linearly independent columns ofA. Ex. 3.17 Letuandvbe two non-zeroN-dimensional complex column vec- tors. Show that the rank of theN by N matrixuv† is one.
Ex. 3.18 Show that the rank of a matrix C = AB is never greater than the smaller of the rank ofAand the rank ofB. Can it ever be strictly less than the smaller of these two numbers?
Ex. 3.19 Show that rank(A+B) is never greater than the sum of rank(A) and rank(B).
Definition 3.11 An M by N matrix A is said to havefull rank or to be afull-rank matrixif the rank of A is the minimum ofM andN.
Proposition 3.4 A square matrix is invertible if and only if it has full rank.
Ex. 3.20 Prove Proposition 3.4.
Corollary 3.2 A square matrix A is invertible if and only if there is a matrixB such that AB=I.
Corollary 3.3 A square matrix A is invertible if and only if there is a matrixGsuch thatAG is invertible.
Corollary 3.4 IfAandB are square matrices andC=ABis invertible, then bothAandB are invertible.
Definition 3.12 AnM byN matrixAis said to haveleft inverseB ifB
is an N by M matrix such that BA = IN, the N by N identity matrix. Similarly, A is said to have a right inverse C if C is anN by M matrix such thatAC=IM, the M by M identity matrix.
Ex. 3.21 Let Abe an M byN matrix. When does A have a left inverse? When does it have a right inverse?
Ex. 3.22 Let A and B be M by N matrices, P an invertible M by M
matrix, and Qan invertibleN by N matrix, such thatB=P AQ, that is, the matrices AandB areequivalent. Show that the rank ofB is the same as the rank ofA. Hint: show thatA andAQhave the same rank.