Lecture2

Chapter 2

Vectors

This chapter is about vectors of real numbers. The set of real numbers, denoted R is characterized by following properties, known as Field Axioms, that allow two operators, sum (+ ) and product (_·):

• Closure with respect to addition: For any two real numbers x and y, x+y is also a real number.

• Closure with respect to multiplication: For any two real numbersxand y, xy is also a real number.

• Commutative operations: For any two real numbers x and y, x+y = y+x and xy =yx.

• Associative operations: For any three real numbersx,yand z,x+ (y+ z) = (x+y) +z and x(yz) = (xy)z.

• Distributive operations: For any three real numbers x, yand z, x(y+ z) =xy+xz.

• Existence of 0 and 1: There exist real numbers x and y such that for any real number z, z+x=z and zy=z.

• Existence of negative: For any real numberx, there exists a real number x0 _{such thatx+x}0 _{= 0.}

• Existence of reciprocal: For any real number x₆= 0, there exists a real number y such thatxy= 1.

Most common results of algebra can be derived from the properties given above.

16 CHAPTER 2. VECTORS

2.1

Vectors and Vector Space

Ann-vector of real numbers can be defined as follows.

Definition 2.1. An n- vector is an ordered n-tuple of real numbers, i.e., an element of the Cartesian product R_{⇥ · · ·}n times_{· · ·⇥}R = Rn_{. We will}

denote a vector in boldface, e.g.,x, whosei-th component is xi.

Example 2.1.1. Following are some special n-vectors:

• Sum vector: (1, . . . ,1

| {z }

ntimes );

• Zero or Null vector: (0, . . . ,0

| {z }

ntimes );

• i-th unit vector, denoted ei_{: (0, . . . ,0}

| {z }

i 1 times

1,0, . . . ,0

| {z }

n itimes ).

We define the following operations on the set of all vectors:

• Equality: Vectors xand y are equal if for all i= 1, . . . , n, xi =yi.

• Addition: For any two vectors x and y, there exists another vector z such that for alli= 1, . . . , n,zi =xi+yi. This vector is called the sum

of x and y, denoted x+y.

• Scalar multiplication: For any vector x and any real number c, there exists another vector w such that for all i = 1, . . . , n, wi = cxi. This

vector is called the c-multiple of x, denoted cx.

Definition 2.2 (Vector Space). The set of all vectors with real components along with equality, addition and scalar multiplication operators, defined as above, is called a vector space.

Since the components of vectors are real numbers, it follows that

• Vector addition is commutative: For all x, y,x+y=y+x.

• Vector addition is associative: For all x,y,z,x+ (y+z) = (x+y) +z.

• Scalar multiplication is associative: For all real numbers a, b and any vector x,a(bx) = (ab)x.

Vectors inR2_andR3_{have nice geometrical representations: vector (x1, x2)} is represented in the XY-plane by an arrow originating from (0,0) and point-ing at (x1, x2) or any parallel arrow of the same size and direction. Similarly, vector (x1, x2, x3) in the 3-dimensional space, is represented by an arrow orig-inating from (0,0,0) and pointing at (x1, x2, x3) or any parallel arrow of the same size and direction. Two vectors are equal if they have the same size and direction. A pair of vectors are added by placing the origin of one at the tip of the other: the vector connecting the tip of the first to the origin of the second is the corresponding sum. A scalar multiple of a vector is obtained by expanding or contracting the original vector such that the scalar equals the ratio of the size of the new vector to the original one.

2.2

Linear Dependence

Definition 2.3 (Linear Dependence). A set of vectors x1_{, . . . ,x}m _{is linearly}

dependent if there exist scalars 1, . . . , m, not all zero, such that 1x1 +

· · ·+ mxm = 0, where the right hand side is the zero vector.

Definition 2.4(Linear Independence). A set of vectorsx1_{, . . . ,x}m_{is linearly}

independent if it is not linearly dependent.

Example 2.2.1. _• The vectors (1,2) and ( 5, 10) in R2 _{are linearly} dependent: 5(1,2) + ( 5, 10) = (0,0).

• The vectors (1,3) and (0,1) are linearly independent: for any and 0_,

(1,3) + 0(0,1) = ( ,3 + 0) = (0,0) if and only if = 0 = 0.

• The vectors (1,1), (2,3) and (3,7) are linearly dependent: 4(2,3) 5(1,1) (3,7) = 0.

• The vectors (1,0) and (0,1) are linearly independent: for any and 0_,

(1,0) + 0(0,1) = ( , 0) = (0,0) if and only if = 0 = 0.

Exercise 2.2.1. Show that any set of vectors containing the zero vector is linearly dependent.

Exercise 2.2.2. Show that any set of vectors containing a subset of linearly dependent vectors is linearly dependent.

Exercise 2.2.3. Show that the set ofn distinct unit vectorsei_{,i= 1, . . . , n,}

18 CHAPTER 2. VECTORS

Definition 2.5(Linear Combination). A vectorycan be written as a linear combination of the vectorsx1_{, . . . ,x}m_{, if there exist scalars 1, . . . ,}

m, such

that y= 1x1_{+· · ·+}

mxm.

Exercise 2.2.4. Show that any vector in Rn _{can be written as a linear}

combination of the unit vectorsei_{,i= 1, . . . , n.}

Theorem 2.1 (Fundamental Theorem on Vector Spaces). A set of m + 1 vectors y0_{, . . . ,y}m _{in R}n _{is linearly dependent if there exist m vectors}

x1_{, . . . ,x}m _{in R}n _{such that each y}j_{, j = 0, . . . , mcan be written as a linear}

combination ofxi_{, i= 1, . . . , m.}

Proof. By induction onm. Letm= 1: supposey0 ₌ 0_x1 _andy1 ₌ 1_x1_{. If} 0 _{= 0 theny}0 _{= 0 or} 1 _{= 0 theny}1 _{= 0, and the claim is true (see Exercise} 2.2.1). If neither 0 _{= 0 nor} 1 _{= 0, theny}0 0

1y

1 _{= 0 where} 0

1 6= 0. Thus,

y0 _{and y}1 _{are dependent. Now suppose the claim holds for m=k. We will} show that it holds for m = k + 1. Suppose yj ₌ j,1_x1 _{+· · ·+} j,k+1_xk+1_, j = 0, . . . , k+ 1. Note that either there exists at least one j and onei such that j,i_xi _{6= 0 or the claim is trivially true since it will imply thaty}j_{s are zero}

vectors. Without loss of generality, let 0,1_x1 _{6= 0. Consider thek+ 1 vectors} yr r,1

0,1y0 = ( r,2

r,1 0,2

0,1 )x2+· · ·+( r,k+1

r,1 0,k+1

0,1 )xk+1,r= 1, . . . , k+1:

each of these k+ 1 vectors can be written as a linear combination of the k vectors x2_{, . . . ,x}k+1_{. By the induction hypothesis, these k+ 1 vectors are} linearly dependent. But each of these k+ 1 vectors is a linear combination of y0 _{and a y}r_{, r= 1, . . . , k. Hence the claim.}

Exercise 2.2.5. Show that any set of n+ 1 vectors in Rn _{is linearly}

depen-dent.

Exercise 2.2.6. Show that any system of n homogeneous linear equations inn+ 1 unknowns

a10x0+· · ·+a1nxn= 0

... an0x0+· · ·+annxn= 0

has a non-zero solution.

2.3

Rank and Basis

Definition 2.6 (Rank). The rank of any set of vectors S⇢Rn _{is the}

Remark. Note that for any S⇢Rn_{, rank(S)n, since any set of n+ 1 or}

more vectors in Rn _{are linearly dependent (see Exercise 2.2.5).}

Definition 2.7 (Basis). For any set of vectorsS _⇢Rn_{, if rank(S) = r, then}

any set ofr linearly independent vectors inS is called a basis of S.

Exercise 2.3.1. Find the ranks of {(0,1),(1,0)} and {(0,0)}.

Exercise 2.3.2. Show thatRn _{has rank n.}

Exercise 2.3.3 (Basis Theorem). Supposex1_{,· · · ,x}m _{are linearly}

indepen-dent vectors inS _⇢Rn_{. Show that}

• If every vector inScan be expressed as a linear combination ofx1_{,· · · ,x}m_,

then (x1_{,· · · ,x}m_{) is a basis of S.}

• If (x1_{,· · · ,x}m_{) is a basis ofS, then every vector inS can be expressed}

as a linear combination of x1_{,· · · ,x}m_.

2.4

Inner Product and Norm

Definition 2.8 (Inner Product). The inner product of any pair of vectors x,y2Rn _{is a scalar given by xy=}P

ixiyi.

The inner product of vectors satisfies the following properties:

• Commutative property: xy=yx.

• Mixed Associative property: ( x)y= (xy).

• Distributive property: (x+y)z=xz+yz.

• xx= 0 if and only if x= 0.

Definition 2.9 (Euclidean Norm). The Euclidean Norm of any vector x ₂ Rn_{, denoted ||x||, is a scalar given by ||x||= (xx)}1_{2 = (}P

ix2i)

1 2.

The Euclidean Norm satisfies the following properties:

• ||x_||= 0 if and only if x= 0.

• || x||=| |||x||.

• ||x+y||||x||+||y|| (Triangle Inequality).

20 CHAPTER 2. VECTORS

An important property of vectors connecting the notions of inner product and norm is the Cauchy-Schwartz inequality.

Theorem 2.2 (Cauchy-Schwartz). For any two vectorsx and y inRn_,

|x_·y_|||x_||||y_||. (2.1)

Proof. Consider the vector ˆy=y x_x·_·y_xx. Notice thatx_·yˆ =x_·y x_x·_·y_xx_·x= x·y x·y= 0. Thus ˆyis orthogonal tox. Sincey= ˆy+x_x·_·y_xx, by Pythagorus theorem,

y·y= ˆy·yˆ+⇣x·y x_·x

⌘2 x·x

= ˆy·yˆ+(x·y) 2

x_·x (x_·y)2

x·x .

Multiplying both sides byx·x, we get

||x_||2_||y_||2 (x_·y)2.

The triangle inequality mentioned in the properties of the Euclidean norm can be obtained as a corollary of the Cauchy-Schwartz inequality.

Exercise 2.4.1. Use the Cauchy Schwartz inequality to prove the triangle inequality.

Definition 2.10 (Orthogonal vectors). A pair of vectorsx and yis orthog-onal if their inner product is zero, i.e.,xy = 0