Chapter 5: Inner Product Spaces

Full text

(1)Chapter 5: Inner Product Spaces. 1. Chapter 5: Inner Product Spaces SECTION A Introduction to Inner Product Spaces By the end of this section you will be able to • understand what is meant by an inner product space • give examples of inner product spaces • prove properties of inner product spaces • define the norm of a vector • prove properties of norms In Chapter 3 we introduced the idea of inner products for a Euclidean n – space. In this chapter we extend the concept of inner product to general vector spaces. This section is more difficult than the equivalent section (3B) in chapter 3 because we cannot visualise the vectors. A1 Definition of Inner Product How did we define the inner product in Chapter 3? It was defined as follows: ⎛ u1 ⎞ ⎛ v1 ⎞ ⎜ ⎟ ⎜ ⎟ u v Let u = ⎜ 2 ⎟ and v = ⎜ 2 ⎟ be vectors in \ n then the inner product of u and v ⎜# ⎟ ⎜#⎟ ⎜ ⎟ ⎜ ⎟ ⎝ un ⎠ ⎝ vn ⎠ denoted by u ⋅ v is (3.9) u ⋅ v = u1v1 + u2 v2 + u3v3 + " + un vn Remember the answer was a scalar not a vector. The inner product was named the dot product (also called the scalar product) in \ n . This is the usual (or standard) inner product in \ n but there are many other inner products in \ n . For the general vector space the inner product u ⋅ v is denoted by u, v rather than u ⋅ v . For the general vector space the definition of inner product is based on Proposition (3.11) of Chapter 3 and is given by: Definition (5.1). An inner product on a real vector space V is an operation which assigns to each pair of vectors, u and v, a unique real number u, v which satisfies the following axioms. for all vectors u, v and w in V and all scalars k. (i) u, v = v, u [Commutative Law] (ii) u + v, w = u, w + v, w. [Distributive Law]. (iii) k u, v = k u, v (iv) u, u ≥ 0 and we have u, u = 0 if and only if u = O A real vector space which satisfies these axioms is called a real inner product space. Note that evaluating , gives a real number (scalar) not a vector. Next we give some examples of inner product spaces..

(2) Chapter 5: Inner Product Spaces. 2. A2 Examples of Inner Product Example 1 Show that the Euclidean n space, \ n , with the dot product as defined in (3.9) above is indeed an inner product space. Solution See Proposition (3.11) of Chapter 3. Example 2. ⎛ 1 2⎞ Let V be the Euclidean 2 space, \ 2 , and A = ⎜ ⎟. ⎝ 2 5⎠ Show that u, w = uT Aw is an inner product for \ 2 . Solution. What are u and v equal to? They are vectors in \ 2 which means they can be written as ⎛ u1 ⎞ ⎛ w1 ⎞ u = ⎜ ⎟ and w = ⎜ ⎟ ⎝ u2 ⎠ ⎝ w2 ⎠. What is uT equal to? T. ⎛u ⎞ u is the vector u transposed, that is u = ⎜ 1 ⎟ = ( u1 u2 ) . How do we show ⎝ u2 ⎠ u, w = uT Aw is an inner product? T. T. By checking all 4 axioms of Definition (5.1) given above. Check (i): ⎛ 1 2 ⎞ ⎛ w1 ⎞ u, w = uT Aw = ( u1 u2 ) ⎜ ⎟⎜ ⎟ ⎝ 2 5 ⎠ ⎝ w2 ⎠ ⎛ w + 2 w2 ⎞ = ( u1 u2 ) ⎜ 1 ⎟ ⎝ 2 w1 + 5w2 ⎠ = u1 ( w1 + 2 w2 ) + u2 ( 2 w1 + 5w2 ) = u1w1 + 2u1w2 + 2u2 w1 + 5u2 w2 Going the other way we have ⎛ 1 2 ⎞ ⎛ u1 ⎞ w, u = w T Au = ( w1 w2 ) ⎜ ⎟⎜ ⎟ ⎝ 2 5 ⎠ ⎝ u2 ⎠. [ Matrix Multiplication ] [ Expanding ]. ⎛ u + 2u2 ⎞ = ( w1 w2 ) ⎜ 1 ⎟ ⎝ 2u1 + 5u2 ⎠ = w1 ( u1 + 2u2 ) + w2 ( 2u1 + 5u2 ). [ Matrix Multiplication ]. = u1w1 + 2u2 w1 + 2w2u1 + 5w2u2. [ Expanding ]. Comparing these two, u, w = uT Aw = u1w1 + 2u1w2 + 2u2 w1 + 5u2 w2 and. w, u = wT Au = u1w1 + 2u2 w1 + 2w2u1 + 5w2u2 , we have.

(3) Chapter 5: Inner Product Spaces. 3. u, w = w, u Hence part (i) of definition (5.1) is satisfied. Check (ii): ⎛ v1 ⎞ Let v = ⎜ ⎟ then we have ⎝ v2 ⎠ u + v, w = ( u + v ) Aw T. ⎛ u1 + v1 ⎞ ⎛ 1 2 ⎞ ⎛ w1 ⎞ =⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎝ u2 + v2 ⎠ ⎝ 2 5 ⎠ ⎝ w2 ⎠ ⎛ 1 2 ⎞ ⎛ w1 ⎞ = ( u1 + v1 u2 + v2 ) ⎜ ⎟⎜ ⎟ ⎝ 2 5 ⎠⎝ w2 ⎠ ⎛ w1 + 2 w2 ⎞ = ( u1 + v1 u2 + v2 ) ⎜ ⎟ ⎝ 2 w1 + 5w2 ⎠ = ( u1 + v1 )( w1 + 2 w2 ) + ( u2 + v2 )( 2 w1 + 5w2 ) T. = u1w1 + 2u1w2 + v1w1 + 2v1w2 + 2u2 w1 + 5u2 w2 + 2v2 w1 + 5v2 w2 Similarly by using the results of part (i) above we have u, w + v, w = u1w1 + 2u1w2 + 2u2 w1 + 5u2 w2 + v1w1 + 2v1w2 + 2v2 w1 + 5v2 w2 . . = u, w. = v, w. = u + v, w [ By Above] Part (ii) of definition (5.1) is satisfied. Check (iii): Need to check that k u, w = k u, w. k u, w = ( k u ) Aw = k ( uT ) Aw. ⎡⎣ Because k is a scalar therefore k T = k ⎤⎦. T. = k uT Aw ⎡⎣ We are given uT Aw = u, w ⎤⎦. = k u, w. Hence part (iii) is satisfied. Check (iv): Need to show u, u ≥ 0 and we have u, u = 0 if and only if u = O : ⎛ 1 2 ⎞ ⎛ u1 ⎞ u, u = uT Au = ( u1 u2 ) ⎜ ⎟⎜ ⎟ ⎝ 2 5 ⎠ ⎝ u2 ⎠ ⎛ u1 + 2u2 ⎞ = ( u1 u2 ) ⎜ ⎟ ⎝ 2u1 + 5u2 ⎠ = u1 ( u1 + 2u2 ) + u2 ( 2u1 + 5u2 ) = ( u1 ) + 2u1u2 + 2u2u1 + 5 ( u2 ) 2. = ( u1 ) + 4u1u2 + 5 ( u2 ) 2. We can rewrite the last line as. 2. 2.

(4) Chapter 5: Inner Product Spaces. 4. u, u = ( u1 ) + 4u1u2 + 5 ( u2 ) 2. 2. = ( u1 ) + 4u1u2 + 4 ( u2 ) + ( u2 ) . 2. 2. = ( u1 + 2 u2 ). 2. 2. = ( u1 + 2u2 ) + ( u2 ) ≥ 0 2. 2. [ Because we have square numbers]. Also u, u = ( u1 + 2u2 ) + ( u2 ) = 0 2. 2. ⇔. u1 = u2 = 0. This means that u = O . Hence part (iv) is fulfilled therefore all 4 axioms are satisfied, so we conclude that u, w = uT Aw is an inner product for \ 2 . Example 3 Let P2 be the quadratic polynomials. Let p = c0 + c1 x + c2 x 2 and q = d 0 + d1 x + d 2 x 2. be polynomials in P2 . Show that. p, q = c0 d0 + c1d1 + c2 d 2 defines an inner product on P2 . [For example if p = 2 + 3 x + 5 x 2 and q = 4 − x + 7 x 2 then. p, q = ( 2 × 4 ) + ( 3 × ( −1) ) + ( 5 × 7 ) = 40 ]. Solution. How do we show p, q = c0 d0 + c1d1 + c2 d 2 is an inner product on P2 ? Checking all 4 axioms of Definition (5.1). (i) Need to check p, q = q, p is true for p, q = c0 d0 + c1d1 + c2 d 2 : p, q = c0 d 0 + c1d1 + c2 d 2 = d 0 c0 + d1c1 + d 2 c2. ⎡ Remember order of multiplication ⎤ ⎢does not matter ⎥ ⎣ ⎦. = q, p. (ii). Need to check p + q, r = p, r + q, r :. Let r = e0 + e1 x + e2 x 2 and. p + q = ( c0 + d0 ) + ( c1 + d1 ) x + ( c2 + d 2 ) x 2. We have. p + q, r = ( c0 + d 0 ) + ( c1 + d1 ) x + ( c2 + d 2 ) x 2 , e0 + e1 x + e2 x 2 = ( c0 + d 0 ) e0 + ( c1 + d1 ) e1 + ( c2 + d 2 ) e2. = c0 e0 + d 0 e0 + c1e1 + d1e1 + c2 e2 + d 2 e2 Evaluating the other inner product p, r + q, r = c0 e0 + c1e1 + c2 e2 + d 0 e0 + d1e1 + d 2 e2 . = p, r. = q, r. = c0 e0 + d 0 e0 + c1e1 + d1e1 + c2e2 + d 2 e2 = p + q, r Hence part (ii) is satisfied.. [ Rearranging ] [ From Above].

(5) Chapter 5: Inner Product Spaces. 5. (iii) Need to check kp, q = k p, q :. k p, q = k ( c0 + c1 x + c2 x 2 ) , d 0 + d1 x + d 2 x 2. [Opening Brackets] [ Using the Definition of I.P.] [ Factorizing ]. = kc0 + kc1 x + kc2 x 2 , d 0 + d1 x + d 2 x 2 = kc0 d 0 + kc1d1 + kc2 d 2 = k ( c0 d 0 + c1d1 + c2 d 2 ). = k p, q Hence part (iii) is satisfied. (iv) Need to show p, p ≥ 0 and we have p, p = 0 if and only if p = O : p, p = c0 + c1 x + c2 x 2 , c0 + c1 x + c2 x 2 = c0 c0 + c1c1 + c2 c2 = ( c0 ) + ( c1 ) + ( c2 ) ≥ 0 2. 2. 2. Also p, p = ( c0 ) + ( c1 ) + ( c2 ) = 0 ⇔ 2. 2. 2. c0 = c1 = c2 = 0. If c0 = c1 = c2 = 0 then p = O . All 4 axioms are satisfied, therefore p, q = c0 d0 + c1d1 + c2 d 2 is an inner product on P2 . In the above example if we define p, q = ( c0 + d 0 ) + ( c1 + d1 ) + ( c2 + d 2 ) then p, q is not an inner product. Why not? Because axiom (iv) of Definition (5.1) fails, that is p, p ≥ 0 is false. For example if p = −1 − x − x 2 then. p, p = −1 − x − x 2 , − 1 − x − x 2 = ( −1 − 1) + ( −1 − 1) + ( −1 − 1) = −3 < 0 Hence p, p ≥ 0 [Not greater than or equal to zero]. Next we state an inner product on the vector space of matrices, M 22 . An example of an inner product on the vector space of matrices is the following: The trace of a matrix is the sum of its leading diagonal elements, that is ⎛a b ⎞ trace ⎜ ⎟=a+d ⎝c d⎠ Let M 22 be the vector space of 2 by 2 matrices and inner product on M 22 be defined by A, B = tr ( BT A ) where tr is the trace. This is an inner product on M 22 . There is a question on this inner product in Exercise 5a. If we define A, B = AB on M 22 then this is not an inner product. Why not? Because we do not have the commutative law, that is A, B = AB ≠ BA = B, A [Not Equal].

(6) Chapter 5: Inner Product Spaces. 6. There are many other examples of inner product spaces which you need to show in Exercise 5a. Next we move onto properties of inner products. A3 Properties of Inner Products Proposition (5.2). Let u, v and w be vectors in a real inner product space V and k be any scalar. We have the following: (i) u, O = O, v = 0 (ii) u, k v = k u, v (iii) u, v + w = u, v + u, w Proof of (i). We can write the zero vector as 0 ( O ) because 0 ( O ) = O . Using the axioms of definition (5.1) we have u, O = u , 0 ( O ) = 0 (O ) , u. ⎡⎣ By Part (i) of (5.1) which is u, v = v, u ⎤⎦. = 0 O, u. ⎡⎣ By Part (iii) of (5.1) which is k u, v = k u, v ⎤⎦. =0. Similarly O, v = 0 . ■ Proof of (ii). Since inner product is commutative, u, v = v, u , by axiom (i) of definition (5.1) therefore proof of part (ii) is straightforward. u, k v = k v , u ⎡⎣ By Part (i) of (5.1) which is u, v = v, u ⎤⎦ = k v, u. ⎡⎣ By Part (iii) of (5.1) which is k u, v = k u, v ⎤⎦. = k u, v. ⎡⎣ By Part (i) of (5.1) which is u, v = v, u ⎤⎦ ■. Proof of (iii). We have u, v + w = v + w , u = v, u + w , u = u, v + u, w. ⎡⎣ By Part (i) of (5.1) which is u, v = v, u ⎤⎦ ⎡ By Part (ii) of (5.1) which is ⎤ ⎢ ⎥ v + w , u = v, u + w , u ⎦ ⎣ ⎡⎣ By Part (i) of (5.1) which is u, v = v, u ⎤⎦. ■ A4 Norm or Length of a Vector Do you remember how the Euclidean norm was defined? The norm of a vector u in \ n was defined by u ⋅ u . The norm is defined in a similar manner for the general vector space V. Let u be a vector in V then the norm denoted by u is defined as (5.3). u =. u, u [Positive Root]. Note that for the general vector space we cannot use the dot product because that is.

(7) Chapter 5: Inner Product Spaces. 7. only defined for Euclidean space, \ n , and in this chapter we are examining inner products on general vector spaces. The norm of a vector u is a real number which gives the size of the vector u. 2 Generally to find the norm u it is easier to determine u = u, u and then take the square root of your result. As before a vector with norm 1 is called a unit vector. In the example below we apply the norm of a vector to vector spaces of continuous functions. Example 3 Let C [ 0, 1] be the vector space of continuous functions on the closed interval [ 0, 1] . Let f ( x ) = x and g ( x ) = x 2 − 1 be functions in C [ 0, 1] . Let the inner product on. C [ 0, 1] be given by 1. f , g = ∫ f ( x ) g ( x ) dx 0. Determine (i) f , g. (ii) f. (iv) f − g. (iii) g. Solution. 1. (i) Using f , g = ∫ f ( x ) g ( x ) dx with f ( x ) = x and g ( x ) = x 2 − 1 we have 0. 1. f , g = ∫ f ( x ) g ( x ) dx 0. 1. = ∫ x ( x 2 − 1) dx. ⎡⎣Substituting f ( x ) = x and g ( x ) = x 2 − 1⎤⎦. 0. 1. ⎡ x4 x2 ⎤ ⎡ 1 1 ⎤ 1 = ∫ ( x − x ) dx = ⎢ − ⎥ = ⎢ − ⎥ = − 4 ⎣ 4 2 ⎦0 ⎣ 4 2 ⎦ 0 1. 3. (ii). As stated above to find f it is easier to determine f. 2. = f , f and then take. the square root of your result: 2 f = f, f 1. = ∫ f ( x ) f ( x ) dx 0. 1. = ∫ xx dx. ⎡⎣Substituting f ( x ) = x ⎤⎦. 0. 1. = ∫ x 2 dx 0. 1. ⎡ x3 ⎤ 1 =⎢ ⎥ = ⎣ 3 ⎦0 3 What is f equal to?. ⎡⎣ Because xx = x 2 ⎤⎦.

(8) Chapter 5: Inner Product Spaces. 8. 1 1 which is f = . 3 3 (iii) Similarly we have 2 g = g, g Square root of f. 2. =. 1. = ∫ g ( x ) g ( x ) dx 0. 1. = ∫ ( x 2 − 1)( x 2 − 1) dx. ⎡⎣Substituting g ( x ) = x 2 − 1⎤⎦. 0. 1. = ∫ ( x 4 − 2 x 2 + 1) dx 0. 1. ⎡ x5 ⎤ x3 = ⎢ − 2 + x⎥ 3 ⎣5 ⎦0 1 2 8 = − +1 = 5 3 15 What is g equal to? We need to take the square root of (iv). (x. ⎡ Expanding ⎣. 2. − 1)( x 2 − 1) ⎤⎦. [ Integrating ]. 8 8 to find g . Hence g = . 15 15. Similarly to find f − g we first determine f − g. 2. and then take the square. root of our answer: 2. f − g = f − g, f − g 1. = ∫ ⎡⎣ f ( x ) − g ( x ) ⎤⎡ ⎦⎣ f ( x ) − g ( x ) ⎤⎦ dx 0. [Substituting ]. 1. = ∫ ⎡⎣ x − x 2 − 1⎤⎦ ⎡⎣ x − x 2 − 1⎤⎦ dx 0 =∫ =∫. 1 0. (x. 1 0. 2. (x. 4. − x3 − x − x3 + x 4 + x 2 − x + x 2 + 1) dx. [ Expanding ]. − 2 x 3 + 3x 2 − 2 x + 1) dx. [Simplifying ]. 1. ⎡ x5 ⎤ x4 x3 x2 = ⎢ − 2 + 3 − 2 + x⎥ 4 3 2 ⎣5 ⎦0. [ Integrating ]. 1. ⎡ x5 x 4 ⎤ ⎡1 1 ⎤ 7 = ⎢ − + x3 − x 2 + x ⎥ = ⎢ − + 1 − 1 + 1⎥ = ⎦ 10 ⎣5 2 ⎦0 ⎣ 5 2 Hence f − g =. 7 . 10. The distance between two vectors u and v is denoted as d ( u, v ) and is defined as (5.4). d ( u, v ) = u − v. In Example 4 above the distance between f and g was given by f − g = 7 10 ..

(9) Chapter 5: Inner Product Spaces. 9. Note that this does not mean there is a distance of 7 10 between the two graphs of f and g. It is the result of applying the definition of an inner product given above. Example 5 Determine norm p of the vector in P2 where the inner product is given by. p, q = c0 d0 + c1d1 + c2 d 2 Solution. By definition (5.3) above we have p =. 2. p, p but first we find p = p, p and. then take the square root: 2 2 2 p, p = c0 c0 + c1c1 + c2 c2 = ( c0 ) + ( c1 ) + ( c2 ) Therefore p =. ( c0 ) + ( c1 ) + ( c2 ) 2. p, p =. 2. 2. .. Some norms are not defined in terms of the inner product as the following example shows: ⎛ x1 ⎞ n Consider the Euclidean space \ and let x = ⎜⎜ # ⎟⎟ be a vector in \ n then the ⎜x ⎟ ⎝ n⎠ following are important norms in this space: x 1 = x1 + x2 + x3 + " + xn [This is called the one norm] 2. x2= x. ∞. 2. 2. x1 + x2 + x3 + " + xn. 2. [This is called the two norm]. ( ) where j = 1, 2, " , n [This is called the infinity norm] = max ( x ) where j = 1, 2, " , n means we select the maximum. = max x j. Note that x. ∞. j. absolute value out of x1 , x2 , x3 , " and xn . ⎛ −1 ⎞ ⎜ ⎟ 4 For example if we have the vector x = ⎜ ⎟ in \ 4 then ⎜ −9 ⎟ ⎜ ⎟ ⎝7⎠ x 1 = −1 + 4 + −9 + 7 = 21. x2= x. ∞. −1 + 4 + −9 + 7 = 147 = 12.12 ( 2 dp ) 2. 2. 2. 2. = max ( −1 , 4 , −9 , 7 ) = max (1, 4, 9, 7 ) = 9. A5 Properties of the Norm of a Vector Next we state certain properties of the norm of a vector. Proposition (5.5). Let V be an inner product space and u and v be vectors in V. If k is any scalar then we have following properties: (i) u ≥ 0 [ Non-negative] (ii) u = 0 ⇔ u = O (iii) k u = k u.

(10) Chapter 5: Inner Product Spaces. 10 k 2 = k where k is the modulus of k].. [Note that for a scalar k we have. Proof of part (i). By the above definition (5.3) we have u =. u, u where the square root is the. positive root, that is u =. u, u ≥ 0. ■ Proof of part (ii). Again from definition (5.3) we have u = u, u = 0 ⇒. u=O. and u=O ⇒. u =. O, O = 0. ■ Proof of part (iii). Applying the definition (5.3) we first find ku. 2. and then we take the square root:. 2. k u = k u, k u = k u, k u = kk u, u = k 2 u, u. Taking the square root gives k u = k 2 u, u = k 2. u, u = k u. This is our required result. ■ SUMMARY (5.1). An inner product is a function on a vector space which satisfies the following: (i) u, v = v, u [Commutative Law] (ii) u + v, w = u, w + v, w. [Distributive Law]. (iii) k u, v = k u, v (iv) u, u ≥ 0 and we have u, u = 0 if and only if u = O (5.2)The inner product has the following properties: (i) u, O = O, v = 0 (ii) u, k v = k u, v (iii) u, v + w = u, v + u, w The norm of a vector denoted u and is defined as (5.3). u =. u, u. Proposition (5.5). Let V be an inner product space then we have the following: (i) u ≥ 0 [ Non-negative] (ii) u = 0 ⇔ u = O (iii) k u = k u.

(11)

No results found