Gram-Schmidt orthonormalization

The dot product defi ned in three-dimensional space has a geometric inter-pretation as the length of the projection of one vector onto another multiplied by the length of the second vector. If the two vectors are mutually orthogo-nal, the projection is zero, and the dot product of the vectors is zero. It is convenient to use a set of mutually orthogonal basis vectors to represent any vector in R³ as a linear combination of these basis vectors. A common choice for an orthogonal set of basis vectors in R³ is

i= j k An orthogonal basis is convenient for the representation of a vector space. It is often easy to obtain a linearly independent basis for a vector space. For exam-ple, it is clear that the vectors p1(x) = 1, p2(x) = x, and p3(x) = x² form a basis for P²[a,b]. However, this basis is not an orthogonal basis for P²[a,b] with respect to the standard inner product. Thus it is useful to develop a scheme to determine an orthogonal basis which spans the same space as a set of basis vectors.

Defi nition 2.11 A set of vectors is said to be normalized with respect to a norm if the norm of every vector in the set is one.

Defi nition 2.12 A set of vectors is said to be orthonormal with respect to an inner product if the vectors in the set are mutually orthogonal with respect to the inner product and the set is normalized with respect to the inner-product-generated norm.

9533_C002.indd 98

9533_C002.indd 98 10/29/08 3:56:59 PM10/29/08 3:56:59 PM

If a set of vectors u₁,u₂,…,un is orthonormal with respect to an inner product, then

( ,u ui j)= =ij ≠

=

⎧⎨

⎪⎪

δ ⎩⎪⎪0 i j

1 i j (2.25)

for all i,j = 1,2,…,n.

A fi nite-dimensional vector space V of dimension n has n basis vectors. The following theorem provides a procedure by which another basis for V, ortho-normal with respect to any valid inner product on V, may be constructed. If a basis can be found for an infi nite-dimensional vector space, the Gram-Schmidt process, outlined in theorem 2.6, can be used to generate an orthonormal basis for the vector space.

Theorem 2.6 (Gram-Schmidt Orthonormalization) Let u1,u2,… be a fi nite or countably infi nite set of linearly independent vectors in a vector space V with a defi ned inner product (u,v). Let S be the subspace of V spanned by the vectors. There exists an orthonormal set of vectors v

1,v

2,v

3,… whose span is also S. The members of the orthonormal basis can be calculated sequentially according to

w u v w

w

w u

1 1 1 1

1

2

= =

= 22 2 1 1 2 2

2

−(u v v, ) v = w w



n n



w =u − (uu v v v w

n i i n wn

n

, )

i n

=

∑

−₁^{1 =}

 

(2.26)

Proof First consider the following lemma:

Lemma A set of vectors which are elements of a vector space V and which are mutually orthogonal with respect to any valid inner product defi ned on V are linearly independent.

Proof of Lemma Let v1,v2,…,vk be elements of a vector space V which are mutually orthogonal with respect to a valid inner product on V, (u,v), Then (vi,vj) = 0 for i = 1,2,…,k and for j = 1,2,…,k, but j ≠ k. Consider a linear combi-nation of the set of vectors set equal to the zero vector,

0=C1v1+C2v2+…+Ckv (a)k

9533_C002.indd 99

9533_C002.indd 99 10/29/08 3:57:00 PM10/29/08 3:57:00 PM

Taking the inner product of both sides of Equation a with v_j for an arbitrary j between 1 and k, and using properties of a valid inner product, leads to

0,vj C1 v v1, j C2 v v2, j C1 v v1, j

( )= ( )+ ( )+…+ ( ) ^(b)

Note that (0,v_j) = 0 and that using the mutual orthogonality of the vectors in Equation b leads to Cj = 0. Since j is arbitrary, Cj = 0 for all j, j = 1,2,…,k. Thus the vectors are linearly independent, and the lemma is proved.

Since an orthogonal set of vectors is linearly independent, then if Equa-tions 2.24 generate an orthonormal set, then v₁,v₂,…,vn,… are linearly inde-pendent. Equations 2.24 also show that each of the vectors in the proposed orthonormal set is a linear combination of the vectors in the basis for S, and thus they are also in S. If S is a space of dimension n, then the proposed set of vectors is composed of n linearly independent vectors, and thus they span S.

If S is an infi nite-dimensional space, then it can be shown that the proposed set is complete in S, but that is beyond the scope of this text.

First note that

v w

w w w

i i

i i

= = 1 i =

1

Hence the set is normalized. It only remains to show that w₁, w₂,… form an orthogonal set of vectors. This is done by induction. First, it is shown that w₂ is orthogonal to v_1´

( , ) ( ( , ), )

( , ) ( , )( ,

w v u u v v

u v u v v

2 1 2 2 1 1

2 1 2 1 1

= −

= − vv1)

However, v1 v v1 1 1 2

= = ( , )1 ^/ . Thus

(w v2, 1)=(u v2, 1) (−u v2, 1)=0 Now assume

( ,v vi j)=δij for i,j=1,2,…,k−1 and consider

( , ) ( ( , ) , )

( , ) (

w v u u v v v

u v u

k i k k j j i

j k

k i

= −

= −

=

∑

−₁¹

kk j j i

j k

,v )(v v, )

=

∑

−₁¹

9533_C002.indd 100

9533_C002.indd 100 10/29/08 3:57:01 PM10/29/08 3:57:01 PM

Since (vi, vj) = 0 for i ≠ j and for i,j < k, the only nonzero term in the sum occurs when j = i. Thus

( , ) ( , ) ( , )( , ) ( , ) ( ,

w v u v u v v v

u v u

k j k i k i i i

k i k

= −

= − vvi)

= 0

The theorem is thus proved by induction.

If a set of vectors u1, u2,…, uk–1, uk, u_{k + 1},… is a complete set of linearly inde-pendent vectors in an infi nite-dimensional vector space V with a defi ned inner product, then the Gram-Schmidt process can be used to determine a set of orthonormal vectors that is complete in V.

Example 2.20 A basis for S, the intersection of P⁶[0,1] with a subspace of C⁴[0,1], defi ned as those functions that satisfy the boundary conditions for the differential equation governing the vibrations of a fi xed-fi xed beam, is developed in Example 2.15. One basis for S is determined as

f x1( )= −x6 4x3+3x2

f x2 x5 x3 x2

3 2

( )= − + (a)

f x3( )= −x4 2x3+ x2

Use the Gram-Schmidt procedure to determine a basis for S that is orthonor-mal with respect to the standard inner product for C⁴[0,1].

Solution Normalizing f

1,

f1 x6 x3 x2 2 dx

0

1 1

2

4 3 0 171

=^⎡

(

− +

)

⎣

⎢⎢

⎢

⎤

⎦

⎥⎥

⎥ =

∫

^{. (b)}

Then

v x f x f x

v x x x x

1 1

1

1 6 3 2

5 84 23 4 17 5 ( ) ( )

( )

( ) . . .

=

= − +

(c)

Calculating w₂^,

w x2( )= f x2( ) ( ( ), ( )) ( )− f x v x v x2 1 1 (d)

9533_C002.indd 101

9533_C002.indd 101 10/29/08 3:57:02 PM10/29/08 3:57:02 PM

where

Thus Equation d becomes

w x2 x x x x x

Calculating w3(x), using Equation (2.26)

w x3( )= f x3( ) ( ( ), ( )) ( ) ( ( ),− f x v x v x3 1 1 − f x v x3 2( ))) ( )v x2 (i)

9533_C002.indd 102

9533_C002.indd 102 10/29/08 3:57:03 PM10/29/08 3:57:03 PM

Substituting Equations j and Equation k into Equation i leads to

which when substituted into Equation l, leads to

v x3 x6 x5 x4 x3

881 2 2667 3 2903 6 1329 9 21

( )= . − . + . − . + 22 4. x (n)²

Example 2.21 Use the Gram-Schmidt process to derive an orthonormal basis for P²[0,1] with respect to the standard inner product on R⁷. Use the basis f₁(x), f₂(x), and f₃(x) from Example 2.16.

Solution Polynomials in P⁶[0,1] can be represented as vectors in R⁷, as illus-trated in Example 2.8. To this end, the polynomials of Example 2.20 can be represented as

The standard inner-product-generated norm for p1 is

p1 = ( )12+ −( 4)2+( )32= 26 (b)

9533_C002.indd 103

9533_C002.indd 103 10/29/08 3:57:04 PM10/29/08 3:57:04 PM

Thus p₁ can be normalized as

Equation 2.26 is used to calculate a vector orthogonal to p₁ as w2 p2 p v v2 1 1

9533_C002.indd 104

9533_C002.indd 104 10/29/08 3:57:04 PM10/29/08 3:57:04 PM

Equation 2.26 is used to construct

Converting the vectors to polynomial form, an orthonormal basis for S with respect to the standard inner product for R⁷ is

v x x x x

In document Advanced Engineering Mathematics - S Graham Kelly.pdf (Page 107-114)