12.1 – Orthogonal Functions

Lecture 1

Kemal Ahmed

Math 2ZZ3 2013-06-24 Dr. French

• Course Address: http://ms.mcmaster.ca/~afrench/wordpress/

• 4 MATLAB assignments, drop lowest 2 assignments

• Most questions can be answered by reading the course outline • TA’s don’t have office hours; only Dr. French

• Not sure if she will give formula sheets

• The suggested problems are not sufficient to covering all the material, so also do sample tests.

• Proofs are fair game; require conceptual understanding of material • Exam: August 7th

12.1 – Orthogonal Functions

A vector in 3

, , ,

x

y x y z

z

   

=_{ } ∈

   

 

xi yj zk = + + 

1 0 0

0 , 1 , 0

0 0 1

i j k

            =_{ } =_{ } =_{ }            



 

, , i j k  

form a basis for3 Notation:u=u

Dot Product

1, 1, 1 2, 2, 2 1 2 1 2 1 2

x y z ⋅ x y z =x x +y y +z z

Properties of the Dot Product

i)

Symmetric: u v⋅ = ⋅v u

ii)

(

) ( )

( )

k k

k

⋅ = ⋅

= ⋅

u v u v

u v

iii)

0

0

⋅ = ⇔ = ⋅ ≥

u u u 0

u u

iv)

(

u v w⋅ ⋅ = ⋅ + ⋅

)

u w v w

Inner Product

Take 2 functions, f₁& f₂and assume they are piecewise continuous on some interval [a, b]. The inner product of two functions f₁& f₂on the interval [a, b] is the number:

Notation:

(

_{1 2}

)

₁

( ) ( )

₂ inner product

, b d

a

f f =

∫

f x ⋅f x x 

We now need to check to see that each of the four properties holds. i)

(

) (

?

)

1, 2 2, 1

f f = f f

(

)

( ) ( )

( ) ( )

(

)

1 2 1 2

2 1

2 1

, d

d

,

b a b a

f f f x f x x

f x f x x

f f

= ⋅

= ⋅

=

∫

∫

This holds because of scalar multiplication. ii)

( ) ( )

(

)

( ) ( )

( ) ( )

(

)

(

)

( )

( )

1 2 1 2

1 2

1 2

1 2

1 2

, d

d

,

,

d

b a b a

b a

kf x f x kf x f x x

k f x f x x

k f f

f kf

f x kf x x =

= = =

= ⋅

∫

∫

∫

This holds because you can take out constants from integrals. iii)

( )

0

f x = is our “zero”

Take the inner product of itself.

( ) ( )

(

)

(

( )

)

2

1 , 1 1 d

b a

f x f x =

∫

f x x

( )

(

)

2

(

( )

)

2

1 0 1 d 0

b a

f x ≥ ⇒

∫

f x x≥

( )

(

)

2

( )

1 d 0 1 0

b

a f x x= ⇔ f x =

∫

iv)

(

)

(

)

(

) (

)

1 2 3 1 2 3

1 3 2 3

1 3 2 3

1 3 2 3

, d

d

d d

, ,

b a b a

b b

a a

f f f f f f x

f f f f x

f f x f f x

f f f f

+ = +

= +

= +

= +

∫

∫

∫

∫

We look at functions as if they were vectors with the dot product applied to them.

Orthogonality

Take a set of functions. A pair of functions is orthogonal if the following if

(

f f₁, ₂

)

=0. A set of functions is orthogonal if every possible pair from the set is orthogonal. We will use orthogonal sets to build basis.

e.g. 1)

( )

( )

( )

{

}

{ }

0

(

(

)

)

sin , sin 3 , sin 5 ,...

, sin 2 1

n _n n

x x x

n x

φ ∞ φ

=

= = +

On the interval_0,π₂_ Is this an orthogonal set?

Let’s show that

(

φ φ_n, _m

)

=0whenever n ≠ m.

(

)

( ) ( )

(

)

(

)

(

(

)

)

(

) (

)

(

)

(

(

) (

)

)

(

)

(

)

(

)

(

)

(

(

)

)

(

)

(

(

)

)

2

2

2

2 2

2 2

0

0

0

0 0

0 0

, d

sin 2 1 sin 2 1 d 1

cos 2 1 2 1 cos 2 1 2 1 d

2

1 1

cos 2 d cos 2 1 d

2 2

1 1

sin 2 sin 2 1

4 4 1

0

n m n m

x x

x x

x x x

n x m x x

n m x n m x x

n m x x n m x x

n m x n m x

n m n m

π π

π

π π

π π

φ φ φ φ

= =

= =

=

= + +

   

= _ + − + _− _ + + + _

 

= _ − _ − _ + + _

= − − + +

− + +

=

∫

∫

∫

∫

∫

Norm

The norm of a function can be expressed in terms of the inner product:

(

)

( ) ( )

2

, b d

a

f = f f ⇒ f =

∫

f x ⋅f x x

Orthonormal

For an orthogonal set of functions

{ }

φm , the set is orthonormal if for every member, φm =1(i.e.

if the length is 1).

If

{ }

φ_m is orthogonal, make it orthonormal by replacingφ_mby , m m m

m

φ ψ ψ

φ =

{ }

ψm is orthonormal:

(

)

(

)

2 , 1

, 1

,

1

m m

m

m m m m

m m m

m m m

m m

φ ψ

φ

φ φ

φ φ

φ φ φ

φ φ φ

φ φ =

 

= _ _

 

=

=

= =

(

)

(

)

0

, ,

1

,

0

m n m n

m n

m n m n

φ φ

ψ ψ

φ φ

φ φ φ φ

=

 

= _ _

 

= =



0 because

{ }

φ_n is orthogonal e.g. 2)

Suppose infinite set of functions

{

( )

}

0

n x _n

φ ∞₌ is orthogonal. Take an arbitrary function, f that is piecewise-continuous on the interval [a, b]. How do we represent f in terms of our orthogonal set?

Detour to 3

{ }

3 1

: _i

i

v ₌

 orthogonal u an arbitrary vector in3

want to write: 1 1 2 2 2 3, i

c c c c

= + + ∈

u v v v 

(

1

)

1 1 1 2_2 1 3_3 1

0 0

2 1 1

,

u v c c c

c v

= =

= ⋅ + ⋅ + ⋅

=

v v v v v v

(

1

)

1 2

1 ,

u v c

v

⇒ = Similarly,

(

)

(

)

2

2 2

2

3

3 2

3 ,

,

u v c

v

u v c

v

=

=

{ }

φ_n ,f are both piecewise continuous on

[ ]

a b,

( )

(

2

) ( )

0 , _n

n n _n

f

f x φ φ x

φ

∞

=

=

∑

You require an infinite set to make a basis, whereas for vectors you can use a finite set.

Note: we aren’t going to define the term basis in this course. To form a basis, the set must also be linearly independent, and span the set.

General Fourier Series

A.K.A. The orthogonal series expansion of f (x). e.g. 3)

Take an orthogonal set,

{ }

φ_n , on

[ ]

a b, , such thatφ₀

( )

x =0. Show b _nd 0, 1, 2, 3,...

aφ x= ∀ ∈ =n I

∫

( )

( )

( ) ( )

( )

( ) ( )

(

)

0

0

0 1

d d

, 0

n n

n

b b

n n

a a

n

x x

x x

x x x x x

φ φ

φ φ

φ φ φ

φ φ

= ⋅

= ⋅

= ⋅

= =

∫

∫

e.g. 4)

Show

( )

( )

2

1 , 2

f x =x f x =x on

[

−2, 2

]

are orthogonal.

(

)

2

( ) ( )

1 2 1 2

2 2 ₃

2

, d

d

f f f x f x x

x x

−

−

= =

∫

∫

Since it’s odd and symmetric across 0, you know it’s 0. Recall:

Even function: f x

( )

= f

( )

−x Odd function: f x

( )

= −f x

( )

Find constantsc c₁, ₂, such that

( )

2 3

3 1 2

f x = +x c x +c x is orthogonal to both f₁and f₂on

[

−2, 2

]

.

(

)

(

)

1 3 2

1 3 2 2

2 3

1 2

2

2 ₂ 2 ₃

2

2 2

odd

, 0

d

d

d d

f f

f f x

x x c x c x x

x x c x x

−

−

− −

= =

= + +

= +

∫

∫

∫

∫

_{} 2 ₄ 2 2

2

2

d

16 64

0

3 5

5 12

c x x

c

c

−

+

= + =

−

⇔ =

∫

(

2 3

)

2 _{2 2 3}

1 2

2 2

3 2

, 0

d

d f f

x x c x c x x

x x

−

−

=

 

= _ + + _

=

∫

∫

2 2

4 5

1 2

2 2

odd

d d

c x x c x x

− −

+

∫

+

∫



odd

1

1

64 0 5

0 c

c

= =

⇔ =



( )

3

3

5 12 f x = −x x

12.2 – Fourier Series

( ) ( ) ( )

( ) ( )

{

_{1, cos} x _{, cos} 2 x _{, cos} 3 x _{, etc., sin} x _{, sin} 2 x _{, etc.}

}

p p p p p

π π π π π

One can show that this set is orthogonal on

[

−p p,

]

.

f is a piecewise-continuous function on

[

−p p,

]

.

Piecewise-continuous: means made of continuous pieces that aren’t necessarily the same function. They may have holes as long as nothing blows up to infinity.

You want coefficientsa b_n, _n, such that:

( )

0

(

( )

( )

)

1

cos sin

2

n x n x

n p n p

n

a

f x a π b π

∞

=

= +

∑

+

( )

0

( )

( )

1

d d cos d sin d

2

p p p p

n x n x

n p n p

p p p p

n

a

f x x x a π x b π x

∞

− − ₌ − −

 

= + _ + _

 

∑

∫

∫

∫

∫

For n≥1,

( )

( )

cos d 0

sin d 0

p n x p p p n x p p x x π π − − = =

∫

∫

( )

( )

0 0 d d 2 1 d p p p p p p a

f x x x

a f x x

p − − − ⇒ = ⇒ =

∫

∫

∫

Zeroth Fourier coefficient

Now, find the rest of the Fourier coefficients.

( )

0

(

( )

( )

)

1

cos sin

2

n x n x

n p n p

n

a

f x a π b π

∞

=

= +

∑

+

Multiply both sides bycos

( )

m x p

π _{, then integrate.}

( )

_cos

( )

_d 0 _cos

( )

_d 2

p p

m x m x

p p

p p

a

f x π x π x

− = −

∫

∫

p cos

( ) ( )

m x cos n x d p cos

( ) ( )

m x sin n x d

n _p p p n _p p p

a π π x b π π x

− − +

∫

+

∫

1 n ∞ =      

∑

The remaining term is 0, except when n = m.

( )

( )

(

( )

)

( )

( )

2 2 2

cos d cos d

1

1 cos d

2

sin 2

p p

n x n x

n

p p

p p

p

n x

n _p p

n x

n p

f x x a x

a x p a p n π π π π π − − − =   = _ + _ = +

∫

∫

∫

x p x p = =−

( )

( )

1 cos d p n x

n _p p

a f x x

p π − =

_∫

Similarly, obtain:

( )

( )

1 sin d p n x

n _p p

b f x x

p

π −

=

∫

Summary

Fourier Series of a function, f, defined on the interval

(

−p p,

)

is given by:

( )

0

(

( )

( )

)

1

cos sin

2

n x n x

n p n p

n

a

f x a π b π

∞

=

= +

∑

+ , where

( )

0 1 d p p

a f x x

p − =

_∫

,

( )

( )

1 cos d p n x

n _p p

a f x x

p π − =

∫

,

( )

( )

1 sin d p n x

n _p p

b f x x

p

π −

=

_∫

The point of the Fourier Series is to represent the function as an orthogonal set / basis.

Ask yourself: does this series converge to the given function? It holds as long as it is continuous; if it is discontinuous, it does not.

Do all the practice problems in this chapter.

e.g. 5)

Find Fourier Series on given interval:

( )

1, 1 0

, 0 1

x f x

x x

− < < 

=  _{≤ <} 

Made with Winplot.

( )

0

(

( )

( )

)

1

cos sin

2 1

n x n x

n p n p

n

a

f x a b

p

π π

∞

=

= + +

=

∑

( )

( )

( )

0

1

1

0 1

1 0

1

d d

1 d d

3 2

p p

a f x x

p

f x x

x x x

−

−

−

= =

= +

=

∫

∫

∫

∫

( )

( )

( ) (

)

( ) (

)

(

)

(

)

1

1

0 1

1 0

0

1 1

cos d

cos d

1 cos d cos d

1

sin integration by parts

p

n x

n _p p

x

x

a f x x

p

f x n x x

n x x x n x x

n x n

π

π

π π

π π

−

−

−

=

=−

= =

= +

= +

∫

∫

∫

∫

(

)

(

)

1

d d

d cos d

sin

n

u x

u x

v n x x

v _π n x

π π =

= = =

Order of choosing u: Log

Inverse trig Algebra Trig

Exponential

Back to the question:

(

)

(

)

(

)

( )

0 1

1

0

1 0

1 1 1

sin sin sin d

1

sin 0

x x

x x

n x x n x n x x

n n n

n

π π π

π π π

π

= =

=− =

= + −

=

∫

(

)

sin nπ

− −

( ) ( )

1 sin n

nπ π

 _{ +}

 

(

)

( )

1

2 2

0

2 2

1

0 cos

1

cos 1

x

x

n x n

n n

π π

π π

=

=

 

− +

 

 

 

= _ − _

Can you write this more nicely? YES!

( )

2 2

1

1 n 1

n

a

nπ

 

= _ − − _

( )

( )

( ) (

)

(

)

0 1

1 0

1

sin d

1 sin d sin d

1

p

n x

n _p p

b f x x

p

n x x x n x x

n

π

π π

π

−

−

=

= +

= −

∫

∫

∫

( )

( )

2 2

(

)

(

)

1

1 1

3 1

cos sin

4

n

n

f x n x n x

n π π nπ π

∞

=

 _{− −} 

= + − 

 



∑



Theorem

Suppose you have f f₁, ₂, both of which are piecewise-continuous on the interval (−p, p). Then, the Fourier Series of f on the interval converges to f (x) at a point of continuity. At a point of discontinuity, Fourier series converges to the average:

( ) ( )

2

f x+ + f x−

. Right-hand limit: f x

( )

+

Left-hand limit: f x

( )

−

e.g. 5) continued

Continued from example 5.

At x = 0, the Fourier series converges to

( ) ( )

0 0 1

2 2

f + + f −

= .

How does the Fourier series behave outside of (−p, p)? It is periodic with period 2p. We call the function ondefined by the Fourier series, the periodic extension of f.

12.3 – Fourier Sine & Cosine Series

Properties:

a) even ∙ even = even b) odd ∙ odd = even c) even ∙ odd = odd d) even ± even = even e) odd ± odd = odd

f) f even

( )

( )

0

d 2 d

a a

a f x x f x x

−

⇒

∫

=

∫

g) f odd a

( )

d 0

a f x x

−

⇒

∫

=

Now, apply the properties to Fourier coefficients!

( )

f x

_{( )}

0 1

d

p p

a f x x

p −

=

∫

1 p

( )

cos

( )

n x d

n _p p

a f x x

p

π −

=

∫

1 p

( )

sin

( )

n x d

n _p p

b f x x

p

π −

=

∫

Even

( )

0 ₀

2

d

p

a f x x

p

=

∫

( )

( )

0 2

cos d

p

n x

n p

a f x x

p

π

=

∫

bn =0

Odd a₀ =0 an =0

( )

( )

0 2

sin d

p

n x

n p

b f x x

p

π

=

_∫

In summary, define:

i. Fourier series of an even function on (−p, p) is the cosine series.

( )

0

( )

1 cos 2

n x n p n

a

f x a π

∞

=

= +

∑

With

( )

( )

( )

0 ₀

0

2

d

2

cos d

p

p

n x

n p

a f x x

p

a f x x

p

π

=

=

∫

∫

( )

( )

1

sin n x n p n

f x b π

∞

=

=

∑

With

( )

( )

0

2

sin d

p

n x

n p

b f x x

p

π

=

_∫

e.g. 6)

Expand in an appropriate sine or cosine series

( )

, 1 0

, 0 1

x f x

x

π π

− < < 

= _{− ≤ <} 

Since f is odd, we choose a sine series.

( )

( )

( )

( )

1

0

sin

2

sin d

1

n x n p n

p

n x

n p

f x b

b f x x

p p

π

π ∞

=

=

= =

∑

∫

( ) (

)

( ) (

)

1

0 1

0

2 sin d

2 sin d

n

b f x n x x

n x x π

π π

=

= −

∫

∫

Yup, we lost half of the interval.

( )

2

1 n 1

n

b n

 

= _ − − _