SFWR ENG 3X03

(1)

SFWR ENG 3X03

Teacher: Dr. Ned2

Fall 2013

Numerical Methods Guy

Chapter 1

Types of errors

The focus in this course is determining the error margin in MATLAB. We will only focus on discretization errors.

Assuming u is an exact result and v is an approximation Absolute error (A): |u – v|

Relative error (R): u v ,u 0 u

− ≠

Cancellation Error (R): when adding 2 numbers of similar absolute value, but opposite sign and there’s a loss of precision

Overflow: when a number is too large (fatal)

Underflow: when a number is too small (usually rounded down to 0)

min

x f : minimum value of f, with changing x Other sources:

• (A) when y >> x, x + y

• (A, R) when |y| >> 1, xy

• (A, R) when |y| << 1, x/y

Condition Number

Forward error: difference between result and solution

Backward error: the difference between the value of x used to find the result and the value of x that would give the solution

(2)

Condition number =

( )

1

ˆ _ˆ

ˆ

f x f x _y _y

f x y

A A

x x x

x x

−

− ₋

= = ⋅

− ∆

Condition number =

relative forward error relative backward error y

y x x ∆

←

∆ ←

Summary: condition number = change in solution/change in input Well-conditioned: low condition #

Ill-conditioned: high condition #

Horner’s Method (or rule or scheme): nested polynomial form, i.e.

( )

0 1

( )

(

1

)

1

)

0

n

n n n n n n

p x = +c c x+ + c x ⇒ p x =  c x+c₋ x+c ₋ x x+c

Chapter 2 – Round-off Errors

Floating Point

Floating Point (FP) System (β, t, L, u) β – base

t – number of digits

L – lower bound for exponent u – upper bound for exponent

( ) (

)

( )

(

)

( )

(

)

1 1

1 x

y

xy f x x

f y y

f xy xy δ

δ δ

= +

Rounding unit (a.k.a. machine epsilon): 1 1

2 t

η₌ β −

flops: the number of elementary floating point operations necessary to compute something; elementary floating point operations are considered equal to each other in terms of time and can be any of addition, subtraction, multiplication, and division

Rounding

Types:

a) To nearest

b) Towards +∞ (i.e. upper bound) c) Towards −∞ (i.e. lower bound) d) Towards 0 / chopping

(3)

Ceiling function: _{ }x

Floor: _{ }x

IEEE Precision

+∞: from overflow of positives; maximum exponent, 0 fraction −∞: from overflow of negatives; same as +∞, but with different sign NaN: division by 0; maximum exponent; fraction is not 0

Single

A.K.A. short Bias = 127

Double

A.K.A. long Bias = 1023

Convergence

x*: root

More accurate as k →∞

Rate of Convergence: givenρ= g x′

( )

* , where 0 < ρ < 1, RoC =−log₁₀ρ

* * *

1 0

* * *

1

, 0

k

k k

x x x x x x

m

x x x x x

m

ρ ₋ ρ

+

− ≤ − ≤ ≤ −

−

− = − =



Taylor Series

Rewrite a function, f(x) as an infinite sum of other functions.

( )

( ) ( ) ( )( )

2 ( )

( ) ( )

1! 2! !

n

f a f a f a

f x f a x a x a x a

n

′ ′′

= + − + − + +_ −

a is a constant that either you choose or it is given to you

It is easiest to do if you start off by finding the first couple derivatives and then plugging in a, so you can see how the function changes with each iteration.

(4)

e.g.)

( )

_{( )}

1

2

3

4 4

ln

1 2

3 2

f x x

f x x

−

= ′ = ′′ = − ⋅ ′′′ = ⋅

= − ⋅

( )

2 ln 2 1 2

2 1 2

4 1 2 2

8 f

f f f

=

′ =

′′ = −

 

′′′ _{=  }

 

As you can see, it seems to have some sort of factorial coefficient

Taylor’s Theorem

If you know the value of f(x0), but want the value of f(x0 + h):

(

)

( )

( ) ( )

( )

_{( )}

2 1

1

0 0 0 0 ... 0

2 ! 1 !

k k

h h h

f x h f x hf x f x f x f

k k ξ

+ +

′ ′′

+ = + + + + +

+

Where ξ is some point between x0 and x0 + h.

Chapter 3 – Solving Non-linear Equations

Find the roots.

Tolerance: a value that tells the computer what precision to stop computation Absolute Tolerance: |xn− xn − 1| < atol

Bisection Method

When given an upper and lower estimate for a number, find the midpoint between the two points. If the midpoint is negative, it becomes the new lower limit. If it is positive, it becomes the new upper limit. Keep doing this until the end of time…

This is used on graphing calculators. It is linearly convergent.

Error

Where c is the solution and cn is the computed solution at step n

error 2

n n

b a c − ≤c −



Newton’s Method

It runs faster than bisection method. When given a single guess (x0),

( )

1

k

k k

k f x

x x

f x + = − _′

How it works:

(5)

2. Find the point on the line that hits y = 0. 3. The point is your new x0. Go to step 1.

One problem with this method is that sometimes the computation gets into an infinite loop, going back and forth endlessly between 2 points. Thus, you require another field, maximum

iterations, to prevent infinite loops.

If you don’t know the derivative of the function, you need to sub in the secant formula. It is super-linearly convergent.

Error

.

Secant Method

( )

(

1

)

1

k k

k

k k f x f x f x

x x − −

−

′ =

−

( )(

)

( )

1

k k k

k k

f x x x

x x

f x f x − +

−

− = −

−

The order of convergence is the golden ratio, 1 5 1.618 2

α = + ≈ . It is super-linearly convergent

Solving Decimal Exponents ln

b b a a =e

Chapter 4 – Linear Algebra Review

Review eigenvalue/vector/matrix stuff

Perturbation: a change in a vector

T

x Ax= scalar if x is a vector and A is a matrix

Algebraic Multiplicity: the number of repeats of a root Geometric Multiplicity: the number of different roots

If 1

B=PAP− , then A is similar to B Vector norm: regular definition of norm;

1

-norm ,1

p

n p

p _p i

i

x p

=

 

= =_ _ ≤ ≤ ∞



∑



x



1

max _i i n x ∞ = _{≤ ≤} x

A norm must have the following properties: 1. p a

( )

v = a p

( )

v

2. p

(

u+v

)

≤ p

( )

u +p

( )

v

(6)

3. Ifp

( )

v =0,v=0

Matrix norm: same as vector norm, but for matrices

1

max A max

A A

≠ =

= =

x 0 x

x

x , i.e. the sum of all the elements on the row of the matrix with the highest sum

Euclidean norm: base 2 norm x ₂; magnitude of something in Euclidean space Two matrices are considered orthogonal to each other if u vT =0

Matrix multiplication review:

Recall a b , x , ax by

c d y cx dy

+

     

=_ _ =_{ } =_ _ +

     

A B AB

Positive definite: a matrix, A, is positive definite if the scalar, x AxT , is positive, as long as

≠

x 0

Residual: r= −b axˆ

Spectral radius: the eigenvalue of a matrix with the highest magnitude ρ

( )

A =max

{ }

λ

Chapter 5 −

Solving a Matrix

When trying to solve for x, where Ax = b, where A is a real, non-singular (determinant ≠ 0), n×n matrix, and b is a real vector, there are two ways:

1. Direct methods (this chapter): exact solutions without round-off errors through variations of Gaussian Elimination

2. Iterative Methods: something like what is done for non-linear equations when direct methods fail

Backward Substitution

To obtain an upper triangular matrix (elements below the diagonal are 0), make your matrix an upper triangular matrix

Cost:

(

)

(

)

( )

1

2

1

2 2

2 n

k

n n

n k O n

−

=

−

− = =

∑

Forward Substitution

To obtain a lower triangular matrix (elements above diagonal are 0), make your matrix a lower triangular matrix through Gauss-Jordan row operations.

Cost: same as backward substitution

Gaussian Naïve Elimination

Use this to solve for your upper and lower triangular matrices of your size n square matrix. First, find the upper triangular matrix:

Start at the top left element 1. Go from left to right:

(7)

a. Compare the current value to the value underneath it to find the value of

, 1

, 1 ,

i j

i j i j x

x +

+

=_ .

b. Multiply each value of xi,j→n by _{i j+}_, ₁.

c. Repeat by comparing the current value to the value under the one you just compared it to, i.e. , repeats

,

i j

i j x

x +

.

2. Repeat starting at the next column, next row, i.e. start each at xi=j Make up your lower triangular matrix:

1. Your diagonal is all 1’s

2. Everything above the diagonal is 0’s

3. Use your l values from the U-matrix to fill in the last gaps

12

13 23

1 0 0

1 0 1

 

 

 



 

Gaussian Elimination

A method of solving matrices that is useful for solving for x.

Cost:

(

)

(

) (

)

( )

1

2 2 ₁ ₃

1

2 2 1 2 ... 1

n

k

n k n n O n

−

=

− = − + − + + =

∑

e.g.)

1 1

2 2

3 3

1 1 1 3 1 1 1 3

0 1 0 4 0 1 0 4

0 0 5 4 0 0 5 4

x x

         

 ₋ ₋   _⇒ ₋     _{= −}

         

 −   _{ } _ −     _{    }

 

3

5 4

4 5 x x

− =

= −

2

4 4 x

x − = −

=

1 2 3

4

1 5

4

1 5

1

3

4 3

1 4 5

5 x x x

x

x x −

+ + =

+ + = = −

− =

1

1 5 x = −

(8)

1 4 , 4,

5 5

₋ ₋ 

 

 

LU Decomposition

It is useful if you aren’t given m number of b vectors. Do not switch rows!

L: lower triangular U: upper triangular

If L is a lower triangular, non-singular matrix, its inverse is also lower triangular. 1. Use Naïve Gaussian to find the upper and lower triangular matrices. 2. A = LU => L(Ux) = b, which splits up into y=Ux&Ly=b

3. Solve Ly = b for y. ← O(n2)

4. Use y to solve the equation y = Ux for x ← O(n2)

( )

( ) ( )

det A =det L det U

Cost: O(n3) for the Gauss, but O(n2) for solving for b.

Finding the Inverse of a Matrix

Given A, find B = A−1 AB=I

GE With Pivoting (GEPP)

Due to floating point arithmetic errors, you sometimes get something where 1 + small# = 1. To avoid this we pivot by switching some of the rows. You do this to avoid division by 0.

Scaled GEPP: multiplying a row by a constant

Cholesky Decomposition

For symmetric, positive-definite matrices ONLY, you can represent the lower triangular as the transpose of the upper triangular, so we can let G represent L, where you try to find G. This halves the required memory and operations required.

T

A=GG

Matrix multiplication

11 12 11 11 21

21 22 21 22 22

0 0

a a g g g

    

=

    

    

(9)

2

11 11 11 11

21 21 11

21 21

11 11

2 2 2

22 21 22 22 22 21

0 0

a g g a

a g g

a a

g g

g a

a g g g a g

= + ⇒ =

= +

= ⇒ =

= + ⇒ = −

O(n3)

Chapter 6 – Linear Least Squares

Data fitting: forming a curve that fits between the points

Interpolation: drawing a curve along the points, guessing the shape

This method is a method of data-fitting

Linear least square: find a function that best fits the given data points. The function that you want is the one that minimizes the sum of the square of the distances from the point to the curve. Why squared?

Usually you’d be given the general shape of function.

Given n points, represent the distance between the points and the line by:

( )

1

(

( )

)

2

0

, n

k k

k

a b f x y

φ −

=

∑

−

Equateφ′

( )

a b, =0to find the coefficients of f because that is when the function is at a minimum. To do this find

( )

a b, 0

a

φ

∂

=

∂ and

( )

, 0 a b

b

φ

∂

= ∂

Matrices

If your system is Ax = b, and your given y equation is y=ax be+ x, represent your y equation by the Ax. Given a set of n points

x 1 2 3

y 2 3 5



A

b

x

2

3

calculated

1

2 2

3 3

5 e

e a

y

e b

x y

 

  

 _{  } _

   _{  }₌ _



 _{  } _

 

    

 

 _

Solve for x: x = b\A

Also, the ‘\’ sumbol is the equivalent of the MATLAB function, mldivide(A,b) or b*inv(A).

(10)

Chapter 8 −

Eigenvalue

n n A R Ax λx

×

∈ =

λ = eigenvalue x = eigenvector

There are 3 methods to compute eigenvalues:

1. Power method: Finds the largest eigenvalue

2. Inverse Power method: Finds the smallest eigenvalue and accelerates

3. QR Algorithm: Finds all eigenvalues, but it’s difficult (graduate level).

Power Method

Finds largest eigenvalue. Works well on large, sparse matrices.

1

i i

i

i i

Ax x

Ax

Ax λ

+

=

Magnitude!

1

i i

x₊ =Ax , but then you need to scale it by factoring out a number, such that one of the elements becomes 1

Deflation: once you have computed the largest eigenvalue, you can subtract it and repeat the method to find the other eigenvalues from highest to lowest

Inverse Power Method

Finds smallest eigenvalue, instead of largest eigenvalue Plug in A−1 instead of A into the regular power method.

Chapter 10 −

Interpolation

Interpolation is determining the value of a point based on the values of the points around it. You do not need to use all given points.

Multiple methods:

1. Monomial basis functions:

( )

j j x X

ϕ =

a. Not useful: only for introducing the concept, yesterday b. Poorly conditioned

c. O(n3) operations

d. Mostly useful for theory 2. Lagrange (easier)

3. Newton Basis functions

Monomial Basis

(11)

Vandermonde Matrix:

2

0 0

0 0 0

2

1 1

1 1 1

2

1 1

1

n

n n

n n n

c y

x x x

c y

x x x

c y

x x x

     

     

     ₌

     

     

 

 

 

    

 e.g. (1,1),(2,3),(4,3)

( )

0 1 2

1 1

2 2 4 3

3 4 16 3

p c c c

= + + =

0

1

2

1 1 1 1

1 2 4 3

1 4 16 3

c

           ₌                  

Error

blah

Lagrange

Polynomial will be of order n for n+1 points

( )

0

weighting function

0

0 1 1 1

i

n

n i i

i

y n

j i

j i j j i

i i n

i i i i i i i n

f x L x f x

x x L x

x x

x x x x x x x x x x

=

= ≠

− +

=

− =

−

 −  −   −  −   −  =  ₋  ₋   ₋  ₋   ₋ 

       

∑

∏

 

 

Error

blah

Newton Basis Functions

Linear

Given you have to find the value on the curve at a certain point, find 2 points, x0 and x1, such that x0 is below the value and x1 is above the value.

(12)

( )

(

)

( )

(

)

( )

(

)

( )

(

)

( )

1 0 1 0

1 0 0 1 0 0

1 0 0

1 1 1 1 1 0

1 1 1 0 1 1 0

1 0

1

1 0

f x c c x x

f x c

f x c c x x

f x f x c x x f x f x

c

x x

= + −

⇒ =

= + −

⇒ = + −

−

⇒ =

−

Plug that back in and get:

( )

₀

( )

1

( ) (

0 ₀

)

1 0

f x f x

f x f x x x

x x −

= + −

−

This can also be written as f x x

[

₀, ₁

]

Polynomial

The order of your polynomial will be the 1 – the number of points you are given. The order of your polynomial will be the subscript of your function. For example, a quadratic equation would have the form:

( )

(

)

(

)(

)

2 0 1 0 2 0 1

f x = +b b x−x +b x−x x−x

To determine the constants, start by plugging in f(x0):

( )

(

)

2 0 0 1 0 0

0

f x = +b b x −x ₂

(

₀ ₀

)

0

b x x

+ −

(

₀ ₁

)

0

x x

b

−

=

Then plug in x1

( )

(

)

(

)

2 1 0 1 1 0 2 1 1

0

f x = +b b x −x +b x −x

(

)

( )

(

)

(

)

( )

(

)

[

]

1 0

0 1 1 0

1 1 0 1 0

1 0

1 1 0

1 0

,

x x

f x b x x

b x x f x f x

f x f x

b f x x

x x

−

= + −

− = −

−

= =

−

Finally, plug in x2

(13)

( )

(

)

(

)(

)

( )

₍

( )

_{) (}

)

(

)(

)

(

)(

)

( )

[

]

(

)

(

)

( )

[

]

[

] [

]

_[

_]

2 2 0 1 2 0 2 2 1 2 0

1 0

0 2 0 2 2 1 2 0

1 0

2 2 1 2 0 2 0 1 0 2 0

2 0

2 2 1 1 0

2 0

0 2 0 1

2 1 0 2

2 1

, ,

, , f x b b x x b x x x x

f x f x

f x x x b x x x x

x x

b x x x x f x f x f x x x x

f x f x

b x x f x x

x x

f x x f x x

b f x x x

x x

= + − + − −

−

= + − + − −

−

− − = − − −

−

− = −

− −

= =

−

You can also write b2 as f x x x

[

0, ,1 2

]

, where you put the number of points inside the square

brackets. The order of the points doesn’t matter-the result will be the same, so b2 can also be re-written as:

[

₂ ₁ ₀

]

[

2 1

] [

1 0

]

2 0

, ,

, , f x x f x x f x x x

x x − =

−

Summary: _i, , _j i 1, , j i, j 1 j i

f x x f x x

f x x

x x

+ −

 −  

   

  =

    ₋ 

Error

blah

Chapter 15 – Integration Methods

There are 3 quadrature integration methods:

Trapezoidal Midpoint Simpson’s

Multiple Segment Trapezoidal Rule

Since this method finds the integration between a and b by finding the area assuming a straight line between f(a) and f(b), you have some error.

b a h

n

− =

(14)

Error: 1

(

)

3

( )

(

( )

)

12 , , max

t

E = − b a− f′′ α a< <α b f′′ α = f′′ t

To reduce the area, break it up into n segments, such that you have segments:a a, +h,,b h b− , , where

Since each segment is now a different trapezoid, instead of having one overall error, you have multiple errors, such that the first error is:

(

)

( )

3 1

1 12 1 1

3 1

1 1

12

,

, ,

E a h a f a a h

h f f a a h

α α

α α α

′′

= − + − < < + ′′ ′′

= − < < +

The textbook summed it up to:

( )

1

d 2

r b

i a

i h

f x x f a f b h f t

−

=

= _ + _+

∑

∫

Absolute Error

Notice how the a cancelled? Well that’s going to happen all the time, so you can simply assume that the error for a given segment, i, the error will be:

( )

3

1 12

i i

E = − h f′′ α

Therefore the total error:

( )

(

)

_{( )}

3

1

3

1

3

3 1

1 12 1 12

12 n

T i

i

n

i i

n

i i

E h f a

b a f n

α

=

′′ = −

−

  _′′

= − _ _

 

−

′′ = −

∑

However, the textbook sums it up to the following equation:

( )

( ) ( )

2 12

f

E f = − ′′ η b a h−

Midpoint Rule

A.K.A. Rectangle Method or Riemann sum

( )

(

1

)

2 1

d r b

a

i

f x x h f a i h =

≈

∑

+ −

∫

Absolute Error

( )

( ) ( )

2

24 f

E f = ′′ ξ b a h−

Simpson’s 1/3 Rule

A type of quadrature that also interpolates the line.

Choose 3 points, instead of 2, where the 3rd point is the midpoint between a and b. In order for the line to go through all 3 points, so it must be a curve => polynomial of order 2

(15)

( )

2

2 0 1 2

2

2 0 1 2

2

2 2 2 0 1 2 2 2

2

2 0 1 2

,

a b a b a b a b

f x a a x a x a x b f a f a a a a a a

f f a a a

f b f b a a b a b

+ + + +

= + + ≤ ≤

= = + +

( )

2

0 2

1

2 2 2

2 2

1 1 1

a b a b a b

a a _a _{f a}

a f

b b a f b

+ + +

 _{ } _ _

 _{  =}_ _

 _{ } _ _

   

 

 

Now solve the matrix for the coefficients a0, a1, and a2. Finally, solve the integral, where:

( )

(

)

( ) ( )

( )

2 2 3 3

2

0 1 2

0 1 2 2 3

2

d d

4 6

b b

a a

b a b a

a b f x x a a x a x x

a b a a a

b a

f a f f b

− −

+

≈ + +

= − + +

−  

= _ + + _

∫

( )

2

d 4

3 b a

b

a b a

h

f x x f a f f b

−

+

=

 

∴

∫

= _ + + _



The textbook gives:

( )

2 1

d 2

3

r

b

i a

i h

f x x f a f t f b

−

=

 

≈ _ + + _

 



∑



∫

Absolute Error

( )

( )4

( ) ( )

4

180 f

E f = − ζ b a h−

Composite Methods

These methods are a version of the quadrature methods that involves splitting up the interval between the points a and b, into intervals of size hi. This h value is consistent between all the points, except in adaptive quadrature.

Adaptive Quadrature

There is another version of the quadrature methods that have a changing h value, such that there is a smaller h for steeper lines, called adaptive quadrature. The less adaptive the method is, the stiffer.

These methods have the same formulas as the other quadrature methods. However, they have a changing h value.

(16)

Chapter 16−ODE’s

To solve ford

( ) ( )

, , 0 ₀ d

y

f x y y y

x = = , you could use Euler’s formula:yi+1= +yi f x y h

(

i, i

)

, where h=x_i+₁−x_i, but this is inaccurate, so use Runge

Runge-Kutta

Uses Taylor Series

(

)

(

)

(

)

(

)

(

)

(

)

2 3

1 1 2 1 3 1

, _, _,

2 3

d 1 d 1 d

d 2! d 3! d

1 1

, , ,

2! 3!

i i i i i i

i i i i i i i i

x y _{x y} _{x y}

i i i i i i i

y y y

y y x x x x x x

x x x

y f x y h f x y h f x y h

+ = + + − + + − + + − +

′ ′′

= + + + +



The second order Runge-Kutta only includes up to the part where you need to find the second

derivative:

(

)

(

)

2

1

, ,

2!

i i i i i i

y₊ = +y f x y h+ f′ x y h

e.g.)

( )

(

)

( )

(

)

2

2 2

2

d

3 , 0 5 d

, 3

d ,

d

2 3 3

5 9

x

x x

x y

e y y

x

f x y e y

f f y f x y

x y x

e e y

e y

−

− −

−

= − =

= −

∂ ∂

′ = +

∂ ∂

= − + − −

= − +

However, you want to avoid having to find the f’, so they re-write it as:

(

)

1 1 1 2 2

i i

y₊ = y = a k +a k h, where:

(

)

(

)

1

2 1 11 1

, , i i

i i

k f x y

k f x p h y q k h =

= + +

With unknowns a1, a2, p1, and q11. However, there are 3 more equations determined that can help to determine the values.

1 2

2 1

2 11

1 1 2 1 2 a a a p a q

+ =

=

Now the question is how much weighting you’re going to apply between the two slopes to solve for the unknowns. How this is done is by different guesses of the value of a2:

• Heun’s Method: ₂ 1 2 a =

• Midpoint method:a₂ =1

(17)

• Ralston’s method: ₂ 2 3 a =

Heun’s Method

If you assume: ₂ 1 2 a = ,

1 2 1

2 1 1

2 11 11

1 1

2 1

1 2

1

1 2

a a a

a p p

a q q

+ = ⇒ =

= ⇒ =

1 1 2

average of 2 slopes

1 1

2 2

i i

y₊ y  k k h

∴ = +_ + _

___

(

)

(

)

1

2 1

, , i i

i i

k f x y

k f x h y k h =

= + +

Midpoint Method

Assumea₂ =1

1 2 1

2 1 1

2 11 11

1 0

1 1

2 2

1 1

2 2

a a a

a p p

a q q

+ = ⇒ =

= ⇒ =

(

)

1 2

1

2 1

,

1 1

,

2 2

i i

y y k h k f x y

k f x h y k h + = +

=

 

= _ + + _

 

Ralston’s Method

2

1 2 1

2 1 1

2 11 11

2 3

1 1

3

1 3

2 4

1 3

2 4

a

a a a

a p p

a q q

=

+ = ⇒ =

= ⇒ =

(18)

(

)

1 1 2

1

2 1

1 2

3 3

,

3 3

,

4 4

i i

y y k k h

k f x y

k f x h y k h +

 

= +_ + _

 

=

 

= _ + + _

 

Forwards Euler

This is an explicit method to find the solution for an ODE. Compared to its implicit counterpart, Backwards Euler, it is more efficient for less stiff equations

Use the tangent line to determine the next point of the line. The higher you set the step size (the less stiff), the more accurate.

( )

(

,

)

( )

f x y x = y x′

( )

(

)

(

)

(

)

( )

1 1

0

0 0 0

1 1 1

1 1

, ,

i i i i

i i i

y y x

y x h

y f x y h

y y f x y h

y y y x h

− − −

− −

=

= +

′

= +

Error

blah

Stability

The quality of stability is mainly to describe how the method responds to different step sizes (h values).

To demonstrate stability, use the Dahlquist Test Equation, which is where we assume

( )

y x′ = −ky x

As an example, observe the analysis of the Dahlquist test equation using forward euler.

(19)

( )

( )(

)

( )

( )(

)

( )(

)

( )(

)(

)

( )(

)

2

0 0

0 1 2

0 1 0 1

0 1 1

0 1 i

y h y y h

y ky h

y kh

y h y h y h h

y h ky h h

y kh ky kh h

y kh kh

y kh ←

′

= +

= −

′

= +

= −

= − − −

= − −

= −

Notice how the power will appear to continuously increase? Stability is making sure the constant does not go to infinity as i approaches infinity. To do this, the initial value of the bracket must be between −1 and 1. To do this, set an h value such that:

1 1

2 2

2 hk

hk

h k − > −

− > − <

<

Backwards Euler

This implicit method for finding ODE’s is more efficient for stiffer equations than its explicit counterpart, forwards Euler.

This is the equation:y_i₊₁= +y_i f x

(

_i₊₁,y_i₊₁

)

h

Since y_i+₁is on both sides of the equation, you need to bring it over to the left side and solve:

(

)

( )

1 1, 1 1 1

i i i i i i i

y₊ − f x₊ y₊ h= y ⇒ y₊ −y x′ ₊ h= y

and solve for y_i₊₁.

If f x y

( )

, = −kxy, you would have:

(

)

(

)

(

)

1 1 1

1 1

1

1 0

1

1 1 1

i i i i

i i i

i i

i

i y kx y h y

y kx h y

y y

kx h

y y

kx h + + +

+ +

+

+ =

= +

 

= _ ₊ _

 

If you take the limit as i → ∞, you can see that the fraction approaches 0, so this method is the more stable one.

Error

blah