Numerical Methods for Differential Equations
Chapter 1: Initial value problems in ODEs
Gustaf S ¨oderlind and Carmen Ar ´evalo Numerical Analysis, Lund University
Textbooks: A First Course in the Numerical Analysis of Differential Equations, by Arieh Iserles and Introduction to Mathematical Modelling with Differential Equations, by Lennart Edsberg
c Gustaf S ¨oderlind, Numerical Analysis, Mathematical Sciences, Lund University, 2008-09
Chapter 1: contents
◮ Course contents
◮ Introduction to initial value problems
◮ The explicit Euler method
◮ Convergence
◮ Order of consistency
◮ The trapezoidal rule
◮ Theta methods
◮ Numerical tests
◮ The linear test equation and numerical stability
◮ Stiff equations
What is Numerical Analysis?
Categories of mathematical problems
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
Algebra: Only finite constructions
Analysis: Limits, derivatives, integrals etc. (transfinite)
Computable: the exact solution can be obtained in a
finite number of operations
What is Numerical Analysis?
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
With numerical methods, problems from all four categories can be solved:
“Numerical analysis aims to construct and analyze
quantitative methods for the automatic computation of approximate solutions to mathematical problems.”
Goal: Construction of mathematical software
0. What will we study in this course?
To solve a differential equation analytically we look for a differentiable function that satisfies the equation
Large, complex and nonlinear systems cannot be solved
analytically
0. What will we study in this course?
To solve a differential equation analytically we look for a differentiable function that satisfies the equation
Large, complex and nonlinear systems cannot be solved analytically
Instead, we compute numerical solutions with standard methods and software
To solve a differential equation numerically we generate a sequence {y k } N k=0 of pointwise approximations to the
analytical solution: y(t k ) ≈ y k
Some differential equations we will solve
◮ Initial value problems (IVP) first-order equations;
higher-order equations; systems of differential
equations
Some differential equations we will solve
◮ Initial value problems (IVP) first-order equations;
higher-order equations; systems of differential equations
◮ Boundary value problems (BVP) two-point boundary
value problems; Sturm-Liouville eigenvalue problems
Some differential equations we will solve
◮ Initial value problems (IVP) first-order equations;
higher-order equations; systems of differential equations
◮ Boundary value problems (BVP) two-point boundary value problems; Sturm-Liouville eigenvalue problems
◮ Partial differential equations (PDE) the diffusion
equation; the advection equation; the wave equation
Some differential equations we will solve
◮ Initial value problems (IVP) first-order equations;
higher-order equations; systems of differential equations
◮ Boundary value problems (BVP) two-point boundary value problems; Sturm-Liouville eigenvalue problems
◮ Partial differential equations (PDE) the diffusion
equation; the advection equation; the wave equation
◮ Applications in all three areas
Initial value problems: examples
A first-order equation: a simple equation without a known analytical solution
dy
dt = y − e −t 2 , y(0) = y 0
Initial value problems: examples
A first-order equation: a simple equation without a known analytical solution
A second-order equation: motion of a pendulum θ ′′ (t) + g
L sin θ(t) = 0, θ(0) = θ 0 , θ ′ (0) = θ 0 ′
where θ(t) is the angle, g is the gravitational constant and
L is the pendulum length
Initial value problems: examples
A first-order equation: a simple equation without a known analytical solution
A second-order equation: pendulum equation A system of equations: the predator–prey model
y 1 ′ (t) = k 1 y 1 (t) − k 2 y 1 (t) y 2 (t) y 2 ′ (t) = k 3 y 1 (t) y 2 (t) − k 4 y 2 (t)
where y 1 (t) is the population of the prey species and y 2 (t)
is the population of the predator species at time t
Boundary value problems: examples
Second-order two-point BVP: the electrostatic potential u(r) between two concentric metal spheres satisfies
d 2 u
dr 2 + 2 r
du
dr = 0, u(R 1 ) = V 1 , u(R 2 ) = 0
at the distance r from the center; R 1 and R 2 are the radii of
the two spheres
Boundary value problems: examples
Second-order two-point BVP: the electrostatic potential u(r) between two concentric metal spheres satisfies
d 2 u
dr 2 + 2 r
du
dr = 0, u(R 1 ) = V 1 , u(R 2 ) = 0
at the distance r from the center; R 1 and R 2 are the radii of the two spheres
A Sturm-Liouville eigenproblem: Euler buckling of column y ′′ + λy = 0, y ′ (0) = 0, y(1) = 0
Find eigenvalues λ and eigenfunctions y
Partial differential equations: examples
The heat equation
u t (x, t) = u xx (x, t), x ∈ [0, a), t ∈ (0, b) u(x, 0) = f (x), x ∈ [0, a]
u(0, t) = c 1 , u(a, t) = c 2 , t ∈ [0, b]
is a parabolic PDE modelling e.g. the temperature in an
insulated rod with constant temperatures c 1 and c 2 at its
ends, and initial temperature distribution f (x)
Partial differential equations: examples
The heat equation The wave equation
u tt (x, t) = u xx (x, t), x ∈ (0, a), t ∈ (0, b) u(0, t) = 0, u(a, t) = 0 for t ∈ [0, b]
u(x, 0) = f (x) for x ∈ [0, a]
u t (x, 0) = g(x) for x ∈ (0, a)
is a hyperbolic PDE e.g. modelling the displacement u of a
vibrating elastic string fixed at x = 0 and x = a
Some applications in ODEs
Initial value problems
• mechanics M ¨ q = F (q)
• electrical circuits C ˙v = −I(v)
• chemical reactions ˙c = f (c)
Boundary value problems
• materials u ′′ = M (u)/EI
• microphysics − 2m ~ ψ ′′ = E
• eigenmodes u ′′ + λu = 0
1. Initial value problems
Standard formulation
y ′ = f (t, y) ; y(0) = y 0
Scalar equation: y ∈ R ; f scalar;
Systems of ODEs: y ∈ R m ; f vector-valued.
Theorem (Cauchy & Peano) A solution to y ′ = f (t, y)
exists if f is continuous.
Existence
Note: Continuity is not sufficient to guarantee uniqueness!
Example:
˙y = 2√y ; y(0) = 0 with solution
y(t) = 0 , t ≤ τ and y(t) = (t − τ) 2 , t > τ
This may happen in real physical models!
Consider the energy of a falling particle of mass m :
1
2 m ˙y 2 + mgy = E := 0 This yields ˙y = − √
2g √
−y with y(0) = 0
Existence and uniqueness
Theorem If f (t, y) is continuous on [0, T ] and satisfies the Lipschitz condition
kf(t, u) − f(t, v)k ≤ L[f] · ku − vk
for all u, v and some Lipschitz constant L[f ] < ∞ , there exists a unique solution to the initial value problem
y ′ = f (t, y)
on [0, T ] for every initial value y(0) = y 0
Existence and uniqueness . . .
Note Most problems do not satisfy a Lipschitz condition on all of R m
Example The problem
˙y = y 2 ; y(0) = y 0 > 0 has solution
y(t) = 1−y y 0
0 t
The solution blows up at t = 1/y 0 (“Finite escape time”)
Existence and uniqueness . . .
Other problems always satisfy Lipschitz conditions on R m
Example
˙y = Ay; y(0) = y 0
Lipschitz constant:
L[f ] = max u6=v kAu−Avk ku−vk = max y6=0 kAyk kyk = kAk
The matrix norm kAk is a Lipschitz constant for f (y) = Ay
Standard form: x ′ = F (t, x)
Example
y ′′ = f (t, y, y ′ ); y(0) = y 0 ; y ′ (0) = y 0 ′
Standard substitution Introduce new variables x 1 = y
x 2 = y ′
Then we get a system of first order equations x ′ 1 = x 2
x ′ 2 = f (t, x 1 , x 2 )
with x 1 (0) = y 0 and x 2 (0) = y 0 ′
2. The Explicit Euler method (1768)
y ′ = f (t, y); y(t 0 ) = y 0
Replace derivative by finite difference approximation y ′ (t n ) ≈ y(t n + h) − y(t n )
h
Let {u n } denote the numerical approximation to {y(t n )} and
h = t n+1 − t n denote the time-step.
2. The Explicit Euler method (1768)
y ′ = f (t, y); y(t 0 ) = y 0
Replace derivative by finite difference approximation y ′ (t n ) ≈ y(t n + h) − y(t n )
h
Let {u n } denote the numerical approximation to {y(t n )} and h = t n+1 − t n denote the time-step. Compute u n from
u n+1 − u n
h = f (t n , u n ), u 0 = y 0
2. The Explicit Euler method (1768)
y ′ = f (t, y); y(t 0 ) = y 0
Replace derivative by finite difference approximation y ′ (t n ) ≈ y(t n + h) − y(t n )
h
Let {u n } denote the numerical approximation to {y(t n )} and h = t n+1 − t n denote the time-step. Compute u n from
u n+1 − u n
h = f (t n , u n ), u 0 = y 0
The Explicit Euler method Given u 0 , t 0 and h , compute u n+1 = u n + hf (t n , u n )
t n+1 = t n + h
Explicit Euler: Taylor series expansion
y ′ (t) = f (t, y), y(t 0 ) = y 0 Use Taylor series expansion
y(t + h) = y(t) + hy ′ (t) + h 2
2! y ′′ (ζ)
= y(t) + hf (t, y(t)) + O(h 2 ) ⇒
y(t + h) ≈ y(t) + hf(t, y(t))
Explicit Euler: Taylor series expansion
y ′ (t) = f (t, y), y(t 0 ) = y 0 Use Taylor series expansion
y(t + h) = y(t) + hy ′ (t) + h 2
2! y ′′ (ζ)
= y(t) + hf (t, y(t)) + O(h 2 ) ⇒ y(t + h) ≈ y(t) + hf(t, y(t))
Construct the numerical method (drop higher order terms) u n+1 = u n + hf (t n , u n ); u 0 = y 0
t n+1 = t n + h
Explicit Euler: Taylor series expansion
y ′ (t) = f (t, y), y(t 0 ) = y 0 Use Taylor series expansion
y(t + h) = y(t) + hy ′ (t) + h 2
2! y ′′ (ζ)
= y(t) + hf (t, y(t)) + O(h 2 ) ⇒ y(t + h) ≈ y(t) + hf(t, y(t))
Construct the numerical method (drop higher order terms) u n+1 = u n + hf (t n , u n ); u 0 = y 0
t n+1 = t n + h
Explicit Euler!
Explicit Euler: graphic interpretation
“Take a step of size h in the direction of the tangent”
0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 1.15 1.2
t
y
Each step introduces an error and ends up on a different
solution trajectory (dashed curves)
Explicit Euler: an example
Compute numerical solution to y ′ = −y cos t ; y(0) = 1 ; t ∈ [0, 2π]
Solution Choose a stepsize h = 2π/N ; N = 24 ⇒ h = π/12 Recursion u n+1 = u n + h (−u n cos t n ); u 0 = 1
t n+1 = t n + h; t 0 = 0
0 1 2 3 4 5 6 7
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
0 1 2 3 4 5 6 7
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
The numerical solution is a sequence of points (t , u )
3. Convergence
Analytical and numerical solutions for h = π/8 and π/128
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=64
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=64*16
As h → 0 the numerical solution approaches the exact solution
3. Convergence
Analytical and numerical solutions for h = π/8 and π/128
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=64
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=64*16
As h → 0 the numerical solution approaches the exact solution A method is convergent if, for every ODE with a Lipschitz
function f and every fixed T , with T = N · h , it holds that
lim ky N,h − y(T )k = 0
Local and global errors
1.3 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75 1.8
0.95 1 1.05 1.1 1.15
t
y
y n+1
• y n
•
ˆ y n+1
• y(t n ) y(t)
• •
y(t n+1 )
Global error e n = y n − y(t n ) and e n+1 = y n+1 − y(t n+1 )
Local error l n +1 = y ˆ n +1 − y(t n +1 )
The local error of the explicit Euler method
Insert exact data ˆ
y n+1 = y(t n ) + hf (t n , y(t n ))
Local error definition l n+1 = ˆ y n+1 − y(t n+1 ) implies residual y(t n+1 ) = y(t n ) + hf (t n , y(t n )) − l n+1
Taylor series y(t n +1 ) = y(t n ) + hy ′ (t n ) + h 2 2 y ′′ (t n ) + . . . ⇒ Local error for explicit Euler l n +1 ≈ − h 2 2 y ′′ (t n )
(along exact solution, solid curve)
Error propagation
Explicit Euler (numerical solution)
y n+1 = y n + hf (t n , y n )
Subtract Taylor series expansion of exact solution
y(t n+1 ) = y(t n ) + hf (t n , y(t n )) + h 2
2 y ′′ (t n ) + . . . Global error recursion
e n +1 = e n + hf (t n , y(t n ) + e n ) − hf(t n , y(t n )) + l n +1
Error propagation . . .
e n+1 = e n + hf (t n , y(t n ) + e n ) − hf(t n , y(t n )) + l n+1 Take norms and use Lipschitz condition:
ke n+1 k ≤ ke n k + hL[f] · ke n k + kl n+1 k
Lemma If {a n }, a 0 = 0, is a sequence of non-negative numbers satisfying
a n+1 ≤ (1 + h µ)a n + ch 2 for µ > 0, then a n ≤ c
µ h [(1 + h µ) n − 1], n = 0, 1, . . .
Convergence of Euler’s method: Theorem
Theorem The explicit Euler method is convergent
Convergence of Euler’s method: Theorem
Theorem The explicit Euler method is convergent
Proof Suppose f sufficiently differentiable. Given h and a
fixed T = N h , let e n,h = y n,h − y(t n )
Convergence of Euler’s method: Theorem
Theorem The explicit Euler method is convergent
Proof Suppose f sufficiently differentiable. Given h and a fixed T = N h , let e n,h = y n,h − y(t n )
Apply the lemma to global error recursion, to get
ke n,h k ≤ c
L[f ] h [(1 + h L[f ]) n − 1], n = 0, 1, . . . with
c = max
n kl n k/h 2 ≈ max t ky ′′ k/2
Convergence of Euler’s method . . .
As (1 + h L[f ]) n < e nhL [f ] ≤ e T L [f ] , we have for n ≤ N ,
ke n,h k ≤ c
L[f ] h (e T L[f ] − 1) So
ke n,h k ≤ C(T ) · h implies convergence because
h→0 lim ke n,h k = 0
Note 1) The global error can be made arbitrarily small!
2) The error is way too large for practical purposes!
3) Better bounds can be obtained
Theoretical error “bound”
Example
y ′ = −100y, y(0) = 1.
Then L[f ] = 100 and the exact solution is y(t) = e −100t with y ′′ (t) = 100 2 e −100t , so c = 100 2 /2 , with bound
ke n,h k ≤ 100 2
2 · 100 h (e 100T − 1)
Theoretical error “bound”
Example
y ′ = −100y, y(0) = 1.
Then L[f ] = 100 and the exact solution is y(t) = e −100t with y ′′ (t) = 100 2 e −100t , so c = 100 2 /2 , with bound
ke n,h k ≤ 100 2
2 · 100 h (e 100T − 1)
Error estimate at T = 1 is ke n,h k ≤ 50 h e 100 ≈ 1.4 · 10 45 h !
Theoretical error “bound”
Example
y ′ = −100y, y(0) = 1.
Then L[f ] = 100 and the exact solution is y(t) = e −100t with y ′′ (t) = 100 2 e −100t , so c = 100 2 /2 , with bound
ke n,h k ≤ 100 2
2 · 100 h (e 100T − 1)
Error estimate at T = 1 is ke n,h k ≤ 50 h e 100 ≈ 1.4 · 10 45 h ! But y n = (1 − 100h) n , so for h < 1/50 , at T = 1 ,
Actual error is ke n,h k = |(1 − 100/N) N − e −100 | ≤ 3.7 · 10 −44 h !
Computational test ˙y = λ(y − sin t) + cos t
λ = −0.2 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/10 h = π/20
Note Local error O(h 2 ) , global error O(h) !
4. Order of consistency
Given a method in the form y n+1 = Φ n (f, h, y 0 , y 1 , . . . , y n ) Insert exact data y(t 0 ), y(t 1 ), . . . , y(t n )
Definition The order of consistency is p if
y(t n+1 ) − Φ n (f, h, y(t 0 ), y(t 1 ), . . . , y(t n )) = O(h p +1 ), ∀n as h → 0 , for every analytic f
Theorem The local error is then O(h p+1 )
4. Order of consistency
Given a method in the form y n+1 = Φ n (f, h, y 0 , y 1 , . . . , y n ) Insert exact data y(t 0 ), y(t 1 ), . . . , y(t n )
Definition The order of consistency is p if
y(t n+1 ) − Φ n (f, h, y(t 0 ), y(t 1 ), . . . , y(t n )) = O(h p +1 ), ∀n as h → 0 , for every analytic f
Theorem The local error is then O(h p+1 )
Alternative The order of consistency is p if the formula is
exact for all polynomials y = P (t) of degree p or less
Order of consistency of Euler’s method
Example Explicit Euler Φ n (f, h, y 0 , . . . , y n ) = y n + h f (t n , y n )
Order of consistency of Euler’s method
Example Explicit Euler Φ n (f, h, y 0 , . . . , y n ) = y n + h f (t n , y n ) Expanding in Taylor series,
y(t n+1 ) − [y(t n ) + h f (t n , y(t n ))] = O(h 2 )
so the method’s consistency order is one
Order of consistency of Euler’s method
Example Explicit Euler Φ n (f, h, y 0 , . . . , y n ) = y n + h f (t n , y n ) Expanding in Taylor series,
y(t n+1 ) − [y(t n ) + h f (t n , y(t n ))] = O(h 2 )
so the method’s consistency order is one
◮ Alternatively, suppose y(t) = 1 with f = y ′ = 0 . Then
1 = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 1 + h · 0 = 1 ⇒ exact!
Order of consistency of Euler’s method
Example Explicit Euler Φ n (f, h, y 0 , . . . , y n ) = y n + h f (t n , y n ) Expanding in Taylor series,
y(t n+1 ) − [y(t n ) + h f (t n , y(t n ))] = O(h 2 )
so the method’s consistency order is one
◮ Alternatively, suppose y(t) = 1 with f = y ′ = 0 . Then 1 = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 1 + h · 0 = 1 ⇒ exact!
◮ Next take 1st degree polynomial y(t) = t with f = y ′ = 1 , then
h = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 0 + h · 1 = h ⇒ exact!
Order of consistency of Euler’s method
Example Explicit Euler Φ n (f, h, y 0 , . . . , y n ) = y n + h f (t n , y n ) Expanding in Taylor series,
y(t n+1 ) − [y(t n ) + h f (t n , y(t n ))] = O(h 2 )
so the method’s consistency order is one
◮ Alternatively, suppose y(t) = 1 with f = y ′ = 0 . Then 1 = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 1 + h · 0 = 1 ⇒ exact!
◮ Next take 1st degree polynomial y(t) = t with f = y ′ = 1 , then h = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 0 + h · 1 = h ⇒ exact!
◮ For a 2nd degree polynomial, y(t) = t 2 with f = y ′ = 2t . Then
h 2 = y(t n+1 ) = y(t n ) + hf (t n , y(t n )) = 0 + h · 0 = 0 6= h 2 ⇒ p = 1 !
5. The trapezoidal rule
Explicit Euler linearizes y at t n with a slope of y ′ (t n )
5. The trapezoidal rule
Explicit Euler linearizes y at t n with a slope of y ′ (t n )
Instead, linearize with a slope equal to the average of y ′ (t n ) and y ′ (t n+1 )
y(t) ≈ y(t n ) + (t − t n ) 1
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]
5. The trapezoidal rule
Explicit Euler linearizes y at t n with a slope of y ′ (t n )
Instead, linearize with a slope equal to the average of y ′ (t n ) and y ′ (t n+1 )
y(t) ≈ y(t n ) + (t − t n ) 1
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]
The numerical method is the trapezoidal rule t n+1 = t n + h
y n+1 = y n + h
2 [f (t n , y n ) + f (t n+1 , y n+1 )]
5. The trapezoidal rule
Explicit Euler linearizes y at t n with a slope of y ′ (t n )
Instead, linearize with a slope equal to the average of y ′ (t n ) and y ′ (t n+1 )
y(t) ≈ y(t n ) + (t − t n ) 1
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]
The numerical method is the trapezoidal rule t n+1 = t n + h
y n+1 = y n + h
2 [f (t n , y n ) + f (t n+1 , y n+1 )]
The method is implicit . Nonlinear equation solving required
Trapezoidal rule: Order and convergence
Order of consistency (sketch):
y(t n+1 ) − {y(t n ) + h
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]}
= y(t n ) + h y ′ (t n ) + h 2
2 y ′′ (t n ) + O(h 3 )
−{y(t n ) + h
2 (y ′ (t n ) + [y ′ (t n ) + h y ′′ (t n ) + O(h 2 )])}
= O(h 3 )
Trapezoidal rule: Order and convergence
Order of consistency (sketch):
y(t n+1 ) − {y(t n ) + h
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]}
= y(t n ) + h y ′ (t n ) + h 2
2 y ′′ (t n ) + O(h 3 )
−{y(t n ) + h
2 (y ′ (t n ) + [y ′ (t n ) + h y ′′ (t n ) + O(h 2 )])}
= O(h 3 )
The method is of order two
Trapezoidal rule: Order and convergence
Order of consistency (sketch):
y(t n+1 ) − {y(t n ) + h
2 [f (t n , y(t n )) + f (t n+1 , y(t n+1 ))]}
= y(t n ) + h y ′ (t n ) + h 2
2 y ′′ (t n ) + O(h 3 )
−{y(t n ) + h
2 (y ′ (t n ) + [y ′ (t n ) + h y ′′ (t n ) + O(h 2 )])}
= O(h 3 )
The method is of order two
Theorem The trapezoidal rule is convergent
(No proof given here)
Trapezoidal rule: the dramatic effect of 2nd order
Compute numerical solution to y ′ = −y cos t ; y(0) = 1 ; t ∈ [0, 8π] . Choose h = π/12 . Note that f is linear in y . The recursion is
u n+1 = 1 − h 2 cos t n
1 + h 2 cos t n+1 u n ; u 0 = 1
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=96 Trapezoidal rule
0 5 10 15 20 25
0 0.5 1 1.5 2 2.5 3
N=96 Explicit Euler
Solutions with Trapezoidal rule and Explicit Euler
The implicit midpoint rule
We can also approximate the derivative y ′ (t) by taking the average of t n and t n+1 as well as y n and y n+1 :
y ′ (t) ≈ f( t n + t n+1
2 , y n + y n+1
2 ), t ∈ [t n , t n+1 ]
The implicit midpoint rule
We can also approximate the derivative y ′ (t) by taking the average of t n and t n+1 as well as y n and y n+1 :
y ′ (t) ≈ f( t n + t n+1
2 , y n + y n+1
2 ), t ∈ [t n , t n+1 ]
The resulting method is the 2nd order implicit midpoint method t n+1 = t n + h
y n+1 = y n + h f ( t n + t n+1
2 , y n + y n+1
2 )
6. Theta methods
We construct methods that linearize y at t n with a slope equal to a convex combination of y ′ (t n ) and y ′ (t n+1 ) :
y n +1 = y n + h [θf (t n , y n ) + (1 − θ)f(t n +1 , y n +1 )], θ ∈ [0, 1]
◮ Explicit Euler: θ = 1
◮ Trapezoidal rule (implicit): θ = 1/2
◮ Implicit Euler: θ = 0 ⇒ y n+1 = y n + h f (t n+1 , y n+1 )
Theta methods: Order and convergence
Use Taylor expansion to get
y(t n+1 ) − y(t n ) − h [θf(t n , y(t n )) + (1 − θ)f(t n+1 , y(t n+1 ))]
= (θ − 1
2 )h 2 y ′′ (t n ) + 1
2 (θ − 2
3 )h 3 y ′′′ (t n ) + O(h 4 )
Theta methods: Order and convergence
Use Taylor expansion to get
y(t n+1 ) − y(t n ) − h [θf(t n , y(t n )) + (1 − θ)f(t n+1 , y(t n+1 ))]
= (θ − 1
2 )h 2 y ′′ (t n ) + 1
2 (θ − 2
3 )h 3 y ′′′ (t n ) + O(h 4 )
If θ = 1/2 the method is of order 2; otherwise it is of order 1
Theta methods: Order and convergence
Use Taylor expansion to get
y(t n+1 ) − y(t n ) − h [θf(t n , y(t n )) + (1 − θ)f(t n+1 , y(t n+1 ))]
= (θ − 1
2 )h 2 y ′′ (t n ) + 1
2 (θ − 2
3 )h 3 y ′′′ (t n ) + O(h 4 ) If θ = 1/2 the method is of order 2; otherwise it is of order 1
Theorem (without proof) The θ methods are convergent
Implicit methods
Implicit Euler y n +1 = y n + h f (t n +1 , y n +1 )
We need to solve a nonlinear equation to compute y n+1 The extra cost is motivated if we can take larger steps
There are some problems where implicit methods can take enormous time steps without losing accuracy!
We will return to how to solve nonlinear equations
7. Tests: Explicit Euler ˙y = λ(y − sin t) + cos t
λ = −0.2 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/10 h = π/20
Local error O(h 2 ) , global error O(h) !
Tests: Implicit Euler ˙y = λ(y − sin t) + cos t
λ = −0.2 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/10 h = π/20
Local error O(h 2 ) , global error O(h) !
Tests: Implicit Euler ˙y = λ(y − sin t) + cos t
λ = −10 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/10 h = π/20
Local error = O(h 2 ) , global error = O(h) , but difficult to see!
Tests: Explicit Euler ˙y = λ(y − sin t) + cos t
λ = −10 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/20
Tests: Explicit Euler ˙y = λ(y − sin t) + cos t
λ = −10 , with initial condition y(π/4) = 1/ √ 2
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t
y
h = π/10 h = π/20
NUMERICAL INSTABILITY! Stability for h small
8. The linear test equation
Defintion The linear test equation is
y ′ = λy; y(0) = 1, t ≥ 0 , where λ ∈ C
8. The linear test equation
Defintion The linear test equation is
y ′ = λy; y(0) = 1, t ≥ 0 , where λ ∈ C
As y(t) = e λt , we have
|y(t)| ≤ K ⇔ Re(λ) ≤ 0
Mathematical stability Bounded solutions if Re(λ) ≤ 0
8. The linear test equation
Defintion The linear test equation is
y ′ = λy; y(0) = 1, t ≥ 0 , where λ ∈ C
As y(t) = e λt , we have
|y(t)| ≤ K ⇔ Re(λ) ≤ 0
Mathematical stability Bounded solutions if Re(λ) ≤ 0 When does a numerical method have the same property?
Does Re(λ) ≤ 0 imply numerical stability?
Numerical stability: The stability region
Definition The stability region D of a method is the set of all
hλ ∈ C such that |y n | ≤ ˜ K when the method is applied to the test equation
4
Numerical stability: The stability region
Definition The stability region D of a method is the set of all
hλ ∈ C such that |y n | ≤ ˜ K when the method is applied to the test equation
Example For Euler’s method, y n+1 = (1 + hλ)y n , so y n remains bounded if and only if |1 + hλ| ≤ 1
4
Numerical stability: The stability region
Definition The stability region D of a method is the set of all
hλ ∈ C such that |y n | ≤ ˜ K when the method is applied to the test equation
Example For Euler’s method, y n+1 = (1 + hλ)y n , so y n remains bounded if and only if |1 + hλ| ≤ 1
−4 −3 −2 −1 0 1 2 3 4
−4
−3
−2
−1 0 1 2 3 4
D Euler = {z ∈ C : |1 + z| ≤ 1}
A-stability
Definition A method is called A-stable if its stability region
contains C − , i.e. C − ≡ {z ∈ C : Re(z) ≤ 0} ⊂ D .
A-stability
Definition A method is called A-stable if its stability region contains C − , i.e. C − ≡ {z ∈ C : Re(z) ≤ 0} ⊂ D .
For the trapezoidal rule D TR =
(
z ∈ C :
1 + 1 2 z 1 − 1 2 z
≤ 1 )
≡ C −
so if Re(z) ≤ 0 then y n remains bounded for any h > 0
A-stability
Definition A method is called A-stable if its stability region contains C − , i.e. C − ≡ {z ∈ C : Re(z) ≤ 0} ⊂ D .
For the trapezoidal rule D TR =
(
z ∈ C :
1 + 1 2 z 1 − 1 2 z
≤ 1 )
≡ C −
so if Re(z) ≤ 0 then y n remains bounded for any h > 0
The explicit Euler method is not A-stable but the implicit Euler method and the trapezoidal rule are A-stable
Usually: “If the original problem is stable, then an A-stable
method will replicate that behavior numerically”
Relevance of the linear test equation
When we have a system of equations y ′ = Ay
and A has a full set of eigenvectors with eigenvalues Λ = diag(λ 1 , λ 2 , . . . , λ N ) , then
◮ A can be diagonalized, T −1 AT = Λ
◮ Putting y = T x transforms the system to x ′ = Λx (scalar systems)
◮ . . . equal to x ′ k = λ k x k for k = 1, . . . , d
◮ The exact solution y(t) = e tA y 0 is stable if and only if
Re(λ k ) ≤ 0 for all k = 1, . . . , d
Relevance of the linear test equation . . .
Applying a method to y ′ = Ay gives (say)
y n+1 = (I + hA)y n
Relevance of the linear test equation . . .
Applying a method to y ′ = Ay gives (say) y n+1 = (I + hA)y n
Noting that T −1 (I + hA)T = I + hΛ , putting y n = T x n produces
x n+1 = (I + hΛ)x n
Relevance of the linear test equation . . .
Applying a method to y ′ = Ay gives (say) y n+1 = (I + hA)y n
Noting that T −1 (I + hA)T = I + hΛ , putting y n = T x n produces x n+1 = (I + hΛ)x n
The same as applying the method to the diagonalized system
x ′ = Λx
Relevance of the linear test equation . . .
Applying a method to y ′ = Ay gives (say) y n+1 = (I + hA)y n
Noting that T −1 (I + hA)T = I + hΛ , putting y n = T x n produces x n+1 = (I + hΛ)x n
The same as applying the method to the diagonalized system x ′ = Λx
Diagonalization and discretization commute! Stability condition
from linear test equation applies to all diagonalizable systems!
9. Stiff ODEs
Example Solve ˙y = λ(y − sin t) + cos t with λ = −50 Solution
Particular: y P (t) = sin t Homogeneous: y H (t) = e λt
General: y(t) = e λ(t−t 0 ) (y(t 0 ) − sin t 0 ) + sin t
Study the flow of this equation and numerical solutions
Flow (solution trajectories)
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
t