Numerical Methods for Differential Equations
Course objectives and preliminaries
Gustaf S ¨oderlind and Carmen Ar ´evalo Numerical Analysis, Lund University
Textbooks: A First Course in the Numerical Analysis of Differential Equations, by Arieh Iserles and Introduction to Mathematical Modelling with Differential Equations, by Lennart Edsberg
Chapter 0: Objectives and Preliminaries
◮ Course structure and objectives
◮ What is Numerical Analysis?
◮ The four principles of Numerical Analysis
1. Course structure and objectives
To solve a differential equation analytically we look for a differentiable function that satisfies the equation
Large, complex and nonlinear systems cannot be solved
analytically
1. Course structure and objectives
To solve a differential equation analytically we look for a differentiable function that satisfies the equation
Large, complex and nonlinear systems cannot be solved analytically
Instead, we compute numerical solutions with standard
methods and software
Course objectives
◮ Understand correspondence between physics and mathematical equations
◮ Learn basic numerical techniques for solving differential equations
◮ Understand mathematics–numerics interaction, and how to match numerical method to mathematical properties
◮ Construct elementary M ATLAB programs for differential
Course topics
◮ IVP Initial value problems first-order equations;
higher-order equations; systems of differential equations
Course topics
◮ IVP Initial value problems first-order equations;
higher-order equations; systems of differential equations
◮ BVP Boundary value problems two-point boundary
value problems; Sturm–Liouville eigenvalue problems
Course topics
◮ IVP Initial value problems first-order equations;
higher-order equations; systems of differential equations
◮ BVP Boundary value problems two-point boundary value problems; Sturm–Liouville eigenvalue problems
◮ PDE Partial differential equations diffusion equation;
advection equation; wave equation
Course topics
◮ IVP Initial value problems first-order equations;
higher-order equations; systems of differential equations
◮ BVP Boundary value problems two-point boundary value problems; Sturm–Liouville eigenvalue problems
◮ PDE Partial differential equations diffusion equation;
advection equation; wave equation
◮ Applications in all three areas
Initial value problems. Examples
A first-order equation A simple equation without elementary analytical solution
dy
dt = y − e − t
2, y(0) = y 0
Initial value problems. Examples
A first-order equation A simple equation without elementary analytical solution
A second-order equation Motion of a pendulum θ + ¨ g
L sin θ = 0, θ(0) = θ 0 , ˙θ(0) = ˙θ 0
where θ(t) is the angle, g is the gravitational constant and L is
the pendulum length
Initial value problems. Examples
A first-order equation A simple equation without elementary analytical solution
A second-order equation Pendulum equation A system of equations Predator–prey model
˙y 1 = k 1 y 1 − k 2 y 1 y 2
˙y 2 = k 3 y 1 y 2 − k 4 y 2
where y 1 (t) is the population of the prey species and y 2 (t) is
the population of the predator species at time t
Boundary value problems. Examples
Second-order two-point BVP The electrostatic potential u(r) between two concentric metal spheres satisfies
d 2 u
dr 2 + 2 r
du
dr = 0, u(R 1 ) = V 1 , u(R 2 ) = 0
at the distance r from the center; R 1 and R 2 are the radii of
the two spheres
Boundary value problems. Examples
Second-order two-point BVP The electrostatic potential u(r) between two concentric metal spheres satisfies
d 2 u
dr 2 + 2 r
du
dr = 0, u(R 1 ) = V 1 , u(R 2 ) = 0
at the distance r from the center; R 1 and R 2 are the radii of the two spheres
A Sturm–Liouville eigenproblem Euler buckling of column y ′′ + λy = 0, y ′ (0) = 0, y(1) = 0
Find eigenvalues λ and eigenfunctions y
Some application areas in ODEs
Initial value problems
• mechanics M ¨ q = F (q)
• electrical circuits C ˙v = −I(v)
• chemical reactions ˙c = f (c) Boundary value problems
• materials u ′′ = M (u)/EI
• microphysics − 2m ~ ψ ′′ = E
Partial differential equations. Examples
The Poisson equation u xx + u yy = f (x, y) subject to u(x, y) = g(x, y) for x = 0 ∨ x = 1 ∨ y = 0 ∨ y = 1
is an elliptic PDE modelling e.g. the displacement u(x, y) of
an elastic membrane under load f
Partial differential equations. Examples
The Poisson equation u xx + u yy = f (x, y) The Diffusion equation u t = u xx subject to
u(x, 0) = f (x), x ∈ [0, a]
u(0, t) = c 1 , u(a, t) = c 2 , t ∈ [0, b]
is a parabolic PDE modelling e.g. the time evolution of temperature u(x, t) in an insulated rod with constant
temperatures c 1 and c 2 at its ends, and initial temperature
Partial differential equations. Examples
The Poisson equation u xx + u yy = f (x, y) The Diffusion equation u t = u xx
The Wave equation u tt = u xx subject to
u(0, t) = 0, u(a, t) = 0 for t ∈ [0, b]
u(x, 0) = f (x) for x ∈ [0, a]
u t (x, 0) = g(x) for x ∈ (0, a)
is a hyperbolic PDE e.g. modelling the displacement u(x, t)
of a vibrating elastic string fixed at x = 0 and x = a
2. What is Numerical Analysis?
Categories of mathematical problems
Category Algebra Analysis
linear computable not computable
nonlinear not computable not computable
2. What is Numerical Analysis?
Categories of mathematical problems
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
Algebra Only finite constructions
2. What is Numerical Analysis?
Categories of mathematical problems
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
Algebra Only finite constructions
Analysis Limits, derivatives, integrals etc. (transfinite)
2. What is Numerical Analysis?
Categories of mathematical problems
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
Algebra Only finite constructions
Analysis Limits, derivatives, integrals etc. (transfinite)
Computable The exact solution can be obtained in a finite
number of operations
What is Numerical Analysis?
Category Algebra Analysis
linear computable not computable nonlinear not computable not computable
With numerical methods, problems from all four categories can be solved:
“Numerical analysis aims to construct and analyze
quantitative methods for the automatic computation of
3. The four principles of numerical analysis
There are four basic principles in numerical analysis. Every computational method is constructed from these principles
◮ Discretization
To replace a continuous problem by a discrete one
◮ Linear algebra tools, including polynomials All techniques that are computable
◮ Iteration
To apply a computation repeatedly (convergence)
◮ Linearization
To approximate a nonlinear problem by a linear one
Discretization. From function to vector
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.5 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.5 1
0 0.5 1
Discretization. . .
Reduces the amount of information in a problem to a finite set Select a subset Ω N = {x 0 , x 1 , . . . , x N } of distinct points from the interval Ω = [0, 1]
Continuous function f (x) is replaced by vector F = {f (x k )} N 0 The subset Ω N is called the grid
The vector F is called a grid function or discrete function
Discretization yields a computable problem. Approximations
are computed only at a finite set of grid points
Discretization. Approximation of derivatives
Definition of derivative
f ′ (x) = lim
∆x→0
f (x + ∆x) − f (x)
∆x
Take ∆x = x k+1 − x k finite and approximate f ′ (x k ) ≈ f (x k + ∆x) − f (x k )
∆x = F k+1 − F k x k+1 − x k
How accurate is this approximation?
Error and accuracy
Expand in Taylor series f (x k + ∆x) − f (x k )
∆x ≈ f (x k ) + ∆xf ′ (x k ) + O(∆x 2 ) − f (x k )
∆x
= f ′ (x k ) + O(∆x) The error is proportional to ∆x
Try this formula for f (x) = e x at x k = 1 for various ∆x
Numerical differentiation
Error in numerically calculated derivative of e x at x = 1
10−10 10−8 10−6 10−4 10−2 100
Error in numerical differentiation
log error
Numerical differentiation
Error in numerically calculated derivative of e x at x = 1
10−15 10−10 10−5 100
10−12 10−10 10−8 10−6 10−4 10−2 100
Error in numerical differentiation
log step size
log error
Log error vs. log step size log ∆x
Another approach
Use a symmetric approximation f (x k + ∆x) − f (x k − ∆x)
2∆x = F k+1 − F k−1 x k+1 − x k−1
= f ′ (x k ) + O(∆x 2 )
The approximation is 2nd order accurate because error is ∆x 2
Whenever you have a power law, plot in log–log diagram
r ≈ C · ∆x p ⇒ log r ≈ log C + p log ∆x
Numerical 2nd order differentiation
Error in numerically calculated derivative of e x at x = 1
10−15 10−10 10−5 100
10−12 10−10 10−8 10−6 10−4 10−2 100
Error in numerical differentiation
log step size
log error
Log error vs. log step size log ∆x
Numerical 2nd order differentiation
Error in numerically calculated derivative of e x at x = 1
10−10 10−8 10−6 10−4 10−2 100
Error in numerical differentiation
log error
1st and 2nd order comparison
Error in numerically calculated derivative of e x at x = 1
10−15 10−10 10−5 100
10−12 10−10 10−8 10−6 10−4 10−2 100
Error in numerical differentiation
log step size
log error
