Brief Notes on Numerical Methods for Ordinary
Differential Equations
The construction of efficient, stable and high order methods for solving initial value problems governed by systems of ordinary differential equations (ODEs) is, in many respects, a mature subject [1, 2, 3, 4, 5, 7]. To fix notation, we assume that the initial value problem to be solved is in the standard form
y0(t) = F (t, y(t)) t∈ [a, b] (1)
y(a) = ya ,
(2)
where ya, y(t) are (possibly complex) n-vectors. As mentioned in class, requiring
that F be continuously differentiable with respect to its arguments is sufficient to guarantee local existence and uniqueness of the problem (1), (2). Since we are interested in high-order methods, however, we suppose throughout that F is sufficiently smooth. Unless otherwise stated, we assume that the dimension of the system n = 1, since it makes much of the discussion less cumbersome.
Suppose now that we are solving the problem (1), (2) numerically on the interval [a, b], obtaining an approximate solutiony(b). The two critical charac-e teristics of such a scheme with which we are concerned are its order of accuracy and its (stiff) stability. A numerical method is said to be of order of accuracy or order k if, for any sufficiently smooth F , there exists a real constant K≥ 0 such that
key(b)− y(b)k < K · (b − a)k+1. (3)
The suitability of a numerical method for stiff problems is generally analyzed by applying it to the equation
y0(t) = λ· y(t) t∈ [0, 1] y(0) = 1 ,
(4)
The amplification factor, Am(λ), for λ∈ C is defined by the formula Am(λ) =y(1).e
(5)
If, for a given value of λ,
| Am(λ) |≤ 1, (6)
then the numerical method is said to be stable for that value of λ. If a method is stable for all λ in the left-half plane (Re(λ)≤ 0), then the method is said to be A-stable. A method is said to be L-stable if
lim
Re(λ)→−∞Am(λ) = 0.
1
Euler’s method
We have discussed the explicit Euler (or forward Euler) method for the solution of the ODE (1):
yi+1 = yi+ hi· F (ti, yi)
(8)
hi= ti+1− ti,
(9)
for i = 0, 1,· · · , m. The implicit (or backward) Euler scheme for the solution of (1) is given by the formula
yi+1= yi+ hi· F (ti+1, yi+1).
(10)
Note that the step-size hiis variable and at our disposal to change from step to
step.
ODE codes require an input , and try to choose an appropriate sequence of steps hi. From Taylor’s theorem, we have
y(ti+1) = y(ti) + hi· F (ti, y(ti)) +
1 2h
2 iy00(ξi)
(11)
where ti≤ ξi≤ ti+1. Subtracting (8) from (11), we have
y(ti+1)− yi+1 = y(ti)− yi+ hi· [F (ti, y(ti))− F (ti, yi)]
+1 2h
2 iy00(ξi)
(12)
The left-hand side is the global error at ti+1. The right-hand side is composed
of three terms: the global error at ti, the propagation error hi· [F (ti, y(ti))−
F (ti, yi)] and the local error 12h2iy00(ξi). The local error is the error made in going
from ti to ti+1 with exact data. The propagation error (using calculus) can be
written as hi·∂F∂y(η)[y(ti)− yi] or (in higher dimensions) as hi· J(η)[y(ti)− yi],
where J denotes the Jacobian. Putting this together with (12) yields
global error(i + 1) = (1 + hiJ )[global error(i)] + local error(i + 1).
(13)
The term (1 + hiJ ) is called the amplification factor (as above). If the factor is
less than one in norm, errors don’t grow. Otherwise, they do and the method is unstable.
Assuming (in the scalar case) that J < 0, this requires that −2 < hiJ < 0.
This interval is called the stability interval for the method, If J > 0, the ODE itself is unstable and hiJ < 0 is never satisfied.
2
Practical issues
Most nonstiff ODE codes use estimates of y00rather than J to pick a time step. This is a form of “local error control.” Consider, however, the following example:
y0=−α(y − sin t) + cos t with initial data y(0) = 1. The exact solution is
y(t) = e−αt+ sin t. The first term is transient. Thus, for large time t,
y00≈ sin t ≤ 1.
Assuming you want three-digit accuracy, a reasonable local time step would appear to be hi≈ 10−3.
We know, from the preceding analysis, however, that the code will blow up. Why?
A different strategy would try to keep hi ≈ 1/|J| to ensure stability. For
the present example, this would keep the step small, even though the solution is smooth.
A possible approach is to estimate y00based on computed values yi0and y0i−1:
yi,i00−1≈ y
0 i− y0i−1
ti− ti−1
. If one is following the solution correctly, then
y0i ≈ F (ti, yi)
y0i−1 ≈ F (ti−1, yi−1)
and the estimate for y00will be small. In reality, the data (ti, yi). and (ti−1, yi−1)
lie on slightly different “solution curves”. Stiffness means that such small changes in the solution curve lead to large changes in y0. SO,
A Strategy
1. Choose hi so that |y00i,i−1h2i/2| < . This tries to enforce the constraint
that the local error be less than . 2. Compute yi+1 via EUler’s method.
3. Compute yi+10 . If it looks crazy, the step must be too large. Estimate yi+100 as yi+1,i00 ≈ y 0 i+1− y0i ti+1− ti .
4. If the step is determined to be too large in step 3, refine the step by a factor of 2 and repeat.
This is certainly a heuristic approach, but it usually works and forces Euler to take small steps for stiff systems. Note that it is purely “data-driven.” It relies on the computed solution alone, not on a study of the function F (t, y).
For systems, note that the condition|1+hJ| < 1 is replaced by k1+hJk < 1.
3
Implicit methods
Many ODE methods can be derived from the exact relation y(ti+1) = y(ti) +
Z ti+1
ti
F (t, y(t)) dt or
y(ti+k) = y(ti) +
Z ti+k
ti
F (t, y(t)) dt by replacing the integral with a quadrature rule.
Problem: What quadrature rules do the forward, Euler, backward Euler and trapezoidal rules correspond to?
Problem: Show that the error analysis for backward Euler yields y(ti+1)− yi+1 =
1
1− hJ(y(ti)− yi) + O(h
2)
and that the error analysis for the trapezoidal rule yields y(ti+1)− yi+1=
1 + hJ/2
1− hJ/2(y(ti)− yi) + O(h
3).
As long as J < 0, both are stable (the amplification factor is less than one). An obvious difficulty with implicit methods is the need to solve a (linear or nonlinear) set of algebraic equations at each time step. Backward Euler, for example, requires finding yi+1 from the equation
yi+1 = yi+ hiF (ti+1, yi+1).
For nonstiff problems, this can be solved by fixed point iteration. For stiff problems, fixed point iteration diverges and you need to use Newton’s method or some variant thereof.
References
[1] K.E. Brenan, S.L. Campbell, and L.R. Petzold, Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations, SIAM, Philadel-phia, 1995.
[2] J. Butcher, The Numerical Analysis of Ordinary Differential Equations: Runge-Kutta and General Linear Methods, Wiley, 1987.
[3] C. W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations, Prentice-Hall, New Jersey, 1971.
[4] E. Hairer, S. P. Norsett and G. Wanner, Solving Ordinary Differential Equa-tions I, Non-Stiff Problems, Springer Verlag, Berlin, 1987.
[5] E. Hairer, G. Wanner, Solving Ordinary Differential Equations II, Springer, 1996.
[6] Arieh Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, 1996.
[7] J. D. Lambert, Numerical Methods for Ordinary Differential Equations, Wiley, 1991.
