Math 128a: Runge-Kutta Methods
GSI: Andrew Shi
Contents
1 Runge Kutta Methods 2
1.1 Setup. . . 2
1.2 Examples of Runge-Kutta Methods . . . 3
1.2.1 Explicit Euler and Implicit Euler . . . 3
1.2.2 Midpoint Methods . . . 5
1
Runge Kutta Methods
1.1
Setup
Our motivation in coming up with Runge-Kutta methods is that we want to be able to construct numerical methods that have the arbitrarily high-order of Local Truncation Error (LTE) we enjoy from Taylor Series Methods without having to compute and evaluate deriva-tives of f . We only want to do direct function evaluations of f .
A general s-stage Runge-Kutta method has the form:
k1 = f (tn+ c1h, wn+ h n X i=1 a1iki) k2 = f (tn+ c2h, wn+ h n X i=1 a2iki) · · · ks= f (tn+ csh, wn+ h n X i=1 asiki) wn+1= wn+ h s X i=1 biki
This looks rather intimidating. It’s worth noting first that what defines the method is the choice of the coefficients aij, bi and ci. This is usually defined in what’s called a Butcher
Table. c1 a11 a12 a13 a1s c2 a21 a22 a23 a2s c3 a31 a32 a33 a3s .. . ... . .. cs as1 as2 as3 ass b1 b2 b3 · · · bs
This is a much more convenient way to represent the Runge Kutta method instead of writing out all the individual k0s.
Each of these ki’s are what’s known as a stage derivative. We call it a derivative because
they are evaluations of f at different times and positions, and f is secretly a derivative by the ODE y0 = f . The idea is that we want to take some linear combination of these f ’s that are evaluated at these various times and positions that will give us the right update to march from wn to wn+1. Any (explicit) one step method has the form
where for a Runge Kutta method, φ(tn, wn) = Psi=1biki. The intuition is that we want
φ(tn, wn) to capture the right “slope” between wn and wn+1 so when we multiply it by h, it
provides the right update wn+1 − wn. This is still rather ambiguous at this point, so let’s
start from first principles and discuss the simplest Runge Kutta methods and see how they fit in this framework.
1.2
Examples of Runge-Kutta Methods
1.2.1 Explicit Euler and Implicit Euler
Recall Euler’s method: wn+1 = wn+ hf (tn, wn). The idea we discussed previously with the
direction fields in understanding Euler’s method was that we just take f (tn, wn) – the slope
at the left endpoint – and march forward using that. So rewriting this as a Runge-Kutta method:
k1 = f (tn, wn)
wn+1= wn+ hk1
So Euler’s method is a one stage Runge-Kutta method with a11 = 0, b1 = 1, c1 = 0. Here is
it’s Butcher Table:
0 0 1
Now I ask myself – over the interval [tn, tn+1], if we are going to pick just one point on the
interval to proxy the slope over the entire interval, why the left endpoint? Shouldn’t the one on the right be equally good (or bad) in the sense that it is equally representative of what goes on over the whole interval?
This leads us to Implicit Euler’s method. To clarify, the usual Euler’s method goes by the name Explicit Euler (or Forward Euler). Here we introduce Implicit Euler (or Backward Euler).
k1 = f (tn+1, wn+1)
wn+1 = wn+ hk1
But this is not quite in the form of a Runge Kutta method, because the second argument of the f evaluation in k1 needs to be expressed as wn+Pni=1a1iki) for some coefficients a1i. So
we rather cleverly substitute the equation for the solution update in the second argument and write tn+1 = tn+ h to get:
k1 = f (tn+ h, wn+ hk1)
wn+1 = wn+ hk1
1 1 1 Here it is written as a one liner:
wn+1= wn+ hf (tn+1, wn+1)
Intuitively we consider Explicit Euler and Implicit Euler as “equally good” since the left or right endpoint should give equally good information about the entire interval. We will quantify this notion later when we see they have the same LTE.
But there’s a good reason not to use this method – given the information (tn, wn) at time
n, the next step wn+1 is only implicitly defined by the method! In order to march forward
and get a value for wn+1, we must solve a nonlinear equation using something like Newton’s
method.
Example 1 Consider the IVP:
y0 = t + sin(y) y(0) = 1 Estimate y(0.1) with h = 0.1 using Implicit Euler.
Solution: One step of Implicit Euler gives us: w1 = w0+ hf ( t1
|{z}
0.1
, w1)
= 1 + (0.1)(0.1 + sin(w1))
So we need to set up a Newton iteration and iterate to convergence to get w1. Define F (x)
to be:
F (x) = x − 1 − (0.1)(0.1 + sin(x)) = x − 0.1 sin(x) − 1.01 F0(x) = 1 − 0.1 cos(x)
So the Newton Iteration to solve F (x) = 0 gives us xn+1= xn−
xn− 0.1 sin(xn) − 1.01
1 − 0.1 cos(xn)
And we can iterate to convergence to get w1 = 1.09908.
As of right now, it seems like there is no good reason to ever use an Implicit Method if we have an Explicit Method that is equally “as good” because of the additional cost of solving a nonlinear equation. We will see next week (Section 5.10) when we discuss Stiffness why for some ODEs, we practically have to use Implicit Methods.
1.2.2 Midpoint Methods
So let’s get back on track – we’ve already discussed Explicit Euler and Implicit Euler, which came from using the left and the right endpoints, respectively. Continuing down this same train of thought we could also use the midpoint to get the so called “Midpoint Method”.
k1 = f tn+1 2, wn+ 1 2 wn+1 = wn+ hk1
But we can’t actually use this method as given. We only have w0 to start from the initial
condition. In order to get to w1, we need the value of w1/2!
One way to do this is obtain w1/2 by Explicit Euler with a stepsize of h/2 as wn+1/2 =
wn+h2f (tn, wn). Here it is written as a Runge Kutta Method:
k1 = f (tn, wn) k2 = f tn+ h 2, wn+ h 2k1 wn+1 = wn+ hk2 Here it is as a one-liner: wn+1= wn+ hf tn+ h 2, wn+ h 2f (tn, wn)
Here is its Butcher Table:
0 0 0 1/2 1/2 0 0 1
This is what’s called the Explicit Midpoint Method (or Midpoint Method with Euler Pre-dictor)
Here’s another idea – instead of obtaining wn+1/2by Euler’s method, why don’t I use Linear
Interpolation and let wn+1/2≈ 12(wn+ wn+1)?
This would give us the Implicit Midpoint Method: wn+1 = wn+ hf tn+ h 2, 1 2(wn+ wn+1)
Let’s write this as a Runge Kutta Method. This is quite challenging! This is a one-stage method, since there is only one f evaluation. Here is the general form for a one-stage method.
k1 = f (tn+ c1h, wn+ a11hk1)
It’s not hard to see that b1 = 1 and c1 = 12. The question is how do we get the second
argument of the f evaluation in the Implicit Midpoint Method 12(wn+ wn+1) to match the
second argument in the f evaluation for k1, (wn+ a11hk1)?
Solve for k1 in the equation that does the solution update:
wn+1 = wn+ hb1k1 =⇒ k1 =
wn+1− wn
h and plug this into the second argument for f for k1 to get
k1 = f tn+ 1 2h, wn+ a11hk1 = f tn+ 1 2h, w|n+ a11(w{zn+1− wn}) a11wn+1+(1−a11)wn
But now it’s quite clear that a11 = 1/2, so the Implicit Midpoint Method in Runge Kutta
Form is: k1 = f tn+ 1 2h, wn+ h 2k1 wn+1 = wn+ hk1
with Butcher Table.
1 2
1 2
1
So that takes care of the one-point rules (left endpoint, right endpoint, and two ways to estimate the midpoint). The natural thing to try next is to consider a two-point rule. The simplest one that comes to mind is taking the average of the left and right endpoint. This is called the Trapezoid Rule:
wn+1 = wn+
h
2(f (tn, wn) + f (tn+1, wn+1))
When we study multistep methods we will see how this is connected (or rather derived from) to the Trapezoid Rule for Integration. Let’s write this as a Runge Kutta Method. Since there are two f evaluations we are going to need two stages. Here’s my first attempt to do this: k1 = f (tn, wn) k2 = f (tn+1, wn+1) wn+1 = wn+ h 2(k1+ k2)
But this is not quite in Runge Kutta form because of the wn+1. So let’s plug in the equation
k1 = f (tn, wn) k2 = f tn+ h, wn+ h 2(k1+ k2) wn+1 = wn+ h 2(k1+ k2) Beautiful. Here’s the Butcher Table:
0 0 0 1 1/2 1/2 1 2 1 2
What do you think comes next now? We have discussed Implicit and Explicit Versions of Euler’s Method and the Midpoint method, and we just did the Implicit Trapezoid Rule. Now for the Explicit Trapezoid Rule. We can accomplish this by predicting wn+1 on the RHS by
Euler’s Method:
wn+1 = wn+
1
2h(f (tn, wn) + f (tn+1, wn+ hf (tn, wn)))
In the textbook and slides, this is also known as Modified Euler’s Method. Let’s write this as a Runge Kutta Method:
k1 = f (tn, wn)
k2 = f (tn+ h, wn+ hk1)
wn+1= wn+
h
2(k1 + k2) with Butcher Table.
0 0 0 1 1 0 1/2 1/2
Up until this point we have done a fairly through discussion of the simplest Runge-Kutta methods and derived them all with fairly basic geometric principles.
1.2.3 Local Truncation Error
Now the question is how good are these methods, which we for now quantify by the Local Truncation Error. Recall the LTE is the one-step error in the method assuming you have an exact solution up until that point. So for a one step method of the form:
It’s LTE is defined by:
yn+1 = yn+ hφ(tn, yn) + hτn+1 =⇒ τn+1 =
yn+1− yn
h − φ(tn, yn)
Let’s compute the Local Truncation Errors for each of these methods we’ve discussed using this definition.
(Explicit) Euler’s Method: wn+1 = wn+ hf (tn, wn)
τn+1 = yn+1− yn h − f (tn, yn) =yn+ hy 0 n+ h2 2!y 00 n+ h3 3!y 000(ξ) − yn h − f (tn, yn) ξ ∈ (tn, tn+1) = yn0 +h 2y 00 n+ h2 3!y 000 (ξ) − f (tn, yn) = O(h)
(Implicit) Euler’s Method: wn+1= wn+ hf (tn+1, wn+1)
τn+1= yn+1− yn h − f (tn+1, yn+1) =yn+ hy 0 n+ h2 2!y 00 n+ h3 3!y 000(ξ) − yn h − f (t| n+1{z, yn+1}) y0 n+1 ξ ∈ (tn, tn+1) = y0n+h 2y 00 n+ h2 3!y 000(ξ 1) − (y0n+ hy 00 n+ h2 2!y (3)(ξ 2)) = O(h)
This quantifies our intuition earlier that Implicit and Explicit Euler should be equally good (or bad).
For the remaining methods we need to be able to do Taylor Expansions in 2-variables. Recall the form for 1D. The Taylor Expansion of f (x) around a point x0 looks like:
f (x) = f (x0) + f0(x0)(x − x0) + f00(x0) 2! (x − x0) 2+ f (3)(x 0) 3! (x − x0) 3+ . . .
We can also write this as:
f (x + h) = f (x) + hf0(x) + h 2 2!f 00(x) + h3 3!f (3)(x) + . . .
For a more thorough explanation why these two are equivalent, see my notes on differentia-tion (the day we missed because of PGE outage).
For 2D, the Taylor Expansion of f (x, y) around a point (x0, y0):
f (x, y) = f (x0, y0) + ∂f ∂x(x0, y0)(x − x0) + ∂f ∂y(x0, y0)(y − y0) + 1 2 ∂2f ∂x2(x0, y0)(x − x0) 2 + ∂ 2f ∂x∂y(x0, y0)(x − x0)(y − y0) + 1 2 ∂2f ∂y2(x0, y0)(y − y0) 2 + . . .
Similarly, this can be written as: f (x + ∆x, y + ∆y) = f (x, y) + ∂f ∂x(x, y)∆x + ∂f ∂y(x, y)∆y + 1 2 ∂2f ∂x2(x, y)(∆x) 2 + ∂ 2f ∂x∂y(x, y)(∆x)(∆y) + 1 2 ∂2f ∂y2(x, y)(∆y) 2+ . . .
So now let’s use this to calculate the LTE of the Midpoint Method(s). (Explicit) Midpoint Method: wn+1 = wn+ hf tn+h2, wn+h2f (tn, wn)
τn+1 = yn+1− yn h − f tn+ h 2, yn+ h 2f (tn, yn) = yn0 + h 2!y 00 n+ h2 3!y (3) n + O(h 3 ) − f (tn, yn) + ∂f ∂t(tn, yn) h 2 +∂f ∂y(tn, yn) h 2 f (tn, yn) + ∂2f ∂t2(tn, yn) h 2 2 + ∂ 2f ∂t∂y(tn, yn) h 2 2 f (tn, yn) +∂ 2f ∂y2(tn, yn) h 2 2 (f (tn, yn))2+ O(h3)
Let me just stop here to marvel just how ugly this is. Let’s start chipping away at it. We are going to need to translate y and its derivatives in terms of f (inside the square brackets) to start canceling terms out. Recall from our study of Taylor Series Methods we have that:
y0 = f (t, y)
y00= ft(t, y) + fy(t, y) f (t, y)
| {z }
y0
y(3) = ftt+ 2ftyf + fyyf2+ fyft+ fy2f
So from the first derivative we can see that the yn0 and the f (tn, yn) cancel out in the LTE:
τn+1 = yn0 + h 2!y 00 n+ h2 3!y (3) n + O(h 3) − f (tn, yn) + ∂f ∂t(tn, yn) h 2 +∂f ∂y(tn, yn) h 2 f (tn, yn) + ∂2f ∂t2(tn, yn) h 2 2 + ∂ 2f ∂t∂y(tn, yn) h 2 2 f (tn, yn) +∂ 2f ∂y2(tn, yn) h 2 2 (f (tn, yn))2+ O(h3)
τn+1 = h 2!y 00 n+ h2 3!y (3) n + O(h 3) − (((( (((( (((( (((( (((( ((( ∂f ∂t(tn, yn) h 2 +∂f ∂y(tn, yn) h 2 f (tn, yn) +∂ 2f ∂t2(tn, yn) h 2 2 + ∂ 2f ∂t∂y(tn, yn) h 2 2 f (tn, yn) +∂ 2f ∂y2(tn, yn) h 2 2 (f (tn, yn))2+ O(h3)
But it doesn’t take much effort to see our luck runs out when trying to cancel out the third derivative terms, so this means our LTE is O(h2). So comparing to the other one
point methods, we see that this is actually better. This is consistent with when we studied Newton-Cotes integration rules, that the midpoint method was O(h2) and the left and right endpoint methods were only O(h).
(Implicit) Trapezoid Rule: wn+1= wn+h2[f (tn, wn) + f (tn+1, wn+1)]
τn+1 = yn+1− yn h − 1 2 f (tn, yn) | {z } y0 n + f (tn+1, yn+1) | {z } yn+10 = yn0 +h 2y 00 n+ h2 3!y (3) n + h3 4!y (4) (ξ1) − 1 2 y0n+ yn0 + hyn00+ h 2 2!y (3) n + h3 3!y (3) (ξ2) = O(h2)
(Implicit) Midpoint Method: wn+1 = wn+ hf tn+h2,12(wn+ wn+1)
(Explicit) Midpoint Method: wn+1 = wn+ 12h(f (tn, wn) + f (tn+1, wn+ hf (tn, wn)))
Exercises for the reader :)
Let’s summarize again – we have derived the simplest Runge-Kutta methods from purely geometric principles and painfully computed their LTEs. But what’s the big picture here? The goal at the beginning was to get Runge-Kutta methods of arbitrarily high order, and it was already so painful to deal with these second order methods. Here is a 4th-order RK method: k1 = f (tn, wn) k2 = f (tn+ h 2, wn+ h 2k1) k3 = f (tn+ h 2, wn+ h 2k2) k4 = f (tn+1, wn+ hk3) wn+1 = wn+ h 6(k1+ 2k2+ 2k3+ k4)
In fact this is frequently known as THE 4th order RK method called RK4 (there are oth-ers, but almost never used). It’s pretty difficult to derive something like this from purely
geometric principles, so you can think of how we get RK methods as more algebraic. In principle you could do Taylor expansions to solve for coefficients in order to get as high order a method as possible, but practically speaking there are some advanced graph theory techniques people use to do this.
In the lecture notes, Prof. Gu derives the order conditions for a 2 stage explicit Runge Kutta method, and this is a more general calculation that encompasses all the ones we have done up until now (which you should review and compare to the painful 2D Taylor calculation we did earlier).
It is also worth making sure we know how to actually use an RK method to step forward: Example 2 Consider the IVP:
y0 = t + y y(0) = 1 Estimate y(0.1) with h = 0.1 using RK4.
Solution: k1 = f (tn, wn) = f (0, 1) = 1 k2 = f tn+ h 2, wn+ h 2k1 = f (0.05, 1.05) = 1.1 k3 = f tn+ h 2, wn+ h 2k2 = f (0.05, 1.055) = 1.105 k4 = f (tn+1, wn+ hk3) = f (0.1, 1.05525) = 1.15525 wn+1= wn+ h 6(k1+ 2k2+ 2k3 + k4) = 1 + (0.1) 6 (1 + 2(1.1) + 2(1.105) + 1.11525) = 1.16400917 Now I offer some final comments and relate what I’ve written here in the notes to the lecture slides (which you should of course study after this).
Stage Derivatives vs. Stage Updates If you look at how the textbook writes RK4, it’s like this: k1 = hf (tn, wn) k2 = hf (tn+ h 2, wn+ 1 2k1) k3 = hf (tn+ h 2, wn+ 1 2k2) k4 = hf (tn+1, wn+ k3) wn+1 = wn+ 1 6(k1+ 2k2+ 2k3+ k4)
The difference is that the ki’s already have the h’s multiplying them in their definition. In
this case, these are called stage updates, because now you can take a direct linear combina-tion of them to move from wn to wn+1 (the k’s have the same units as the w’s now).
Textbook Notation for the 2 stage Explicit Case
In class, Prof. Gu really only describes the case of the 2-stage Explicit method written as follows:
wj+1 = wj + h(a1f (tj, wj) + a2f (tj+ α2, wj + δ2f (tj, wj)))
But written in the more general notation we’ve discussed, this is the same thing as: k1 = hf (tn, wn)
k2 = hf (tn+ c2h, wn+ a21k1)
wn+1= wn+ a1k1+ a2k2
So apologies for any slightly conflicting notation, but make sure you understand the differ-ence (and why they’re really the same).
Explicit vs. Implicit Methods
For a method to be explicit, this requires c1 = 0 and the A matrix in the Butcher table
to be strictly lower triangular (0 on the diagonal and above). We see this means that each of the ki’s can be calculated in terms of all the previous ones, hence there is no need to
solve any nonlinear equations and we can march forward explicitly. In practice we almost al-ways use Implicit methods, but as of this point we have yet to explain why they are useful...
