In this section we present our Lyapunov function based step size control algorithm. We start by stating and analyzing an “abstract” version of this algorithm and then describe the details of our implementation and illustrate it by means of two numerical examples.
3.1 An abstract step size control algorithm
The following algorithm provides an abstract step size selection method based on a Runge-Kutta scheme for (1), expressed via (7) as a hybrid system (6), a Lyapunov
function V for (1) according to Definition 5 and its derivative∇V. Moreover, we
assume that we have defined two functions
hreduce∶ R+×Rn→ R+ and hnew∶ R+×Rn→ R+,
which for a given h> 0 and x ∈ Rnproduce a reduced step size hreduce(h,x) < h and a
new step size hnew(h,x) > 0. In order to simplify the presentation, the algorithm uses
a maximal step size hmax> 0 which does not depend on the state x, but the inclusion
of a state dependent upper bound is straightforward.
Algorithm 3: Lyapunov function based step size control algorithm
1: inputs
Initial value x0∈ Rn, initial step size h0> 0, maximal step size hmax> 0,
parameter λ∈ (0,1)
2: set x(0) ∶= x0, τ0∶= 0, i ∶= 0
3: set hi∶= min{hi,hmax} and compute x(τi+hi) ∶= Φ(x(τi),hi) 4: while V(x(τi+hi))−V(x(τi)) > λh∇V(x(τi)) f (x(τi)) do
5: set hi∶= hreduce(hi,x(τi)) and recompute x(τi+hi) ∶= Φ(x(τi),hi)
6: end while
7: set hi+1∶= hnew(hi,x(τi)), τi+1∶= τi+hi, i∶= i+1 and goto step 2
Note that this algorithm does not have a termination criterion and hence — in principle — produces infinitely many values x(τi). Of course, if only a solution on some interval
Since the above algorithm only produces one single sequence of step sizes hifor
each initial value x0, it does not make sense to talk about robust stability concepts
anymore. Moreover, since the Lyapunov function condition in step (2) is only ensured at the discrete times τi, we cannot in general expect stability properties for all t≥ 0
(although under suitable conditions they could be recovered by continuity arguments, see, e.g., [24]). However, under appropriate assumptions on hreduceand hnewwe can
still recover the required properties from Definition 3 at the discrete time instants τi,
as the following theorem states.
Theorem 7. Consider Algorithm 3 in which Φ is a consistent Runge-Kutta scheme with order p≥ 1. Let V ∈ C1(Rn;R+) be a Lyapunov function for system (1) and λ∈ (0,1). Assume that hmax> 0 is chosen such that Φ(x,h) is defined for all x ∈ Rn
and h∈ (0,hmax] and that the functions hreduceand hnewsatisfy
• there exist real values0< ρ1≤ ρ2< 1 such that hreduce(x,h) ∈ [ρ1h,ρ2h] holds
for all x∈ Rnand all h> 0 for which the condition in Step (2) of Algorithm 3 is satisfied
• hnew(x,h) ≥ h holds for all x ∈ Rnand all h> 0 for which the condition in Step
(2) of Algorithm3 is not satisfied.
Then, for each initial value Algorithm3 generates an infinite sequence of time steps hi(i.e., the while loop in steps (2)–(4) always terminates) and at the times τi the
resulting trajectories are
(i)Lagrange stable, i.e., for each R> 0 it holds that sup{∥x(τi,x0)∥∣i ∈ N, ∥x0∥ ≤ R}
< ∞.
(ii)Lyapunov stable, i.e., for each ε> 0 there exists δ > 0 such that ∥x(τi,x0)∥ ≤ ε for
all i∈ N and all x0∈ Rnwith∥x0∥ ≤ δ
(iii)uniformly attractive, i.e., for each R> 0 and ε > 0 there exists T > 0 such that ∥x(τi,x0)∥ ≤ ε for all τi≥ T and all x0∈ Rnwith∥x0∥ ≤ R.
If, in addition, there exists a continuous function ϕ∈ C0(Rn;(0,r]) for some r > 0 such that
V(Φ(x,h))−V(x) ≤ hλ∇V(x) f (x) (13)
holds for all x∈ Rnand all h∈ (0,ϕ(x)), then for each initial value x0∈ Rnthere
exists hmin> 0 such that hi≥ hminholds for all i∈ N. In particular, in this case the
sequence of times τigenerated by Algorithm3 tends to∞ as i → ∞.
Proof. Let(ak)k∈Nbe an arbitrary sequence with 0< ak+1< ak, limk→∞ak→ 0 and
pick ˜λ ∈ (λ,1) arbitrarily. Define the sets M ∶= {x ∈ Rn∣V(x) ≤ a1}, Mk∶= {x ∈
Rn∣V(x) ∈ [a
k+1,ak]} and let k ∈ N be arbitrary. Since V is continuous, positive
definite and radially unbounded, it follows from Lemma 3.5 in [19] that there exist functions α1,α2∈ K∞with
α1(∥x∥) ≤ V(x) ≤ α2(∥x∥) for all x ∈ Rn. (14)
This implies that the sets M and Mkare compact and there exists Rk> εk> 0 such that
Festschrift in Honor of Uwe Helmke L. Grüne et al. there exists hεk,Rk∈ (0,hmax] such that the condition (11) holds for ˜λ in place of λ
for all x∈ Mkand all h∈ (0,hεk,Rk].
Since the scheme is consistent and V is Lipschitz, this implies
V(Φ(h,x))−V(x) ≤ V(z(h,x))−V(x)+LChp+1≤ h˜λ∇V(x) f (x)+LChp+1
for all x∈ Mkand all h∈ (0,min{¯h,hεk,Rk}], where ¯h is the step size from the consis-
tency condition for the compact set M and L is the Lipschitz constant of V on the set
K= {Φ(x,h)∣x ∈ M, h ∈ [0, ¯h]}∪{z(h,x)∣x ∈ M, h ∈ [0, ¯h]}, which is compact since
M×[0,h] is compact and both z(⋅,⋅) and Φ(⋅,⋅) are continuous. Setting
γk∶= max x∈Mk∇V(x) f (x) < 0 and h ′ k∶= (( λ− ˜λ)γ LC ) 1 p > 0, for ¯hk∶= min{¯h,hε,R,h′} we thus obtain
V(Φ(x,h))−V(x) ≤ hλ∇V(x) f (x) ≤ hλγk (15)
for all x∈ Mkand all h∈ (0, ¯hk].
Now consider the while loop in the steps (2)–(4) of Algorithm 3 for some x(τi) ∈ Mk
and denote by hi,old the time step for which step (1) is executed before entering
this loop. Inequality (15) and the inequalities for hreducethen implies that for any
x(τi) ∈ Mk the loop terminates with a time step hi≥ min{hi,old,ρ1¯hk} for which
V(Φ(x(τi),hi)) ≤ V(x(τi)) ≤ hiλ γkholds. Moreover, since hnew(hi,x(τi)) ≥ hi, the
subsequent time steps hj, j≥ i+1, will satisfy the same lower bound as hias long
as xj∈ Mkholds. Hence, as long as xj∈ Mkholds, V(x(τi)) decreases monotonically
with a fixed amount of decay and a uniform positive lower bound on the time step. Thus, by definition of the sets Mk, for each k∈ N there exists a time tk> 0 such that
whenever x(τi) ∈ Mk there exists a τj≤ τi+tksuch that either x(τj) = 0 (and thus
x(τm) = 0 for all τm≥ τj) or x(τj) ∈ Ml holds for some l≥ k +1. By induction, for
each k∈ N one thus obtains a time Tk> 0 such that for each x0∈ M there exists some
ik∈ N such that τik≤ Tkand either x(τi) ∈ Mkor x(τi) = 0 holds for all i ≥ ik.
The three stability properties (i)–(iii) are now readily concluded from this property: (i) Given R> 0 we choose a1= α2(R) which implies that each x0∈ Rnwith∥x0∥ ≤
R lies in M. Then the whole solution x(τi) lies in M which implies ∥x(τi)∥ ≤
α1−1(α2(R)) which implies Lagrange stability.
(ii) Given ε> 0 we choose δ = α2−1(α1(ε)). Then the inequality needed for Lyapunov
stability follows as in (i) with δ in place of R.
(iii) Given R and ε, choose a1as in (i) and k so large that ak≤ α1(ε) holds. Then, for
T= Tkand all τi≥ T, the solution x(τi) is either equal to 0 or lies in Mlfor some l≥ k.
This implies∥x(τi)∥ ≤ α1−1(V(x(τi))) ≤ α1−1(ak) = ε for all τi≥ T.
We finish the proof by proving the additional property of the hi in case that ϕ∈
C0(Rn;(0,r]) exists such that (13) holds. To this end, pick an arbitrary x0∈ Rnand
choose a1so large that x0∈ M holds. Then, (13) implies that the values ¯hkdefined in
the proof, above, can be bounded from below by h∗∶= minx∈Mϕ(x) > 0. By induction, the inequality hi≥ min{hi,old,ρ1¯hk} then implies hi≥ min{h0,ρ1h∗} > 0 for all i ∈ N
We note that various sufficient conditions ensuring the existence of ϕ∈C0(Rn;(0,r]) with (13) can be found in [18, Section 4]. We emphasize, however, that even without this condition the stability properties defined in Theorem 7 and in particular the convergence x(τi) → 0 is ensured. In particular, the numerical solution will converge
to the origin even if Problem (P1) does not have a solution.
3.2 Implementation of the algorithm
There are various ways of defining hreduce and hnew such that the conditions of
Theorem 7 are satisfied. A simple way, proposed in [18] is to define hreduce(h,x) ∶= h/2 and hnew(h,x) ∶= 2h.
This choice produces reasonable results (cf. [18]) but due to the “try and error” nature of the resulting algorithm it has the disadvantage that typically the while loop is executed at least once for each i, implying that Φ is usually evaluated at least twice for each i.
A more efficient way of defining hreduceand hnewis obtained by using ideas from the
classical error estimation based step size control, cf. e.g. [14, Section II.4]. To this end, define the Lyapunov differences
∆V(x,h) ∶= V(z(h,x))−V(x) and ̃∆V(x,h) ∶= V(Φ(x,h))−V(x)
for the exact solution and the numerical approximation. If V and f are sufficiently
smooth, then for a p-th order scheme there exists a constant c∈ R such that the
approximate equality
̃
∆V(x,h) ≈ ∆V(x,h)+chp+1
holds for all sufficiently small h> 0. Hence, we can approximately compute c as
c≈∆Ṽ(x,h)−∆V(x,h)
hp+1 .
We now intend to find a time step ˜h> 0 such that ̃
∆V(x, ˜h) ≤ λ ˜h∇V(x) f (x) holds, which will be approximately satisfied if the inequality
∆V(x, ˜h)+c˜hp+1≤ λ ˜h∇V(x) f (x) holds, i.e, if ˜h≤ (λ∇V(x) f (x)−∆V(x, ˜h)/˜h c ) 1 p
holds. Inserting the approximate value for c, we obtain the condition
˜h≤ h(λ̃∇V(x) f (x)−∆V(x, ˜h)/˜h
∆V(x,h)/h−∆V(x,h)/h )
1 p
Festschrift in Honor of Uwe Helmke L. Grüne et al. This suggests to use the expression on the right hand side of this inequality as a candidate for a new step size in our algorithm for h= hi. However, this expression is
implicit (as it contains the unknown ˜h) and contains the values ∆V(x,h) which are
not available in practice as they depend on the exact solution.
Both problems vanish if we replace the term ∆V(x,h) by its first order approximation h∇V(x) f (x) (both for h and ˜h) which leads to the expression
h( (λ−1)∇V(x) f (x) ̃ ∆V(x,h)/h−∇V(x) f (x)) 1 p .
Although the first order approximation ∆V(x,h) ≈ h∇V(x) f (x) introduces an error
of higher order than the error of the scheme, the resulting step size control mechanism shows very good results (cf. the discussion at the end of Example 8), probably due to the fact that the choice of λ< 1 introduces some tolerance against additional approximation errors.
For the practical implementation, we moreover need to take into account that the
denominator ̃∆V(x,h)/h−∇V(x) f (x) may become negative or 0 — this happens if
the discretization error speeds up the convergence to the origin instead of slowing
down the convergence. To this end, we replace the denominator by max{̃∆V(x,h)/h−
∇V(x) f (x),ε(λ −1)∇V(x) f (x)}, where ε > 0 is a small positive constant. Finally, in order to compensate for the various approximations during the derivation, we multiply our formula with a security factor ρ∈ (0,1). All in all, we end up with
hreduce(h,x) ∶= ρh( (λ −1)∇V(x) f (x)
max{̃∆V(x,h)/h−∇V(x) f (x),ε(λ −1)∇V(x) f (x)})
1 p
. (16) For hnewwe may use the same formula, i.e.,
hnew(h,x) ∶= hreduce(h,x), (17)
although this formula does not rigorously ensure the condition hnew(x,h) ≥ h imposed
in Theorem 7 (it would satisfy this condition if all approximate equations in the
derivation were exact). As an alternative, one might use the definition hnew(h,x)
∶= max{h,hreduce(h,x)}, however, the difference between these two choices turned
out to be marginal in our numerical simulations and since (17) yields a slightly lower number of evaluations of Φ we have used this variant in our simulations.
3.3 Examples
In our simulations we run Algorithm 3 with (16) for the Euler, the Heun and the classical Runge-Kutta scheme. All these schemes are explicit and thus satisfy ajl= 0
for all j,l= 1,...,s with l ≥ j. The Euler scheme is a scheme of order p = 1 with s = 1 stages and coefficient b1= 1, the Heun scheme is an s = 2 stage scheme of order p = 2
with
s= 2, a21= 1, b1= b2= 1/2
and the classical Runge-Kutta scheme (henceforth abbreviated as RK4) is of order p= 4 with s = 4 stages and
The standard parameters in all examples were h0= 0.1, hmax= 1, and ρ = 0.9 and
ε= 0.01 in Formula 16.
Example 8. The first example we investigate is the 2d differential equation ˙z1= −z1+z22, ˙z2= −z2−z1z2
with Lyapunov function V(x) = ∥x∥2, cf. [18, Example 4.15]. The Figures 1–3 show
simulation results (phase portrait, Lyapunov function V(x(τi)) over time and time
steps) on the time interval[0,20] with λ = 0.5 and x0= (5,5)⊺.
0 1 2 3 4 5 6 7 −1 0 1 2 3 4 5 x 1(t) x2 (t) Euler Heun RK4
Figure 1: Phase portrait for Example 8 with λ= 0.5 and x0= (5,5)⊺
All solutions approach the equilibrium x= 0 very quickly. The total number of steps
for the three schemes on the time interval[0,20] were 28 for the Euler scheme, 42
for the Heun scheme and 52 for the RK4 scheme. Here, two facts are worth noting. First, although the Euler scheme is the scheme with the lowest order, it allows for the largest steps and needs the fewest total number of steps. This is due to the fact that for asymptotic stability not only the size of the error matters but also the direction in which the error distorts the solution. Here, the error in the Euler scheme speeds up the convergence towards 0 and hence there is no reason for the scheme to reduce the time step. In contrast to this, while the errors for the Heun and the RK4 scheme are smaller, they have a tendency to slow down the convergence to 0 and hence the time steps have to be chosen more cautiously.
Second, it is clearly visible that our step size control Algorithm 3 does by no means ensure that the numerical solution is close to the exact solution during the transient phase, i.e., until a small neighborhood of the equilibrium is reached. In fact, the
Festschrift in Honor of Uwe Helmke L. Grüne et al. 0 2 4 6 8 10 12 14 16 18 20 10−15 10−10 10−5 100 t V(x(t)) Euler Heun RK4
Figure 2: Lyapunov function (logarithmic scale) for Ex. 8, λ= 0.5, x0= (5,5)⊺
0 2 4 6 8 10 12 14 16 18 20 10−2 10−1 100 t time step h Euler Heun RK4
three numerical solutions differ considerably and the Euler solution actually looks quite irregular. This is not a drawback of our algorithm but actually intended, since all the algorithm cares about is the convergence to x= 0 which is perfectly achieved for all schemes. If, in addition, a faithful reproduction of the exact solution during the transient phase is desired, our step size control algorithm could be coupled with traditional error estimation based techniques.
In order to illustrate the effects of different choices of λ∈ (0,1), Figure 4 shows the
time steps for the RK4 scheme for λ= 0.1,0.5,0.9.
0 2 4 6 8 10 12 14 16 18 20 10−1 100 t time step h λ=0.1 λ=0.5 λ=0.9
Figure 4: Time steps from Algorithm 3 (logarithmic scale) using the RK4 scheme applied to Example 8 with λ= 0.1,0.5,0.9 and x0= (5,5)⊺
As expected, the time steps become the smaller the closer the value λ is to 1, i.e., the more decay of the Lyapunov function is supposed to be guaranteed. The total
number of steps for the simulations on the time interval[0,20] was 28 for λ = 0.1,
52 for λ= 0.5 and 290 for λ = 0.9.
In order to investigate the efficiency of Formula (16), we have changed the definition of hnewin (17) by using Formula (16) with ρ= 1.1 instead of 0.9 (the ρ in the formula
for hreduceremains unchanged). With this enlarged ρ, it turns out that the condition
in step (2) of Algorithm 3 is violated in more than 90% of the iterations (similar results have been obtained for Example 9, below). In contrast to that, with the choice of ρ= 0.9 this typically happens in less than 5% of the iterations, which shows that Formula (16) with ρ= 0.9 very precisely predicts a good (i.e., small enough but not too small) time step hi+1for the next time step.
Festschrift in Honor of Uwe Helmke L. Grüne et al. Example 9. Our second example is [18, Example 4.11(ii)], given by the 2d differen- tial equation
˙z1= −∥z∥2z1+z2, ˙z2= −z1−∥z∥2z2
with Lyapunov function V(z) = ∥z∥2. Since the vector field for this example satisfies ∥ f (z)∥ = O(∥z∥) in a neighborhood of z = 0, one can show that in a neighborhood of z= 0 the consistency error of a p-th order Runge-Kutta scheme satisfies
∥Φ(z0,h)−z(h,z0)∥ = O(hp+1∥z0∥)
which, since V is quadratic, implies
∣̃∆V(z0,h)−∆V(z0,h)∣ = ∣(V(Φ(z0,h))−V(z0))−(V(z(h,z0))−V(z0))∣
= O(hp+1∥z 0∥2).
On the other hand, writing the system in polar coordinates one verifies that ∆V(z0,h) = O(h∥z0∥4),
again in a neighborhood of 0. Hence, for each fixed h> 0 and all z0sufficiently close
to 0 the inequality
̃
∆V(z0,h) < 0 (18)
can not be guaranteed from these estimates, since the Lyapunov difference consistency error∣̃∆V(z0,h) − ∆V(z0,h)∣ is not guaranteed to be smaller than the exact decay
∆V(z0,h). Since the analysis in [18] uses similar estimates, this explains why none
of the sufficient conditions in [18] guaranteeing asymptotic stability for h /→ 0 is satisfied for this example.
However, the fact that (18) is not guaranteed by this analysis does, of course, not imply that this inequality does not hold. Indeed, the fact that the difference∥Φ(z0,h)−
z(h,z0)∥ is large does not necessarily imply that the difference ∣̃∆V(z0,h)−∆V(z0,h)∣
is large: it may well happen that the error included in Φ(z0,h) is large compared to
∆V(z0,h) but nevertheless does not act destabilizing, because it changes the exact
solution z(h,z0) into a direction in which V does not grow — or does even further
decrease. In fact, we already observed this behavior in Example 8 for the Euler scheme and will also observe it for this example, but now for the RK4 scheme. The Figures 5–7 show the simulation results (phase portrait, Lyapunov function V(x(τi)) over time and the time steps) on the time interval [0,200] with λ = 0.5 and
x0= (5,5)⊺.
The total number of steps is 24925 for the Euler scheme, 621 for the Heun scheme and 240 for the RK2 scheme. Hence, in this example the benefit of using higher order schemes is clearly visible.
However, the advantage of the RK4 is not only due to the higher order. Looking at the step sizes one sees that for the Euler and the Heun scheme the step size is strictly decreasing after the first couple of time steps. Longer simulations indicate that the sequences indeed converge to 0 which is in accordance with the discussion above, i.e., that decay of the Lyapunov function can only be guaranteed for vanishing step