Numerical Methods for Scenario Tree Nonlinear Model Predictive Control

(1)

INAUGURAL-DISSERTATION zur

Erlangung der Doktorwürde der Naturwissenschaftlich-Mathematischen Gesamtfakultät der Ruprecht-Karls-Universität Heidelberg vorgelegt von M. Sc. Conrad Leidereiter aus Prenzlau

Tag der mündlichen Prüfung:

(2)

(3)

Numerical Methods for

Scenario Tree Nonlinear Model Predictive Control

Gutachter:

Prof. Dr. Dr. h. c. mult. Hans Georg Bock

Prof. Dr. Ekaterina Kostina

(4)

(5)

Danksagung

Meine tiefe Dankbarkeit gilt an dieser Stelle allen, die mich bei dieser Arbeit inspiriert und unterstützt haben. Ich danke meinen Lehrern und Mentoren der Fakultät für Mathematik und Informatik, Hans Georg Bock, Christian Kirches, Ekaterina Kostina, Johannes Schlöder und Andreas Potschka, für das Teilen ihres umfangreichen Wissens und für das Vertrauen in meine Arbeit.

Für die anregende Umgebung danke ich dem Interdisziplinären Zentrum für Wissen-schaftliches Rechnen der Ruprecht-Karls-Universität Heidelberg, deren Lehre, Forschung und Entwicklung die Basis dieser Arbeit formen. Ich bedanke mich bei allen Kollegen der Arbeitsgruppen Simulation und Optimierung, Numerische Optimierung, Model-Based Op-timizing Control, Optimization of Uncertain Systems und Optimum Experimental Design für die freundliche und kooperative Atmosphäre, für die fachlichen und die überfachlichen Gespräche.

Der Graduiertenschule HGS MathComp als Struktur aus der Exzellenzinitiative danke ich für ihre Angebote, die ich während der Promotion gern genutzt habe. Für die erfolgrei-che Zusammenarbeit im Kreis der Officer des Heidelberg Chapter of SIAM danke ich Anja Bettendorf, Dominik Cebulla, Diana-Patricia Danciu, Jürgen Gutekunst, Robert Kircheis, Ole Klein, Felix Lenders, Robert Scholz, Ruobing Shen und Christoph Weiler.

Dem DFG-Cluster Optimizing Control for Uncertain Systems und dem ERC Grant Model-Based Optimizing Control gebühren Dank für die finanzielle Unterstützung.

Meinen Dank für die Unterstützung in organisatorischen Angelegenheiten richte ich an die Verwaltung der Arbeitsgruppe Simulation und Optimierung, Abir Al-Laham, Margret Rothfuß, Anastasia Valter und Anja Vogel, sowie an die HGS Verwaltung, Ria Hillenbrand-Lynott und Sarah Steinbach. Für die Rahmenbedingungen der Promotion danke ich der Fakultät für Mathematik und Informatik der Ruprecht-Karls-Universität Heidelberg und für die Unterstützung in formalen Promotionsangelegenheiten danke ich Dorothea Heukäufer vom Dekanat.

Schließlich danke ich von Herzen meinen Eltern und meiner Schwester, dass sie immer für mich da sind.

(6)

(7)

Abstract

In this thesis we propose new methods in the field of numerical mathematics and stochastics for a model-based optimization method to control dynamical systems under uncertainty. In model-based control the model-plant mismatch is often large and unforeseen external in-fluences on the dynamics must be taken into account. Therefore we extend the dynamical system by a stochastic component and approximate it by scenario trees. The combination of Nonlinear Model Predictive Control (NMPC) and the scenario tree approach to robus-tify with respect to the uncertainty is of growing interest. In engineering practice scenario tree NMPC yields a beneficial balance of the conservatism introduced by the robustification with respect to the uncertainty and the controller performance. However, there is a high numerical effort to solve the occuring optimization problems, which heavily depends on the design of the scenario tree used to approximate the uncertainty. A big challenge is then to control the system in real-time. The contribution of this work to the field of numeri-cal optimization is a structure exploiting method for the large-snumeri-cale optimization problems based on dual decomposition that yields smaller subproblems. They can be solved in a massively parallel fashion and additionally their discretization structure can be exploited numerically. Furthermore, this thesis presents novel methods to generate suitable scenario trees to approximate the uncertainty. Our scenario tree generation based on quadrature rules for sparse grids allows for scenario tree NMPC in high-dimensional uncertainty spaces with approximation properties of the quadrature rules. A further novel approach of this thesis to generate scenario trees is based on the interpretation of the underlying stochastic process as a Markov chain. Under the Markovian assumption we provide guarantees for the scenario tree approximation of the uncertainty. Finally, we present numerical results for scenario tree NMPC. We consider dynamical systems from the chemical industry and demonstrate that the methods developed in this thesis solve optimization problems with large scenario trees in real-time.

(8)

(9)

Zusammenfassung

Diese Arbeit stellt neue Methoden aus dem Bereich der numerischen Mathematik und der Stochastik für ein modellbasiertes Optimierungsverfahren zur Regelung unsicherheitsbe-hafteter dynamischer Systeme vor. Bei der Anwendung modellbasierter Regelungsansätze ist die Diskrepanz zwischen dem mathematischen Modell und der zu steuernden Anlage oft groß. Auch unvorhersehbare externe Einflüsse müssen berücksichtigt werden. Daher erweitern wir das dynamische System um eine stochastische Komponente und approximie-ren diese durch einen Szenarienbaum. Nichtlineare modellprädiktive Regelung (NMPC) in Kombination mit dem Szenarienbaumansatz als Robustifizierung gegen die Unsicherheit stößt auf wachsendes Interesse in der Regelungstechnik, denn in der Praxis hat sich gezeigt, dass Szenarienbaum-NMPC eine gute Balance zwischen dem Konservatismus von Robu-stifizierungsmethoden und der Performanz des Reglers schafft. Der numerische Aufwand für die Lösung der auftretenden Optimierungsprobleme hängt stark von der Struktur des Szenarienbaumes ab. Dieser wiederum soll die Unsicherheit möglichst gut approximieren. Eine große Herausforderung von Szenarienbaum-NMPC ist die Lösung der auftretenden Optimierungsprobleme in Echtzeit. Der Beitrag dieser Arbeit im Forschungsfeld der nume-rischen Optimierung ist ein strukturausnutzendes Verfahren, welches die auftretenden Op-timierungsprobleme mit Hilfe von dualer Dekomposition in kleinere Teilprobleme zerlegt. Die Teilprobleme können parallel gelöst werden unter zusätzlicher numerischer Ausnut-zung ihrer Diskretisierungsstruktur. Desweiteren stellt diese Arbeit neuartige Verfahren vor, die passende Szenarienbäume generieren. Unsere Szenarienbaumgenerierung basierend auf Quadraturformeln mit dünnen Gittern ermöglicht Szenarienbaum-NMPC in mehrdimensio-nalen Unsicherheitsräumen mit den Approximationseigenschaften der Quadraturformeln. Ein weiterer neuartiger Ansatz der Arbeit zum Generieren von Szenarienbäumen basiert auf der Interpretation des zugrundeliegenden stochastischen Prozesses als Markovkette. Unter der Markov-Annahme geben wir Garantien, wie gut die Unsicherheit durch den Szenarien-baum approximiert wird. Schließlich präsentieren wir in der Arbeit numerische Resultate für Szenarienbaum-NMPC. Wir betrachten dynamische Systeme aus industriellen Anwen-dungen der chemischen Verfahrenstechnik und belegen, dass mit den entwickelten Metho-den dieser Arbeit Optimierungsprobleme mit großen Szenarienbäumen in Echtzeit gelöst werden können.

(10)

(11)

Introduction

Mathematical optimization under uncertainty is a growing field of research as the aware-ness of the uncertain influence has increased. Steering real-world systems into a desired state by dynamical model-based optimization approaches requires to make decisions within a specified time. Full information that can serve as basis for the decision is hardly ever avail-able. Therefore uncertainty occurs in a natural way and has to be considered in dynamical model-based optimization methods. Robert K. Greenleaf, a pioneer of modern manage-ment, leadership, organizational development and education approaches wrote in his essay [49]:

”On an important decision one rarely has 100 % of the information needed for a good decision no matter how much one spends or how long one waits. And, if one waits too long, he has a different problem and has to start all over.”

We emphasize three insights from the citation that originates from a management context but fits perfectly in our framework. First, uncertainty is present in decision making. Second, the uncertainty does not vanish completely over time. And third, the problem that requires decisions, will change over time. Especially the third insight will lead us later to the fast feedback principle.

Scenario tree Nonlinear Model Predictive Control accounts for the uncertain influence and generates decisions to control a dynamical system in a robust sense - robust against the uncertainty. Nonlinear Model Predictive Control (NMPC) is a model-based mathemati-cal optimization framework to obtain feedback control inputs for a dynamimathemati-cal system. The framework has been applied in various fields such as physics, chemistry, biology, engi-neering, social sciences and psychology, overviews of NMPC applications can be found for example in [85, 95]. Techniques to numerically address the mathematical optimization problems in NMPC have been developed e.g. in [17, 109, 4, 11] and continuously enhanced by real-time and multi-level components [27, 15, 114, 67, 44]. However, the mismatches

(14)

2 CHAPTER 1. INTRODUCTION

between the real-world system and the mathematical model as well as disturbances are often large. Therefore NMPC can only be successful when the feedback controls are computed in a robust sense as [10] pointed out. Otherwise, in the case of a chemical plant for example, violations of system constraints can cause severe safety-critical situations.

Robust NMPC techniques treat model-plant mismatch and disturbances as probabilistic perturbations to a nominal model. One technique is the game-theoretic worst case approach, mathematically a bilevel optimization problem [26]. It suffers from high conservatism, be-cause it safeguards against all possible realizations of the perturbation at the same time. Lucia et al. [84] have shown that using scenario tree NMPC as proposed in [23] can effi-ciently reduce this conservatism while remaining feasible with high probability.

The main assumption of the scenario tree approach is of stochastic nature. We assume that the uncertainty can be approximated by a discrete and finite tree process. Usually the scenario tree construction starts with choosing a finite number of realizations of the uncertainty. Then a number of decision points in time are specified. At these points the realizations are allowed to change. The sequences of those realizations are regarded as scenarios that must be coupled according to the tree structure. Figure 1.1 illustrates the classical tree construction. There are alternatives to generate the scenario trees. In this thesis we develop tree generation algorithms based on the investigation of the underlying stochastic process.

The major challenge of scenario tree NMPC is the exponential growth of the full tree in the number of decision points. This yields a high demand for fast structure-exploiting numerical methods. Especially in high dimensional uncertainty spaces the classical tree construction itself must be questioned. How should the uncertainty space be discretized and how should we choose a finite number of realizations? Are there alternative tree structures? This thesis aims to provide answers to the questions based on a mathematical investigation of the stochastic process behind the scenario tree and practical implementations of demand-ing case studies. Contributions of the thesis to scenario tree NMPC are named in the next section.

1.1 Aims and Contributions of this Thesis

The overarching goal is to control dynamic processes under uncertain influence in real-time using scenario tree NMPC. This thesis contributes to the field of applied mathematics by the following topics.

(15)

1.1. AIMS AND CONTRIBUTIONS OF THIS THESIS 3 t0 t1 t2 x1 0 x x1 1 x x2 1 x x3 1 x u1 0 u1 1 u2 1 u3 1 p₁ p₂ p₃ tpresent = t0 t1 t2

Figure 1.1: Single scenarios (left) and scenario tree (right) with three realizations of the uncertainty and two decision points. The three realizations are represented by the dotted, the dashed and the two-dot-one-dash line styles. We have two decision points t0and t1where

the realization can change. Between the time points t0and t1(stage 1) and between t1 and

t2(stage 2) the parameter realizations stay constant respectively. The nine scenarios on the

left are all combinations of the parameter realizations for the first two stages. We couple the scenarios due to a tree-specific stochastic principle and arrive at the scenario tree structure on the right. The colors indicate the controls we have at hand for optimization according to the stochastic tree process. For the continuation of the scenarios beyond t2there are various

possibilities indicated by dots. Most commonly the realizations stay constant over the rest of the considered time horizon.

(16)

Formulation of stochastic properties of the scenario tree process

Scenario tree NMPC is located at the intersection of the fields numerics, optimization, pro-cess control and stochastics and has been investigated by researchers of all those fields. However, the stochastic process behind scenario trees requires a rigorous formulation. This work contributes a mathematical tree process formulation and rigorous investigation of a special process property, the non-anticipativity.

Fast structure-exploiting numerical methods for scenario tree NMPC

For fast NMPC with scenario trees the exploitation of the discretization structure, tree struc-ture and state-of-the art methods to deal with real-time requirements are inevitable. In this thesis we contribute a fast numerical method for the solution of large-scale tree-structured problems based on a dual decomposition approach that exploits the inherent tree structure. In combination with discretization structure exploitation and the Real-Time Iteration or Multi-Level Iteration scheme we can speed up the necessary computations of the process feedback.

Quadrature-based scenario tree generation

In high-dimensional uncertainty spaces the major challenge of generating scenario trees is the choice of finitely many realizations of the uncertainty. We present a contribution of scenario tree generation based on sparse grid quadrature rules, a commonly used tool in the field of uncertainty quantification. With the approach the number of realizations can be significantly reduced and there is still a reasonable approximation quality determined by error formulars of sparse grid quadrature.

Scenario tree generation based on Markov chains

In the classical tree construction the scenarios are determined by the finitely many real-izations and the decision points where the tree can branch. As a consequence the number of scenarios grows exponentially in the number of decision points. Hence, for practical computations the number of decision points must be restricted to a robust horizon in the beginning. This thesis contributes an alternative scenario tree generation approach that is based on the stochastic properties of the tree process and does not require the choice of a ro-bust horizon. We provide a fast implementation of the alternative tree generating algorithm that builds on interpreting scenarios as realizations of Markov chains over time and pruning the most unlikely scenarios.

(17)

1.1. AIMS AND CONTRIBUTIONS OF THIS THESIS 5

Implementation of scenario tree NMPC

We contribute a software implementation of the structure-exploiting numerical algorithms of the thesis to demonstrate the effectiveness and efficiency of the methods. The STMLI package (Scenario Tree Multi Level Iteration package) is the scenario tree extension of the MLI software (Multi Level Iteration software) that implements the Multi Level Iteration idea [16, 15].

Demanding case studies

One important contribution of this work is to apply the developed methods to challenging problems. First, we study a basic chemical reactor, a continuous stirred tank reactor. Then we continue with a demanding real-world industrial problem, a biochemical batch reactor. The plant model is provided by BASF. As a third example we consider a pharmaceutical problem on Penicillin production. All problems have in common that there is uncertainty present in the dynamical system model. We showcase the performance and robustness of scenario tree NMPC with respect to the uncertainty.

Published contributions

The scenario tree generating approach based on sparse grids has been published in

[74]: C. Leidereiter, A. Potschka, and H. G. Bock. Quadrature-based scenario tree generation for Nonlinear Model Predictive Control. In Proceedings of the 19th IFAC World Congress, volume 47, pages 11087–11092, 2014.

To demonstrate performance and robustness of NMPC with scenario trees generated from sparse grids we study the control of a simulated distillation column with three-dimensional uncertainty space on a Monte-Carlo testbed and statistically evaluate the results.

The scenario tree structure exploiting numerical method based on dual decomposition has been published in

[75]: C. Leidereiter, A. Potschka, and H. G. Bock. Dual decomposition for QPs in scenario tree NMPC. In Proceedings of the European Control Conference (ECC15), pages 1608–1613, 2015.

We present the structure-exploiting numerics and computational results for NMPC with large scenario trees.

(18)

[73]: C. Leidereiter, D. Kouzoupis, M. Diehl, and A. Potschka. Pruning for scenario tree NMPC with uncertainties described by Markov chains. In prepa-ration.

In the above publication we will discuss numerical results for an uncertainty that has an intrinsic Markovian property.

1.2 Organization of this Thesis

Chapter 1 is this introduction. In Chapter 2 of problem classification we put scenario tree NMPC into context and view the main object of this thesis from a numerical, optimization, process control and stochastic perspective. The latter focuses on properties of the stochas-tic tree process, especially non-anstochas-ticipativity. In the three subsequent chapters we present numerical methods for scenario tree NMPC. We discuss the Direct Multiple Shooting dis-cretization including the exploitation of disdis-cretization structure in Chapter 3. It serves as the basis for fast feedback NMPC, that we survey in Chapter 4. We propose our tree structure exploiting numerical method in Chapter 5. The dual decomposition approach with non-smooth Newton method is the main contribution of numerical structure exploitation in this thesis. The contributions based on the stochastic nature of the scenario tree process follow in Chapter 6 with quadrature-based scenario tree generation and in Chapter 7 with Markov chain scenario tree pruning. In Chapter 8 we describe the software implementation of the numerical methods. With numerical results for demanding applications we demonstrate the efficiency of our methods in Chapter 9. In the last Chapter 10 we draw conclusions and give an outlook on topics that arise from the results of this thesis.

(19)

Chapter 2

Problem Classification

In this chapter we start with the mathematical formulation of the scenario tree optimization problem, also known as multi-stage optimization problem. We approach the core object from various perspectives. The first perspective is to consider the dynamical aspects and focus on the optimal control formulation. We continue with the optimization perspective and focus on the discretized finite nonlinear optimization problem in the second section. The next perspective, the dynamical process control, underlines the requirement of fast feedback methods. At the end of this chapter we present the stochastic properties of the scenario tree process. Theorem, proof and examples in the last section are contributions of this thesis.

2.1 The Scenario Tree Optimization Problem

Generally, optimization with scenario trees is a systematic and efficient approach to deal with uncertainty in nonlinear model predictive control (NMPC) as we have pointed out in the introductory chapter. At this point we already state the scenario tree optimal control problem as one core optimization problem in scenario tree NMPC being aware that back-ground from different perspectives in the next sections is required to fully explain it.

We set I := [t0,tf] ⊂ R as the underlying time domain, define the states x : I → Rnx and

the controls u : I → Rnu_{. In this section we denote the possibly uncertain parameters simply} as p ∈ Rnp_{. A further investigation and rigorous definition follows later on. We consider} a finite number S ∈ N of scenarios. From now on the elements of the set of scenarios S = {1, . . . , S} have index j. We assign a weight wj∈ [0, 1] to all scenarios j ∈ S.

Further-more, we define the set K = {0, . . . , M − 1}. The time interval I is partitioned by

t0< t1< · · · < tM−1< tM= tf

into intervals [tk,tk+1], k ∈ K, the so-called stages that are eponymous for the multi-stage

(20)

8 CHAPTER 2. PROBLEM CLASSIFICATION

approach.

The sufficiently smooth functions

f _{: R}nx_{× R}nu_{× R}np _{→ R}nx_,

Φ : R × Rnx× Rnu× Rnp → R

describe the system dynamics and the cost function. Furthermore we require sufficiently smooth descriptions of the feasible regions X and U for the states and controls. At the current system state x(t0) = x0the optimal control problem reads

min x(t),u(t)

∑

_j∈Swj Z t_f t0 Φ(t, xj(t), uj(t), pj(t))dt (2.1a) s.t. x˙j(t) = f (xj(t), uj(t), pj(t)), t∈ [t0,tf], j ∈ S, (2.1b) xj(t0) = x0, j∈ S, (2.1c) xj(t) ∈ X , t∈ [t0,tf], j ∈ S, (2.1d) uj(t) ∈ U , t∈ [t0,tf], j ∈ S, (2.1e) ui(t) = uj(t), t∈ [tk,tk+1], (i, j) ∈ Ck, k ∈ K. (2.1f)

In the above formulation every scenario has its own controls. We define the set Ck for

constraint (2.1f) to encode the tree structure,

Ck:=(i, j) ∈ S2

p_i(t) = p_j(t) for all t ∈ [t₀,t_k] .

The set Ckis required for the scenario-wise view of the tree structure. For every stage k, the

set Ckcontains tuples of scenarios with common history of parameter values on all previous

stages. The respective control variables of the scenario tuples in Ck are coupled on stage k

according to (2.1f). We emphasize that the constraints (2.1f) are the only coupling of the scenarios. Problem (2.1) without the linear coupling constraints completely decouples into Sscenario optimal control problems. To sum up, we minimize a weighted sum of scenario objectives with respect to dynamic constraints, an initial value, upper and lower variable bounds and linear scenario coupling constraints.

(21)

2.2. DYNAMICAL SYSTEM PERSPECTIVE: OPTIMAL CONTROL 9

2.2 Dynamical System Perspective: Optimal Control

The evolution of processes that change in time can be modelled by dynamical systems. Transferring the observations from a physical, biological or chemical process to a dynamical model and applying mathematical tools helps to gain more insight into the process behavior. This applies as well to processes from economical, social and life sciences. For abstract dynamical systems we refer to [52]. The ordinary differential equation (ODE) of the form

˙

x= f (t, x)

corresponds in a natural way to a dynamical system. The function f : D → Rd is a time-dependent continuous vector field on D ⊂ R×Rd. In the autonomous case f does not depent on time. But we can transform every ODE to an autonomous ODE by introducing an extra state τ that represents the time. We then replace all t-dependencies of f by τ-dependencies and add the equations ˙τ = 1 and τ (0) = t0to the ODE.

For an ODE ˙x= f (t, x) and (t0, x0) ∈ D we can solve initial value problems (IVPs). In

the following we state the fundamental existence and uniqueness theorem of IVP solutions.

Theorem 2.1 (Existence and uniqueness of solutions). Let f : D → Rd, D ⊂ R × Rd be a vector field that is continous and Lipschitz continuous in the second argument. Now consider the ODEx˙= f (t, x). Then for all initial values (t0, x0) ∈ D there exists a unique

solution x(t;t0, x0) to the ODE with initial condition x(t0) = x0 on a maximum existence

interval It0,x0 ⊂ R with t0∈ It0,x0.

The existence and uniqueness theorem is usually proven by the Banach contraction map-ping theorem. Often the theorem is referred to Picard and Lindelöf.

Ordinary differential equations implicitly define the dynamical evolution constraint of optimal control problems. At this point we state an optimal control problem (OCP) in a general form to optimize a process with underlying dynamical system. We transfer the assumptions of Theorem 2.1 to all following OCPs with underlying ODE.

The general optimal control problem reads

min x,u Z t_f t0 Φ(t, x(t), u(t), p(t))dt (2.2a) s.t. x(t) = f (x(t), u(t), p(t)),˙ t∈ [t₀,tf] , (2.2b) 0 = x(t0) − x0, (2.2c) 0 ≤ r(t, x(t), u(t), p(t)), t∈ [t0,tf] . (2.2d)

Our goal is to minimize the objective over the states x : [t0,tf] → Rnx and the controls

u: [t0,tf] → Rnu. We do not regard the possibly uncertain parameters p ∈ Rnp as

(22)

The objective function (2.2a) is a Lagrange term

Z tf

t0

Φ(t, x(t), u(t), p(t))dt

with Φ : [t0,tf] × Rnx+nu+np → R. The function Φ is assumed to be twice continuously

differentiable.

The dynamical evolution of the system is implicitly given by the constraint (2.2b) and the initial value of the dynamical evolution is set by (2.2c).

Additionally, we have path constraints (2.2d) described by a twice continuously differ-entiable r : [t0,tf] × Rnx+nu+np → Rnr. The so-called box constraints of the form

x(t) ≤ x(t) ≤ x(t) u(t) ≤ u(t) ≤ u(t)

are a special case of (2.2d).

Further possible components of optimal control problems such as interior point con-straints are not considered in this thesis. Objective formulations that depend on (tf, x(tf))

can be reformulated to fit (2.2a).

Now the question arises how to solve the OCP (2.2) as the optimization variables are functions and we therefore have an infinite-dimensional problem. For an in-depth introduc-tion to funcintroduc-tional analysis and the relaintroduc-tion to optimal control and calculus of variaintroduc-tions we refer to the textbook [21].

Since the middle of the 20th century indirect approaches to solve optimal control prob-lems emerged from Pontryagin’s maximum principle [91]. The maximum principle states necessary conditions of optimality which can be solved analytically for certain examples. We first transform the optimal control problem to a boundary value problem (BVP) by the maximum principle and then solve the BVP numerically by collocation or shooting meth-ods.

In contrast to the indirect approach we choose direct methods - we first discretize the infinite-dimensional optimization problem and then optimize. The direct multiple shooting method [17] and the direct collocation method [109, 4, 11] yield finite dimensional opti-mization problems from optimal control problems. For an efficient and fast solution the resulting discretized problems must be treated with tailored numerical methods. The direct multiple shooting discretization and a related discretization structure exploiting method is explained in Chapter 3.

We conclude the section with the remark that the scenario tree problem (2.1) is a special case of the OCP (2.2). It has to be emphasized that the methods developed for problems of the form (2.2) are fully applicable to the scenario tree problem (2.1). Furthermore we can

(23)

2.3. OPTIMIZATION PERSPECTIVE: NONLINEAR PROGRAMMING 11

understand the OCP (2.2) as problem (2.1) with only one scenario and weight w1= 1. The

so-called nominal problem corresponding to a scenario tree optimization problem is such a one-scenario problem with the nominal parameter realization.

2.3 Optimization Perspective: Nonlinear Programming

In Section 2.2 we have stated that the OCP (2.2) or its special case with tree structure (2.1) can be transferred to a finite dimensional optimization problem by direct methods. Those finite dimensional optimization problems are the mathematical objects of interest in the field of numerical optimization. The general constrained nonlinear optimization problem (NLP) in Rn_{is defined as follows.}

Definition 2.1 (Nonlinear optimization problem).

min

z∈Rn F(z) (2.3a)

s.t. ci(z) = 0, i∈ E (2.3b)

ci(z) ≥ 0, i∈ I. (2.3c)

The functions F and ci are assumed to be twice continuously differentiable functions

mapping a subset of Rn to R. We call F the objective function, ci, i ∈ E , |E | = ce equality

constraints and ci, i ∈ I, |I| = ci inequality constraints. In this formulation the sets E and

I are disjoint. If the objective function and all the constraint functions are linear, (2.3) becomes a linear programming problem (LP). The quadratic programming problem (QP) is (2.3) with linear constraint functions and an objective function of the form

F(z) = 1 2z

T

Hz+ gTz.

The symmetric matrix H ∈ Rn×n is addressed as the Hessian of the QP and the vector g_{∈ R}n as the QP gradient. In the following we state only the most important definitions and theorems of NLP theory as this is not the main focus of thesis. A comprehensive introduction to numerical optimization is for example the book [88].

Definition 2.2 (Feasible). A point z ∈ Rnis called feasible point of the problem(2.3) if it satisfies

ci(z) = 0 for all i∈ E,

(24)

Definition 2.3 (Local optimum). A point z∗∈ Rn _{is called locally optimal point of the}

problem(2.3) if for ε > 0 there exists an open ball Bε(z∗) with ε > 0 such that

F(z) ≥ F(z∗)

for all feasible z∈ B_ε(z∗).

If additionally F(z) > F(z∗) holds for all feasible z 6= z∗, then z∗is a strict local optimum.

Definition 2.4 (Active set). For a feasible point z we call an inequality constraint ci active

for some i∈ I if ci(z) = 0. Equality constraints are always active in a feasible point. The

active set associated to a feasible point z is the set of indices of all active constraints

A(z) = {i|ci(z) = 0}.

Constraint qualifications (CQs) are important to ensure that the feasible set in a neigh-borhood of a feasible point is well-behaved. There is a whole theory on how to derive CQs for certain types of NLPs [55]. We often assume the set of active constraint gradients to be linearly independent.

Definition 2.5 (Linear independence constraint qualification). Given the feasible point z and its active setA(z) according to Definition 2.4. We say that the linear independence con-straint qualification (LICQ) holds if the set of active concon-straint gradients{∇ci(z), i ∈ A(z)}

is linearly independent.

Definition 2.3 characterizes a locally optimal point. However, this characterization is computationally expensive to check. Therefore we state optimality conditions that can be handled more conveniently from a computational perspective. For this purpose we need the Lagrangian function.

Definition 2.6 (Lagrangian function). The function L : Rnz_{× R}nce_{× R}nci_{→ R,} L(z, µ, λ ) := F(z) −

_∑

i∈E

λici(z) −

∑

i∈I

µici(z)

with Lagrangian multipliers λ ∈ Rnce_{and µ ∈ R}nci _{is called the Lagrangian function of the} NLP(2.3).

Theorem 2.2 (1st order neccesary conditions of optimality). Let z∗∈ Rn_{be a point}

satisfy-ing LICQ and a local minimum of (2.3). Then there exist λ∗∈ Rnce_{and µ}∗_{∈ R}nci _{such that} (z∗, λ∗, µ∗) satisfies the following:

Stationarity: ∇zL(z∗, λ∗, µ∗) = ∇F(z∗) −

∑

i∈E λ_i∗∇ci(z∗) −

∑

i∈I µ_i∗∇ci(z∗) = 0 (2.4)

(25)

2.3. OPTIMIZATION PERSPECTIVE: NONLINEAR PROGRAMMING 13 Feasibility: ci(z∗) = 0 for i∈ E, (2.5) ci(z∗) ≥ 0 for i∈ I, (2.6) Dual feasibility: µ∗≥ 0 (2.7) Complementarity: µ_i∗ci(z∗) = 0, for i∈ I. (2.8)

Theorem 2.2 is often referred as Karush-Kuhn-Tucker theorem, named after [64] and [71]. A point (z∗, λ∗, µ∗) satisfying the conditions of Theorem 2.2 is called KKT-point. We notice that Theorem 2.2 provides necessary conditions for optimality that can be checked computationally. To distinguish between minimum, maximum and saddlepoints we state 2nd order conditions. Therefore we require the following two sets representing cones for a feasible point z ∈ Rnsatisfying LICQ:

ˆ

T(z) = {p ∈ Rn| ∇ci(z)p = 0 for i ∈ E(x), ∇cj(z)>p= 0 for j ∈ I(x)},

T(z) = {p ∈ Rn| ∇ci(z)p = 0 for i ∈ E(x), ∇cj(z)>p= 0 for j ∈ I(x) and µj> 0}.

Furthermore, ∇zzL(z∗, λ∗, µ∗) = ∇2F(z∗) −

∑

i∈E λi∗∇2ci(z∗) −

∑

i∈I µi∗∇2ci(z∗)

is the Hessian matrix of the Lagrangian function with respect to z.

Theorem 2.3 (2nd order neccesary conditions of optimality). Let z∗∈ Rn_{be a local}

mini-mum satisfying LICQ and let λ∗, µ∗be Lagrange multipliers such that(z∗, λ∗, µ∗) is a KKT point. Then

p>∇zzL(z∗, λ∗, µ∗)p ≥ 0 for all p∈ ˆT(z∗).

Theorem 2.4 (2nd order sufficient conditions of optimality). Let z∗∈ Rn_{be a local}

mini-mum satisfying LICQ and let λ∗, µ∗be Lagrange variables such that(z∗, λ∗, µ∗) is a KKT point. If

p>∇zzL(z∗, λ∗, µ∗)p > 0 for all p∈ T (z∗)

holds, then z∗is a strict local minimum.

In the broad field of nonlinear optimization various concepts have been developed to solve (2.3). We mention three algorithmic classes and refer the interested reader to [88].

(26)

Interior-point methods

The interior-point or barrier methods are a class of efficient NLP algorithms. We treat the combinatorial complexity introduced by inequality constraints (2.3c) through smoothing and obtain iterates that are strictly in the interior of the set of feasible points. A compre-hensive overview is in [112]. Interior-point method tailored to tree-structured NLPs can be found in [60].

Augmented Lagrangian methods

In augmented Lagrangian methods we define a minimizer function combining the La-grangian and a penalty term depending on the constraints. Some aspects of augmented Lagrangian methods are present in the algorithms for dynamic optimization. Especially in the dual-decomposition based algorithm in Chapter 5 a part of the augmented Lagrangian term becomes relevant.

Sequential quadratic programming

In sequential quadratic programming (SQP) methods we pass the combinatorial complexity of determining the correct active set to a simpler subproblem. This simpler problem is often a convex quadratic programming problem (QP). QP subproblems have some favorable properties and characteristics making SQP one of the most successful methods for nonlinear programming. If we need to repeatedly solve similar NLPs for example in the context of real-time optimization, these features have a strong effect, see Chapter 4.

The Tree QP

The tree QP is a special case of a QP, therefore a finite dimensional scenario tree optimiza-tion problem. We want to state it here to finish the secoptimiza-tion on the optimizaoptimiza-tion perspective.

min z1,...,zS _j∈S

∑

wj 1 2z T jHjzj+ gTjzj (2.9a) s.t. xj,0= x0, j∈ S, (2.9b) xj,k+1= Aj,kxj,k+ Bj,kuj,k, k∈ K, j ∈ S, (2.9c) x≤ xj,k≤ x, k∈ K ∪ {M}, j ∈ S, (2.9d) u≤ uj,k≤ u, k∈ K, j ∈ S, (2.9e) Ej+1zj+1= Cjzj, j∈ S\S. (2.9f)

(27)

2.4. PROCESS CONTROL PERSPECTIVE: FAST FEEDBACK IN REAL-TIME 15

The set K = {0, . . . , M − 1} contains the time indices, and the set S = {1, . . . , S} the scenario indices. Corresponding scenario weights are wj≥ 0 for all j ∈ S. State and control variables

are grouped in z|_j = [x|_j,0, u|_j,0, . . . , x|_j,M−1, u|_j,M−1, x|_j,M]|_{∈ R}nz_{for all j ∈ S. At this point we} require box constraints for states and controls. We exploit the inherent tree structure of (2.9) for fast numerical solution methods in Chapter 5.

2.4 Process Control Perspective: Fast Feedback in Real-Time

Up to now we have introduced the optimization problems in the context of scenario tree NMPC. For the online control of an application such as a chemical plant we do not only solve one optimization problem. It is required to solve a sequence of dynamical opti-mization problems and with that, new challenges arise. In the moving horizon framework [25, 98] we solve problem (2.2) or a scenario tree optimization problem (2.1) for every in-stant t of a sampling time grid in order to drive a process according to the desired objective. The problems (2.2) and (2.1) depend parametrically on the initial value x0representing the

current state of the plant that further depends on the current sampling time t, so x0= x0(t). In

ideal NMPC and the special case scenario tree NMPC the optimization problem is assumed to be solved instantaneously. The resulting optimal control u∗(t) is immediately fed back to the process. Therefore ideal NMPC can be regarded as a control feedback law u∗(t, x(t)) implicitly given by the solution of optimization problems (2.2) or (2.1). The dynamical evolution of the process is then described by the ODE

˙

x(t) = f (t, x(t), u∗(t, x(t)), p(t)).

The practical implementation of NMPC differs from the idealized NMPC by the fol-lowing main points. First, we do not solve the infinite-dimensional problem (2.2) or (2.1) at every sampling time, but a finite dimensional approximation, and second, the solution u∗(t) is not obtained and applied to the process immediately. There is a delay between receiving the current process state x0(t) and feeding the control back to the process,

deter-mined mainly by the cost of solving the discretized optimization problem. In practice these circumstances affect the performance and stability of the controller [38].

For fast feedback we need to minimize the delay by fast NMPC schemes. We empha-size that fast NMPC is not just NMPC with a fast solver, it rather requires sophisticated methods that we describe in Chapter 4. One algorithmic concept to reduce the time de-lay between retrieving the process state and the control feedback is the Real-Time Iteration scheme [28] that we focus on in Section 4.2. Major ingredients are the initial value embed-ding idea and splitting the computation into phases. A further contribution to fast NMPC is the Multi-Level-Iteration scheme [16]. We describe it in Section 4.3. The fast feedback

(28)

NMPC methods tackle the model-plant mismatch and disturbances. Further robustification against uncertainty in the process is achieved by stochastic approaches. We continue with the stochastic nature of scenario tree NMPC in the next section.

2.5 Stochastic Perspective: Discrete Tree Process

At this point we focus on the stochastic nature of the uncertainty. In the field of optimization under uncertainty, especially in stochastic finance applications and optimal portfolio selec-tion problems [30, 51, 58], the evoluselec-tion of a discrete stochastic process is often illustrated as a scenario tree. We adapt this interpretation and regard the uncertain parameter process as a finite stochastic process. In this section we consider two processes and define math-ematically rigorously a central property of the tree process. Uncertainty is present in the underlying parameter process. In traditional theory, which concentrates mainly on stability and tractability assertions for robust optimal control, there are two basic approaches to in-corporate parameter uncertainty into optimization problems. While in robust optimization the uncertainty model is basically deterministic and set-based, the second perspective builds upon a probabilistic description of the uncertainty. The thesis [61] extensively investigates the effects from the robust and probabilistic optimization perspective. A main focus there lies on optimal control problems driven by stochastic differential equations. The scenario tree approach is located between the two basic approaches. In this section we aim to clarify the stochastic properties of the scenario tree uncertainty model. For this purpose we require basic stochastic definitions. In the following we give a condensed overview. For a detailed introduction and further background in the field we refer to the textbooks [31, 68].

(29)

2.5. STOCHASTIC PERSPECTIVE: DISCRETE TREE PROCESS 17

Definition 2.7 (Sigma-Algebra). Let Ω be a non-empty set and P(Ω) its power set. A subsetF ⊂ P(Ω) is a sigma-algebra if it satisfies the following properties:

1. F contains the universal set: Ω ∈ F .

2. F is closed under complementation: If A ∈ F , then Ω \ A ∈ F .

3. F is closed under countable unions: If (Ai)i∈Nwith Ai∈ F for all i, then S

i∈NAi∈ F .

We call the elements ofF measurable sets and (Ω, F ) a measurable space.

Definition 2.8 (Generated Sigma-Algebra). Let G be any subset of P(Ω). Then the smallest sigma-algebra containingG, σ (G) =T

{F |F is sigma-algebra on Ω, G ⊂ F }, is called the sigma-algebra generated byG.

Definition 2.9 (Measurable Function). Let (Ω, F ) and (Ω0, F0) be measurable spaces. A function f : Ω → Ω0is a measurable function if for the preimages it holds that

f−1(A0) := {x ∈ Ω| f (x) ∈ A0} ∈ Ω, for all A0∈ F0.

Definition 2.10 (Sigma-algebra generated by a function). Let Ω be a non-empty set, (Ω0, F0) a measurable space and f : Ω → Ω0 a function. Then the sigma-algebra on Ω generated by the function f , denoted as σ ( f ), is the collection of all preimages of the sets in F0: σ ( f ) := { f−1(A0)| A0∈ F0}. It is the smallest sigma-algebra such that f is measurable.

Definition 2.11 (Measure). A measure µ on a measurable space is a non-negative function µ : F → [0, ∞[ such that

1. µ( /0) = 0,

2. for(Ai)i∈Nwith Ai∩ Aj= /0 for all i 6= j ∈ N the equality

µ ∞ [ i=1 Ai ! = ∞

∑

i=1 µ (Ai) holds.

The triple(Ω, F , µ) is called measure space.

If additionally µ(Ω) = 1, we call (Ω, F , µ) a probability space.

Definition 2.12 (Random Variable). Let (Ω, F , P) be a probability space and (Ω0, F0) a measurable space. A random variable X is a measurable function X: (Ω, F , P) → (Ω0, F0). Every random variable X induces a probability measure µX on(Ω0, F0) through

(30)

Definition 2.13 (Stochastic Process). Let I ⊂ R. A collection of random variables X = (Xt,t ∈ I) on a probability space (Ω, F , P) with values in a measurable space (Ω0, F0) is a

stochastic process with time domain I and state space Ω0.

If I is discrete or even finite, the process X is named discrete or finite stochastic process. In case of(Ω0, F0) = (R, B(R)) (B(R) the Borel sigma-algebra), X is a real-valued stochastic process.

Definition 2.14 (Filtration). A collection F = (Ft,t ∈ I) of sigma-algebras with Ft⊂ F for

all t∈ I is a filtration, if Fs⊂ Ft for all s,t ∈ I with s ≤ t.

Definition 2.15 (Adapted to a Filtration). A stochastic process X = (Xt,t ∈ I) is adapted to

the filtration F = (Ft,t ∈ I) if Xt is measurable with respect toFt for all t∈ I.

At this point we return to the scenario tree approach. We assume that the parameters representing the uncertainty are constant on every stage. In the following we consider two stochastic processes, the uncertain parameter process and the scenario tree process.

Definition 2.16 (Discrete uncertain parameter process). The underlying parameter pro-cess is the collection of random variables X= (Xt,t ∈ {0, . . . M}), M ∈ N, Xt: (Ω, F , P) →

(R, B(R)) for all t.

Then we define the discrete scenario tree process on the finite subset E ⊂ Ω, as we have to choose a finite number of parameter realizations.

Definition 2.17 (Scenario tree process). The scenario tree process is the collection of ran-dom variables Y = (Yt,t ∈ {0, . . . , M}), M ∈ N, Yt : (E, E, P0) → (R, B(R)) with E ⊂ Ω a

finite set,E = σ (E) and P0a probability measure onE.

Remark 2.1. The measure P0of the scenario process is chosen either as branch probabil-ities P0(Yt = wi) = pi for all realizations wi ∈ E, or transition probabilities depending on

the previous realization P0(Yt+1= wj|Yt = wi) = pi jfor wi, wj∈ E.

We now define the most prominent term used in the context of scenario tree NMPC. The definition is mathematically rigorous in the sense that it relates a stochastic process to a filtration containing the probabilistic information of another stochastic process.

Definition 2.18 (Non-anticipativity). A stochastic process Y = (Yt,t ∈ J) is called

non-anticipative with respect to the stochastic process X = (Xt,t ∈ I) if Y is adapted to the

filtration σ (Xt,t ∈ I).

Remark 2.2.

The definition of non-anticipativity is independent of the measures of the two stochastic processes.

(31)

2.5. STOCHASTIC PERSPECTIVE: DISCRETE TREE PROCESS 19

Remark 2.3.

Non-anticipativity means that we can observe the realization of the stochastic process Y at times until t if we can observe process X until time t. It does not mean that we can predict the process Y . We want to clearly distiguish between the term non-anticipativity and the term of a predictable stochastic process.

The process X= (Xn, n ∈ N0) is predictable with respect to the Filtration F = (Fn, n ∈ N0)

if X0is constant and it holds for all n∈ N that XnisFn−1measurable.

We continue with the central theorem of this section.

Theorem 2.5. The scenario tree process Y is non-anticipative with respect to the underlying process X .

Proof.

We recall from the definition of non-anticipativity that we have to show the following state-ment: The process Y is is adapted to the filtration σ (Xt,t ∈ I) of the underlying parameter

process X .

By definition of the stochastic tree process Y , each YtisE-measurable for all t ∈ {0, . . . , M}.

The sample space E of the tree process is a finite subset of Ω, therefore E ⊂ F . As both processes X and Y are defined for t ∈ {0, . . . M}, we deduce the statement that Yt is F

-measurable for all t∈ {0, . . . , M}.

We now focus on the filtrations Et := σ (Ys, 0 ≤ s ≤ t) for s,t ∈ {0 . . . M}. The random

variables Yt are measurable with respect to the filtrationEtfor all t∈ {0, . . . , M}. Because Yt

isF -measurable for all t ∈ {0, . . . , M}, we conclude that Ytis also measurable with respect

to the filtrationFt := σ (Xs, 0 ≤ s ≤ t) for all s,t ∈ {0, . . . , M}. Then by the definition of

adaptivity, Def. 2.15, the scenario tree process Y is adapted to σ (Xt,t ∈ {0, . . . , M}).

We close this section on the stochastic perspective with two examples illustrating the concept on non-anticipativity.

Example 2.1 (Anticipativity). Consider Ω = {A, B} and a discrete constant process X , with Xt(ω) = 10 for all t ∈ N, ω ∈ Ω, and the process Y with

Yt(ω) =    0 for ω = A 1 for ω = B for all t∈ N.

Because X is constant, the generated sigma-algebra of all random variables Xt is trivial:

σ (Xt) = { /0, Ω} for all t ∈ N. Hence, the filtrations Et = E := σ (Xt) stay constant for

all t _{∈ N. The generated sigma-algebra is σ(Y ) = { /0, {A}, {B}, Ω}, and the filtration is} F₁= F = σ (Y ).

(32)

For completeness we specify the probability measures and refer to Remark 2.2 that non-anticipativity is measure-independent. Process X is equipped with measure P₁on F such that P1(Xt = 10) = 1. For process Y we can define a measure P2onE such that

P2(ω) =    0.6 for ω = A 0.4 for ω = B.

The event(ω = B) is an element of the filtration F1, because it is possible to observe Y1= 1.

From the process X we can only observe that it has constant value 10. We do not know the elementary random event that is happening at a time t. Regarding non-anticipativity we state formally that the event(ω = B) is not contained in E, therefore Y1 is not measurable

with respect to E and Y is not adapted to σ (X1). Hence, the process Y on (Ω, E, P2) is

anticipative (not non-anticipative) with respect to the process X on(Ω, F , P1).

Example 2.2 (Events of the tree process). Let X be the uncertain parameter process from Def. 2.16 and Y the tree process from Def. 2.17. We now consider a few events to explain the non-anticipativity we have proven above.

In the following we need the filtrations of the underlying process X ,

Ft = σ (Xs, 0 ≤ s ≤ t), s,t ∈ {0, . . . , M}

and the filtrations of the tree process Y ,

Et = σ (Ys, 0 ≤ s ≤ t), s,t ∈ {0, . . . , M}.

If we look at the first stage, then the event(Y1= p1) is contained in E1⊂ F1. The events

(Y1= p1,Y2= p1) and (Y1= p1,Y2= p2), that represent scenarios of length two, are not in

F1. But the considered events(Y1= p1,Y2= p1) and (Y1= p1,Y2= p2) are elements of the

filtrationsE₂⊂ F₂. Therefore those two events cannot be distinguished until t= 2. Consid-ering the two events as two possible scenarios, we have to ensure later that they coincide until time t= 2. We shall reflect this property in the so-called non-anticipativity control constraints for the branch-wise scenario tree formulation to get scenario tree formulation that is consistent with the underlying stochastics.

Up to now, we have introduced non-anticipativity and illustrated the concept in our framework. There are two chapters in this thesis that especially rely on the stochastic nature of the scenario tree approach. In Chapter 6 we present an approach how to generate sce-nario trees based on quadrature rules and in Chapter 7 we develop an efficient tree reduction algorithm for a process with Markovian properties.

(33)

2.6. SUMMARY 21

2.6 Summary

The present chapter has introduced the mathematical formalism of the scenario tree ap-proach and classified the arising problems in different fields of applied mathematics. From a dynamical systems perspective we have formulated an optimal control problem with sce-nario tree structure. In the direct optimization approach the optimal control problem is discretized. We have seen from the optimization perspective that we arrive at a specific structured finite-dimensional constrained scenario tree optimization problem. The process control perspective section has emphasized that fast feedback NMPC methods like the Real-Time Iteration scheme and the Multi-Level-Iteration scheme are required to minimize the feedback delay. In the last section we have defined the scenario tree process from a stochas-tics perspective and introduced the principle of non-anticipativity. This chapter has set the perspectives and directions to approach scenario tree NMPC. In the following chapters we always start discussing the problem from a specific viewpoint to derive the contributions of this thesis.

(34)

(35)

Chapter 3

Discretization Structure Exploitation

Numerical algorithms exploiting an inherent problem structure are indispensable for an ef-ficient solution process. In case of optimal control, direct methods such as Direct Multiple Shooting [17] or Direct Collocation [109, 4, 11] have proven to be versatile and efficient whereas the initial value problems in Direct Single Shooting and the boundary value prob-lems resulting from indirect methods suffer from the fact that the trajectories of nonlinear ODE systems do not necessarily exist on a long time period. If they exist, the high non-linearity can cause a blowup of the solution trajectories such that boundary conditions can-not be satisfied. In this chapter we focus on the direct multiple shooting method, its structure and tailored methods to solve the resulting finite-dimensional optimization problems.

3.1 The Direct Multiple Shooting Method

The foundations of multiple shooting are improved solution methods for boundary value problems [89, 18, 12]. In the thesis [90] supervised by Hans Georg Bock and the contri-bution [17] the multiple shooting idea has been extended to optimal control problems. The Direct Multiple Shooting Method combines several advantages of other direct approaches. For instance, the state discretization allows to incorporate a priori knowledge of the process for initialization. The underlying dynamics can be computed by fast and also adaptive state-of-the art integrators because we solve initial value problems (IVPs). Moreover, the Direct Multiple Shooting method can cope with highly nonlinear dynamics because we solve IVPs on subintervals of the solution horizon and do not require an all-at-once solution over the whole horizon. For an explanation of the method let us state again the optimal control

(36)

24 CHAPTER 3. DISCRETIZATION STRUCTURE EXPLOITATION

problem (2.2) in the following. We assume that all functions are sufficiently smooth.

min x,u Z t_f t0 Φ(t, x(t), u(t), p(t))dt (3.1a) s.t. x(t) = f (x(t), u(t), p(t)),˙ t∈ [t₀,tf] , (3.1b) 0 = x(t0) − x0, (3.1c) 0 ≤ r(t, x(t), u(t), p(t)), t∈ [t₀,tf] . (3.1d)

The problem (3.1) is a constrained infinite-dimensional optimization problem on a time horizon I := [t0,tf] ⊂ R with state variables x : I → Rnxand control variables u : I → Rnu. The

dynamical evolution of the states is described by a system of ordinary diffential equations (ODEs) (3.1b) with right hand side f : Rnx_{× R}nu_{× R}np _{→ R}nx _{and initial value x}

0∈ Rnx.

For the ODE right hand side we assume continuity and Lipschitz continuity in x such that local solutions to IVPs are guaranteed by Theorem 2.1. We aim to minimize a performance criterion (3.1a) while satisfying the dynamical evolution constraints (3.1b) with initial value (3.1c) and path constraints (3.1d) with r : R × Rnx_{× R}nu_{× R}np _{→ R}nr_{. The system behavior} is affected by uncertain parameters p ∈ Rnp_{. The solution trajectories x and u have infinitely} many degrees of freedom. Hence, for numerical computations we must discretize the opti-mal control problem. After the discretization procedure we arrive at a structured NLP.

Discretization of the optimal control problem

First, the time horizon I is partitioned into a non-necessarily equidistant shooting grid {tk}

with

t0< t1< . . . < tM= tf.

On each interval Ik:= [tk,tk+1], 0 ≤ k ≤ M − 1 we use a control discretization

uk(t) = ψk(t, qk)

with basis functions ψk: Ik× Rnqk → Rnu. The basis functions have local support such that

we get separability of the discretized problem. A common and easy variant is the piecewise constant control parameterization. For every Ikwe choose ψk= qk with qk∈ Rnu. In other

control discretization cases we may require continuity of the control. It can be achieved by adding the constraints

ψk(tk+1, qk) − ψk+1(tk+1, qk+1) = 0 for k = 0, . . . , M − 1.

These constraints are linear and affect only neighboring controls. Therefore the separability of the discretized problem will not get lost by adding continuity constraints.

(37)

3.1. THE DIRECT MULTIPLE SHOOTING METHOD 25

We continue with the state discretization. The continuity assumptions on the ODE right hand side of the dynamical evolution constraint (3.1b) ensure the existence of a unique solution in the neighborhood of (t, x, u) ∈ I × Rnx_{× R}nu _{by Theorem 2.1. Therefore we} can write the ODE solution as function of initial value x0and control q, namely x(t; x0, q).

The fundamental idea of direct multiple shooting is the splitting of the ODE trajectory computation. We do not integrate the ODE on the whole interval I at once. Instead we divide the integration task into subproblems. We aim to compute IVPs on the intervals Ikfor k = 0, . . . , M − 1. Therefore we introduce the variables sk ∈ Rnx, k = 0, . . . , M and

solve local IVPs of the form

˙

x(t) = f (t, x(t), ψk(t, qk)) for t ∈ Ik (3.2a)

x(tk) = sk. (3.2b)

The solution of (3.2) is denoted by xk_{(t; s}

k, qk). Then we concatenate the IVP solutions on

the whole interval I to the function

x(t) =    xk(t; sk, qk) for t ∈ [tk,tk+1) sM for t = tM.

We have to ensure continuity of the concatenated function on I, because this is a fundamen-tal property of the ODE solution, which yields matching conditions of the form

xk(tk+1; sk, qk) − sk+1= 0 for j = 0, . . . , M − 1

as additional constraints. Now the evaluation of x(t) does not require an integration over I at once. Only integration over the subintervals Ik must be performed. The shorter

integra-tion intervals reduce the error propagaintegra-tion and yield a better convergence behavior of the multiple shooting method, especially compared to single shooting. In addition to that, the multiple shooting formulation allows a parallel integration of subintervals, yielding compu-tation time reduction on multicore systems.

Finally we discretize the continuous path constraints (3.1d) and require

r(tk, x(tk; sk, q), ψk(tk, qk)) ≥ 0 for k = 0, . . . , M − 1.

We define the discretized constraint function rk: R × Rnx× Rnu→ Rnr with

rk(tk, sk, qk) := r(tk, x(tk; sk, q), ψk(tk, qk)) ≥ 0 for k = 0, . . . , M.

Alternatives to treat the path constraints are described in [93].

At this point we state the discretized optimal control problem that is a nonlinear opti-mization problem as in Definition 2.3. The structured nonlinear direct multiple shooting optimization problem then reads

(38)

26 CHAPTER 3. DISCRETIZATION STRUCTURE EXPLOITATION min s,q φ (s, q) := M−1

∑

k=0 Z t_k+1 tk Φ(t, xk(t; sk, qk), ψk(t, qk))dt (3.3a) s.t. 0 = xk(tk+1; sk, qk) − sk+1 for k = 0, . . . , M − 1 (3.3b) 0 = x(t0) − x0 (3.3c) 0 ≤ rk(tk, sk, qk) for k = 0, . . . , M. (3.3d)

The inherent multiple shooting structure of the problem (3.3) can be exploited efficiently due to the separability of the objective function (3.3a) and the constraints with respect to the optimization variables s and q. Only the matching conditions (3.3b) couple unknowns of neighboring shooting nodes linearly. This is the reason why the Hessian of the Lagrangian of the NLP (3.3) exhibits block-diagonal structure.

time state / control t₀ t₁ t₂ t_M−1 tM q₀ q1 q₂ s0 s1 s2 sM−1 sM time state / control t₀ t₁ t₂ t_M−1 tM q0 q1 q2 s0 s1 s2 sM−1 sM

Figure 3.1: Illustration of state and control variables of the direct multiple shooting dis-cretization applied to an optimal control problem. On the left the shooting nodes are initial-ized linearly and the trajectory violates the matching conditions. On the right the solution of the NLP has converged with feasible matching conditions. Inspired by [65, 41, 101].

We conclude with the remark that the multiple shooting method only solves the dis-crete approximation (3.3) of the continuous problem (3.1). Investigations on approximation properties and asymptotic behavior can be found in [45].

(39)

3.2. STRUCTURE EXPLOITING SEQUENTIAL QUADRATIC PROGRAMMING 27

Control Move Regularization

At this point we introduce the Control Move Regularization (CMR). We formulate the CMR as an addition to the NLP (3.3). The NLP with CMR reads

min s,q φ (s, q) (3.4a) s.t. 0 = xk(tk+1; sk, qk) − sk+1 for k = 0, . . . , M − 1 (3.4b) 0 = x(t0) − x0 (3.4c) 0 ≤ rk(tk, sk, qk) for k = 0, . . . , M, (3.4d) 0 ≤ αCMR− (qk+1− qk) for k = 0, . . . , M − 1, (3.4e) 0 ≤ αCMR− (qk− qk+1) for k = 0, . . . , M − 1. (3.4f)

The constraints (3.4e) and (3.4f) state that the differences of subsequent control variables are less than or equal to the CMR parameter αCMR∈ Rnu. All components of αCMR are

strictly positive. It is also possible to add an objective term φCMR(q) to (3.4a) penalizing

the difference of control variables. For our numerical results in Chapter 9 it is sufficient to use the CMR parameter αCMR. Engineers use CMR extensively as a means to tune

NMPC controllers. For instance, let the coolant temperature to control a reactor be 80◦C. Assuming that there is a fine sampling grid, it is impossible to apply 60◦C at the next sampling point to the plant. Therefore the constraint bounding the difference of subsequent coolant temperature variables is added to the discretized prediction problem in NMPC. We remark that the CMR couples neighboring stages only linearly and that the CMR parameter depends on the discretization grid.

3.2 Structure Exploiting Sequential Quadratic Programming

We employ Sequential Quadratic Programming (SQP) techniques [88] and solve the con-strained NLP (3.3) with multiple shooting structure as described in [77, 78]. For notational convenience we write the NLP (3.3) in the more generic form

min

z φ (z) (3.5a)

s.t. 0 = d(z) + Λx0 | λ (3.5b)

0 ≤ h(z) + h | µ (3.5c)

with z = (s0, q0, . . . , sM−1, qM−1, sM) and Λ = (I, 0, 0, . . .) ∈ Rnx×(Mnx+(M−1)nq). The line

(3.5b) comprises the initial value and dynamical evolution constraints. The line (3.5c) com-prises all further constraints. Furthermore, the symbols | λ and | µ on the right represent

(40)

the dual variables corresponding to the constraints. The concept is introduced in Definition (2.6) of the Lagrangian function. We emphasize that (3.5) is parametric in x0.

The standard full-step SQP technique is a Newton-type iterative method. We start with an initial guess of primal and dual NLP variables (z0, λ0, µ0). Within every SQP iteration

we compute the primal-dual solution of the structured QP

min ∆zi 1 2∆z > i Bi∆zi+ b>i ∆zi (3.6a) s.t. 0 = Di∆zi+ d(zi) + Λx0 | λQP (3.6b) 0 ≤ Hi∆zi+ h(zi) + h | µQP (3.6c)

to obtain (∆zi, λQP, µQP). The QP (3.6) is a local quadratic model of the multiple shooting

NLP (3.5) at zi. The matrix Bi denotes an approximation of the Hessian of the Lagrangian

of the NLP (3.5), bi is the objective gradient, Di is the Jacobian of the function d and Hi is

the Jacobian of the function h.

A full step SQP iteration is performed by updating the variables in every iteration ac-cording to

zi+1= zi+ ∆zi, λi+1= λQP, µi+1= µQP.

We use the principle of Internal Numerical Differentiation (IND) to solve the IVPs and to compute the sensitivities. Regarding the whole topic of numerical integration and sensitivity calculation we refer to [13, 2, 1, 7]. For SQP variants, especially the choice of Bi, see

[62, 63]. For globalization strategies such as line search SQP or trust-region SQP methods we refer to the textbook [88].

Various structural features as the separable Lagrangian, the block diagonal Hessian, and the block structure of the Jacobians of the matching conditions can be extensively exploited if the control functions, constraints, and multiple shooting variables are discretized on a common grid.

3.3 Condensing

In this section we focus on the multiple shooting structure exploiting method called Con-densing. It has been described already in [14, 17, 90, 76] and yields small dense QPs. Solving the tree QP benefits from Condensing because our branchwise formulation pre-serves the multiple shooting structure. The basic idea of Condensing is a splitting of the optimization variables z ∈ Rnz _{into (z}

1, z2) ∈ Rn1+n2 and a structure exploiting elimination

of z2. After Condensing the resulting QP is of much smaller size if n2dominates n1and can

be solved with a state-of-the-art dense QP solver. At first we present the interpretation of Condensing as a partial nullspace approach as described in [92].

(41)

3.3. CONDENSING 29

We consider a structured QP in the optimization variables z = (z1, z2) ∈ Rn1+n2 of the

form min (z1,z2)∈Rn1+n2 1 2 " z1 z₂ #>" B11 B12 B₂₁ B₂₂ # " z1 z₂ # + " b1 b₂ #>" z1 z₂ # (3.7a) s.t. G1z1+ G2z2= g, (3.7b) D1z1+ D2z2= d, (3.7c) H₁z₁+ H2z2≥ h, (3.7d)

with matrices G1∈ Rn1×n1, G2∈ Rn1×n2, matrices D1∈ Rm2×n1, D2∈ Rm2×n2 and matrices

H1∈ Rm3×n1, H2∈ Rm3×n2.

QP (3.7) is formulated as in [92, 101]. The following theorem from [92, 101] describes how to eliminate variables z1that are coupled with variables z2by the equality constraints

(3.7b). It is known as partial nullspace approach and has been discussed in similar form in [8].

Theorem 3.1. Let G1from QP(3.7) be invertible. We introduce the notation

B= " B11 B12 B₂₁ B₂₂ # , b= " b1 b₂ # , Z= " −G−1₁ G2 I # , B0= Z>BZ, g0= G−1₁ g, b0= B21g0+ b2− G>2G −> 1 (B11g0+ b1), d0= d − D1g0, D0= D2− D1G−11 G2, h0= h − H1g0, H0= H2− H1G−11 G2.

Furthermore let(z∗₂, λ₂∗, µ∗) ∈ Rn2+m2+m3 _{be a solution of the QP} min z2∈Rn2 1 2z > 2B 0 z2+ b0>z2 (3.8) s.t. D0z₂= d0, H0z2≥ h0. Then(z∗, λ∗, µ∗) := (z∗₁, z∗₂, λ₁∗, λ₂∗, µ∗) with z∗₁= G−1₁ (g − G2z∗2) and (3.9a) λ₁∗= G−>₁ ((B12− B11G−1₁ G2)z∗2+ B11g0+ b1− D>1λ ∗ 2− H > 1 µ ∗₎ (3.9b) is a solution of the QP(3.7).

(42)

Theorem 3.1 does not specify how to split the optimization variables z = (z1, z2). This

allows for tailoring the approach to the specific QP structure. We only have to ensure that G1is well-conditioned.

In our case the QP originates from the direct multiple shooting discretization and a stan-dard SQP method. We apply Theorem 3.1 to QP (3.6) and focus on the constraint (3.6b). First, we drop ∆ and i from the SQP formulation. We call the matrix representing the Jaco-bian of the matching conditions in this section G. It exhibits block-sparse structure,

G=        G0_s G0_q −I G1_s G1_q _−I . .. ... . .. GM−1_s GM−1_q _−I        . (3.10)

The matrix blocks Gk_s,q represent the sensitivities of the IVP solutions on the kth shooting interval for k = 0, . . . , M − 1, Gk_s= ∂ ∂ sx k (tk+1; sk, qk), Gkq= ∂ ∂ qx k (tk+1; sk, qk). (3.11)

In the following we discuss two established Condensing variants.

Block Gauss Condensing

The first variant is named Block Gauss Condensing. We arrange the variables z = (z1, z2)

as follows

z1= (s1, . . . , sM) ,

z2= (s0, q0, q1, . . . , qM−1) .

For the critical constraint matrix G the first variant yields a permutation of its columns. We arrive at the structure

G₁=        −I G1_s . .. . .. _−I GM−1_s _−I        and G₂=        G0_s G0_q G1_q . .. GM−1_q        .

The matrix G1is lower-triangular with negative identity matrices on the diagonal.

There-fore G1is invertible and we can apply Theorem 3.1. As we know the structure of G1, we

can exploit it for the matrix-vector operations with G−1₁ . Of course we do not explicitly invert G1 but perform Block Gauss eliminations that are eponymous for the Condensing

variant. This variant has good stability properties for stable systems yielding kGisk2< 1

for all i ∈ {0, . . . , M − 1}. In some applications Block Gauss Condensing has shown poor stability properties, an example can be found in [34].

Numerical Methods for Scenario Tree Nonlinear Model Predictive Control

Numerical Methods for

Scenario Tree Nonlinear Model Predictive Control

Prof. Dr. Dr. h. c. mult. Hans Georg Bock

Prof. Dr. Ekaterina Kostina

Danksagung

Abstract

Zusammenfassung

Contents

Chapter 1

Introduction

1.1

Aims and Contributions of this Thesis

1.2

Organization of this Thesis

Chapter 2

Problem Classification

2.1

The Scenario Tree Optimization Problem

∑

2.2

Dynamical System Perspective: Optimal Control

2.3

Optimization Perspective: Nonlinear Programming

∑

∑

∑

∑

∑

∑

∑

2.4

Process Control Perspective: Fast Feedback in Real-Time

2.5

Stochastic Perspective: Discrete Tree Process

∑

2.6

Summary

Chapter 3

Discretization Structure Exploitation

3.1

The Direct Multiple Shooting Method

∑

3.2

Structure Exploiting Sequential Quadratic Programming

3.3

Condensing

_∑