Probability Calculation in Subset Simulation based Chance-constrained

First of all, it should be noted that there are two different implemented strategies to calculate the probability of the rare-event failure in the ROCP of (6.1) by SubSim (de-sired fulfillment probability: ξ): Within the NLP iteration the sampling-based approach (Subsection 6.1.1 using Algorithm 6.2) using the sigmoid (Subsection 6.1.2) is applied.

Here, the sigmoid is used to evaluate the failure probability by (2.43) or (2.47). Take into account that the fulfillment probability of this CC, ξ_OCP, can be updated while solving the CC-OCP with SubSim (Steps 11–17 in Algorithm 6.2). This might be necessary if the initial guess does not allow the NLP iteration to converge and a relaxed NLP should therefore be solved initially. It is further reminded here that the formulation in (2.47) is generally preferred because it gives a continuous representation of the failure probability (due to the Beta PDF approximation), while it also provides the possibility to calculate the standard deviation, which can be used to robustify the solution by CIs and calculate the coefficient of variation (CoV). Thus, (2.47) is applied in the following derivations and the examples of this thesis.

For the probability calculation, e.g., in the ROCP, it is further important to note that the failure probability in both (2.43) and (2.47) is calculated for being inside the failure domain F . As also already stated, it is generally preferred to calculate the probability to not be in the failure domain, due to the numeric scaling of the NLP, which has the simple relation P [y (z; θ) /∈ F ] = 1 − P [y (z; θ) ∈ F]. Thus, it is easy to change between the two descriptions. In Algorithm 6.2, the calculated probability at the optimal point of the NLP is further on denoted by POCP[y (z; θ) /∈ F ].

6.2 Subset Simulation based Chance-constrained Optimal Control

Furthermore, the SubSim in the homotopy step (Step 18 in Algorithm 6.2) is based on a full simulation of the model (e.g., by a standard Runge-Kutta scheme [16, p. 145f.]

or a hybrid Matrix Laboratory ^R (MATLAB ^R) differential equation integrator¹). Here, the robust control history from the previous optimization result is used for the simulation.

The results can then be evaluated using the indicator function to have an independent formulation of the ROCP, which is based on the sigmoid. By this, it is secured that the calculated control history does also fulfill the rare-event CC probability without the er-rors introduced by e.g., using the gPC expansion for the sampling, the tolerances of the NLP algorithm (i.e., the optimality and feasibility tolerances), as well as the sigmoid approximation. Additionally, the errors occurring by taking the random samples from the previous SubSim run are also mitigated. The corresponding probability calculated by the SubSim in the homotopy is further on denoted by PSubSim[y (z; θ) /∈ F ] (see Step 18 of Algorithm 6.2).

It should be further noted that the random samples, created in Step 18 of Algorithm 6.2, as well as the corresponding number of SubSim levels n_ssfor these samples, are directly used in the NLP algorithm to evaluate the CC and remain constant during the NLP solution process. As mentioned, this moves the stochasticity of the SubSim to the homotopy step.

As also already stated, the basic procedure of the SubSim within CC-OC, and especially the two step SubSim evaluation procedure, once within the NLP using the gPC expansion with constant level samples and the other time within the homotopy step using a simulation with the calculated robust control history, is illustrated in Algorithm 6.2 as well: Here, it is clear that the CC-OCP is only considered to be solved if both the CC within the NLP as well as the SubSim homotopy step fulfill the desired CC probability level Pdes[y (z; θ) /∈ F ] = ξ (Step 7 in Algorithm 6.2).

It is important to note that the procedure of assigning constant samples as well as using the number of SubSim from the homotopy step (Steps 9 and 19 in Algorithm 6.2), makes it straightforward to calculate the Jacobian and Hessian of the failure probability estimate from the Beta PDF in (2.46) and (2.47). This is based on the fact that the derivative must only be taken with respect to the expansion coefficients. To illustrate this, the parameters c₁ and c₂, which are used for the calculation of the Beta PDF shape and rate parameter (see (2.46)), are looked at:

c₁ = (p₀· n_s+ 1)^nss (n_s+ 2)^nss

| {z }

const.

·

"_ns X

i=1

Ie^hy⁽ⁱ⁾_nss^z; θ⁽ⁱ⁾_nss^i+ 1

#

c₂ = (p₀· n_s+ 2)^nss (n_s+ 3)^nss

| {z }

const.

·

"_ns X

i=1

Ie^hy⁽ⁱ⁾_nss^z; θ⁽ⁱ⁾_nss^i+ 2

# (6.17)

1https://mathworks.com/help/matlab/math/choose-an-ode-solver.html (Retrieved April 23, 2019)

148

Chapter 6: Chance-constrained Optimal Control with Polynomial Chaos

Here, the indicator function is approximated by the sigmoid representation within the NLP for smoothness reasons (it should be reminded that the sigmoid in (6.9) is defined for the non-failure domain and thus, the sigmoid must be subtracted from one to get the failure domain probability):

Then, the derivative with respect to the expansion coefficients is given by the chain rule combining (6.3), (6.10), and (6.13) as follows:

∂c1

Thus, the necessary Jacobian of the probability for the NLP algorithm using (6.19) can be calculated.

Take into account that the Hessian can be derived in the same manner as (6.19) using (6.14). Additionally, the chain rule for the product of derivatives in (6.19) as well as the fact that the derivative of (6.3) with respect to the expansion coefficients is zero must be considered. Then, the Hessian is given as follows:

∂²c₁

Then, (6.19) and (6.20) provide the Jacobian and Hessian for the coefficients used to calculate the shape and rate parameter of the Beta PDF respectively, which is used to estimate the failure probability. Thus, the Jacobian and Hessian calculation must be ex-tended for the NLP to the e.g., mean and standard deviation value in (2.47) applying (2.46)

6.2 Subset Simulation based Chance-constrained Optimal Control

(using the coefficients in (6.19) and (6.20)). These results can directly be calculated using e.g., symbolic derivations and the chain rule and are thus, left out here for the sake of compactness.

To conclude the SubSim-based CC-OC framework, the following two paragraphs discuss the convergence as well the choice of SubSim parameters for the developed algorithm:

Convergence Discussion of Proposed Algorithm

As a general statement, it can be said that the proposed SubSim homotopy strategy is viable for the use within NLPs as especially the Jacobian and Hessian can be provided efficiently. On the other hand, it should be noted that by using the SubSim samples calculated from the previous optimal solution within the new optimization, a bias might be introduced as the samples drawn from the Markov chain are based on the optimal result created by the previous NLP solution. Generally, the samples would have to adapted in each iteration of the NLP, as the system response changes and therefore, the MMHA might create different results. As this is not done within the NLP, but within the homotopy step after a new optimal solution has been calculated, the CC and its rare-event probability is solved using “non-ideal” samples compared to the ones that would be calculated within the SubSim (still, the used samples could be a possible outcome set of the random MMHA within the SubSim nonetheless). This issue is coped with in this thesis by checking the fulfillment of the CC probability both in the CC-OCP as well as after the CC-OCP is solved by the SubSim in the homotopy step, i.e., with the new response surface. Thus, the CC-OCP is only considered to be solved if both results show that the CC is fulfilled to the desired probability level. As stated, within this thesis, the CC-OCP converges fine, but further studies can explore the effects and influences of this bias and how to reduce it (e.g., by introducing importance sampling techniques [81, p. 23ff.]).

Additionally, sensitivity analysis techniques could be used to update the samples based on the difference to the previous optimal solution, i.e., the change in the response surface (this would require “inverse” sensitivities, i.e., parameter updates based on the decision variable changes rather than decision variable updates based on the sensitive parameters changes). Still, the application cases of this thesis show that the proposed algorithm converges well even omitting these issues. As mentioned, this is due to the two-step procedure and independent SubSim evaluation in the NLP and the homotopy to assure a feasible solution.

Choice of Subset Simulation Proposal Distribution

A further important aspect of the SubSim, and especially the used MMHA within it, is the choice of the proposal PDF around the current samples in Step 0 of Algorithm 2.2.

Here, generally the choice of a proposal PDF is difficult, because the required “spread” is largely depending on the original PDF of the samples and the weighting of acceptance rate and spatial dependence. Using the gPC method, this choice of a proposal PDF becomes

150

Chapter 6: Chance-constrained Optimal Control with Polynomial Chaos

more straightforward: This is due to the fact that the samples are created in a way that they can be evaluated in the orthogonal polynomial domain. This consequently means that the samples are always defined such that they are in their “standard” domains of gPC expansion as defined in Table 2.2. Thus, it is not necessary to adapt the proposal PDF choice for different parameter settings, but instead it is possible to define a standard relation between the parameter’s PDF and the proposal PDF. As stated in [9, p. 123], a safe strategy to choose the standard deviation is the use of the standard deviation of the target distribution. Thus, for an uncertain parameter with a Gaussian PDF, the proposal PDF can also be chosen as the standard Gaussian PDF. By this, a good exploration of the failure region can be assured. Furthermore, for a Uniform PDF, the exploration can be done with the standard Uniform spread around the current sample and be clipped at the standard domain bounds accordingly. This has proven to be efficient as well. Therefore, the gPC algorithm actually has some benefits for SubSim, due to its standard definition in terms of the PDFs.

Concluding, both introduced CC-OC framework, with MCA-based sampling for “fre-quent” events (Section 6.1) as well as the SubSim-based sampling for rare-events, are tested in Chapter 9 to show their applicability in the OC context.

Chapter 7

Design of Robust Gains for Control Loop

This chapter covers the first application example of the developed robust open-loop di-rect optimal control (ROC) frameworks of this thesis. Within this chapter, the bi-level framework, introduced in Chapter 3, is applied to calculate robust controller gains and thus, results for Contribution 2 are shown. An adaptive control loop is considered to show the applicability of the methodology to calculate robust, optimal adaptation gains.

Generally, the calculation is done by at first applying the framework in Section 3.1, i.e., with the differential evolution algorithm (DEA) in the upper level, to restrict the domain of the (globally) optimal gains. Afterward, the framework from Section 3.2 is used, in the domain around the optimal gains from the DEA result, to find the numerically exact optimum. This procedure is applied to be more certain that the calculated optimal gains are globally optimal.

To show the procedure and the results, the chapter is organized as follows: In Sec-tion 7.1, the dynamic model for the control loop is introduced. Then, SecSec-tion 7.2 depicts the application of the bi-level framework to the calculation of the adaptation gain in an adaptive control loop with Section 7.3 illustrating the results. Here, especially proba-bilistic constraints, i.e., chance constraints (CCs), are applied to achieve a robustification.

7.1 Dynamic Model for Gain Design

In this example, a short period approximation of an F-16 is used to show the robust gain design based on the bi-level generalized polynomial chaos (gPC) method (Chapter 3;

after [122, p. 259] and [59, p. 29f.]):