Stochastic Gradient Method: Applications

(1)

Two Elementary Exercices on the Stochastic Gradient Option Pricing Problem and Variance Reduction Optimal Control Under Probability Constraint

Stochastic Gradient Method: Applications

February 03, 2015

(2)

Lecture Outline

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

P. Carpentier Master MMMEF — Cours MNOS 2014-2015 115 / 267

(3)

Two-Stage Recourse Problem

Trade-off Between Investment and Operation

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(4)

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(5)

A Basic Two-Stage Recourse Problem

We consider the management of a water reservoir. Water is drawn from the reservoir by way of random consumers. In order to ensure the reservoir supply, 2 decisions are taken at successive time steps.

A first supply decision q

1

is taken without any knowledge of the effective consumption, the associated cost being equal to

1 2

c

1

q

1

2

, with c

1

> 0.

Once the consumption d has been observed (realization of a r.v. D defined over a probability space (Ω, A, P)), a second supply decision q

2

is taken in order to maintain the reservoir at its initial level, that is, q

₂

= d − q

₁

, the cost associated to this second decision being equal to

¹₂

c

2

q

2

, with c

2

> c

1

> 0.

The problem is to minimize the expected cost of operation.

(6)

Mathematical Formulation and Solution

Problem Formulation

q

1

is a deterministic decision variable,

whereas q

₂

is the realization of a random variable Q

₂

.

min

(q1,Q2)

1 2 c

₁

q

₁

2

+ 1 2 E

c

₂

Q

₂

2

s.t. q

₁

+ Q

₂

= D .

Equivalent Problem

q

min

1∈R

1 2 E

c

1

q

1

2

+ c

2

D − q

₁

2

Analytical solution: q

₁^]

= c

₂

c

₁

+ c

₂

E D .

(7)

Stochastic Gradient Algorithm

Q

₁^(k+1)

= Q

₁^(k)

− 1 k

(c

₁

+ c

₂

)Q

₁^(k)

− c

₂

D

^(k+1)

.

Algorithm (initialization)

//

// Random generator //

rand(’normal’); rand(’seed’,123);

//

// Random consumption //

moy = 10.; ect = 5.;

//

// Criterion //

c1 = 3.; c2 = 1.;

//

// Initialization //

x = [ ]; y = [ ];

Algorithm (iterations)

//

// Algorithm //

qk = 0.;

for k = 1:100

dk = moy + (ect*rand(1));

gk = ((c1+c2)*qk) - (c2*dk);

ek = 1/k;

qk = qk - (ek*gk);

x = [x ; k]; y = [y ; qk];

end //

// Trajectory plot //

plot2d(x,y);

xtitle(’Stochastic Gradient ’,’Iter.’,’Q1’);

(8)

A Realization of the Algorithm

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

0 10 20 30 40 50 60 70 80 90 100

Stochastic Gradient Algorithm

Iter.

Q1

(9)

More Realizations. . .

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

0 50 100 150 200 250 300 350 400 450 500

Stochastic Gradient Algorithm

Q1

(10)

Slight Modification of the problem

As in the basic two-stage recourse problem,

a first supply decision q

1

is taken without any knowledge of the effective consumption, the associated cost being equal to

1

2

c

₁

q

₁

2

,

a second supply decision q

2

is taken once the consumption d has been observed (realization of a r.v. D ), the cost of this second decision being equal to

¹₂

c

2

q

2

.

The difference between supply and demand is penalized thanks to an additional cost term

¹₂

c

3

q

1

+ q

2

− d

2

. The new problem is : min

(q1,Q2)

1 2 E

c

1

q

1

2

+ c

2

Q

₂

2

+ c

3

q

1

+ Q

₂

− D

2

. Question: how to solve it using a stochastic gradient algorithm?

(11)

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(12)

Trade-off Between Investment and Operation (1)

A company owns N production units and has to meet a given (non stochastic) demand d .

For each unit i , the decision maker first takes an investment decision u

_i

∈ R , the associated cost being I

_i

(u

_i

).

Then a discrete disturbance w

_i

∈ {w

_{i ,a}

, w

_{i ,b}

, w

_{i ,c}

} occurs.

Knowing all noises, the decision maker selects for each unit i an operating point v

_i

∈ R , which leads to a cost c

_i

(v

_i

, w

_i

) and a production e

_i

(v

_i

, w

_i

).

The goal is to minimize the overall expected cost, subject to the following constraints:

investment limitation: Θ(u

₁

, . . . , u

_N

) ≤ 0, operating limitation: v

_i

≤ ϕ

_i

(u

_i

) , i = 1 . . . , N, demand satisfaction: P

_N

i =1

e

_i

(v

_i

, w

_i

) − d = 0.

(13)

Trade-off Between Investment and Operation (2)

Questions.

1

Write down the optimization problem. Is it possible to apply the stochastic gradient algorithm in a straightforward manner?

2

Extract the optimization subproblem obtained when both the investment u = (u

1

, . . . , u

_N

) and the noise w = (w

1

, . . . , w

_N

) are fixed. The value of this subproblem is denoted f

^]

(u, w ).

Discuss the resolution of this subproblem.

Give assumptions in order to have a “nice” function f

^]

. Compute the partial derivatives of f

^]

w.r.t. u.

3

Reformulate the initial optimization problem using this new function f

^]

and apply the stochastic gradient algorithm in the two following cases:

the investment limitation reduces to u

i

∈ [u

_i

, u

i

], i = 1, . . . , N ,

the investment limitation has the form Θ(u

1

, . . . , u

N

) ≤ 0.

(14)

Financial Problem Modeling Computing Efficiently the Price Two Algorithms

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(15)

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(16)

The Financial Problem

The price of an option with payoff ψ(S

_t

, 0 ≤ t ≤ T ) is given by P = E

e−rT ψ(S

_t

, 0 ≤ t ≤ T ) ,

where the dynamics of the underlying n-dimensional asset S is described by the following stochastic differential equation

dS

_t

= S

_t

r dt + σ(t, S

_t

)dW

_t

, S

₀

= x ,

r being the interest rate and σ(t, y ) being the volatility function.

(17)

Discretization

Most of the time, the exact solution is not available. To overcome the difficulty, one considers a discretized approximation of S (by using an Euler’s scheme), so that the price P is approximated by

P = E b

e−rT ψ( b S

_t

1

, . . . , b S

_t

d

) .

In such cases, the discretized function can be expressed in terms of the Brownian increments, or equivalently using a random normal vector. A compact form for the discretized price is

P = E φ(G ) b ,

where G is a n × d -dimensional Gaussian vector with identity

covariance matrix and zero-mean.

(18)

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(19)

A Parameterized Change of Variable

The problem is now to compute P = E φ(G ) b

using Monte Carlo simulations. By a change of variables, we obtain that

P = E b

φ(G + θ)e

^{−hθ ,G i−}^kθk2²

,

for any θ ∈ R

^n×d

. Let us denote by V (θ) b the associated variance:

V (θ) = E b

φ(G + θ)

²

e

−2hθ ,G i−kθk²

− E φ(G )

2

,

= E

φ(G )

²

e

^{−hθ ,G i+}^kθk2²

− E

φ(G )

2

.

The last expression shows that function V b is strictly convex and differentiable without any specific assumptions on φ. Moreover,

∇ b V (θ) = E

(θ − G )φ(G )

²

e

^{−hθ ,G i+}^kθk2²

,

so that the gradient of V b does not involves any derivative of φ.

(20)

Variance Minimization

The goal is to compute the parameter θ such that the variance V (θ) b related to P b is as small as possible, in order to optimize the convergence speed of the Monte Carlo simulations:

min

θ∈R^d

E

φ(G )

²

e

^{−hθ ,G i+}^kθk2²

.

This optimization problem is well suited for being solved by the stochastic gradient algorithm:

θ

^(k+1)

= θ

^(k)

−

^(k)

θ

^(k)

−G

^(k+1)

φ G

^(k+1)

2

e

−hθ(k) ,G (k+1)i+^kθ(k)_k2

2

,

and its unique solution is denoted θ

^]

.

(21)

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(22)

Non adaptive Algorithm

1

Using a N-sample of G , obtain an approximation θ

^(N)

of θ

^]

by N iterations of the stochastic gradient algorithm.

2

Once θ

^(N)

has been obtained, use the standard Monte Carlo method to compute an approximation of the price P b by using another N-sample of G :

P b

^(N)

= 1 N

N

X

k=1

φ(G

^(N+k)

+ θ

^(N)

)e

−hθ(N) ,G (N+k)i−^kθ(N)_k2

2

.

This algorithm requires 2N evaluations of the function φ, whereas a crude Monte Carlo method evaluates φ only N times. The non adaptive algorithm is efficient as soon as V (θ b

^]

) ≤ b V (0)/2.

(23)

Adaptive Algorithm

Combine the 2 previous algorithms, and compute simultaneously approximations of θ

^]

and P b by using the same N-sample of G :

¹⁰

θ

^(k+1)

= θ

^(k)

−

^(k)

θ

^(k)

− G

^(k+1)

φ G

^(k+1)

2

e

−hθ(k) ,G (k+1)i+^kθ(k)_k2

2

,

P b

^(k+1)

= b P

^(k)

− 1 k + 1

P

^(k)

− φ(G

^(k+1)

+ θ

^(k)

) e

−hθ(k) ,G (k+1)i−^kθ(k)_k2

2

.

A Central Limit Theorem is available for this algorithm:

√ N

P b

^(N)

− b P

_D

−→ N

0, b V (θ

^]

)

.

10

the last relation being the recursive form of:

Pb^(N)= 1 N

N

X

k=1

φ(G^(k+1)+ θ^(k)) e−hθ(k) ,G (k+1)i−^kθ (k)_k2

2 .

(24)

Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(25)

Numerical Results

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(26)

Numerical Results

Mission to Mars

(27)

Numerical Results

Satellite Model

dr

dt = v , dv

dt = −µ r kr k

³

+ F

m κ , (8a)

dm

dt = − T

g

₀

I

_sp

δ . (8b)

(8a): 6-dimensional state vector (position r and velocity v).

(8b): 1-dimensional state vector (mass m including fuel).

κ involves the direction cosines of the thrust and the on-off switch δ of the engine (3 controls), and µ, F , T , g

0

, I

sp

are constants.

The deterministic control problem is to drive the satellite from the

initial condition at t

i

to a known final position r

_f

and velocity v

_f

at

t

_f

(given) while minimizing fuel consumption m(t

_i

) − m(t

_f

).

(28)

Numerical Results

Deterministic Optimization Problem

Using equinoctial coordinates for the position and velocity

; state vector x ∈ R

⁷

,

and cartesian coordinates for the thrust of the engine

; control vector u ∈ R

³

,

the deterministic optimization problem writes as follows:

min

u(·)

K x (t

_f

) subject to:

x (t

_i

) = x

_i

, x (t) = f x (t), u(t) ,

^•

ku(t)k ≤ 1 ∀t ∈ [t

_i

, t

_f

] ,

C x (t

_f

) = 0 .

(29)

Numerical Results

Engine Failure

Sometimes, the engine may fail to work when needed: the satellite drifts away from the deterministic optimal trajectory.

After the engine control is recovered, it is not always possible to drive the satellite to the final target at t

_f

.

By anticipating such possible failures and by modifying the trajectory followed before any such failure occurs, one may increase the possibility of eventually reaching the target.

But such a deviation from the deterministic optimal trajectory results in a deterioration of the economic performance.

The problem is thus to balance the increased probability of

eventually reaching the target despite possible failures against

the expected economic performance, that is, to quantify the

price of safety one is ready to pay for.

(30)

Numerical Results

Stochastic Formulation (1)

A failure is modeled using two random variables:

t

_p

: random initial time of the failure, t

d

: random duration of the failure.

For any realization (t

_p^ξ

, t

_d^ξ

) of a failure:

1

u(·) denotes the control used prior to the failure

; u is defined over [t

_i

, t

_f

] but implemented over [t

_i

, t

p^ξ

] and corresponds to an open-loop control,

2

the control during the failure is 0 over [t

p^ξ

, t

p^ξ

+ t

_d^ξ

],

3

v

^ξ

(·) denotes the control used after the failure

; v

^ξ

is defined over [t

p^ξ

+ t

_d^ξ

, t

_f

] (if nonempty) and corresponds to a closed-loop strategy V.

The satellite dynamics in the stochastic formulation writes:

x

^ξ

(t

_i

) = x

_i

, x

^•^ξ

(t) = f

^ξ

x

^ξ

(t), u(t), v

^ξ

(t) .

(31)

Numerical Results

Stochastic Formulation (2)

The problem is to minimize the expected cost (fuel consumption) w.r.t. the open-loop control u and the closed-loop strategy V, the probability to hit the target at t

f

being at least equal to p.

min

u(·)

min

V(·)

E

K x

^ξ

(t

_f

)

min

u(·)

min

V(·)

E

K x

^ξ

(t

_f

)

C x

^ξ

(t

_f

) = 0 subject to:

x

^ξ

(t

_i

) = x

_i

, x

^•^ξ

(t) = f

^ξ

x

^ξ

(t), u(t), v

^ξ

(t) ,

ku(t)k ≤ 1 ∀t ∈ [t

_i

, t

f

] , kv

^ξ

(t)k ≤ 1 ∀t ∈ [t

_p^ξ

+ t

_d^ξ

, t

f

] , P

C x

^ξ

(t

_f

) = 0

≥ p .

(32)

Numerical Results

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(33)

Numerical Results

Indicator Function

Consider the real-valued indicator function:

I(y ) =

( 1 if y = 0, 0 otherwise.

Then

P

C x

^ξ

(t

_f

) = 0

= E I

C x

^ξ

(t

_f

)

,

and

E

K x

^ξ

(t

_f

)

C x

^ξ

(t

_f

) = 0

= E

K x

^ξ

(t

_f

) × I

C x

^ξ

(t

_f

)

E

I

C x

^ξ

(t

_f

)

.

(34)

Numerical Results

Problem Reformulation

The problem is (shortly) reformulated as

min

u(·)

min

V(·)

E

K x

^ξ

(t

f

) × I

C x

^ξ

(t

f

)

E

I

C x

^ξ

(t

f

)

s.t. E

I

C x

^ξ

(t

_f

)

≥ p .

Such a formulation is however not well-suited for a numerical implementation (e.g. Arrow-Hurwicz algorithm), because

a ratio of expectations is not an expectation!

(35)

Numerical Results

An Useful Lemma

Using compact notation, the previous problem writes:

min

u

J(u)

Θ(u) s.t. Θ(u) ≥ p , (9)

in which J and Θ assume positive values.

1

If u

^]

is a solution of (9) and if Θ(u

^]

) = p, then u

^]

is also a solution of

min

u

J(u) s.t. Θ(u) ≥ p . (10)

2

Conversely, if u

^]

is a solution of (10), and if an optimal Kuhn-Tucker multiplier β

^]

satisfies the condition

β

^]

≥ J(u

^]

)

Θ(u

^]

) ,

then u

^]

is also a solution of (9).

(36)

Numerical Results

Back to a Cost in Expectation

Using the previous lemma, we aim at solving a problem in which the cost and the constraint functions correspond to expectations:

min

u(·)

min

V(·)

E

K x

^ξ

(t

_f

) × I

C x

^ξ

(t

_f

)

s.t. E

I

C x

^ξ

(t

f

)

≥ p ,

or equivalently (Interchange Theorem [R&W, 1998]):

min

u(·)

E

min

v^ξ(·)

K x

^ξ

(t

_f

) × I

C x

^ξ

(t

_f

)

s.t. E

I

C x

^ξ

(t

f

)

≥ p .

(37)

Numerical Results

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(38)

Numerical Results

Lagrangian Formulation

min

u(·)

E

min

v^ξ(·)

K x

^ξ

(t

f

) × I

C x

^ξ

(t

f

)

s.t. p − E

I

C x

^ξ

(t

_f

)

≤ 0 ! µ

Assume there exists a saddle point for the associated Lagrangian.

In order to solve max

µ≥0

min

u(·)

(

µ p + E

min

v^ξ(·)

K x

^ξ

(t

_f

) − µ × I

C x

^ξ

(t

_f

)

| {z }

W (u, µ, ξ)

)

.

that is,

max

µ≥0

min

u(·)

n

µ p + E W (u, µ, ξ) o ,

we use an adapted Arrow-Hurwicz algorithm ([Culioli, 1994]).

(39)

Numerical Results

Algorithm Overview

Arrow-Hurwicz algorithm At iteration k,

1

draw a failure ξ

^k

= (t

p^ξ^k

, t

_d^ξ^k

) according to its probability law,

2

compute the gradient of W w.r.t. u and update u(·):

u

^k+1

= Π

_B

u

^k

− ε

^k

∇

_u

W (u

^k

, µ

^k

, ξ

^k

) ,

3

compute the gradient of W w.r.t. µ and update µ:

µ

^k+1

= max

0, µ

^k

+ ρ

^k

p + ∇

_µ

W (u

^k+1

, µ

^k

, ξ

^k

)

.

(40)

Numerical Results

First Difficulty: I Is Not a Smooth Function

At every iteration k , we must evaluate function W as well as its derivatives w.r.t. u(.) and µ. But W is not differentiable!

I(y ) =

( 1 if y = 0,

0 otherwise, I

r

(y ) =







1 −

^y_r2²

2

if y ∈ [−r , r ],

0 otherwise.

0.2 0.4 0.6 0.8 1.0

There are specific rules to drive r to 0 as the iteration number k goes to infinity in order to obtain the best asymptotic Mean Quadratic Error of the gradient estimates ([Andrieu et al., 2007]).

(41)

Numerical Results

Second Difficulty: Solving the Inner Problem

The approximated closed-loop problem to solve at each iteration is:

W

_r^k

(u

^k

, ξ

^k

, µ

^k

) = min

v^ξ(·)

n

K x

^ξ

(t

f

) − µ

^k

× I

_r^k

C x

^ξ

(t

f

)

o . In this setting, we have to check if the target is reached up to r

^k

. Different cases have to be considered:

1

the target can be reached accurately,

2

the target can be reached up to r

^k

only,

3

the target cannot be reached up to r

^k

.

Note that if reaching the target is possible but too expensive (that is, if K x

^ξ

(t

_f

) ≥ µ

^k

), the best thing to do is to stop the engine!

In practice, the solution of the approximated problem is derived

from the resolution of two standard optimal control problems. . .

(42)

Numerical Results

Parameters Tuning

Gradient step length:

ε

^k

= a

b + k , ρ

^k

= c d + k ,

usual for a stochastic gradient algorithm.

Smoothing parameter:

r

^k

= α β + k

¹³

,

MQE reduced by a factor 1000 in about 100.000 iterations.

(43)

Numerical Results

1

Two Elementary Exercices on the Stochastic Gradient Two-Stage Recourse Problem

Trade-off Between Investment and Operation

2

Option Pricing Problem and Variance Reduction Financial Problem Modeling

Computing Efficiently the Price Two Algorithms

3

Optimal Control Under Probability Constraint Satellite Model and Optimization Problem Probability and Conditional Expectation Handling Stochastic Arrow-Hurwicz Algorithm

Numerical Results

(44)

Numerical Results

Example: Interplanetary Mission

t

i

= 0.70 and t

f

= 8.70 (normalized units),

t

_p

: exponential distribution: P t

p

≥ t

_f

≈ 0.58 = π

_f

, t

_d

: exponential distribution: P 0.035 ≤ t

d

≤ 0.125 ≈ 0.80.

Comp. normale w Comp. tangentielle s Comp. radiale q

0 1 2 3 4 5 6 7 8 9

−0.4

−0.2 0.0 0.2 0.4 0.6 0.8 1.0

The deterministic optimal control has a “bang–off–bang” shape.

Along the optimal trajectory, the probability to recover a failure is:

p

^det

≈ 0.94.

(45)

Numerical Results

Masse finale / iterations

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.674

0.675 0.676 0.677 0.678 0.679 0.680 0.681

Multiplicateur probabilite / iterations

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.314

0.316 0.318 0.320 0.322 0.324 0.326

0 1 2 3 4 5 6 7 8 9

−0.4

−0.2 0.0 0.2 0.4 0.6 0.8 1.0

Figure: Probability level p = 0.750

(46)

Numerical Results

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.673

0.674 0.675 0.676 0.677 0.678 0.679 0.680 0.681

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.318

0.320 0.322 0.324 0.326 0.328 0.330 0.332

0 1 2 3 4 5 6 7 8 9

−0.4

−0.2 0.0 0.2 0.4 0.6 0.8 1.0

Figure: Probability level p = 0.960

(47)

Numerical Results

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.674

0.675 0.676 0.677 0.678 0.679 0.680 0.681

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 0.320

0.325 0.330 0.335 0.340 0.345 0.350 0.355 0.360 0.365

0 1 2 3 4 5 6 7 8 9

−0.4

−0.2 0.0 0.2 0.4 0.6 0.8 1.0

Figure: Probability level p = 0.990

(48)

Numerical Results

The Price of Safety. . .

Consommation sans panne / Probabilite

0.85 0.90 0.95 1.00

0.3204 0.3206 0.3208 0.3210 0.3212 0.3214 0.3216 0.3218 0.3220 0.3222

Figure: Fuel consumption versus probability level p

(49)

Numerical Results

H. Robbins and S. Monro.

A stochastic approximation method.

Annals of Mathematical Statistics, 22:400–407, 1951.

B. T. Polyak and A. B. Juditsky.

Acceleration of stochastic approximation by averaging.

SIAM Journal on Control and Optimization, 30(4):838–855, 1992.

M. Duflo.

Algorithmes stochastiques.

Springer Verlag, 1996.

H. J. Kushner and D. S. Clark.

Stochastic Approximation Methods for Constrained and Unconstrained Systems.

J.-C. Culioli and G. Cohen.

Decomposition-coordination algorithms in stochastic optimization.

SIAM Journal on Control and Optimization, 28(6):1372–1403, 1990.

J.-C. Culioli and G. Cohen.

Optimisation stochastique sous contraintes en esp´erance.

CRAS, t.320, S´erie I, pp. 75-758, 1995.

Laetitia Andrieu, Guy Cohen and Felisa J. V´azquez-Abad.

Gradient-based simulation optimization under probability constraints.

European Journal of Operational Research, 212, 345-351, 2011.

R. T. Rockafellar and R. J.-B. Wets.

Variational Analysis, volume 317 of A series of Comprehensive Studies in Mathematics.

Stochastic Gradient Method: Applications