Smart Power Systems: the Optimization Challenge

(1)

Smart Power Systems:

the Optimization Challenge

MichelDE LARA

Cermics_{, ´}_{Ecole des Ponts ParisTech, France}

and

Pierre Carpentier, ENSTA ParisTech, France

Jean-Philippe Chancelier, ´Ecole des Ponts ParisTech, France

Pierre Girardeau, post-doc EDF R & D

Jean-Christophe Alais, ´Ecole des Ponts and EDF R & D PhD student

Vincent Lecl`ere, ´Ecole des Ponts PhD student

Cermics, France

(2)

Outline of the talk

In 2000, theOptimization and Systemsteam was created at ´Ecole des Ponts

ParisTech, withstochastic optimizationas major theme. The idea was to

extenddecomposition-coordinationoptimization methods forlarge systems

— developed in the deterministic setting withElectricit´´ e de France(EDF

R&D) — to the stochastic case

During ten years, we havetrained PhD studentsin stochastic optimization,

mostly with EDF R&D. Since 2011, we witness a growing demand from

energy firms for stochastic optimization, fueled by adeep and fast

transformation of power systems

We discuss to what extent optimization is challenged by the smart grid perspective

We shed light on the two maindifferencesofstochastic controlwith respect

todeterministiccontrol: riskandinformation

(3)

Outline of the presentation

1 _{Long term industry-academy cooperation}

2 The “smart grids” seen from an optimizer perspective

3 _{Moving from deterministic to stochastic dynamic optimization}

4 _{Two snapshots on ongoing research}

(4)

Outline of the presentation

(5)

Outline of the presentation

´

Ecole des Ponts ParisTech–Cermics–Optimization and Systems Industry partners

2 _{The “smart grids” seen from an optimizer perspective}

What is the “smart grid”? Optimization is challenged

Dam models

The deterministic optimization problem is well posed

In the uncertain framework, the optimization problem is not well posed There are two ways to express the information constraints

Decomposition-coordination optimization methods under uncertainty Risk constraints in optimization

(6)

(7)

´

Ecole des Ponts ParisTech

TheEcole nationale des ponts et chauss´ees´ was founded in1747and isone of the world’s oldest engineering institutes

´

Ecole des Ponts ParisTech is traditionally considered as belonging to the

5 leading engineering schools in France

Young graduates find positions in professional sectors like transport and urban planning, banking, finance, consulting, civil works, industry, environnement, energy, etc.

Faculty and staff

217 employees (including 50 subsidaries). 165 module leaders, including 68 professors. 1509 students.

´

(8)

Research at ´

Ecole des Ponts ParisTech

Figures on research

Research personnel: 220

About 40 ´Ecole des Ponts PhDs students graduate each year

10 research centers

*CEREA (atmospheric environment), joint ´Ecole des Ponts-EDF R&D * CEREVE (water, urban and environment studies)

*CERMICS (mathematics and scientific computing)

* CERTIS (information technologies and systems) *CIRED (international environment and development)

* LATTS (techniques, regional planning and society) * LVMT (city, mobility, transport)

* UR Navier (mechanics, materials and structures of civil engineering, geotechnic) *Saint-Venant laboratory (fluid mechanics), joint ´Ecole des Ponts-EDF R&D * Paris School of Economic PSE

(9)

CERMICS, Centre d’enseignement et de recherche en

math´ematiques et calcul scientifique

The scientific activity of CERMICS covers several domains in

scientific computing modelling

optimization

14 senior researchers 14 PhD

11 habilitation `a diriger des recherches

Three missions

Teaching and PhD training Scientific publications Contracts

550 000 euros of contracts per yearwith

research and development centers of large industrial firms: CEA, CNES, EADS, EDF, Rio Tinto, etc.

(10)

Optimization and Systems Group

Three senior researchers J.-P. CHANCELIER M. DE LARA F. MEUNIER Five PhD students

J.-C. ALAIS (CIFRE EDF)

S. SEPULVEDA (foreign co-supervision) V. LECL`ERE (IPEF)

T. PRADEAU P. SARRABEZOLLES Four associated researchers

P. CARPENTIER (ENSTA ParisTech) L. ANDRIEU (EDF R&D)

K. BARTY (EDF R&D) A. DALLAGI (EDF R&D)

(11)

Optimization and Systems Group research specialities

Methods

Stochastic optimal control (discrete-time)

Large-scale systems

Discretization and numerical methods Probability constraints

Discrete mathematics; combinatorial optimization System control theory, viability and stochastic viability Numerical methods for fixed points computation Uncertainty and learning in economics

Applications

Optimized management of power systems under uncertainty

(production scheduling, power grid operations, risk management)

Transportmodelling and management

Natural resourcesmanagement (fisheries, mining, epidemiology)

Softwares

Scicoslab, NSP Oadlibsim

(12)

Publications since 2000

24 publicationsinpeer-reviewed international journals 3 publicationsin collective works

3 books

Modeling and Simulation in Scilab/Scicos with ScicosLab 4.4

(2e ´edition, Springer-Verlag)

Introduction `a SCILAB

(2e ´edition, Springer-Verlag)

Sustainable Management of Natural Resources. Mathematical Models and Methods(Springer-Verlag)

2 books to be finished soon:

Stochastic Optimization. At the Crossroads between Stochastic Control and Stochastic Programming

(13)

Teaching

Masters

Math´ematiques, Informatique et Applications ´

Economie du D´eveloppement Durable, de l’Environnement et de l’ ´Energie Renewable Energy Science and Technology Master ParisTech

´

Ecole des Ponts ParisTech

Optimisation et contrôle (J.-P. CHANCELIER) Modéliser l’aléa (J.-P. CHANCELIER)

Introduction `a Scilab (J.-P. CHANCELIER, M. DE LARA)

Mod´elisation pour la gestion durable des ressources naturelles (M. DE LARA) ´

(14)

Industrial and public contracts since 2000

Industrial contracts

Conseil fran¸cais de l’´energie(CFE)

SETEC Energy Solutions ´

Electricit´e de France(EDF R&D)

Thales

Institut fran¸cais de l’´energie(IFE)

Gaz de France(GDF)

PSA

Public contracts

STIC-AmSud (CNRS-INRIA-Affaires étrangères) Centre d’étude des tunnels

CNRS ACI ´Ecologie quantitative RTP CNRS

(15)

Outline of the presentation

´

Dam models

(16)

We cooperate with industry partners

As academics, wecooperate withindustry partners,

looking forlonglasting close relations

We are not consultants working for clients Our job consists mainly in

training Master and PhD students, workingin the companyand interacting with us, on subjects designed jointly

developingmethods, algorithms

contribute tocomputer codes developed within the company training professional engineersin the company

(17)

´

Electricit´e de France R & D / D´epartement OSIRIS

´

Electricit´e de France is the French electricity main producer 159 000 collaborateurs dans le monde

37 millions de clients dans le monde 65,2 milliards d’euros de chiffre d’affaire 630,4 TWh produits dans le monde ´

Electricit´e de France Research & Development 486 millions d’euros de budget

2 000 personnes

D´epartementOSIRIS

Optimisation, simulation, risques et statistiques pour les march´es de l’´energie

Optimization, simulation, risks and statistics for the energy markets 145 salari´es (dont 10 doctorants)

(18)

The Optimization and Systems Group has trained

8 PhD from 2004 to 2011 + 2 PhD students,

the majority related with EDF and energy management

* Laetitia ANDRIEU, former PhD student at EDF, now researcher EDF * Kengy BARTY, former PhD student at EDF, now researcher EDF * Daniel CHEMLA, former PhD student

* Anes DALLAGI, former PhD student at EDF, now researcher EDF * Laurent GILOTTE, former PhD student with IFE, researcher EDF * Pierre GIRARDEAU, former PhD student at EDF, now with ARTELYS * Eug´enie LIORIS, former PhD student

* Babacar SECK, former PhD student at EDF

* Cyrille STRUGAREK, former PhD student at EDF, now with Munich-R´e * Jean-Christophe ALAIS, PhD student at EDF

(19)

PhD subjects

Contributions to the Discretization of Measurability Constraints for Stochastic Optimization Problems,

Optimization under Probability Constraint,

Uncertainty, Inertia and Optimal Decision. Optimal Control Models Applied to Greenhouse Gas Abatment Policies Selection,

Variational Approaches and other Contributions in Stochastic Optimization,

Particular Methods in Stochastic Optimal Control,

From Risk Constraints in Stochastic Optimization Problems to Utility Functions,

Resolution of Large Size Problems in Dynamic Stochastic Optimization and Synthesis of Control Laws,

Evaluation and Optimization of Collective Taxis Systems,

Risk and Optimization for Energies Management,

(20)

Other contacts with French small consulting companies

ARTELYSis a company specializing inoptimization, decision-making and modeling. Relying on their high level ofexpertise in quantitative methods, the consultants deliver efficient solutions to complex business problems. They

provide services to diversified industries: Energy & Environment, Logistics &

Transportation, Telecommunications, Finance and Defense.

Cr´e´ee en 2011,SETEC Energy Solutionsest la filiale du groupe SETEC

spécialisée dans les domaines de laproductionet de lamaˆıtrise de l’énergie

en France et à l’étranger. SETEC Energy Solutions apporte à ses clients la maˆıtrise des principaux process énergétiques pour la mise en œuvre de solutions innovantes depuis les phases initiales de définition d’un projet jusqu’à son exploitation.

(21)

Summary

The following slides on “smart grids” express a viewpoint from an optimizer perspective

working in an optimization research group in an applied mathematics research center in a French engineering institute

having contributed to train students now working in energy having contacts and contracts with energy/environment firms

(22)

´

Dam models

(23)

Outline of the presentation

´

Dam models

(24)

Three key drivers for the “smart grid” concept

Environment Markets Technology

(25)

Key driver: environmental concern

Strong impulse towards adecarbonized energy system

Expansion ofrenewable energy sources

⇒Successfullyintegrating renewable energy sourceshas become critical,

(26)

Key driver: deregulation

Apower system(generation/transmission/distribution)

less and less vertical(deregulation of energy markets) hence withmany players with their own goals

with somenew players

industry (electric vehicle)

regional public authorities (autonomy, efficiency)

with anetwork in horizontal expansion

(the Pan European system counts 10,000 buses, 15,000 power lines, 2,500 transformers, 3,000 generators, 5,000 loads)

with more and more exchanges (trade of commodities)

⇒Achange of paradigm for management from centralized tomore and moredecentralized

(27)

Key driver: telecommunication, metering and computing

technology

A power system withmore and more technology

due to evolutions in the fields of computing and telecoms smart meters

sensors controllers

grid communication devices, etc.

⇒Ahuge amount of datawhich, one day, will be

(28)

The “smart grid”? An infrastructure project with promises

to be fulfilled by a “smart power system”

Hardware/ infrastructures / smart technologies

Renewable energies technologies Smart metering

Storage Promises

Quality, tariffs More safety

More renewables (environmentally friendly) Software/ smart management

(energysupplybeingless flexible, make thedemand more flexible)

smart management, smart operation, smart meter management, smart distributed generation, load management, advanced distribution management systems, active demand management, diffuse effacement, distribution management systems, storage management, smart home, demand side management, etc.

(29)

Outline of the presentation

´

Dam models

(30)

A call from the industry to optimizers

Every three years,´Electricit´e de France(EDF) organizes an international

Conference on Optimization and Practices in Industry(COPI)

At the last COPI’11, Jean-Fran¸cois Faugeras from EDF R&D opened the

conference with aplenary talkentitled“Smart grids: a wind of change in

power systems and new opportunities for optimization”

He claimed that “power system players are facing high level problems to solve

requiring new optimization methods and tools”,

with “not only a ’smart(er)’ grid buta ’smart(er)’ power system”

and called on the optimizers to develop new methods

EDF R&Dhas just created anew program Gaspard Monge pour l’Optimisation et la recherche op´erationnelle(PGMO)

(31)

What is “optimization”?

Optimizing is obtaining the best compromise between needs and resources

Marcel Boiteux (pr´esident d’honneur d’EDF)

Resources: portfolio of assets

production units

costly/not costly: thermal/hydropower

stock/flow, predictable/unpredictable: thermal/wind

tariffs options, contracts Needs: energy, safety, environment

energy uses

safety, quality, resilience (breakdowns, blackout)

environment protection (pollution) and alternative uses (dam water) Best compromise: minimize socio-economic costs (including externalities)

(32)

Electrical engineers metiers and skills are evolving

Unit commitment, optimaldispatchof generating units:

finding the least-cost dispatch of available generation resources to meet the electrical load

which unit? 0/1 variables

which power level? continuous variables

subject to more unpredictable energy flows (solar, wind) and demand (electrical devices, cars)

Markets: day-ahead, intra-day(balancing market):

dispatcher takes bids from the generators, demand forecasts from the distribution companies and clears the market

subject to more unpredictability, more players

Long term planning

subject to more unpredictability (technologies, climate), more players . . . Without even speaking of voltage, frequency and phase control

(33)

Optimization skills will follow the power system evolution

We focus on generation and trading, not on transmission and distribution

Less base production andmore wind and photovoltaic fatal generationmakes

supply more unpredictible

→stochasticoptimization

Hence morestorage(batteries, pumping stations)

→dynamicaloptimization, reservesdimensioning

Theshape of the load is changingdue to electric vehicle penetration

→demand-side management, “peak shaving”,adaptive tariffs

New subsystems emergewith local information and means of action: smart

meters, new producers,micro-grid,virtual power plant

→agregation,coordination,decentralizedoptimization

Markets(day-ahead, intra-day)

→optimization under uncertainty

Environmental constraintson production (CO2) and resources usages (water)

(34)

A context of increasing complexity

Multiple energy resources: photovoltaic, solar heating, heatpumps, wind, hydraulic power, combined heat and power

Spatially distributed energy resources (onshore and offshore windpower, solarfarms), producers, consumers Strongly variable production: wind, solar Intermittent demand: electrical vehicles Two-ways flows in the electrical network Environmental and risk constraints (CO2, nuclear risk, land use)

(35)

An optimization birdview

Complex power systems

stocks: water reservoirs, storage, accumulators, fuels, etc.

uncertain demand and production multiple constraints: costs, environment, blackout risk, etc.

large interconnected networks multiple producers and consumers information is not shared by all the actors

Complex decision problems

dynamics uncertainties multiple objectives large scale multiple decision-makers decentralized information

(36)

Summary

Three major key factors — environmental concern, deregulation, telecommunication, metering and computing technology — drive the power systems mutation

This induces a change of paradigm for management: from vertical centralized predictible “stock” energies to more horizontal centralized unpredictible “flow” energies Ripples are touching the optimization community

Specific optimization skills will be required, because optimal solution based on a nominal forecast (deterministic setting) might perform poorly under off-nominal conditions

Roger Wets’ illuminating example: deterministic vs. robust

of a furniture manufacturer deciding how many dressers of each type to make with man-hours and time as constraints; when they are uncertain, the stochastic

optimal solution considers all 106_{possibilities and provides a robust solution,}

(37)

Outline of the presentation

(38)

´

Dam models

(39)

At

St

Qt

Rt

(40)

A single dam nonlinear dynamical model where turbinating

decisions are made before knowing the water inflows

We can model the dynamics of the water volume in a dam by

S(t+ 1) | {z } future volume = min{S♯, S(t) |{z} volume − q(t) |{z} turbined + a(t) |{z} inflow volume }

S(t) volume(stock) of water at the beginning of period [t,t+ 1[

a(t) inflow water volume(rain, etc.) during [t,t+ 1[ decision-hazard:

a(t) is not available at the beginning of period [t,t+ 1[ q(t) turbined outflow volumeduring [t,t+ 1[

decided at the beginning of period [t,t+ 1[

supposed todepend on the stockS(t)butnot on the inflow watera(t)

(41)

A single dam nonlinear dynamical model in decision-hazard:

equivalent formulation

S(t+ 1) | {z } future volume = S(t) |{z} volume − q(t) |{z} turbined − r(t) |{z} spilled + a(t) |{z} inflow volume

a(t) inflow water volume(rain, etc.) during [t,t+ 1[

q(t) turbined outflow volumeduring [t,t+ 1[

supposed todepend on the stockS(t)butnot on the inflow watera(t)

chosen such that 0≤q(t)≤S(t)

(42)

When turbinating decisions are made after knowing the

water inflows, we obtain a linear dynamical model

S(t+ 1) | {z } future volume = S(t) |{z} volume − q(t) |{z} turbined − r(t) |{z} spilled + a(t) |{z} inflow volume

a(t), inflow water volume(rain, etc.) during [t,t+ 1[; hazard-decision:

a(t)isknown and available at thebeginning of period [t,t+ 1[ q(t) turbined outflow volumeand r(t) spilled volume

supposed todepend on the stockS(t)andon the inflow watera(t)

chosen such that

(43)

(44)

Typology of hydro-valleys

(c)

(a) (b)

(45)

Sketch of a cascade model with dams

i

= 1

, . . . ,

N

Dam 2 Dam 3 Dam 1 S₁_,t S₂_,t S₃_,t Q₁_,t R₁_,t Q₂_,t R₂_,t Q₃_,t R₃_,t A₁_,t A₂_,t A₃_,t

ai,t : inflowinto dami at timet (rain, run off water)

Si,t : volumein dami at timet (water volume)

qi,t : turbinedfrom dami at timet (valued at pricepi,t)

(46)

(47)

Description of variables:

ai

,t

Inflow water volume

ai,t: amount of water inflowingwithout any controlinto the dam

Inflow water volumesai,t are part of theproblem data, and may be

eitherdeterministic, in which case the values(ai,t0, . . . ,ai,T)are unique and

are exactly known, for every dami

oruncertain, and many possibilities can be considered

inflows chroniclesmay be available:

(as i,t0, . . . ,a

s

i,T)wheres belongs to a setS ofhistorical scenarios in addition, theprobabilityπs _{of scenario}_s _{may be known}

more generally, the sequence(ai,t0, . . . ,ai,T)may be considered as a

random processwith knownprobabilitydistribution

Pricespi,t can also be part of the problem data

In the same way, the prices of turbined volumes can be part of the problem data, with an uncertain or deterministic status

(48)

Description of variables:

qi

,t

Turbined water volume

qi,t: amount of water turbined to produce electricity

Turbined volumesqi,t arecontrol variables, the possible values of which are

in the hands of the decision-makerto achieve different goals

In thedeterministicframework, one looks for

a single sequence of values (qi,t0, . . . ,qi,T−1)for each dami

In theuncertainframework, the situation is more complex

one has to specify theonline available information

upon which decisions are made

thecontrolsqi,t are alsouncertain variables

and they can be obtained by means offeedback policiesfeeding on online information

(49)

Description of variables:

ri

,t

Spilled water volume

ri,t: amount of water spilled from the dam with or without control

The status of the spilled volumes depends on the context, andri,t can represent

an amount taken in the dam (for irrigation purposes, for instance), and can

be adata(deterministic or uncertain)

a volume deliberately taken in the dam and released without being turbined,

and it is then acontrolvariable

the amount of water which overflows in case of excess

ri,t = [Si,t+ai,t+qi−1,t−qi,t−Si♯]+

and it is then anoutputvariable which results from a calculation based upon

(50)

The dynamics describes the temporal evolution

of the water stock volume in the dam

Dam

i

Qi

−

1

,t

Si,t

Qi,t

Ai,t

The water volume in the dam evolves when time goes by and it is given by

Si,t+1=Si,t+ai,t+qi−1,t−qi,t

A general form is

Si,t+1=Dyni,t Si,t,qi,t,ai,t,qi−1,t

(51)

The payoffs are decomposed in instantaneous and final

The payoff obtained by turbining the volumeqi,t is

pi,tqi,t

We shall use the general form

Utili,t Si,t,qi,t,ai,t,pi,t

To avoid emptying the dam at the end of the optimization process, we introduce a “water value” term bearing on the final stock volume

UtilFini Si,T

(52)

The management of the dams cascade can be formulated

as an intertemporal optimization problem

The payoff attached to the dam cascade is

N X i=1 T−1 X t=t0 Utili,t Si,t,qi,t,ai,t,pi,t+UtilFini Si,T

This payoff must be optimized

under constraints ofdynamics Si,t+1=Dyni,t

`

Si,t,qi,t,ai,t,qi−1,t

´

, i∈J1,NK, t∈Jt0,T−1K

and underboundsconstraints

qi,t∈ ˆ q i,qi ˜ , i ∈J1,NK, t∈Jt0,T−1K Si,t ∈ ˆ S_i,Si ˜ , i∈J1,NK, t∈Jt0,TK

(53)

´

Dam models

(54)

The deterministic optimization problem

In the deterministic case, inflowsai:= ai,t0, . . . ,ai,T

and prices

pi:= pi,t0, . . . ,pi,T

are unique and are exactly known, for every dami

Withqi := qi,t0, . . . ,qi,T−1

andSi:= Si,t0, . . . ,Si,T

, the optimization problem is

max (qi,Si)i=1,...,N N X i=1 T−1 X t=t0 Utili,t Si,t,qi,t,ai,t,pi,t +UtilFini Si,T

under constraints of dynamics

Si,t0 given, i ∈J1,NK Si,t+1=Dyni,t Si,t,qi,t,ai,t,qi−1,t

, i ∈J1,NK, t ∈Jt0,T−1K

and under bounds constraints

qi,t ∈ q i,qi , i∈J1,NK, t ∈Jt0,T−1K Si,t ∈ S_i,Si , i∈J1,NK, t ∈Jt0,TK

(55)

The deterministic optimization problem is well posed

If, to make things simple, we assume all variables to be scalar, the optimization must be made with respect to all control variables (∈RN(T−t0)_{) and to all “state” variables} (∈RN(T−t0+1)₎

The optimization yields for every dam

trajectories (q⋆_i_,_t₀, . . . ,q_i⋆_,_T₋₁)and

(S_i⋆_,_t₀, . . . ,S_i⋆_,_T)of the controls and of the

(56)

Here are some characteristics

of the deterministic optimization problem

The resolution of the deterministic optimization problem can be made in the

framework ofmathematical programming, that is, the maximization of a

function overRn_{under equality and inequality constraints}

The volumesSi,t areintermediary variables,

completely determined by the choice of theqi,t

(however, such intermediary variables may be used to compute gradients by means of an adjoint state)

The optimization problem is often solved after formulating it as a

linear problemand with softwares such asCPLEX

In thenonlinear case, the optimization problem can bedifficultto solve, due

to itslarge size, but it is known how to applydecomposition and coordination

(57)

Outline of the presentation

´

Dam models

(58)

0 50 100 150 200 250 300 350 400 0 10 20 30 40 50 60 70 80 90 100

Stock volumes in a dam following an optimal strategywith a final value of water

(time)

(59)

In the uncertain framework,

two additional questions must be answered

Question (expliciting risk attitudes)

How are the uncertainties taken into account in the payoff criterion and in the constraints?

Question (expliciting available online information)

Upon which online information are turbinating decisions made? In practice, one assumes

more or less precise knowledge of the past (online information) statistical assumptions about the future (offline information)

(60)

How are the uncertainties taken into account

in the payoff criterion and in the constraints?

In aprobabilistic setting, where uncertainties are random variables, a classical answer is

to take themathematical expectationof the payoff (risk-neutral approach)

E N X i=1 T−1X t=t0 Utili,t Si,t,qi,t,ai,t,pi,t +UtilFini Si,T

and to satisfy all (physical) constrainstsalmost surelythat is, practically,

for all possible issues of the uncertainties (robust approach)

P qi,t ∈ q i,qi , i∈J1,NK, t∈Jt0,T−1K Si,t∈ S_i,Si , i ∈J1,NK, t∈Jt0,TK ! = 1

(61)

Upon which online information

are turbinating decisions made?

On the one hand, it issuboptimalto restrict oneself,

as in the deterministic case, toopen-loop controlsdepending only upon time

On the other hand, it is impossible to suppose that we know in advance

what will happen for all times: clairvoyance is impossible

The in-between is non anticipativity constraint

In our case, thedecisionqi,t will be looked after as a

fonction of past uncertainties,

that is, of the water inflows and the prices aj,τ,pj,τ

(62)

On-line information feeds turbinating decisions:

the non anticipativity constraint

As a consequence, the controlqi,t is looked after under the form

qi,t=φi,t

`

a1,t0,p1,t0, . . . ,aN,t0,pN,t0, . . . ,

a1,τ,p1,τ, . . . ,aN,τ,pN,τ, . . .a1,t−1,p1,t−1, . . . ,aN,t−1,pN,t−1´

Denote the uncertainties at timet by

wt = a1,t,p1,t, . . . ,aN,t,pN,t

When uncertainties are considered asrandom variables (measurable

mappings), the above formula forqi,t expresses themeasurabilityof the

control variableqi,t with respect to the past uncertainties, also written as

σ(qi,t)⊂σ wt0, . . . ,wt−1

⇐⇒ qi,tσ wt0, . . . ,wt−1

(63)

On-line information structure can reflect

decentralized information

When the controlqi,t is looked after under the form

qi,t=φi,t ai,t0,pi,t0, . . . ,ai,τ,pi,τ, . . . ,ai,t−1,pi,t−1

(64)

The stochastic optimization problem

In aprobabilistic setting, where uncertainties are random variables and where a probabilityP_{is given, a possible formulation is}

max (q_i,S_i)i=1,...,N E „ N X i=1 “TX−1 t=t0 Utili,t ` Si,t,qi,t,ai,t,pi,t ´ +UtilFini ` Si,T ´” «

under constraints of dynamics

Si,t0 given, i∈J1,NK Si,t+1=Dyni,t ` Si,t,qi,t,ai,t,qi−1,t ´ , i∈J1,NK,t∈Jt0,T−1K

under bounds constraints P qi,t ∈ ˆ q i,qi ˜ ,i∈J1,NK,t∈Jt0,T−1K Si,t∈ ˆ S_i,Si ˜ ,i∈J1,NK,t∈Jt0,TK ! = 1

and undermeasurability constraints qi,tσ

`

wt0, . . . ,wt−1

´

(65)

The outputs of a stochastic optimization problem

are random variables

210000 215000 220000 225000 230000 235000 240000 245000 250000 0.0e+00 1.0e−05 2.0e−05 3.0e−05 4.0e−05 5.0e−05 6.0e−05 7.0e−05 8.0e−05 9.0e−05 1.0e−04

(66)

´

Dam models

(67)

There are two ways to express the information constraints

Functional approachqi,t =φi,t wt0, . . . ,wt−1

In the special case of theMarkovianframework withwhite noise, it can be

shown that there isno loss of optimalityto look for solutions under the form

qi,t=φi,t S1,t, . . . ,SN,t

| {z }

state

Measurability constraints qi,tσ wt0, . . . ,wt−1

In the special case of thescenario tree approach, uncertainties are supposed to be observed, and all possible scenarios`

wts0, . . . ,w

s T ´

,s∈S, are organized in a tree, andcontrolsqi,t are indexed by nodes on the tree

Equivalently,qi,t are indexed bys∈S with the constraint that if two scenarios coincide up to timet, so must do the controls at timet

` wt0s, . . . ,w s t−1 ´ =` wt0s′, . . . ,w s′ t−1 ´ ⇒qsi,t =q s′ i,t

(68)

General framework

Stochastic Optimal Control (SOC)

Consider adynamical systemin discrete time influenced by exogenousnoise

At each time step:

observationsare made on the system and kept in memory adecision/control is chosen, and the system evolves

Our aim is tominimize some objective functionthat depends on the evolution of

(69)

A motivating example

Typical application of SOC: portfolio management

Consider a power producer which has to

manage anamountxt of stockat each time stept

usingdecisionut for each time stept

while dealing withstochastic data wt, such as the power demand, unit

breakdowns, market prices

(70)

Mathematical formulation

Standard problem min x,u E( T−1 X t=0 Ct(xt,ut,wt+1) +K(xT)), s.t. xt+1=ft(xt,ut,wt+1), ∀t = 0, . . . ,T−1, x0=w0,

P-a.-s. constraints (e.g. bounds onxt andut),

ut isFt-measurable, whereFt :=σ{w0, . . . ,wt}

(71)

Classical methods

Stochastic Programming

Discretize randominputs of the problem

usingscenario trees

Write the constraints and objective function on this structure (time has become

arborescent)

Use standard numerical methodsfrom mathematical programming (linear/convex programming to overcome the combinatorial explosion of the tree)

Carpentier/Cohen/Culioli (1995); Birge/Louveaux (1997); Pflug (2001); Bacaud/Lemaréchal/Renaud/Sagastizábal (2001); Heitsch/Römisch (2003);

Dupaˇcová/Gröwe-Kuska/Römisch (2003); Barty (2004); Shapiro/Dentcheva/Ruszczyński (2009) ...

(72)

Classical methods

Stochastic Programming

Discretize randominputs of the problem

usingscenario trees

Write the constraints and objective function on this structure (time has become

arborescent)

Use standard numerical methodsfrom mathematical programming (linear/convex programming to overcome the combinatorial explosion of the tree)

Complexity of multistage stochastic programs

Theamount of scenariosneeded to achieve a given precisiongrows exponentiallyw.r.t. the number of time steps

(73)

Classical methods

Dynamic Programming (1)

Markovian framework

Noise variableswt are independent over time Cost-to-go function, Bellman function, value function

Vt(x) := min u E T−1 X s=t Cs(xs,us,ws+1) +K(xT)|xt =x ! , ∀x ∈X, subject to constraints of the original problem

(74)

Classical methods

The Dynamic Programming (DP) principle Bellman (1957)

There is no loss of optimality in looking for theoptimal strategy as feedback functions onxt only.

Moreover, DP provides a way tocompute Bellman functions backwards:

VT(x) =K(x), ∀x ∈XT, and, for allt=t0, . . . ,T−1:

Vt(x) = min

u∈U_tE(Ct(x,u,wt+1) +Vt+1(ft(x,u,wt+1))), ∀x∈Xt

(75)

Classical methods

Most of the time, theDP equation cannot be solved analytically Numerical methodshave to be used

Curse of dimensionality

The complexity associated with the resolution of the DP equations by discretization

(76)

Classical methods

Most of the time, theDP equation cannot be solved analytically Numerical methodshave to be used

Curse of dimensionality

The complexity associated with the resolution of the DP equations by discretization

grows exponentially w.r.t. the dimension of the state space

Some attempts of breaking the curse are through functional approximations: Approximate Dynamic Programming Bellman/Dreyfus (1959); Gal (1989);

de Farias/Van Roy (2003); Longstaff/Schwartz (2001); Bertsekas/Tsitsiklis (1996)... Stochastic Dual Dynamic Programming Pereira/Pinto (1991); Philpott/Guan (2008); Shapiro (2010)

(77)

Summary

Stochastic optimization highlightsrisk attitudestackling

Stochastic dynamic optimization emphasizes the handling of

online information

Many open problems, because

many ways to represent risk (criterion, constraints) many information structures

(78)

(79)

Outline of the presentation

´

Dam models

(80)

Decomposition-coordination: divide and conquer

Spatialdecomposition

multiple players with their local information local / regional / national /supranational Temporaldecomposition

Astateis aninformation summary

Time coordination realized throughDynamic Programming, by value functions Hard nonanticipativity constraints

Scenariodecomposition

Along each scenario,sub-problemsaredeterministic(powerful algorithms) Scenario coordination realized throughProgressive Hedging, by updating nonanticipativity multipliers

(81)

Problems and methods: a very tentative table

Feel free to comment and criticize! ;-)

Decompose and coordinate

Space Time Scenarios Scenarios Trees

DP PH 2SSP

decentralized units OK

storage OK

unit commitment OK

short term market OK

Dynamic Programming: DP Progressive Hedging: PH

Two and Multi-Stage Stochastic Programming : 2SSP Use specific analytical features

Stochastic Programming: linearity, convexity

Stochastic Dual Dynamic Programming: linear dynamics, convex Bellman function approximated by “cuts”

(82)

We have a nice decomposed problem but. . .

Flower structure

We are almost in the case where units could bedriven independently

one from another Unit 1

Unit 2 Unit 3 Unit 4 Unit 5 Unit 6 Unit 7 Unit 8

(83)

We have a nice decomposed problem but. . .

Flower structure Unfortunately... Unit 1 Unit 2 Unit 3 Unit 4 Unit 5 Unit 6 Unit 7 Unit 8 Coupling constraint

(84)

The associated optimization problem may be written as

min (u1,...,uN) N X i=1 Ji(ui) | {z } costs minimization under N X i=1 Θi(ui) =D | {z } offer = demand where

ui is thedecisionof each uniti

Ji(ui)is thecostof making decisionui for uniti

(85)

Proper prices allow decentralization of the optimum

min (u1,...,uN) N X i=1 Ji(ui) under N X i=1 Θi(ui) =D

The simplestdecomposition/coordination schemeconsists in

buying the production of each unit at aprice λ(k)

and letting each unit minimizing its costs min ui Ji(ui) +λ(k) |{z} price Θi(ui)

then, modifying the price depending on the coupling constraint

λ(k+1)=λ(k)+ρ N X i=1 Θi(ui)−D

(86)

Extension to dynamics and stochasticity

If we explicitely take into account time and uncertainties, a classical approch consists in writing the new problem

min {ui,t}ti∈{0∈{1,,TN−}1} E N X i=1 TX−1 t=0

Li,t xi,t,ui,t,wi,t

+Ki xi,T

under the constraints N

X

i=1

Θi,t xi,t,ui,t,wi,t−dt = 0, t ∈J0,T−1K

(87)

We need to specify information constraints

The optimization problem isnot well posed, because we have not specified

upon what depends the controlui,t of each uniti at each time t!

In thecausal and perfect memory case, we say that the controlui,t depends

of all past noises up to timet

ui,t=φi,t w1,0, . . . ,wN,0,d0. . . .w1,t, . . . ,wN,t,dt

When the controlui,t is looked after under the form

ui,t =φi,t wi,0,d0. . . .wi,t,dt

this expresses that each unit is managed withlocal information,

(88)

Looking after decentralizing prices models

Going on with the previous scheme, each uniti solves

min ui,0,...,ui,T−1E „T−1 X t=0 “ Li,t ` xi,t,ui,t,wi,t ´ +λ(_i_,k_t)Θi,t ` xi,t,ui,t,wi,t ´” +Ki ` xi,T ´ «

under the constraints

xi,t+1=fi,t xi,t,ui,t,wi,t, t ∈J0,T−1K

In all rigor, the optimal controlsu⋆_i,t of this problem depend uponxi,t and

uponall past prices(λ(_i,k₀), . . . , λ(_i,tk))

Research axis: find an approximatedynamical model for the prices, driven by proper information (for instance, replaceλ(_i,tk) byEλ(_i,tk)|yt

, where the

(89)

Dual Approximate Dynamic Programming

λk

Samples/scenarios of dual variable at iterationk

(90)

Dual Approximate Dynamic Programming

λk Samples/scenarios of dual variable at iterationk Subproblemi Subproblem 1 SubproblemN We solve subproblems usingE(λk|y) by Dynamic Programming

(91)

Policy Φ1“x1,y” Policy Φi“xi,y” Policy ΦN“xN,y” We obtain policies

(92)

We simulate scenarios of strategies and prices and update prices using a gradient step λk_t+1=λk_t+ρ×PN i=1g it “ xi_t,k,ui_t,k,wt ” We update prices using a gradient step

(93)

Dual Approximate Dynamic Programming

λk At iterationk+ 1 Subproblemi Subproblem 1 SubproblemN We solve subproblems usingE(λk|y) by Dynamic Programming

We simulate scenarios of strategies and prices and update prices using a gradient step λk_t+1=λk_t+ρ×PN i=1g it “ xi_t,k,ui_t,k,wt ” We update prices using a gradient step

(94)

(95)

Contribution to dynamic tariffs

Spatial decomposition of a dynamic stochastic optimization problem Lagrange multipliers attached to spatial coupling constraints are stochastic processes (prices)

By projecting these prices, one expects to identify approximate dynamic models

(96)

Outline of the presentation

´

Dam models

(97)

Dam management under “tourism” constraint

Maximizing the revenue from turbinated water under a tourism constraint of having enough water in July and August

(98)

(99)

A single dam nonlinear dynamical model in decision-hazard

S(t+ 1) | {z } future volume = min{S♯, S(t) |{z} volume − q(t) |{z} turbined + a(t) |{z} inflow volume }

a(t) inflow water volume(rain, etc.) during [t,t+ 1[ decision-hazard:

a(t) is not available at the beginning of period [t,t+ 1[ q(t) turbined outflow volumeduring [t,t+ 1[

decided at the beginning of period [t,t+ 1[ supposed todepend onS(t)butnot ona(t)

(100)

The traditional economic problem is

maximizing the expected payoff

Suppose that

aprobabilityPis given on the set Ω =RT−t0 _of_{water inflows scenarios}

`

a(t0), . . . ,a(T−1)´

turbined waterq(t) is sold at pricep(t), related to the price at which energy can be sold at timet

at the horizon,the final volumeS(T) has a valuegiven byUtilFin`S(T)´

, the “final value of water”

The traditional (risk-neutral) economic problem is to maximize the intertemporal payoff (without discounting if the horizon is short)

maxE    T−1 X t=t0 price z}|{ p(t) turbined q(t) |{z} +

final volume utility

z }| {

UtilFin S(T)   

(101)

Dam management under “tourism” constraint

Traditional cost minimization/payoff maximization

maxE    T−1 X t=t0

turbined water payoff

z }| {

p(t)q(t) +

z }| {

UtilFin S(T)   

For “tourism” reasons:

volume S(t)≥S♭, ∀t∈ {July, August}

In what sense should we consider this inequalitywhich involves the

(102)

Robust / almost sure / probability constraint

Robustconstraints: for all the scenariosin a subset Ω⊂Ω

S(t)≥S♭, ∀t∈ {July, August}

Almost sureconstraints

Probability

  

water inflowscenariosalong which

the volumesS(t) are above the

thresholdS♭ for periodst in summer

  

= 1

Probabilityconstraints, with “confidence” levelp∈[0,1]

Probability

  

the volumesS(t) are above the

thresholdS♭ _{for periods}_t _{in summer}

  

≥p

(103)

Environmental/tourism issue leads to dam management

under probability constraint

The traditional economic problem ismaxE[P(T)]

where the payoff/utility criterion is

P(T) =

T−1 X t=t0

turbined water payoff z }| {

p(t)q(t) +

z }| {

UtilFin S(T)

and a failure tolerance is accepted

Probability

  

the volumesS(t)≥S♭

for periodst in July and August

  

(104)

(105)

(106)

However, though the mean is optimal,

(107)

We propose a stochastic viability formulation

to treat symmetrically and

to guarantee both environmental and economic objectives

Given two thresholds to be guaranteed

a volumeS♭(measured in cubic hectometershm3) a payoffP♭(measured in numeraire$)

we look after policies achieving themaximal viability probability

Π(S♭,P♭)= maxProba       

water inflow scenarios along which thevolumesS(t)≥S♭

for all time t ∈ {July, August}

andthe finalpayoff P(T)≥P♭

      

The maximal viability probability Π(S♭_,_P♭₎_{is the}_{maximal probability to}

(108)

A stochastic viability formulation

State, control and dynamic

The state is the couplex(t) = S(t),P(t)volume/payoff

The controlu(t) =q(t)is the turbined water

The dynamics is S(t+ 1) | {z } future volume = min{S♯, S(t) |{z} volume − q(t) |{z} turbined + a(t) |{z} inflow volume }, t =t0, . . . ,T −1 P(t+ 1) | {z } future payoff = P(t) |{z} payoff + p(t)q(t) | {z } turbined water payoff

, t =t0, . . . ,T −2

P(T) = P(T−1) +UtilFin S(T)

| {z }

(109)

A stochastic viability formulation

State and control constraints

The control constraints are

u(t)∈B t,x(t) ⇐⇒ 0≤q(t)≤S(t)

The state constraints are

x(t)∈A(t) ⇐⇒

S(t)≥S♭ , ∀t∈ {July, August}

(110)

Dynamic programming equation

Abstract version V(T,x) = 1A(T)(x) V(t,x) = 1A(t)(x) max u∈B₍t,x)Ew(t) h V“t+ 1,Dyn`t,x,u,w(t)´”i Specific version V(T,S,P) = 1_{_P_≥_P♭_} V(T−1,S,P) = max 0≤q≤S E_a₍_t₎ h V“t+ 1,S−q+a(t),P+UtilFin` S´”i V(t,S,P) = max 0≤q≤S

E_a₍_t₎hV“t+ 1,S−q+a(t),P+pq”i,t6∈ {July, August} V(t,S,P) = 1_{_S_≥_S♭_} max

0≤q≤S

E_a₍_t₎hV“t+ 1,S−q+a(t),P+pq”i,

(111)

Maximal viability probability Π(

S

♭

,

P

♭

)

as a function of guaranteed thresholds

S

♭

and

P

♭

For example, the probability to guarantee

a final payoff above

P♭=1 Meuros

and a volume above

S♭=40hm3_{in July}

and August

(112)

Iso-values for the maximal viability probability

as a function of guaranteed thresholds

S

♭

and

P

♭

(113)

Contribution to quantitative sustainable management

Conceptual frameworkfor

quantitativesustainable management

Managing ecological and economic

conflicting objectives

Displayingtradeoffsbetween

ecology and economysustainability

(114)

(115)

Trends are favorable to statistics and optimization

More telecom technology

→more data

More data, more unpredicability

→more statistics

More unpredicability→more storage

→more dynamic optimization

More unpredicability

(116)

Opportunities for higher education

Masters

Math´ematiques, Informatique et Applications ´

Economie du D´eveloppement Durable, de l’Environnement et de l’ ´Energie Renewable Energy Science and Technology Master ParisTech

and others ;-)

Research centers

CERMICS(mathematics and scientific computing)

´

Electricit´e de France Research & Development D´epartement OSIRIS

Optimization, simulation, risks and statistics for the energy markets and others ;-)

(117)

Pedagogical material

CEA / EDF / INRIA Summer School on Stochastic Optimizationslides http://www-hpc.cea.fr/SummerSchools2012-SO.htm

Advanced Course on Sustainable Optimizationslides http://cermics.enpc.fr/~delara/ENSEIGNEMENT/

Sustainable_Optimization/Sustainable_Optimization.html

Scicoslab practical computer works http://cermics.enpc.fr/~scilab

More on theOptimization and Systemsweb page

http://cermics.enpc.fr/equipes/optimisation.html More on my web page

(118)

Some related initiatives

Gaspard Monge Program for Optimization and operations research

http://www.fondation-hadamard.fr/pgmo

From January 7 to April 5, 2013,Institut Henri Poincar´e (IHP)thematic

quarterMathematics of Bio-Economicsas a contribution toMathematics of

Planet Earth - MPE2013

http://www.ihp.fr/en/ceb/mabies

From January 7 to March 1, 2013,Master Course on Stochastic Optimization

(MMMEF)

http://cermics.enpc.fr/equipes/optimisation.html

From April 8 to April 12, 2013,CIRMSchool onStochastic Control for the

Management of Renewable Energiesas a contribution toMathematics of Planet Earth - MPE2013

http://cermics.enpc.fr/~delara/ENSEIGNEMENT/ CIRM2013/Optim_ENR_CIRM2013/

(119)

References

Bacaud, L., Lemar´echal, C., Renaud, A., and Sagastiz´abal, C. A. (2001). Bundle methods in stochastic optimal power management: A disaggregated approach using preconditioner.Computational Optimization and Applications, 20(3):227–244.

Barty, K. (2004).Contributions à la discrétisation des contraintes de mesurabilité pour les problèmes d’optimisation stochastique. Thèse de doctorat, École Nationale des Ponts et Chaussées.

Barty, K., Carpentier, P., and Girardeau, P. (2010). Decomposition of large-scale stochastic optimal control problems.RAIRO Operations Research, 44(3):167–183.

Bellman, R. (1957).Dynamic Programming. Princeton University Press, New Jersey.

Bellman, R. and Dreyfus, S. E. (1959). Functional approximations and dynamic programming.Math tables and other aides to computation, 13:247–251.

Bertsekas, D. P. (2000).Dynamic Programming and Optimal Control. Athena Scientific, 2 edition. Bertsekas, D. P. and Tsitsiklis, J. N. (1996).Neuro-Dynamic Programming. Athena Scientific. Birge, J. R. and Louveaux, F. V. (1997).Introduction to Stochastic Programming. Springer Series in

Operations Research. Springer Verlag, New York.

Carpentier, P., Cohen, G., and Culioli, J.-C. (1995). Stochastic optimal control and decomposition-coordination methods. In Durier, R. and Michelot, C., editors,Recent Developments in Optimization. Lecture Notes in Economics and Mathematical Systems, 429, pages 72–103. Springer Verlag, Berlin.

de Farias, D. P. and Van Roy, B. (2003). The Linear Programming Approach to Approximate Dynamic Programming.Oper. Res., 51(6):850–856.

Dupaˇcová, J., Gröwe-Kuska, N., and Römisch, W. (2003). Scenario reduction in stochastic programming. An approach using probability metrics.Mathematical Programming, 95(3):493–511.

Gal, S. (1989). The Parameter Iteration Method in Dynamic ProgrammingManagement Science, 35(6):675–684.

Girardeau, P. (2010). A comparison of sample-based Stochastic Optimal Control methods. Stochastic Programming E-Print Series,http://www.speps.org. submitted to Optimization and Engineering. Heitsch, H. and R¨omisch, W. (2003). Scenario Reduction Algorithms in Stochastic Programming.

Computational Optimization and Applications, 24:187–206.

http://www.speps.org.