Approximate multi-parametric programming based B&B algorithm for MINLPs

ContentslistsavailableatSciVerseScienceDirect

Computers

and

Chemical

Engineering

jo u r n al h om ep a ge : ww w . e l s e v i e r . c o m / l o c a t e / c o m p c h e m e n g

Approximate

multi-parametric

programming

based

B&B

algorithm

for

MINLPs

Taoufiq

Gueddar, Vivek

Dua

∗

CentreforProcessSystemsEngineering,DepartmentofChemicalEngineering,UniversityCollegeLondon,TorringtonPlace,LondonWC1E7JE,UnitedKingdom

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received5October2011

Receivedinrevisedform5March2012 Accepted8March2012

Available online 22 March 2012 Keywords:

Parametricprogramming Branch&Bound MINLP

a

b

s

t

r

a

c

t

InthisworkanimprovedB&BalgorithmforMINLPsisproposed.Thebasicideaoftheproposedalgorithm

istotreatbinaryvariablesasparametersandobtainthesolutionoftheresultingmulti-parametricNLP

(mp-NLP)asafunctionofthebinaryvariables,relaxedascontinuousvariables,attherootnodeofthe

searchtree.Itisrecognizedthatsolvingthemp-NLPattherootnodecanbemorecomputationally

expen-sivethanexhaustivelyenumeratingalltheterminalnodesofthetree.Therefore,onlyalocalapproximate

parametricsolution,andnotacompletemapoftheparametricsolution,isobtainedanditisthenused

toguidethesearchinthetree.

© 2012 Elsevier Ltd. All rights reserved.

1. Backgroundandproblemformulation

Theprocesssynthesisarearemainsa veryimportantsubject

withinchemicalprocessdesign andoptimizationresearchfield.

Theneedforoptimizingplantconfigurationsandflowsheet

struc-turesisevenmorecriticalnowwithconstantlytighteningmarket

conditionsandcommercialspecificationsfortheproducts.Thereis

alsomoreconcernforenvironmentalissuesandfortheabilityand

theflexibilityofproducingawiderangeofproducts.Specialclasses

of process synthesis problems include: heat recovery systems,

heat and mass exchangersnetwork synthesis, utilities systems

andseparationprocesses.Theoverallprocesssynthesisproblem

aimstofindthebestconfiguration(selectionofunitsand

inter-actionbetweenthedifferentblocksoftheprocessflowsheet)that

allowsonetotransformtherawmaterialsintothedesiredproducts

whilstmeetingthespecifiedperformancescriteriaofmaximum

profit,minimumoperatingcost,energyefficiencyandgood

oper-abilitywithrespecttoflexibility,controllability,reliability,safety

and environmentalregulations. Processsynthesis problemscan

bemodeledasaMixedIntegerNon-linearProgramming(MINLP)

problem(Grossmann,1996;Grossmann&Daichendt,1996).The

nonlineartermsin theMINLPareusuallydue tothenon-linear

natureofthechemicalprocessesandenergynetworks,wherethe

integervariablescanrepresentexistenceornotofaprocessunitor

aheatexchanger,existenceoftraysinadistillationtower,routing

possibilitiesofaby-producttoafinalblend,etc.Theseproblems

∗ Correspondingauthor.Tel.:+4402076790002;fax:+4402073832348. E-mailaddress:[email protected](V.Dua).

canalsobeformulatedasMINLPproblems(Kallrath,2000).

Stabil-ityanalysisofnonlinearmodelpredictivecontrolproblems(Dua,

2006)andoptimalconfigurationofartificialneuralnetwork

prob-lems(Dua,2010)canalsobeformulatedasMINLPs.

InthisworkanimprovedBranch&Bound(B&B)algorithmfor

MINLPs ispresentedand itsperformanceisanalyzed.The

algo-rithmisbaseduponthefundamentalsofparametricprogramming.

Consideringtheintegervariablesasparametersandsolvingatthe

rootnode,theobjectivefunction,Lagrangianfunction,andthe

con-tinuous variablesare approximatedas afunctionof theinteger

variables. Using themulti-parametric programmingframework,

theapproximateobjectiveandLagrangianfunctionsarethen

eval-uatedattheterminalnodesandusedtoguidethesearchinthetree.

Theproposedapproachedimprovesthecomputationaltime

com-paredtostandardtraditionalBranch&Boundalgorithmthrough

reduction of number ofnodes exploredin thesearch tree.The

detailsofthealgorithmarediscussednext.

1.1. Mixed-IntegerNonlinearProgramming

Consider the following Mixed-Integer Nonlinear Program

(MINLP),problemP1: z1=min x,y f(x,y) subject to: h(x,y)=0 g(x,y)≤0 x∈nx y∈{0,1}ny

Inthisformulationxisavectorofcontinuousvariables,yisa

vectorofbinaryvariables,hisavectorofequalityconstraints,gisa

0098-1354/$–seefrontmatter © 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.compchemeng.2012.03.001

Table1

mp-LPsolutionoftheMILP.

i CRi _{Optimalsolutiongivenbyz(y}

1,y2)andx(y1,y2) 1 −827.59y1+2103.45y2≤896.55 z(y1,y2)=27931.04y1+43758.63y2+286758.6 0≤y1≤1 x1(y1,y2)=10344.83y1−3793.10y2+26206.897 0≤y2 x2(y1,y2)=−5172.41y1+6896.55y2+6896.552 2 −827.59y1+2103.45y2≥896.55 z(y1,y2)=45147.55y1+305409.86 0≤y1≤1 x1(y1,y2)=8852.459y1+24590.164 y2≤1 x2(y1,y2)=−2459.016y1+9836.065

vectorofinequalityconstraintsandfisthescalarobjectivefunction. Typically,themassandenergybalancesaregivenbyhandprocess specificationbyg.

ThedeterministicalgorithmsforMINLPscanbebroadly classi-fiedasthosebasedupondecompositionprinciplesandBranch& Bound(B&B)techniques(Bonamietal.,2008;Floudas,1995).Two

mostcommonlyuseddecompositionalgorithmsarebasedupon

GeneralizedBendersDecomposition(GBD)(Geoffrion,1972)and

OuterApproximation(OA)(Duran&Grossmann,1986).IntheGBD

andtheOAalgorithmsasequenceofiteratingprimalandmaster

sub-problemsisconstructedthatconvergesinafinitenumberof

iterations.Thesequenceofprimalsub-problemsrepresents

non-increasingupperboundsandthesequenceofmastersub-problems

representsnon-decreasinglowerbounds.Theprimalsub-problem

isformulatedbyfixingthebinaryvariablesresultingina

nonlin-earprogram(NLP).ThemaindifferencebetweentheGBDandthe

OAalgorithmisintheformulationofthemastersub-problem.In

theGBDalgorithmthemastersub-problemisbasedupon

dual-itytheorywhereasintheOAalgorithmthemastersub-problem

isobtainedbylinearizingtheconstraintsandtheobjective

func-tion.IntheGBDandtheOAalgorithmsthemastersub-problemsis

formulatedasamixed-integerlinearprogram(MILP).Ithasbeen

shownthat thelower boundgenerated bytheOA algorithm is

greaterthanorequaltothatgeneratedbytheGBDalgorithm.The

OAalgorithmthereforetakesfeweriterationsthantheGBDto

con-vergeand hasbeensuccessfully appliedtoseveralprocess and

productdesign problems.Decompositionalgorithm basedupon

simplicalapproximation(Goyal&Ierapetritou,2004)andcutting

planemethodsrequiringrepetitivesolutionofMILPshavealsobeen

presented(Westerlund&Pettersson,1995).

B&B algorithms are based upon a systematic tree search

methodology (Borchers & Mitchell, 1994; Gupta & Ravindran,

1985).At therootnodeof thetreeall thebinaryvariables are

relaxedascontinuousvariables,attheterminalnodesallthebinary

variablesarefixedandattheintermediatenodessomeofthebinary

variablesarefixedandtheremainingonesarerelaxed.The

prob-lemateachnodeofthetreecorrespondstoanNLP;thesolutionat

therootnoderepresentsalowerboundandataterminalnode

rep-resentsanupperboundonthesolution.TheefficiencyoftheB&B

algorithmsdependsuponenumeratingasfewnodesaspossible.

Thedecisionofwhethertoenumerateorfathomanodedepends

uponthesolutionobtainedatitspredecessornodeandthebest

upperboundthatisavailable.

1.2. Multi-parametricprogramming

Considerthefollowing multi-parametricNonlinear

Program-ming (mp-NLP) problem (Pistikopoulos, 2009; Pistikopoulos,

Georgiadis,&Dua,2007a,2007b),problemP2:

z2(␪)=min x f(x,␪) subject to: h(x,␪)=0 g(x,␪)≤0 x_∈nx ␪∈n_␪

Parametricprogrammingprovidesx*,theoptimalvalueofx,as

asetofexplicitfunctionsof␪ withoutexhaustivelyenumerating

theentirespaceof␪,theregionswheretheseexplicitfunctions

arevalidareknownascriticalregions(CRs).Forthesolutionofthe

mp-NLPs,thenonlineartermsareouter-approximatedand

multi-parametriclinearprogram(mp-LP)isformulatedandsolved.The

pointsinthespaceof␪wherethedifferencebetweenthe

solu-tionoftheNLPandthemp-LPismaximumareidentifiedandat

thosepointsmp-LPsareformulatedandsolved.Thisprocedureis

repeateduntilthisdifferenceiswithinacertaintolerance(Dua&

Pistikopoulos,1999).

InthenextsectionanewB&BalgorithmforsolvingP1based

uponparametricprogrammingispresented,inSection3

illustra-tiveexamplesarepresentedandconcludingremarksarepresented

inSection4.

2. Multi-parametricprogrammingbasedB&BAlgorithm forMINLPs

InthisworktheMINLP(problemP1)isreformulatedasan

mp-NLP(problemP2)byrelaxingthebinaryvariables,y,ascontinuous

variablesboundedbetween0and1andtreatingyasparameters,

problemP3: z3(y)=min x f(x,y) subject to: h(x,y)=0 g(x,y)≤0 x∈nx y∈[0,1]ny

ThesolutionofproblemP3providestheobjectivefunction,z3,

andthecontinuousvariables,x,asafunctionofygivenbyz3(y)

andx(y)respectively.Theoptimalsolutioncanthenbeobtained

byfixingallthepossiblecombinationsofyandevaluatingz3(y)

throughsimplefunctionevaluationsatthosefixedvaluesandthen

selectingthebestsolution.

2.1. Motivatingexample

ConsiderthefollowingMILP,formulatedasanmp-LP:

z(y1,y2)=max x 8.1x1+10.8x2 subject to: 0.80x1+0.44x2≤24000+6000y1 0.05x1+0.10x2≤2000+500y2 0.10x1+0.36x2≤6000 0≤y1,y2≤1 (1)

The solutionof this problemis given in Table 1. Evaluating

z(y1,y2)byfixingy1andy2atthebinaryvaluesgivesz(0,0)=2.87E5,

z(0,1)=3.05E5,z(1,0)=3.15E5,z(1,1)=3.51E5.Theoptimalsolution

oftheMILPisthereforegivenbyz=3.51E5,y1=1,y2=1,x1=3.34E4

andx2=7.38E3.

Thisapproachingeneralcanbemorecomputationally

expen-sivethanexhaustivelysolvingtheMILPorMINLPforallthepossible

fixedvalues of y.In the nextsectionan algorithm based upon

solutionattheterminalnodesoftheB&Btreefromthesolution

attherootnodeandintermediatenodesispresented.

2.2. Approximateparametricprogramming

Inthisworkacompleteparametricsolutionprofileofproblem

P3isnotobtainedandinsteadanapproximateparametricsolution

ofproblemP3isobtained.Theseapproximationsareobtainedby

solvingtheNLPinP3foracertainvalueofy,attherootnodeof

theB&Btree,andaregivenbyexplicitfunctionsofy.Evaluating

theseapproximatesolutionsattheterminalnodesoftheB&Btree

providesanestimateofthesolutionoftheoriginalMINLPforfixed

binaryvaluesofy.Theseestimatedsolutionsarethenranked,based

uponwhetherthesolutionisfeasibleaswellasthevaluesofthe

estimates.ThisrankingisthenusedtoguidethesearchintheB&B

tree– soastomakedecisionswhichnodestofathomandwhich

onestobranchon.

TheNLPproblemforfixedintegervariabley=ykisformulated

asproblemP4: zk UB=min_x f(x,yk) subject to: h(x,yk)=0 g(x,yk)≤0 x∈nx

Thesolutionofthisproblemprovidesanupperboundonthe

solutionofP1.Assumingthatf,gandharetwicecontinuously

dif-ferentiableinxthefirst-orderKKTconditionsforproblemP4are

givenasfollows: L(x,yk)=

∇

f(x,yk)+ p



i=1 i

∇

gi(x,yk)+ q



j=1 j

∇

hj(x,yk) i

∇

gi(x,yk)=0,i≥0,

∀

i=1,...,p hj(x,yk)=0,

∀

j=1,...,q

∇

L(x,yk)=0 (2)

In(2)ListheLagrangianfunction,itheLagrangianmultipliers

fortheinequalityconstraintsandjtheLagrangianmultipliersfor

theequalityconstraints.

Theorem1. BasicSensitivityTheorem(Fiacco,1976;Jackson& McCormick, 1988). Let y0 be a vector of binary variables

con-sideredasparametervaluesand(x0,␭0,␮0)aKKTtriplewhere

␭0isnon-negativeandx0 isfeasible.Alsoassumethat:(i)strict

complementaryslackness(SCS)holds;(ii)thebindingconstraint

gradients are linearly independent (LICQ: LinearIndependence

Constraint Qualification); and (iii) the second-order sufficiency

conditions(SOSC)hold.Then,inneighbourhoodofy0,thereexistsa

unique,oncecontinuouslydifferentiablefunction[x(y),␭(y),␮(y)]

satisfying the SOSC of problem P4 with [x(y₀),␭(y0),␮(y0)]=

(x0,␭0,␮0),wherex(y)isauniqueisolatedminimizerforP4and:

U=−(M0)−1N0 (3) U=

⎛

⎜

⎜

⎜

⎝

dx(y₀) dy d␭(y0) dy d_␮(y0) dy

⎞

⎟

⎟

⎟

⎠

(4) M0=

⎛

⎜

⎜

⎜

⎜

⎜

⎜

⎜

⎜

⎜

⎝

∇

2_L

_∇

_g 1 ···

∇

gp

∇

h1 ···

∇

hq −1

∇

Tg1 −g1 . . . . .. −p

∇

Tgp −gp

∇

T_h 1 . . .

∇

T_h q

⎞

⎟

⎟

⎟

⎟

⎟

⎟

⎟

⎟

⎟

⎠

(5) where L(x,␭,␮,y)=f(x,y)+ p



i=1 igi(x,y)+ q



j=1 jhj(x,y) N0=(

∇

2yxL·−1

∇

Tyg1,...,−p

∇

Tygp,

∇

Tyh1,...,

∇

Tyhq)T (6)

Under the assumption of Theorem 1, the first order

Taylor–Lagrangeexpansionof[x(y),␭(y),␮(y)]Tinaneighborhood

ofy0canbestatedas:

x(y) ␭(y) ␮(y)

=

x0 ␭0 ␮0

−(M0)−1·N0·(y−y0)+o(||y||) (7)

2.3. ImprovedB&BalgorithmforMINLPs

Baseduponthetheorydescribedintheprevioussection,the

stepsoftheproposedalgorithmarepresentedasfollows:

Step1:InitializetheupperandlowerboundsoftheMINLP

prob-lem(LB=−∞,UB=+∞),thenrelaxtheintegervariablesandsolve

thecorrespondingNLPproblem(P3).Theintegervectorwillbe

consideredasavectorofcontinuousparametersbelongingtothe

[0,1]interval.IftheNLPrelaxationsubproblemisinfeasiblethen

theproblemisinfeasibleaswell.IftheNLPrelaxationsolutionis

integerthealgorithmwillterminateandthesolutionofthis

sub-problemwillbethesolutionoftheinitialMINLPproblem(P1).If

thesolutionisfractional(eitherallorsomeoftherelaxedinteger

variablesareinthe[0,1]interval)thenfurtherbranchingwilltake

place.ThelowerboundisupdatedZLB=Z*,whereZ*istheoptimal

objectivefunctionoftherelaxedMINLPproblemattherootnode.

Step2:Generatetheterminalnodesmatrix(i,j),the

dimen-sionsofthismatrixaren×2n_{,nbeingthedimensionoftheinteger}

vectorand2n_{thenumberofintegerenumerationsofthetree.Each}

columnwillrepresenta potentialintegersolutionvectorof the

MINLPproblem(P1).WhenanNLPsubproblemrelaxationleads

toafractionalsolutioninsidethetreethenthedimensionsofthe

terminalnodesmatrixbelowthisnodewillben_−kand2n−k_,where

kisthelevelwherethenodeislocatedinthetree.Forinstancek=0

correspondstotherootnodeandk=ntotheterminalnodes.

ForagivenMINLPproblemwith3integervariablestheterminal

nodesmatrixattherootnodecanbepresentedas:

n×2n=



1 0 1 0 1 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 0 0 0



Step3:Theintegervectorisconsiderednowasaparameteras

statedinproblemP3.Obtain[x(y),␭(y),␮(y)]byusingEq.(7).

Themultiparametricapproximationswillbeperformedevery

timewhennofathomingactionscanbetakeninthenode.Thelinear

sensitivitysystem’sdimensionsdependonthenumberoffractional

integervariables.InotherwordstheU(y)vectorwillhaveasmany

columnsasfractionalintegervariablesaftersolvingarelaxedNLP

subproblem(sensitivitiesfortheintegervariablesthatarealready

integerarenotrequired)

Step4:UsingtheestimatedvaluesofthevariablesfromEq.(7),

Fig.1.Structureofthetree.

theLagrangianfunctionvaluesforthedifferentterminalnodesare

estimatedasfollows:

L(x(y),␭(y),␮(y),y)=f(x(y),y)+

p



i=1 i(y)gi(x(y),y) + q



j=1 j(y)hj(x(y),y) (8)

Step5:TheideaistotransformtheconstrainedproblemP3

intoanunconstrainedproblem(Fiacco,1983;Floudas,1995).Such

transformationinvolvestheuseoftheLagrangianfunction:

L(x,y,,)=f(x,y)+ p



i=1 igi(x,y)+ q



j=1 jhj(x,y) (9)

Thetransformedunconstrainedproblemthenbecomesfinding

thestationarypointsoftheLagrangefunction.

min

x,y,,≥0L(x,y,,)=f(x,y)+

T_g(x,y)+T_{h(x,y) (10)}

Therefore,aftercalculationoftheLagrangiansfunctionvalues

attheterminalnodes,theyareranked(fromminimumto

maxi-mumvalue),thebestcandidateisselectedcorrespondingtothe

minimumLagrangianfunctionvalue.Theintegervariablevalues

correspondingtotheselectedcandidatearefixedandtheNLP

sub-problemisthensolved(P4).IftheNLPisfeasibleitwillprovidean

upperboundtothetree,ifnot,thenextcandidateswillbeselected

successivelyuntilafeasiblesolutionisreached.Thisnodewillbe

usedtodirectthebranchinginthetree.

Step6:Ingeneral,whentherelaxedNLPsubproblemissolvedat

treelevelkandatthe2n−k_{terminalnodestheLagrangianfunctions}

areestimatedandifk,m(i)=ym(i)isthebestcandidateinteger

vec-toritissolvedasanNLP(asinStep5)thenthenextstepistocheck

itsoptimality.Thisisachievedthroughexploringthe

complemen-tarysectionsofthetreeasshowninFig.1.Inthisfigureatlevelk=p

arelaxedNLPproblemissolvedandtheLangrangefunction

esti-mationsoftheterminalnodesarederivedfromtherelaxednode.

TochecktheoptimalityoftheLagrangiancandidatethebranching

steptakesplaces.Explicitenumerationofalltreenodesis

prac-ticallyimpossibleduetotheexponentiallyincreasingnumberof

possiblesolutions.Theuseofboundsfortheobjectivefunctionto

beminimizedcombinedwiththevalueofthecurrentbestsolution

enablesthealgorithmtosearchsectionsofthesolutionspaceonly

implicitly.ForinstancefromtheoriginalrelaxedNLPaconstraint

(y(i=p)=|1−p,m(i)|)isaddedtoperformthebranching.

Thebranchednodesfallintoaclassofproblemsthat canbe

formulatedasProblemsPk(p)(pisatreelevelbetweenkandn):

zp(y)=min x f(x,y) subject to: h(x,y)=0 g(x,y)≤0 x∈nx y(i)∈[0,1]ny _{for i>p}

y(i)_=|1−k,m(i)| for i=p

y(i)=k,m(i) for i<p

Wecanclearlyverifythatthesub-problems{Pk(p),k≤p≤n}are

asubdivisionoftheinitialsolutionspaceofproblemP1asthey

satisfythefollowingconditions:

• Afeasiblesolutionofanyofthesub-problems{Pk(p),k≤p≤n}is

afeasiblesolutionof(P1)

• Everyfeasiblesolutionof(P1)isafeasiblesolutionofexactlyone

ofthesub-problems.

Thisimplies that thesub problems Pk(p)are disjoint, hence

thesamefeasiblesolutionappearingindifferentsubspacesofthe

searchtreeisavoided.

Step7:whenallthetreenodesareexploredthealgorithm

ter-minates.Figs.1and2providemorein-depthinformationtohelp

understandthealgorithmbetter.

Thedetailed stepsof theproposedimprovedB&Balgorithm

outlinedinthissectioncanbesummarizedintheboxbelow:

Algorithmstatement

Step1:Solve(P3)bytreatingyasfreevariableinthe[0,1]

interval

Step2:Generateterminalnodesmatrix.Eachcolumnwill

rep-resentapotentialintegersolutionvectoroftheMINLPproblem

Step3:Obtain[x(y),␭(y),␮(y)]fromEq.(7)

Step4:EvaluateL(x(y),␭,␮)attheterminalnodesbyusing

Eq.(8)

Step5:RanktheLagrangianfunctions,selectthebest

can-didateintegersolutionandforthiscandidatesolvetheNLP

(P4).Iftheselectedcandidateisinfeasiblethengotothenext

candidate.

Step6:Solvesubsetnodesfortherelaxedintegervectorand

fixingthebranchingvariables(11)andapplyfathomingwhen

criteriabelowapply.

Criteria1:Theoptimalvalueofthesubproblemis_≥Z*(Z*is

thebestintegersolutionfound)

Criteria2:TheNLPsubproblemisinfeasible

Criteria3:TheoptimalsolutionoftheNLPsubproblemis

integer

Ifanodeisfathomedthenselectnewnode.Ifanodecannot

befathomedyetgotostep3andrepeatstep3–5.

Step7:programstopswhenthewholetreespaceisexplored

Examplesfromtheliteraturewherethisapproximate

paramet-ricprogrammingapproachhasbeenappliedarepresentednextto

demonstratethemainideasoftheapproach.

3. Illustrativeexamples

3.1. Example1:linearbinaryvariable

Thisexampleisrelativelysimplebuthelpstodemonstratethe

basicconceptsoftheproposedalgorithm.Theproblemhasone

binaryvariableandonecontinuousvariableandisdescribedby

theformulationbelow:

z=min x −2.7y+x 2 subject to: 1)_{−ln(1+x)+y≤0} 2)_{−ln(x−0.57)+y−1.1≤0} 3)_x≤2 4)_−x≤0 y∈{0,1}

The NLP relaxation is solved to get z0=−0.479, x0=1.35,

y0=0.856,*1=0.869,*2=1.831,*3=0.000and*4=0.000.

Theintegervariableisfixedtotherelaxedvaluey=y0andsolved

toobtainthesensitivities.Theorem1isusedtoobtainthe

para-metricsensitivitiesofthecontinuousvariablesandtheLagrangian

multipliers

x(y+y)=x∗+∂x_∂yy and (y+y)=∗+∂_∂yy

∂x ∂y=2.353, ∂1 ∂y =18.449, ∂2 ∂y =3.336, ∂3 ∂y =0, ∂4 ∂y =0 x(y)=x0+2.353(y−y0) 1=1,0+18.449(y−y0) 2=2,0+3.336(y−y0) 3=3,0=0 4=4,0=0

TheLagrangianfunctionisconstructedusingtheapproximate

parametricfunctionsforxand

L(y)=−0.479−0.58(y−0.856)+(0.869+18.449(y−0.856))

×(ln(2.35+2.353(y−0.856))+y)

+(1.83+3.336(y−0.856))(−ln(0.78+2.353(y−0.856))

+y−1.1)

TheLagrangiansareestimatedfortheintegersolutions

candi-dates(nooptimizationproblemissolved,itisafunctionevaluation

only).L(y=1)=−1.028whereL(y=0)isundefinedbecauseof

unac-ceptablevaluesforthelogfunctionL(y=1)=−1.056,L(y=0)=

UNDEF.ThebestLagrangianwasobtainedforthecandidate(y=1),

anNLPproblemisthensolvedbyfixingy=1andtheoptimal

solu-tionisZ∗=0.2525,x∗=1.72,y∗=1.

Thebranchingprocedureisrelativelysimpleinthiscaseand

consistsofsolvingtheproblemfory=0inordertocoverthewhole

feasibilityspace.Fory=0,Z*=0.815>0.2525.Thereforethis

solu-tioncanbepruned.

FinallythesolutiontothisproblemisZ*=0.2525,x*=1.72,y*=1.

3.2. Example2:quadraticbinaryvariable

z=min x −2.7y 2_+x2 subject to: −ln(1+x)+y2_≤0 ln(x−0.57)+y2_{−1.5y−1.1≤0} x≤2 −x≤0 y∈{0,1}

The NLP relaxation is solved to get z0=−0.553, x0=0.760,

y0=0.752,*1=2.000,*2=0.073,*3=0.000and*4=0.000.

Theintegervariableisfixedtotherelaxedvaluey=y0andsolved

toobtainthesensitivities.Theorem1isusedinordertoobtain

theparametricsensitivities ofthecontinuousvariables and the

Lagrangianmultipliers

Fig.2.FlowchartoftheimprovedB&Balgorithm. ∂x ∂y =−2.646, ∂1 ∂y =11.634, ∂2 ∂y =−0.921, ∂3 ∂y =0, ∂4 ∂y =0 x(y)=x0−2.646(y−y0) 1=1,0+11.634(y−y0) 2=2,0−0.921(y−y0) 3=3,0=0 4=4,0=0

TheLagrangianfunctionisconstructedusingtheapproximate

parametricfunctionsforxand

L(y)=−0.553+3.006(y−0.752)+(2.000+11.634(y−0.752))

×(−ln(1.76−2.646(y−0.752))+y2₎

+(0.073−0.921(y−0.752))(−ln(0.19−2.646(y−0.752)

TheLagrangiansareestimatedfortheintegersolutions

candi-dates(nooptimizationproblemissolved,itisafunctionevaluation

only).L(y=1)isundefinedbecauseofunacceptablevaluesforthe

logfunction,L(y=0)=9.189.ThebestLagrangianwasobtainedfor

thecandidate(y=0),anNLPproblemisthensolvedbyfixingy=0:

Z*=0.815,x*=0.903,y*=0.SimilarlytoExample1thebranching

procedureissimpleinthiscaseandconsistsofsolvingthe

prob-lemfory=1inordertocoverthewholefeasibilityspace.Fory=1,

Z*=0.952>0.815.Thereforethissolutioncanbepruned.Finallythe

solutiontothisproblemisZ*=0.815,x*=0.903,y*=0.

3.3. Example3:ProcesssynthesisoptimizationI(Duran&

Grossmann,1986) z=min 5y1+8y3+6y2+10x1−7x3−18ln(x2+1)−19.2 ×ln(x1−x2+1)+10 Subject to : −0.8ln(x2+1)+0.96ln(x1−x2+1)+0.8x3≤ 0 ln(x2+1)−1.2ln(x1−x2+1)+x3+2y3−2≤ 0 x2−x1≤0 x2−2y1≤ 0 x1−x2−2y2≤ 0 y1+y2−1≤ 0 y∈{0,1}3_{,a≤X≤b,X={x} j,j=1,2,3},aT=(0,0,0), bT_=(2,2,1) Step1:RelaxedNLP

_x∗ 1 x∗ 2 x∗ 3

=

1.147 0.547 1.000

,

_y∗ 1 y∗ 2 y∗ 3

=

0.273 0.300 0.000

,z∗=0.759

Step2:Generatetheterminalnodesmatrix

n,2n=



1 0 1 0 1 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 0 0 0



Numberofrows=dimensionoftheintegervectorn;numberof

columns=2n

Therootnodeissolvedwithz=0.759.

Eqs.(7)and(8)areusedtoestimateLagrangiansfornodes#8–15

(theyareonlyestimatedandarenotsolved)–seeFig.3.Thebest

estimateisforterminalnode#13.Thisnodeissolvedandgives

Z=6.010providinganupperboundtotheproblem.

Thebranchingstep startsbysolving node#2byaddingthe

constraintbelowasstatedin(11):

y1=|1−1,13(2)|=|1−0|=1.

0_≤y2≤1

0≤y3≤1

Theobjectivefunctionisgreaterthanthebestupperboundso

faritisthenfathomed.Similarlythenextnodeexploredisnode#7

byaddingtheconstraints:

y1=1,13(1)=0.

y2=|1−2,13(2)|=|1−1|=0.

0≤y3≤1

whichgivesgreaterobjectivefunctionthantheupperbound.Itis

thenfathomed.Thenextonetobeexploredisnode#12which

yieldsanobjectivefunctionZ=14.010givesalsogreaterobjective

functionthanthecurrentupperboundandisthenfathomed.

Thesolutionisfinallynode#13y=(0,1,0).

3.4. Example4:ProcesssynthesisoptimizationII(Duran&

Grossmann,1986)

Consider the following MINLP example problem (Duran &

Grossmann,1986): z=min 5y1+6y3+8y2+10y4+6y5−10x1−15x2−15x5 +15x3 +5x4−20x6+exp(x1)+exp(x2/1.2)−60 ln(x3+x4+1)+140 Subject to : −ln(x3+x4+1)≤ 0 −x1−x2−2x5+x3+2x6≤ 0 −x1−x2−0.75x5+x3+2x6≤ 0 x5−x6≤0 2x5−x3−2x6≤ 0 −0.5x3+x4≤ 0 0.2x3−x4≤ 0 exp(x1)−10y1≤ 1 exp(x2/1.2)−10y2≤ 1 1.25x5−10y3≤ 0 x3+x4−10y4≤ 0 −2x5+2x6−10y5≤ 0 y4+y5≤ 1 y1+y2=1 y∈{0,1}5_{,a≤X≤b,X={x} j,j=1,2,3,4,5,6}, aT_{=(0,0,0,0,0,0),b}T_{=(2,2,−,−,2,3)}

AB&BtreedepictingthepathtakeninthetreeisshowninFig.4.

Thedarkcolourednodesarethenodesthatwereexploredandlight

colourednodeswerenotexplored.Thenodesarenumberedfor

reference,therootnodeisnumberedas1andtheterminalnodes

arenumberedfrom32to63.

TheapproximateparametricsolutionoftheNLPatnode1is

usedtoestimatethesolutionatnodes32to63.Estimatesatnodes

32–45giveinfeasiblesolution.

Therootnodeissolvedandgivesz=−0.554andy=(0.571,0.429,

0.250,0.210,0).

Thesolutionatthis nodeisusedtoestimateLagrangiansfor

terminal nodes (#32–63),thesenodes are notsolved. Thebest

Lagrangianbasedcandidateisnode#47y(0,1,1,1,1),thisnodeis

thensolvedbutisinfeasible,thereforethenextbestLagrangian

basedcandidateisselected.ThesecondbestLagrangianbased

can-didateisnode#46y=1,46=(0,1,1,1,0),thisnodeisthensolvedto

provideanupperboundfortheproblem(Z=73.03).

Node#46isthenastartingpointforthebranchingstepinorder

toexplorethewholefeasibilityspace.Tobeabletocheckthe

opti-mality,ornot,ofthebestlagrangiancandidate,thecomplementary

sectionsofthetreewillhavetobeexplored.

Node#3issolvedasanNLPrelaxationbyaddingthefollowing

constraints:

y1=|1−1,46(1)|=|1−0|=1.

0≤y2≤1,0≤y31,0≤y4≤1,0≤y5≤1

Itleadstoafractionalsolutionwithanobjectivefunctionof

z=67.9whichmeannofathomingcriteriacanbeappliedyet.

Fur-therbranchingshouldtakeplacefromthisnode.Thesameconcept

isthenappliedtothispartofthetreetoestimatesolutionsatthe

terminalnodes48–63.

Thebestcandidate(node#54)issolvedwithanobjective

func-tionofz=82.13andy=2,54=(1,0,1,1,0).Thisnodeisinitsturna

startingpointforsub-branchinginthesectionoftreeboundedby

node#3andnodes#63–48.

Thenodethatisfirstexploredisnode#7throughtheaddition

Fig.3.ApproximateparametricprogrammingbasedB&BtreeforExample3.

y1sameasfornode#3

y2=|1−2,50(3)|=|1−0|=1.

0≤y3≤1,0≤y4≤1,0≤y5≤1

Thisnodeyieldsaninfeasiblesolutionitwillthenbefathomed.

Thenextnodetobeexploredisnode#12throughtheadditionof

thefollowingconstraints:

y1sameasfornode#3

y2=2,54(2)

Y3=|1−2,54(3)|=|1−1|=0.

0≤y4≤1,0≤y5≤1

Thisnodeyieldsanobjectivefunction(Z=86.45)thatisgreater

thanthecurrentbestbound(73.03)itwillthenbefathomed.The

nextonetobeexploredisnode#26:

Table2

Computationalresultsforexample2.

Algorithm TraditionalBranch&Bound OA ImprovedB&B

NumberofMIPcalls – 3MIP –

NumberofNLPcalls 4NLP 3NLP 3NLP

Solutionfoundin 2ndnode – 2ndnode

Table3

Computationalresultsforexample3.

Algorithm TraditionalBranch&Bound OA ImprovedB&B

NumberofMIPcalls – 3MIP –

NumberofNLPcalls 6NLP 4NLP 5NLP

Solutionfoundin 2ndnode – 2ndnode

CPUtime(s) 0.064 0.613 0.051

Table4

Computationalresultsforexample4.

Algorithm TraditionalBranch&Bound OA ImprovedB&B

NumberofMIPcalls – 7MIP –

NumberofNLPcalls 15NLP 7NLP 12NLP

Solutionfoundin 12thnode 6thmajoriteration 1stnode

CPUtime(s) 0.141 1.224 0.080

Table5

Computationalresultsforexample4. B&Balgorithm Depthfirst

search(DFS)

Bestbound (BB)

Bestestimate (BE)

DFS/BBmix DFS/BEmix DFS/BB/BE

mix

Automatic Improved

B&B

NumberofNLPcalls 22 8 8 15 12 12 15 12

Solutionfoundin 20thnode 5thnode 6thnode 12thnode 7thnode 7thnode 12thnode 1stnode

Obj 73.03 73.03 73.03 74.29 73.03 73.03 74.29 73.03

CPUtime(s) 0.157 0.074 0.081 0.128 0.105 0.110 0.128 0.080

y1sameasfornode#3 y2=2,54(2)

y3=2,54(3)

y4=|1−2,54(4)|=|1−1|=0. 0≤y5≤1

Thisnodeyieldsanobjectivefunction(Z=83.39)thatisgreater thanthecurrentbestbound(73.03)itwillalsothenbefathomed. Thenextonetobeexploredisnode#55:

y1sameasfornode#3 y2=2,50(2)

y3=2,50(3) y4=2,50(3)

y5=|1−2,50(5)|=|1−0|=1.

Thisnodeyieldsaninfeasiblesolutionitwillthenbefathomed Hence,thesectionoftreeboundedbynode#3andnodes#63–48 willbefathomed.Theinitialbranchingprocedurewhichstarted fromtheterminalnode#46cannowcarryon.Therefore,nowthat thesectionbelownode3iscleared,thenextnodetoexploreisnow node#4.

y1=1,46(1)=0

y2=|1−1,46(2)|=|1−1|=0. 0≤y3≤1,0≤y4≤1,0≤y5≤1

ThisNLPproblemisinfeasiblehencefathomed.Thenextoneis node#10

y1=1,46(1)=0 y2=1,46(2)=1

y3=|1−1,46(3)|=|1−0|=1. 0≤y4≤1,0≤y5≤1

Thisnodeyieldsanobjectivefunction(Z=79.12)thatisgreater thanthecurrentbestbound(73.03)itwillbefathomed.

Thelastnodetobeexploredisnode#22 y1=1,46(1)=0

y2=1,46(1)=1 y3=1,46(1)=1

y4=|1−1,46(4)|=|1−1|=0. 0≤y5≤1

Similarly node#22(z=74.30) isalsofathomed forthesame reason.Thenode#47wassolvedasapotentialLagrangianbased candidateandhasbeenfoundinfeasible,therefore,noneedtosolve thisnodeagain,itisthenfathomedalso.Thesolutionisfinallynode #46y=(0,1,1,1,0).

4. Algorithmcomputationalresults

Inthissection,acomparisonbetweentheproposedimproved B&Balgorithm, thetraditionalBranch&Bound(GAMS/SBB)and OuterApproximation(GAMS/DICOPT)algorithmsispresentedfor examples2(Table2),3(Table3)and4(Table4).

Amoredetailedcomparisonbetween(GAMS/SBB)andthe

pro-posedimprovedB&BalgorithmforExample4isalsopresentedin

Table5.

Thecomputationalrequirementsforthe proposedalgorithm

arecomparabletoor betterthan for5 outof7 traditionalB&B

algorithms(Table5).Amoredetailedcomputationalcomparison

willbepresentedinthefuturepublications;themainobjective

ofthisworkwastopresentanddemonstrateapplicabilityofthe

basicconceptsof theproposedalgorithm. Different treesearch

techniques(e.g.DepthFirst Search(DFS))willalsobeexplored

inconjunction withthe proposedalgorithm. Thefinite

conver-genceandguaranteedoptimalityproperties,forconvexfunctions

forthe traditionalB&B algorithm,are retainedin theproposed

algorithm.

5. Concludingremarks

Anapproximateparametricprogrammingsolutionattheroot

nodeandotherfractionalnodesoftheB&Btreeareobtainedand

usedtoestimatethesolutionattheterminalnodesindifferent

sectionsofthetree.Theseestimatesarethenusedtoguidethe

searchintheB&Btree,resultinginfewernodesbeingevaluated

andreductioninthecomputationaleffort.Preliminary

computa-tionalresultsareencouraging,futureworkwillinvolvetestingthe

proposedalgorithmonlargerscaleproblemsandcomparingwith

otheralgorithmsreportedintheliterature.

Acknowledgment

Financial support from EPSRC (EP/G059195/1) is gratefully

acknowledged.

References

Bonami,P.,Biegler,L.T.,Conn,A.R.,Cornuejols,G.,Grossmann,I.E.,Laird,C.D., etal.(2008).Analgorithmicframeworkforconvexmixedintegernonlinear programs.DiscreteOptimization,5,186–204.

Borchers,B.,&Mitchell,J.E.(1994).Animprovedbranchandboundalgorithm formixed-integernonlinearprograms.ComputersandOperationsResearch,21, 359–367.

Dua,V.(2006).Stabilityanalysisofnonlinearmodelpredictivecontrol:An opti-mizationbasedapproach.In16thEuropeansymposiumoncomputeraidedprocess engineeringand9thinternationalsymposiumonprocesssystemsengineering. Ams-terdam:Elsevier.,pp.1287–1292.

Dua,V.(2010).Amixed-integerprogrammingapproachforoptimal configura-tionofartificialneuralnetworks.ChemicalEngineeringResearchandDesign,88, 55–60.

Dua,V.,&Pistikopoulos,E.N.(1999).Algorithmsforthesolutionof multipara-metricmixed-integernonlinearoptimizationproblems.Industrial&Engineering ChemistryResearch,38,3976–3987.

Duran,M.A.,&Grossmann,I.E.(1986).Anouter-approximationalgorithmfor aclassofmixed-integernonlinearprograms.MathematicalProgramming,36, 307–339.

Fiacco,A.V.(1976).Sensitivityanalysisfornonlinearprogrammingusingpenalty methods.MathematicalProgramming,10,287–311.

Fiacco,A.V.(1983).Introductiontosensitivityandstabilityanalysisinnonlinear pro-gramming.NewYork:AcademicPress.

Floudas,C.A.(1995).Nonlinearandmixed-integeroptimization:Fundamentalsand applications.OxfordUniversityPress.

Geoffrion,A.M.(1972).GeneralizedBendersDecomposition.JournalofOptimization TheoryandApplications,10,237–260.

Goyal,V., & Ierapetritou, M.G. (2004). Computational studiesusing a novel simplical-approximationbasedalgorithmforMINLPoptimization.Computers andChemicalEngineering,28,1771–1780.

Grossmann,I.E.(1996).Mixed-integeroptimizationtechniquesforalgorithmic pro-cesssynthesis.Advancesinchemicalengineering:ProcesssynthesisElsevier.,pp. 171–246.

Grossmann,I.E.,&Daichendt,M.M.(1996).Newtrendsinoptimizationbased approaches forprocesssynthesis.Computers andChemicalEngineering, 20, 665–683.

Gupta,O.K.,&Ravindran,A.(1985).Branchandboundexperimentsinconvex nonlinearintegerprogramming.ManagementScience,31,1533–1546. Jackson,R.H.F.,&McCormick,G.P.(1988).Second-ordersensitivityanalysisin

factorableprogramming:Theoryandapplications.MathematicalProgramming, 41,1–27.

Kallrath,J.(2000).Mixedintegeroptimizationinthechemicalprocessindustry: Experience,potentialandfuture.Trans.I.ChemE.PartA,78,809–822. Pistikopoulos,E.N.(2009).Perspectivesinmulti-parametricprogrammingand

explicitmodelpredictivecontrol.AIChEJournal,55,1918–1925.

Pistikopoulos,E.N.,Georgiadis,M.C.,&Dua,V.(2007a).Multi-parametric program-mingWiley-VCH.

Pistikopoulos,E.N.,Georgiadis,M.C.,&Dua,V.(2007b).Multi-parametricmodel basedcontrolWiley-VCH.

Westerlund,T.,&Pettersson,F.(1995).Anextendedcuttingplanemethodforsolving convexMINLPproblems.ComputersandChemicalEngineering,19,S131–S136.