2.5 Stochastic Programming Algorithms
2.5.1 Progressive Hedging
The Progressive Hedging (PH) algorithm for continuous stochastic problems was originally for- malised by Rockafellar and Wets [98]. Both the current name and operation of the algorithm are based on what Rockafellar and Wets term the principle of progressive hedging in optimisation under uncertainty. The essence of this principle is that by finding a solution to a stochastic prob- lem without enforcing non-anticipativity, and then gradually demanding greater conformity to the non-anticipativity constraints, a good solution will be found.
The Progressive Hedging algorithm is outlined below. A more formal definition of the algorithm is deferred until the introduction of Chapter 4.
Initialise Determine the starting values for the primal variables x0s, y0s and dual variables λ0s for all s P S. Set the consensus variables ¯x0 °sPSpsx0s. Choose a value for the penalty parameter
35 ρ. Set k 1.
The starting values for the primal variables xs and ys are obtained by finding the optimal
anticipative decisions for each scenario. It suffices to initially set the dual variables to zero. Step 1 Update x and y for each scenario:
pxk s, y k sq P argmin px,yqPXs Lρpx, y, ¯xk1, λk1q @s P S
Note that since ¯xk1 is constant the parts of the augmented Lagrangian which were non-
separable are now constant and can be ignored. Therefore, this minimisation is separable into subproblems, each of which contains the variables and constraints from one scenario. This greatly decreases the computational difficulty of this step.
Step 2 Update ¯x: ¯ xk P argmin ¯ x Lρpxk, yk, ¯x, λk1q
Step 3 Update dual variables λ for each scenario:
λks λks1 psρkpxks ¯xq @s P S
Step 4 Check for convergence; if the method has not yet convereged, set k k 1 and return to Step 1.
Progressive Hedging is guaranteed to converge to the optimal primal solution when applied to a convex stochastic problem even if the subproblems used to update the primal variables are not solved exactly [98]. In particular it is possible to use a computationally simple heuristic to obtain an approximate solution to the subproblems quickly, rather than use a slower method to obtain an unnecessarily precise solution. This idea has been applied to control problems related to fishery management [57]. In practice, finding a more accurate solution to the subproblems can actually slow the convergence of the Progressive Hedging algorithm, not only in terms of computation time but also with respect to the number of iterations [65].
Since the feasible region of a SIP is non-convex by virtue of the integrality restriction, the above convergence result does not apply in the SIP context. In an intuitive sense, taking the probability- weighted average of the first-stage decisions is problematic for an integer problem. If any first-stage
variables are integer-valued, their average is unlikely to be an integer itself. Even if all of the first- stage variables are integer-valued, the integer-valued second-stage variables cause the second-stage problem (2.3) to be non-convex, which means that the objective function of ζSIP (as defined in
(2.2)) is non-convex.
Progressive Hedging may be used to solve the continuous relaxation of an SIP with relative ease, since this is simply an ordinary continuous stochastic program. An intuitive approach to solving the SIP (and integer programs in general) is to first solve their linear relaxation and then examine feasible integer points with values near to the fractional optimal point of the relaxation. The notion of finding a “nearby point” is not always practical, especially in the case of a binary program, and in general there is no guarantee of finding an optimal or even feasible integer point in this way. Nevertheless, this approach has been applied to some problems with success [77].
The first application of Progressive Hedging to general multi-stage stochastic integer program- ming was made by Løkketangen and Woodruff [78], using tabu search to solve the integer program- ming subproblems.
Since the Progressive Hedging algorithm is not guaranteed to converge to the optimal primal solution when applied to integer problems, solving the subproblems approximately is attractive in this context also. Progressive Hedging has been used in this way to solve stochastic lot-sizing problems, solving the scenario subproblems with a dynamic programming algorithm only guaranteed to yield an optimal solution in the initialisation step [54].
Penalty Parameters
The value of the penalty parameter is especially important when applying Progressive Hedging to integer stochastic problems, since for these problems the parameter determines not only the rate of convergence and difficulty of the subproblems (as in continuous stochastic programming), but also the quality of the solution found by the algorithm [88, 77].
A downside of the conventional approach of choosing a single penalty parameter which applies to every non-anticipative constraint is that the decision variables may not all have the same scale. For example, if a problem contains a binary variable and another variable with a large range of possible values, failure to achieve consensus with respect to the binary variable will be penalised to a lesser degree, even if its value is a more important component of the policy. This downside can
37 be overcome by choosing separate penalty parameters for the non-anticipativity constraints which pertain to each decision variable.
Watson and Woodruff [112] proposed the rule
ρpiq ci
maxsPSpx0sq minsPSpx0sq 1
for choosing a penalty corresponding to each integer first-stage variable xi, based on the anticipative
first-stage decisions x0
sobtained for each scenario in the initialisation step. They proposed the similar
rule
ρpiq ci
maxp°sPSps|x0s ¯x0| , 1q
for continuous variables.
In addition, the results covered in the previous section regarding penalty parameters for ADMM are also applicable to Progressive Hedging.
Variable Fixing
Once the Progressive Hedging algorithm has obtained consensus as to the optimal value of a given decision variable, that variable may be “fixed” to that value to decrease the computational difficulty of the subproblems to be solved.
The difficult part of this procedure is to find a reliable heuristic to determine that the opti- mal value of the variable has in fact been reached. If the heuristic is too conservative, then the improvement in performance will be small. If the heuristic is too aggressive it may lock variables prematurely to a non-optimal value.
One possible approach is to wait until the value of all integer variables has achieved consensus, then fix the integer values and determine the value of the continuous variables by directly solving the IP-equivalent problem [78, 77]. Since the resulting problem is a continuous LP it can be computationally tractable despite its large size.
Alternatively, variables which have not yet achieved consensus may be “slammed” by fixing them to a consensus value which seems reasonable based on the present value of the variables. This method is easiest to apply to problems where all of the bounds on the variables are either from above or below, since in this case it is possible to guarantee admissibility of the slammed variable by taking the most pessimistic value for the variable [112].
Obtaining Lower Bounds
A downside of Progressive Hedging as applied to SIPs is that despite being a Lagrangian dual-based approach it does not directly yield lower bounds. Gade et al. [45] showed that the dual variables obtained at each step may be used to compute a lower bound for the SIP. The lower bounds are computed by solving the following mixed integer linear program for each scenario s:
Dspλq min xs,ys
pspcTxs dTsysq λTsxs
s.t.pxs, ysq P Ks
The sum°sPSDspλq is a lower bound for the SIP. This mixed-integer linear program is of comparable
complexity to the mixed-integer quadratic program solved for each scenario in each Progressive Hedging step, and requires a similar amount of computational effort to solve.
The lower bounds obtained by this method, using the dual variables from successive steps of Progressive Hedging, are not guaranteed to converge to the optimal Lagrangian dual value ζLD
when Progressive Hedging is applied to an integer problem. However, computational experiments show that in some cases tight bounds can be obtained by this method [45]. In particular, when the penalty parameter is chosen to be small the quality of the lower bound tends to improve. Unfortunately, this low penalty parameter also causes the method to converge very slowly. The developments in Chapters 4 and 5 of this work address this weakness of the Progressive Hedging algorithm.