A robust approach to the chance-constrained knapsack problem
Olivier Klopfenstein
1,2, Dritan Nace
21
France T´el´ecom R&D, 38-40 rue du gl Leclerc, 92794 Issy-les-Moux cedex 9, France
2Universit´e de Technologie de Compi`egne, Laboratoire Heudiasyc UMR CNRS 6599,
60205 Compi`egne Cedex, France.
[email protected], [email protected]
June 6, 2006
Abstract
Chance-constrained programming is a relevant model for many concrete problems. However, it is known to be very hard to tackle directly. In this paper, the chance-constrained knapsack problem (CKP) is addressed. Relying on the recent advances in robust optimization, a tractable combinatorial algorithm is proposed to solve CKP. It always provides feasible solutions for CKP. Moreover, for two specific classes of uncertain knapsack problems, it is proved to solve CKP at optimality.
1
Introduction.
Many concrete problems require to take into account uncertainty on input data. Several frameworks have been proposed these last fifty years to deal with uncertainty in optimization problems. Among them, three main families of approaches can be distinguished: sensitivity analysis, stochastic optimization and robust optimization. As already remarked by many authors, sensitivity analysis is an approach deeply different from the two others, since it does not impact the optimization process: it only tries to say how good is a given solution with respect to uncertainty (post-optimization analysis). In other words, sensitivity analysis does not propose any solution. By contrast, stochastic and robust optimization aim at providing a solution adapted to the uncertain framework considered. The first one relies on probabilistic information, supposed to be available, and tries to find a solution which is “good” in a probabilistic sense. For instance, the expected value of the objective function can be optimized with respect to the considered possible events. Robust optimization aims generally at providing solutions feasible for all the uncertain events considered.
Chance-constrained programming is a very attractive part of stochastic optimization (see for instance [9, 13]). It is devoted to finding the best solution which remains feasible with probability at least 1 − ε, for a given ε > 0. Such a model is relevant for many problems, where it is acceptable that a given solution is not feasible for all the events taken into account, as soon as the unfeasibility probability is controlled. For instance, this approach becomes particularly interesting when dealing with problems for which uncertainty assumptions remain approximative. In this case, there is often no sense for looking for a solution always feasible, since the worst case (if there is one) is very likely to never occur.
While the direct resolution of chance-constrained programs is often very difficult, some recent robust optimization frameworks bring a new way to deal with these problems [4, 8]. More precisely, given an initial set of uncertain events, the authors propose a way to relax it into a subset of events of probability not less than 1 − ε, so that the associated robust optimization problem is tractable. As a result, solving the robust problem provides a feasible solution for the chance-constrained problem. The links between chance-constrained programming and robust optimization have been underlined only recently. [10] compares both approaches and proposes an intermediate resolution framework. The authors study the randomized construction of a set of events with the following property: a solution robust to this set of events is a solution of a chance-constrained program. [12] follows the same line, while dealing with uncertainty on data probability distributions themselves. [15] shows how the robust optimization framework developed in [4, 5] provides an approximation of chance-constrained programming. [11] also proposes robust optimization as a technique to obtain feasible solutions to chance-constrained programs.
In the same spirit, we propose to use robust optimization techniques to provide a solution to chance-constrained problems. This work focuses on the knapsack problem, because of its importance for integer linear programming. When dealing with uncertain linear programs, the approach of Bertsimas and Sim [8] appears as particularly appropriate, especially because it preserves the linearity of the basis problem. Furthermore, it can be readily used for integer linear programs. Note that this approach can be seen as a particular case of the robustness model of [2, 3], extended with a probabilistic analysis. In the present paper, the links between this robustness model and chance-constrained programming are investigated for the specific case of the knapsack problem. A pseudo-polynomial time resolution algorithm is designed for the chance-constrained knapsack problem. It is proved to provide always feasible solutions, and even optimal solutions for two specific classes of uncertain knapsack problems. To the best of our knowledge, this is the first time an optimal resolution process is designed for a chance-constrained integer linear program where constraint coefficients are uncertain.
2
Problems and formulations.
2.1
The knapsack problem.
Let I = {1, . . . , n}. Given a profit vector p ∈ IRn+, a weight vector w ∈ IRn+ and a knapsack capacity
c > 0, the classical knapsack problem is:
max P i∈Ipixi s.t. P i∈Iwixi ≤ c x ∈ {0, 1}n (1) For an extensive study of the knapsack problem, we refer for instance to [14]. It is known to be NP-hard, even though there exists a pseudo-polynomial dynamic programming algorithm to solve it.
In the rest of this paper, the scalar product of two vectors a and b in IRn will sometimes be denoted
by: a.b =P
i∈Iaibi.
2.2
The Chance-constrained Knapsack Problem.
Let us suppose that the weight vector w is in fact not known with accuracy, that is, w can take values
in a set W ⊆ IR+n. Consider a probability measure P on W (P (w ∈ W ) = P (W ) = 1). The
Chance-constrained Knapsack Problem (CKP) is:
max P i∈Ipixi s.t. P (P i∈Iwixi≤ c) ≥ 1 − ε x ∈ {0, 1}n (2) where ε ≥ 0. That is, we look for the best solution x such that x remains feasible with probability at least 1 − ε. As noted in most of the introductions to chance-constrained programming, the set of
fractional points {x ∈ [0, 1]n|P (w.x ≤ c) ≥ 1 − ε} is often intractable because of multivariate integral
calculations (see e.g. [9, 13] for more details).
Example: Let us present a very simple example in IR2by describing the set: X = {x ∈ [0, 1]2|P (w1x1+
w2x2 ≤ 1) ≥ 3/4} (ε = 1/4). Suppose that both random weights w1 and w2 are uniformly distributed
on [0,1]. Let us denote d their density function: d = 1I[0,1] (characteristic function of [0,1]).
Let (x1, x2) ∈ [0, 1]2. We denote by W1,2 the random variable x1,2w1,2. Each Wi has a density
function di= 1/xi.1I[0,xi]. Then
P (w1x1+ w2x2≤ 1) = P (W1+ W2≤ 1) = Z 1 t=0 d1(t).P (W2≤ 1 − t)dt = Z x1 t=0 d1(t).P (W2≤ 1 − t)dt
Easy calculations lead to: P (W2≤ 1 − t) =
1 if t ≤ 1 − x2 (1 − t)/x2 if t ≥ 1 − x2 If x1+ x2 ≤ 1 ⇔ 1 − x2 ≥ x1, it follows that: P (w1x1+ w2x2≤ 1) = 1. If x1+ x2> 1 ⇔ 1 − x2 < x1, we obtain: P (w1x1+ w2x2≤ 1) = R1−x2 t=0 d1(t)dt + Rx1 t=1−x2d1(t).(1 − t)/x2dt = 1/x1+ 1/x2− 1/2.(x2/x1+ x1/x2) − 1/(2x1x2)
The set X is represented on Figure 1. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.2 0.4 0.6 0.8 1 proba 0.75 x_1 x_2 proba
Figure 1: Probability of solutions x ∈ [0, 1]2for the example
Hence, this very basic example shows that chance-constrained problems are quite hard to tackle in a direct way. But observe that CKP can be equivalently written:
max V ⊆W :P (V )≥1−ε max x∈{0,1}n{p.x | ∀w ∈ V, w.x ≤ c} (3) (V, x) is said to be a feasible solution of (3) if V ⊆ W , P (V ) ≥ 1 − ε and: ∀w ∈ V, w.x ≤ c. It is optimal if the profit p.x dominates this associated to any other feasible solution.
Proposition 1 x is an optimal solution of CKP if, and only if, there exists V ⊆ W of probability
P (V ) ≥ 1 − ε such that (V, x) is an optimal solution of (3).
Proof: It is sufficient to observe that x is a feasible solution of CKP if, and only if, there exists V ⊆ W
of probability P (V ) ≥ 1 − ε such that (V, x) is a feasible solution of (3). For the direct sense, consider V = {w ∈ W |w.x ≤ c}. For the reverse one, observe that P (w.x ≤ c) ≥ P (V ).
This formulation relies on robust optimization subproblems defined for subsets V of possible weights.
For any subset V of W of probability measure P (V ) ≥ 1 − ε, the resolution of maxx∈{0,1}n{p.x | ∀w ∈
V, w.x ≤ c} provides a feasible solution to CKP. This motivates the recourse to robust optimization models for solving CKP. More specifically, the aim of this study is to rely on an adaptation of the appealing framework proposed in [8] to provide good feasible solutions to CKP, and even optimal ones for some specific cases.
2.3
Solving CKP with robust models.
From now on, we assume that the coefficients wi lie in intervals [wi, wi]: wi ≥ 0 is the lowest possible
weight value for element i, while wi≥ wiis the largest one. Thus, W is included in the cartesian product
of intervals [wi, wi]. To each weight is associated a random variable (r.v.); for the sake of simplicity the
random variables and their realizations are denoted by the same symbol wi. Let us also introduce, for
all i ∈ I, the random variable ηi: ηi= (wi− wi)/(wi− wi) ∈ [0, 1].
Let Γ ∈ {0, . . . , n}, the idea of the robust formulation is to find a solution feasible even though up to Γ coefficients of w take their largest values. Thus, in particular, if Γ = 0, we consider only the best scenario, where all weights take their lowest values; if Γ = n, the worst case is taken into account. Note that [8] presents a very similar model where Γ can be non-integral: Γ ∈ [0, n]. Although this could directly be adapted here, for the sake of simplicity, our models will assume the integrality of this robustness coefficient.
Within this framework, let us introduce the Robust Knapsack Problem (RKP), parameterized by Γ: max P i∈Ipixi s.t. P i∈I\Swixi+ P i∈Swixi≤ c, ∀S ⊆ I s.t. |S| = Γ x ∈ {0, 1}n (4)
For any x ∈ {0, 1}n, we denote: I(x) = {i ∈ I|xi= 1}.
Lemma 1 For any feasible solution x of RKP: P (w.x ≤ c) ≥ P P
i∈Iηi≤ Γ.
Proof: Let x be a feasible solution of RKP. If |I(x)| ≤ Γ, there exists S ⊆ I of cardinality Γ such
that I(x) ⊆ S. Then: P
i∈Iwixi = P i∈I(x)wixi = P i∈Swixi+ P
i /∈Swixi ≤ c (since x is feasible for
RKP). This implies that: P (w.x ≤ c) = 1, and thus: P (w.x ≤ c) ≥ P (P
i∈Iηi≥ Γ). Suppose now that
|I(x)| ≥ Γ + 1. For the sake of clarity, let us denote: δi= wi− wi. For any subset S ⊆ I(x) of cardinality
Γ: P (w.x > c) ≤ P w.x ≥P i∈Swixi+ P i /∈Swixi = PP i∈I(x)wi ≥ P i∈Swi+ P i∈I(x)\Swi = PP i∈I(x)ηiδi≥ P i∈Sδi = PP i∈I(x)\Sηiδi≥ P
i∈Sδi(1 − ηi)
≤ PP
i∈I(x)\Sηiδi≥ minj∈Sδj. P
i∈S(1 − ηi)
= PP
i∈I(x)\Sηi[δi/ minj∈Sδj] + P
i∈Sηi≥ Γ
The first inequality comes from the feasibility of x for RKP. Now, we can choose S ⊆ I(x) so that:
∀i ∈ I(x) \ S, δi ≤ minj∈Sδj. In this case, we obtain: P (w.x > c) ≤ PP
i∈I(x)ηi≥ Γ
, and thus,
clearly: P (w.x > c) ≤ P P
i∈Iηi≥ Γ.
Note that this result is general, since it requires no assumption on probability distributions of weights. The robust model proposed, directly inspired by this of [8], appears as particularly relevant for approximating our chance-constrained program. First, the probability results of [8] can be adapted to our framework, for instance:
Proposition 2 Assume that the random variables wi are independent and symmetrically distributed.
Let ε ∈ (0, 1). If Γ ≥ 1/2.n +p−2n ln(ε), a feasible solution of RKP will be feasible for CKP.
Proof: Let us introduce ˜ηi = 2ηi− 1. The r.v. {˜ηi}i∈I are independent and symmetrically distributed
on [-1,1]. Suppose that Γ ≥ n/2. From the probability results of [8], we know that: P (P
i∈Iηi˜ ≥
2Γ − n) ≤ exp −(2Γ − n)2/(2n). On the other hand: P (Pi∈Iηi˜ ≥ 2Γ − n) = P (P
i∈Iηi ≥ Γ), and
thus: P (P
i∈Iηi≥ Γ) ≤ exp −(2Γ − n)2/(2n). We conclude thanks to Lemma 1.
Better, but also more complex, probability bounds are proved to exist and can lead to a better choice of Γ. Secondly, the robust set of events w associated to the set of feasible solutions of RKP (characterized in Lemma 1) seems appropriate, since in particular it discards the worst-case, where all weights take their maximal values wi. Finally, as stated in [8], the multi-knapsack formulation (4) can also be equivalently written: max x∈{0,1}n{p.x | ∃z ≥ 0, y ≥ 0 s.t. P i∈Iwixi+ Γz + P i∈Iyi≤ c }
and ∀i ∈ I, z + yi≥ (wi− wi)xi
This formulation makes the computational resolution tractable when using a branch-and-bound algo-rithm.
One of the greatest advantages of the robust framework proposed in [8] is to preserve the linearity of the initial problem. Thus, as clearly expressed by the authors in their initial paper, it is directly usable for integer programming. However, this aspect has not been extensively investigated so far. [8, 7] provide some numerical tests on the robust formulation of some classical combinatorial problems. In particular, a robust knapsack problem very close to ours is introduced, and the authors show numerically
the impact of protection level Γ on the optimal profit. But to the best of our knowledge, the previous theoretical studies on applications of [8] for robust integer programming have focused on cases where only objective coefficients are uncertain. While [7] provides some complexity results, [16] proves a probability bound easier to compute than this proposed by [8]. [1] investigates some reformulations and tight linear programming formulations for this particular case of uncertain cost coefficients. By contrast, the present paper deals with robust integer programming applied to the knapsack problem when constraint coefficients are uncertain. Finally, with respect to classical stochastic programming approaches, the present study is somewhat original since it is fully combinatorial. That is, no convex optimization tools are needed.
3
On the theoretical link between CKP and RKP.
Let I0⊆ I and Γ ∈ {0, . . . , |I0|}, we introduce the problem RKP(I0,Γ):
max p.x s.t. P i∈Swixi+ P i∈I0\Swixi+ P i /∈I0wixi≤ c, ∀S ⊆ I0 s.t. |S| = Γ x ∈ {0, 1}n (5)
RKP(I0,Γ) can be seen as a robust knapsack problem (4) with all weights of elements in I \ I0 set to
their worst possible value, with no uncertainty for these values. From now on, to be more precise, RKP
will be denoted by RKP(I,Γ). Observe that for any two subsets I0 and I00 of I, I0⊆ I00implies that any
feasible solution of RKP(I0,Γ) is also feasible for RKP(I00,Γ). On the other hand, considering Γ0 ≤ Γ,
any feasible solution of RKP(I0,Γ) is also feasible for RKP(I0,Γ0). In particular, any feasible solution of
RKP(I0,Γ) is also feasible for RKP(I,Γ).
Lemma 2 Let Γ ∈ {0, . . . , n}. Suppose that x∗ is an optimal solution of RKP(I,Γ):
(i) if Γ ≤ |I(x∗)|, x∗ is an optimal solution of RKP(I(x∗),Γ).
(ii) If Γ ≥ |I(x∗)|, x∗ is an optimal solution of RKP(I,n).
Proof: Suppose that Γ ≤ |I(x∗)|. Since I(x∗) ⊆ I, any feasible solution of RKP(I(x∗),Γ) is feasible for
RKP(I,Γ). But it can be seen that x∗ is a feasible solution of RKP(I(x∗),Γ). Then, as it is optimal for
RKP(I,Γ), it is optimal also for RKP(I(x∗),Γ).
If now Γ ≥ |I(x∗)|, there exists S ⊆ I of cardinality Γ such that I(x∗) ⊆ S. Then: P
i∈Iwix∗i = P
i∈I(x∗)wix∗i =Pi∈Swixi∗+Pi /∈Swix∗i ≤ c. The last inequality comes from the fact that x∗is feasible
for RKP(I,Γ). Thus, x∗ is shown to be a feasible solution of RKP(I,n). But since Γ ≤ n, RKP(I,Γ)
is a relaxation of RKP(I,n): x∗ is also optimal for RKP(I,n). Finally, observe that in the case where
Γ = |I(x∗)|, both statements (i) and (ii) hold. Then, RKP(I(x∗),Γ) and RKP(I,n) both admit x∗ as an
optimal solution.
The following result shows the relevance of the robust setting proposed to deal with CKP when all
weight variations (wi− wi) are identical:
Theorem 1 Suppose that: ∀i ∈ I, wi− wi = δ > 0, and that {wi/δ}i∈I and c/δ are integers. Then,
there exist I∗ ⊆ I and Γ∗ ∈ {0, . . . , |I∗|} such that any optimal solution of RKP(I∗,Γ∗) is an optimal
solution of CKP.
Proof: Let x∗be an optimal solution of CKP, we define: I∗= I(x∗). Since x∗is feasible with probability
at least 1−ε, we have: P (w.x∗ ≤ c) = P P
i∈I∗wi≤ c ≥ 1−ε. Let us denote V = {w ∈ W | Pi∈I∗wi≤
c}. Let consider now the following problem:
max p.x
s.t. w.x ≤ c, ∀w ∈ V
x ∈ {0, 1}n (6)
Let us prove first that any optimal solution of (6) is an optimal solution of CKP. Any feasible solution x of (6) is feasible for CKP, since: P (w.x ≤ c) ≥ P (w ∈ V ) ≥ 1 − ε. Thus, we just have to show that
(6) has the same optimal value as CKP. This is clear, since by construction, the optimal solution x∗ of CKP is a feasible solution of (6).
The goal now is to find Γ∗ ∈ {0, . . . , |I∗|} such that (6) is equivalent to RKP(I∗,Γ∗). Observe that:
w ∈ V ⇔ P
i∈I∗ηi ≤ c −
P
i∈I∗wi /δ, and let us denote: Γ∗ = min|I∗|, c −
P
i∈I∗wi /δ . We
have: w ∈ V ⇔ P
i∈I∗ηi ≤ Γ
∗. Note that V 6= φ, which implies that c −P
i∈I∗wi /δ ≥ 0. On the
other hand, from the hypothesis, Γ∗is integral. Let us show that (6) is in fact equivalent to RKP(I∗,Γ∗).
Let us prove that x is a feasible solution of (6) if, and only if, x is a feasible solution of RKP(I∗,Γ∗). Let
x be feasible for (6). For a subset S ⊆ I∗with |S| = Γ∗, let us prove thatP
i∈Swixi+ P
i∈I∗\Swixi+ P
i /∈I∗wixi ≤ c. Let w(S) denote the weight vector whose components are wi if i ∈ I∗\ S, and wi
otherwise. We have: P i∈I∗w(S)i = P i∈Swi+ P i∈I∗\Swi = Γ∗δ + P
i∈I∗wi ≤ c, and consequently:
w(S) ∈ V . Then, since x is feasible for (6), we have: w(S).x =P
i∈Swixi+ P
i∈I∗\Swixi+ P
i /∈I∗wixi≤
c. Hence, x is also feasible for RKP(I∗,Γ∗).
Suppose now that x is not feasible for (6), that is, there exists w ∈ V such that w.x > c, and let
us prove that there exists S ⊆ I∗ with |S| = Γ∗ such that: P
i∈Swixi+ P
i∈I∗\Swixi+Pi /∈I∗wixi =
P
i∈I∗wixi+ P
i∈Sδxi+ P
i /∈I∗wixi> c. Let look at I(x) ∩ I∗: if |I(x) ∩ I∗| ≥ Γ∗ (a), we construct S
with |S| = Γ∗ such that S ⊆ I(x) ∩ I∗. When |I(x) ∩ I∗| < Γ∗ (b), let S be any extension of I(x) ∩ I∗
in I∗ of cardinality Γ∗: I(x) ∩ I∗⊆ S ⊆ I∗and |S| = Γ∗ (recall that, by construction, Γ∗≤ |I∗|).
Then, if (a) is satisfied we obtain: P
i∈I∗wixi+ P i∈Sδxi+ P i /∈I∗wixi = P i∈I∗wixi + δΓ∗+ P i /∈I∗wixi≥ P i∈I∗wixi+ P i∈I∗ηiδ + P i /∈I∗wixi≥ P i∈I∗wixi+ P i∈I∗ηiδxi+ P i /∈I∗wixi= w.x > c.
The first inequality comes from w ∈ V ; for the last equality, recall that wi = wi+ ηiδ. Similarly if (b)
is satisfied we obtain: P i∈I∗wixi+ P i∈Sδxi+ P i /∈I∗wixi= P i∈I∗wixi+ δ|I(x) ∩ I∗| + P i /∈I∗wixi≥ P i∈I∗wixi+ P i∈I(x)∩I∗ηiδxi+ P i /∈I∗wixi= P i∈I∗wixi+ P i∈I∗ηiδxi+ P i /∈I∗wixi= w.x > c.
By contraposition, this proves that any feasible solution of RKP (I∗,Γ∗) is also feasible for (6). Hence,
it is shown that x is a feasible solution of (6) if, and only if, x is a feasible solution of RKP(I∗,Γ∗). As a
consequence, since both objective functions are identical, both problems are equivalent, and in particular, they admit the same optimal solutions. Since any optimal solution of (6) is an optimal solution of CKP,
it is proved that any optimal solution of RKP(I∗,Γ∗) is also optimal for CKP.
One of the strengths of the above theorem is that it requires absolutely no assumption on probability
distributions for weights. The conditions on integrality of {wi/δ}i∈I and c/δ are necessary to ensure
that Γ can be chosen integral. Nevertheless, recall that in a more general setting, Γ may also be chosen non-integral (cf Section 2.3): in this case, the theorem could be easily generalized without the integrality
conditions on {wi/δ}i∈I and c/δ. However, these assumptions do not seem very constraining for many
practical applications, where δ will in fact be small.
In the rest of this section, two particular cases are more specifically addressed, when weights or profits are all identical. In each case, the above theorem is specified, since we show that there exists a coefficient Γ such that all optimal solutions of RKP(I,Γ) are optimal also for CKP.
Lemma 3 Suppose that:
• weights and profits can be sorted so that: i < j ⇒
wi≤ wj
pi≥ pj ,
• for all i ∈ I, wi− wi= δ > 0,
• the r.v. {ηi}i∈I are independent and identically distributed.
There exists an optimal solution x∗ of CKP such that I(x∗) = {1, . . . , |I(x∗)|}.
Proof: Let x be an optimal solution of CKP, and let us build x∗ such that I(x∗) = {1, . . . , |I(x)|}. As
|I(x∗)| = |I(x)| and profits pi are sorted in non-increasing order: p.x∗≥ p.x. Moreover, observe that by
construction: P i∈I(x∗)wi≤ P i∈I(x)wi. Then: P (w.x∗ ≤ c) = P (P i∈I(x∗)wi≤ c) = P (δ. P i∈I(x∗)ηi≤ c − P i∈I(x∗)wi) = P (δ.P i∈I(x)ηi≤ c − P i∈I(x∗)wi) (a) ≥ P (δ.P i∈I(x)ηi≤ c − P i∈I(x)wi) (b) = P (P i∈I(x)wi≤ c) = P (w.x ≤ c)
As the r.v. {ηi}i∈I are supposed independent and identically distributed, and since |I(x∗)| = |I(x)|,
the r.v. P
i∈I(x)ηi and P
i∈I(x∗)ηi are identically distributed: this ensures (a). (b) is a direct
conse-quence ofP
i∈I(x∗)wi≤ P
i∈I(x)wi. Thus, as x is feasible for CKP, x∗ is also feasible for this problem.
Hence, x∗ is an optimal solution of CKP.
Theorem 2 Suppose that:
• weights and profits can be sorted so that: i < j ⇒
wi≤ wj
pi≥ pj ,
• for all i ∈ I, wi− wi= δ > 0,
• {wi/δ}i∈I and c/δ are integers,
• the r.v. {ηi}i∈I are independent and identically distributed.
Then, there exists Γ∗ ∈ {0, . . . , n} such that any optimal solution of RKP(I,Γ∗) is an optimal solution
of CKP.
Proof: From Theorem 1, we know that there exist I∗⊆ I and Γ∗ ∈ {0, . . . , |I∗|} such that an optimal
solution of RKP(I∗,Γ∗) is an optimal solution of CKP. Moreover, from the proof, we know that I∗
can be chosen so that I∗ = I(x∗) for some optimal solution x∗ of CKP, and that Γ∗ can be chosen
so that: Γ∗ = min|I∗|, c −P
i∈I∗wi /δ . Without loss of generality, from Lemma 3, we consider:
I(x∗) = {1, . . . , |I∗|}. Suppose first that Γ∗ = |I∗|. In this case, problems RKP(I∗,Γ∗) and RKP(I,n)
are equivalent.
Suppose now that Γ∗ = c −P
i∈I∗wi /δ < |I∗|. Let us prove that any optimal solution of
RKP(I,Γ∗) is optimal also for CKP. Let x0 be an optimal solution of RKP(I,Γ∗). If |I(x0)| ≤ Γ∗,
x0 is feasible for RKP(I,n) and consequently: P (w.x0 ≤ c) = 1. Thus, in this case, x0 is feasible for CKP.
Consider now that |I(x0)| ≥ Γ∗+ 1, and let us prove that x0 is a feasible solution of CKP. Suppose that:
P
i∈I(x0)wi>Pi∈I∗wi, then for any subset S ⊆ I(x0) of cardinality Γ∗: X i∈S wix0 i+ X i∈I\S wix0i= X i∈I(x0) wi+ Γ∗δ > X i∈I∗ wi+ Γ∗δ = c − Γ∗δ + Γ∗δ = c
This is a contradiction with the feasibility of x0 for RKP(I,Γ∗). As a consequence: P
i∈I(x0)wi ≤ P
i∈I∗wi. Furthermore, as I∗is the collection of the |I∗| lowest values wi, this implies that |I(x0)| ≤ |I∗|.
On the other hand, as I∗ ⊆ I, x∗ is a feasible solution of RKP(I,Γ∗), and we have: p.x∗ ≤ p.x0, i.e.:
P
i∈I∗pi≤
P
i∈I(x0)pi. As I∗ is the collection of the |I∗| largest profit values, this implies that |I(x0)| ≥
|I∗|. As I∗ is also the collection of the |I∗| lowest values w
i, this leads to:
P
i∈I(x0)wi≥ P
i∈I∗wi. As
a result: |I(x0)| = |I∗| andP
i∈I(x0)wi= P
i∈I∗wi.
Now, let us see that x0 is feasible for CKP:
P (w.x0 ≤ c) = P (δ.P i∈I(x0)ηi≤ c − P i∈I(x0)wi) = P (δ.P i∈I(x0)ηi≤ c − P i∈I(x∗)wi) (a) = P (δ.P i∈I(x∗)ηi≤ c − P i∈I(x∗)wi) (b) = P (w.x∗ ≤ c)
(a) comes fromP
i∈I(x0)wi= P
i∈I∗wi. (b) comes from |I(x0)| = |I∗| and probability assumptions on
r.v. ηi. This shows that x0 is a feasible solution of CKP. Finally, we have already seen that p.x∗≤ p.x0,
which ensures that x0 is an optimal solution of CKP.
As previously for Theorem 1, the integrality of c/δ and of wi/δ for all i ∈ I is not really a requirement,
since Γ∗ may also be chosen non-integral (cf Section 2.3). As particular cases of the above theorem, the
two following results hold:
Corollary 1 Suppose that:
• ω/δ and c/δ are integers,
• the r.v. {ηi}i∈I are independent and identically distributed.
Then, there exists Γ∗ ∈ {0, . . . , n} such that any optimal solution of RKP(I,Γ∗) is an optimal solution
of CKP.
Corollary 2 Suppose that:
• for all i ∈ I, pi= ρ > 0 and wi− wi= δ > 0,
• for all i ∈ I, wi/δ and c/δ are integers,
• the r.v. {ηi}i∈I are independent and identically distributed.
Then, there exists Γ∗ ∈ {0, . . . , n} such that any optimal solution of RKP(I,Γ∗) is an optimal solution
of CKP.
4
Solving CKP.
4.1
Complexity of RKP and resolution algorithm.
Within this section, RKP(I,Γ) is denoted simply RKP. Moreover, all data {wi}i∈I, {wi}i∈I and c are
suppose to be non-negative integers. It is shown that the classical dynamic programming algorithm can be adapted, and that the pseudo-polynomiality of the classical knapsack problem is preserved:
Theorem 3 RKP is weakly NP-hard: there exists a pseudo-polynomial time algorithm to solve it.
Proof: That RKP is NP-hard comes immediately, since the particular case Γ = 0 is in fact the classical
knapsack problem with weights {wi}. On the other hand, the classical dynamic programming algorithm
for knapsack problems can be adapted to solve RKP. Let us denote by RKPk(Γ, b) the problem RKP for
robust parameter Γ ∈ {0, . . . , n} and capacity b ∈ {1, . . . , c}, considering only elements in Ik= {1, . . . , k}
for k ∈ I. Let Fk(Γ, b) denote its optimal solution value. We suppose without loss of generality that the
elements of I are sorted so that: i < j ⇒ (wi− wi) ≤ (wj− wj).
For any k, we assume that: b < 0 ⇒ Fk(., b) = −∞. For k = 1, F1 is defined as follows:
if Γ = 0: F1(0, b) = 0 if b < w1 p1 if b ≥ w1 if Γ ≥ 1: F1(Γ, b) = 0 if b < w1 p1 if b ≥ w1
Consider b ∈ {1, . . . , c} and Γ ∈ {1, . . . , n}. Then the optimal value of RKPk(Γ, b) can be computed by the recurrence formula:
Fk(Γ, b) = max {Fk−1(Γ, b), pk+ Fk−1(Γ − 1, b − wk)}
Let us prove that Fk(Γ, b) is the optimal value of RKPk(Γ, b). The result is obvious for k = 1. Let k ≥ 2.
Let us show first that any feasible solution xk−1∈ {0, 1}k−1of RKPk−1(Γ, b) or RKPk−1(Γ−1, b−wk)
can be extended into a feasible solution xk ∈ {0, 1}k of RKPk(Γ, b) by adding a kthelement to xk−1. If
xk−1 is feasible for RKPk−1(Γ, b), then considering xk
k= 0 ensures that xk is feasible for RKPk(Γ, b). If
xk−1 is feasible for RKPk−1(Γ − 1, b − wk), let us consider the vector xk with xk
k = 1. Then, we have: P i∈Ikwix k i + max{ P
i∈S(wi − wi)xki : S ⊆ Ik, |S| ≤ Γ} =
P i∈Ik−1wix k−1 i + wk + max{ P i∈S(wi−
wi)xk−1i : S ⊆ Ik−1, |S| ≤ Γ − 1}. This comes from the order assumed on weight variations. Combining
this with: P
i∈Ik−1wixk−1i + max{
P
i∈S(wi− wi)xk−1i : S ⊆ Ik−1, |S| ≤ Γ − 1} ≤ b − wk, we obtain that
xk is also feasible for RKPk(Γ, b).
The reverse, that is for any feasible solution of RKPk(Γ, b), the first k − 1 components give a feasible
a feasible solution of RKPk(Γ, b). Let xk−1 ∈ {0, 1}k−1 denote the vector composed of the first k − 1
elements in xk. Two cases can be distinguished:
(a) xk
k = 0: then xk−1 is feasible for RKPk−1(Γ, b).
(b) xk
k = 1: then we can prove by contraposition that xk−1is a feasible solution of RKPk−1(Γ − 1, b −
wk). Suppose that there exists S ⊆ Ik−1 with |S| = Γ − 1, such that: P
i∈Ik−1wixk−1i + P
i∈S(wi−
wi)xk−1i > b − wk. Then, xk cannot be feasible for RKPk(Γ, b) since for S0 = S ∪ {k} we obtain:
P i∈Ikwix k i + P i∈S0(wi− wi)xki = P i∈Ikwix k i + P i∈S(wi− wi)xki + (wk− wk) = P i∈Ik−1wix k−1 i + P i∈S(wi− wi)x k−1 i + wk > b.
Thus, we have shown that the set of vectors composed of the k − 1 first components of any feasible
solution xk of RKPk(Γ, b) is equal to the union of all feasible solutions of respectively RKPk−1(Γ, b) and
RKPk−1(Γ − 1, b − wk). Hence we have: Fk(Γ, b) = max {Fk−1(Γ, b), pk+ Fk−1(Γ − 1, b − wk)}.
Finally, observe that the required value Fn(Γ, c) can be computed in pseudo-polynomial time O(nΓc).
4.2
An iterative algorithm.
In the following algorithm, the value of Γ is progressively increased until the optimal solution of RKP(I,Γ) is a feasible solution of CKP.
Algorithm 1:
Step 1: Let k = 0.
Step 2: Solve the problem RKP(I,k). Let x(k) denote its optimal solution.
Step 3: Set I0 = I(x(k)). Compute Γ0, the largest value of {0, . . . , |I0|} such that
x(k) is feasible for RKP(I0,Γ0).
Step 4: Compute a bound B ≤ P (P
i∈I0wi≤ c). If B ≥ 1 − ε, STOP.
Step 5: Set k ← Γ0+ 1 and go to Step 2.
At Step 4, observe that: P (P
i∈I0wi≤ c) = P (w.x
(k)≤ c). Thus, we know that the algorithm stops
only with a solution feasible for CKP. Moreover, as x(k) is a feasible solution of RKP(I0,k) (cf Lemma
2), we have Γ0 ≥ k. Hence, the increase of index k at each loop ensures the convergence: indeed, when
k = n = |I|, the feasibility probability of x(n) is equal to 1 and the algorithm stops at Step 4. Thus, k
can not exceed this value. As a result:
Lemma 4 Algorithm 1 provides a feasible solution to CKP by solving at most n + 1 robust knapsack
problems.
Some further observations have to be made. First, from a given iteration k, it is unnecessary to
consider the problems RKP(I,l), with k + 1 ≤ l ≤ Γ0. Indeed, x(k) is already feasible for RKP(I,Γ0),
since it is feasible for RKP(I0,Γ0). That is the reason for considering directly Γ0+ 1 at Step 5. On the
other hand, the value Γ0 at Step 3 can be computed in linear time. Indeed, it is the optimal solution of:
max Γ
s.t. P
i∈I0wi+ maxS⊆I0:|S|=ΓPi∈S(wi− wi) ≤ c
Γ ∈ {0, . . . , n}
It is sufficient to consider the empty set S = φ and to fill it progressively with the indices of the largest
weight variations wi− wi. As soon asPi∈Swi− wi> c −Pi∈I0wi, we have: Γ
0= |S| − 1.
To compute B at Step 4, we refer for instance to the probability bounds proposed in [8]. The bound already used in the proof of Proposition 2 under specific probability assumptions may be used. It is clear that the quality of the bound used will directly impact the number of iterations of the algorithm and
the quality of the final solution. It has to be noted that if |I0| = Γ0, the feasibility probability is 1 and
the algorithm stops (cf Lemma 2).
As a consequence of Lemma 4, we have:
Proposition 3 Algorithm 1 runs in pseudo-polynomial time by using the dynamic programming
4.3
Two particular cases.
Let us finally consider the case of uniform weights, and this of uniform profit values.
Lemma 5 Suppose that for all i ∈ I, wi = ω > 0 and wi− wi= δ > 0. Then an optimal solution x∗ of
RKP(I,Γ) satisfies: X i∈I x∗i = min {n, b(c − Γδ)/ωc} if Γ(ω + δ) ≤ c bc/(ω + δ)c otherwise
Proof: Observe first that any optimal solution of the uniform RKP(I,Γ) will maximize the sum of its
components (P
i∈Ixi). Suppose that Γ(ω + δ) ≤ c. This means that an optimal solution x∗of RKP(I,Γ)
has at least Γ elements: P
i∈Ix∗i ≥ Γ. Then, x∗ is in fact an optimal solution of the problem with the
only constraint: P
i∈Iωxi≤ c − Γδ. The result follows.
Now, if Γ(ω +δ) > c, a feasible solution of RKP(I,Γ) can not have more than Γ−1 elements. Then, an
optimal solution x∗ of RKP(I,Γ) is in fact an optimal solution of the problem with the only constraint:
P
i∈I(ω + δ)xi ≤ c. This ends the proof.
From this lemma, an optimal solution of RKP(I,Γ) can be built in linear time by considering succes-sively the elements of largest profit values.
Theorem 4 Suppose that:
• for all i ∈ I, wi= ω > 0 and wi− wi= δ > 0,
• ω/δ and c/δ are integers,
• the r.v. {wi}i∈I are independent and identically distributed,
• P (P
i∈I0wi≤ c) is known for any subset I
0⊆ I.
Then Algorithm 1 provides an optimal solution to CKP in polynomial time O(n3).
Proof: It has just been seen that for any value of Γ, an optimal solution of RKP(I,Γ) can be
com-puted in linear time O(n). From Lemma 4, Algorithm 1 stops in at most n + 1 resolutions of robust
problems, and since Step 3 is solved in time O(n), it runs in time O(n3). From corollary 3.1, we know
that one of the solutions x(k) explored with Algorithm 1 is optimal for CKP. Moreover, the sequence
of objective values p.x(k) is non-increasing. As P (P
i∈I0wi ≤ c) is supposed to be exactly known for
any I0⊆ I, the algorithm stops at Step 4 with the first solution feasible for CKP, which then is optimal.
Similar results hold for uniform profits. As the proofs use exactly the same ideas as previously, they are not detailed.
Lemma 6 Suppose that, for all i ∈ I, pi= ρ > 0 and wi− wi = δ > 0. If c ≥ Γδ, let x0 be an optimal
solution of: max P i∈Ixi s.t. P i∈Iwixi≤ c − Γδ x ∈ {0, 1}n (7)
and let x00 be an optimal solution of:
max P i∈Ixi s.t. P i∈Iwixi≤ c x ∈ {0, 1}n (8) If c ≥ Γδ andP
i∈Ix0i≥ Γ, x0 is optimal for RKP(I,Γ). Otherwise, x00 is optimal for RKP(I,Γ).
Note that both problems (7) and (8) are solved in linear time. On the other hand, observe that (7) has no feasible solution if c − Γδ < 0.
• for all i ∈ I, pi= ρ > 0 and wi− wi= δ > 0,
• {wi/δ}i∈I and c/δ are integers,
• the r.v. {ηi}i∈I are independent and identically distributed,
• P (P
i∈I0wi≤ c) is known for any subset I
0⊆ I.
Then Algorithm 1 provides an optimal solution to CKP in polynomial time O(n3).
4.4
An example.
In this last section, the algorithm 1 is used to solve a simple uniform chance-constrained knapsack problem to the optimum. Consider the following problem:
max P10i=1ixi
s.t. P10i=1wixi≤ 80
x ∈ {0, 1}10
where all weights wi are uniformly distributed on [8, 12], i.e. wi = 8 and wi = 12 for all i ∈ I =
{1, . . . , 10}, and δ = 4. Let us consider ε = 0.1, that is, we look for a filling x of the knapsack which will be feasible with probability at least 90%. Note that under the above probability assumptions, the
probabilities P (P
i∈I0wi≤ c) are known analytically for all subsets I0⊆ I. In the following, let eidenote
the 0-1 vector such that ei
i= 1 and eij= 0 for i 6= j.
k=0: From Lemma 5, we know that: P
i∈Ix (0)
i = b80/8c = 10, and thus: x(0)=
P10 i=1ei.
We have: Γ0= b(80 − 10 ∗ 8)/4c = 0, and we compute: P (P10
i=1wi≤ 80) = 0
(the only event possible is w = w).
k=1: From Lemma 5: P i∈Ix (1) i = b(80 − 4)/8c = 9, and thus: x(1)= P10 i=2ei.
We have: Γ0= b(80 − 9 ∗ 8)/4c = 2, and we compute: P (P10
i=2wi≤ 80) = 0.001... < 0.9. k=3: From Lemma 5: P i∈Ix (3) i = b(80 − 12)/8c = 8, and thus: x(3)= P10 i=3ei. We Γ0= b(80 − 8 ∗ 8)/4c = 4, and we compute: P (P10 i=3wi≤ 80) = 0.5 < 0.9. k=5: From Lemma 5: P i∈Ix (5) i = b(80 − 20)/8c = 7, and thus: x(5)= P10 i=4ei. We compute: P (P10i=4wi≤ 80) = 0.99... > 0.9.
STOP: x(5) is feasible, and thus optimal.
Note that considering a feasibility probability of 100% would require to put less elements in the knapsack. Indeed, assuming that all weights take their worst values leads to put at most b80/12c = 6 items in the knapsack.
5
Conclusion.
The chance-constrained knapsack problem has been addressed. A relevant robust optimization problem has been proposed. Its theoretical links with the chance-constrained problem have been investigated. Then, a tractable combinatorial algorithm has been designed to obtain good solutions to the chance-constrained problem, by running a sequence of robust problems. When profits are identical, or when uncertain weights present all the same characteristics, this algorithm is proved to provide an optimal solution.
References
[1] A. Atamt¨urk, Strong formulations of robust mixed 0-1 programming, submitted to Math. Program.. Available at www.optimization-online.org/DB FILE/2004/01/816.pdf
[2] W. Ben-Ameur and H. Kerivin, Offres de R´eseaux Priv´es Virtuels Flexibles, Tech. Report NT/FTRD/7358, France T´el´ecom R&D (2001).
[4] A. Ben-Tal and A. Nemirovski, Robust solutions of Linear Programming problems contamined with uncertain data, Math. Program., A88 (2000), pp. 411-424.
[5] A. Ben-Tal and A. Nemirovski, Robust optimization - methodology and applications, Math. Program., B92 (2002), pp. 453-480.
[6] D. Bertsimas, D. Pachamanova and M. Sim, Robust Linear Optimization under General Norms, Oper. Res. Lett., Vol. 32, Issue 6 (2004), pp. 510-516.
[7] D. Bertsimas and M. Sim, Robust discrete optimization and network flows, Math. Program., B98 (2003), pp. 49-71.
[8] D. Bertsimas and M. Sim, The Price of Robustness, Oper. Res., Vol. 52, no. 1 (2004), pp. 35-53. [9] J.R. Birge and F. Louveaux, Introduction to Stochastic Programming, Springer-Verlag, 1997.
[10] G. Calafiore and M.C. Campi, Uncertain convex programs: randomized solutions and confidence levels, Math. Program., A102 (2005), pp. 25-46.
[11] X. Chen, M. Sim and P. Sun, A Robust Optimization Perspective of Stochastic Programming, submitted to Oper. Res. (2005).
[12] E. Erdo˘gan and G. Iyengar, Ambiguous chance constrained problems and robust optimization, Math. Pro-gram. (online) DOI: 10.1007/s10107-005-0678-0 (2005).
[13] P. Kall and S.W. Wallace, Stochastic Programming, Wiley, Chichester, 1994.
[14] S. Martello and P. Toth, Knapsack Problems: Algorithms and Computer Implementations, Wiley, 1990. [15] A. Nemirovski and A. Shapiro, Convex Approximations of Chance Constrained Programs, submitted to
SIAM J. Optim..
[16] M.C. Pinar, A Note on Robust 0-1 Optimization with Uncertain Cost Coefficients, 4OR, Vol. 2 (2004), pp. 309-316.
[17] N.V. Sahinidis, Optimization under uncertainty: state-of-the-art and opportunities, Computers & Chemical Engineering, 28 (2004), pp. 971-983.
