As the constraints of the scenario problem SBN are randomly extracted, its optimal solutions are random variables which depend on the set of extractions {ξ(s)}N
s=1. These solutions typically
fail to satisfy all constraints of the robust problem RB. However, they can be shown to satisfy the constraints of RB with high probability. The following fundamental question is addressed,
2.4. Complexity Analysis 31
among others, by Calafiore and Campi [2005, 2006] and by Campi and Garatti [2008]: what confidence do we have that an optimal solution of SBN will violate the constraints of RB
with probability less than ǫ, where ǫ is a prescribed probability level as in chance-constrained programming (see Section 2.2)? To the best of our knowledge, the most general answer to this question is provided by Campi and Garatti [2008] who establish, for any solvable convex optimization problem and any probability distribution P with support Ξ, the so-called “exact feasibility” of the randomized solution. We repeat their results here, together with the definition of violation probability as introduced in Calafiore and Campi [2005].
Definition 2.1 (Calafiore and Campi [2005]) The violation probability of a given w ∈ Rnw
is defined as V (w) := P (ξ ∈ Ξ : f (w, ξ) > 0).
Theorem 2.1 (Campi and Garatti [2008]) For any given probability level ǫ ∈ (0, 1) and confidence level β ∈ (0, 1), let
N(ǫ, β) := min ( N ∈ N : nXw−1 i=0 (N i ) ǫi(1 − ǫ) N −i ≤ β ) .
Suppose that RB is solvable. If problem SBN is solvable and N ≥ N(ǫ, β), then we have
PN(V (wN∗) > ǫ) ≤ β,
where w∗N denotes the (w.l.o.g.) unique optimal solution of SBN, and PN:= P × P × · · · × P (N
times) is the probability distribution of the sample (ξ(1), ξ(2), . . . , ξ(N )).
Remark 2.2 Theorem 2.1 remains applicable in the case when problem SBN has multiple optimal solutions, provided a suitable “tie-breaking” rule is used to systematically select a single optimal solution, see e.g., Section 4.1 in Calafiore and Campi [2005] or Section 2.1 in Campi and Garatti [2008].
The result from Theorem 2.1 guarantees that any random solution of SBN satisfies most of the original constraints with high confidence provided that the sample size N is chosen large
enough. Thus, the solution of SBN is feasible in CCǫ with high confidence 1 − β. We emphasize
that the distribution P is not needed to compute this solution.
We note that by continuity of f , RB is always solvable when its feasible set is nonempty and bounded. A bound on the violation probability that remains valid for possibly infeasible instances of RB is provided by Calafiore [2010].
The following corollary is an immediate consequence of Theorem 2.1 and generalizes a result of Bertsimas and Caramanis [2007] for polynomial decision rules.
Corollary 2.1 (Complexity of problem SBN) For any fixed probability level ǫ ∈ (0, 1) and confidence level β ∈ (0, 1), the number of samples N needed such that the optimal solution w∗N of SBN satisfies PN(V (w∗
N) > ǫ) ≤ β remains polynomially bounded in n and k.
Remark 2.3 (Computational tractability) Corollary 2.1 implies that problem SBN can be
solved in polynomial time with respect to the size of the input parameters, provided that for any fixed ξ ∈ Rk, the set {w ∈ Rn : f (w, ξ) ≤ 0} admits an efficient separation oracle, see Gr¨otschel
et al. [1981].
Proof of Corollary 2.1 In Calafiore [2009b] it was shown that
N(ǫ, β) ≤ 2 ǫ ln1 β + nw .
By construction of problem RB (see Section 2.3.1), we have that
nw = 1 + X t∈T ntsd kt ≤ 1 +X t∈T ntsd(k) = 1 + nsd(k) ,
where the inequality holds since sdis increasing and k ≥ ktfor each t ∈ T. As sdis polynomially
bounded by condition (C2), the number of decision variables nw, and hence the number of
samples N(ǫ, β) required for the prescribed violation probability ǫ and confidence level β, remain
2.4. Complexity Analysis 33
Remark 2.4 (Scalability) Typically, n and k are both linear in T , in which case the number of samples N needed to ensure PN(V (w∗
N) > ǫ) ≤ β is also polynomially bounded with T .
Remark 2.5 The basis size sd(kt) can be exponential in the complexity parameter d for fixed
values of kt. In such cases, the number of samples required to sustain a maximum violation
probability of ǫ at confidence 1 − β is exponential in d. If P is unknown and only ˜N samples from P are available, one may opt for “simpler” decision rules, i.e., low values of d, in order to guarantee the required level of feasibility. Indeed, the maximum admissible value of d for given values of ǫ, β and ˜N amounts to
d := max d ∈ N : minnN ∈ N : nwX(d)−1 i=0 (N i ) ǫi(1 − ǫ) N −i≤ βo≤ ˜ N ,
where nw(d) := 1 +Pt∈Tntsd(kt). In particular, any d with sd(k) ≤ n−1(ǫ ˜N /2 − ln(1/β) − 1)
satisfies d ≤ d and ensures that no more than ˜N samples are needed to sustain a maximum violation of ǫ with confidence 1 − β.
For high values of d and low values of ǫ, problem SBN can become computationally expensive or memory intensive. The following algorithm, which relies on the observation that only a small portion of the sampled constraints are active at optimality, provides an iterative solution approach that substantially reduces the sizes of the problems to be solved. It is specifically designed to solve SBN efficiently while keeping the memory requirements of each iteration low, as it only involves the solution of subproblems with significantly fewer constraints. For ease of exposition, we assume that RB is feasible.
Algorithm 2.1 (Iterative constraint addition/removal procedure)
1. Initialization. Partition the set {1, . . . , N} into nit subsets, letting Ii denote the i th
subset. Set the iteration counters to i ← 1 and j ← 1. Also set I ← ∅ and u ← false. 2. Optimization. Set I ← I ∪ Ii. Solve the relaxation of SBN involving only the con-
straints f (w, ξ(s)) ≤ 0, s ∈ I. If the problem is solvable, denote the solution by wi N and
go to Step 3, else if i < nit, set i ← i + 1 and repeat Step 2, else set u ← true and go to
Step 6.
3. Constraint removal. Set I ← I\{s ∈ I : f (wi
N, ξ(s)) < 0}. If i = nit, set ˆw1N ← wiN
and go to Step 4, else set i ← i + 1 and go to Step 2.
4. Constraint addition. If ˆwjN is feasible in SBN, go to Step 6, else set I ← I ∪ {s ∈ {1, . . . , N} : f ( ˆwjN, ξ(s)) > 0}.
5. Re-optimization. Set j ← j + 1 and solve the relaxation of SBN involving only the constraints f (w, ξ(s)) ≤ 0, s ∈ I. Denote the resulting solution by ˆwj
N. Go to Step 4.
6. Termination. If u = false, set wN∗ ← ˆwjN and stop, else declare SBN to be unbounded.
More sophisticated algorithms for solving SBN that remain applicable when RB is infeasible can be designed by adapting known semi-infinite optimization algorithms.