D Proof Proposition 9

Before proving Proposition 9, we prove Lemmas 11 to 16. For convenience, we dene functions f(pⁱ_i+1), similar as in the proof of Proposition 8 (see Appendix C). Using Equa-tion (27), it is straightforward to show that all f are polyhedral convex funcEqua-tions. For POS-AW, we use Equation (34) to obtain

f (pⁱ_i+1) = g_i+1(0, pⁱ_i+1) + Eh_i+1(max{x_i+ t_ii+1, a_i+1+ pⁱ_i+1}, pⁱ_i+1) +

E¯ci+1 max{x_i+ t_ii+1, a_i+1+ pⁱ_i+1}
 . (50) For EXT-VW and POS-VW, we use Equation (47) to obtain

f (pⁱ_i+1) =gi+1(0, pⁱ_i+1) + Ehi+1(xi+1, pⁱ_i+1) + E [¯cⁱ⁺¹(xi+1)] , (51) with in the right-hand side

• x_i+1= t_i+1+ min{X_i+1^∗ , a_i+1} − t_i+1
+

and ti+1 = x_i+ t_ii+1 for EXT-VW,

• xi+1= ti+1+ min{X_i+1^∗ , ai+1+ pⁱ_i+1} − ti+1

+

and ti+1 = xi+ tii+1 for POS-VW.

For brevity, we dene Assumption 1, which we refer to in the lemmas.

Assumption 1. Assume that the combination of adjustment type and waiting behavior is POS-AW, EXT-VW, or POS-VW. Let i ∈ {0, . . . , n − 1} and assume that ¯ci+1 is piece-wise linear and all breakpoints are integer.

Lemma 11. Under Assumption 1, every breakpoint pⁱ_i+1 of f satises at least one of the equations as marked in the table below:

Description Equation POS-AW EXT-VW POS-VW

No adjustment: pⁱ_i+1 = 0

× × ×

Arrive at start time window POS for some tii+1: xi+ tii+1 = ai+1+ pⁱ_i+1

× ×

Arrive at end time window for some tii+1: xi+ tii+1 = bi+1+ pⁱ_i+1

× × ×

Start time window POS is integer: ai+1+ pⁱ_i+1∈ Z

× ×

Time window POS start at preferred departure time: Xi+1^∗ = a_i+1+ pⁱ_i+1

×

Time window ends at preferred departure time: X_i+1^∗ = b_i+1+ pⁱ_i+1

× ×

Time window has width zero: ai+1= bi+1+ pⁱ_i+1

×

Proof. If pⁱi+1 is a breakpoint of f, then pⁱi+1 must also be a breakpoint of one of its summands. Recall that by assumption, the expectation can be written as a nite sum.

Consider the case POS-AW. If pⁱi+1 is a breakpoint of gi+1, then it follows from the denition of gi+1, Equation (20), that pⁱi+1 = 0. Next, consider the hi+1 term. Note that hi+1 is dened by Equation (22). Without loss of generality, we may assume that δ_i+1 = 0, i.e., there is no penalty for early delivery. This is possible because we always wait until the time window opens. It follows that the hi+1 term within the expectation can be written as (max{xi+ t_ii+1, a_i+1+ pⁱ_i+1} − (b_i+1+ pⁱ_i+1)
+

γ_i+1.

We observe two potential breakpoints for each hi+1 term. First, we may have a breakpoint when the two arguments of the maximum are equal, i.e., xi+ t_ii+1 = a_i+1+ pⁱ_i+1. Second, if xi + tii+1 > ai+1 + pⁱ_i+1 then xi + tii+1 = bi+1 + pⁱ_i+1 is a poten-tial breakpoint. Note that if xi + tii+1 < ai+1 + pⁱ_i+1, then the hi+1 term is equal to

a_i+1+ pⁱ_i+1− (b_i+1+ pⁱ_i+1)
+

γ_i+1 = (a_i+1− b_i+1)⁺γ_i+1, which is constant in pⁱi+1 and does not result in any breakpoints.

Next, we consider breakpoints due to the ¯ci+1 term. Again, we can have a breakpoint if the arguments of the minimum are equal. By Assumption 1, the function ¯ci+1only has integer breakpoints. Hence, if xi+ t_ii+1< a_i+1+ pⁱ_i+1, we have a potential breakpoint for a_i+1+ pⁱ_i+1 ∈ Z. Note that if xi+ t_ii+1 > a_i+1+ pⁱ_i+1, then the ¯ci+1 term is constant in pⁱ_i+1. This concludes the proof for POS-AW.

Next, consider the case EXT-VW. For the gi+1 term, we again obtain pⁱi+1= 0. In the denition of f, Equation (51), we have

xi+1=







x_i+ t_ii+1 if xi+ t_ii+1 ≥ min{X_i+1^∗ , a_i+1}

min{X_i+1^∗ , ai+1} if xi+ tii+1 ≤ min{X_i+1^∗ , ai+1}. (52) That is, if the vehicle arrives at customer i + 1 before min{Xi+1^∗ , ai+1}, then it waits until that time. If the vehicle arrives later, customer i + 1 is served immediately.

Now consider the hi+1 terms. From the denition, Equation (21), it follows that these terms are given by xi+1− (b_i+1+ pⁱ_i+1)
+

γ_j + (a_i+1− x_i+1)⁺δ_j. If xi+1 = x_i+ t_ii+1, we have a potential breakpoint for xi+ t_ii+1= b_i+1+ pⁱ_i+1. Note that (ai+1− (x_i+ t_ii+1))⁺δ_j is constant in pⁱ_i+1, and thus does not yield breakpoints. If xi+1= min{X_i+1^∗ , a_i+1}, then depending on whether Xi+1^∗ or ai+1 is the minimum, we obtain potential breakpoints for X_i+1^∗ = b_i+1+pⁱ_i+1and ai+1= b_i+1+pⁱ_i+1. Finally, we consider xi+t_ii+1 = min{X_i+1^∗ , a_i+1}, i.e., the value for which (52) switches cases. Again, we nd potential breakpoints for X_i+1^∗ = bi+1+ pⁱ_i+1 and ai+1= bi+1+ pⁱ_i+1.

Next, we consider breakpoints due to the ¯ci+1 term. It follows from the denition of x_i+1, Equation (52), that ¯ci+1(x_i+1) is constant in pⁱi+1. Hence, the ¯ci+1 term does not provide additional potential breakpoints. This completes the proof for EXT-VW.

Finally, we consider POS-VW. For the gi+1 term, we again obtain pⁱi+1 = 0. In

the denition of f, Equation (51), we have waits until that time. If the vehicle arrives later, customer i + 1 is served immediately.

The hi+1 terms are dened by Equation (22). As in the POS-AW case, we assume without loss of generality that δi+1 = 0. It follows that the hi+1 terms are given by

x_i+1− (b_i+1+ pⁱ_i+1)
+ poten-tial breakpoints. Finally, there can be a breakpoint if the arguments of the minimum are equal, i.e., Xi+1^∗ = a_i+1+ pⁱ_i+1.

Now consider the breakpoints due to the ¯ci+1 term. By Assumption 1, we have a potential breakpoint due to ¯ci+1(x_i+1)is xi+1 is not constant in pⁱi+1, and xi+1 is integer.

It follows that we can only get a breakpoint if xi+1= a_i+1+pⁱ_i+1. The potential breakpoints that we nd are given by ai+1+ pⁱ_i+1 ∈ Z. This completes the proof of POS-VW. After proving the cases POS-AW, EXT-VW, and POS-VW, we have proven the lemma.

Lemma 12. Under Assumption 1, the preferred departure time Xi+1^∗ from customer i+1 (see Proposition 10, Appendix B) can be chosen such that Xi+1^∗ ∈ Z ∪ {−∞, ∞}.

Proof. By denition, X_i+1^∗ = arg min_X

i+1∈R{¯c_i+1(X_i+1) − X_i+1δ_i+1}. Under Assump-tion 1, the funcAssump-tion ¯ci+1(X_i+1) − X_i+1δ_i+1 is piece-wise linear and only has breakpoints on the integers. As Xi+1^∗ is the arg min of this function, we may assume that Xi+1^∗ ∈ We refer to this set as the neighborhood of ¯xi. We use ¯f to denote the function f in which xi is replaced by ¯xi.

We dene Q to be the set of potential breakpoints of f. That is, for a given value of x_i, the set Q contains all values of pⁱi+1 that satisfy at least one of the relevant equations

in Lemma 11. Similarly, we dene ¯Q to be the set of potential breakpoints of ¯f. By denition, there is a bijection between Q and ¯Q.

Consider q ∈ Q and ¯q ∈ ¯Q, both dened by the same equation. By going over the equations in Lemma 11, and using that ¯xi ∈ Z and x/ i ∈ (b¯x_ic, d¯x_ie), the following three implications can be veried.

1. If ¯q ∈ Z then q − ¯q = 0, i.e., the dierence between q and ¯q is zero.

2. If ¯q /∈ Z then ¯q − ¯xi ∈ Z, i.e., ¯q and ¯xi have the same fractional part.

3. If ¯q /∈ Z then q − ¯q = xi − ¯xi, i.e., the dierence between q and ¯q is equal to the dierence between xi and ¯xi.

For example, consider the third equation in Lemma 11. We obtain xi+ t_ii+1= b_i+1+ q and ¯xi + t_ii+1 = b_i+1+ ¯q. We rst verify the rst statement. If ¯q ∈ Z, then it follows from the integrality of tii+1 and bi+1 that ¯xi ∈ Z. This contradicts the assumption that

¯

x_i ∈ Z, and the implication trivially holds. Next, we consider ¯/ q /∈ Z. Rewriting the equations yields q − xi = t_ii+1− b_i+1 and ¯q− ¯xi = t_ii+1− b_i+1, which shows ¯q− ¯xi ∈ Z and q − ¯q = x_i− ¯x_i. Hence, both implications hold. The verication of the three implications for the other equations in Lemma 11 is similar.

It can be seen that the subdierential ∂f(pⁱi+1) is uniquely determined by the set of potential breakpoints q such that q < pⁱi+1, and whether pⁱi+1 is a potential breakpoint itself. This follows from the fact that the slopes of the summands of f can only change at the potential breakpoints, and that replacing xi by ¯xi only changes the locations of the potential breakpoints, and not the slopes before and after the potential breakpoints.

Next, we show that the ordering of the potential breakpoints is the same for f and f¯. Earlier, we have shown that if ¯q ∈ Z, then q − ¯q = 0, i.e., the potential breakpoints are equal (Implication 1). In the case of ¯q /∈ Z we have shown that q − ¯q = xi − ¯x_i (Implication 3) and that ¯q and ¯xi have the same fractional part (Implication 2). By assumption, ¯x /∈ Z, and xi ∈ (b¯xic, d¯xie). It follows immediately that q ∈ (b¯qc, d¯qe).

In summary, f and ¯f have the same integer breakpoints, and all fractional breakpoints change by the same amount. The fractional breakpoints remain fractional, which implies that the ordering of the potential breakpoints is the same for f and ¯f.

As an immediate consequence, we have ∂f(q) = ∂ ¯f (¯q) for corresponding potential breakpoints q and ¯q. Furthermore, the subdierential of f at a point between two se-quential potential breakpoints q and q⁰ is equal to the subdierential of ¯f at a point between ¯q and ¯q⁰. A similar argument can be made for a point between a boundary point of Pi+1 and the closest potential breakpoint. Note that the lemma is trivially satised if Pi+1 is a singleton.

We are now ready to prove that if ¯xi ∈ Z, then p/ ^∗(x_i) = C or p^∗(x_i) = x_i + C, for x_i in the neighborhood of ¯xi, for an integer value C. Consider an optimal adjustment

¯

pⁱ_i+1 ∈ R for a given departure time ¯x_i ∈ Z. Because ¯/ f is a polyhedral convex function, we may assume that ¯pⁱi+1 is a breakpoint of ¯f with 0 ∈ ∂ ¯f (¯pⁱ_i+1), or ¯pⁱi+1 is a boundary point of Pi+1.

We rst consider the breakpoints of ¯f. If ¯pⁱi+1 is a breakpoint of ¯f and ¯pⁱi+1∈ Z, then pⁱ_i+1= ¯pⁱ_i+1is the corresponding breakpoint of f. It follows that ∂ ¯f (¯pⁱ_i+1) = ∂f (pⁱ_i+1) 3 0, which implies by convexity that pⁱ_i+1 is optimal for xi. Hence, we have p^∗(x_i) = C with C = ¯pⁱ_i+1, which is integer by assumption. Similarly, if ¯pⁱi+1 is breakpoint of ¯f and

¯

pⁱ_i+1∈ Z, then p/ ⁱi+1− ¯pⁱ_i+1= x_i− ¯x_i (Implication 3) ⇔ pⁱi+1= x_i+ ¯pⁱ_i+1− ¯x_i is optimal for xi. That is, p^∗(xi) = xi+ C, with C = ¯pⁱi+1− ¯xi. Note that C is integer by Implication 2.

Next, we consider the case that ¯pⁱi+1 is a boundary point of Pi+1. By assumption, the boundary points of Pi+1 are integral. It follows that pⁱi+1= ¯pⁱ_i+1. Hence, we can use p^∗(x_i) = C, with C = ¯pⁱi+1 integer. This completes the proof.

Lemma 14. Under Assumption 1, for case POS-AW, we have that ¯ci is piece-wise linear and all breakpoints are integer.

Proof. From the denition of ¯ci, Equation (27), it is straightforward to show that ¯ci is a polyhedral convex function. Hence, ¯ci is piece-wise linear. It remains to prove that ¯ci

has no breakpoints on the integers.

We prove this fact by contradiction. First, we assume that xi ∈ Z is a breakpoint of/

¯

c_i(x_i). By Lemma 13, we have that the optimal adjustment in the neighborhood of xi is given by p^∗(x_i) = C or p^∗(x_i) = x_i+ C for some integer value C.

If we substitute the optimal actions into the denition of ¯ci(x_i), we obtain a straight-forward expression for this function that is valid in the neighborhood of the current state x_i. By analyzing this expression, we show that it only has breakpoints on the integers.

By contradiction, xi ∈ Z cannot be a breakpoint./ By denition, ¯ci(x_i) = min_pⁱ

i+1∈P_i+1f (pⁱ_i+1). Instead of minimizing over pⁱi+1, we use the optimal adjustment p^∗(xi). We then obtain ¯ci(xi) = f (p^∗(xi)).

First consider p^∗(xi) = C. Recall that C ∈ Z by Lemma 13. It follows from Equa-tion (50) that

¯

ci(xi) = gi+1(0, C) + E [hⁱ⁺¹(max{xi+ tii+1, ai+1+ C}, C)] +

E [¯c_i+1(max{x_i+ t_ii+1, a_i+1+ C})] . (54) Next, we consider the breakpoints of ¯ci(x_i). Note that the gi+1 term is constant in x_i, and does not result in any breakpoints. It is straightforward to verify that hi+1 only has breakpoints for integer xi. This follows from the fact that all parameters are integer.

By Assumption 1, ¯ci+1 only has breakpoints for the integers. If xi + t_ii+1 < a_i+1+ C, then the ¯ci+1 term is constant in xi and does not give any potential breakpoints. If xi + tii+1 > ai+1+ C, then potential breakpoints are given by xi + tii+1 ∈ Z, which

implies that the breakpoints xi of ¯ci are integer. Finally, we may have a breakpoint for x_i+ t_ii+1 = a_i+1+ C, which also corresponds to xi ∈ Z.

Hence, for p^∗(x_i) = C, we have shown that if xi ∈ Z is a breakpoint of ¯c/ i, then the function ¯ci dened in the neighborhood of xi only has breakpoints on the integers. By contradiction, xi ∈ Z cannot be a breakpoint. It follows that ¯c/ i only has breakpoints on the integers.

We have to show the same result for p^∗(x_i) = x_i+ C. In this case, we have

¯

c_i(x_i) = g_i+1(0, x_i+ C) + E [hi+1(max{x_i+ t_ii+1, a_i+1+ x_i + C}, x_i+ C)] +

E [¯c_i+1(max{x_i+ t_ii+1, a_i+1+ x_i+ C})] . (55) Again, by combining Assumption 1 with the fact that all parameters are integer, it is straightforward to show that ¯ci only has breakpoints on the integers. Applying the same contradiction argument as for p^∗(x_i) = C completes the proof.

Lemma 15. Under Assumption 1, for case EXT-VW, we have that ¯ci is piece-wise linear and all breakpoints are integer.

Proof. We use the same argument as in Lemma 14, but for a dierent function ¯ci. Specif-ically, we show that the function ¯ci(x_i) = f (p^∗(x_i)) only has breakpoints for integer xi, with f as in Equation (51). The other parts of the proof are identical.

In this case, we obtain the parameters, and using that ¯ci+1 only has breakpoints on the integers (Assumption 1).

The arguments are straightforward, and similar to those in Lemma 11 and Lemma 14.

As such, we omit them here.

After it is shown that (56) only has integer breakpoints, the proof for the EXT-VW case is the same as the proof for the POS-AW case (Lemma 14).

Lemma 16. Under Assumption 1, for case POS-VW, we have that ¯ci is piece-wise linear and all breakpoints are integer.

Proof. Similar to the proof of Lemma 15, but for a dierent function ¯ci. From

Equa-tion (51) we obtain In this case, we obtain

¯

ci(xi) = gi+1(0, p^∗(xi))+

E h

h_i+1

x_i+ t_ii+1+ min{X_i+1^∗ , a_i+1+ p^∗(x_i)} − (x_i+ t_ii+1)
+

, p^∗(x_i)i + E

h

¯ ci+1



xi+ tii+1+ min{X_i+1^∗ , ai+1+ p^∗(xi)} − (xi+ tii+1)
+i

. (57)

Similar as in Lemma 11, Lemma 14, and Lemma 15, it can be proven that ¯ci only has integer breakpoints. These steps are omitted here. Using the argument of Lemma 14 completes the proof.

Proposition 9. The simple DTWAP with discretized time allows for optimal integer decisions in the cases of POS-AW, EXT-VW, and POS-VW.

Proof. By denition, ¯cn = 0, which is linear and does not have breakpoints. Lemmas 14 to 16 show for all i ∈ {0, . . . , n − 1} that if ¯ci+1 is piece-wise linear and only has integer breakpoints, then the same is true for ¯ci. By induction, it follows that ¯ci is piece-wise linear and only has integer breakpoints for all i ∈ {0, . . . , n}.

It then follows from Lemma 11 that for a given x ∈ Z, there exists an optimal adjustment pⁱi+1 ∈ Z. It follows from Lemma 12, and from the denition of the optimal waiting time, that there also exist optimal voluntary waiting times that are integer.

Hence, for a given integer state, there exist optimal integer actions. If integer actions are taken, the next state will again be integer. This can be seen from Equation (29), the denition of ¯ci. It follows that the simple DTWAP with discretized time allows for optimal integer decisions in the cases of POS-AW, EXT-VW, and POS-VW.