Numerical Experiment

In this section, we report on the numerical experiments conducted to implement and com- pare the performance of the dynamic SAP using the three queueing models. The first queueing model (M/G/c/c) assumes that the arrival process is Poisson over each period. We use sta- tistical tests from Brown et al. [2005] to validate this assumption, and show that overall there is no evidence to reject the null hypothesis that the arrival process is NHPP with piecewise constant (PC) arrival rates. See Appendix A.1.3 for details. This PC arrival rate function can also be used in the dynamic SAP calculation with theM(t)/G/∞ model.

To implement the dynamic SAP, we forecast arrival ratesλn,iusing the methods introduced in Section 2.5. We assume that the mean service timessn,i are time-independent and simply take the sample mean sn for application n as the estimate of sn,i for all the periods. The approaches we use to estimate other parameters such as θn,i,pn,i of theMX/M/∞ queue are introduced later.

We simulate the system with the trace data from the VCL to compare the efficacy of all the methods introduced earlier. Each simulation setup can be described by four attributes described below:

Attribute 1. Forecasting Method: MA vs. SVD;

Attribute 2. Forecasting Level: Mean vs. upper bound of 95% prediction interval; Attribute 3. Estimation Method: SIPP vs. SDPP;

Attribute 4. Queueing Model: M/G/c/cqueue vs. MX/M/∞queue vs. M(t)/G/∞ queue. Thus there are a total of 2×2×2×3 = 24 simulation setups.

As a general parameter setting, we assume the targeted global probability of delay α∗ is 5%, and the targeted global blocking probability β∗ is 0.1%. There are 400 dedicated pools and one flexible pool. To forecast the arrival rates, we choose the length of rolling horizon

w = 30 days, and assume that there are P = 24 hourly periods in a day. Applying methods described in Section 2.5 for each application, we obtain four different sets of forecastsλ(1)_n,i and

λ(2)_n,i using the MA method, λ(3)_n,i and λ_n,i(4) using the SVD approach (λ(1)_n,i and λ(3)_n,i are at the mean forecasting level, and λ(2)_n,i and λ(4)_n,i are at the upper bound of 95% prediction interval level). Besides, we estimate the mean service time sn by the sample mean of all the recorded

service times of applicationn, and we set loading timeun= 5 mins for all the applications. To determine the number of servers assigned over the ith interval, we letτi =i−1 (i≥1, in hours), and record the number of type-n users using the dedicated servers (Bn(i−1)) and the flexible servers (Fn(i−1)) at the end of the earlier period. Then the SIPP method and the SDPP method yield the following estimates:

ˆ λn,i=λ(n,ij), ˆ λmod(n,i)= [λ (j) n,i+Bn(i−1) +Fn(i−1)]/(1 +sn). 2.6.1 M/G/c/c Queue.

In this case, all of the estimated arrival rates are used directly as the input to the greedy algorithm of Section 2.4.2.1. We record the numbers of users blocked from all of the dedicated pools (overflows) and use them to forecast the arrivals to the flexible pool. The same procedures are used as those applied to the dedicated pools. The simulation results are presented in Figure 2.7.

The three plots on the left show the results using the mean forecasting level, while that right ones use the upper bound of 95% prediction interval. Panels (a) and (b) show the average probability of delay over the 24 hours in a day. We see that using the SVD forecasting method produces lower probabilities, and SDPP outperforms SIPP. In panel (a) we find all of the results are above the target level ofα∗ = 5%, which means our policies underestimate the number of servers needed. With the upper bound of 95% prediction interval, in panel (b), the two curves using the MA forecasting methods again fail to meet the target level. The other two curves using the SVD forecasting method are very close to, and hit our target 5% during the off-peak hours (3am-8am), but during the peak hours, the probability of delay goes up to 9%.

In Figure 2.7, Panels (c) and (d) plot the average blocking probability over the 24 hours. At the mean forecasting level, we see that policies using SDPP have the best performance, though none of them hit the targetβ∗ = 0.1%. In Panel (d), all of the blocking probabilities are much lower, even though only the policy using MA combined with SDPP method frequently hits the target.

We also plot the average number of on servers (dedicated + flexible) over the 24 hours in Figure 2.7(e) and (f). Policies using the SVD forecasting method usually allocate more servers during the off-peak hour, which explains the lower probabilities of delay in (a) and (b). Besides, SDPP methods require less on servers than SIPP, while achieving the similar performance as shown in panel (b). Hence, for this queueing model, considering Attributes 1-3, we recommend the upper bound of 95% prediction interval as forecasting level, SDPP method, combined with SVD for dedicated pools and MA for flexible servers.

2.6.2 MX/M/∞ Queue.

Letσ2_n,i be the variance of the number of arrivals. From Eq. 2.7 and 2.8 we see that

pn,i= 2/(

σ_n,i2 λn,i

+ 1), (2.19)

θn,i=λn,ipn,i, which suggests,

rn,i=λn,isnpn,i/(1−pn,i).

Note thatQd(dn,i, rn,i) is a function ofrn,iand pn,i, and hence can be viewed as a function of λn,i and pn,i. Again we estimate λn,i using one of the combinations of our forecasting and estimation methods, while we estimate pn,i separately by computing the sample mean and variance of the corresponding hourly arrivals of the preceding 30 days, before plugging it into Eq. 2.19.

Furthermore, for a small fraction of the cases where pn,i >0.99, we use the Erlang-B loss formula to approximate the blocking probability instead of Eq. 2.9 to avoid computational instability. This makes sense since the Poisson distribution is the limiting case of the negative binomial distributionN B(r,1−p) when p→1 (or equivalently, asr→ ∞).

We collect the historical hourly overflow to the flexible pool and estimateλf,iandpf,iin the same way as for the dedicated pools. These parameters can also be computed using Eq. 2.10 and 2.11.

they are quite similar to those shown in Figure 2.7. (See Appendix A.1.4 for the corresponding figure.) Again, we find that the best policy uses 95% forecasting level, SDPP, combined with SVD forecasting for the dedicated pools and MA forecasting for the flexible pool. We will present the differences among the various queueing models (Attribute 4) later in Section 2.7, by using the same Attributes 1-3 for all queueing models.

2.6.3 M(t)/G/∞ Queue.

In this case, we first obtain the arrival function λn,i(t) by formulating a piecewise linear function using the hourly estimates ˆλn,i (or ˆλmod(n,i)). Then we approximate the CDF of the

service time of each application by its own empirical CDF. Finally we compute the integral in Eq. 2.12 numerically, and obtainmn(t), the mean number of busy servers of application nat timet. We use the same techniques for the flexible pool as in the case ofM/G/c/cqueue. We arrive at the same conclusion in comparing Attributes 1-3. See Section 2.7 for the comparison among the three queueing models.