Discussion on Time-Stability

In this section we discuss details of time-stability obtained by our suggested approach in terms of its extension and limitations.

2.6.1 Extension to Sojourn Times

As introduced in Section 2.5.1, our suggested approach stabilizes queue length dis-tribution. Then we consider sojourn times as users of data center need to get Quality of Service (QoS) guarantees in terms of sojourn times. In fact, when the distribution of the queue lengths is stabilized, performance analysis of system is very straightforward in terms of sojourn times. Since the distribution of number of jobs in each powered-on class a server is time-stable, the amount of work brought by jobs is time-invariant, and service speed is constant for each server, the sojourn time distribution would also be time-stable.

Therefore based on queue length distribution, we can derive time-stable sojourn time dis-tribution which enables us to provide probabilistic guarantees of the response times for incoming requests. In other words, under a FCFS regime, distribution of sojourn time of class a at time t, W_a(t), can be defined as Ψ_a(w) = P [W_a(t) ≤ w] (which is not de-pendent on time t). Providing probabilistic guarantees on sojourn times (as well as queue lengths) based on time-stability has significant benefits since for transient system with time-varying and non-stationary load, it is extremely difficult to provide guaranteed SLA without assumption for steady-state. For example, our approach is able to provide a bound τ on average sojourn time such that E[W_a(t)] ≤ τ , or tail probability of response time for bound τ such that P [W_a(t) ≤ τ ] ≥ 1 −  which would remain unchanged across time.

Without assuming that system reaches steady-state in each time interval, the only way to provide guarantees is running a large number of servers which causes a much higher energy consumption. In this context, achieving time-stability and providing performance bounds and guarantees based on dummies is the key benefit of our suggested approach.

As described in Section 2.5.2.1, under suggested framework we can decompose our system into simpler homogeneous queues, D/G/1 queue for round-robin routing and M/G/1 queue for Bernoulli routing. In this case, for the M/G/1 queue we have the

LST of the sojourn time distribution ˜Ψ_a(s) for class a request as (Gautam [32]),

Ψ˜_a(s) = (1 − ρ)s ˜G(s)

s − λ_a(1 − ˜G(s)) (2.17)

where λa = ρφθa and ˜G(s) = R∞

0 e^−sxdG(x), the LST of service time distribution. Al-though we do not have a specific formula for the sojourn time distribution of D/G/1 queue case, we can derive the sojourn time distribution from simulation with D/G/1 queue set-ting. Note that it is not easy to derive continuous sojourn time distribution, thus we can derive it based on queue length distribution, π_a(i) itself. Indeed, we can apply derived sojourn time distribution to each server under round-robin routing in time-stable man-ner. Based on our analysis, we can also obtain time-stable performance bound on sojourn times as well as queue lengths. In Section 2.7.2.2 we will introduce simulation results which show that the mean and standard deviation of sojourn times are stabilized with our suggested approach.

2.6.2 Time Interval Length

In order to model time-varying arrivals of requests, we assume that requests arrive according to a piecewise constant non-homogeneous Poisson process where arrival rates of requests of application classes stay constant in each time interval. In this situation, we need to carefully think about the effect of time interval length in terms of whether our suggested approach would be robust to time interval length. In other words, we need to check whether distribution of queue lengths or sojourn times would be time-stable with small time interval length when arrival rates change very fast. In this context, we would like to note that our suggested approach would perform well when time interval length too small to reach steady-state within each time interval and has a sense of the robustness to time interval length. Note that for implementation it is reasonable to assume that the service times and inter arrival times of requests are much smaller than time interval length

since the case where the service times are longer than time interval length is unlikely in practice for data centers. In Section 2.7.2.4, we will compare the simulation results with different time interval lengths to show robustness of our approach.

2.6.3 System Size (Total Number of Servers)

In this study, we consider a fairly common situation in data centers where the traffic of requests is very high and a large number of servers are necessary, and thus we use the asymptotic scaling where both the arrival rates and number of servers are scaled with size N . In fact, our suggested approach itself has limitation with small size N , since for round-robin routing arrival rate into each powered-on server would not be time-homogeneous if size N is small as shown in proof of Theorem 3. Therefore queue length distribution is also non-homogeneous with small size N . Note that for the case of using Bernoulli routing, arrival rate into each powered-on server would be time-stable regardless of size N . In Section 2.7.2.5 we will compare simulation results with small size N for both round-robin and Bernoulli routing cases to check the limitation of our approach.