Demand Forecasting

In the current implementation of the Tasklet system, each Tasklet execution requires a resource request from the consumer to the broker and a resource response from the broker back to the consumer. This approach is feasible for applications that spontaneously offload workload to remote resources. However, in stream processes, where work packets are offloaded continuously, requesting a resource for every Tasklet will lead to a considerable communication overhead and unacceptable delays. To reduce both, we propose a demand estimation mechanism that predicts the future demand of stream processes [160]8. Thus, we can supply each process with a number of resources even before tasks are created. A feedback mechanism from the process helps to make the prediction more accurate. This is an essential part of the mechanism as overestimating the demand would lead to resources that are reserved but would not be used. Whereas the prediction is straightforward in processes with a uniform workload, it gets nontrivial for event-based processes such as in [211] where a face recognition task is issued as soon as an infrared sensor detects a person moving into a security zone. The occurrence of such an event is randomly distributed and can be estimated by fitting a probability distribution function to observed data.

Implementation

Predicting future workload for stream processes is, for example, discussed in [212] and [213]. Future workload can be estimated by using fundamental knowledge of statistical distribution and time series analysis. It is based on two main parts: (i) fitting the distribution to a historical workload and (ii) calculating the

8 _{[160] is joint work with , S. Choochotkaew, H. Yamaguchi, T. Higashino, D. Sch¨afer, and C.}

6.4. Further Scheduling Features 145 Consumer Broker T T T T T T T T T T T T T T T t1 t0 W0 W1 t2 W2 tp t3 1 1 1 1 1 5 -1 4 +1 1 Estimate next: [(t1,5)(t2,?)] →5 Estimate next: [(t1,5) (t2,4)(t3,?)] →4 Reestimate: [(t₁,5) (t₂,4) (t_p,5)(t₃,?)] →5 1 5 -1

Tasklet Resource Request Resource Response Resource Supply Feedback

Figure 6.30.: Demand forecasting mechanism. For each window W the resource demand is predicted based on the demand in the previous windows.

future workload by a time series analysis [213]. We propose a demand estimation mechanism for stream processes that splits up the timeline into evenly sized windows. For each window W , we count the number of tasks and use this data to predict future demands. Figure 6.30 shows the demand estimation mechanism. In the initial window (W0), the broker performs the task allocation on a per-request basis and counts the number of requests. It fits a probability distribution function (PDF) to the data which is used for future resource prediction. As the tasks appear at random points in time, we have selected a Poisson distribution, estimate the parameter λ, and determine the goodness of fit with chi-square (χ2_{). In the} beginning of the next window (W1), the broker sends the number of resources proactively to the consumer. In this way, it does not only reduce the delay but also eliminates the need for a resource request. In case not all resources are used at the end of a window, the consumer sends a feedback to the broker. In case a consumer needs more resources than it received from the broker (as in W2), it also sends a feedback to the broker indicating that it requires additional resources. The feedback messages help the broker to learn about the request pattern of the broker. After each window, the PDF is updated.

In general, there is a trade-off between sending too few or too many resources to the consumer. If the number of required resources is underestimated, the consumer has to send another resource request which will cause additional delay and communication overhead. If the number of required resources is overestimated, the resources remain unused as they are reserved for one particular consumer. We note that in some cases the number of requests may be not be significant and using the estimation may incur unnecessary overhead. For such a case, we

6.4. Further Scheduling Features 146

Application Task Arrival Pattern

TRACK Every time when a person enters the common area, based on the data of a double passive infrared sensor (PIR)

ENV0 When the temperature is out of range from 20◦C to 30◦C.

ENV1 When the temperature is out of range from 20◦_{C to 30}◦_{C and the humidity is}

out of range from 40% to 60%.

ENV2 When the average temperature of one hour is out of range from 20◦C to 30◦C. ENV3 When the average temperature and humidity in one hour is out of range from

20◦C to 30◦C and from 40% to 60%, respectively.

VIDEO We feed uniform requests every 100s based on camera sensors.

Table 6.5.: Applications used in the evaluation and their task arrival patterns.

additionally specify the minimum threshold, NwT H, to determine whether the broker should preallocate resources or not by considering the trade-off between feedback-message overhead plus wasteful allocation and request-message frequency. If the calculated number of resources from the currently considered window is less than NwT H, the broker will allocate one virtual machine responding to each request only, considered as an on-demand response. Otherwise, it will use the estimated response.

Evaluation

To evaluate our approach, we conducted an experiment on collected data from a real deployment in our laboratory that provided realistic arrival patterns. There were six sampling applications that initiated a task based on the data of three sensors under specific conditions. The task arrival patterns are shown in Table 6.5. We simulated a resource sharing system written in Java and measured (i) the number of communication packets including task request and feedback count from a consumer to a broker, (ii) the response count from the broker back to the consumer, and (iii) the number of unused resources at the consumer due to overestimation. We also measured the effect of setting the minimum threshold NwT H, to activate the advanced allocation. Further, we compared our Poisson- based prediction model with the trivial approach called Lookback, which predicts the next window to have the same demand as the previous one.

We compared the total communication costs after running 50 windows for all models against results without estimation. This is shown in Table 6.6 with a window period of 10, 000 seconds. We observe, that applying the Poisson model

6.4. Further Scheduling Features 147

NO EST LOOKBACK POIS-0 POIS-1 POIS-2

VIDEO 8448 241 139 214 230 TRACK 104 123 73 96 96 ENV0 264 107 119 147 195 ENV1 264 107 119 147 208 ENV2 58 89 55 54 55 ENV3 58 89 55 54 55

Table 6.6.: Total communication packets during 50 windows.

(POIS-N : fitting with POISSON distribution and NwT H = N )

Sampling Data N 0 50 100 150 4000 4100 4200

VIDEO TRACK ENV0 ENV1 ENV2 ENV3 NO_EST EST_LOOKBACK EST_POISSON (NwTH=0) EST_POISSON (NwTH=1) EST_POISSON (NwTH=2) Sampling Data N 0 50 100 150 4000 4100 4200

VIDEO TRACK ENV0 ENV1 ENV2 ENV3 NO_EST

EST_LOOKBACK EST_POISSON (NwTH=0) EST_POISSON (NwTH=1) EST_POISSON (NwTH=2)

Figure 6.31.: Request (left) and response (right) count from demand estimation mechanism during 50 Windows

is always beneficial, especially when the number of requests per window is high, like with the VIDEO sampling stream. In general, the Poisson model reduced

the communication costs and the lower minimum threshold (NwT H = 0) mostly

provided a better estimation. For some cases, Lookback estimation incurred slightly lower communication costs. However, the simplicity of this model can also cause unexpected larger costs due to estimation failures.

Figure 6.31 shows that the estimation model can reduce the request count in most cases. Although, in a uniform requesting case, such as VIDEO, the Lookback model can perform better than the more sophisticated models, but it usually suffers from false estimation when the request arrival is fluctuating, as observed in TRACK, ENV2 and ENV3. The increased response count with periodic feedback might affect the total communication if we do not limit the minimum threshold of activation due to sparse requests. It might also be noted that if we set NwT H to 1, the response count will not be larger than the base case NO EST. Resource

6.4. Further Scheduling Features 148

overestimation can cause wasted allocation, which keeps other consumers from using the resources. A higher threshold, NwT H, can reduce the wasted allocation, especially in the low arrival sampling stream.

Overall, the evaluation results show that we can apply the pre-allocation mecha- nism, a proper distribution model, and threshold activation, to reduce the total communication cost, especially for highly frequent request interval applications.