Priority Tasklets

Our current implementation of the Tasklet system does not support queuing of Tasklets and requests to busy providers are rejected. However, supporting queues would allow for flexibility as it decouples the scheduling process from the execution. While queuing does not solve the problem of a permanently congested execution environment, it makes a system tolerant to temporary high loads. Tasklets that are scheduled during a high load are not rejected but added to a queue and will eventually be executed. On the one hand, this reduces the workload for the scheduler. On the other hand, it introduces the risk that Tasklets are delayed as they are queued for some time. Thus, in this section, we introduce priority Tasklets that can skip the lines and are executed before any non-prioritized Tasklet [208]5.

Implementation

To implement priority Tasklets we used the concept of QoC. Priority can be either implemented as a QoC goal which developers can specify for a Tasklet or a QoC mechanism that enforces one or more QoC goals. As priority is not a goal per se but rather facilitates a timely execution and avoids delays, we implemented it as a mechanism that can be used to enforce the speed goal. To avoid the excessive use of the Priority QoC mechanism, the usage might be charged.

6.4. Further Scheduling Features 136

In a Tasklet system that supports queuing, resource providers execute Tasklets in a FIFO order. One Tasklet is executed at one TVM at a time. Hence, the waiting period for a Tasklet is determined by the workload of the provider it has been assigned to.

Introducing a simple model of execution levels, we distinguish between two different queues. In order to execute the incoming Tasklets based on their priority declaration, the modification of the providers is minimal. Instead of holding only one queue for Tasklets, each provider holds two queues – the economy queue and the priority queue. Having completed the execution of a Tasklet, the provider first checks the priority queue for further assignments. Only if the priority queue is empty, the provider checks and processes its economy queue. Hence, as long as a provider has prioritized Tasklets in its queue, the economy Tasklets are neglected. In case both queues are empty, the provider will proceed with the next incoming Tasklet, whether priority or not. As there are no guarantees for the best-effort Tasklets, priority execution does not guarantee an immediate execution or a maximum waiting period either, but ensures that each priority Tasklet is processed before the next economy Tasklet will be executed.

Developers do not directly request Tasklets to have priority as they cannot know whether Tasklets are executed in a congested environment or in an environment where multiple idle resources are available. Thus, developers only set the Speed QoC goal for time-critical Tasklets. The Tasklet middleware then evaluates the current state of the system and decides how this goal should be enforced. In Section 6.1, we have already demonstrated how the QoC mechanisms Strong Distribution and Speed Filter can be used to reduce the execution time. When the middleware detects that the system is congested, it can use Priority as a (further) mechanism to implement a timely execution. In case of an underutilized system, this mechanism might not be used to avoid the charges for prioritized Tasklets. Evaluation

We evaluated priority Tasklets on two computers and a blade server connected via Ethernet. The applications that created the Tasklets were executed on a Lenovo T430s laptop equipped with a 64-Bit Windows 7 Professional, an Intel i7-3520 Dual-Core 2.9 GHz CPU, and 8 GB RAM. For the providers we used a desktop PC running a 64-Bit Windows 8 OS equipped with an Intel i5-2500K Quad-Core 3.3

6.4. Further Scheduling Features 137

Scenario 1a 1b 2a 2b 3

# of applications 20 20 20 20 10 + 10 after 120s

# of Tasklets per application 100 200 100 200 200 / 200

# of providers 10 10 10 10 10

Seconds between requests 2-8 1-4 2-8 1-4 3-9 / 1-3

Economy/priority ratio 100/0 100/0 80/20 80/20 80/20

Table 6.4.: Overview of evaluation scenarios.

(a) Underutilization (1a). (b) Overutilization (1b). Figure 6.23.: Waiting period per Tasklet in Scenario 1a and 1b.

GHz CPU and 8 GB RAM. Finally, the single broker ran on the blade server with a 2x Quad-Core Intel Xeon with 2.33 GHz CPU, 6 GB RAM, and a 64-Bit Windows Server 2008 Standard Edition. During the evaluation, the applications repeatedly issued Tasklet execution requests to the broker based on the parameters of the individual scenario. For each scenario, we specified the number of applications in the system, the number of Tasklets per application, the number of providers, the frequency of Tasklet requests and the ratio between priority and non-priority (economy) Tasklets. Table 6.4 summarizes these scenarios.

The first two scenarios (1a, 1b) served as a benchmark and only involved economy Tasklets, meaning that all Tasklets had the same priority. Providers strictly stuck to the FIFO order for Tasklet execution and there was no way to avoid a delayed execution in case of a congested system.

In the second two scenarios (2a, 2b), we used the same basic settings as before, but labeled 20% of the Tasklets as priority Tasklets. In both, Scenario 1a and 2a, we underutilized the capacity of the providers in order to examine the difference in waiting time between economy and priority Tasklets. This means that the

6.4. Further Scheduling Features 138

(a) Underutilization (2a). (b) Overutilization (2b). Figure 6.24.: Waiting period per Tasklet in Scenario 2a and 2b.

providers could execute the Tasklets faster than they came in. In scenarios 1b and 2b, in contrast, we increased the number of Tasklets per application and reduced the waiting period between Tasklet execution requests, in order to overutilize the providers. Here, the waiting time of the economy Tasklets was expected to increase.

Scenario 3 featured both under- and overutilization in order to see how the system recovers from overload. That is, we created a steady workload that underutilizes the providers comparable to Scenario 2a. After two minutes, we started a second set of applications and flooded the providers with Tasklet execution requests, in order to create a peak in the workload. When the second set of applications has finished sending Tasklets, we expected the providers to slowly work off the build-up. Even during peak load, we expected priority Tasklets to be executed with a minimal waiting period.

Figure 6.23 shows the waiting period of the Tasklets in Scenarios 1a and 1b during underutilization (Figure 6.23a) and overutilization (Figure 6.23b). Whereas in the first case, the Tasklet execution was almost immediate, the waiting period increases linearly when the queue for economy Tasklets built up in size during overutilization.

Similar to above, Figure 6.24 shows the waiting periods of the Tasklets in Scenario 2a and 2b. During underutilization (Figure 6.24a) Tasklet execution was almost immediate, and there was no measurable difference in the waiting periods of the economy and priority Tasklets. During overutilization (Figure 6.24b) we see that

6.4. Further Scheduling Features 139

(a) Waiting period per Tasklet. (b) Queue sizes per Tasklet.

Figure 6.25.: Waiting time and queue size in Scenario 3 in which the system is temporarily overloaded.

even though the providers could not keep pace with their economy queue, the waiting time for priority Tasklets stayed consistently small as intended. Naturally, this would no longer be the case when the amount of priority Tasklets exceeded the provider capacities.

Figure 6.25a shows the waiting period per Tasklet in Scenario 3. At first, the providers were not fully utilized and so the waiting periods for both economy and priority Tasklets were very short. After two minutes, the second set of applications was launched and the number of Tasklet execution requests increased enough to exceed the computing capacity of the providers. However, as in Scenario 2b, the priority Tasklets were still executed almost instantly, whereas the economy Tasklets’ waiting periods increased dramatically and peaked at about the eight minute mark, at which the second set of applications had finished requesting Tasklet executions (see Figure 6.25b). Following this peak, the system recovered and worked off the build-up.

Figure 6.25b shows the respective queue sizes with regard to the accumulative number of requests (dashed line) in Scenario 3. The behavior of the queue sizes was consistent with the waiting periods shown in Figure 6.25a. The size of the economy queue increased dramatically with the start of the second set of applications, while the size of the priority queue stayed almost constant. Overall, our evaluation shows that priority Tasklets were always executed after a consistently short delay, even if the overall workload exceeded the capacity of the providers.

6.4. Further Scheduling Features 140 Tasklet 1 Tasklet 2 Tasklet 3 Tasklet 4 Deadline Tasklet 1 Tasklet 2 Tasklet 3 Tasklet 4 Performance Reference provider

(with estimated runtime = actual runtime)

Estimated runtime Actual runtime

Figure 6.26.: Greedy scheduling in a moderately utilized environment. All Tasklets can be computed within the deadline.