Problems With Combining These Methods

Unfortunately, the methods described in Sections 3.2.1 and 3.2.2 cannot be combined effec- tively. This was hinted at when describing the limitations of megatasks, since megatasks limit the parallelism available to MTTs.

More generally, megatasks achieve performance gains by discouraging parallelism (Sec- tion 3.2.1), while MTT co-scheduling achieves gains by encouraging parallelism (Section 3.2.2). There is no straightforward way to combine these mechanisms, since encouraging and discour- aging are opposite goals. The static restrictions on parallelism that are imposed by megatasks make it very difficult to encourage co-scheduling, even if such co-scheduling would actually contribute towards the goal of reducing cache miss rates and preventing thrashing. Upon further reflection on this point, we observed it is necessary only to discourage co-scheduling of tasks fromdifferent MTTs—encouraging co-scheduling of tasks within thesame MTT im- plicitly satisfies this goal. This observation led to the work in [17], which will be discussed further in Chapter 4.

3.3

Conclusion

In this chapter, we have formally introduced the cache-aware co-scheduling problem that we seek to address in this dissertation, and proven that finding an exact solution to the problem is NP-hard in the strong sense. We also described two earlier attempts to address aspects of the problem and discussed limitations that make them both individually restrictive and impractical to combine. In the remaining chapters, we will present a more complete solution to the problem that also has low enough scheduling overheads to be used in a real operating system in practice.

CHAPTER 4

CACHE-AWARE REAL-TIME

SCHEDULING

In this chapter,1 _{we present our methods for cache-aware real-time scheduling. These meth-} ods influence co-scheduling through the use of a technique, called a job promotion, that we introduce to reduce cache miss rates. This chapter is organized as follows. First, we discuss how job promotions can be used to influence co-scheduling, and present an example of how certain co-scheduling choices can affect the cache miss rates experienced by real-time work- loads. We then introduce a large number of cache-aware real-time scheduling policies that dictate when and how job promotions are used; these policies are employed withinheuristics. A large number of heuristics are evaluated in Chapter 6 based on their ability to reduce cache miss rates. Next, we present the tardiness bounds of our method, and explain how buffering can sometimes be used to hide tardiness from an end user. Finally, we present a detailed example of our best-performing heuristic, and discuss implementation concerns that guided our selection of the heuristic to implement withinLITMUSRT, and then conclude.

4.1

Influencing Co-Scheduling: Job Promotions

Co-scheduling decisions can have a significant impact on the cache miss rates of real-time tasks. Our scheduler is concerned with reducing cache miss rates for periodic tasks within soft real-time workloads, where tasks are grouped into MTTs.

1

Contents of this chapter previously appeared in preliminary form in the following papers:

Calandrino, J., and Anderson, J. (2008). Cache-aware real-time scheduling on multicore platforms: Heuristics and a case study. InProceedings of the 20st Euromicro Conference on Real-Time Systems, pages 299–308. Calandrino, J., and Anderson, J. (2009). On the design and implementation of a cache-aware multicore real- time scheduler. InProceedings of the 21st Euromicro Conference on Real-Time Systems, pages 194–204.

768K Job K 768K Job J Priority points moved here Job W Job V MTT: 256K 3 2 1 0

Figure 4.1: An example of where promoting jobs can reduce cache miss rates, assuming the WSSs shown and a cache size of 1 MB.

As stated in Chapter 1, our cache-aware scheduler attempts to reduce cache miss rates by encouraging the co-scheduling of tasks within the same MTT, and discouraging the co- scheduling of tasks within different MTTs when doing so would cause shared cache thrashing. This is achieved through job promotions, wherein a job is given a temporary increase in priority by moving its priority point, originally located at the job deadline, to the current time. Cache impact is determined by examining per-job working set sizes (WSSs), which are specified for each MTT. The per-job WSS of an MTT indicates the amount of memory referenced by all tasks of that MTT while executing one “job” of the MTT, where the ith

job of an MTT consists of the ith jobs of all tasks in the MTT. Shared cache thrashing is assumed to occur during a quantum if the sum of the WSSs of all MTTs with jobs scheduled in that quantum exceeds the shared cache size. For the time being, we simply assume that accurate WSS information already exists for each MTT. In practice, the profiler described in Chapter 5 is responsible for providing this information.

Example (Figure 4.1). Figure 4.1 presents an example for a three-core platform where influencing co-scheduling in the ways previously stated can be useful. Assuming the use of

GEDF _{scheduling, jobs J and K have the highest priority at time 0, but would thrash the}

shared cache if co-scheduled, since the sum of the WSSs of jobsJ andKis 768+768 = 1536K, which exceeds the shared cache size of 1 MB, or 1024K. We can avoid thrashing by scheduling job V instead of job K. Additionally, since jobs V and W are part of the same MTT, we want to encourage job W to be scheduled when job V is scheduled. To accomplish this,

the priorities of jobs V and W need to be increased, which has the potential to negatively impact real-time guarantees later in the schedule. However, if we were to use job promotions, then priority points would remain window-constrained, as defined in Chapter 2; that is, the priority point of each job would remain between its release time and its deadline, and as a result, tardiness would remain bounded as it is under GEDF_{. Knowing this, we can promote}

jobsV andW in Figure 4.1 at time 0. Doing so causesV andW to be scheduled next, which should reduce cache miss rates, while still allowing soft real-time guarantees to be made. This method of promoting jobs indirectly discourages the co-scheduling of certain groups of tasks by encouraging other groups to be co-scheduled instead.

While promoting jobs can lead to reduced cache miss rates in the near term, it might result in higher cache miss rates, or cache thrashing, later. For example, if we always promote jobs from MTTs with the smallest WSSs, then eligible jobs from MTTs with larger WSSs will be pushed later in the schedule, which may be problematic. Thus, the choice of when to promote jobs, and which jobs to promote, can have a substantial impact on the effectiveness of our scheduler. Moreover, this choice is not always straightforward. For this reason, we propose (in this chapter) and evaluate (in Chapter 6) a large number of heuristics within this dissertation. Each heuristic represents a set of policies that dictatewhen to promote jobs and

which jobs to promote. These heuristics, and the policies that they employ, are described next.