Shared Cache Memory

Modern computing systems supplement the main memory of the computer with a set of local registers and one or more levels of cache in a memory hierarchy, where the cache level closest to the processing core(s) is small and fast, but contains only a subset of the contents of memory. Caches are transparent to the programmer, with hardware determining which parts of memory should be placed at any given time in the cache.

Cache memory is the part of the memory hierarchy that has seen the most change over the years, increasing the size and number of cache layers that has been introduced between the register le and memory [31]. For the sake of scalability, exibility, and to deal with power limitations, it has become mainstream to group multiple cores sharing a local cache memory.

The majority of the algorithms proposed to guarantee the schedulability of real-time systems (e.g. RMS or EDF), especially hard real-time ones, assume that tasks are fully

preemptive. This means that tasks can be suspended at arbitrary points in favor of higher-priority tasks. Preemption simplies the schedulability analysis but introduces a runtime overhead (sometime called preemption cost) during task execution, due to the context switching between tasks, the pipeline invalidation delay, and the cache-related preemp-tion delay (CRPD), especially for shared caches in multi-core platforms [16,17]. The preemption cost is often assumed to be constant and speed independent. However, the CRPD is dened as the delay that a preempted job incurs due to a loss of cache afn-ity after resuming execution. This delay may vary greatly, depending on the architecture characteristics and the workload characteristics. Thus, such CPRD delay introduces a sig-nicant source of uncertainty in the timing analysis of real-time systems. In essence, the sharing of local cache memory helps to improve the average case execution time of each task, but can be hazardous to the estimation of the worst-case execution time (WCET).

This is because the number of memory accesses, locations in time, and bus loads origi-nated from other concurrent tasks are difcult to determine precisely. Fig. 2.7shows an example of cache-related preemption delay. It can be seen that a low-priority task t1with T1=10ms and C1=5ms (Fig. 2.7a) is scheduled along with a high-priority task t2 with T1=10ms and C1=3ms (Fig.2.7b). However, if t2unloads sufcient cached data of t1, C1increases and the second job of t2misses its deadline (Fig. 2.7c) [112].

Multiple research works have acknowledged the problem of preemption cost, and specically the CRPD increasing the uncertainties in the execution of tasks, which leads to even more pessimistic estimated WCET for each task, and one of the most common methodologies to tackle such a problem is based on the concept of cache partitioning.

Shared cache partitioning approaches are widely applied in both general-purpose and real-time computing systems [113–116]. However, since a major source of pessimism in WCET estimation comes from shared cache memories, cache memory partitioning has proven to be one of the most effective methods to improve the predictability and

Figure 2.7: Example of cache related preemption delay. A low priority task t₁with T₁=10ms and C1=5ms (a), is scheduled along with a high priority task t2with T1=10ms and C1=3ms (b). If t2unloads sufcient cached data of t1, C1 increases and the second job of t2 misses its deadline (c) [112].

schedulability of real-time systems. Fig 2.8 shows a typical conguration of computing platforms with two cores and two levels of cache memory. The rst level is private to each core, but the second level is shared among the two cores. Such second level of cache memory is divided into partitions. In the gure, two partitions are assigned privately to core 0 and one partition is assigned privately to core 1. Cache partitions can be managed at the hardware level, the compiler level or the operating system level. Therefore, it is possible to assign partitions privately to, not only cores, but also to real-time tasks.

In essence, WCETs are bounded and controlled much more tightly when the cache is partitioned. This allows the estimation of real-time tasks' WCETs relatively tight, yet safe, which promotes processor utilization in both single-core and multi-core platforms.

The main goal of implementing shared cache partitioned approaches, in hard and soft real-time embedded systems, is to improve the predictability of the WCET, and conse-quently improve the response time of the real-time system [62,117–121]. For exam-ple, authors in [122] propose a compiler-based method that partitions the cache among

Figure 2.8: Example of dual-core conguration with cache partitioning [12]

tasks at compilation time. Authors in [113] propose a dynamic cache partitioning scheme that decreases the allocated spaces to the faster threads and allocates them to the critical ones to minimize the system overall response time. Authors in [114] and [115] suggest a software-based page coloring technique, while in [116] a hardware-based partitioned method is introduced.

Existing work on cache partitioning can be largely categorized into two groups: cache allocation policies and cache management schemes [123]. The rst ones focus on policies to dictate how to allocate available cache resources to different tasks to achieve different objectives, such as fairness, priorities, and performance maximization (e.g. [116,118, 124]). The second ones intend to enforce, by means of hardware or software, the distribu-tion of the outcomes of the cache allocadistribu-tions so that each program can access its allocated cache memory (e.g. [62,125,126]). For instance, at the operating system level, cache coloring is an important cache allocation policy to optimize the performance of real-time and general purpose systems by cache allocating the contiguous pages from the cores

point of view. This technique sets up the virtual-to-physical-address translation so that no two tasks access the same cache set in the shared cache and hence one task cannot evict a cache block that another task has fetched to the shared cache [127].

As mentioned in chapter1, multiple research works have realized the need to address the problems of the increasing power consumption in memory systems and thermal is-sues related to such a high power. Some of those research works have focused on the timing problems seen by real-time systems when power and thermal issues are taken into consideration. The power consumption of on-chip cache memories is usually assumed to be a constant value in many research works [128]. Since cache memories are imple-mented close to the CPU, their power can be associated with the power consumed by the entire CPU. Unlike cache memories, external main memory manufactured with DRAM technology usually has an additional power consumption, comparable to that of the CPU in today's platforms and dependent on multiple factors, e.g. the memory capacity and bandwidth. Thus, in what follows, we provide a general introduction of the role and or-ganization of DRAM memories, along with the challenges of real-time system design, considering main memory power and thermal issues.