Memory Model

Parallel programming in multicore systems is challenging. Even in a shared-memory sys- tem, programmers can still get unexpected results due to misunderstanding of the under- lying memory consistency model adopted by the platform. A memory model is the set of

specifications that describe how the different components of a system should behave in re- gards to memory operations. The choice of memory model is affected by many components in the system, such as CPUs, caches, buses or interconnects, memory controllers, and com- pilers. For example, whether the CPU issues memory operations in-order or out-of-order, the CPU has write buffer or not, the cache is write-though or write-back, and the compiler optimizes and reorders memory operations or not. By understanding the adopted memory model, programmers can code their programs accordingly to produce functionally correct applications. The memory model acts as an agreement between the computer platform and programs or programmers, in which both hardware and software have to adhere to the memory model; thus providing correct functionality. Memory model is very important and must be considered when porting applications from one platform to another.

Coming from a uniprocessor system in which all threads or tasks have consistent mem- ory view, Sequential Consistency (SC) is a natural extension of the uniprocessor notion of correctness and the most commonly assumed notion of correctness for multiprocessors [14, 153]. In the SC all memory operations are serialized and viewed by the memory sys- tem in the original program order. When local caches are present, SC requires that write operations to the same memory address must be atomic, e.g, no other write to the same address is permitted until the current write is completed and propagated to all nodes. This model is considered as the natural successor to the uniprocessor memory model as programmers can confidently assume that any read will always return the most up-to-date value. The SC model makes migration of applications that share data from a uniprocessor system to a multicore system easy [14].

Although SC is good from a programming point of view, SC imposes some performance challenges such as preventing out-of-order memory operations and enforcing to wait for a previous write operation to complete [14]. Furthermore, some performance optimizations that were valid in a uniprocessor system, such as utilizing processor’s write buffer, now can lead to problems. These restrictions are important because compilers and out-of-order CPUs generally guarantee the order of the dependent memory operations only, assuming that independent memory operations will not affect the same thread or task. In multicores however, this might affect threads or tasks running simultaneously on other cores. In addition, IO devices can be affected by the order of independent memory operations because the order of setting some devices’ registers can be important.

Other less strict or more relaxed models have been proposed to allow for performance optimizations, such as Processor Consistency model (PC), and Total-Store-Order model (TSO) [153]. For example, TSO relaxed SC by allowing the use of the write buffer, hence allowing read after write re-ordering. Generally, different relaxed models allow different op- timizations. To enforce program order, relaxed-modeled CPUs implement specific instruc-

tions to enforce order between memory operations, such as memory fences. Programmers and compilers can utilize these instructions to achieve the desired functionality. Although relaxed models offer better performance, they introduce more complex programming model as programs need to use low-level instructions for synchronization.

Another set of models are the Weak Ordering models (WO) [153]. In a weak ordering model, memory operations are classified as synchronization operations and data opera- tions. The WO models are based on the intuition that reordering memory operations to data regions between synchronization operations does not typically affect the correctness of a program. Therefore, it offers more space for performance optimizations with less complex programming model than the relaxed memory models. To enforce program order between two operations, the programmer is required to identify at least one of the operations as a synchronization operation. In a WO model, all types of re-ordering are allowed between data memory operations. However, all memory operations issued before the synchroniza- tion memory operation have to complete before issuing any following memory operation.

Release Consistency model (RC) follows the philosophy of the WO. However, RC pro- vides a clear and higher-level programming model. In RC synchronization operations are divided into acquire and release operations; synchronized memory operations are enveloped between acquire and release. Acquire and release are analogues to lock and unlock respec- tively. Acquire operations tries to lock (has exclusive access) to a flag that is used to sy nchronize accesses between parallel threads to what is called a Critical Section (CS). A CS is a piece of code that accesses memory addresses subjected to synchronization. On the other hand, release operation unlocks (allow access) the locked flag to allow other threads to lock it again and gain access to the critical section. All memory operations between acquire and release are not restricted to be in the program order. However, RC provides correctness similar to SC as it guarantees atomicity and sequential ordering between criti- cal sections. For example, depending on the implementation, a release operation might be delayed until all memory operations enveloped between the acquire and release are com- pleted and delivered to all relevant nodes. The implementation of this model can be done in different ways. For example, compilers and programming languages can implement the semantics of the model utilizing the low-level memory fences, and read-and-modify instruc- tions. In addition, operating systems can provide a synchronization library that implement acquire and release semantics.

In this work, RC is chosen to allow more room for performance optimization while keeping the programming complexity minimal. This become relevant when we discuss the intra-task communication of parallel tasks in Chapter 8. In particular, our methodology is to keep the hardware design simple, which what the RC model allows.

Cache Coherency

As mentioned earlier, different components in the system, such as cache controller and interconnect, affect the consistency model. When private caches are not coherent, violation to the memory consistency can happen. In a shared memory system, a coherence problem arises if multiple cores have access to multiple copies of the same data, such as in private caches. Therefore, memory coherence is required in shared memory systems to keep only one version of data that is equally seen by all cores [115,148].

A hardware platform implements a coherence protocol to adhere to a specific memory model. Based on the number of shared or distributed memories in the system, such as private/shared caches and main memory, and how they are connected, the implementation complexity of the protocol can vary. There are many cache coherence protocols that have been introduced in the literature. Examples are MSI, MESI, MOSI, MOESI, MERSI, MESIF, write-once, Synapse, Berkeley, Firefly, Dragon, and ARM AMBA 4 ACE [21,85] Generally, there are two broad classes of coherence protocols, snooping and directory- based. Snooping protocols tend to be faster, if a common bus with enough bandwidth is available, since every request is broadcasted to all nodes in the system. However, the snooping protocol is not scalable as the bus will not be able to provide enough bandwidth as the number of cores increases. Directory-based, on the other hand, is scalable and can be distributed but is however slower than snooping, as the directory transactions have to traverse through the interconnect. In addition, directory-based protocol requires dedicated memory to store the directory.

Independently of snooping or directory-based protocols, the other major design decision in a coherence protocol is to decide what to do when a core writes to a block. There are two options, invalidate and update. Invalidate protocol is when a core invalidates the copies in all other caches before writing to the block. If another core wishes to read the block after its copy has been invalidated, it has to initiate a new coherence transaction to obtain the block, and it will obtain a copy from the core that wrote it, thus preserving coherence. On the other hand, in update protocol, when a core wishes to write a block, it updates (pushes) the copies in all other caches to reflect the new value it wrote to the block. Update protocols reduce the latency for a core to read a newly written block. However, update protocols typically consume substantially more bandwidth than invalidate protocols because update messages are larger than invalidate messages. Most of currently implemented protocols are based on invalidation protocols.

Although there has been a lot of work in improving cache coherency in general-purpose architecture, real-time systems are still behind in analyzing and introducing predictable cache coherency protocols.

In the real-time community, Sarkar et al. [143] introduced a push mechanism that pushes some or the whole content of a cache to another cache. This work is presented in the context of tasks migration, in which a migrated task does not suffer cache warm-up delay. Similarly, Pyka et al. [138,137] introduced the on-demand coherent cache (ODC2_). Unlike [143], this work aims specifically for a predictable coherence mechanism for real-time systems. The ODC2 _{has two modes, share mode and private mode. In the private mode it} works as a regular non-coherent cache. Whereas, in the share mode it provides coherency. The share mode is only activated in critical sections. When exiting a critical section, all the cachelines accessed during the critical section are invalidated and flushed-back to main memory. Although the (ODC2_{) provides predictable coherency mechanism, it still uses} main memory for data sharing, which can reduce the performance.

Nowotsch et al. [121] indicates that in COTS systems performance can be degraded by just enabling cache coherence even with no data sharing. In addition, [166] studied MESI- based coherence protocols for real-time suitability. The study concluded that to provide a statically predictable timing behavior of the MESI-based cache coherence protocol, few conditions have to be met. It requires a time predictable TDMA bus interconnect with an analyzable memory controller and an update-based dual-ported direct-mapped cache using bus-snarfing.

Haasn et al. [72] recently proposed (PMSI): a predictable cache coherence for multi- core systems. The authors extends the classic MSI protocol and enhanced it with transient coherence states to bound the worst-case access latency. This work also reports possible sources of unpredictable behavior on conventional coherence protocols. The results show that, in the worst-case, the arbitration latency scales linearly, whereas the coherence latency scales quadratically with the number of cores. Although, quadratic coherence latency is observed for wost-case scenarios, for general-purpose computing the cost is more reasonable for the average-case scenarios [107]

One-Way Shared Memory [144], proposed a direct communication between cores’ local memories via a time predictable NoC without relying on main memory. This is to avoid the unpredictable access time to local memory and the contention on main memory which is a communication bottleneck in the cache-based system. In [144], the distributed local memories are connected in a predictable and coherent way so they appear as one shared memory. Specifically, each local memory is split into TX and RX communication channels such that each core has (m-1) TX and (m-1) RX channels. The Network Interface (NI) continuously synchronizes the local memory with the rest of the network. This NoC adopts a static TDM schedule to arbitrate between injectors and guarantees conflict free commu- nication. The all-to-all communication mechanism implemented in the NoC guaranteed that all communications data are fully propagated to all other destinations within bounded

time (hyper period).