Hint-Guided Data Placement

After various memory access patterns are recognized for static and dynamic memory areas, an efficient hint representation method becomes necessary so that these patterns can be utilized by later executions. Since hints for dynamically allocated data and static data are different, they are presented separately. A static hint gives the target cache slice for a static page explicitly, thus can be used directly at run time. A dynamic hint only tells the type of a malloc instance in the source code. It needs to be translated into actual mappings when the starting address and the size of a memory area are determined by the corresponding malloc. This section discusses how to utilize these dynamic pattern hints at run time to guide data placement.

The OS reads carried hints when loading a program for execution. Each hint is repre- sented by the OS internally using a data structure such as following:

struct DynamicHint { Integer MallocID Integer HintType Integer Counter Integer Target }

When malloc is called, the corresponding MallocID (e.g., a combination of the containing file name and the line number) is searched against the hint list. If there is a match, a new pattern descriptor is created. The following pseudo-code describes how a dynamic hint is translated into a pattern descriptor at the time of malloc:

struct PatternDescriptor { Integer PatternType Addr_t Start Addr_t End Integer Target Pointer Next }

MallocHandler() {

normal malloc operation ...

ID = current malloc identifier

Start = start address of the allocated memory End = end address of new allocated memory ThreadID = the calling thread ID

for each hint item h in hint list if ID equals h.MallocID

p = new PatternDescriptor structure p.PatternType = h.HintType p.Start = Start p.End = End if p.PatternType is OrderScattered p.Target = h.Counter h.Counter++ if p.PatternType is PrivateScattered p.Target = ThreadID if p.PatternType is PrivateOwner p.Target = h.Target

add p into the pattern list exit the loop

return start address }

The following text describes how our scheme generates target tile ID (i.e., where the page is placed in the cache) from pattern descriptors at run time.

Even Partition Pattern: In the above MallocHandler(), only the whole memory range is generated for the even partition pattern. The actual data placement decision is deferred until a page fault is triggered. The whole memory range is artificially partitioned into 16 equal sub-ranges. Then the triggering virtual address is checked to find out which range it falls in. The corresponding sub-range index is the target tile ID. The following pseudo-code illustrates this process:

PageFaultHandler() {

VA = virtual address that triggers the page fault

/* the default value -1 indicates a shared pattern */ TargetTID = -1

for each pattern p in the pattern list

if VA falls in the range [p.Start, p.End] if p.PatternType is EvenPartition

PartitionSize = (p.End - p.Start) / NCore TargetTID = (VA - p.Start) / PartitionSize save TargetTID into the corresponding page table entry }

Ordered Scattered Pattern: As described in MallocHandler(), the target tile ID is determined during the process of translation from the hint to its pattern descriptor. A counter with an initial value of 0 is associated with the hint structure. Each malloc call uses the counter value as the target tile ID and increments the counter. For example, if the counter value in the hint structure for a malloc is 2, then the memory area returned by the next call to this malloc is assigned to the cache slice in tile 2. The counter value is then increased to 3. Given the pattern descriptor, the following pseudo-code illustrates how tile ID is determined:

VA = virtual address that triggers the page fault

/* the default value -1 indicates a shared pattern */ TargetTID = -1

for each pattern p in the pattern list

if VA falls in the range [p.Start, p.End] TargetTID = p.Target

save TargetTID into the corresponding page table entry }

Private Scattered Pattern: If a hint indicates that a malloc has this pattern, the memory area returned by this malloc is placed directly to the tile, who calls this malloc instance. The implementation of MallocHandler() is the same as Ordered Scattered Pattern.

Private Owner Pattern: For this pattern, a target tile ID comes with the pattern type in the hint explicitly. The allocated dynamic area is simply placed in the specified target cache slice. The implementation of MallocHandler() is the same as Ordered Scattered Pattern.

Shared Pattern: As shown in MallocHandler(), when there is no match in the pattern list, -1 is assigned to the target tile ID to indicate a shared pattern for a page. By default, no effort is made to optimize for a dynamic data area of this pattern. Since the data in a page are shared by all threads, placing this page in any single cache slice would cause uneven access latency among all sharers. It can also create a hot-spot in the hosting cache slice. As a result, it seems reasonable to distribute the data in this page at cache block granularity instead of page size to balance L2 cache accesses among all cache slices. Moreover, it is beneficial to combine other hardware-based techniques to improve access latency for highly shared data since shared memory accesses account for a significant portion of all L2 cache accesses. To capture temporal data affinity for shared data, one can benefit from the victim replication scheme [111]. As the private data are correctly placed, the amount of data replication and the number of resulting conflict misses can be reduced significantly. Other hardware techniques such as multicast can also be employed. The motivation here is that

highly shared data are likely possessed by multiple tiles simultaneously in their L1 caches. If the load request misses in the local L1 cache, it is very likely to find the data in one of its neighbors’ L1 caches. Based on this observation, a multicast scheme can work as follows: A read miss leads to an access to the home directory to find out where to fetch the data. On its way to the remote directory, L1 caches of all visited tiles are checked for the data. If a copy of the data is found, it is replied immediately to the requester, which could resume execution earlier. A similar multicast scheme was recently proposed in [19], albeit in a different context (private L2 cache).

5.3 EVALUATION RESULTS

Five cache schemes are evaluated and compared in order to demonstrate the benefit of the proposed data placement scheme. These five of them are: the shared cache scheme (L2S), the private cache scheme (L2P), the victim replication scheme (L2VR), the hint-guided data placement scheme (L2H), the hint-guided data placement and data replication scheme L2HR. For L2HR, the page placement in the L2 cache for private data is guided by L2H as usual, but data replication is used to save remote access latency for shared data blocks. The data replication is also guided by the derived hints.