imbalanced memory consumption may saturate the local node’s memory capacity and band- width while other nodes still have enough resources. As a result, operating systems introduce interleaved memory policy, which trades off the locality for memory balance. Interleaved memory policy allocates memory from the system’s nodes in a round robin order.
Linux, since kernel 2.6, uses the interleaved policy as the default policy on boot-up for the kernel processes. Since kernel structures are shared among running processes, it is advan- tageous to distribute the kernel structures across all nodes. The interleaved policy avoids excessive load on a single node when processes access kernel data structures. The default policy changes to local node when the first userspace process is started [Lameter, 2013]. Processes inherit the default policy, which is local node, when they start running. The Linux scheduler prefers to keep the process running on the same node to benefit from cache locality. In particular, the scheduler leaves the process to run on cores that share L2 cache, then cores that shares L3 cache of the last run. At last and for load balancing needs, the scheduler will move the process to another node.
Operating systems provide tools to control processes with a specific NUMA execution en- vironment. The main Linux tool is numactl, which can display the system’s NUMA con- figuration and set a user-specified NUMA scheduling or memory allocation policy. When a policy is set for a process, all its children inherit the same policy setting. The Linux man numactlcommand displays the tool’s usage and examples.
3.5
Virtual to Physical Memory Page Mapping
Operating systems execute a computer program in an autonomous entity called a process, which is a dynamic instance of a program. Modern computer machines provide physical resources with high capacity, for example manycore processors, large memory space, and video accelerators. Therefore, many processes can run simultaneously by sharing the system resources. One of these resources is the memory, and operating systems incorporate many techniques to organize the memory usage.
Since every process is an autonomous entity, a process creates its own linear and contiguous virtual address space to allocate program code and data. The process treats its virtual memory as if it is the sole owner of the physical memory. The main advantage of using virtual memory is to enable many processes to run at the same time without any interference between the processes.
3.5. VIRTUAL TO PHYSICAL MEMORY PAGE MAPPING
Figure 3.7: Hierarchical Page table for 4KB page size on 64-bit x86 Linux. Source [AMD, 2015]
3.5.1
Memory Pages
The physical memory is divided into chunks of memory called frames. Operating systems set different sizes for memory frames. For instance, Linux sets the default physical mem- ory frame size to 4KB. Bigger frame sizes are available and the system developers provide various configurations to suit different needs. Linux also provides 2MB and 1GB memory frames.
Processes allocate code and data in the virtual memory pages. When a process attempts to access a memory location, the operating system needs to translate the virtual memory address to a physical memory address. The virtual address is divided into two segments: a virtual memory page and an offset address. The translation operation is done with the help of a hardware cache component called Translation Lookaside Buffer (TLB), which stores the recent memory mapping entries. If the virtual memory page is not cached in the TLB, the operating system issues a page fault signal to retrieve the translation entry from the page table (page walk), which contains virtual to physical mapping entries. The operating system walks through the page table and searches for the mapping entry of the requested virtual memory page. If the page is touched for the first time, the operating system allocates a physical page and updates the page table. Once it retrieves the mapping entry, it updates the TLB cache and restarts the memory transaction. This mechanism is called on-demand paging to manage the physical memory efficiently between the running processes.
3.5. VIRTUAL TO PHYSICAL MEMORY PAGE MAPPING
Figure 3.8: Hierarchical Page table for 2MB page size. Source [AMD, 2015]
For 64-bit processor architecture, the address space is huge. AMD Opteron uses 48 bits for physical address space and 48 bits for virtual memory address space. Walking the page table of such a huge memory spaces incur space and time overhead. For example, for 48 bit virtual memory page address and 4KB page size, we get 64GB (36 bits) of page table entries. Therefore, operating systems incorporate a hierarchical page table, which implements multi- level paging tables to manage the page translation entries in a more efficient way. AMD Opteron processors implement four paging levels as described in Figure 3.7. The virtual address contains the address offset (12 bits) and other paging level fields (36 bits). Thus, walking the page table with a hierarchical page table requires less space when compared to the flat paging mechanism.
Memory-intensive and big data applications require large memory capacity that the system provides. These applications allocate large data structures, which cross the memory page boundaries and use many memory pages. Although the virtual memory space for such data structures could be contiguous, the physical memory space is unlikely to be contiguous. For NUMA systems, these memory pages could be mapped to different NUMA nodes. In addi- tion, every memory access requires a translation operation and if the page entry is not cached in the TLB, accessing a data structure would require many page table walking operations. This additional overhead may degrade the application performance.
Operating systems provide a solution to minimize memory page mapping and page table walking. Instead of dividing the physical memory into frames of size 4KB, the frame size
3.6. JAVA VIRTUAL MACHINE AND GARBAGE COLLECTION