This chapter covers virtual memory as implemented in Intel 64. We are going to start by motivating an abstraction over physical memory and then getting a general understanding of how it looks like from a programmer’s perspective. Finally, we will dive into implementation details to achieve a more complete understanding.
4.1 Caching
Let’s start with a truly omnipresent concept called caching.
The Internet is a big data storage. You can access any part of it, but the delay after you made a query can be significant. To smoothen your browsing experience, web browser caches web pages and their elements (images, style sheets, etc.). This way it does not have to download the same data over and over again. In other words, the browser saves the data on the hard drive or in RAM (random access memory) to give much faster access to a local copy. However, downloading the whole Internet is not an option, because the storage on your computer is very limited.
A hard drive is much bigger than RAM but also a great deal slower. This is why all work with data is done after preloading it in RAM. Thus main memory is being used as a cache for data from external storage.
Anyway, a hard drive also has a cache on its own...
On CPU crystal there are several levels of data caches (usually three: L1, L2, L3). Their size is much smaller than the size of main memory, but they are much faster too (the closest level to the CPU is almost as close as registers). Additionally, CPUs possess at least an instruction cache (queue storing instructions) and a Translation Lookaside Buffer to improve virtual memory performance.
Registers are even faster than caches (and smaller) so they are a cache on their own.
Why is this situation so pervasive? In information system, which does not need to give strict guarantees about its performance levels, introducing caches often decreases the average access time (the time between a request and a response). To make it work we need our old friend locality: in each moment of time we only have a small working set of data.
The virtual memory mechanism allows us, among other things, to use physical memory as a cache for chunks of program code and data.
4.2 Motivation
Naturally, given a single task system where there is only one program running at any moment of time, it is wise just to put it directly into physical memory starting at some fixed address. Other components (device drivers, libraries) can also be placed into memory in some fixed order.
In a multitasking-friendly system, however, we prefer a framework supporting a parallel (or pseudo parallel) execution of multiple programs. In this case an operating system needs some kind of memory management to deal with these challenges:
• Executing programs of arbitrary size (maybe even greater than physical memory size). It demands an ability to load only those parts of program we need in the near future.
• Having several programs in memory at the same time.
Programs can interact with external devices, whose response time is usually slow.
During a request to a slow piece of hardware that may last thousands of cycles, we want to lend precious CPUs to other programs. Fast switching between programs is only possible if they are already in memory; otherwise we have to spend much time retrieving them from external storage.
• Storing programs in any place of physical memory.
If we achieve that, we can load pieces of programs in any free part of the memory, even if they are using absolute addressing.
In case of absolute addressing, like mov rax, [0x1010ffba], all addresses including starting address become fixed: all exact address values are written into machine code.
• Freeing programmers from memory management tasks as much as possible.
While programming, we do not want to think about how different memory chips on our target architectures can function, what is the amount of physical memory available, etc. Programmers should pay closer attention to program logic instead.
• Effective usage of shared data and code.
Whenever several programs want to access the same data or code (libraries) files, it is a waste to duplicate them in memory for each additional user.
Virtual memory usage addresses these challenges.
4.3 Address Spaces
Address space is a range of addresses. We see two types of address spaces:
• Physical address, which is used to access the bytes on the real hardware. Naturally, there is a certain memory capacity a processor cannot exceed. It is based on addressing capabilities. For example, a 32-bit system cannot address more than 4GB of memory per process, because 232 different addresses roughly correspond to 4GB of addressed memory. However, we could put less memory inside the machine capable of addressing 4GB, like, 1GB or 2GB. In this case some addresses of the physical address space will become forbidden, because there are no real memory cells behind them.
• Logical address is the address as an application sees it.
In instruction mov rax, [0x10bfd] there is a logical address: 0x10bfd.
A programmer has an illusion that he is the sole memory user. Whatever memory cell he addresses, he never sees data or instructions of other programs, which are running with his own in parallel. Physical memory holds several programs at time, however.
In our circumstances virtual address is synonymous to logical address.
Translation between these two address types is performed by a hardware entity called Memory Management Unit (MMU) with help of multiple translation tables, residing in memory.
4.4 Features
Virtual memory is an abstraction over physical memory. Without it we would work directly with physical memory addresses.
In the presence of virtual memory we can pretend that every program is the only memory consumer, because it is isolated from others in its own address space.
The address space of a single process is split into pages of equal length (usually 4KB). These pages are then dynamically managed. Some of them can be backed up to external storage (in a swap file), and brought back in case of a need.
Virtual memory offers some useful features, by assigning an unusual meaning to memory operations (read, write) on certain memory pages.
• We can communicate with external devices by means of Memory Mapped Input/
Output (e.g., by writing to the addresses, assigned to some device, and reading from them).
• Some pages can correspond to files, taken from external storage with the help of the operating system and file system.
• Some pages can be shared among several processes.
• Most addresses are forbidden—their value is not defined, and an attempt to access them results in an error.1 This situation usually results in abnormal termination of program.
Linux and other Unix-based systems use a signal mechanism to notify
applications of exceptional situations. It is possible to assign a handler to almost all types of signals.
Accessing a forbidden address will be intercepted by the operating system, which will throw a SIGSEGV signal at the application. It is quite common to see an error message, Segmentation fault, in this situation.
• Some pages correspond to files, taken from storage (executable file itself, libraries, etc.), but some do not. These anonymous pages correspond to memory regions of stack and heap —dynamically allocated memory. They are called so because there are no names in file system to which they correspond. To the contrary, an image of the running executable data files and devices (which are abstracted as files too) all have names in the file system.
1An interrupt #PF (Page Fault) occurs.
A continuous area of memory is called a region if:
• It starts at an address, which is multiple of a page size (e.g., 4KB).
• All its pages have the same permissions.
If the free physical memory is over, some pages can be swapped to external storage and stored in a swap file, or just discarded (in case they correspond to some files in file system and were not changed, for example).
In Windows, the file is called PageFile.sys, in *nix systems a dedicated partition is usually allocated on disk.
The choice of pages to be swapped is described by one of the replacement strategies, such as:
• Least recently used.
• Last recently used.
• Random (just pick a random page).
Any kind of a system with caching has a replacement strategy.
■
Question 47 read about different replacement strategies. What other strategies exist?
Each process has a working set of pages. It consists of his exclusive pages present in physical memory.
■
Allocation What happens when a process needs more memory? it cannot get more pages on its own, so it asks the operating system for more pages. the system provides it with additional addresses.
Dynamic memory allocation in higher-level languages (C++, Java, C#, etc.) eventually ends up querying pages from the operating system, using the allocated pages until the process runs out of memory and then querying more pages.
4.5 Example: Accessing Forbidden Address
Now we are going to see a memory map of a single process with our own eyes. It shows which pages are available and what they correspond to. We will observe different kinds of memory regions:
1. Corresponding to executable file, loaded into memory, itself.
2. Corresponding to code libraries.
3. Corresponding to stack and heap (anonymous pages).
4. Just empty regions of forbidden addresses.
Linux offers us an easy-to-use mechanism to explore various useful information about processes, called procfs. It implements a special purpose file system, where by navigating directories and viewing files, one can get access to any process’s memory, environment variables, etc. This file system is mounted in the /proc directory.
Most notably, the file /proc/PID/maps shows a memory map of process with identifier PID.2
2To find the process identifier, use such standard programs as ps or top.
Let’s write a simple program, which enters a loop (and thus does not terminate) (Listing 4-1). It will allow us to see its memory layout while it is running.
Listing 4-1. mappings_loop.asm
Now we have to launch a file /proc/?/maps, where ? is the process ID. See the complete terminal contents in Listing 4-2.
Listing 4-2. mappings_loop
> nasm -felf64 -o main.o mappings_loop.asm
> ld -o main main.o
> ./main &
[1] 2186
> cat /proc/2186/maps
00400000-00401000 r-xp 00000000 08:01 144225 /home/stud/main 00600000-00601000 rwxp 00000000 08:01 144225 /home/stud/main 7fff11ac0000-7fff11ae1000 rwxp 00000000 00:00 0 [stack]
7fff11bfc000-7fff11bfe000 r-xp 00000000 00:00 0 [vdso]
7fff11bfe000-7fff11c00000 r--p 00000000 00:00 0 [vvar]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Left column defines the memory region range. As you may notice, all regions are contained between addresses ending with three hexadecimal zeros. The reason is that they are composed of pages whose size is 4KB each (= 0x1000 bytes).
We observe that different sections defined in the assembly file were loaded as different regions. The first region corresponds to the code section and holds encoded instructions; the second corresponds to data.
As you see, the address space is huge and spans from 0-th to 264 −1-th byte. However, only a few addresses are allocated; the rest are being forbidden.
The second column shows read, write, and execution permissions on pages. It also indicates whether the page is shared among several processes or it is private to this specific process.