Basic Concepts
Virtual shared memory (VSM) was first developed in a Ph.D. thesis ky Kai Li (1986). The
idea is to implement a coherent shared memory on a network of processors without physi-cally shared memory. The system uses virtual addresses for memory references. The VSM address space is organized in pages which can be accessed by any node in the system. A me-mory-mapping manager on each node views its local memory as a large cache of pages for its associated processor.
Pages that are marked read-only can have copies in the physical memory of other nodes. A page currently written may reside in only one local memory.
The page fault mechanism of the MMU is used to handle the access and an VSM manager called by the MMU trap can perform the consistency protocol and the copy of pages.
• page based access control • page fault handler
• Home node
The node holding the primary page • primary page
The page located at the home node, only copies of this page are distributed to other nodes • copied page
A Page which is a copy of a primary page
Read access Read access vaddr fault vaddr Translation Cache Page Table copy Paged Memory 1. read Translation Cache Page Table primary Paged Memory page vaddr vaddr 2. read request 3. store copy Node # i Node # j
1. access to variable by a read => page fault 2. find the appropriate primary page (Node #j) 3. copy the page to the requester
and send a read request to this node
4. enter new address
4. enter new address into page table addr
and restart page access
Page fault handler is software-based. The MMU traps into the OS on a page miss. Instead of transferring the missing page from swap space into memory, the page fault handler sends a message to the home node requesting a copy of the page. Upon reception of the answer, the page is stored at the appropriate location in main memory and the process can resume exe-cution.
Functions of the page fault handler
• finding the source of state information by using a directory; a page has a "ho-me" determined by (a part of) the virtual address
• finding the appropriate copies
• communicating with the copies and copy the page to the requester; setting page access rights
Write access Write access vaddr.m fault vaddr.m Translation Cache Page Table Paged Memory object access vaddr.n Translation Cache Page Table Paged Memory object access vaddr.o Translation Cache Page Table Paged Memory object vaddr.o 1. write Node # i co of 5. invalidate
4. store primary page
5. invalidate 3. new owner 3. new owner Node # j Node # k addr.n
1. access to variable by a write => page fault 2. find the appropriate primary page (Node #j)
4. store primary page to write requester and send a write request to this node
6. enter new address into page table and restart page access
2. request page
6. enter new address
pr pa
3. messages to all nodes with read copies to enter new single write owner
5. invalidate all copies
addr
The write access in a virtual shared memory (VSM) is much more complicated than a read. The single writer coherency scheme requires the invalidation of all copies handed out for
Example of a Coherency Protocol for VSM INVALID SHARED OWNED LOCAL remote read request define local remote write request remote write request read miss write miss write hit define local define local define global
States of the coherence protocol
• invalid: access not possible
• shared: only read-access allowed, copies are present on other nodes • owned: write access possible, copies are not allowed
Methode for unification of partly changed copies copy difference replicate update replicate update sequential consistency orig data orig data copy difference orig data updated data
local changes local changes
0.8 1.5 copy
orig data
1. copy original data 1.1 2.2 3.3 4.4 5.5 6.6 1.1 2.2 3.3 4.4 5.5 6.6 orig data 2. local changes 1.1 2.2 3.3 4.4 5.5 6.6 0.8 1.5 3.3 4.4 5.5 6.6 local changes 0.8 1.5 copy orig data 3. determine changes 1.1 2.2 3.3 4.4 5.5 6.6 0.8 1.5 3.3 4.4 5.5 6.6 local changes subtraction 0.3 0.7 0 0 0 0 orig data 1.1 2.2 3.3 4.4 5.5 6.6 partial changed subtraction updated data orig data 1.1 2.2 3.3 4.4 5.5 6.6
