An IPC Software Layer for Building a Distributed Virtualization System

An IPC Software Layer for Building a Distributed

Virtualization System

Pablo Pessolani1, Toni Cortés2, Fernando G. Tinetti3,4 and Silvio Gonnet1,5,

'F R S F , U n iv ersid a d T e c n o ló g ic a N a c io n a l, Santa F e , A rg en tin a ^ B arcelon a S u p ercom p u tin g C en ter & U P C , B a r c e lo n a , E spaña

3III-L ID I F acu ltad d e In fo rm á tica - U N L P , L a Plata, 4C o m is ió n d e In v e stig a c io n e s C ie n tífica s, B u e n o s A ir e s , A rg en tin a

5I N G A R - C O N IC E T , Santa F e , A r g e n tin a

Abstract. H a rd w are virtu a lization te c h n o lo g ie s w e r e p rin cip a lly d e sig n e d fo r server c o n s o lid a tio n , a llo w in g m u ltip le O p era tin g S ystem s in stan ces to b e c o ­ lo c a te d o n a sin g le p h y s ic a l com p u ter. B u t, IaaS p ro v id e rs a lw a y s n e e d h ig h er le v e ls o f p e rfo rm a n ce , sca la b ility and a v aila bility fo r th eir v irtu a lization services. T h e se requ irem en ts c o u ld b e m et b y a distribu ted v irtu a lization t e c h n o lo g y , w h ic h ex ten d s th e b ou n d a ries o f a v irtu a lization ab straction b e y o n d a host. A s a distribu ted system , it d ep e n d s o n the co m m u n ica tio n s b e tw e e n its co m p o n e n ts scattered in several n o d e s o f a virtu a lization cluster. T h is w o r k con trib u tes M 3 - IP C , an IP C softw a re la y er d e s ig n e d to fa cilitate the d e v e lo p m e n t o f an O S -b a s e d distribu ted virtualization.

Keywords: D istrib u ted S y stem s, V irtu a liza tion , IP C .

1 Introduction

Hardware virtualization is the most common technology that provides Virtual Machines (VM) as abstractions. Operating System (OS) level virtualization is another technology that is located at a higher level, encapsulating user-space applications within Containers or Jails or Virtual Operating Systems (VOS) [1]. On the other hand, in the 1980s, several Distributed Operating Systems (DOS) [2, 3, 4, 5] were developed as a consequence of the limited CPU performance of a single host and the need for scalability and higher computing power.

Thinking of a distributed virtualization technology seems to make sense to achieve higher performance and increase service availability. OS-based virtualization and DOS technologies lead authors to think about their convergence to achieve these goals, extending the boundaries of the virtualized abstraction to multiple hosts and thereby running multiple isolated instances of DOSs sharing the same cluster.

A distributed OS-based virtualization approach will explore aggregation with partitioning. In such systems, a set of server processes constitutes a distributed OS running within an execution context which authors call “D i s t r i b u t e d C o n t a in e r " (DC).

Processes belonging to a DC may be scattered on several nodes of a cluster (aggregation); and processes of different DCs could share the same host (partitioning).

The development of such distributed systems would be facilitated with an IPC infrastructure that provides uniform semantics without considering process location. But, as it is also a virtualization system, this IPC infrastructure should support confinement to isolate communications among processes of different DCs. In addition to these features, other characteristics are required which are related to generic distributed systems. They refer to keeping IPC operational even during process migration and supporting fault-tolerant applications which use the P r i m a r y - B a c k u p

approach [6]. Therefore, there is also a need to redirect communications addressed to the failed P r i m a r y towards one of its B a c k u p processes that becomes the new primary. All these features should be complemented by a suitable performance to make Distributed Virtualization a feasible approach.

This paper proposes M3-IPC, a general purpose IPC software layer which is the foundation of a " D i s t r i b u t e d V ir tu a liz a tio n S y s te m " (DVS) [7], a new model of OS-

level virtualization for Cloud Computing (IaaS). Distributed Virtualization must not be confused with Clustered Virtualization [8, 9] in which an application could run in a distributed way across a group of Containers located on several nodes of a cluster. On such systems, the boundaries of each container are limited by the node where they run, and applications must be developed using special middleware to extend APIs, which avoid the direct migration of legacy applications.

M3-IPC was developed to communicate distributed components of a VOS running within a DC, but it can be used as a powerful infrastructure for designing generic distributed applications. Although the IPC mechanism could be embedded within the DVS, the authors considered useful to build it as an independent software module. Before M3-IPC design, several IPC mechanisms were evaluated, but none of them meet the aforementioned requirements.

The next sections present some works related to IPC mechanisms, followed by a sketch of DVS components, M3-IPC concepts, design goals and implementation issues. Then, results of performance tests are discussed in Evaluation Section. Finally, Conclusions and Future Works Sections summarize the main features of the resulting software, including future improvements.

2 Motivation and Related Works

M3-IPC focuses on providing an IPC software layer to enable N-to-N communications between Client and Servers without any intermediary broker. It should facilitate the development of complex distributed systems such as an OS-level DVS. A great deal of effort had been already taken in providing IPC mechanisms for distributed systems, some of them being integrated as a component within classical DOS [10, 11] and others as added software through libraries, patches, modules, etc. [12, 13, 14, 15, 16]. Their features, semantics and performance, were previously evaluated and during M3-IPC design, implementation, and testing stages. For space limitation reasons, only a representative set of these suitable IPC software is considered here:

• Synchronous Inter-process Messaging Project for Linux (SIMPL) [12]. • Send/Receive/Reply (SSR) [13].

• Remote Procedure Call (RPC) [15]. • Telecommunications IPC (TIPC) [16].

All of the above presented IPC mechanisms were used for performance comparison against M3-IPC as it is shown in Evaluation Section. URPC [17] was not considered because, although it may reach a good performance, it is weak on issues related to security. Messages Queues, Pipes, FIFOs, and Unix Sockets do not have the ability to communicate with remote processes. DIPC presents interesting features, but it is no longer maintained.

One of the most important features needed to build a DVS is IPC isolation, which includes the following properties:

1. C o n fin e m e n t: A process running within a DC cannot communicate with any process

within another DC.

2. P r iv a te A d d r e s s in g : An endpoint number (an M3-IPC address which identifies a

process or thread) allocated to a process running within a DC could be allocated to other processes running within other DCs.

3. V ir tu a liz a tio n T r a n s p a r e n c y : A process does not need to know that it is a member of a DC or the node in which is running, or in which nodes other processes of the same DC are running. To allow DVS management, privileged processes could allocate endpoints and DCs for other non-related processes.

None of the evaluated IPC software meet all of these requirements considering that a DC could span several nodes of a cluster.

3 M3-IPC Design

This section describes the design outline and rationale for M3-IPC to provide local and remote IPC for centralized and distributed systems. It presents the DVS topology model for which it was originally designed, its design goals, main concepts and implementation issues.

3.1 DVS Components

A DVS consists of the following components (Fig. 1):

• D V S : It is the top level layer that assembles all cluster nodes and it embraces all DCs. • N o d e : It is a computer that belongs to the DVS where processes of several DCs are

able to be run. All nodes are connected by a network infrastructure.

• D C : It is the group or set of related processes that might be scattered on several nodes. M3-IPC only allows communications among processes that belong to the same DC. The boundary of each DC can be based on administrative boundaries. A DC hides its internals from the outside world and hides network communication issues from its processes.

• E n d p o in t: An endpoint is an integer number which identifies a process/thread registered in a DC. Endpoints are unique and global within a DC, but could be repeated within other DCs.

M3-IPC does not impose a network/transport protocol to be used for inter-node communications. It allows programmers to choose the protocol that best fit their needs.

Nodes communicate among them through p r o x i e s .

NODE1 N O D E 2 N O D E 3 D C l DC2 DC3 DC4 NODE1 NODE2 NODE3

DVS

3.2 M3-IPC Design Goals

The following design goals were established:

• E a s y - to - u s e : Defining, implementing, and deploying applications using M3-IPC

should be straightforward.

• L o c a t i o n T r a n s p a r e n c y: The application should not need to consider the location of the involved processes in order to facilitate programming.

• C o n fin e m e n t: IPC must be confined so as to achieve a proper degree of isolation. • C o n c u r r e n c y : It should support threads to maximize throughput and efficiency. • T r a n s p a r e n t L i v e M i g r a t i o n S u p p o r t: The application would use process migration

[18] as a mechanism to achieve workload balancing and downtime avoidance. • T r a n s p a r e n t F a u l t T o le r a n c e S u p p o r t: IPC must remain operational even during

process switching from Primary to Backup [6] without the awareness of the other peer processes.

• P e r f o r m a n c e a n d S c a la b ility : Performance goals are considered satisfied when meeting minimal thresholds established for local and remote transfers. IPC throughput between processes located on different nodes must be limited by the network capability (latency and bandwidth) and the transport protocol being used. • C o m m u n ic a tio n P r o t o c o l A g n o s tic : Developers can choose the Network/Transport

• M o d u l a r a n d C u s to m iz a b le : It should be able to be used by any kind of application

with similar requirements.

• C l ie n t /S e r v e r a n d R P C O r ie n te d : M3-IPC must cover typical communication

patterns between multiple clients against multiple servers.

• S e c u r ity : Basic security principles must be applied to its design and implementation.

M3-IPC is based on the assumption of a cluster of network nodes that are connected to a switched LAN where the packet loss rate is low and the available bandwidth is high.

3.3 M3-IPC Main Concepts

M3-IPC was designed to emulate the semantics of a multi-server OS within the Linux kernel providing user-space APIs. Furthermore, it expands its use to processes located on remote nodes of a cluster, and includes features that could be required by distributed applications such as application isolation, location transparency, and message redirection on process migration or process switching on replication.

As M3-IPC supports threads to maximize concurrency, the following references to processes are also valid for threads.

M3-IPC APIs are classified as:

• C o m m u n ic a tio n A P I s : Those APIs emulate Minix 3 [19] IPC primitives. They are related to message and data transfers among processes. They do not include any reference to the DC a process belongs to, or to the node where it resides.

• M a n a g e m e n t A P I s : Those related to DVS, DCs, proxies, nodes, and process management, which allow the mapping of applications to nodes and DCs.

A process which will use M3-IPC must be previously registered at the kernel to bind a DC to it. The process can register by itself, or by another local process with management privileges. Every process bound to a DC will have a unique e n d p o i n t

number with backup endpoints as exceptions.

Management APIs consider: L o c a l e n d p o i n ts are those allocated to processes

running on the same node; and R e m o t e e n d p o i n ts are those allocated to processes running on other nodes.

Every remote server should be first registered in the client’s node specifying in which node is running. Once a remote endpoint is registered by the kernel, all local registered processes (including other clients) within the same DC are able to refer to that server endpoint in communication APIs. Remote client binding is easier because M3-IPC can automatically bind remote unprivileged endpoints when their messages arrive to the local node.

M3-IPC supports message redirection on node switching when a Primary process (on node A) is replaced by a Backup process (on node B). The local M3-IPC kernel

automatically unlinks the endpoint from the previous P r im a r y process (on node A) and

links it to the new P r im a r y process (on node B). A local process can be registered as a

B a c k u p of a remote P r im a r y process with the same endpoint in the same DC, but messages sent by other local processes addressed to that endpoint will be sent to the

P r im a r y process. The B a c k u p process will not be able to communicate with any other process until it is promoted as the new P r i m a r y by a privileged process. Message

replication among P r i m a r y and B a c k u p processes does not concern M3-IPC.

M3 -IPC can keep communications in operational state and can automatically redirect messages on live process migration. Before a process begins to migrate, a management application must call m n x _ m ig r _ s ta r t( ) . At that moment, all messages addressed to the

migrating endpoint will be queued up. Once the process has successfully migrated, the management application must call m n x _ m ig r _ c o m m it( ) . All queued messages will be sent to the process on its new location. If the migration fails, the management application must call m n x _ m ig r _ r o llb a c k ( ) ; then the process can resume its execution as if nothing

had happened by receiving the queued messages.

3.4 M3-IPC Implementation Issues

This section describes the implementation details of the M3-IPC core within the Linux kernel.

M3-IPC APIs use a software interrupt vector or interrupt gate that differs from that used by Linux system calls. This issue allows both APIs to share the same kernel with minimal interference or crosstalk. It also facilitates future maintenance of M3-IPC in current and future Linux kernels.

Two important decisions made before developing M3-IPC were: its construction being based on software components that are available in the Linux kernel, and taking advantage of the parallelism in SMP and multi-core systems. Before starting the development, several Linux kernel synchronization mechanisms and mutual exclusion facilities were evaluated. K e r n e l s e m a p h o r e s and R C U (Read-Copy-Update) were too slow; and s p in l o c k s were somewhat slower than Read/Write locks ( r w lo c k s ) . Finally, m u te x e s were used for mutual exclusion, but since they have a similar performance to

that of r w lo c k s . A compile option is available for system programmers to choose which

synchronization mechanisms and mutual exclusion facilities want to be used.

Another design decision was related to the mechanism for data transfer from user­ space to kernel-space and vice versa. N e t l i n k s o c k e ts [20] have a quite complex start-up

which do not meet DVS project needs. E f f i c i e n t C a p a b i lity - B a s e d M e s s a g i n g (ECBM)

[21] has an impressive performance, but basic security issues were neglected. Finally, functions c o p y _ to _ u s e r ( ) and c o p y _ fr o m _ u s e r ( ) provided by the Linux kernel were used. A custom function named c o p y _ u s r 2 u s r ( ) was built to copy data from the user-space buffer of a source process to the user-space buffer of a destination process.

The most outstanding characteristics of how M3-IPC was finally implemented are summarized in the following list:

• APIs support threads and were implemented using Linux kernel provided mutual exclusion facilities; Task Queues and Event Waiting were used for process synchronization; and Reference Counters were used to hold a count of processes from which a data structure is referenced.

• The granularity of internal critical sections was maximized at the process/thread level, allowing parallel message transfers among multiple pairs of processes. • Higher performance is achieved because DCs do not share any data structure among

one another during concurrent message transfers between pairs of processes that belong to different DCs.

• Data structures that are frequently used for registered processes ( s t r u c t p r o c ) are

aligned with L1 cache lines to reduce access time.

• An Affinity Inheritance Protocol (similar to the well-known Priority Inheritance Protocol) was implemented to minimize performance impact of cache ping-pong. Thus, the facility provided on Linux to specify process affinity with a set of processors/cores was used. With this approach, there is a greater chance for the

message to remain in L1 cache when the destination process is scheduled, thus reducing access time.

• A CPU mask could be allocated by each DC to specify on which CPUs its local processes are able to run (only meaningful within the each node).

• The copy of data blocks and message transfers among co-located processes are made from the source process address space to the destination process address space without any intermediate copy through the kernel. The copy is made by the Linux kernel through the copy-on-write mechanism.

• Debugging information is sent to the Linux kernel ring buffer and can be shown by

means of d m e s g command.

• Data blocks which are page-aligned and whose lengths are greater than or equal to the page size are copied using the kernel provided p a g e _ c o p y ( ) function, which is very efficient because it uses MMX instructions.

• Information about configuration, status, and statistics of M3-IPC abstractions is presented as directories and files within Linux /p r o c filesystem.

M3-IPC has been implemented in C programming language on Linux for Intel x86 32-bit and it is distributed as a kernel patch, a kernel module, and a set of libraries.

Proxies and IPC through the Network

M3-IPC uses application level proxies for communications between nodes, but also support kernel-level proxies too. Therefore, there is no restriction about the network/transport protocol proxies can use.

M3-IPC APIs provides a function to get messages from the kernel that need to be sent to remote nodes, and another function to insert messages into the local kernel coming from remote nodes to local processes as destinations.

The current M3-IPC distribution provides the use of one pair of proxies (sender­ receiver) for each remote node. Proxies exchange p r o x y m e s s a g e s which consist of fixed

length headers and, eventually, variable length payloads (data blocks).

The provided proxies send/receive messages and data through the network without taking account to which DC those messages or data belong to. Custom proxies may consider implementing encryption, compression, filtering, message logging, QoS, etc.

4 Evaluation

This section is devoted to M3-IPC performance evaluation against other IPC mechanisms. The compliance of the other design goals was verified during and after the development stage using several testing scenarios.

Two types of communication tests are presented in Fig. 2: 1) between co-located processes; 2) between processes located on different nodes. Furthermore, two types of micro-benchmarks were performed on each one: a) message transfer; b) data copy. The following common communication pattern was used: a server waits to receive a message from a client, and immediately replies. Once the client sends the request to the server, it waits for the reply. The reply may be a message (36 bytes) or a block of data (36 bytes to 64 Kbytes).

Although several performance metrics were evaluated as latency, CPU usage, network usage, by space limitations, only the throughput results are presented here.

The tests were made on a cluster of 8 (eight) quad-core Intel(R) i5 CPU [email protected] with a memory bandwidth of 20,841 Gbytes/s for 64 Kbytes blocks

(reported by b m m e m ) , linked by a 1 Gbps dedicated LAN switch.

TCP) Block Size |Bytes|

(C) Remote message tra n sfe r throughput (D) Remote data transfer throughput

Fig. 2 : R esu lts o f p e rfo rm a n ce tests.

4.1 Tests between Co-located Processes

Tests between co-located processes allow the comparison of M3-IPC performance versus other IPC mechanisms available on Linux.

One of the design goals states that the expected performance should be as good as the fastest IPC mechanisms available on Linux. The following IPC mechanisms were tested using custom and [22, 23] provided micro-benchmarks: M e s s a g e Q u e u e s , R P C , T IP C , F IF O s , p ip e s , U n ix S o c k e ts , T C P S o c k e ts , S R R , S I M P L .

4.1.1 Message Transfer Performance

The presented results (Fig. 2-A) summarize the best throughput achieved by the IPC mechanisms running a single pair of client/server processes. Linux IPC mechanisms with the highest performance were pipes and named pipes (or FIFOs) followed by M3- IPC (925,314 [msg/s]).

Another micro-benchmark of message transfers between multiple pairs of Client/Server processes was run to evaluate performance in concurrency. The highest average throughput was 1,753,206 [msg/s], which was reached with 4 pairs of Client/Server processes (4 cores).

4.1.2 Data Copy Performance

As it is shown in Fig. 2-B, M3 -IPC performance surpasses other IPC mechanisms on Linux. The reasons of this behavior are: 1) M3-IPC performs a single copy of data between address spaces while the others perform at least two copies (Source to Kernel, Kernel to Destination); 2) it requires a lower number of context switches; 3) it uses the Linux kernel provided p a g e _ c o p y ( ) function which uses MMX instructions.

4.2 Tests between Processes Located on Different Nodes.

This section presents performance results of M3-IPC against RPC and TIPC. M3-IPC does not consider flow control, error control, or congestion control. Those issues are delegated to proxies and the protocol that the use. Reference implementations of M3-IPC proxies use TCP and TIPC as transport protocols.

4.2.1 Message Transfer Performance

As it can be seen in Fig. 2-C, TIPC has the highest throughput. M3-IPC using TCP on proxies has throughput similar to RPC.

The remarkable performance of TIPC suggested that it could be a good option to be used by M3-IPC proxies as transport protocol. M3-IPC versatility and flexibility in proxy programming allowed authors to modify the source code of proxies in a few minutes so as to use TIPC instead of TCP. These changes result in an improvement of performance, emphasizing the impact that the transport protocol has on its throughput.

4.2.2 Data Copy Performance

As shown in Fig. 2-D, TIPC presents the highest throughput of 81[MB/s] that confirms results presented in [16]. The highest throughput achieved by RPC and M3- IPC was about 60[MB/s]. Fig.2-D also shows that there is no noticeable difference in performance when using TIPC instead of TCP as transport protocol on M3-IPC proxies to copy data blocks.

5 Conclusions and Future Works

IaaS providers always need higher levels of performance, scalability and availability for their virtualization services. These requirements can be met by a distributed virtualization technology. As a proof of concept, a DVS prototype using M3-IPC was developed.

M3-IPC addresses some issues about thread support, location transparency, message redirection on process migration, network-transport protocol agnostic, and IPC confinement for virtualization. The results show that M3-IPC achieves all its performance related design goals, with a high throughput for both intra-node and inter­ node messages and data transfers.

To improve M3-IPC security and isolation issues, its integration to Linux Capabilities and c g r o u p s [24] are being considered for further research and future works.

Until the development of M3-IPC, to the best of authors’ knowledge, there was not a modular IPC software for Linux that fully satisfied the particular communication needs

of a DVS. This suggests that many questions remain about software for programming communications related to virtualization.

References

1. D . H a ll, D . Scherrer, J. Sventek: A V irtu al O p era tin g System . Journal C o m m u n ic a tio n o f the A C M , 19 80

2. D a v id R . C h eriton ; W illy Z w a e n e p o e l: T h e D istrib u ted V K e rn e l an d its P e rfo rm a n ce fo r D isk le ss W ork sta tion s. P ro c e e d in g s o f the 9th A C M S y m p o s iu m o n O p era tin g S y stem s P rin cip le s (1 9 8 3 )

3. M o r in , C ., at al.: K e r r ig h e d and data parallelism : clu ster c o m p u tin g o n sin g le sy stem im a g e op era tin g system s. IE E E C o m p u te r S o c ie ty (2 0 0 4 )

4. B arak A .; L a'ad a n O ; S h ilo h A .: S ca la b le C lu ster C o m p u tin g w ith M O S I X fo r L in u x . P roc. 5th A n n u a l L in u x E x p o R a le ig h (1 9 9 9 )

5. P fister, G r e g o ry F.: In S earch o f C lusters. P ren tice H a ll 1 9 9 8 , IS B N 0 -1 3 -8 9 9 7 0 9 -8 .

6. N a v in B u d h iraja, K e ith M a r z u llo , F red B . S ch n eid er, and S am T o u e g : T h e p rim a ry -b a ck u p a p p roa ch . In D istrib u ted sy stem s (2 n d E d .), S ap e M u lle n d e r (E d .). A C M P r e s s /A d d is o n - W e s le y P u b lish in g C o ., N e w Y o r k , N Y , U S A 1 9 9 -2 1 6 , 1993

7. P a b lo P esso la n i, T o n i C ortes, S ilv io G on n et, F ern an do G. T inetti: S istem a d e V irtu a liza ció n c o n R e c u r s o s D istrib u id os. (S p a n is h ) W I C C 2 0 1 2 . A rg en tin a , 2 0 1 2

8. J. T urnbull: T h e D o c k e r B o o k . IS B N 9 7 8 -0 -9 8 8 8 2 0 2 -0 -3 . 2 0 1 4

9. H in d m an , B e n ja m in , et a l.: M e s o s : A P la tfo r m f o r F in e -G r a in e d R e s o u r c e Sharing in the D ata Center. In: N S D I. 11: 2 2 -2 2 . R e tr ie v e d 12 January 2 0 1 5 .

10. A n d r e w S. T an en bau m , G r e g o ry J. Sharp an d D e B o e le la a n A : T h e A m o e b a D istrib u ted O p era tin g System . 19 92

11 . Barrera, J.: A F ast M a c h N e t w o r k IP C Im plem en tation . In P ro c e e d in g s o f th e U S E N IX M a c h S y m p o siu m , N o v e m b e r 1991.

1 2 . Joh n C o llin s and R o b e r t F in d lay: P rog ra m m in g the S IM P L W a y . I S B N 0 5 5 7 0 1 2 7 0 8 , 2 0 0 8 . 1 3 .S R R - Q N X A P I co m p a tib le m e ssa g e p a ssin g fo r L in u x.

h t t p ://w w w .o p c d a t a h u b .c o m /D o c s /b o o k s r .h tm l

14 . M . Sharifi, K . K arim i: D IP C : A H e t e r o g e n e o u s D istrib u ted P rog ra m m in g System . In P ro c e e d in g s o f the 3rd A n n u a l C o m p u te r C o n fe r e n c e o f the C o m p u te r S o c ie ty o f Iran, 19 97 15. A n d r e w D . B irre ll and B r u c e Jay N e ls o n : Im p lem en tin g r em ote p r o c e d u r e calls. A C M

T ra n sa ction s o n C o m p u te r S y stem s, 19 84

1 6 . Jon P au l M a lo y : T IP C : P r o v id in g C o m m u n ic a tio n fo r L in u x C lusters. P r o c e e d in g s o f the L in u x S y m p o s iu m , 2 0 0 4

17 . B ria n N . B ersh a d , T h o m a s E. A n d e r s o n , E d w a rd D . L a z o w s k a , H en ry M . L e v y : U s e r -L e v e l In terp rocess C o m m u n ic a tio n fo r S ha red M e m o r y M u lt ip r o c e sso r s. 1991

18 . D e ja n S. M ilo ji c i c , F red D o u g lis , Y v e s P a in d a v ein e, R ic h a r d W h e e le r , and S on g n ia n Z h ou . “ P r o c e s s m ig ra tio n ” . A C M C o m p u te r S urv ey 3 2 , 3, S ep tem b er 2 0 0 0

1 9 . T an en bau m A ., W o o d h u ll A .: O p eratin g S y stem s D e s ig n an d Im plem en tation . T h ird E d ition P ren tice-H a ll. 2 0 0 6

2 0 . R y a n S lom in sk i: F ast U se r /K e r n e l D ata T ransfer; M a ster T h esis. A p r il 2 0 . 2 0 0 7 2 1 . L u c a V era ld i: E ffic ie n t C a p a b ility -B a se d M e s s a g in g (E C B M ). 2 0 0 3 (Italian )

2 2 . M ic h a e l K errisk: T h e L in u x P rog ra m m in g In terfa ce; N o S tarch P ress, IS B N 9 7 8 -1 - 5 9 3 2 7 ­ 2 2 0 -3 , 2 0 1 0

2 3 .ip c -b e n c h : h ttp ://w w w .c l. cam . ac .u k /re s e a r ch /s r g /n e to s /ip c-b e n c h /.