much faster compared to previous generations of processors and have special mecha- nisms to serve multiple traps or faults more efficiently. Nowadays, most kernels can perfectly operate without utilizing all four privilege rings. In fact, most modern kernels utilize only two privileged levels, 0 and 3 [85].
Next, Figure 2.2b shows how a type-1 hypervisor utilizes the rings on the server. The type-1 hypervisor is widely used in data centers and the most efficient way to run multiple VMs on a server concurrently. Throughout this dissertation, the type-1 hypervisor is used for conducting experiments. As the figure shows, type-1 hypervisor must be placed in ring-0 as it requires the highest privilege in the system. The host operating system (of the server) is placed in the ring-1; thus, having the second highest privilege. Hypervisor still has higher precedence than that of the host operating system and ultimately controls all the system resources.
The guest operating system (on VM) is placed on the next available ring, and it has lower privilege than both the guest operating system and hypervisor. Finally, applica- tions on VMs are run in level 3, which is the least privileged of all the levels.
As mentioned above, a system does not need to use all four rings to operate, and a non-virtualized system can operate perfectly on two or three privilege levels. How- ever, when virtualization is implemented the guest OS, host OS, and hypervisors are generally placed on the rings as shown in Figure 2.2b.
After the advent of Cloud computing and reincarnation of virtualization technol- ogy, the x86 architecture has been modified to accommodate an extra privilege level [81– 83]. All modern x86 processors (produced within last decade or so) have a new priv- ilege level, which is commonly referred to as “Ring -1”. In hardware-assisted virtu- alization, the hypervisor utilizes the new ring to operate at the most privileged level. The extra ring makes it possible for hardware-assisted hypervisors to run unmodified operating systems on the VMs.
2.5
Categorization of virtualization methods
In an x86 system, virtualization can be implemented at different levels and ways to serve different functional purposes. Various researchers have classified virtualization technology differently; however, there is no consensus among the researchers. In this dissertation, virtualization methods of x86 processors are divided into six categories
2.5. CATEGORIZATION OF VIRTUALIZATION METHODS 22
based on the purpose of their use. In this dissertation, the categorization is made de- pending on at what level the virtualization is applied and what type of service it has to offer to the system.
The virtualization technology is not monolithic rather it is a highly dynamic one. It has a different meaning to different people, and it can be used in a wide range of scenarios. A categorization of virtualization is necessary to know which category is essential for data centers and Cloud.
In this dissertation, virtualization is divided into six categories according to levels and types of services they offer. The six categories are as follows:
1. Emulation,
2. Full virtualization, 3. Para-virtualization,
4. Hardware assisted virtualization, 5. OS-level virtualization, and 6. Application level virtualization.
Table 2.1 compares the important characteristics of all six virtualization categories. The categories of virtualization are briefly discussed below.
A full discussion about the categories of virtualization is out of scope for this dis- sertation. The categories are introduced here to make it clear to users that the virtual- ization technology has many applications and functions. In the context of this disser- tation, it is necessary to know what type of virtualization is essential for data centers, so that experiments can be designed and conducted accordingly.
2.5.1
Emulation
In this case, the hypervisor emulates all the components of the hardware. Every hard- ware component is emulated for the VM, and any OS can be ported to run on the VM without significant modification. Emulation is used for running OS with one system architecture, on a machine with another system architecture.
The instruction set of the guest system can be completely incompatible with that of the host. It is the most costly form of virtualization as it requires to translate all the
2.5. CATEGORIZATION OF VIRTUALIZATION METHODS 23
Table 2.1: Comparison of virtualization categories.
Virtualization category Operation overhead VM dependent on Hypervisor operating level Consolidat- ion of many VMs Examples
Emulation High Nothing Usually
over OS Infeasible
QEMU, Bochs, VirtualPC
Full Medium Host H/D Architecture Usually over OS Feasible VMWare Worksta- tion, Parallels Para Medium to low Host H/D Architecture Below OS Most feasible Xen, Citrix XenServer, VMWare ESX Server Hardware assisted Medium to low Special host H/D support Usually below OS Most feasible Intel VT-x, AMD-V OS-level Usually low Shared host OS libraries Closely connected with OS Only for the same OS version Solaris Containers, BSD Jails, Docker Application level Usually low Shared host application libraries Over OS Only for the same application JVM
guest instructions into native instructions. The hypervisor is responsible for the binary translation of all guest OS instructions to make them compatible with host hardware.
Emulation is mainly utilized for the development of new systems and debugging. With this type of virtualization, an OS can be ported to any machine even with en- tirely different system architecture. QEMU is a popular example [86]. Other examples include Bochs and VirtualPC.
2.5. CATEGORIZATION OF VIRTUALIZATION METHODS 24
2.5.2
Full virtualization
Full virtualization is possible if the architectures of the physical host and virtual ma- chine are the same. It uses just “enough” virtualization so that unmodified OS can run on the VM. In this kind of virtualization, hardware architecture swap is not possible for the OS. The IBM VM family fall into this category. The VM-CMS is a family of virtual machine operating systems developed for IBM mainframe computers [87–98]. They are the ancestors of the modern hypervisors, both sharing many characteristics.
The Conversational Monitor System or CMS is one of the earliest examples of software that acted as a hypervisor. Many of the design principles of modern hyper- visors are invented initially for the VM-CMS family. Two modern examples of full virtualizations are the VMWare Workstation [77, 78] and Parallels.
2.5.3
Para-virtualization
Para-virtualization is by far the most common form of virtualization deployed today. In this case, the hardware simulation is not used for VMs; however, the guest OS inside the VM has to be modified. The guest OS uses hypercalls to interrupt the hypervisor. In this model the guest OS is trapped by the application then the hypervisor is trapped by the OS. The application uses the syscall to interrupt the OS, then, in turn, OS uses the hypercall to interrupt the hypervisor.
The guest OS needs to be modified and can not access the underlying hardware. An application can send a service request to the guest OS through syscall. Instead of servicing the interrupt by itself the modified OS interrupts the hypervisor through the hypercall. In this case, OS must be aware of uses the hypervisor API and the guest OS is modified to accomplish this.
In modern data centers and clusters, this type of virtualization is widely used. Ex- amples include Xen [38, 39, 53], XenServer [99] and VMWare ESX Server [35–37].
2.5.4
Hardware virtualization
Hardware virtualization technology [100] transfers some of the hypervisor responsi- bilities to the hardware. One of the problems with the para-virtualization is that the OS needs to be modified. The modification is only possible for open source OS. Pro- prietary software like the Windows is not open source and this is a problem for the
2.5. CATEGORIZATION OF VIRTUALIZATION METHODS 25
para-virtualization technique. To overcome such problem, the hardware manufactur- ers have introduced additional features to the hardware to assist virtualization. In this case, the hardware is aware of the hypervisor and interrupts from the guest OS are transferred to the hypervisor automatically.
Initially, the x86 family of processors was not designed for virtualization, and it became a significant drawback after the advent of Cloud. The hardware assistance virtualization was introduced to overcome this issue. In this case, the hypervisor gets direct support from the hardware. Both modified and unmodified OS can be run on the hardware-assisted virtualization. Examples include the Intel Hardware-assisted
Virtualization(Intel VT-x) [82] and AMD-V [101].
2.5.5
OS-level virtualization
In this case, the primary focus of the virtualization is in the OS. A single instance of OS is divided into multiple partitions. Each partition acts as a fully functioning OS. This case, only one kernel is active, and it provides services to all the OS partitions. Nonetheless, all the partitions are isolated from each other. Applications running on one partition is unaware of the other partitions.
In this case, guest OS is the same as the host OS, and only appears to be isolated. Any other type or version of OS cannot be run on the guest. Examples include Solaris
Containers[102], BSD Jails, Linux Vserver [103] and Docker [104].
2.5.6
Application level virtualization
This type of virtualization is applied at the application level. Each application gets a full execution environment, and they do not share system variables. Each application controls own separate copy of the components like own registry files and global ob- jects. The application level virtualization allows applications to avoid deadlocks and conflicts over the resources.
Multiple application level VMs can be run on one host OS. Multiple copies of the application and runtime environment can be made and run in parallel. The main advantage is that the virtualization overhead is low. The tasks run on the applica- tion level VM, and the VM communicates with the OS. The Java Virtual Machine (JVM) [105, 106] is an example of application level VM.