The idea of virtualization was originally conceived in late 1964 by Creasy and Comeau from IBM as a way to partition the resources of mainframe computers [169]. Initially, and even today, virtualization was simply used to create the abstraction of multiple virtual machines running on a single hardware platform. However, more recently virtualization has been used as a supporting infrastructure for a variety of applications, including security.
We take this approach in this thesis, although we do so cautiously noting that off-the-
shelf virtualization solutions likely do not satisfy the requirements for a robust security architecture. Custom solutions will be required for deployable security solutions in most
cases. However, for the work in this thesis, we recognize that off-the-shelf virtualization can be used to demonstrate the viability of key technologies before investing in expensive, custom virtualization solutions.
In this section, we review previous work in virtualization as it relates to its suitability for supporting security architectures and applications. We start with a brief history of virtualization, then look at how it has been used for isolation purposes, and finally consider the question of virtualization’s suitability for security applications directly.
A Brief History The earliest implementations of virtualized systems were built in the late
1960’s and early 1970’s [169]. These systems were largely motivated by the desire to allow multiple users to work on the same hardware, while allowing each user direct access to the hardware for applications such as operating system debugging and development, running
diagnostic software, or running a different operating system [74]. By 1974, there were at
least ten virtual machine systems either available for use or under development including
the VM/370, CP-40, CP-67, 360/30, and the PDP-10 [73]. With the shift to x86 hardware
– which is harder to virtualize – for both server and desktop machines, virtualization fell out of favor during most of the 1980’s and 1990’s.
By 1998, VMWare had solved many of the challenges associated with virtualizing the x86 architecture. Soon after, they released both desktop and server class virtualization
products that effectively renewed interest in virtualization technologies [172]. In the fol-
lowing years, virtualization became increasing ubiquitous, in part because of open source projects such as Xen [16] and KVM [134]. While many people currently use virtualization products for the same reasons cited in the 1970’s (e.g., running multiple operating sys- tems or improved hardware utilization), a new trend has emerged with people looking at ways to creatively use virtualization as part of a comprehensive rethinking of host-based system architectures. Virtualization is now used to support a wide variety of services in- cluding migration, fault tolerance, debugging, host monitoring, and sand boxing. Whitaker
et al. designed an extensible virtual machine monitor for supporting these types of services [182]. An ongoing debate questions if these applications of virtualization are actually turn- ing virtualized systems into a modernized form of microkernel [79, 82]. Regardless of the answer, this debate illustrates that virtualization is now used to support a wide variety of applications.
Isolation Isolation is one of the key virtualization properties used to motivate its use
for security applications and architectures. Traditionally, virtualization provides complete isolation between virtual machines. In this case, virtual machines have no more or less con- nectivity to each other than to non-virtualized hosts. Madnick and Donovan were the first to recognize the security benefits of this in 1973, suggesting that the isolation between vir- tual machines would provide stronger security than the isolation between processes within a given operating system [110]. Around the same time, Lampson defined a set of rules for confining an arbitrary program [104]. Drawing on the work from Madnick and Donovan, these rules would be much easier to enforce using virtualization instead of a traditional operating system. Similarly, Saltzer and Schroeder discussed the concept of controlled
sharing noting that it is a simple concept that is difficult to implement given the mecha-
nisms required [148], but this can also be simplified using virtualization.
Many researchers have leveraged this isolation property to make stronger statements about the security of their systems. Rushby proposed a separation kernel as a small kernel that provides strong isolation between the various processes that it runs [141]. This sep- aration kernel concept was implemented using virtualization by Kelem and Feiertag [97]. Shockley and Schell proposed used “TCB subsets” to simplify the process of evaluating the security of large systems [158]. These subsets must be isolated from each other, only communicating through well defined channels, which is easier to implement using the vir- tualization abstraction.
virtual machines. One technique to control this sharing, and to enable the types of isolation applications discussed above, is to implement a mandatory access control policy within the hypervisor. Karger discussed the requirements for this approach in 2005 [93]. Around the same time, Sailer et al. implemented mandatory access control for Xen [146]. These mechanisms provide the foundation for controlling interactions between virtual machines, however identifying the techniques and policies required to achieve these goals remains an active research area [140].
Suitability In recent years, some researchers have questioned the suitability of using vir-
tualization to address security problems. For example, Bellovin expressed concern over the potentially vulnerable interfaces required for interaction between virtual machines and the higher administrative burden imposed by virtualization [22]. And Garfinkel and Rosenblum identified several ways in which the rapid provisioning of virtual machines can complicate security administration [70]. All of these concerns are valid, but they are only critical of specific virtualization use cases.
The architecture proposed in this thesis does not correspond to these use cases. In- stead, we use virtualization strictly as a technique to provide controlled isolation between the security-critical software and the user or server operating environment. This use case leverages previous work on isolation in a virtualized environment, while also benefiting
from the application flexibility afforded by virtualization [33]. Using a custom hypervisor
could further improve on this deployment scenario by reducing the size of the trusted com- puting base (TCB) [84]. Finally, the major challenges for implementing a secure hypervisor on the x86 platform [138] are no longer a concern due to the recent virtualization exten- sions added by both Intel and AMD to the base x86 architecture. For these reasons, we believe that virtualization is well suited for supporting the security architecture proposed in this thesis.