VirtualBox within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 7.0.1 build-227600 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338 L2 guest: Ubuntu 9.04 32bit
The L1 guest booted and ran correctly using dynamic binary translation. The image of the L2 guest was copied to the L1 guest. When starting the L2 guest, the bootloader showed
Boot from ( hd0 , 0 ) e x t 3 <HDD−ID> S t a r t i n g up . . .
and afterwards, VirtualBox aborted the start and showed following message:
1 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 5 0 0 6 7 0 2 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 5 0 0 5 6 0 3 PATM: F a i l e d t o r e f r e s h d i r t y p a t c h a t c 0 1 3 f 3 7 0 . D i s a b l i n g i t . 4 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 5 4 b c 0 5 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 0 9 d 0 0 6 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 2 2 d 5 0 7 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 5 0 0 6 d 0 8 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 0 9 c 8 0 9 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 8 0 a c 0 10 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 5 a 0 4 0 11 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 3 f 9 8 0 12 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 5 4 8 8 0 13 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 5 4 a 0 0 14 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 3 2 3 c 0 15 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 5 1 6 0 0 16 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 7 e d 0 0 17 PATM: F a i l e d t o r e f r e s h d i r t y p a t c h a t c 0 1 3 f 3 7 0 . D i s a b l i n g i t . 18 PATM: patmR3RefreshPatch : s u c c e e d e d t o r e f r e s h p a t c h a t c 0 1 0 5 1 d 0 19 PIT : mode=0 c o u n t=0x10000 ( 6 5 5 3 6 ) − 1 8 . 2 0 Hz ( ch =0) 20 21 ! ! A s s e r t i o n F a i l e d ! ! 22 E x p r e s s i o n : pOrgInstrGC
23 L o c a t i o n : /home/ vbox / vbox − 3 . 1 . 6 / s r c /VBox/VMM/PATM/VMMAll/PATMAll . cpp ( 1 5 9 ) v o i d PATMRawLeave(VM∗ , CPUMCTXCORE∗ , i n t )
B.2. PARAVIRTUALIZATION 79
VMware Workstation within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 7.0.1 build-227600 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 6.5.3 build-185404 L2 guest: Ubuntu 9.04 32bit
The L1 guest booted and ran correctly using dynamic binary translation. The image of the L2 guest was copied to the L1 guest. The L2 guest tried to start but did not show any activity. The screen stayed black and nothing happened, even after several hours.
Xen within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 7.0.1 build-227600 L2 hypervisor: xen-3.0-x86 32p
Domain 0 (L2): openSUSE 11.3 build 0475 L2 guest: openSUSE 11.3 build 0475
The L1 guest booted and ran the Xen hypervisor correctly using dynamic binary translation. The image of the L2 guest was copied to the L1 guest. The paravir- tualized L2 guest booted rather slow but a user could log in and use the nested guest.
B.2
Paravirtualization
None of the setups with paravirtualization as bottom layer work. The configuration of the setups are given but, as explained in section 5.1.2, the problem is that nested hypervisors need modifications.
VirtualBox within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338
VMware Workstation within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 6.5.3 build-185404
Xen within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L2 hypervisor: xen-3.0-x86 32p
B.3. FIRST GENERATION HARDWARE SUPPORT 80
B.3
First generation hardware support
In this section, the results are given of the setups with a L1 hypervisor based on hardware support for x86 virtualization. All tests were conducted on a processor that only has first generation hardware support, namely Intel R CoreTM2 Quad Q9550.
The following subsections combine the setups that use the same nested hypervisor technique.
B.3.1 Dynamic binary translation VirtualBox within VirtualBox
Host operating system: Ubuntu 9.10 64bitor Ubuntu 9.04 32bit L1 hypervisor: VirtualBox 3.1.6 r59338
L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338 L2 guest: Ubuntu 9.04 32bit
The L2 guest hung in this setup. It did not crash, did not show information in the log during several hours, and was probably just very slow.
VMware Workstation within VirtualBox
Host operating system: Ubuntu 9.10 64bit or Ubuntu 9.04 32bit L1 hypervisor: VirtualBox 3.1.6 r59338
L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 6.5.3 build-185404 L2 guest: Ubuntu 9.04 32bit
This setup was rather unstable. One configuration allowed booting the nested guest. This working configuration used the graphical user interface for the L1 and L2 guest, while other setups used a text based environment. The setup was tested on a Ubuntu 9.10 64bit and Ubuntu 9.04 32bit operating system, with a graphi- cal user interface and with a text based environment. The test on the Ubuntu 9.04 32bit operating system with the graphical user interface was the only one that worked.
VirtualBox within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 6.5.3 build-185404 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338 L2 guest: Ubuntu 9.04 32bit
Both L1 and L2 guest were able to boot and run correctly. However, in order to get the L2 guest started, one must append the line
B.3. FIRST GENERATION HARDWARE SUPPORT 81
to the configuration file (.vmx) of the L1 guest. This allowed to run a hypervisor within the L1 guest. Upon starting the L2 guest, the L1 guest displayed the following warning:
The v i r t u a l machine ’ s o p e r a t i n g s y s t e m h a s a t t e m p t e d t o e n a b l e p r o m i s c u o u s mode on a d a p t e r E t h e r n e t 0 . T h i s i s n o t a l l o w e d f o r s e c u r i t y r e a s o n s . P l e a s e go t o t h e Web page h t t p : / / vmware . com/ i n f o ? i d =161 f o r h e l p e n a b l i n g
t h e p r o m i s c u o u s mode i n t h e v i r t u a l machine .
One can get around this message by starting VirtualBox as root instead of a normal user.
VMware Workstation within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 6.5.3 build-185404 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 6.5.3 build-185404 L2 guest: Ubuntu 9.04 32bit
Both L1 and L2 guest were able to boot and run correctly. However, in order to get the L2 guest started, one must append the line
m o n i t o r c o n t r o l . r e s t r i c t b a c k d o o r = ”TRUE”
to the configuration file (.vmx) of the L1 guest. This allowed running a hypervisor within the L1 guest. Upon starting the L2 guest, the L1 guest displayed the following warning:
The v i r t u a l machine ’ s o p e r a t i n g s y s t e m h a s a t t e m p t e d t o e n a b l e p r o m i s c u o u s mode on a d a p t e r E t h e r n e t 0 . T h i s i s n o t a l l o w e d f o r s e c u r i t y r e a s o n s . P l e a s e go t o t h e Web page h t t p : / / vmware . com/ i n f o ? i d =161 f o r h e l p e n a b l i n g
t h e p r o m i s c u o u s mode i n t h e v i r t u a l machine .
One can get around this message by starting the L2 VMware Workstation as root instead of a normal user.
VirtualBox within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338 L2 guest: Ubuntu 9.04 32bit
The L2 hung while booting inside the Xen guest. The L2 guest was probably just very slow since it stayed unresponsive for several hours.
VMware Workstation within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 7.0.1 build-227600 L2 guest: Ubuntu 9.04 32bit
B.3. FIRST GENERATION HARDWARE SUPPORT 82
The L2 guest did not start because VMware Workstation checks whether there is an underlying hypervisor. VMware Workstation noticed that there was a Xen hypervisor running and displayed the following message:
You a r e r u n n i n g VMware W o r k s t a t i o n v i a an i n c o m p a t i b l e h y p e r v i s o r . You may n o t power on a v i r t u a l machine u n t i l t h i s h y p e r v i s o r i s d i s a b l e d .
VirtualBox within KVM
Host operating system: Ubuntu 9.10 64bit L1 hypervisor: kvm-kmod 2.6.32-9
L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VirtualBox 3.1.6 r59338 L2 guest: Ubuntu 9.04 32bit
The L1 guest booted and ran correctly. Upon starting the L2 guest, the following error was shown:
Boot from ( hd0 , 0 ) e x t 3 <HDD−ID> S t a r t i n g up . . .
. . .
[ 3 . 9 8 2 8 8 4 ] K e r n e l p a n i c − n o t s y n c i n g : Attempted t o k i l l i n i t !
VMware Workstation within KVM
Host operating system: Ubuntu 9.10 64bit L1 hypervisor: kvm-kmod 2.6.32-9
L1 guest: Ubuntu 9.04 32bit
L2 hypervisor: VMware Workstation 6.5.3 build-185404 L2 guest: Ubuntu 9.04 32bit
The setup worked for newer versions of KVM. In older versions and in the devel- oper version (kvm-88), the L2 guest hung during start-up. With the newer version, the L2 guest booted and ran successfully. Note that newer versions of VMware Workstation check if there is an underlying hypervisor and in those cases it will refuse to boot a virtual machine. In the newest version, VMware Workstation 7.0.1 build-227600, this setup no longer worked due to the check for an underlying hyper- visor.
B.3.2 Paravirtualization
All four setups could successfully nest a paravirtualized guest inside the L1 guest. However, the setup where Xen is nested inside VirtualBox was not very stable.
Xen within VirtualBox
Host operating system: Ubuntu 9.10 64bitor Ubuntu 9.04 32bit L1 hypervisor: VirtualBox 3.1.6 r59338
L2 hypervisor: xen-3.0-x86 32p
Domain 0 (L2): openSUSE 11.3 build 0475 L2 guest: openSUSE 11.3 build 0475
B.4. SECOND GENERATION HARDWARE SUPPORT 83
Sometimes during the start-up of domain 0 several segmentation faults occurred. Domain 0 was able to boot and run successfully but the creation of another paravir- tualized guest was sometimes impossible. Xen reported that the guest was created, however it did not show up in the list of virtual machines indicating that the guest crashed immediately. The setup worked most of the times on host operating system Ubuntu 9.04 32bit. On the Ubuntu 9.10 64bit operating system, there were always segmentation faults.
Xen within VMware Workstation
Host operating system: Ubuntu 9.10 64bit
L1 hypervisor: VMware Workstation 7.0.1 build-227600 L2 hypervisor: xen-3.0-x86 32p
Domain 0 (L2): openSUSE 11.3 build 0475 L2 guest: openSUSE 11.3 build 0475
Both L1 and L2 guest were able to boot and run correctly.
Xen within Xen
L1 hypervisor: xen-3.0-x86 32p
Domain 0 (L1): openSUSE 11.3 build 0475 L2 hypervisor: xen-3.0-x86 32p
Domain 0 (L2): openSUSE 11.3 build 0475 L2 guest: openSUSE 11.3 build 0475
Both L1 and L2 guest were able to boot and run correctly.
Xen within KVM
Host operating system: Ubuntu 9.10 64bitor Ubuntu 9.04 32bit L1 hypervisor: VirtualBox 3.1.6 r59338
L2 hypervisor: xen-3.0-x86 32p
Domain 0 (L2): openSUSE 11.3 build 0475 L2 guest: openSUSE 11.3 build 0475
Both L1 and L2 guest were able to boot and run correctly.
B.4
Second generation hardware support
This section summarizes the results of the setups with a L1 hypervisor based on
hardware support for x86 virtualization. The tests were conducted on a newer
processor with second generation hardware support, namely Intel R CoreTM i7-860.
Only the setups are given where the outcome differed from section B.3. These setups are the setups that worked using the second generation hardware support and did not work using the first generation. All the other setups had the same output.