Virtual Switching Without a Hypervisor for a More Secure Cloud

(1)

Virtual Switching Without a

Hypervisor for a More Secure Cloud

Xin Jin

Princeton University

Joint work with Eric Keller(UPenn)

and Jennifer Rexford(Princeton)

(2)

Public Cloud Infrastructure

• Cloud providers offer computing resources on demand to multiple “tenants”

• Benefits:

– Public (any one can use)

– Economies of scale (lower cost) – Flexibility (pay-as-you-go)

2

(3)

Hardware

Server Virtualization

• Multiple VMs run on the same server

• Benefits

– Efficient use of server resources – Backward compatibility

• Examples

– Xen – KVM

– VMware

VM VM VM

Hypervisor

(4)

Network Virtualization

• Software switches

– Run in the hypervisor or the control VM (Dom0)

• Benefits: Flexible control at the “edge”

– Access control

– Resource and name space isolation

– Efficient communication between co-located VMs

• Examples

– Open vSwitch

– VMware’s vSwitch

– Cisco’s Nexus 1000v Switch

Hardware

VM VM Software

Switch Hypervisor

4

(5)

Security: a major impediment for moving to the cloud!

Let’s take a look at where the

vulnerabilities are…

(6)

Guest VM 2 Guest VM 3

Hardware Hypervisor

Guest VM 1

Vulnerabilities in Server Virtualization

• The hypervisor is quite complex

• Large amount of code ―> Bugs (NIST’s National Vulnerability Database)

6

(7)

Hardware Hypervisor

Guest VM 1

Vulnerabilities in Server Virtualization

• The hypervisor is an attack surface (bugs, vulnerable)

―> Malicious customers attack the hypervisor

(8)

Vulnerabilities in Network Virtualization

• Software switch in control VM (Dom0)

• Hypervisor is involved in communication

Hardware Hypervisor

Dom0 Software

Switch

Physical NIC

Virtual Interface Virtual

Interface

8

(9)

Hardware Hypervisor

Dom0 Software

Switch

Physical NIC

Virtual Interface Virtual

Interface

• Software switch is coupled with the control VM

―> e.g., software switch crash can lead to a complete system crash

Vulnerabilities in Network Virtualization

(10)

Dom0 Disaggregation [e.g., SOSP’11]

• Disaggregate control VM (Dom0) into

smaller, single-purpose and independent components

• Malicious customer can still attack hypervisor!

Hardware

Hypervisor

Guest VM 1 Dom0 Guest VM 2

Service

VM Service

Service VM

VM Service VM

10

(11)

NoHype [ISCA’10, CCS’11]

• Eliminate the hypervisor attack surface

• What if I want to use a software switch?

Hypervisor Dom0 Emulate, Manage

Hardware

Virtualized Physical NIC Physical

Device Driver

Physical Device Driver

• Pre-allocating memory and cores

• Using hardware

virtualized I/O devices

• Hypervisor is only used to boot up and shut down guest VMs.

(12)

Software Switching in NoHype

• Bouncing packets through the physical NIC

• Consumes excessive bandwidth on PCI bus and the physical NIC!

Hypervisor

Hardware

Virtualized Physical NIC Physical

Device Driver

Physical Device

Driver Guest VM 2

Software Switch Dom0

Emulate, Manage

12

(13)

Our Solution Overview

• Eliminate the hypervisor attack surface

• Enable software switching in an efficient way

Hardware

Guest VM 1 DomS Guest VM 2

Virtual Interface

Software Switch

Physical NIC

Virtual Interface Hypervisor

Dom0 Emulate, Manage

(14)

Hypervisor

Eliminate the Hypervisor-Guest Interaction

• Shared memory

– Two FIFO buffers for communication

• Polling only

– Do not use event channel; no hypervisor involvement

FIFO FIFO

Polling

Guest VM 1

Virtual Interface

Dom0 Software

Switch

14

(15)

Limit Damage From a Compromised Switch

• Decouple software switch from Dom0

– Introduce a Switch Domain (DomS)

• Decouple software switch from the hypervisor

– Eliminate the hypervisor attack surface

Hypervisor FIFO

FIFO

Polling

Guest VM 1

Virtual Interface

DomS Software

Switch

Dom0

FIFO FIFO

Polling

Guest VM 1

Virtual Interface

DomS

Software Switch

Dom0

Hypervisor Hypervisor

FIFO FIFO

Polling

Guest VM 1

Virtual Interface

Dom0 Software

Switch

(16)

Preliminary Prototype

• Prototype based on

– Xen 4.1: used to boot up/shut down VMs

– Linux 3.1: kernel module to implement polling/FIFO – Open vSwitch 1.3

16 Hypervisor

Dom0 Software

Switch Guest VM 1

Virtual Interface

Native Xen

Hardware

FIFO FIFO

Polling

Guest VM 1

Virtual Interface

DomS

Software Switch

Our Solution

Hardware

Hypervisor Dom0

(17)

Preliminary Evaluation

• Evaluate the throughput between DomS and a guest VM, compared with native Xen

• Traffic measurement: Netperf

• Configuration: each VM has 1 core and 1GB of RAM

Hypervisor

Dom0 Software

Switch Guest VM 1

Virtual Interface

Native Xen

Hardware

FIFO FIFO

Polling

Guest VM 1

Virtual Interface

DomS

Software Switch

Our Solution

Hardware

Hypervisor Dom0

(18)

Evaluation on Throughput

• FIFO Size

– Polling period is fixed to 1ms – Reach high throughput with

just 256 FIFO pages (Only 1MB)

• Polling Period

– Shorter polling

period, higher throughput – CPU resource consumption?

―> Future work

0 300 600 900 1200 1500 1800

4 8 16 32 64 128 256 512

Throughput (Mbps)

FIFO Pages (1 page = 4 KB)

0 1000 2000 3000 4000

0.050.25 0.5 0.75 1 1.25 1.5 1.75 2

Throughput (Mbps)

Polling Period (ms)

18

(19)

Comparison with Native Xen

• Outperforms native Xen when message size is smaller than 8 KB.

• Future work: incorporate more optimization

0 2000 4000 6000 8000 10000

64 128 256 512 1K 2K 4K 8K 16K

Throughput (Mbps)

Message Size(Bytes) Our Solution Native Xen

(20)