Research and Evaluation of Network Virtualization
in Cloud Computing Environment
Zongjian He
School of Software Engineering Tongji University
Shanghai, China
Email: [email protected]
Guanqing Liang
Department of Computing The Hong Kong Polytechnic UniversityHong Kong, China
Email: [email protected]
Abstract—Network virtualization in cloud computing environ-ment has attracted many attentions recently. With the help of network virtualization, VMM is able to provide fine-grained isolation among different VMs. However, the performance and overhead introduced by network virtualization is also important concerns when applying network virtualization to the cloud computing environment. In this paper, we evaluated the security, performance and QoS of popular virtual switch software Open vSwitch. The results can provide several guidelines for cloud application developer and administrators.
Index Terms—cloud computing, SDN, performance evaluation, open vswitch.
I. INTRODUCTION
Virtualization is an important component in the definition of cloud computing [1]. In cloud environment, resources are virtualized and then can be managed more efficient (a.k.a. on demand or pay as you go). From computing’s perspective, virtualization means that computing tasks or programs are run under the virtual environment rather than physical hardware, namely real environment. With the help of virtualization, a single CPU / RAM / hard disk could be emulated as multiple parallel CPUs / RAM / hard disks and many various operating systems are enabled to run on one platform independently, so as to improve the utility of the computer.
The virtualization of CPU / RAM / hard disks have been studied in many articles. However, a new issue arises after a physical device has been virtualized into multiple virtual devices. How does these virtual devices communicate with each others? Though virtualization provides a lot of conve-nience, and simplifies the system implementation, it lacks the support for network administration among virtual machines. Meanwhile, it is very hard to monitor the communication flow among virtual machines (VM) and manage the virtual network, since the packages will be sent directly from VM to other VM on the same physical host without pass though physical switch. So virtual switch is developed to support the network management and enable monitoring of communication flow among virtual machines.
In this paper, we are going to present the implementation and analysis of software named Open vSwitch [2] [3] which is a typical implementation of this concept. It can act as a virtual software switch in the virtual environment. Open vSwitch
provides different features for the network management such as VLAN, flow control, QoS, etc.. After installing the software, we design the test methodologies and conduct the tests. We’ve designed thee experiments to evaluate different functionality and performance of the Open vSwitch software. The list below summarized our work.
∙ Security Evaluation: We first create two VLANs among four virtual machines and each virtual machine will send ARP broadcast to other VMs in different LAN, and see whether ARP broadcast can transverse different VLAN or not. Additionally, we also conduct the ping test among different virtual machines to see whether VM could successfully connect to other VM in different VLAN. ∙ QoS Evaluation: We utilize the QoS feature of Open
vSwitch to implement the flow control for each network interface. The objective is to show the fine-grained man-agement support for Open vSwitch.
∙ Network Performance Evaluation: We want to know the overhead caused by the software switching. We want to compare the performance with and without virtual switch functions.
The contribution of our work could be summarized as below:
1) We compiled the latest source code of Open vSwitch and upgrade it from 1.0.99 to the latest version 1.4 for Xen Server 6, since Open vSwitch 1.0.99 was built in XenServer 6.0 system by default.
2) We conduct the VLAN tests with two PCs and each PC runs two virtual machines. In each PC, one virtual machine is designated as VLAN1 and other VLAN2. Furthermore, we investigated the implementation of VLAN technology of Open vSwitch.
3) We evaluate the overhead introduced by software switch. The evaluation is focused on VLAN and QoS. It can be used as a reference or guide line for cloud application designers and developers.
4) We conduct the QoS function, especially flow control. We’ve shown that Open vSwitch can limit the maximum bandwidth for each virtualized network interface. The rest of the paper is organized as follows. Section II introduces related works regarding virtual switches in cloud
TABLE I: Environment Configuration
Component Hardware Configuration Software Configuration
Server
Intel T7300 Core 2 Duo 2GB RAM 100GB 7200RPM Citrix XenServer 6.0 Open vSwitch 1.2.2 Virtual Machine Intel Core 2.4GHZ 512M RAM 20GB SCSI HardDisk 100 Ethernet Card Centos 6.0 Or Ubuntu 10.04 Or Windows XP SP3 Remote Host Intel Core 2 4GB RAM 500GB 5400-rpm Xen Center 6.0
computing. Section III introduces the setup of the experimental environment. In section IV, V and VI, security, network per-formance and QoS evaluation are described in detail. Finally, Seciont VII concludes this work.
II. RELATEDWORKS
The concept of introducing virtualization into networks has attracted the attention of both industry and academic bodies. Several works have been done on the virtualization of network devices [4]. [5] uses a higher concept of virtulizing the I/O port to implements network virtualization. In summary, the state of art progress and challenges for network virtualization have been summarized in [6] and [7].
Among these works, Open vSwitch has attracted more attentions than others. Cloud Rack [8] utilizes Open vSwitch to implement live migration among physical machines. CloudPo-lice [9] is a system based on Open vSwitch that implements a hypervisor-based access control mechanism that support com-plicated access control. Meanwhile, the performance issue is another interesting matric for cloud application designer. The network performance of Amzaon EC2 data center is evaluated in [10]. In this paper, we will focus on the performance of Open vSwitch.
III. EXPERIMENTALSETUP
A. Hardware and Software
To evaluate the performance and functionality of network virtualization, Table I describes the main components and configurations of each system.
B. Test Network Topology
Figure 1 depicts the test network infrastructure. We use two Laptop PCs to be two virtualization servers and one PC to act as the cloud control center. On top of the each virtualization server, several virtual machines are created, and designated as VLAN1 and VLAN2 respectively.
The two Xen Servers and the control machine are connected together using a router. All the physical machines and virtual machines are configured to be in the same subnet with IP address 192.168.1.x and network mask 255.255.255.0.
XenCenter VM1: Ubuntu VM 2: Ubuntu Tap 0 Tap 1 OVS eth0 Bro VM 3: Windows VM 4: CentOS 6.0 Tap 0 Tap 1 OVS eth0 Bro IP: 192.168.1.21 IP: 192.168.1.22 IP: 192.168.1.11 IP: 192.168.1.12
IP: 192.168.1.156
VLAN 1 VLAN 2 VLAN 1 VLAN 2
IP: 192.168.1.20 IP: 192.168.1.10
Fig. 1: Test Network Topology TABLE II: XenServer Configuration
XenServer 1 XenServer 2
IP address 192.168.1.20 192.168.1.10
Subnet mask 255.255.255.0 255.255.255.0
Default gateway 192.168.1.1 192.168.1.1
Default time zone Asia/Hong Kong Asia/Hong Kong
C. Installation and Configuration
We downloaded the Xen Server 6.0 iso file and burn it into a USB flash stick and insert the USB flash and boot up the system. We have installed two Xen Servers on two separate laptop computers, and each Xen Server system is configured as Table II.
Since Open vSwtich 1.0.99 is built in Xen server 6.0 system, but it is not the latest version. We download the Open vSwitch source file from Open vSwitch official site, and compile the OVS source file to rmp file on Xen Server 6.0 DDK VM according to the manual provided by Open vSwitch and updated the Open vSwitch on each Xen server to make sure that our work is based on the latest version of the software.
IV. SECURITYEVALUATION
Physical isolation is an important mechanism to improve the security of networks. In physical network environment, if networks can not be isolated into different physical subnets, virtual LANs are used to isolate the communication between machines in the same physical network but different virtual networks. In fact, machines in different virtual subnets could not communicate with each other, and ARP request sent by one machine will not be received by other machine in a different virtual subnet.
In this section, we will conduct the VLAN test to see whether Open vSwitch is able to isolate VM traffic using VLANs. In this test, we set up two PCs for the VLAN test. The table III below shows the VLAN test configuration of each server in details.
A. Ping and ARP test
Firstly, we try the basic ping command used to diagnose the connectivity of networks between two network peers. Then, we use tcpdump command to dump the received ARP package from each machine. Since ping command will first use ARP to find the physical address of the destination machine and the ARP protocol works in a broadcast way, we want to test that the VLAN can block the broadcast that does not belong to the same VLAN.
Table IV shows the result of our test. In the table, Y indicates VMs can communicate with each other and N means can not. The conclusion is obvious from the table that virtual machines allocated into same VLAN can communicate with each other and VMs in different VLAN can not.
In this way, with the help of network virtualization, VMs can be organized using virtual LAN rather than physical ma-chines or physical networks. The granularity of virtualization can be further guaranteed.
B. VLAN implementation analysis
The implementation of VLAN has significant impact on the VMs which use Open vSwitch. As claimed in the documen-tation, open vSwitch supports standard 802.1Q VLAN model. There are two possible implementations of VLAN mechanism: Native VLAN tagging support or Software based. The major difference of the two is whether the Ethernet frames received by the inner VM are tagged.
As stated in IEEE 802.1q standard, 802.1q adds a 32-bit field between the source MAC address and the EtherType fields of the original frame. Two bytes are used for the tag protocol identifier (TPID), the other two bytes for tag control information (TCI). The TCI field is further divided into PCP, CFI, and VID. If Open vSwitch support native vlan tagging, the Ethernet frames received by the inner VMs should be tagged. To verify the detailed implementation, we have designed the following experiment.
Step 1. Environment setup. Install two VMs running Win-dows XP on XEN Server. Install WireShark which is used to capture the physical ethernet framews on the VM. The main reason we use Windows rather than the Linux system used in previous experiment is that WireShark can not be used under Linux. Set their IP addresses to the same subnet.
Step 2. Place the two VMs in the same VLAN with tag=4. Ping each other to make sure that they can be reached. At the beginning, we find that the pings between the two VMs
TABLE III: VLan Test Configuration
System IP Address NIC VLan
Server 1 XenServer 192.168.1.20 Eth0
VM 1 (Ubuntu 10) 192.168.1.21 Tap5.0 Vif5.0 xenbr0 tag=1
VM 2 (Ubuntu 10) 192.168.1.22 Tap6.0 Vif6.0 xenbr0 tag=2
Server 2 XenServer 192.168.1.10 Eth0
VM 3 (Centos 6) 192.168.1.11 Vif5.0 xenbr0 tag=1
VM 4 (Centos 6) 192.168.1.12 Vif6.0 xenbr0 tag=2
TABLE IV: Summary of the VLAN test VM1 VM1 VM3 VM4
VM1 / N Y N
VM2 N / N Y
VM3 Y N / N
VM4 N N Y /
Fig. 2: WireShark Capture without / with native VLAN
are always timeout. Later we find out that this is due to the Windows Firewall, which prevent the machine from replying. After turning off the Windows firewall. The two VMs can get the reply from each other.
Step 3. Using WireShark to capture the ICMP package in-side the VM. The package captured by WireShark is illustrated as:
As can be seen from the upper part of Figure 2, the package type field follows immediately after the source MAC address, there is no VLAN tag in the Ethernet frames. In contrast, we present the lower part of Figure 2, which capture inside a VM with native VLAN support. It is clear that the Ethernet frames are all tagged by IEEE 802.1q tags.
We can conclude that the default implementation of Open vSwitch stripped off the VLAN tag before transferring the package to the VMs. After checking the source code, we’ve found the implementation in strip vlan function located as Action.c. This function is invoked each time Open vSwitch receives a package from XEN kernel driver hook. After invoking this function, the package is passed to the VMs. That’s the reason why we can not capture the IEEE 802.1Q inside the XEN VMs.
V. NETWORKPERFORMANCEEVALUATION
In this section, we would like to investigate the overhead of software switch using Open vSwitch. Intuitively, applying software switch technology can bring additional overhead due to the following reasons:
Router
XEN XEN
NTttcps VMs NTttcpr VMs
Fig. 3: NTttcp architecture
1) VLAN and vBridge operation: the software switch must check the VLAN and vBridge configuration and operate according to the user preferences like dropping package. 2) QoS management. If QoS is applied, the software switch must have some statistic mechanism to count the sent package number for each port, which brings additional overhead.
Obviously there’re other overheads. In our experiment, we will just focus on the two mentioned above.
A. Approach and Methodology
In our experiment, we use Microsoft NTttcp to test the network performance. NTttcp is a multithreaded, asynchronous application that sends and receives data between two or more endpoints and reports the network performance for the duration of the transfer. It works using client / server mode, as illustrated in the following figure. NTttcpr is the server and NTttcps is the client. The client will sent package as fast as possible to the server to test the performance of the network. To verify the overhead of software switch, we’ve set up two configurations to compare with. Configuration 1 has no VLAN and QoS support and configuration 2 has both VLAN and QoS enabled. Notice that for configuration2, we set the QoS rate to 100MB, which is above the maximum bandwidth and thus will not affect the throughput.
For each configuration, we test both TCP stream and UDP diagram for ten times. To simplify the test process, we’ve written a batch file to complete the process automatically. The batch file will execute the NTttcps for ten times, and at each loop, the execution time is increased by 1 second. This is controlled by the for statement. -u means UDP test. By default, it will use TCP to perform the test. -m 1,0 means we use single thread and the thread is attached to processor 0. The reason we use single thread is the limitation of hardware. In our XEN server, we only assign each VM a single CPU, if we use multi-thread here, more overhead will be introduced by context switch of thread. The output will be logged into a text file named a.txt.
B. Test Result
The raw data is difficult to understand. In the paper, we only provide the processed data in the form of plot.
Figure 4 shows the comparison between TCP with or without QoS and VLAN. It is obvious that significant over-head has been introduced due to the QoS and VLAN. The
Fig. 4: TCP Throughput Comparison
Fig. 5: TCP CPU Usage for VMs
Fig. 6: UDP Throughput Comparison
Fig. 7: UDP CPU Usage for VMs
average bandwidth for connections without software switching is 25.789 MB/S. However, if we add flow control and VLAN, the bandwidth dropped significantly to 11.131 MB/S. Only half of the performance is achieved.
is only for information purpose. No much information can be concluded from the CPU overhead inside VM. We can see that for Open vSwitch without flow control and VLAN, the CPU is about 100%. This is caused by the best effort delivery behavior of the CPU without network flow control.
In terms of UDP, Figure 6 and Figure 7shows the UDP throughput with / without QoS and VLAN support. The aver-age bandwidth of UDP without QoS and VLAN is 2.41875M-B/S. However, if we add VLAN and QoS support, the average speed will drop to 1.5 MB/S.
The reason why TCP throughput is higher than UDP throughput is interesting. Intuitively, the performance of using TCP should be lower than UDP due to the overhead of TCP functions such as sequence control, connection maintenance and etc. Currently, we can only guess that since UDP pro-vides connectionless and data gram service. The bandwidth calculation is different from TCP. Since NTttcp is not open source software, we can not analysis from source level. Here is another future work for us.
VI. QOS EVALUATION
In this experiment, we test the QoS function of Open vSwitch. We’ll focus on the flow control function that can limit the bandwidth of specific interface.
The configuration process is shown as follows.
Firstly, we limit the virtual port bandwidth to 1 MByte every second using the following command:
# ovs−v s c t l s e t I n t e r f a c e t a p 5 . 0 i n g r e s s p o l i c i n g r a t e =1024
In addition, we set the burst to 0.1MB. # ovs−v s c t l s e t I n t e r f a c e t a p 5 . 0
i n g r e s s p o l i c i n g b u r s t =100
After the configuration, we use NTttcp again to test the result of flow control. The results are depicted as Figure 8. Considering the accuracy of the NTttcp tools and compared with Figure 4 without QoS control, the result is acceptable.
The QoS function is extremely useful when cloud adminis-trators want to provide different quality of services to different customers.
Fig. 8: QoS evaluation result
VII. CONCLUSION
In this work, we have implemented the evaluation environ-ment and conducted different tests on Open vSwitch including VLAN test, Qos Test and Overhead evaluation. The result shows that it can successfully isolate the traffic between virtual machines which are in different virtual subnets, and block the unnecessary broadcast. The fine-grained QoS setup brings additional convenience to network administrators in cloud platform. Overall, we believe that software switching technology will show more power with the development of cloud computing.
However, our work can still be improved in several aspects. Due to the limitation of our experiment, we can only use laptops to set up XEN server. The laptops have only one NIC to communicate with outside. Some functions of Open vSwitch can not be evaluated such as OpenFlow, sFlow, etc. Moreover, the accuracy of result can be improved if we isolate the network communication and remote control communication.
In the future, we are planning to analysis the inside and outside communication of VM individually. That means we will separate the package send between VMs organized by same open vSwitch and VMs under different machines. In ad-dition, we will compare open vSwitch with other competitive implementations. We believe that more interesting result can be found.
REFERENCES
[1] L. M. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, “A
break in the clouds: towards a cloud definition,” SIGCOMM Comput.
Commun. Rev., vol. 39, no. 1, pp. 50–55, Dec. 2008. [Online]. Available: http://doi.acm.org/10.1145/1496091.1496100
[2] J. Pettit, J. Gross, B. Pfaff, M. Casado, and S. Crosby, “Virtual switching
in an era of advanced edges,” in 2nd Workshop on Data Center–
Converged and Virtual Ethernet Switching (DC-CAVES), 2010. [3] B. Pfaff, J. Pettit, T. Koponen, K. Amidon, M. Casado, and S. Shenker,
“Extending networking into the virtualization layer,” Proc. HotNets
(October 2009), 2009.
[4] J. Lee, J. Tourrilhes, P. Sharma, and S. Banerjee, “No more
middlebox: integrate processing into network,” SIGCOMM Comput.
Commun. Rev., vol. 41, no. 4, pp. –, Aug. 2010. [Online]. Available: http://dl.acm.org/citation.cfm?id=2043164.1851262
[5] Y. Dong, X. Yang, J. Li, G. Liao, K. Tian, and H. Guan, “High
performance network virtualization with sr-iov,” Journal of Parallel
and Distributed Computing, no. 0, pp. –, 2012. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0743731512000329 [6] N. Chowdhury and R. Boutaba, “Network virtualization: state of the
art and research challenges,”Communications Magazine, IEEE, vol. 47,
no. 7, pp. 20 –26, july 2009.
[7] N. M. K. Chowdhury and R. Boutaba, “A survey
of network virtualization,” Computer Networks, vol. 54,
no. 5, pp. 862 – 876, 2010. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1389128609003387 [8] Y. Pu, Y. Deng, and A. Nakao, “Cloud rack: Enhanced virtual topology
migration approach with open vswitch,” in Information Networking
(ICOIN), 2011 International Conference on, jan. 2011, pp. 160 –164. [9] L. Popa, M. Yu, S. Y. Ko, S. Ratnasamy, and I. Stoica, “Cloudpolice:
taking access control out of the network,” in Proceedings of the
Ninth ACM SIGCOMM Workshop on Hot Topics in Networks, ser. Hotnets ’10. New York, NY, USA: ACM, 2010, pp. 7:1–7:6. [Online]. Available: http://doi.acm.org/10.1145/1868447.1868454
[10] G. Wang and T. Ng, “The impact of virtualization on network
perfor-mance of amazon ec2 data center,” inINFOCOM, 2010 Proceedings
