Load Balancing Algorithm based Virtual Machine Dynamic Migration Scheme for Datacenter Application with Optical Networks

Load Balancing Algorithm based Virtual Machine

Dynamic Migration Scheme for Datacenter

Application with Optical Networks

Xinyu Zhang

1

_{, Yongli Zhao}

2

_{, Xin Su}

1

_{, Ruiying He}

1

_{, Weiwei Wang}

1

_{, Jie Zhang}

2

(1) School of Information and Communications Engineering, (2) State Key Laboratory of Information Photonics and Optical Communication, Beijing University of Posts and Telecommunications, Beijing, 100876, P.R.China, [email protected] Abstract—This paper proposes a load balancing algorithm based

virtual machine dynamic migration scheme for datacenter application with optical networks. The combination of the virtual machine dynamic migration scheme and the load balancing algorithm can help the virtual machines’ migration with a dynamic adjustment on the basis of the current data center’s situation in real-time. And in this way, it can reach the minimum average load among data center’s occupation of service resource. Besides, the experiment results show that the time of migration process is very short, and users’ service level agreement (SLA) can be met well.

Keywords-optical networks; datacenter; virtual machine; dynamic migration; load balacing algorithm

I. INTRODUCTION

With the rapid development of cloud computing, the datacenters are widely deployed in various areas. Connected by optical networks, the datacenters are always distributed unbalanced [1] according to the service requirement of different regions. For the uncontrollable randomness of the requirements from large amount of clients to the servers, more tasks might focus on a small amount of datacenters, which would cause part of the network servers’ paralysis [2]. Upgrading the hardware’s processing power is not a fundamental solution. What’s more, it can improve equipment cost rapidly. Therefore, it is very useful if the service which focuses on some datacenters can be dispatched to some other idle datacenters through migrating virtual machines dynamically [3]. Through these operations, it will develop efficient use of data centers’ resources, improve the quality of users’ service, and prolong the life of each physical host computer. This dynamic scheduling strategy is based on a load balance algorithm [4]. Using load balance algorithm, dynamic dispatching of virtual machine is more reasonable.

Nowadays, there are many research works on the virtual machines dynamic migration and load balancing algorithm. Xen [5] virtual machines developed by University of Cambridge can migrate among a number of physical host computers instead of shutting down the virtual machines. However, the dynamic migration of virtual machines is initiated manually, and only between the same virtual machine monitors [6]. In addition, dynamic migration also requires intensive CPU and network bandwidth resources. Virtual machine’s service level agreement (SLA), in the process of transformation, must suffer great destruction if the process of mobile transformation is controlled unsuccessfully. Besides, the research on the load balancing algorithm has been extended

to various areas, such as in the data center of cloud computing, the algorithm of image processing optimization and so on [7]. However, among the existing dynamic algorithms, the best standard of judging if the node reaches the best balancing state is still not to reach agreement.

The load balancing algorithm based virtual machine’s dynamic migration scheme for datacenter application with optical networks, this paper proposes can satisfy the service demand of the front users and dispatch the resources of datacenter’s virtual machines. It combines load balancing algorithm to dispatch the virtual machine’s resource. After migration of virtual machines, each datacenter’s load is balanced so well that the service on it can be run freely. At the same time, the users from the front stage can only feel their service demands are fulfilled very well and have no idea of the operation at the backstage [8]. In addition, the load balancing algorithm in this paper is based on the analysis for average variance of each node’s load in the cluster. And the algorithm is aiming at making every node’s load in a balanced way. Since variance is a way to measure the degree of each variable’s balance [9], the nodes could reach the best state when the cluster’s variance reaches the minimum.

The rest of this paper is organized as follows. Section II introduces a virtual machine dynamic migration scheme for datacenter applications with optical networks. It describes a whole scheme’s architecture and work process of main modules. Section III proposes a novel load balancing algorithm. Then experiment demonstration and results analysis are given in Section VI. At last, section V is going to conclude the entire paper and propose directions for future research.

II. VIRTUAL MACHINE DYNAMIC MIGRATION SCHEME FOR DATACENTER APPLICATION WITH OPTICAL NETWORKS

A. Virtual Machine Dynamic Migration Framework

A good resource management and migration scheme is a scheme which can meet the front users’ real-time service needs and dispatch the datacenters’ resources rationally and effectively. Besides, the front users are not aware of all of these operations. The virtual machine dynamic migration scheme for datacenter application with optical networks is a scheme just like this. It makes meeting the user’s requirements as its main goal and focuses on datacenter’s resources management and dispatch. At the same time, it combines load balancing algorithm to make the migration more rational and effective. 2012 7th International ICST Conference on Communications and Networking in China (CHINACOM)

The whole scheme’s architecture of virtual machines dynamic migration is shown as in Fig. 1. Some servers are set up on each datacenter, and many virtual machines are contained by each server. There are different kinds of services from the same and different users running on each virtual machine. Different datacenters are linked by optical networks. The virtual machines’ using situation can be learned through collecting their status constantly. When there is a datacenter A running too many services to run anymore, a datacenter B that runs little service is chosen by this scheme. Then the service on it is migrated through dispatching a virtual machine from the datacenter A to datacenter B. While all these operations are based on load balancing algorithm. The combination of the virtual machine dynamic migration scheme and the load balancing algorithm can help the virtual machine’s migration have a dynamic adjustment on the basis of the current data center’s situation in real-time. During the whole process, the virtual machine needn’t to be closed and the front user’s services are not affected.

Figure 1. Virtual Machine Dynamic Migration Architecture

B. Main Modules’ Functions

1) Datacenter and Optical Networks Module

Datacenter, a specific device network, has a strong capacity of data service carrying, high density, flexibility and reliability. Datacenter has already replaced traditional server cluster in present application. And it provides a flexible and reliable service. Different datacenters are connected by optical network. Optical network is a network structure of fiber transmission. Network element of optical network selects path automatically, and establishes the connection by controlling signals. In the process of virtual machine migration, optical network is so fast and efficient that it contributes to migrate the virtual machines successfully. The combination of optical network and datacenter makes virtual machine resources dispatch rapidly from one datacenter to another in the process of virtual machine migration. Meanwhile, the service from front users is served pretty well.

2) Virtule Machine Dynamic Migration Module

Fig. 2 shows virtual machine dynamic migration framework, it contains 5 sub modules: Monitor the migration, Run the migration, Freeze, and Target activation.

Monitor migration module: It decides who should be responsible for starting the migration, especially when and where to migrate. To achieve the goal of load balancing, a monitoring signal at the virtual machine monitor must be set up to monitor the operation of all datacenters. If it decides to migrate the virtual machine A, Monitor migration module is issued to the virtual machine A ‘need to migrate’ signal, and at the same time to communicate with other datacenters on the other modules to determine the migration field goal.

Figure 2. Virtual Machine Dynamic Migration Module

Run migration module: It is responsible for the specific implementation. The operation of the module is the key to the entire migration process. It includes reservation, iterative pre-copy, stop - copy submitted, and activation process.

Freeze module: It is used to guarantee the consistency of source host A and the destination host B, which solves when the sources host A is to be frozen. It is also used to ensure continuity of the original system’s service after freezing. When the memory is copied to a certain extent, it’s needed to terminate the iteration. The freeze module is called to freeze the migration source host, enter the shutdown copy stage, and terminate all activities of the migration source host. After doing these, it can stop the generation of dirty pages in order to maintain the consistency of the source host A and the destination host B.

Target activation module: It is used to determine when to activate the target host, and when to create a new virtual machine. It is also used to ensure the consistency of the target host B and the source host A service after the activation.

The above modules and method ensures that at least one host has a consistent virtual machine images during the dynamic migration process. Until the dynamic migration of the transaction is committed the original host should remain stable, because once the migration has an error, the dynamic migration process will be terminated and the virtual machine will be re-activated in the original host.

III. LOAD BALANCING ALGORITHM

A. Assumption and Description

The algorithm is aiming at making every node’s load in a balanced way after requested distribution. Since variance is a way to measure the degree of each variable’s balance, the nodes could reach the best state when the cluster’s variance reaches the minimum. The average variance of this paper is to keep the variance in a low number after an amount of time of distributing the requests.

There is a cluster, in which each datacenter’s load is A at t_1. If the cluster has no algorithm on it, and at t_1, there is a possibility that one of the datacenters is working overload. Therefore, the purpose to use the algorithm is to make each datacenter achieve a more balanced state as much as possible. And in this paper, average variance load balancing algorithm, based on the analysis of CPU usage, strives to balance the load after redistribution.

Overall flow chart is shown as Fig. 3:

Figure 3. Flow Chart of the Algorithm

B. Load balancing algorithm

Before the algorithm illustration, some issues should be explained at first. The node of the algorithm is the usage of CPU. The purpose of this algorithm is to guarantee the variance of each node reach a low degree. Now, there are n datacenters to form a cluster. At the time

t

₁, the total amount of service requests isG , and the usage of CPU in each datacenter is 1

i

k and the capacity of each CPU isc . To facilitate the _i description of the problem, it can be assumed that all thec _i equals to

c

. And G is: 1

1 1 1 i n i n i i i i i G = c k = c k = = =

∑

× =

∑

× .

( )

1

After a period of time, at the time t , the new service ₂ requests, with amount G , are coming. In this case, the total _a amount of service requests is G : ₂

2 1 1 i n a i a i G G G = c k G = = + =

∑

× + .

( )

2 At this time, the load of CPU:

1i i

g = ×k c.

( )

3 In theory, if the additional overhead of system is not taken into account, the best way to make the variance reach the minimum is to redistribute all service requests on average absolutely to each datacenter, because of the same capacity of each CPU. The average number g is defined as: ₂

2 2 G g n = .

( )

4 In fact, only the new service requests will be distributed to each CPU. And each datacenter’s load is to increase. As a result, it is almost impossible to reach the absolute load balancing. At this point, to make the variance as small as possible, the aim of average variance load balancing algorithm is to make each CPU’s load, it is defined asg_2i, approaches

2

g as close as possible after the new coming task requests been distributed. At this time, if the amount of service requests distributed to Datacenter I is

r

_i, then g2iis:

2i 1i i

g =g +r.

( )

5 After distribution, to make sure that each load approaches to the average number, only the new coming requests can be distributed to the datacenter, whoseg_1iis smaller thang₂. And in this way, the light load CPU has more work to do while the amount of work in overload CPU is not changing.

That is to add the new coming service requests as r_i to the datacenter which can make the following inequality hold：

1i i 2

g = × <k c g .

( )

6 Assume the number of datacenters which hold the inequality (6) is

m

and the r_i is:

i a i

r =G × . δ

( )

7 Among the equation (8), the _{δ is: i}

2 2 1 ( ) i i i m i i g c k g c k δ = = − × = − ×

∑

.

( )

8

The equation (7) is explained as follows: at timeti, there

are

m

datacenters, whose load is smaller thang₂ . And among these datacenters, the difference between Datacenter I

(1≤ ≤i m) and g₂ is g₂− ×c k_i .So, the overall difference

is ₂ 1 ( ) i m i i g c k = = − ×

∑

. And δ is the amount of the proportion of _i the total service request that the Datacenter should deal with.

In general, all c is not the same. When new service _i

requests comes, the average load of each datacenter is different，which depends on the CPU capacity. '

2t g is defined as: ' 2 2 1 i t i n i i c g G c = = = ×

∑

.

( )

9

So, the equation (8) changes into:

' 2 ' 2 1 ( ) t i i i i m t i i i g c k g c k δ = = − × = − ×

∑

.

( )

10

Because of the services’ randomness, different datacenters’ services have different granularities. The varieties of the services’ granularities will contribute to the different loads of the datacenters. The load balancing algorithm in this case is more meaningful. Through this algorithm, the variance of each datacenter load maintains a small value, which realizes the load balancing.

IV. EXPERIMENTAL RESULTS

A. Experimental eniviroment

Figure 4. System’s Testbed

In the experiment, two servers which belong to two different datacenters are employed altogether. The two datacenters are connected by optical networks. Virtual machines states are monitored by VMware vCenter Server. The system’s testbed is shown in Fig. 4. Configurations of the servers are shown as table I.

TABLEI. CONFIGURATIONS

Configuration Server CPU Intel(R) Xeon(R) E5520 @2.27GHz

Operating System System x3650 M2-[7947RDI]-

Memory 20466.5 MB

NIC 1000 Full-duplex

In this part, experiment has been made to test the load balancing algorithm based virtual machine dynamic migration scheme for datacenter application with optical networks. This experiment focuses on the following three factors: migration based on load balancing, heterogeneous migration time, and experiment demonstration of user’s SLA.

B. Migration based on load balancing

When one datacenter’s load is overloaded and the other datacenter’s is idle correspondingly, the migration framework can discover this situation and send out the order of virtual machine’s dynamic migration automatically.

At the beginning of this experiment, datacenter A and B run separate virtual machine a1 and b1.Virtual machine a1 is idle. Virtual machine b1 runs FTP service and responses to FTP request from the outside. During this stage, it isn’t necessary to balance the load although there are some differences between the loads of these two datacenters. At the second stage of experiment, another virtual machine b2 in datacenter B is started and it runs FTP service. And that causes two virtual machine compete bandwidth, which makes datacenter B overloaded. The system finds out this situation through collecting virtual machine’s states. Then it fulfills a migration of virtual machine b2 automatically from datacenter B to datacenter A. After the migration, datacenter A runs two virtual machines a1 and b2.a1 is idle and b1 runs FTP service. Datacenter B’s virtual machine b1 runs FTP service. The two datacenters’ load gradually becomes balanced. The migration situation of datacenter A and datacenter B is shown in table II and table III.

TABLEII. DATACENTER A

Stage Virtual machine 1 Virtual machine 1

Stage 1 Idle Null

Stage 2 Idle Null

Stage 3 Idle FTP service

TABLEIII. DATACENTER B

Stage Virtual machine 1 Virtual machine 2

Stage 1 FTP service Null Stage 2 FTP service FTP service stage 3 FTP service Null

C. Heterogeneous Migration Time

The experiment mainly studies isomerization virtual machine dynamic migration (Xen-KVM, KVM-Xen) downtime and total time competed with isomorphic migration. The experiment employs four different intensity types of load to study the time of the migration process, including idle, kernel-compile, memtest86 and apache ab.

Figure 5. Downtime

Fig. 5 describes different types of virtual machine’s downtime during the dynamic migration under different load conditions. In the experiment, downtime is related to source-side virtual machine monitor, while it doesn’t have influence on destination-side virtual machine. For example, dynamic migration from KVM to Xen virtual machine’s downtime and from KVM to KVM’s are roughly equal, while time from XEN to XEN has a large discrepancy. Generally speaking, heterogeneous migration downtime is longer than isomorphic migration time, both of which are less than 1 second. Fig. 5 also describes the fact that dynamic migration of Xen virtual machine monitor downtime is longer than that from KVM to KVM. This is mainly because Xen takes hundreds of milliseconds to pause the virtual machine and write virtual machine’s state to a file during the dynamic migration process.

2) Migration Total Time

Migration of the total time includes the time of memory pre-copy and downtime. For memory pre-copy, the Xen virtual machine monitor employs the bulk transmission, while KVM transmits one by one. Dynamic migration of virtual machine framework is designed in this paper, and a bulk transfer is also employed. Fig.6 shows heterogeneous migration and constitutive migration is roughly equal in migration time. When pre-copy is modified to bulk transfer, the migration time from the KVM to Xen is even better than the KVM to KVM isomorphic migration. Xen to KVM heterogeneous migration time is better than isomorphism migration from Xen to Xen, because migration from Xen to KVM requires less iteration.

Apart from the test program of memtest86, the time of migration from KVM to KVM migration is better than that from Xen to Xen because memtest86 writes memory very fast and needs more iterations of memory pre-copy. In this situation, Xen bulk pre-copy format shows the advantages. In all cases, the frame’s migration time this paper designs is equal to Xen.

Figure 6. Migration Total Time

D. Experiment demenstration of user’s SLA

An experimental demonstration has been employed to illustrate the user’s SLA in the process of virtual machine’s dynamic migration.

Figure 7. Experimental Demostration

As what has shown in Fig. 7, a user has applied two videos from background service. At first two video services run separately on two datacenter’s virtual machines. However, these two video services have the competition on memory and network resources. The scheme of virtual machines dynamic migration based on load balancing algorithm makes an order of the virtual machine migration. After series of operations, a virtual machine running video service migrates to another datacenter. While during this process, the front user can not feel the operation action, which ensures the user’s SLA, and accomplishes the final goal of the framework designed.

V. SUMMARY

In this paper, the virtual machines dynamic migration scheme is proposed. It is based on load balancing algorithm and faced with datacenter application in optical networks, which to some degree solves the problem of resource mass and hardness of management. The experiment results show that the appearance of this scheme guaranteed the independence,

mobility and flexibility of virtual machine coordination under the load balancing algorithm. Our next step is the study of resource management strategy covering the whole base and the virtual machine switching technology based on many virtual machines dynamic migrations.

ACKNOWLEDGEMENTS

This work was supported in part by 973 program (2010CB328204), NSFC project (60932004), 863 program (2012AA011301), RFDP Project (20090005110013), and the Fundamental Research Funds for the Central Universities (2011RC0406). Research innovation fund for college students of Beijing University of Posts and Telecommunications.

REFERENCES

[1] Niu J, Gu H, Wang C. A new load balanced routing algorithm for Torus networks. Proceedings of the 1st International Symposium on Combinatorics, Algorithms, Probabilistic and Experimental Methodologies (ESCAPE’07), Apr 7−9, 2007, Hangzhou, China. LNCS 4614. Berlin, Germany: Springer-Verlag, 2007: 495−503

[2] Melvin Ver. Dynamic Load Balancing Based On Live Migration Of Virtual Machines: Security Threats and Effects. Thesis report Rochester

Institute of Technology, B. Thomas Golisano College of Computing and Information Sciences (GCCIS), Rochester, NY, U.S.A.

[3] Robert Bradford, Evangelos Kotsovinos, Anjia Feldmann, et a1., “Live wide-area migration of virtual machines including Iocalpersistent state,” In Proceedings of the 3rd international conference on Virtual execution environments, pp. 169-179, San Diego, CA, 2007.

[4] Hung T C, Tanh N H, Thang N D, et al. Advanced routing algorithms and load balancing on MPLS. Proceedings of IEEE 9th International Conference on Advanced Communication Technology (ICACT’07): Vol3, Feb 12−14, 2007, Phoenix Park, Korea. Piscataway, NJ, USA: IEEE, 2007: 1886−1891

[5] Jin Heo Xiaoyun, Zhu, Pradeep Padala, Ann Arbor, Zhikui Wang, “Memory Overbooking and Dynamic Control of Xen Virtual Machines in Consolidated Environments “, 2009.

[6] Jon Oberheide, Evan Cooke, Farnam Jahanian. Empirical Exploitation of live migration of virtual machines. Proc of Black Hat DC, March 24, 2008.

[7] Martin Randles, EnasOdat, David Lamb, Osama Abu- Rahmeh and A. Taleb-Bendiab, ”A Comparative Experiment in Distributed Load Balancing”, 2009 Second International Conference on Developments in eSystems Engineering.

[8] Jon Oberheide, Evan Cooke, Farnam Jahanian. Empirical Exploitation of live migration of virtual machines. Proc of Black Hat DC, March 24, 2008.

[9] Ali M. Alakeel, A Guide to Dynamic Load Balancing in Distributed Computer Systems, IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.6, June 2010.