4.3 Task execution time variation (ETV) as a performance metric
4.3.1 Combination of VMs
Several VMs may be consolidated on a server at a time. The objective of this section is to investigate whether the location of the VMs on the server influences the performance or not. The problem is explained in an example next.
Assume one server has the physical resources to run 16 VMs and currently only 5 VMs are running. The 5 VMs may be placed in any location on the server. Placement of the VMs can change for several reasons. It is possible that some of the VMs have finished their tasks leaving some vacant places in-between the running VMs. Another scenario would be a VM is added to the server and it is placed in a random location. In this way a server may end up with the same number of VMs; however, they are located in different places. In either case, the server has the same number of VMs; however, the combination is different.
If all combinations of the five VMS are to be considered, then there would be 4368 combinations in total. Because there are
C
516 or 4368 ways to select five locations for VMs from sixteen possible available locations on the server. Now the question is, do all of those combinations have the same effect on the server performance or not? Profiling VMs with a series of micro-benchmarks is already a time-consuming4.3. TASK EXECUTION TIME VARIATION (ETV) AS A PERFOR ... 116
process [271]. Furthermore, profiling for all of the combinations of VMs would require even more time.
The experiments in this section aim to show that different combination of the same number of VMs has the same overall effect on the consolidation. In this section, ex- periments are conducted with different combinations of VMs, and they show almost similar results. Since all of the VM combinations show almost similar effect; profiling the performance of all combinations of VM is not necessary. Thus, much VM profiling time can be saved. Next, the experimental setup for this section is described.
A consolidated server can have many different combinations of co-located VMs, the number of consolidated VMs can change at any time. Figure 4.2 depicts a con- solidated hypervisor with ten different VM sets. Each row independently represents a set of VM combination at a particular time. Thus, ten rows represent ten different combinations or sets.
In each set, on one VM the Filebench [229] is executed and the completion time is recorded. Each line of Figure 4.2 represents a different combination of VMs. The top line shows that one VM is running the Filebench (shown in yellow), while other VMs are idle (shown in white). The Filebench is run eight times in eight randomly selected locations and the arithmetic mean of all the execution times is recorded. That is the
R.8 R.7 R.6 R.5 R.4 R.3 R.2 R.1 Set.1 Hypervisor
CPU intensive VM I/O intensive VM Filebench
Set.0
4.3. TASK EXECUTION TIME VARIATION (ETV) AS A PERFOR ... 117
setting of the first line is repeated eight times and each time a random VM was chosen to run the Filebench.
Next, the second line depicts that two CPU (shown in red) and two I/O-intensive (shown in green) VMs are added to the system. In this case, one VM runs the Filebench as before, and another four co-located VMs are also run simultaneously. The rest of the eleven VMs do not run any task (shown in white). Eight different random combinations of those five VMs are run and the arithmetic mean of the execution time of Filebench is calculated. The process of running those random combinations are further explained next.
In the second row, there are sixteen VMs in total, and only five are busy. Those five VMs are all placed at the left end of the row. However, it is possible to place those five VMs anywhere on the server. Like the eight random combinations of the VMs are shown in the next eight lines (
R.1
-R.8
) of Figure 4.2. Although they are in random locations, each line contains exactly five VMs.The experimental results of this section show that all random combinations of the same number of VMs suffer similar performance degradation. In other words, the results here show that the VM performance variation does not depend on locations of the VMs on the server rather it depends on the total number of VMs and their types. For example, the execution times of the eight random rows (
R.1
-R.8
) of Figure 4.2 are 20.32, 19.5, 18.51, 20.22, 20.23, 19.74, 20.48, and 19.51 minutes, respectively. Their arithmetic mean is 19.985 minute.The results show that the locations of the VMs do not influence the performance. Rather the total number of VMs and their resource intensities that are responsible for the task ETV. Thus, performance interference of the second VMs set (second line) of Figure 4.2 is the same as that of the eight random sets (
R.1
-R.8
).Above result has important significance for the rest of the experiments. It means that for a fixed group of VMs, there is no need to examine their performance interfer- ence for all possible VM combinations on the server. Experimental results show that the performance data collected for a combination is equivalent to all other combina- tions.
In the above experiment, only a set of 5 VMs is used. However, those experiments can be done with any number of VMs. In the next section, experiments are done with different sets of VMs with different number of VMs. Each time the similar results are observed.
4.3. TASK EXECUTION TIME VARIATION (ETV) AS A PERFOR ... 118
In this section experiments are done with a fixed number of VMs, in the next section experiments are done by changing the number of VMs. That is in the next section, during experiments the total number of VMs is also changed. Again, for each of those set of VMs different combinations are used in the experiments.