the database access is required, and the database access operations consume multiple system resources. The goal of the OLTP test is to put pressure on the MySQL database server, which in turn puts pressure on multiple system resources like, CPU, memory, and disk. The benchmark is very suitable for reproducing the resource usage footprint of most database applications.
Two observations play essential parts in performance benchmark design. The first is that applications usually spend most of the time executing a small segment of code [234]. The second is that the application performance depends on some fac- tors like the system call handling efficiency of a particular machine [222]. Those two factors significantly affect the performance of an application. The benchmarks are designed based on statistical analysis of popular applications.
There is no universal consensus on the two issues; therefore, statistical analysis from different angles have given rise to different benchmark suites. No single bench- mark is sufficient enough to provide the complete picture of the VM performance in- terference. The goal of this work is to analyze the performance of co-located VMs, and different benchmarks suites are used for this purpose. Each benchmark has some unique features and representative of a particular type of application. In the experi- ments, workloads are build by combining those benchmarks.
3.5
Data collection process
The previous section describes the benchmarks used in the experiments in this chapter. This section describes how the data is collected from the VMs during the experiments. Linux distributions are accompanied with some built-in tool to gather data about the running processes, these tools along with some other utilities are used. The resource usage data is collected from all the VMs and physical host.
Figure 3.1 shows the diagram of the experiment setup and data collection process. The middle section of the figure shows that several hosts are running different numbers of consolidated VMs. In a data center, different hosts may concurrently run different numbers of VMs at different times. The experimental setup replicates this situation by running different numbers of VMs on different hosts.
A virtualized host is capable of running several VMs concurrently. Figure 3.1 shows the VMs that are running on the host with solid lines. Empty VM slots on the
3.5. DATA COLLECTION PROCESS 64
Hardware
configure scripts Remote vm Remote data collection scripts
Benchmarks
control scripts Remote benchmarks
Data collection tools
Iostat Diskbench Filebench Iozone Lmbench Nbench Sysbench Unixbench Stream Cachebench Top Store data
Raw data files
Configured to run workload on 1 vm
Configured to run workload on 2 vms
Configured to run workload on 4 vms
Configured to run workload on 8 vms
Configured to run workload on 14 vms vm 0 vm 0 vm 0 vm 0 vm 0 vm 1 vm 2 vm 3 vm 1 vm 2 vm 8 vm 1 vm 2 vm 3 vm 13 vm 1 Host Host Host Host Host
CPU, Mem, I/O data
CPU, Mem, I/O data
CPU, Mem, I/O data
CPU, Mem, I/O data
CPU, Mem, I/O data
Figure 3.1: Data collection process from the co-located VMs.
server are shown with dashed lines; the empty slots are shown to indicate that the host is capable of consolidating more VMs.
The right side of Figure 3.1 shows the scripts are used to control the benchmarks and VMs. During the stages of experiments, several benchmarks need to be run. Also, the number of concurrently running VMs need to change at each stage. The shell scripts on the right are used to control the benchmarks and VMs. The scripts them- selves run on a separate machine and connects to the host through SSH connection.
On the left side of Figure 3.1 several other scripts are running to collect resource usages data from the hosts. The scripts run tools to collect information about CPU, memory, and I/O usages. These scripts are also run on a separate machine and store all data on the remote machine. Data from different tools have the different format, and they are stored in files for later analysis.
Next, Figure 3.2 shows how the usage data collected from the physical host and each VM on the host. As mentioned before, that three resources usages are monitored, they are CPU, memory, and disk I/O.
The Linux top command is used for collecting data of CPU and memory usage data. For the CPU various data are gathered from all VMs and host. Those include the
3.5. DATA COLLECTION PROCESS 65 ... ... Hardware Counters Control Doamin VM 0 Physical Hardware Hypervisor VM 1 VM N VM 0 Counters VM 1 Counters VM N Counters
Figure 3.2: System data collected from the co-located VMs and host.
percentage of CPU usage by user and system level processes, system idle time and I/O- wait percentages. The top command provides information about used memory pages; however, they are not in the same format as the CPU data. Therefore, memory usage data need to be refined and scaled.
The Iostat [201] is used to measure the disk I/O activity of the VMs and physical host. The Iostat provides various data about device utilization, including the number of kilobytes written and read from the device per second. The data provided by the Iostat also need to be scaled and formatted. The system parameters collected with the above tools are summarized in Table 3.2.
The experiments were conducted with different numbers of simultaneously running VMs. First data is collected by running each benchmark separately on a single VM on the host; resource usages trace for each benchmark is collected separately. Afterward, VMs are run in groups of two, four, eight and fourteen co-located VMs. This process is repeated for all nine benchmarks.
Shell scripts were used to run the experiments from a remote machine over the
Secure Shell (SSH). The shell scripting has been used for two reasons. The first is shell scripts can efficiently run benchmarks and collect data from remote machines. The second is that the scripts can be easily created and manipulated. The shell scripting is a powerful way to run processes and collect information about the running process from the system. For each benchmark, a separate script is written to run the experiments and collect information. All the collected data are filtered and scaled using another shell script for analysis.