With prediction module

In previous chapter it was confirmed that all of the components (except the Prediction module) were wired together correctly and the virtual cluster is scaled up and down correctly in response to the workload, therefore now it would be time to test the auto-scaling application with the Prediction module enabled and using real world data. Originally I planned to use Wikipedia’s access trace provided by [66], but due to the number of requests in the access trace, my tool couldn’t send the requests as fast as it was reading the log file (the tool became a bottleneck), therefore I used access trace of ClarkNet’s (former internet service provider) web server [44]. Because the requests were sometimes sparse, the requests were resent twice as fast, e.g. requests received in 3 hour timeframe were resent in 1.5 hours. Because of this speedup I also halved the window’s size in the prediction module from 15 minutes to 7.5 minutes.

To calculate the power consumption I will use an exponential power model proposed by [67]. In their work the researchers show that it is possible to derive an estimate of power consumption by using CPU’s utilisation rate and CPU’s average and maximum power consumption. The formula to derive the power consumption is presented in Equation 1 where P is power consumption, D – average power consumption during idle time, M – average power consumption during heavy load and U – CPU’s utilisation expressed in percentages. This formula will be used to calculate total power consumed by all VMs.

𝑃 = 𝐷 + (𝑀 − 𝐷)𝑈0.5

Equation 1

Obviously, the estimation of power consumption by this formula will not be correct in the context of virtualisation as each VM is allocated only a slice of physical CPU, therefore 100% of CPU utilisation on VM might not correspond to 100% utilisation of physical CPU, but for the purposes of comparison this is assumption is acceptable. Also, during the tests all VMs were allocated on physical hosts which had the same CPU model (Intel Xeon X3360) and all VMs were given the same CPU share. Values for parameters

D and M will be used as provided by [68].

In order to measure how the Prediction module works I ran six tests – three 3 hour slices from ClarkNet’s access trace were replayed twice – once with the Prediction module enabled and once with it disabled. Every request in the access log was substituted to be /DemoWebsite/performance?n=330

which corresponds to workload equivalent to multiplying two 330 by 330 matrices. Results from the tests are presented in Table 13.

45

With Prediction module Without Prediction module Percentage change with Prediction module

enabled Average response time (ms) Total power consumed (kWh) Average response time (ms) Total power consumed (kWh) Average response time Total power consumed Data Set 1 1117 0.3376 1402 0.3208 -20.3% +5.2% Data Set 2 2166 0.3332 2155 0.3320 +0.5% +0.4% Data Set 3 1406 0.2530 2089 0.2100 -32.7% +20.5%

Table 13: Results from testing the Prediction module.

The results from the experiments show that the Prediction module improved the average response time significantly, although it led to higher power consumption.

8.1

Summary

In this final implementation chapter the 6th Milestone of this project was achieved – all components developed in previous chapters were wired together into working software – the auto-scaling solution responded to changes in workload by adjusting the cluster’s size and was able to make adequate predictions about future workload and preparing for it accordingly.

46

9

Evaluation

9.1

Introduction

In this chapter I will give an overview of the results from the experiments conducted, evaluate the solution’s design and implementation, choice of methodology and present possible extensions to the auto-scaling tool which was not implemented in this project due to time and resource constraints.

9.2

Summary of results

Originally, it was planned to evaluate how different VM allocation strategies (random vs. greedy) contribute to energy consumption, but due to technical limitations this was impossible to do, therefore instead I would like to reflect how the predictive model influenced the energy consumption.

Runs which had the Prediction module enabled used more energy – 0.0168 kWh more using the first data set, 0.0012 kWh using the second data set and 0.043 kWh with the third one. However, the average response time was improved only in the first and third case – by 285ms and 683ms respectively. The reason why the response time did not improve in the second test was because the predictive model triggered one VM to be removed, but it did not trigger VMs to be added, whereas in test cases 1 and 2, the predictive model triggered the Brain module to add VMs to the cluster, thus improving the average response, but also worsening the power consumption. From this, it is clear that average response time and power consumption are inversely proportional – higher power consumption results in faster response time and lower power consumption leads to longer response times.

The auto-scaling solution can function without the predictive model and it will have lower power consumption that way, but the web application will become less responsive, as additional resources are provisioned only when the all the VMs in the virtual cluster are over capacitated.

9.3

Evaluation of solution

9.3.1 Scalability

Scalability defines how well the system copes if the problem’s size increases [69]. The auto-scaling tool should scale very well even with high workload – at any time it will be communicating with at most two VMs – it will be communicating with the load balancer when it checks the response time and gets the request count or with the load balancer and a newly instantiated worker when the processes on the worker VM have to be started and the load balancer has to be configured. The only bottleneck for this

47

setup is the load balancer, e.g. a case where a workload is so heavy that two load balancers are required.