Performance Modelling and Optimization

Prediction-based performance modelling and optimisation of resources and pro- visioning based on characteristics of the workload are not new topics. They have been widely discussed in the literature using historical information to pre- dict different performance metrics such as execution time for different field of applications in various computing environments such as datacentres [146], Cloud Computing[157][98][165] and Hadoop-MapReduce[102] report different approaches ranging from online/offline instrumental profiling, machine learning techniques to statistical modelling and control theory. Furthermore, within Hadoop-based ap- plications, a number of optimisation methods were investigated with different ob-

jectives and constraints, such as cost, energy, quality and reducing job execution time. The management of resources for Hadoop includes issues with VM place- ment to physical machines [108][169], or placing applications to VM. Within this research, the focus is proactive prediction and optimisation of resources required by the application, and on the basis that cluster nodes deploy homogeneous VM types.

In this section we will review some relevant existing research on using machine learning for performance prediction in a given resource and using optimization approach for performanc optimization and resource provisioning.

Resource allocation and performance analysis for Cloud-based media applica- tions have been studied in terms of considering their individual performance met- rics. These exciting approaches focus on online resource adaptation (i.e. runtime estimation of resource usage) for media tasks being processed to meet QoS, job deadlines or performance goals. The authors [144] modelled resource allocation as a bin packing problem, considering only the CPU usage. They proposed dynamic resource allocation predictions based on Machine Learning algorithms SVM and KNN, to estimate tasks requirements and survival functions in order to estimate how much load a single VM can handle in parallel and shared environments so as not to violate QoS constraints. The authors [100] also aimed to predict the dynamic resource allocation, but specifically for real time video transcoding. This depends on the prediction of future user load demand using time series models. While the researchers have applied Machine Learning for resource prediction, no one has considered video analytic performance metrics in a Hadoop based distrib- uted environment.

However, there are existing research conducted on a general Hadoop-based MapReduce using statistical models and Machine Learning techniques. The re- lated work in this area was proposed by Kambatla et al.[102] who proposed the use of the online RSmaximizer tool that searches for optimum configuration para- meters in terms of Hadoop applications by statistically matching its resource con- sumption with already known applications resource consumption signature and the optimal configuration stored in a database, using a brute force method. However, the optimisation techniques were built on fixed nodes and slots, which we proved in our study of Chapter-4 to have a great impact to overall performance. Verma et al. [160] developed online SLO-based resource provisioning, which can predict a reduced job completion time using the parameters from job profiles (Map, shuffle, reducer phases, completion times), input size and allocated resources. Herodotos et al. [87] proposed the use of an online Elastisizer tool which is an automated technique to optimise different configuration parameter settings and cluster re- sources (sizing) for a Hadoop job to meet performance needs. The authors used a

mix of a statistical blackbox model and a white-box model for estimation. They also used an instrumental application profiler which has high overheads. How- ever, our application is processor-intensive, so this method will make it slower. Moreover, they used a fixed slot number when collecting job profiling data. This approach suffers from resource underutilisation or overutilisation. In addition, the method requires intensive job profiling. The author [107] proposed AROMA system that followed the same approach as [102] in classifying applications using clustering technqiues and SVM algorithm for hadoop job performance modelling. The author applied a pattern search technique that is based upon a SVM model to find the optimal resources to meet the required target at a reduced cost. The variation of slot and task numbers were ignored by AROMA when the model was built, which had a negative impact upon the accuracy of the model, and the op- timisation goal. In addition did not describe how the problem was mathematically formulated.

The author in [104] improvd th work proposed by [160] and proposed a classical language multiplier to optimise resource provision, again based on the Hadoop per- formance model, which was generated by using locally weighted linear regression. The application specific characteristics were ignored and the aim was to focus on map and reduce task durations during modelling to estimate resources (i.e. map/reduce slot).

CRESP applies a search technique that is brute force in order to provision optimal resources within a Hadoop cluster[154] [58]. This is based upon a cost model deployed, which estimates job performance, as well as organising the jobs resources for the job through the use of a regression technique. The brute-force method generates a single solution for testing when there is a large search space with a number of representative and attributes that have a wide distribution of values, which takes longer to process when attempting to reach an optimal solu- tion.

Our technique is different from all previous works in that we evaluate various Machine Learning algorithms using the WEKA tool [39] to find the best model with high prediction accuracy, modelled using feature vectors specifically related to video analytic application performance metrics such as resolution, file size, Hadoop configuration parameters and system performance (i.e. resource consumption) that affects video analytic services and uses them as input to the Machine Learning algorithms. In addition, for performance optimization and resource provisioning we have mathematically formulated the optimisation problem and presenting the genetic algorithm method for fast and effective results. We finally compared the result with other optimization technqiues Pattern Search and Language Multiplier technqiues that were proposed in literature.