This chapter presents a detailed description of the background and previous related works in the areas of job scheduling, resource allocation and image processing on distributed systems. The definitions of the various terms and concepts used in this thesis, as well as a summary of literature survey were presented. In addition, the chapter provides an overview of the Apache Spark architecture.
Spark applications are divided at runtime into Jobs, Stages, and Tasks. These applications consist of various components including Driver, Executors and SparkContext, and can connect and request resources from various cluster managers including Spark Standalone, Apache Mesos, and Hadoop YARN. Furthermore, Spark applications can access data stored in local or distributed filesystem such as HDFS. Spark implements two different resource scheduling and allocation policies: static and dynamic, and provides many configuration parameters to control the runtime behaviour of the Spark system.
Chapter 3
Experimental Design
This chapter presents the experimental design methodology used in this thesis. The first part of the chapter gives a detailed description the tools used in the experiments, including the hardware configurations of both the testbed server and cluster, as well as their software specifications. The second part of the chapter describes the configuration parameters used and how they were set for the different experiments to evaluate the performance impact of (Spark’s job scheduling and resource allocation) configuration parameters on image processing applications. A description of the three heterogeneous workload applications used in this thesis, as well as the performance metrics selected for their evaluation are also presented. The last part of the chapter presents the description of the multiple experiments done.
3.1
Hardware and Software Configurations
This section describes the hardware setup as well the software configurations of the two testbeds used in the different experiments of this thesis. Information about operating system, processor, memory, disk, and number of cluster nodes is provided. All the nodes of the Spark cluster and the testbed server run Ubuntu 16.04.3 LTS Operating System.
3.1.1
Sever Configurations
The server machine onomi.usask.ca (called Onomi in the rest of this document) was used as a testbed for running the sequential versions of the three image processing applications. The computer is configured with two Intel® Xeon® E5-2690 v4 CPUs (14 cores @ 2.6 GHz on each CPU, hyper-threaded, for a total of 56 virtual cores), 620 GB RAM and 43 TB of disk space. This machine runs Python version 3.5.2.
Exclusive access to Onomi was provided for a limited time frame to run the sequential version of the three image processing applications and evaluate their performance on a single large computer.
3.1.2
Spark Cluster Configurations
The DISCUS Lab at the University of Saskatchewan has a heterogeneous Spark cluster consisting of 12 virtual machines on 12 physical machines, with 1 master node and 11 worker nodes. The nodes are interconnected
using a 1 Gigabit private network. This cluster was used as a testbed for running the distributed version of the image processing applications.
The master node and each of the first nine worker nodes are configured with a single Intel®Core™i7-2600 CPU (4 cores @ 3.4 GHz, hyper-threaded, for a total of 8 virtual cores). In addition, the master node is equipped with 11 GB RAM and 396 GB of disk space, while the nine worker nodes are equipped with 14 GB RAM and 7.4 TB of disk space. Each of these worker nodes is configured to provide the cluster with 7 virtual cores and 13 GB of memory. The remaining two worker nodes have slightly different configurations. They are each equipped with a single Intel®Xeon® E5-2403 CPUs (4 cores @ 1.8 GHz, hyper-threaded, for a total of 8 virtual cores), 35 GB RAM and 7.7 TB of disk space, and provide the cluster with 7 virtual cores and 34 GB of memory.
With these configurations, 1 virtual core and 1 GB memory were reserved for the Operating System and other background processes on each of the 11 worker nodes. The cluster runs Spark version 2.1.0, Hadoop YARN version 2.7.2, Mesos version 1.2.0, and Python version 2.7.12. Table 3.1 shows the hardware configurations of the two testbeds. In all, the server has a total resource of 56 virtual cores and 620 GB memory, while the cluster has a total resource of 77 virtual cores and 185 GB memory for scheduling and allocating to different Spark applications by the cluster managers as shown in Table 3.2.
The heterogeneity of the testbed cluster comes from the fact that two of the worker machines have slower CPU cores than the rest (approximately half the speed). Experiments show that tasks scheduled on these slower machines sometimes tend to take longer time to finish than the rest. To overcome the effect of having slower machines slow down the processing of an application, speculative execution of tasks was evaluated to verify its impact on the makespan of the set of jobs. With speculative execution of tasks enabled, the tasks that are running slowly in a given stage may be re-launched on a different executor.
Table 3.1: Testbeds Hardware Configurations
Node CPU Memory Disk
Master Node Intel® Core™i7-2600 @ 3.4 GHz 11 GB 396 GB 9 Worker Nodes Intel® Core™i7-2600 @ 3.4 GHz 14 GB 7.4 TB 2 Worker Nodes Intel®Xeon® E5-2403 @ 1.8 GHz 35 GB 7.7 TB Onomi 2 Intel®Xeon® E5-2690 v4 @ 2.6 GHz 620 GB 43 TB