Workflow tasks parallelism
Experimentation demonstrates that intrinsic characteristics of applications play an important role in the scheduling process. For instance, the Epigenomics application that exhibits parallelism on 96% of its nodes has the potential to employ 24 VMs for its execution. Nevertheless, when HEFT and GA-ETI distribute these tasks among 12 VMs, the execution time reaches almost the minimum runtime as shown in Figure 43a. Notice that doubling the numbers of resources to 24 does not necessarily reduce execution time by half. This is because parallel tasks require transfer of replica files causing extra file transfer that increments runtime.
The aforementioned findings imply that GA-ETI and other scheduling algorithms such as [7, 13, 72] do not necessarily exploit parallelism to the fullest. For instance refer to the results of the Cybershake workflow from
Figure 43b. This is a data orientated workflow with its majority of nodes distributed on two workflow levels. These nodes can be distributed over 48 resources but instead the GA-ETI selects only two VMs. The reason for this decision is that the time to transfer files is significant greater than the execution of tasks in the workflow.
Nevertheless the GA-ETI exhibited a particular favouritism for specific solutions. Figure 44 expresses results from Figure 43a in a pareto style. On it, the GA-ETI shows that the majority of solutions are concentrated on the center of the graph. An important implication of these findings is that configurations have 6, 8, and 12 machines that completely exploit application parallelism.
System utilization
The results obtained in Sections 4.6 4.7 showed that system utilization has a direct relation to task parallelism. The data replication technique from BaRRS produces a strong influence on parallel tasks, consequently the system utilization increases. To illustrate this refer to Figure 19 Figure 22 where BaRRS and Provenance [72] produce trade-off for applications exhibiting parallelism. On them, the proposed BaRRS presented superior outcomes. The reason that Provenance does not produce similar results is that it allows users to freely manipulate monetary constraints without advising of the potential degradation in the
Chapter 7: Discussion
104
system utilization. On the contrary, the proposed BaRRS allows a user to change the priority to the monetary cost optimization but it also keeps control of the runtime; as a consequence, the system utilization is not affected.
An additional finding regarding the system utilization is the strong dependence of this parameter on the nature of each application. The tested workflows have the same number of tasks but each one requires different computational power and consequently their analysis is different. To illustrate this refer to Figure 23,Figure 26,Figure 29 andFigure 32, graphs exhibit different shapes for each workflow type. The shape of these graphs is dictated by the number of data files, tasks dependencies and computational demands of each workflow.
Number of resources
This thesis makes a contribution to open discussion about the number of resources to execute applications. Results from Section 4.7 exhibited that the number of resources directly affects the scheduling process. The number of nodes in an application does not directly influence the final number of resources as exhibited in Table 7. For this reason, the number of resources was included as a variable in the GA-ETI. This approach provides an interesting tactic to solve the problem of selecting the number of resources to run a workflow based on its nature.
Randomness of the genetic algorithm
An important contribution of GA-ETI was the reduction of randomness compared to the original GA as presented in Figure 45. This contribution relies on the GA-ETI mutation operator. This operator introduces additional resources into the internal genetic process causing diversity among chromosomes. This diversity causes the GA-ETI to expand the range of scheduling options. A deep analysis of chromosome evolution in the GA-ETI revealed that after each generation the quality of each chromosome significantly increments converging to a final solution with fewer iterations.
A trade-off of optimization objectives
This thesis introduced a novel view to build a trade-off that a user employs to decide which of the optimization objectives must have higher priority. The objective of the BaRRS
105
trade-off is to offer the user a general view to execute an application considering its characteristics such as computational demands and file data usage. Each of the different options from the trade-off offers advantages and disadvantages that only a user can decide based on his/her particular interests. To illustrate this concept Figure 19Figure 22 present a trade-off exhibiting the different options to execute each application. Figure 19 shows that a user has the option to execute application at a minimal time of ~20 000 seconds at the cost of 20.74 dollars or reduce monetary cost to 10.676 dollars at the expense of increasing execution time to ~120 000 seconds. The main objective of the BaRRS is to guide a user in selecting the best option according to his/her needs.
System utilization and evaluation function values
Another interesting finding was the similarities between system utilization and evaluation function. Experimentation in Section 6.7 shows that not all results exhibit an evaluation function value above 80% when testing the PSO-DS as shown in Figure 53. Shapes of these graphs have strong similarities with the system utilization results from Section 4.7 . Readers must also notice that the number of machines in the shaded area in columns 2 and 3 from Figure 53 matches the ones that obtained the highest evaluation function values in column 1 from the same figure. Moreover, runtime and monetary cost uniformly decrease and increase, respectively, as the number of resources increases. Nevertheless, only the system utilization exhibits a peak at a particular number of machines (see Figure 23Figure 26 and Figure 32).