Comparative Analysis

The experimental results of the algorithm’s performance evaluation are summarised in Table 5.17, Table 5.18, Table 5.19, and Table 5.20 for J30, J60, J90, and J120 sets, respectively. The first column denotes the name of the algorithm, column “Author(s)” shows the name of the original author. Column “Dev. (%)” shows average deviation of solutions from the critical path (CP). Column “Comp. time” displays the average computational time required to solve each of the benchmark instances. All results in the tables are sorted with respect to the average deviation. In accordance to standard RCPSP performance evaluation experiments, here as the main performance factor is only considered deviation from optimal solutions, whereas computational time is left out and provided strictly as a reference.

Table 5.17 - Experimental evaluation results for J30 dataset

Algorithm Author(s) Dev. (%) Comp. time

DSCCS Bibiks et al. 0.00 29.7

TS Nonobe and Ibaraki (2002) 0.06 21.3

SA Bouleimen and Lecocq (2003) 0.08 17.5

GA Hartmann (1998) 0.09 23.6

Table 5.18 - Experimental evaluation results for J60 dataset

Algorithm Author(s) Dev. (%) Comp. time

DSCCS Bibiks et al. 4.36 63.1

SA Bouleimen and Lecocq (2003) 6.81 38.6

TS Nonobe and Ibaraki (2002) 7.25 41.2

GA Hartmann (1998) 7.49 45.3

Table 5.19 - Experimental evaluation results for J90 dataset

Algorithm Author(s) Dev. (%) Comp. time

DSCCS Bibiks et al. 13.93 95.5

GA Hartmann (1998) 15.85 74.1

TS Nonobe and Ibaraki (2002) 16.01 71.9

151

Table 5.20 - Experimental evaluation results for J120 dataset

Algorithm Author(s) Dev. (%) Comp. time

DSCCS Bibiks et al. 25.14 130.9

GA Hartmann (1998) 29.18 113.3

TS Nonobe and Ibaraki (2002) 30.01 112.9

SA Bouleimen and Lecocq (2003) 32.38 101.2

The above-presented results show that by managing to obtain lowest deviation from optimal solution in all experiments, DSCCS achieves the highest performance between all compared algorithms for all datasets. The performances of other implemented algorithms are in line with the performance evaluations that were by done by Kolisch and Hartmann [22]. These results demonstrate that the application of the algorithm to the proposed model does not impact its performance. It also was worth mentioning that in all experiments DSCCS was able to obtain from three to six solution candidates, whereas other algorithms, due to their limitations, could obtain only one solution.

Computational time, however, shows a different picture. Here, DSCCS demonstrated the worst result, mainly due to the additional computational overhead that is caused by the species conservation procedure. Among all tested algorithms, the fastest to solve all benchmark instances was SA. The main reason for such fast computational speed is the work only with one solution and reduced amount of operations that it makes at each iteration. The computational time of TS is close to the one of the SA, primarily because of the fact that both these methods are trajectory-based. Computational time of GA is somewhere in the middle between the ones of SA and DSCCS.

5.3 Summary

The work in this chapter focused on the application of the DSCCS on the optimisation model proposed in Chapter 3 and activity scheduling with varying resource efficiencies. For this, two sets of experiments are carried out: scheduling and sequencing of the activities of the real practical project, and execution of the edited PSPLIB benchmark instances and subsequent comparison of the received results with the results obtained by other implemented algorithms.

For the first experiment, the HARNet project is selected as the practical example, which is then used to demonstrate the applicability of the proposed

152

optimisation model for scheduling development projects, and analyse the behaviour of the DSCCS when applied to the real-world project. HARNet represents a large-scale aeronautical project which relied on the collaboration of many partners and consisted of many subprojects (WPs). In this case study, the DSCCS is applied to schedule the activities of the WP9, mainly due to the reasons that people who have worked on this sub-project had very little of relevant experience and as the project went on, their effectiveness improved. The WP9 consisted of 51 activities and 6 resource types. The resource from 1 to 4 represent a group of researchers, whereas resources 5 and 6 represent specialised equipment. Because of that, only resources 1-4 can impact the activity durations. To analyse the difference between deterministic and stochastic scheduling, and study the resource experience gain and its effect on the activity durations, the DSCCS is applied to solve two instances of the case study: deterministic, in which activity durations do not vary, and stochastic, in which the activity durations depends on the execution time and applied resources. The reference schedule received in deterministic mode had the makespan of 113 weeks. The reference makespan of stochastic schedule was 97 weeks. Further, two best schedules (one from the deterministic mode and one from stochastic) are selected for comparative analysis. The analysis has shown close to the end of the project’s execution, the duration of the activities has reduced on average by 20%.

For the second experiment, three among the most popular methodologies for the deterministic RCPSP are implemented and are applied to solve the PSPLIB benchmark instances. To correlate the benchmark instances to the proposed optimisation mode, the instances are modified to include additional parameters for resource efficiency and learnability. In total 204 instances are selected and are divided into four datasets. Implemented algorithms, along with the DSCCS, are applied to solve each of the benchmark instances. To evaluate the performances of the algorithms, two criteria are considered: average deviation from CP, and average computational time. As the result of the experimental evaluation, the DSCCS showed the best level of performance among all algorithms, however, at the same time, it had the worst computational time, mainly due to the additional computational overhead that was caused by the application of the species conservation technique.

153