Experimental Results

Table 3 to 6 summarize the computational results for the two objectives with a differentiation between sparse and dense problem instances. The formulations are analysed with respect to the average relative percentage deviation (RPD(%)) from the best known solution found by either the solver or the heuristic, as well as the average total runtime in seconds (t(s)) and the number of instances solved to optimality (#opt) over 10 variations of a problem type. Bold face values highlight the best results. Statistical significance was tested with the Wilcoxon signed-rank test (* p < 0.05).

For M = 2 machines and J = 5 jobs the position-based model and the precedence-based model are capable of finding the optimum solution values for both objectives as well as for dense and sparse matrices within a few seconds. The time-index-based model requires a

significant higher computation time to solve the problems to optimality. Moreover, it is not able to solve all dense instances with M = 2 machines, J = 5 jobs and a maximum of T = 15 different tools to optimality. The time-index-based model is not able to find integer solution values for larger problems with either more jobs or tools within 1800 seconds since the number of variables depends highly on K.

In the large, the position-based model requires less time to reach optimality than the precedence-based model. For instances with up to J = 15 jobs both the position-based model and the precedence-based model find good or even the best solution very quickly, however, the time limit of 1800 seconds is not enough for the models to prove optimality. When the problem size is increased to J = 25 jobs, the RPD of the mathematical models becomes noticeable high.

Figure 3 and Figure 4 show the box plots for the average relative deviation of sparse and dense matrices for minimizing makespan and total flowtime respectively. For minimizing makespan, it is shown that the average RPD as well as the average deviation of the RPD across dense instances is lower for the precedence-based formulation than for the position-based formulation. For minimizing total flowtime, both models yield lower RPD values for sparse instances in average across all instances. Figure 5 to Figure 8 present the comparison of the solution quality (bars) and the computation time (lines) of the position-based and the precedence based model and the benchmark ILS algorithm. The performance is measured by the RPD and the average computation time given in seconds. For minimizing makespan and all sparse problems, the position-based formulation shows the most robust results in terms of solution quality. Yet, the ILS is able to find better solutions for large problems with M = four machines, J = 25 jobs and T = 20 tools within a short time. For dense matrices, however, the precedence-based model outperforms the position-based model in terms of solution quality but not in terms of computation time. Still, the ILS is able to yield a significant lower average RPD and runtime for problems with M = 4 machines.

For minimizing total flowtime, the position-based model yields a high solution quality and thus a low RPD value for up to J = 15 jobs for sparse and dense problems. However, for instances with more than J = 15 jobs and up to T = 15 tools, the precedence-based model yields better results within the given time limit. For the group of the largest problem instances, the time limit is not sufficient for the precedence-based model to find good solutions. Still, the performance of the mathematical models cannot compete with the ILS heuristic for larger problems. It is interesting to note that the position-based formulation and the precedence-based formulation require a higher computation time in average for sparse matrices. Although sparse matrices require fewer tool switches, the number of empty slots increases the number of tools that can be kept in the magazine because they may be used in the future.

Overall, the precedence-based formulation and the position-based formulation work better, in terms of solution quality and computation time, on instances where the minimum number of tools for a job is close to the magazine capacity, i.e. instances with a high job-tool-matrix density.

The assumption of Burger et al. (2015) and Laporte et al. (2004) regarding the performance

A Comparison of Different Mathematical Models

differences for sparse and dense matrices for the SSP can thus be confirmed for the SSP-NPM.

However, their claim that the precedence-based formulation outperforms the position-based formulation for dense problems can only be confirmed if the time limit is sufficient so that the precedence-based model can converge to good solutions.

Table 3. Comparison of the performance of sparse matrices for minimizing makespan.

position-based precedence-based time-index-based ILS M J T RPD

* for instances with integer solution found n/a: no integer solutions found within 1800 seconds

Table 4. Comparison of the performance of dense matrices for minimizing makespan.

position-based precedence-based time-index-based ILS

* for instances with integer solution found n/a: no integer solutions found within 1800 seconds

Table 5. Comparison of the performance of sparse matrices for minimizing total flowtime.

position-based precedence-based time-index-based ILS M J T RPD

(%) t(s) #opt RPD

(%) t(s) #opt RPD

(%) t(s) #opt RPD (%) t(s)

2 5 10 0 0 10 0 0 10 0 36 10 2 12

2 5 15 0 0 10 0 0 10 0 214 10 3 12

2 10 10 0 321 10 0 575 10 n/a 1800 0 6 17

2 10 15 0 818 10 0 1800 0 n/a 1800 0 3 20

2 15 15 0 1800 0 1 1800 0 n/a 1800 0 6 49

3 15 15 1 1800 0 3 1800 0 n/a 1800 0 6 28

3 25 15 7 1800 0 5 1800 0 n/a 1800 0 3 115

4 25 15 13 1800 0 8 1800 0 n/a 1800 0 1 90

4 25 20 13 1800 0 17 1800 0 n/a 1800 0 1 91

n/a: no integer solutions found within 1800 seconds

Table 6. Comparison of the performance of dense matrices for minimizing total flowtime.

position-based precedence-based time-index-based ILS M J T RPD

(%) t(s) #opt RPD

(%) t(s) #opt RPD

(%) t(s) #opt RPD (%) t(s)

2 5 10 0 0 10 0 0 10 16* 661 9 4 12

2 5 15 0 0 10 0 0 10 9* 1672 2 0 12

2 10 10 0 298 10 0 772 10 n/a 1800 0 4 21

2 10 15 0 356 10 0 1673 2 n/a 1800 0 4 22

2 15 15 0 1800 0 4 1800 0 n/a 1800 0 5 56

3 15 15 2 1800 0 2 1800 0 n/a 1800 0 6 36

3 25 15 11 1800 0 6 1800 0 n/a 1800 0 0 145

4 25 15 11 1800 0 8 1800 0 n/a 1800 0 1 95

4 25 20 14 1800 0 31 1800 0 n/a 1800 0 1 99

* for instances with integer solution found n/a: no integer solutions found within 1800 seconds

A Comparison of Different Mathematical Models

Figure 3. Percentile deviation of the position-based and precedence-based models when compared to the best known solutions for minimizing makespan.

Figure 4. Percentile deviation of the position-based and precedence-based models when compared to the best known solutions for minimizing total flowtime.

Figure 5. Comparison of the computation time and the average percentage deviation of sparse matrices for mimizing makespan (lines: average computation time, bars: RPD).

Figure 6. Comparison of the computation time and the average percentage deviation of dense matrices for mimizing makespan (lines: average computation time, bars: RPD).

A Comparison of Different Mathematical Models

Figure 7. Comparison of the computation time and the average percentage deviation of sparse matrices for mimizing total flowtime (lines: average computation time, bars: RPD).

Figure 8. Comparison of the computation time and the average percentage deviation of dense matrices for mimizing total flowtime (lines: average computation time, bars: RPD).