Preliminary Results for State of the Art Algorithms
A.1 Statistical Results
In the following, the results reflecting each indicator are compiled into tables. As IGD and spread need a reference solution set, we build it by consolidating the approximated fronts from 30 runs of all algorithms into one reference set for each instance. Then, we employ Kruskal-Wallis test and Wilcoxon test for statistical analysis with the confidence interval of 95%.
A.1.1 Results for 5000-node Instances
The followings present the statistical analysis for 5000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
100
Table A.1: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
5000 1 0.156 (0.018) 0.221 (0.028) 0.227 (0.028) 2.43e-12 5000 2 0.179 (0.025) 0.190 (0.025) 0.207 (0.024) 3.68e-5 5000 3 0.174 (0.021) 0.202 (0.023) 0.188 (0.021) 5.06e-05 5000 4 0.174 (0.020) 0.212 (0.024) 0.186 (0.022) 5.13e-08 5000 5 0.192 (0.020) 0.210 (0.023) 0.181 (0.019) 2.12e-05 5000 6 0.166 (0.015) 0.203 (0.020) 0.179 (0.017) 2.45e-09 5000 7 0.188 (0.030) 0.194 (0.024) 0.184 (0.029) 3.50e-01 5000 8 0.178 (0.020) 0.189 (0.020) 0.177 (0.025) 3.00e-02 5000 9 0.169 (0.018) 0.204 (0.025) 0.183 (0.024) 4.67e-07 5000 10 0.156 (0.017) 0.200 (0.022) 0.176 (0.024) 3.31e-09
Table A.2: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
5000 1 0.555 (0.042) 0.605 (0.030) 0.591 (0.046) 4.27e-05 5000 2 0.572 (0.035) 0.630 (0.043) 0.601 (0.051) 1.40e-05 5000 3 0.560 (0.038) 0.620 (0.041) 0.606 (0.044) 2.38e-06 5000 4 0.568 (0.044) 0.626 (0.049) 0.617 (0.050) 4.48e-05 5000 5 0.584 (0.038) 0.623 (0.035) 0.605 (0.033) 5.59e-04 5000 6 0.571 (0.039) 0.619 (0.049) 0.595 (0.038) 6.63e-4 5000 7 0.573 (0.045) 0.621 (0.032) 0.601 (0.032) 1.05e-04 5000 8 0.580 (0.026) 0.616 (0.049) 0.601 (0.048) 4.18e-03 5000 9 0.586 (0.037) 0.611 (0.041) 0.595 (0.051) 8.66e-03 5000 10 0.568 (0.040) 0.623 (0.041) 0.610 (0.049) 1.80e-05
Table A.3: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
5000 1 0.397 (0.011) 0.377 (0.014) 0.366 (0.015) 1.21e-10 5000 2 0.368 (0.013) 0.371 (0.013) 0.355 (0.011) 5.85e-05 5000 3 0.412 (0.011) 0.405 (0.012) 0.388 (0.020) 1.68e-06 5000 4 0.381 (0.012) 0.371 (0.013) 0.373 (0.015) 4.95e-03 5000 5 0.376 (0.010) 0.378 (0.014) 0.381 (0.012) 3.50e-01 5000 6 0.375 (0.009) 0.368 (0.013) 0.365 (0.011) 3.81e-03 5000 7 0.407 (0.013) 0.412 (0.012) 0.407 (0.017) 5.30e-01 5000 8 0.370 (0.011) 0.372 (0.011) 0.370 (0.016) 7.10e-01 5000 9 0.383 (0.010) 0.376 (0.010) 0.373 (0.017) 2.18e-02 5000 10 0.380 (0.010) 0.370 (0.010) 0.371 (0.014) 1.39e-03
A.1.2 Results for 10000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.4: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
10000 1 0.173 (0.016) 0.190 (0.018) 0.188 (0.025) 3.66e-03 10000 2 0.196 (0.019) 0.180 (0.024) 0.189 (0.034) 7.13e-03 10000 3 0.174 (0.017) 0.189 (0.017) 0.181 (0.032) 8.16e-03 10000 4 0.227 (0.031) 0.196 (0.020) 0.185 (0.023) 3.07e-07 10000 5 0.204 (0.029) 0.191 (0.027) 0.170 (0.021) 1.09e-05 10000 6 0.198 (0.025) 0.177 (0.021) 0.171 (0.019) 2.98e-05 10000 7 0.177 (0.022) 0.178 (0.023) 0.160 (0.023) 3.59e-03 10000 8 0.175 (0.018) 0.205 (0.021) 0.171 (0.024) 8.78e-08 10000 9 0.197 (0.022) 0.178 (0.019) 0.162 (0.019) 1.38e-07 10000 10 0.179 (0.022) 0.169 (0.022) 0.164 (0.021) 3.96e-02
Table A.5: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
10000 1 0.548 (0.039) 0.630 (0.047) 0.582 (0.043) 3.23e-08 10000 2 0.570 (0.041) 0.624 (0.036) 0.597 (0.050) 1.05e-05 10000 3 0.564 (0.037) 0.595 (0.040) 0.577 (0.039) 1.82e-02 10000 4 0.602 (0.037) 0.643 (0.050) 0.628 (0.054) 3.95e-03 10000 5 0.578 (0.035) 0.621 (0.043) 0.603 (0.042) 5.82e-04 10000 6 0.580 (0.035) 0.641 (0.048) 0.600 (0.043) 1.43e-06 10000 7 0.572 (0.034) 0.610 (0.049) 0.583 (0.043) 1.66e-03 10000 8 0.575 (0.034) 0.621 (0.043) 0.590 (0.045) 2.13e-04 10000 9 0.592 (0.044) 0.627 (0.046) 0.610 (0.048) 2.70e-02 10000 10 0.568 (0.027) 0.632 (0.049) 0.599 (0.044) 1.47e-06
Table A.6: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
10000 1 0.361 (0.009) 0.356 (0.011) 0.354 (0.014) 4.01e-02 10000 2 0.367 (0.010) 0.371 (0.013) 0.368 (0.015) 3.69e-01 10000 3 0.347 (0.012) 0.350 (0.010) 0.347 (0.013) 3.21e-01 10000 4 0.366 (0.013) 0.383 (0.014) 0.380 (0.014) 2.43e-05 10000 5 0.349 (0.014) 0.363 (0.015) 0.360 (0.015) 3.23e-04 10000 6 0.344 (0.013) 0.356 (0.013) 0.353 (0.012) 2.16e-03 10000 7 0.355 (0.011) 0.358 (0.013) 0.362 (0.017) 2.13e-01 10000 8 0.347 (0.012) 0.341 (0.010) 0.346 (0.015) 2.69e-01 10000 9 0.350 (0.010) 0.367 (0.013) 0.364 (0.016) 3.98e-06 10000 10 0.333 (0.012) 0.341 (0.012) 0.334 (0.014) 1.45e-02
A.1.3 Results for 15000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.7: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
15000 1 0.188 (0.021) 0.167 (0.019) 0.159 (0.016) 3.23e-07 15000 2 0.198 (0.021) 0.182 (0.027) 0.169 (0.021) 1.89e-05 15000 3 0.162 (0.021) 0.183 (0.013) 0.159 (0.025) 1.78e-05 15000 4 0.182 (0.025) 0.165 (0.019) 0.156 (0.018) 1.55e-04 15000 5 0.170 (0.024) 0.172 (0.018) 0.162 (0.025) 2.16e-01 15000 6 0.159 (0.017) 0.177 (0.024) 0.159 (0.019) 4.62e-03 15000 7 0.175 (0.024) 0.174 (0.020) 0.165 (0.013) 6.93e-02 15000 8 0.174 (0.020) 0.185 (0.016) 0.151 (0.021) 9.68e-08 15000 9 0.205 (0.023) 0.183 (0.016) 0.179 (0.018) 8.89e-06 15000 10 0.169 (0.017) 0.193 (0.024) 0.178 (0.020) 3.79e-04
Table A.8: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
15000 1 0.589 (0.040) 0.627 (0.043) 0.600 (0.044) 4.63e-03 15000 2 0.566 (0.036) 0.612 (0.041) 0.606 (0.047) 1.48e-04 15000 3 0.561 (0.032) 0.606 (0.039) 0.590 (0.035) 8.18e-05 15000 4 0.585 (0.037) 0.621 (0.036) 0.600 (0.041) 1.69e-03 15000 5 0.571 (0.036) 0.625 (0.048) 0.584 (0.049) 3.96e-05 15000 6 0.562 (0.039) 0.618 (0.037) 0.590 (0.040) 1.09e-05 15000 7 0.574 (0.037) 0.619 (0.037) 0.590 (0.036) 1.13e-04 15000 8 0.561 (0.043) 0.611 (0.038) 0.592 (0.052) 1.35e-04 15000 9 0.579 (0.039) 0.637 (0.037) 0.615 (0.053) 1.36E-05 15000 10 0.559 (0.037) 0.628 (0.044) 0.590 (0.030) 1.02E-07
Table A.9: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
15000 1 0.332 (0.011) 0.349 (0.011) 0.346 (0.012) 7.76e-07 15000 2 0.330 (0.011) 0.341 (0.012) 0.339 (0.015) 1.87e-03 15000 3 0.329 (0.012) 0.324 (0.008) 0.332 (0.017) 4.63e-02 15000 4 0.321 (0.012) 0.334 (0.014) 0.334 (0.015) 5.44e-04 15000 5 0.337 (0.014) 0.345 (0.011) 0.340 (0.020) 1.28e-01 15000 6 0.359 (0.011) 0.354 (0.012) 0.356 (0.016) 4.08e-01 15000 7 0.352 (0.013) 0.357 (0.012) 0.355 (0.011) 3.02e-01 15000 8 0.340 (0.010) 0.349 (0.010) 0.351 (0.019) 1.08e-02 15000 9 0.365 (0.012) 0.387 (0.015) 0.377 (0.012) 4.60e-07 15000 10 0.345 (0.010) 0.338 (0.012) 0.339 (0.014) 3.3e-02
A.1.4 Results for 20000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.10: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
20000 1 0.199 (0.031) 0.176 (0.015) 0.171 (0.020) 4.37e-04 20000 2 0.180 (0.025) 0.190 (0.025) 0.171 (0.019) 6.42e-03 20000 3 0.177 (0.024) 0.222 (0.021) 0.190 (0.024) 4.15e-09 20000 4 0.190 (0.021) 0.183 (0.021) 0.168 (0.020) 7.59e-04 20000 5 0.175 (0.021) 0.184 (0.021) 0.163 (0.023) 1.16e-03 20000 6 0.183 (0.024) 0.175 (0.017) 0.164 (0.022) 9.42e-03 20000 7 0.169 (0.017) 0.174 (0.018) 0.160 (0.018) 3.18e-03 20000 8 0.183 (0.017) 0.203 (0.021) 0.172 (0.022) 1.58e-06 20000 9 0.170 (0.022) 0.188 (0.024) 0.176 (0.032) 8.39e-03 20000 10 0.192 (0.028) 0.163 (0.016) 0.161 (0.023) 2.52e-06
Table A.11: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
20000 1 0.581 (0.038) 0.640 (0.038) 0.617 (0.037) 1.11e-06 20000 2 0.587 (0.036) 0.639 (0.043) 0.599 (0.032) 1.37e-05 20000 3 0.553 (0.039) 0.591 (0.035) 0.583 (0.040) 6.14e-04 20000 4 0.558 (0.039) 0.628 (0.034) 0.574 (0.046) 5.00e-08 20000 5 0.579 (0.041) 0.620 (0.033) 0.597 (0.030) 1.64e-04 20000 6 0.588 (0.035) 0.624 (0.042) 0.597 (0.040) 1.64e-03 20000 7 0.562 (0.039) 0.617 (0.035) 0.584 (0.035) 5.87e-06 20000 8 0.565 (0.035) 0.609 (0.047) 0.586 (0.048) 2.08e-03 20000 9 0.559 (0.030) 0.607 (0.033) 0.593 (0.043) 7.48e-06 20000 10 0.578 (0.037) 0.635 (0.052) 0.612 (0.047) 8.35e-05
Table A.12: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
20000 1 0.367 (0.014) 0.389 (0.012) 0.382 (0.017) 1.11e-06 20000 2 0.357 (0.014) 0.357 (0.013) 0.362 (0.013) 1.73e-01 20000 3 0.340 (0.015) 0.324 (0.010) 0.329 (0.015) 1.81e-04 20000 4 0.332 (0.011) 0.336 (0.013) 0.338 (0.015) 1.89e-01 20000 5 0.334 (0.014) 0.337 (0.013) 0.341 (0.019) 3.05e-01 20000 6 0.331 (0.012) 0.341 (0.014) 0.340 (0.016) 2.19e-02 20000 7 0.343 (0.011) 0.343 (0.014) 0.343 (0.010) 9.83e-01 20000 8 0.337 (0.010) 0.337 (0.013) 0.342 (0.014) 1.79e-01 20000 9 0.354 (0.013) 0.349 (0.017) 0.347 (0.018) 3.83e-01 20000 10 0.346 (0.013) 0.369 (0.013) 0.364 (0.016) 7.63e-08
A.1.5 Results for 25000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.13: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
25000 1 0.226 (0.028) 0.184 (0.031) 0.189 (0.027) 1.31e-06 25000 2 0.181 (0.025) 0.164 (0.020) 0.157 (0.014) 1.26e-04 25000 3 0.170 (0.019) 0.174 (0.015) 0.173 (0.029) 6.32e-01 25000 4 0.188 (0.021) 0.182 (0.023) 0.165 (0.021) 4.21e-04 25000 5 0.183 (0.022) 0.183 (0.024) 0.169 (0.025) 2.14e-02 25000 6 0.210 (0.025) 0.201 (0.023) 0.185 (0.028) 2.43e-04 25000 7 0.214 (0.030) 0.194 (0.026) 0.179 (0.026) 1.86e-04 25000 8 0.184 (0.026) 0.182 (0.013) 0.170 (0.020) 3.02e-02 25000 9 0.178 (0.021) 0.158 (0.017) 0.153 (0.020) 8.71e-05 25000 10 0.200 (0.021) 0.179 (0.019) 0.171 (0.021) 8.66e-06
Table A.14: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
25000 1 0.579 (0.039) 0.620 (0.041) 0.581 (0.036) 1.06e-04 25000 2 0.585 (0.040) 0.631 (0.037) 0.598 (0.036) 6.89e-05 25000 3 0.577 (0.041) 0.602 (0.040) 0.585 (0.038) 5.82e-02 25000 4 0.573 (0.043) 0.609 (0.038) 0.599 (0.033) 8.87e-03 25000 5 0.562 (0.037) 0.624 (0.037) 0.598 (0.044) 2.12e-06 25000 6 0.592 (0.054) 0.639 (0.043) 0.606 (0.054) 1.23e-03 25000 7 0.581 (0.038) 0.630 (0.052) 0.601 (0.038) 1.03e-03 25000 8 0.551 (0.043) 0.629 (0.044) 0.580 (0.038) 5.26e-08 25000 9 0.584 (0.035) 0.643 (0.044) 0.600 (0.041) 1.60e-06 25000 10 0.568 (0.056) 0.624 (0.038) 0.590 (0.049) 1.46e-04
Table A.15: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
25000 1 0.326 (0.012) 0.346 (0.014) 0.338 (0.018) 6.50e-06 25000 2 0.335 (0.013) 0.348 (0.012) 0.347 (0.013) 1.14e-04 25000 3 0.316 (0.012) 0.319 (0.009) 0.314 (0.015) 3.04e-01 25000 4 0.308 (0.011) 0.318 (0.013) 0.320 (0.015) 8.13e-04 25000 5 0.335 (0.011) 0.340 (0.013) 0.343 (0.020) 5.26e-02 25000 6 0.338 (0.013) 0.352 (0.018) 0.351 (0.018) 1.20e-03 25000 7 0.333 (0.012) 0.344 (0.016) 0.346 (0.017) 4.39e-03 25000 8 0.341 (0.013) 0.349 (0.012) 0.347 (0.016) 6.89e-02 25000 9 0.332 (0.010) 0.344 (0.013) 0.343 (0.015) 6.62e-04 25000 10 0.331 (0.011) 0.344 (0.013) 0.343 (0.016) 2.64e-04
A.1.6 Results for 30000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.16: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
30000 1 0.329 (0.022) 0.234 (0.027) 0.276 (0.025) 6.60e-15 30000 2 0.326 (0.030) 0.191 (0.024) 0.230 (0.022) 3.24e-16 30000 3 0.279 (0.027) 0.176 (0.018) 0.194 (0.023) 2.38e-14 30000 4 0.326 (0.033) 0.188 (0.017) 0.241 (0.025) 2.20e-16 30000 5 0.244 (0.023) 0.174 (0.018) 0.175 (0.016) 1.99e-13 30000 6 0.287 (0.025) 0.191 (0.028) 0.234 (0.023) 6.16e-15 30000 7 0.242 (0.019) 0.173 (0.014) 0.188 (0.019) 7.14e-14 30000 8 0.273 (0.022) 0.196 (0.029) 0.222 (0.020) 1.55e-13 30000 9 0.254 (0.026) 0.201 (0.022) 0.210 (0.022) 5.57e-10 30000 10 0.266 (0.025) 0.195 (0.021) 0.196 (0.020) 5.01e-13
Table A.17: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
30000 1 0.591 (0.047) 0.643 (0.045) 0.582 (0.045) 2.48e-06 30000 2 0.595 (0.044) 0.631 (0.046) 0.606 (0.046) 2.90e-02 30000 3 0.610 (0.041) 0.644 (0.033) 0.614 (0.043) 3.66e-03 30000 4 0.594 (0.038) 0.646 (0.041) 0.607 (0.035) 5.52e-06 30000 5 0.595 (0.036) 0.627 (0.046) 0.606 (0.049) 3.49e-02 30000 6 0.593 (0.042) 0.634 (0.036) 0.610 (0.043) 5.41e-04 30000 7 0.589 (0.034) 0.642 (0.046) 0.586 (0.046) 1.16e-05 30000 8 0.584 (0.045) 0.639 (0.046) 0.599 (0.041) 6.60e-05 30000 9 0.583 (0.045) 0.651 (0.041) 0.583 (0.047) 2.24e-07 30000 10 0.589 (0.039) 0.634 (0.040) 0.599 (0.038) 2.43e-04
Table A.18: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
30000 1 0.273 (0.008) 0.299 (0.012) 0.292 (0.013) 7.10e-11 30000 2 0.295 (0.010) 0.348 (0.017) 0.332 (0.014) 2.88e-14 30000 3 0.286 (0.010) 0.333 (0.013) 0.325 (0.016) 1.96E-13 30000 4 0.275 (0.013) 0.324 (0.010) 0.307 (0.015) 3.36e-14 30000 5 0.287 (0.011) 0.326 (0.014) 0.328 (0.014) 5.54e-13 30000 6 0.281 (0.011) 0.317 (0.015) 0.302 (0.015) 1.35e-11 30000 7 0.282 (0.010) 0.314 (0.012) 0.307 (0.013) 3.54e-12 30000 8 0.275 (0.010) 0.304 (0.016) 0.295 (0.014) 1.05e-09 30000 9 0.296 (0.010) 0.314 (0.015) 0.315 (0.013) 2.12e-07 30000 10 0.307 (0.013) 0.338 (0.013) 0.340 (0.014) 1.20e-11
A.1.7 Results for 35000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.19: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
35000 1 0.268 (0.022) 0.165 (0.015) 0.179 (0.020) 2.03e-14 35000 2 0.324 (0.023) 0.195 (0.030) 0.244 (0.021) 2.44e-16 35000 3 0.210 (0.032) 0.167 (0.012) 0.172 (0.023) 5.46e-08 35000 4 0.323 (0.025) 0.236 (0.019) 0.258 (0.030) 4.96e-13 35000 5 0.313 (0.031) 0.218 (0.024) 0.240 (0.024) 1.03e-13 35000 6 0.309 (0.019) 0.260 (0.024) 0.275 (0.028) 6.94e-09 35000 7 0.358 (0.024) 0.248 (0.032) 0.281 (0.029) 2.27e-14 35000 8 0.336 (0.040) 0.193 (0.028) 0.243 (0.019) 5.13e-16 35000 9 0.304 (0.026) 0.188 (0.020) 0.237 (0.025) 2.33e-16 35000 10 0.329 (0.026) 0.190 (0.030) 0.237 (0.025) 3.11e-16
Table A.20: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
35000 1 0.600 (0.035) 0.652 (0.042) 0.599 (0.038) 2.38e-06 35000 2 0.583 (0.037) 0.637 (0.040) 0.588 (0.034) 1.19e-06 35000 3 0.574 (0.036) 0.624 (0.045) 0.586 (0.034) 2.55e-05 35000 4 0.594 (0.044) 0.651 (0.037) 0.611 (0.041) 6.01e-06 35000 5 0.585 (0.044) 0.640 (0.042) 0.593 (0.042) 2.09e-05 35000 6 0.561 (0.034) 0.638 (0.047) 0.573 (0.030) 2.23e-08 35000 7 0.588 (0.040) 0.644 (0.052) 0.595 (0.040) 4.07e-05 35000 8 0.584 (0.034) 0.653 (0.042) 0.609 (0.040) 6.42e-08 35000 9 0.578 (0.037) 0.640 (0.033) 0.580 (0.035) 6.83e-09 35000 10 0.599 (0.041) 0.644 (0.039) 0.606 (0.028) 5.29e-05
Table A.21: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
35000 1 0.275 (0.011) 0.320 (0.012) 0.316 (0.015) 2.04e-13 35000 2 0.273 (0.010) 0.319 (0.017) 0.301 (0.013) 9.58e-14 35000 3 0.332 (0.014) 0.356 (0.014) 0.348 (0.018) 1.50e-06 35000 4 0.278 (0.011) 0.304 (0.012) 0.299 (0.013) 2.57e-10 35000 5 0.274 (0.012) 0.300 (0.011) 0.297 (0.014) 1.02e-09 35000 6 0.277 (0.009) 0.283 (0.011) 0.283 (0.015) 2.90e-02 35000 7 0.274 (0.010) 0.302 (0.016) 0.300 (0.016) 4.17e-10 35000 8 0.289 (0.015) 0.335 (0.014) 0.317 (0.010) 3.79e-14 35000 9 0.290 (0.012) 0.333 (0.013) 0.316 (0.012) 2.51e-14 35000 10 0.291 (0.008) 0.335 (0.014) 0.322 (0.012) 4.11e-14
A.1.8 Results for 40000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.22: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
40000 1 0.272 (0.023) 0.210 (0.027) 0.214 (0.019) 4.07e-12 40000 2 0.252 (0.026) 0.179 (0.014) 0.168 (0.018) 3.62e-14 40000 3 0.275 (0.020) 0.193 (0.020) 0.201 (0.022) 2.22e-13 40000 4 0.302 (0.029) 0.204 (0.030) 0.234 (0.023) 2.49e-14 40000 5 0.347 (0.033) 0.209 (0.030) 0.267 (0.024) 2.20e-16 40000 6 0.276 (0.028) 0.210 (0.023) 0.206 (0.022) 8.65e-12 40000 7 0.352 (0.032) 0.176 (0.030) 0.257 (0.023) 2.20e-16 40000 8 0.314 (0.029) 0.217 (0.021) 0.250 (0.027) 7.35e-14 40000 9 0.328 (0.019) 0.204 (0.028) 0.247 (0.028) 7.92e-16 40000 10 0.298 (0.024) 0.190 (0.026) 0.220 (0.022) 7.29e-15
Table A.23: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
40000 1 0.578 (0.042) 0.637 (0.043) 0.575 (0.037) 6.94e-07 40000 2 0.576 (0.038) 0.644 (0.042) 0.609 (0.044) 3.32e-07 40000 3 0.577 (0.040) 0.635 (0.035) 0.591 (0.031) 2.05e-07 40000 4 0.589 (0.038) 0.639 (0.035) 0.589 (0.043) 3.92e-06 40000 5 0.600 (0.049) 0.633 (0.049) 0.603 (0.042) 3.45e-02 40000 6 0.585 (0.044) 0.627 (0.051) 0.605 (0.036) 3.65e-03 40000 7 0.610 (0.040) 0.651 (0.041) 0.608 (0.050) 7.11e-04 40000 8 0.593 (0.045) 0.650 (0.047) 0.588 (0.041) 2.91e-06 40000 9 0.588 (0.036) 0.642 (0.041) 0.605 (0.042) 1.91e-05 40000 10 0.604 (0.038) 0.663 (0.035) 0.607 (0.040) 1.15e-07
Table A.24: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
40000 1 0.286 (0.011) 0.311 (0.017) 0.313 (0.015) 1.53e-09 40000 2 0.269 (0.013) 0.309 (0.011) 0.319 (0.015) 4.61e-14 40000 3 0.286 (0.011) 0.320 (0.013) 0.319 (0.013) 2.34e-12 40000 4 0.272 (0.011) 0.308 (0.017) 0.302 (0.012) 5.17e-12 40000 5 0.273 (0.014) 0.314 (0.015) 0.300 (0.013) 5.45e-13 40000 6 0.265 (0.011) 0.293 (0.014) 0.295 (0.015) 1.56e-10 40000 7 0.290 (0.011) 0.351 (0.017) 0.323 (0.013) 4.13e-16 40000 8 0.279 (0.012) 0.318 (0.012) 0.310 (0.015) 1.44e-12 40000 9 0.276 (0.008) 0.320 (0.015) 0.306 (0.014) 1.70e-13 40000 10 0.280 (0.010) 0.320 (0.016) 0.313 (0.013) 3.60e-13
A.1.9 Results for 45000-node Instances
The followings present the statistical analysis for 10000-node synthetic instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.25: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
45000 1 0.302 (0.033) 0.159 (0.015) 0.212 (0.022) 2.20e-16 45000 2 0.309 (0.029) 0.175 (0.021) 0.228 (0.027) 2.20e-16 45000 3 0.354 (0.024 0.233 (0.031) 0.289 (0.025) 4.19e-16 45000 4 0.314 (0.028) 0.188 (0.018) 0.230 (0.020) 2.20e-16 45000 5 0.297 (0.026) 0.208 (0.023) 0.229 (0.025) 4.05e-14 45000 6 0.345 (0.025) 0.225 (0.030) 0.288 (0.027) 5.11e-16 45000 7 0.355 (0.029) 0.227 (0.032) 0.278 (0.028) 1.70e-15 45000 8 0.411 (0.027) 0.248 (0.038) 0.326 (0.025) 2.20e-16 45000 9 0.336 (0.033) 0.200 (0.025) 0.257 (0.022) 2.20e-16 45000 10 0.350 (0.029) 0.170 (0.025) 0.259 (0.021) 2.20e-16
Table A.26: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
45000 1 0.597 (0.050) 0.641 (0.038) 0.606 (0.039) 5.37e-04 45000 2 0.587 (0.042) 0.663 (0.048) 0.601 (0.041) 1.24e-07 45000 3 0.569 (0.032) 0.639 (0.044) 0.599 (0.041) 1.51e-07 45000 4 0.587 (0.038) 0.657 (0.052) 0.608 (0.042) 3.30e-06 45000 5 0.592 (0.034) 0.651 (0.044) 0.599 (0.034) 1.19e-06 45000 6 0.569 (0.045) 0.642 (0.051) 0.590 (0.046) 2.35e-06 45000 7 0.581 (0.034) 0.638 (0.032) 0.602 (0.052) 3.45e-06 45000 8 0.589 (0.046) 0.638 (0.046) 0.601 (0.047) 3.62e-04 45000 9 0.595 (0.046) 0.652 (0.038) 0.615 (0.047) 2.01e-05 45000 10 0.578 (0.037) 0.643 (0.054) 0.606 (0.029) 7.67e-06
Table A.27: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
45000 1 0.279 (0.014) 0.340 (0.012) 0.315 (0.013) 5.92e-16 45000 2 0.283 (0.012) 0.335 (0.015) 0.314 (0.018) 9.75e-14 45000 3 0.282 (0.009) 0.321 (0.017) 0.303 (0.010) 1.09e-13 45000 4 0.288 (0.012) 0.335 (0.012) 0.322 (0.014) 5.78e-14 45000 5 0.275 (0.011) 0.306 (0.013) 0.305 (0.017) 1.83e-11 45000 6 0.269 (0.008) 0.304 (0.014) 0.286 (0.012) 2.10e-13 45000 7 0.274 (0.011) 0.312 (0.014) 0.296 (0.014) 8.99e-13 45000 8 0.256 (0.009) 0.302 (0.015) 0.283 (0.012) 8.33e-15 45000 9 0.284 (0.012) 0.327 (0.014) 0.311 (0.010) 4.16e-14 45000 10 0.288 (0.011) 0.351 (0.015) 0.324 (0.011) 2.20e-16
A.1.10 Results for Berlin and Dusseldorf Instances
The followings present the statistical analysis for Berlin and Dusseldorf instances. The first table is for IGD. The second table presents spread indicator and the last table shows the hypervolume indicator for each algorithm.
Table A.28: Average results obtained for the IGD metric.
Instance IGD (×10−2)
SPEA-II NSGA-II NSGA-III p-value
Berlin 0.270 (0.027) 0.236 (0.015) 0.330 (0.087) 3.48e-07 Dusseldorf 0.256 (0.024) 0.180 (0.016) 0.188 (0.052) 1.22e-11
Table A.29: Average results obtained for the spread metric.
Instance Spread
SPEA-II NSGA-II NSGA-III p-value
Berlin 0.606 (0.043) 0.635 (0.042) 0.662 (0.205) 9.16e-03 Dusseldorf 0.542 (0.035) 0.619 (0.041) 0.557 (0.045) 8.48e-09
Table A.30: Average results obtained for the hypervolume metric.
Instance Hypervolume
SPEA-II NSGA-II NSGA-III p-value
Berlin 0.346 (0.021) 0.348 (0.010) 0.336 (0.020) 6.66e-03 Dusseldorf 0.396 (0.014) 0.425 (0.012) 0.431 (0.015) 2.05e-11
All the statistical results for all instances has been presented. We have seen the trend that persists throughout all experiments and also a new find on SPEA-II. In the next section, we present the convergence plot of all performance indicators mentioned in this thesis, IGD, spread, and hypervolume.