5.2 IP Model
5.2.3 Experiments using the IP Model
The aim of the experiments is to obtain optimal solutions if possible. Since the IP model is based on the VRPTW problem, only the data sets based on such problem are used (Sol and Mov). Another reason for using only those data sets, is that the model requires to complete all activities in the instances. From the analysis performed on the Sec data set, it is concluded that in many instances there are not enough employees available to perform all activities, as consequence the Sec data set was discarded in the experiments in this section.
The weights used in the objective function αp and αT are given both the same value,
in this case (0.5) since no particular additional component is more important than the other one.
The experiments were carried out using Gurobi solver version 5.1 and executed on a X64-based PC running Microsoft Windows 7 Enterprise with 2 Duo CPU (3.16 GHz) and four gigabytes of RAM.
5.2.3.1 Tackling the first objective
The objective of tackling WSRP with a model based on VRPTW is investigated by performing the following experiments. Both data sets Sol and Mov are solved by a commercial solver (Gurobi) in order to obtain optimal, and if not possible, at least good integer feasible solution. No time limit was set for the solver, therefore the
optimisation process stops when finding the optimum value, when the solver runs out of memory, or when the solver proves the instance being integer infeasible.
5.2.3.2 Overview of Results
The experiments used in total 183 instances (Sol and Mov). The solver was not able to load instances of 150 and 250 activities due to the size of the model (the amount of memory required), such instances belong to the Mov data set and in total are 10, five of each number of visits. For the remaining ones 173, Table 5.1 shows the overall results. There were four types of results among the 173 instances: instances for which the solver found optimal solutions (Optimal), instances for which the solver reported as infeasible (Infeasible), instances where the solver run out of memory (OoM) without giving any result, and instances where the solver reported errors (Error).
From Table 5.1 we observed that optimal solutions were only reported in two instances both with 25 activities. It took 67 hours for the longest one to find the optimal solution and 21 hours for the shortest one. Almost half of the instances of 25 and 50 visits run out of memory when searching for the optimal solution, nevertheless feasible solutions were found. The time by which the solver reports the first available solution is provided in parentheses. It is noticeable that the first feasible solution is reported as fast as eight seconds and maximum of four hours, mean computation time is between 10.5 (for 25 visits instances) and 5365 (for 50 visits instances) seconds. Infeasible solutions could be identified almost immediately, in less than one second after the optimisation commences, but for other cases after 45 hours.
The overall observation is that these WSRP instances are computationally difficult to solve for the mathematical solver with this IP formulation. More importantly, it is observed that for a good proportion of them no feasible solution was obtained. For example, none of the 100 activities instances was solved. It is clear that for instances of that size the solver using this model is not a good option as solution method. Figure 5.1 shows the gap reduction of an optimal instance found by the mathematical solver. The outer graph represents all time required to achieve optimality (241620 seconds). The inner graph only shows the first two hours (7200 seconds) which in this case is the amount of time required for the solver to achieve almost a 10% gap. Reducing the gap further takes the solver 65 additional hours. Hence, setting a maximum time limit of two hours for further experiments is a reasonable compromise. It should be remembered that the instances represent daily planning problems where two hours waiting for a result from the solver is a significant but manageable time.
Total Size Outcome # Instances Min(t) Mean(t) Max(t) Std Dev(t) 56 25 Infeasible 30a 0 5417.80 162340 29637.88 Optimal 2a 79072 160346.00 241620 114938.79 (8) (10.50) (13) (3.53) OoM 21a 38430 117198.48 348129 75519.83 (188) (2329.08) (9293) (2599.13) Error 3a - - - - 61 50 Infeasible 31a 0 24.29 331 73.49 Optimal 0a - - - - OoM 26a,b 1651 67294.31 269150 61362.68 (63) (5365.00) (14898) (8273.17) Error 4a - - - - 56 100 Infeasible 53a 0 55.64 123 32.27 Optimal 0a - - - - OoM 0a - - - - Error 3a - - - -
Table 5.1: Summary of the outcome infeasible, optimal, out of memory (OoM) reported by Gurobi. Computation times t (Minimum, Mean, Maximum, Standard Deviation) are in seconds. Times in parenthesis show when the solver found the first feasible solution when available. a Sol data set. b Mov data set.
In the next round of experiments we concentrate in the 25 and 50 visits size instances.
5.2.3.3 Tackling the second objective
Two of the constraints, 5.8 and 5.9, are included to tackle the case when activities require more than one employee and when activities have time-dependent relation- ships. It could be argued that the presence of these two additional type of constraints could make the search easier since the search space is reduced. Or, it might be the case that introducing the constraints makes the problem computationally harder to search because, although the search space might be reduced, it might also be divided in small separated feasible areas, and finding those areas could be more difficult. A way of testing whether Teaming and time-dependent activities constraints make the search easier or more difficult for the solver is to use the same model with some in- stances that include those constraints and other instances that do not include them and compare results.
0
0.5
1
1.5
2
2.5
·10
50
0.1
0.2
0.3
0.4
0
2,000
4,000
6,000
0.1
0.2
0.3
0.4
Time in seconds
R
ep
or
te
d
G
AP
Optimal instance solver performance
Figure 5.1: Gap reduction as computation time progresses in a case in which the math- ematical solver found the optimal solution. The optimal solution is reported after 241620 seconds (approximately 67 hours) but a considerable gap reduction is achieved during the first two hours, as shown in the close up
5.2.3.4 Experiment Design
The experiments use only the instances of activities size of 25 and 50 within the Sol (112 instances) data set. Two runs of experiments are performed. The first run has no changes of the instances. In the second run, all teaming and time-dependent activities constraints are removed from the instances. For both runs of experiments the same IP model, computer and mathematical solver settings are used. The time limit set was 15 minutes. This time limit differs from the suggested two hours from the first objective as the purpose is to see the effect of having time-dependent activities or not and not to find the optimum value. The same objective function is used (5.1). At the end of both experiment runs a comparison is made to identify which group of instances the solver achieves better results. Such comparison might provide guidance in answering whether teaming and time-dependent activities constraints make the search more difficult or not.
5.2.3.5 Experimental Results
To facilitate the report of experiments we divided the instances according to the original Solomon groups, i.e. C100, C200, R100, R200, RC100, RC200. Table 5.2 shows the number of instances in each group for which the solver found a feasible solution. The solver found more feasible solutions in all groups where the version was without Teaming and Time-dependent Activities Constraints (TTC). In total there were 59 feasible solutions compared with 26 for the version with TTC constraints.
TTC Constraints C100 C200 R100 R200 RC100 RC200 Total (18) (16) (24) (22) (16) (16) (112)
With TTC 10 8 0 4 0 4 26
Without TTCs 13 13 3 12 8 10 59
Table 5.2: Number of feasible solutions found in every group. Values in parentheses indi- cate the number of instances per group. Teaming and time-dependent activities constraints (TTC)
The version without the teaming and time-dependent constraints finds more feasible solutions (59) compared to the version that includes them (26). Although better results are obtained the version without constraints still remains a difficult problem for the solver since just 59 out of 112 instances could be solved. Table B.1 (see Appendix B section B.1) provides details of the results of each instance with TTC constraints. The results include best objective, lower bound found, and the respective gap. It also shows whether the solver identifies the instance as infeasible. The solver reported overall: 26 instances with feasible solutions, out of which seven were optimal ones; seven instances were infeasible; and, for the remaining 79 instances, the solver time out without reporting any feasible solution. Similarly, Table B.2 (see Appendix B section B.2) provides detail of individual instances for the version without TTC constraints. The solver found feasible solutions for 59 instances, out of which 15 were optimal. There were two infeasible instances. And, for the remaining 51 instances the solver timed out without reporting any feasible solutions. There are 26 instances that have feasible values for both experiments (with and without TTC), the solver obtains the same results for seven of them. These seven are the optimal ones reported with TTC. For the rest, the version without TTC achieves a better gap. A series of Figures showing the reduction in gap as computation time progresses for both experiments runs is included in Appendix B for results with TTC refer to B.1, B.2, B.3, B.4 for results without TTC refer to B.5, B.6, B.7, B.8, B.9, B.10.
Regarding infeasible results in both experiments, two instances are infeasible due to the lack of employees to cover all activities, whether because there are not enough
or they do not have the required skills. The remaining five infeasible instances are due to conflicts when introducing TTC, particularly those constraints that require the simultaneous performance of two different activities, because they require two or more employees available at a specific time.
5.2.3.6 Review of the First objective: Varying time limit
In the experiments of section 5.2.3.2 there was no limit in computation time, in the hope that the solver could eventually find optimal solutions. In this section, three additional time limits are used for instances with 25 and 50 visits. The time limits are 15 minutes, 60 minutes and 240 minutes. The need for repeating the experiments with an specific time limit was to obtain more information regarding instances that run out of memory in the previous section. 5.2.3.2.
Results from section 5.2.3.5 for instances with TTC with time limit of 15 minutes are re-used here. There are 79 instances for which the solver timed out after 15 minutes providing no result, for those instances only the time limit is increased to an hour. Table B.3 (see Appendix B section B.3) shows detailed results of individual instances. The solver only found three instances with new feasible solutions. Figure B.11 (Appendix B) shows the gap reduction of the three instances, notice how the x-axis starts around 2000 seconds.
There are still 76 instances for which the solver does not provide any information apart from the lower bound. A third increase in time limit is performed (240 minutes). The number of minutes is chosen to maintain the same ratio (four times) as for the previous two set of experiments (15 to 60) and (60 to 240). Table B.4 (see Appendix B section B.4) contains detailed information for each instance. The solver found 16 new feasible solutions. The new results belong to all groups except R100. Figures B.12, B.13 and B.14 also in Appendix B illustrate the gap reduction for the 16 instances. Among all groups R100 contains instances with shorter time horizon with respect to the other groups. The duration of the time horizon is equivalent to employees working time. In other words, instances in R100 have the same work with less resources (employees- hours). For the same reason six out of seven infeasible instances belong to that group R100.
5.2.3.7 Tackling the Third objective: Gurobi results
Gurobi provides information regarding the current gap achieved while performing the optimisation. In the experiments, Gurobi is set up to report the gap reduction every 15 seconds. When a gap reduction is achieved, the method used is reported by the solver. The objective in this set of experiments is to identify which method is used by Gurobi when finding better solutions for each instance. For every new feasible solution Gurobi reports whether the solution was found by branching or by heuristics as specified in the reference manual of the solver Inc. (2013). If most of the time new feasible solutions are found by heuristics, it would justify developing a tailored one for WSRP. In all previous experiments, without exemption Gurobi found more gap improvements when using heuristics. It is expected that MIP heuristics find more feasible solutions than the branching process for the VRPTW. The adaptations to the data set and modification of the VRPTW model to tackle WSRP have similar results. In fact, the number of times a heuristic within Gurobi finds a better solution is in general larger for instances that include the additional constraints in the WSRP instances.
Table 5.3 summarises the number of times a gap reduction was achieved for every group of instances in all experiments. The table has four rows but split in two parts vertically, each part has three groups of instances. Note that the second row in each part, marked with (*), refers to all instances without the teaming and time-dependent activities constraints. The third row in each part shows the 79 instances that timed out after 15 minutes in the first set of experiments but then executed for up to 60 minutes. The number in parentheses after the time limit is the number of instances used in that set of experiments. In all groups there are more gap reductions achieved by heuristics than by branching (H/B values).