Implementation 77

The designed algorithms are tested by using random generated test data. However, before applying them in reality some implementation recommendations and requirements are given. In this section those recommendations and requirements are discussed. The key issues relate to face validation of the tested scheduling engine (Section 6.2.1.) and the settings of the priority values (Section 6.2.2.).

6.2.1. Validation and benchmarking

Although the algorithm and their result are tested for face validity (see Section 5.3.1.), actual face validity must be created during the implementation phase of the automated scheduling tool. Face validity is a result of testing and analysing data gathered under real life circumstances. As a part of the development phase, the scheduling engine must be put in practice and feedback from its first user should be gathered. Gathering and processing the feedback is an iterative loop during the development phase. Once the performance of the scheduling engine is satisfying, the scheduling engine can be prepared for real implementation. However the feedback loop never ends, as changes in the process or branch may occur. In order to deliver good results (i.e., useful schedules) the scheduling engine should keep up with customer and market developments. Figure 6.1 gives the different phases in the development of new systems. This research gives direction to the design. Before the scheduling engine is ready to be developed, face validity must be created for the application of the scheduling engine. This must be done for each individual company.

Figure 6.1 Systems development life cycle (Sparkdawn)

In addition to (individual) face validity benchmarking is recommended. Benchmarking is a systematic comparison of (organizational) processes and performances within the same niche or branch. The general process and their belonging characteristics are discussed in Chapter 2. However, the demarcation made there should also be verified. Verification is concerned with the quality of the provided solution (i.e., does the solution meets the specifications?), validation is concerned with the usefulness of the provided solution (i.e., does the specification meet the customer needs?). (de Graaf, 2014) Together verification and validation are used to demonstrate that the given solution meets the requirements and needs of the customers, and thus the solution fits within the solution space.

Benchmarking focuses on verification issues, and therefore the focus is not primarily on collecting and analysing data, rather the focus is on the thoughts behind the system. The thought behind the system are likely to be similar within the same niche, as the same key characteristics are involved. Reliability and

Plan Analyse Design Develop Implement Maintain

representativeness are therefore two important elements in benchmarking. However, in order to compare individuals within the same niche one should be aware to compare the same issues. Therefore definitions of what is measured/analysed should be clearly defined. Newminds’ position as a solution provider to multiple customers within the same branch is a good starting point for initializing benchmarking. From their position they are able to compare the process and belonging main characteristics of the scheduling process as practiced within a certain niche of branch.

6.2.2. Sensitivity analysis on priority values

While testing the algorithms on random data, the priority values assigned to the distinct factors are similar during each run because otherwise the results are non-comparable. However, the values of those factors are set based on instinct rather that they are well substantiated. The initial priority values are adapted while testing in order to create face validity, however customers might require different setting. The priority value assigned to the distinct factors will differ per company, dependent on their priorities and the interrelations between the different factors. Some factors relate to one another, therefore their value is also important. For example, the value of a preferred resource depends on others, as the preference of one resource does not say that resource must be assigned to that particular event. The consideration whether to assign the preferred resource depends for example on the required additional travel distance. If the preferred resource is close (e.g., less than 20 km additional travel distance compared to other resources) one would assign the preferred resource. In this case, the priority value assigned to the preference of resource therefore depends on the priority value assigned to travel distance (per travel distance unit).

The dependencies between those factors are company dependent. Each company that uses the scheduling engine should set their own priority values and validate the scheduling engine performance. However, they should be aware of the fact that one factor can overrule the entire system. This is the case when the different priority values are not balanced. In that case the system becomes more sensitive for the appearance of certain characteristics than others. In addition to verification (i.e., benchmarking) and validation also sensitivity analysis on the priority values is recommended.

Setting the priority values within a company can initially best be based on a combination of gut feeling and experience. However, many companies have similar settings. For example, almost any company will prioritize urgent events. Therefore the priority value of such events must be higher compared to others. Which characteristics to favour are often known within the company, and so they can initially be rated. With the initial settings the company should test the automated GP for face validity. The priority values create a priority rule that is specific for each company. Each individual company should be aware that the set priority values are hard restrictions, and so all decisions are made on those values. Although the algorithms can be expanded with many characteristics, it is advised to limit the number of characteristics at first. While putting the automated GP in practice, the priority values can recurrent be adjusted and missing characteristics can be added.