VALIDATION AND VERIFICATION

In simulation modelling, the models developed and used in mimicking real life systems or problems need to be verified and validated. Model verification and validation needs to be conducted throughout the modelling phase and the experimental phase of simulation. According to Sargent (2010 p. 166), model

66

verification is defined as “ensuring that the computer program of the computerized model and its implementation are correct” while Cimino et al. (2010 p. 6) adopted the following definition for model validation; “the process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended use of the model”. The conceptual model, which is a mathematical and/or logical model has to be an accurate representation of the real system being studied, hence validation is done at this stage to ensure this. Once this is done, the computerisation aspect of the modelling process has to be verified to assure that the conversion of the conceptual model into a computer model and its implementation is correct (Sargent 2010). Lastly, during the experimentation phase the output of the simulation experiment has to be validated as well to ensure the result exist within an acceptable range of accuracy and to assure the decision maker or users of the information of the correctness of the model for the intended use. These validation and verification needs to be done to assure the integrity and credibility of the simulation model in addressing the objective of the real life system. One commonly used method for validation is comparison with other models that have been validated in literature.

3.7.1 Conceptual Model Validation

The conceptual model used this study has already been validated by previous work in literature. Several authors have used the same representation of the three ordering policies used in this study. Option I has been used by Cimino et al. (2010), Chatfield et al. (2004), and (Agrawal et al., 2009, Bensoussan et al., 2007, Beamon and Chen, 2001, Chen et al., 2000). Option II was developed by (Axsäter, 1996) and option III has been used by Lau et al. (2002). The mathematical representation of the levels of integration used in this study was developed and validated by Lau et al. (2002) and Lau et al. (2004). Validating the conceptualisation of the supply chain structure is rather simple. The three supply chain structures are simplification (WH and MF) and networking strategies (NT) being considered as an alternative to the serial type structure. The structures are merely a reduction in the number of agents within a tier or simply a splitting of orders coming from downstream agents. Reducing the number of agents in each tier translate into a single agent (wholesaler in WH structure or manufacturer in MF structure) aggregating and fulfilling the orders from downstream agents. These type of structures have been studied in the past, however,

67

under different circumstances. The splitting of order is conceptualised in this study as equal sharing between agents in the same tier. While it is possible that equal splitting may not be agreed to by members in many supply chains, the point is that the proportion of split can be made based on the size and level of commitment of the supply agents. In this study, the assumption is that all supply agents are of equal size and the level of commitment to the chain is the same, which is the case for many supply chains.

The supply chain parameters and assumptions used in this study are similar to the ones used in Lau et al (2002) and Lau et al. (2004) and the result of the study is used to validate the output in this study. Here is a summary of all the main assumptions made in the supply chain

 Demand is normally distributed with mean of 10 quantities and a standard deviation of 2

 All the lead times are constant.

 All members of the supply chain use the same ordering policy

 If on-order quantity cannot be met with current on hand inventory, then the on-hand inventory is shipped and the rest is back ordered leaving the agent with zero inventories.

 Each unfulfilled order is backordered and a shortage or back log cost is incurred per unit item including a fixed shortage cost once an order is unfilled or partly filled

 The performance of each tier is seen as an average of the performance of all the agents within that tier.

 Agents in the same echelon use the same sharing mode.

 The total production capacity at the manufacturer tier is equal to 80 equally split between all manufacturers.

 A unit of production capacity makes a unit of the product for the duration of the production lead time.

 The manufacturer has an unlimited and unfettered supply of raw materials. 3.7.2 Computer Model Verification

The programmed model is verified to ensure the implementation of the codes is correct. The Java Development Kit (JDK) includes development tools such as Java

68

compiler, Javadoc, Jar and a debugger. The debugger is an excellent tool that aids the user in determining programming errors which might affect the output or running of the simulation model. In other words it is a useful verification tool (Kleijnen 1995) that helps to ensure (in part) the integrity of the written programme. Each time there is an error in coding, the java compiler registers an error and the programme will not be executed until the error is found and fixed. On the other hand, a very useful and widely used verification method is structured walkthrough or traces. According to Sargent (2010) traces is defined as following (tracing) the behaviours of different types of specific entities through the model to determine if the model’s logic is correct and if the necessary accuracy is obtained. In this study, each of the 240 scenarios is run for a simulation time of 10 days and the predicted result (obtained by manual calculation) is compared with the simulation output. This confirmed that the computerised model was properly implemented using the java programming language.

3.7.3 Experimental Output Validation

Experimental output validation is also known as operational validation. The aim is to check if the output of the computerised model exists within the level of accuracy required for the model’s intended purpose over the domain of the model’s intended applicability. To ensure this accuracy, during the experimentation phase the simulation warm up period was determined and the output was computed over the effective simulation period which is the total simulation time minus the warm-up period. This was done to remove the warm-up effect and ensure that the output was determined over a steady state. In addition, a confidence level (98%) was built into the result by conducting multiple replications (predetermined to be 45) for each of the 240 scenarios (making the total number of experiments 10800). The output for each scenario was averaged over the 45 replications meaning the result was computed with 98% confidence.