Chapter 6 Model Experimentation
6.3 Experimental Design Methods
When the arduous process of building, verifying and validating a simulation model is completed, it is time to have the model work for you; and one extremely effective way of accomplishing this task is to apply experimental design methods to help explore the model (Sanchez 2007). In this study, two formal experimental design methods are adopted: fractional factorial design and response surface design.
6.3.1 Fractional Factorial Design
The most commonly applied experimental design method appears to be full2k
factorial design. According to the method, all k experimental factors will be set at two levels (i.e., the base level and the alternative level), and a full combination of the k experimental factors with two levels will result 2k design points in total. Normally
multiple replication runs are performed at each design point, and mean model responses are estimated. Standard statistical techniques exist to analyse the responses and calculate the main effect of each experimental factor as well as arbitrary higher- order interaction effects among the factors.
The main effect of each experimental factor measures the average change of the model response due to a change of the factor value from its base to the alternative level, considering all possible combinations of the other factors. In this study, the main effect of each experimental factor indicates the average effectiveness of the intervention policy or the average sensitivity of the influencing factor, considering all possible combinations of the rest of the experimental factors. The two-way interaction effect measures the level of interactions between two factors, i.e., the effect of one factor depends in some way on the level of the other factor. The two-way interaction effect between factor i and factor j is the difference between the average effect of factor i when factor j is at its base level and the average effect of factor i when factor j is at its alternative value. In this study, the two-way interaction effect may disclose the
Model Experimentation and Analysis
Given a limited number of experimental factors, a full 2kdesign has relatively few
design points, is easy to implement, and allows you to examine any higher-order interactions without confounding among factors.
One assumption of the factorial design is that each experimental factor only has two levels tested: base and alternative level. Therefore, it is not possible to explore the factor on more than two levels and linearity is implicitly assumed between the experimental factor and model responses, and between different experimental factors. Another drawback of the method is that when the number of experimental factors increases, the number of total design points increases exponentially and will soon become intractable.
The reason to use the fractional factorial design (with resolution V), rather than a full factorial design, is to reduce the number of design points and subsequently the computational time and effort. A full factorial design with eight experimental factors (the number of experimental factors in this study) needs 256 design points (28 =256),
each representing a unique combination of the base/alternative levels of all eight factors. Like model validation, for each design point, the simulation model will run 500 replications to estimate the mean model responses; and each simulation run will last 415 days (365 days for data collection preceded by 50 day warm-up period). This implies a total of 128,000 (256×500=128000) runs are required for a full factorial design experiment. The fractional factorial design can significantly reduce the number of design points; and the resolution V can guarantee that not only main effects but also the two-way interaction effects can be estimated without confounding with each other (Law 2007).
Table 6.2 gives the definition of resolution III, IV, and V in 2k−p fractional factorial
design according to Law (2007). Standard techniques exist to construct a 2k−p design
matrix and estimate the main and non-confounding two-way interaction effects (if the resolution allows) of the experimental factors (Law 2007).
Model Experimentation and Analysis
Table 6.2 Definition of resolution III, IV and V of fractional factorial design (source: Law 2007)
Resolution Definition
III No main effect is confounded with any other main effect, but main effects are confounded with two-way interactions and some two-way interactions may be confounded with each other.
IV No main effect is confounded with any other main effect or with any two-way interaction, but two-way interactions are confounded with each other.
V No main effect or two-way interaction is confounded with any other main effect or two-way interactions.
With eight experimental factors, fractional factorial design with resolution V needs 64 design points. According to the rules for determining the combinations of experimental factors in each design point, the design matrix is constructed (see the first ten columns of Table 6.4). During model experimentation, the model responses of each design point are estimated based on 500 replications. Since multiple replications are performed for each design point, the model responses may be used not only to estimate the mean main and two-way interaction effects, but also the distribution and confidence interval for each main and two-way interaction effect.
6.3.2 Response Surface Design
Compared to factorial design, response surface design can test the experimental factor on many different levels, so that non-linear and more detailed relationships can be revealed. A response surface design tests two experimental factors at a time while keeping all other experimental factors and input parameters at default values.
If the first experimental factor has n levels and the second factor has m levels to be tested, then the total number of design points for the response surface is the product of n and m (i.e.,
n×m
). For each design point, the model response will be estimated based on 500 replications and a three-dimensional diagram will be constructed to show the response surface with the two horizontal axes representing the levels of the two experimental factors and the vertical axis representing the corresponding mean model responses. The detailed and potential non-linear relationships between each of the two experimental factors and the model response and the potential non-linearModel Experimentation and Analysis
The drawback of response surface design is that it needs a large number of design points, especially if more levels of each factor are tested. Another drawback of the design is that it only tests two factors at a time while the values of other potential experimental factors need to be fixed. This assumption means response surface design can not reveal the potential interactions between the two factors tested and other experimental factors.
Following the fractional factorial design, the response surface design will be carried out on those pairs of experimental factors that both demonstrate significant two-way interaction effects in the factorial design and have potential practical implications on MRSA control and prevention. Experimental design itself is a vast research area in the operational research and statistics domain. Detailed discussion of experimental design methods (including factorial design, response surface design and other experimental design methods) can be found in the relevant subject literature (Chapter 9 of Kleijnen and Groenendaal 1992; Kleijnen et al. 2005; Chapter 12 of Law 2007).