SPC method

The SPC method described by Robinson (2002, 2007), and implemented by Hoad et al. (2009), uses common process control methods to identify if a time series is in steady state based on the assumption that the warm up period section of the output data is considered to be ‘out of control’ due to variation. The following section describes how this method was implemented. It is based on an algorithm described by Robinson (2002, 2007), with key changes made to how the data is analysed for serial correlation and normality. An outline summary of the method is given as follows:

1. Perform replications and collect data.

2. Determine the appropriate batch means using: - the Von Neumann test for serial correlation, - the Anderson Darling test for normality.

3. Construct control parameters and identify steady state.

Perform replications and collect data

The length of the simulation run is first calculated using the Whitt approximation, as discussed in Section 3.3, using a black-box fit of an appropriate queueing model for the system. Then five initial replications are performed and the data is collected. To ensure that a sufficient number of replications are performed, the algorithm discussed in Section 3.2 is used to identify the recommended number of runs. If there are i = 1, 2, 3, . . . , m observations collected from each replication. The data is then averaged across the replications, which gives the time series of the average sample means for all replications performed as ¯y = ( ¯Y1, ¯Y2, ¯Y3, . . . , ¯Ym)

Determining the appropriate batch means

The next step involves creating a batch means of the data series to reduce or remove the autocorrelation of the time series. Autocorrelation (also known as serial correlation)

happens as a consequence of monitoring observations in a time series data from simulation output. Any observation collected is somewhat affected by the previous observation. The aim is to reduce this correlation as much as possible. The process of averaging across replications, as in Section 3.4.1, helps to remove some of the serial correlation, and grouping observations into batches aids in further reducing it. This method is known as the batch means method and was popularised by Law and Carson (1979); Law and Kelton (1997).

This method divides the (possibly correlated) time series ¯y = ( ¯Y1, ¯Y2, ¯Y3, . . . , ¯Ym) into

a number of batches h, where the batch size is k = bm/hc. Then, each batch mean is,

¯ ¯ Yi = hk P i=(h−1)k+1 ¯ Yi k , (3.9)

for h = 1, 2, . . . , bm/kc, giving the vector ¯y = ( ¯¯ Y¯1, ¯Y¯2, ¯Y¯3, . . . , ¯Y¯h). A coded implementa-

tion of this method can be found in Appendix A.5.2.

A technique proposed by Fishman (1978) and used subsequently by Hoad et al. (2009) and Robinson (2002, 2005) is used to locate the appropriate batch size. The initial batch size k is set to 2, the batch means are formed and tested for autocorrelation and normality using the Von Neumann and Anderson-Darling test, respectively. If the data ‘fail’ either test, i.e., data is correlated or not normal, tests that are required to satisfy the assumptions of the SPC method, then the batch size is increased twofold. These tests are carried out repeatedly and any doubling of the batch size is performed until both tests pass, meaning that the data is not correlated and is normal, or the number of batches has reached a critical lower limit (h > 20). The next step is then to test the midway point between the ‘failed’ batch size and the ‘passed’ batch size. This process is repeated until the lowest batch size is found that passes both test. A coded implementation of this algorithm is included in Appendix A.3.

Identifying steady state

A time series trend or transient can be considered as ‘in control’ as long as a number of control parameters are not violated. Three sets of control parameters can then be

constructed UL3, UL2, UL1, LL1, LL2, LL3.

Beginning at the end of the time series and working backwards towards the start, the series is assumed to fail at the first point where any of the following four control limits fail;

Test 1: A point plots below LL3 or above UL3,

Test 2: Two out of three consecutive points plot below LL2 or above UL2,

Test 3: Four out of five consecutive points plot below LL1 or above UL1,

Test 4: Eight consecutive points plot on the same side of the mean µ.

The simulation run length must be at least twice the length of the warm up period, a recommendation given by Kelton (1980). Iterating through each batch mean, if any of the batch means fail in the last half of the series, then it is assumed that steady state was reached in the latter half of the simulation runs and the run length was insufficient. If the test fails in the first half of the series, then the first point where it failed (working backwards) is considered as the point where steady state is assumed to have begun, and any data before this is disregarded.

Another scenario occurs if the time series is ‘in control’ from start to finish, this often happens when the batch sizes are high and much of the initial transient is captured in the first batch. In this case, steady state is assumed to have begun when the time series first crosses the mean line.

An example is shown in Fig. 3.7 for an M/M/1 queue with te = 1 hour. At a particular

design point, the time batched transient ¯Y¯i of the performance indicator (cycle time) with

mean µ is plotted with the control limits. Working backwards at ¯Y¯53 the test results are

given by Table 3.2. Test 1 fails at ¯Y¯36because this batch mean plots above UL3. However,

0 0.5 1 1.5 2 2.5 0 5 10 15 20 25 30 35 40 45 50 cycle time (hrs) batched series

failed test 1 (outside UL₃)

warm up period steady state transient mean (µ) UL₃ UL₂ UL₁ LL₁ LL₂ LL₃

Figure 3.7: Batched time series transient and SPC control parameters for an M/M/1 queue, showing failure of Test 1 at ¯Y¯36.

point along the transient. Therefore, it is assumed that the series is in control after ¯Y¯36,

which is in the latter half of the time series (i.e., ¯Y¯36 >

j_¯¯ Y53

2 k

= ¯Y¯26) meaning that the

run length is deemed insufficient and the experiment should be rerun with twice the run length.

Table 3.2: SPC test results for M/M/1 queueing system.

SPC Tests Result

Test 1 Fails at ¯Y¯36

Test 2 Pass

Test 3 Pass

Test 4 Pass

to be the point where the transient crosses the mean line the first time. In this example,

that point would be at ¯Y¯7. The code for this algorithm is included in Appendix A.4