Multiple Independent Replications in Parallel

The estimators based on the method of independent replications (IR) can be adapted for applications in MRIP parallel simulations. In this applica- tion of IR in parallel simulation, each processor simulates multiple indepen- dent realisations of the stochastic simulation process [HEID86], [GLYN91], [GLYN91b], [REGO92], [HEID88], [SUND91]. The results of these runs are then averaged together. We will call such a modication of IR the method of

parallelized independent replications (PIR). Let us note that in this case, the simulation is replicated in two dimensions: we have the freedom in selecting the number of processors, and each of them executes a dierent (random) number of replications until a given stopping criterion is met. A simulation executing according to the PIR method is stopped when the global point estimator = XPIR(P) = 1P XP j=1 0 @1 kj kj X i=1 Yij 1 A (4.1)

reaches the stopping criterion for the rst time. Here Yij is the mean over

kj replications executed by processor j. The basic properties of this esti-

mator were discussed in [HEID86]. Taking into account the random size of replications, another PIR estimator of the mean, formulated after the ratio estimator of the sample mean used in regenerative simulation, was proposed in [HEID86]; see also [GLYN92]. Namely they assumed

= X PIR(P) = P Pj=1 1 kj Pkj i=1Yij P Pj=1kj : (4.2)

Theoretically, PIR estimators are statistically more ecient than IR es- timators used in SRIP (i.e. when multiple processors cooperate on a single realisation of a simulated process), in the sense that PIR produces nal es- timates of sample means with smaller mean squared error, if the problem of the initialisation bias is properly solved. Properties of =

XPIR (P) and

=

X

PIR (P) have been studied empirically in [GLYN92], where the coverage of

their non-sequential versions was analysed, assuming the same deterministic duration of the initial transient period and the same time of each replication,

for a dierent number of processors. It was shown that in such applications the estimator =

X

PIR (P) can yield a much better coverage than

=

XPIR (P),

and its quality improves when the degree of parallelism increases, i.e. with an increasing number of processors. In [GLYN89a] and [GLYN89b] some asymptotic properties of these estimators were studied, but their full analy- sis has yet to be done, especially in fully sequential versions of steady-state simulations.

The PIR estimators for MRIP simulation can potentially be applied to any model and is suitable for execution on multiprocessors as well as multi- computer networks. Fig. 4.1 illustrates the behaviour of PIR simulation and compares that with the sequential independent replications simualtion technique. In Fig. 4.1a we see that with classical independent replications, k runs (replications) are made, each run using a dierent random number stream. The means of the parameter of interest of each run are independent, therefore classical statistical techniques can be applied to thek means to form a point estimate, and an estimate of the precision of results. Fig. 4.1b shows that the Parallel Independent Replications case is the same as the sequential simulation, except that several processors, P1 to PP, are used simultaneously

when making replicated runs.

PIR simulation is well suited for user transparent implementation. As an example, the EcliPse [REGO92], [REGO91], SUND91] system was spe- cially developed for user transparent, and portable PIR simulation. Eclipse included monitors for combining an equal number of samples from each repli- cated instance, or for a xed number of samples (hence a random comple- tion time, or a random total number of samples). It was also discussed in [SUND91] that EclipSe may also be appropriate for batch means and regen- erative simulation, although a batch means analysis procedure, and one for batch size determination is not provided.

There is also a possibility of using other IR-type estimators in MRIP scenario. For example, one could think about adapting estimators proposed in [HEID88] and [GLYN91] for estimating transient quantities.

Notwithstanding, the question on the choice of a proper replication size needs to be answered. The lack of a method for choosing an appropriate repli- cation size may limit applications of PIR, since an inappropriate replication size can result in very biased estimates [PAWL90]. The main weakness of

Figure 4.1: Virtual-Real time diagram fo the method of independent replica- tions executed on (a) a single processor, and (b) on P Processors (concurrent simulation in MRIP scenario). Pi denotes the i-th processor.

PIR methods is that they suer from greater overhead because they lead to discarding more initial observations, as each replication must go through its own warm-up period. The number of initial data sets that has to be deleted grows linearly with the number of replications. For example in a 10 processor system where each is used to make 10 replications, 100 times the number of observations in an average initial transient period must be discarded. This situation suggested to us a novel scenario for MRIP that would be a parallel generalisation of the SA/HW method of sequential estimation : The method of Spectral Analysis in Parallel Time Streams.