Guidance for Implementers

When developing an MCMC algorithm, readers are advised to incorporate the par- allelisation methods described in this document into their planning and designs from the outset, as some of the parallelisation and optimisation techniques place constraints on the MCMC implementation. (for example, it must be possible to cal- culate what the prior and likelihood of a state would be after some proposed move is applied to it, without actually applying the move or making any other modifica- tion to the existing state - see page 64). Whilst image-splitting based techniques may easily be adapted to a MCMC implementation through the addition of pre and post processing operations, speculative moves and speculative chains have very specific requirements. For instance, it must be possible to treat potential moves as discrete objects whose processing and/or application may be deferred or prevented, and that can be evaluated without any modification (even temporary) of the model representing the MCMC chain’s state.

To avoid future scientists reinventing the wheel, as the example applica- tions featured in this document were constructed and tested the parallelisation, core

MCMC mechanics, boilerplate, performance measuring and recording functions were abstracted out into a extensible framework called pMCMC, which is documented in appendix A. pMCMC automates the implementation of the parallel MCMC meth- ods described in chapters 3 and 4 and some of those in chapter 5. It also provides (through static code and autogeneration) most input/output functionality that is required for a basic MCMC application, including for the handling of all the prior and likelihood that are needed for determining the posterior probability, the move proposal probabilities, and any ‘tweak’ values that can be used to fine-tune the ex- ecution of the parallelisation. This allows developers using the pMCMC framework to focus on the specifics of the MCMC algorithm relating to the program’s applica- tion - the specification of the simulation state and the calculation of the prior and likelihood probabilities. Appendix B shows an example implementation of one of the section 2.5 circle-finding algorithms and appendix C demonstrates the runtime usage of this application.

When faced with either implementing a new MCMC application or adapting an existing one, the first matter to consider is whether some form of image splitting is applicable to the application and datasets that are expected to be processed, and whether image splitting is acceptable considering the purpose to which the resultant models will be put. Depending on the form of image splitting employed and the information that can be gathered about the input data before employing MCMC, image splitting may or may not introduce anomalies or result in uneven matching throughout the dataset. In almost all cases, image splitting will not result in statistically ‘pure’ results though there is the potential for substantial reductions in runtime, even with a limited number of processors. In many cases the only way to determine if the potential loss of reliability and accuracy is worth the reduced runtime is to try it and see.

When it comes to the actual MCMC code, there will be considerable variation in the types and behaviours of various MCMC simulations, and different paralleli-

sation methods will yield significantly different results depending on the properties of the simulation and hardware in question. If the MCMC algorithm has been implemented using the pMCMC framework, determining the appropriate parallel processing methods to employ is simply a matter of trying each of the paralleli- sation options in turn and observing the results (the parallel settings to use are specified either on the command line or in the XMLjobfile defining the simulation

to run). If pMCMC is not being used, a selective approach to which parallelisa- tion methods are implemented is desirable to avoid unnecessary work. Analyse the workings of each move type and determine the relative time spend in the prior cal- culations, likelihood calculations, and move proposal construction/application. If the bulk of the time is spent in the prior and likelihood calculations and the time for the prior calculation is ≈ the time for the likelihood calculation, consider per- forming the calculations for the prior and likelihood in parallel as this can halve the time spent calculating them. Unless a high proportion of iterations are accepted, speculative moves will provide a reliable and predictable performance improvement (see section 3.3 for best-case estimates based upon move rejection rate). Speculative chains should be implemented only if there are move types whose proposals involve substantially more processing time than the other move types (see section 4.4 for an idea of how significant the differences in move processing time should be before considering speculative chains). Periodic parallelisation will yield good results so long as there are have large datasets containing relatively small features that can be modified with localised moves. Unlike image splitting, periodic parallelisation is statistically sound (given sufficiently frequent phase switching) but in general slower than the direct image splitting methods. If image splitting has been deemed acceptable, employing periodic parallelisation as well would be redundant.