We have made two contributions to ABSS/ABM in this thesis: 1. An approach to improving replication in ABM;
2. The production of alternative synchronous algorithms for ABSS.
8.2.1 Improving Replication of ABM through Formal Specification
Replication is an open problem in ABM [Edmonds and Hales, 2003, Drummond, 2009]. There is an inability to compare different implementations of the same simulation due to ambiguities in the simulation definition. To allow for proper comparisons to be made between our approach and other approaches to ABM we required re- peatable results. That is, we need to be able to present our new approach in such a manner that they can be compared fairly against the alternatives.
Our solution is to use formal specifications, already widely studied in computer science, to document precise definitions of simulations. Formal methods have been used up to now mainly for safety critical systems in order to prove the correctness of the system definition and the algorithms that implement them. Up until now they have not been used to help researchers create simulations that are reproducible by other researchers. We demonstrate that they can also be used to provide a high level description or specification of an ABM. This specification can then be used to verify that different implementations of the same model conform to the same precise model definition.
We achieved this by choosing a representative ABSS, Sugarscape, that is well- known [Epstein and Axtell, 1996] and contains a wide variety of agent interactions of varying complexity and formally specifying it using the Z notation [Spivey, 1989]. This specification allows modellers who wish to compare simulation approaches to use a well defined simulation and so make proper comparisons. At present each Sugarscape simulation is a unique interpretation [Bigbee et al., 2007, Gilbert, 2014] making it difficult to fairly compare the outcomes between any two implementa- tions. Our specification has identified ambiguities in the original definitions and is specified in a manner agnostic to updating approach. Once an ABM has been spec- ified formally it solves one of the major problems of replication, that of different researchers misinterpreting the simulation definition.
The issue of whether modellers are able or willing to undertake an extensive formal specification process is not directly addressed in this thesis. However, if it is the case that modellers want the precision required to allow for replication then there are few alternatives available and none that give the same level of precision as is offered by formal specification.
Our specification, summarised briefly in Chapter 5 and more extensively in Ap- pendix C, has been made available online [Kehoe, 2015a] along with our reference implementation [Kehoe, 2015b].
This approach of using formally specification to help tackle the replication problem in ABM has been published and presented at the Multi Agent Based Sim- ulation Workshop held at AAMAS in 2016 [Kehoe, 2016a].
8.2.2 Detecting Artefacts through Multiple Updating Strategies
The use of AU within ABM [Huberman and Glance, 1993] and the attendant issues were surveyed [Cornforth et al., 2005], [Caron-Lormier et al., 2008]. Our analysis shows that AU can, especially in higher density populations, allow agents to gain
non-local knowledge instantaneously - something not allowed within ABM. This allows us to explain some of the differences in outcomes between synchronous and asynchronous updating already identified by other researchers. These are :
1. Higher density populations in a simulation where bounded rationality is im- portant will result in bigger differences between the asynchronous and syn- chronous outcomes [Caron-Lormier et al., 2008] than lower density popula- tions in the same simulations;
2. Asynchronous updating will cause simulations to be less likely to display periodicity [May, 1973].
The identification of these issues provides guidance for modellers choosing between an asynchronous or synchronous approach to simulation design.
Following on from our study of these updating techniques we developed al- gorithms that allow SU to be extended to ABM. Our algorithms handle all types of interaction, even the most complex, thus increasing the scope of synchronous updating from simple CA-based simulations to the more complex ABSS.
We proved, using asymptotic analysis that our algorithms belong to the n log n complexity class. Although this is slightly worse than asynchronous updating which is linearly bound, n, it is still polynomially bound - a time complexity that is generally considered acceptable. We also established that the algorithms demonstrate good Weak Scalability and are Deadlock and Livelock free. These SU algorithms are deterministic in execution, further differentiating them from the alternative AU algorithms.
Finally, we produced a framework for our algorithms to show their practicality. This framework serves as a starting point for any researchers thinking of using a synchronous approach in an ABM. We used our framework to implement a cross section of Sugarscape rules (or behaviours) ensuring that we implemented at least one rule belonging to each category or type of interaction. We verified that they could all be implemented synchronously and with empirical testing (summarised in Appendix A) confirmed that they conform to our asymptotic analysis.
Using this framework this thesis demonstrated that Sugarscape is robust to the updating strategy employed, something that was unproven until now. The emergent properties of Sugarscape were shown to be independent of the updating strategy employed. In all cases the results show little divergence and identical emergent properties. This is a new finding as until now the use of SU within ABM has been
rare, due to the lack of any useful SU algorithms that can handle ABM interactions. The comparisons shown here have previously only been attempted for simple (CA- based) simulations that do not contain the complex types of interactions present in an ABM such as Sugarscape.
The analysis in section 6.5.1 showed that CA simulations are more suspectable than ABM to instantaneous information transfer accross localities. It suggests that the major differences that have been shown to occur in CA-based simulations under SU and AU updating strategies [Huberman and Glance, 1993] do not necessarily occur in more complex ABM. This is bourne out by the comparisons in Chapter 7. The results obtained by our framework (Chapter 7) on the use of multiple up- dating strategies in reproducibility have been published at the European Simulation and Modelling conference in 2016 [Kehoe, 2016b].
Appendix A
Empirical Benchmarks
A.1
Overview of Benchmarks
These benchmarks contain behaviours as complex (if not more so) as those dis- played in any other synchronous algorithms from the literature. We have shown that we can implement complex behaviours, their time complexity matches the ex- pectations from the asymptotic analysis and that the algorithms scale as expected.
Sequential execution of our algorithms, using pseudo random numbers gives deterministic execution. In the concurrent version of our algorithms there is some nondeterminism caused by the sharing of a single random number generator amongst all threads. This can be removed by a more sophisticated approach to pseudo ran- dom number generation.
As we have already stated the framework is not optimised. This is to make the code as easy to understand as possible. When performing agent behaviours the framework iterates over all locations instead of iterating over only locations that contain agents. For this reason we are not overly concerned with the absolute speeds obtained, although they are fast, but with how it scales as (i) the size and complexity of the simulation grows and (ii) the number of simultaneous threads increases.
Every major simulation toolkit (Swarm, Repast, Mason and NetLogo) comes with a partial implementation of Sugarscape that demonstrates that toolkit’s ap-
proach to simulation [Railsback et al., 2006a, Berryman, 2008, Inchiosa and Parker, 2002]. In chapter four we produced a formal specification of Sugarscape thus making
proper analysis of our results possible and allowing other researchers to compare their own implementations to ours. If we compare Sugarscape to the StupidModel
[Railsback et al., 2005] benchmark we can see that although StupidModel has been used by some ABM researchers to show how fast their approach to implementing ABM is [Lysenko and D’Souza, 2008], its most complex behaviour only matches that of the simplest benchmark presented here. This makes the results from our framework more complete and relevant. We note in passing that the implemen- tation of even the sophisticated Sugarscape algorithms is simplified by the use of our algorithm. This is because all concurrency is handled transparently by our algorithm.