Two paradigms for simulation modelling

How do we make contributions to knowledge using simulation modelling? In this section we outline two approaches, representing different scientific paradigms (Pidd, 2004) and competing for our attention - though in practice they complement each other as the one concentrates on structuring problems while the other focuses on solving them. In one the simulation model is intended to be a representation of some aspect of a real-world system. In the other it is intended to form a representation of people’s mental models, and where more than one person is involved in its creation, it may represent a fusion of different mental models - a group consensus among various subjective viewpoints. It is this second paradigm that best serves our research objectives, so we must spell out its details carefully.

The distinction here has an origin in the philosophy of “truth” (Kirkham 1998a; 1998b). A “true” model in the first approach is one that corresponds to reality, and accuracy is the mark of success. In the second approach coherence takes priority. The success of the modelling activity depends primarily on how well the distinct viewpoints sit together encoded in the model, including whether they are logically consistent. Success is affected by whether or not a simulation model has represented each participant’s mental model accurately. But when business or decision-making success depends on coordinated efforts and group solidarity, participants may be willing to trade personal representation for consensus - or may even be willing to adjust their own viewpoints in the light of the others’ views and the model’s behaviour.

These two approaches should also be familiar from the history of Operational Research. They are represented by “hard” and “soft” OR techniques respectively

(Pidd, 2004), and sometimes characterised as “problem solving” and “problem structuring”. The motivation for the emergence of Soft OR is given in (Ackoff, 1979). Over-focus on textbook optimisation and forecasting exercises had taken academics away from OR’s world-war-two origins as the application of science to decision making of vital practical importance. Real-world situations in business and other organisations had a complexity and instability far exceeding that of textbook exercises. They were “messes”, not clear-cut problem scenarios, and they involved people as stakeholders, whose buy-in to any proposed solution was essential for its success. After a brief “Kuhnian crisis” and scientific “revolution” (Dando & Bennett, 1981), room was found for a new, “soft” paradigm alongside the old (Rosenhead & Mingers, 2001). As Mingers (1992) points out, the two paradigms reflect different philosophies of social science. Hard OR exemplifies Positivism, and aims at prediction and explanation of some objective reality. Soft OR exemplifies a

Interpretivist or hermeneutic philosophy, and aims at facilitating communication and understanding among many people with their own subjective mentalities. Mingers, following a prediction by Dando & Bennett (1981), identifies a third approach -

Critical OR - aimed at emancipation, but our focus here is on the first two. Reflections on the practice of simulation modelling in OR have concluded that it commonly sits somewhere between hard and soft (Robinson, 2001). It enjoys the rigour of hard OR through the precision of its computer modelling and the statistical analyses of its outputs, but through the discipline of “conceptual modelling” (Robinson, 2007a; 2007b) it serves as a “tool for thinking” (Pidd, 1996) among groups of stakeholders, promoting soft OR ends instead. This view of simulation modelling places it closer in use to system dynamics modelling (Sterman, 2000; Morecroft & Sterman, 1994; Morecroft, 2007).

Reality Observer / Data Collector Model Building Model V & V Experiment Intervention Prediction Model Design Theorist Steps:

1. Objective reality causes observation to be made, data to be collected.

2. Reflection on data in the light of past theory and observations leads to new theoretical understanding.

3. Theories drive the design of a conceptual model.

4. A computer model is constructed to this design.

5. The model is verified and validated. 6. Once satisfactory, experiments are

conducted with the new model.

7. On the basis of the model behaviour predictions are made concerning the real world system.

8. An attempt is made to intervene in the real world on the basis of the experience with the model.

9. Objective reality is altered by the intervention.

Figure 8 Example of a linear, positivist approach to understanding simulation modelling

In order to emphasise the reality-modelling interaction loop we have removed any other cycles.

To illustrate the difference between these two paradigms consider some diagrams (Figure 8, Figure 9). Figure 8 shows a linear, positivist approach to understanding simulation modelling. In order to emphasise the reality-modelling interaction loop we

have removed any other cycles - in real examples of this approach we would expect some repetition of steps before a particular stage was passed. (Following poor experience at verification and validation, for instance, we could alter the model construction or rethink the design.)

In addition, some steps are undertaken with awareness of the potential requirements for future steps - the model may be designed to address a question concerning options for the future intervention. Finally, some interventions alter not the objects depicted in the model but our behaviour towards them - such as when a clockwork model of the solar system leads not to attempts to play with real planets, but can affect our decisions on where to point our telescopes on future occasions. But these points aside, the flow chart illustrates how simulation models are determined by the real-world system they are supposed to represent, and how they can be used to make predictions, answer what-if questions and guide actions concerning those same systems.

The mark of a good model here seems to be correspondence between model behaviour and reality. This correspondence is believed to be measured when we ask in the light of later observations:

• Did our model-inspired predictions come to pass?

• Did our attempted interventions result in the changes we expected?

We might wonder how easy this correspondence is to achieve when the reality in question involves people capable of reacting to the very processes of being observed and modelled. Attempts to capture their reactability in the model can themselves be

reacted against, and an infinite regress threatens. In addition, human systems are complex and shifting - can the modelling project make a contribution before its depiction of reality becomes obsolete? These problems would suggest this approach is not suitable for social simulation.

Researcher 1 Modelling: Modeller + Models Resear cher 3 Researcher 4 Researcher 2

Figure 9 Example of an interpretivist approach to understanding simulation modelling

The modelling brings together researchers of different types and different views, and becomes the focus of interactions between them, though other avenues for dialogue may exist at the same time. Outcomes depend on the qualities of both the modeller(s) and the models.

Contrast this situation with that of Figure 9, an illustration of the interpretivist approach. Various researchers - who may be theorists or data analysts, qualitative or quantitative researchers, academics or industrial practitioners, and who may come from different disciplines - have different viewpoints. The modelling project is one of maybe several attempts at dialogue between these researchers. The modeller attempts interpretations of the researchers’ output - their analysis and theories - by representing them in computer code. (Depending on the software package in use, this modelling may actually be something performed by researchers themselves.)

When the simulation models represent the mental models of multiple researchers, there may be tacit tensions between those viewpoints that are revealed during the design, construction and exploration of the simulation. In addition, there may be unforeseen implications of those viewpoints surprising to the researchers. In becoming simulation models, mental models are made explicit (Epstein, 2008). Researchers can react to the public event of the modelling and the shared experience of the computer models, and perhaps adapt their private mental models in the light of this experience. Group consensus is not guaranteed, but it may be facilitated by the modelling project, and decisions based on this group interaction may have more support by the participants.

The mark of a good model is whether it leads to greater understanding between the participants in the modelling process, even if this means they agree to differ. So rather than ask about correspondence to a common reality we ask:

• Can the participants make their views clear enough to be represented in code?

• Where do they agree? Where do they disagree?

• What follows logically and probabilistically from their views?

• Do the participants understand each other's perspective better as a result of the modelling?

Note, however, the mark of a good model is not how well it represents the mental models of all the researchers - they may be incoherent or incompatible with each other.

Researcher 1 Modelling: Modeller + Models Resear cher 3 Researcher 4 Researcher 2 Researchers' Costs Modelling & Discussion Costs

Figure 10 Recognition of the fact that the modelling process has costs determined by extrinsic factors

Success in communication is not guaranteed - for example, Researcher 4 may prove too hard to understand in the time and with the available resources.

This modelling process does not take place in a vacuum. Material reality plays its part - as suggested by the addition of costs in Figure 10. The researchers’ activities carry costs, including the expression of their mental models, and some actions may be more expensive than others in physiological, physical, financial and social terms. Likewise, the other components of the modelling process involve costs. Hardware and software has to be paid for, as does computing time. Some simulation models are easier than others to create, verify, modify and interpret. Modelling outcomes are social constructions - but some things are easier to construct socially than others. Unlike the positivist account earlier, though, we make no claims about depicting within models the external reality. One can certainly reflect on the outcomes of a past or an on-going modelling project, and have an opinion concerning the costs and benefits. But accurate knowledge of costs is not a prerequisite to meeting them.

This concludes our presentation of the two modelling paradigms. Like the positivist approach, the interpretivist approach faces difficulties. People’s mental models may be complex and shifting, and their expressions of them ambiguous and incoherent. Participants may not have the same level of interest in reaching consensus and supporting group decisions. But these are problems faced in all attempts at communication, not just in simulation projects, and some level of communication would seem to be possible - we find sufficient value in it to continue our attempts. So in this light we feel no hesitation in advocating the interpretivist paradigm use of simulation modelling for our research objectives. If, as a result of this hermeneutic process, clarity and consensus emerge as to what organisational problems with energising are being faced, then focus may switch to the approach of Hard OR and the solving of these problems.