System dynamics modeling

The outcome of the previous research phases formed the input for a system dynamics modeling exercise we undertook to model the reliability of the organization’s Incident Management process as determined by the capability of its supporting emergency response information system (Van Den Eede, Muhren, Smals, & Van de Walle, 2006). This work is based on the Capability Trap model by Repenning & Sterman (2001) and draws from the many insights on emergency response information systems design as described in the DERMIS (Dynamic Emergency Response Management Information System) framework established by Turoff et al.

(2004). Whereas the latter authors describe the premises that underlie an Information System (IS) that is capable of ensuring a reliable and flexible emergency response, our model contributes to the research field by looking at the interrelations of the aforementioned premises. We take a System Dynamics approach and gain insights in the key determinants of IS Capability by highlighting the mutual interdependences grouped around the concepts of adaptability, control, implicit knowledge and explicit knowledge. The following sections subsequently explain the technique (5.6.2) and the need of system dynamics for studying HROs (5.6.3).

5.6.2 System dynamics

System Dynamics was developed by Jay W. Forrester (1961) during the mid-1950s at the Massachusetts Institute of Technology (MIT). It is a method to gain insight – i.e. identifying, explaining, and eliminating problem behavior (Marais & Leveson, 2003) – into the behavior of (especially socio-economic) complex systems over time and enhances learning from these complex systems (Sterman, 2000, p. 4). Principally by identifying feedback loops in the system, it provides a framework for dealing with dynamic complexity, where cause and effect are not obviously related. System dynamics is grounded in the theory of non-linear dynamics and feedback control, but also draws on cognitive and social psychology, organization theory, economics, and other social sciences (Marais & Leveson, 2003). It does so by representing the

processes, structure, strategies and information flows of systems (Wolstenholme, Henderson, &

Gavine, 1993).

Forrester (1993) states that all decisions are made in an on-going circular environment in which each action is based on information that is the result of earlier actions. System dynamics is based on the idea that people cannot oversee the consequences of these feedback mechanisms in complex systems. There is a bias of looking for problem causes in convenient places (Moberg, 2001). “[There] is a fundamental characteristic of complex human systems: ‘cause’ and ‘effect’

are not close in time and space. *…+ Why is this a problem? Because most of us assume, that cause and effect, most of the time are close in time and space” (Senge, 1990, p. 63). Therefore, much of the art of system dynamics modeling is discovering and representing the different feedback processes in a system (Sterman, 2000, p. 12). A System dynamics model is meant to give people a more effective understanding about an important system that could have controversial behavior. Influential system dynamics models are those that change the way people think about a system (Forrester, 1993). Wolstenholme (2004) distinguishes five components of system ‘structure’ in the concept of system dynamics: (1) processes, created using stock-flow chains; (2) information feedback; (3) policy; (4) time delays; and (5) boundaries.

5.6.3 The need of system dynamics for studying HROs

System Dynamics has been applied in all kinds of domains and for all kinds of problems, like the cold war arms race, the human immune system, the aircraft industry, and business and strategy issues (Sterman, 2000, p. 41-42) but the technique can play an equally important part in the study of High Reliability. We posit that HRO literature can do with more system dynamics exercises. In our case, for instance, by building an explicit representation of the incident management process, it should/might be possible to predict, simulate, and/or explain the resultant behavior of the system form the structure causality, functional and behavior of its components (Lewis, 2004). “If we do establish a connection between given features of an organization and its operational reliability, can we assume that these features are necessary, rather than merely sufficient? That is, are there alternative strategies possible for the production of the same outcome, or do the requirements for organizational reliability impose an inevitable solution on an organization? Finally, even if an organization has evolved features that are necessary to its reliability, can we be sure that its evolutionary adaptation is stable, that what adds reliability today will also work tomorrow under changes in the character of technology, the work force, or in the life cycle of the organization?” (Schulman, 2001, p. 347-348). Hence, this is a useful approach to deal with non-linear problems in a non-linear way (contrary to a classical risk management approach). While it is tempting to attribute the kind of reliability described in this dissertation to organizational culture and values (Little, 2005, p. 5) it is not sufficient. What we need is a laboratory setting that permits to verify independently the different building blocks⁵¹. System dynamics modeling is a relevant technique for our research because of the complex nature of the emerging interactions (i.e. complexity), the choice of our model is vital.

Efforts to model HRO characteristics so far have mainly concentrated on unreliability, in studies of calamities and mishaps: for instance, the 1992 Westray mine disaster (Cooke, 2003), the 1986 Chernobyl nuclear disaster (Salge & Milling, 2006). In part this is due to the prescription that system dynamics should model problems, not systems (Sterman, 2000), but such can be remedied by formulating high reliability as a problem, e.g. by modeling the absence of HRO

characteristics and investigating the consequences thereof on organizational reliability.

Examples of application of system dynamics in HRO contexts include (for instance) outage planning and plant improvement efforts and maintenance (Caroll, Perin, & Marcus, 1993);

patient safety (Cook & Rasmussen, 2005); security incident management (Gonzalez, 2005) and railways safety (Hale et al., 2003). Notable studies building on system dynamics for conceptualizing on HRO and NAT include the analyses by Cooke & Rohleder (2006) and Rudolph

& Repenning (2002).

The latter type of studies is interesting because they have the potential of combining system and culture, as “the two great tropes of the 20th century *…+. Whereas systems thinking holds out hope for the precise mapping and managing of complexity, cultural approaches are less focused on outcomes and more on understanding and appreciating the meanings people ascribe to collective behavior, and how these meanings help constitute communities and societies”

(Eisenberg, 2006, p. 1697). For instance, Romme and van Witteloostuijn’s (1999) notion of

‘circular design’ combines (1) insights from Organizational Learning Theory with (2) insights from Systems Theory. More specifically, they combine the concept of triple loop learning, manifesting itself in the form of ‘collective mindfulness’ with three important notions of system dynamics:

(a) the pervasive influence of the deeper structure of a system on the behavior of its constituent elements; (b) the difference between static and dynamic systems; and (c) the dilemma between global patterns (e.g., corporate patterns and synergies) and local behavior (e.g., creative actions and findings at the unit or team level). The analysis of circular designs relates strongly to the HRT concepts as it also examines the relationship between structure and behavior. More in particular, circular design, like HRT, stresses the importance of the opportunity to switch between the circular and the hierarchical mode of organizing, as a pivotal element (Romme &

van Witteloostuijn, 1999).

6 Survey

Without prejudice to the relevance of the previously described research techniques (5.3 to 5.6), this research’s main data collection technique is a self-administered survey. For this reason this separate section is dedicated to its description. The survey questions respondents in our case study organizations on those elements that have become apparent in the previous research phases. In this subsection we first point out why we have chosen for the survey technique (6.1).

Next, we explain the way the conceptual model is operationalized (6.2) and how the survey is administered (6.3). We conclude this section with a description of the data analysis process (6.4).

6.1 Rationale

In an era where ‘what cannot be measured, is not true’, a particularly difficult task emerges to translate constructs like mindfulness and resilience into workable indicators (Le-Coze, 2006, p.

8). As we have experienced ourselves, this exercise is necessary in order to get stakeholder’s – and especially management’s – commitment to cooperate. The buy-in ⁵² is obtained more easily with figures. The survey we have designed hence is a mixture of trying to measure the immeasurable and returning value for money. We have tried to find our way out of this dilemma by gaining trust of the organization by doing ‘consultancy’ jobs of a smaller order.

The use of quantitative research techniques in the field of organizational behavior is not very common (Balthazard & Cooke, 2004, p. 240). Qualitative and quantitative methods nevertheless are complementary approaches to the study and assessment of organizational processes and attributes. The advantages of qualitative methods can lead to intensive and in-depth information, especially useful in research on issues and processes about which little information exists (Balthazard & Cooke, 2004, p. 240). Quantitative methods on the other hand have the advantage of cross-sectional assessments and comparisons (across individuals, organizations, or sub-units). It is this last characteristic we have relied on since we deal with the phenomenon of an organization’s reliability through its subunits as level of analysis. A greater understanding of the factors and values that affect intra-organizational contextual and structural variations can only provide an even richer picture of how to optimize organizational systems to promote reliability (Balthazard & Cooke, 2004, p. 240).

Quantitative methods facilitate asking large numbers of people their opinions in a relatively time- and cost-effective manner (Child & McGrath, 2001). It should not come as a surprise therefore that over the years, the survey method was extensively used and is still in predominant use. While the method can attain high levels of external validity, it is known to suffer from lack of control and internal validity (Palvia et al., 2003). Another critique on surveys in the field of reliability is that they are used merely to refine the question sets in order to improve face validity, and in some cases to conduct factor analyses and internal consistency checks (Child & McGrath, 2001). Yet, quantitative research that goes beyond this type of analysis is necessary in order to bring the discipline of reliability studies to reach a higher level of maturity.