for cross-boundary coordination
3.4 Methods 1 Research Setting and Data Collection
In order to obtain access to the short-cycle coordination processes through which the officers tailor the boundary object, we conducted a qualitative field study of emergency management exercises. This allowed us to study how boundary objects were configured in the interaction between Police, Fire Department, Medical and Municipal officers during a response operation. The data presented in this article was collected in the winter of 2010, in which 10 field exercises were observed, recorded, and transcribed in detail. In addition, 20 in-depth interviews were conducted with all the officers that participated in these exercises.
We chose to focus in depth on the collaboration processes in these exercises, favoring accuracy over generality in the first stages of our research (Brinberg and McGrath 1985, Langley 1999).This enabled us to capture the process of creating the COP as a boundary object, which is grounded in the ongoing interpersonal interaction between the officers. Our data collection focused on how officers interpreted the information shared concerning this boundary object and how it influenced their consequent actions. For us it was very important to understand how these actions were embedded in context of response operations, to be able to capture the richness of the coordination process without abstracting it away from the interdependencies in the environment (Barley and Kunda 2001).
Emergency response field exercises offer a unique opportunity to study the use of boundary objects in situ. They are scenario-based exercise sessions in which emergencies are simulated, that start with a predefined script, but continue in any possible direction as the events unfold. As such, the exercises resemble the reality of emergency response, since actions of the officers have direct influence on subsequent events and outcomes. Exercises differ from training sessions in that events are not interrupted for moments of reflection, but the response operation is left to unfold. As such, exercises are a good focus for analysis because the emergency management organization is similar to events in the field, and actors are obliged to act and forced to make decisions that influence the way in which the exercise unfolds. Still, it is also necessary to acknowledge the limitations of exercises: because scenarios are removed from reality they may lack the same emotional aspects, decision impact and in some cases delay in the recruitment of units (Latiers and Jacques 2009).
The exercises themselves were staged in different locations on training grounds specially designed for emergency response exercises. These training grounds were located at a former military airbase, on which abandoned buildings like an energy plant and a small village were transformed into a disaster area with caved-in buildings, a gas station, train yard, a section of highway, etc. These enhanced the realism and feel for the situation, as for example an exercise that involved a highway traffic accident was located at the realistically reconstructed highway with crashed vehicles and trucks. The exercise scenarios included: a fire in a youth hostel; fire in an adult club; a highway collision; a hostage situation; carbon monoxide diffusion in an elder home; a fire in a tire factory; an explosion after a failed SWAT raid; and a Cessna plane crash on a gas station. The average length of the exercises was 1h26m, ranging from 1h2m to 1h54m. Each exercise consisted of two or three command meetings, with approximately 15 to 30 minutes of
3
time in between to carry out the tasks set out in the meeting. This corresponds to the phase of group activity that Marks, Mathieu and Zacarro (2001) call transition. Following the exercise, an after action review was held.
In the exercises the COP is created on a white board or flip chart in a mobile field command center. A command meeting was held approximately every half hour to create or update the COP on a common display, and to discuss the progress and challenges of the response operation. In many countries including the Netherlands, the COP is a common part of response operation training and contains two elements: a geographical representation of the situation (drawing) and a checklist of the actions (both completed and “to-do”) (Wolbers and Boersma 2013).
The sequences of activities during the exercise were observed and audio recorded by the first author by shadowing lead officers. The first author also took field notes on the interactions between officers from different organizations, in order to analyze and reconstruct the way in which the COP was used to guide actions and decision making. Observations are important tools to capture this process since people often cannot articulate how they do what they do unless they are in the process of doing it (Barley and Kunda 2001). In addition, informal conversations were held before and after the exercises to allow participants to reflect on their actions. After the exercises, participants were interviewed in the spring of 2011 to ascertain their view on the response exercises in order to find out how they understood the COP and the actions of other emergency services in specific exercises. Discussions in the interviews suggested that the interviewees still recalled the exercises in quite some detail and felt confident commenting on them. The analysis of the exercise transcripts and field notes provided the input for the interviews. We asked the officers how they constructed the COP, both mentally and on paper, how they perceived the information and collaboration with other officers, and how this influenced their decisions for specific actions. As such, moving back and forth between analysis of the transcripts and the interpretations of the officers in the interviews allowed us to explore and explicate the dynamic of tailoring the boundary object.
The prolonged period of data collection allowed us to zoom in and out on the data (Nicolini 2009), and enabled us to critically assess, compare and broaden our understanding of the phenomena we observed in the field exercises and heard described in the interviews. The research process developed iteratively as we analyzed transcripts of the recordings, interviewed the participants of the exercises based on the previous analysis and combined this with analysis of documents on the use of the COP. During this
process we realized that the officers tailored the COP in unique ways, each of which had its own temporal trajectory. This guided our focus towards the sequential order in which the tailoring occurred, requiring a mixed methods design to investigate this.
3.4.2 Analytical Process
We have divided our analytical approach into five stages based on a conversion mixed design (Teddlie and Tashakkori 2009), to integrate our qualitative and quantitative analysis. A general overview of this mixed methods design is visualized in Figure 4.
Quantization
Modes identified with Lexical Markers Observation and data
gathering
Analysis
Member check
Analysis
Analysis Timeline and Markov Analysis
Interpretation Interpretation of Timeline and Markov Analysis Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Qualitative Step Quantitative Step
Figure 4. Design of the study, using notation described in Teddlie and Tashakkori (2009)
In the first stage of our analysis we started with inductive theory building (Corbin and Strauss 2008) to develop the theoretical concepts and connections among them from our process data (Langley 1999). The transcripts from the interviews and exercises were combined with the help of the data analysis tool MaxQDA (Corbin and Strauss 2008) to inform our analysis. The fully transcribed exercises allowed us to reconstruct the interactions of the officers with each other and the COP chronologically. We coded the elements of this process, which provided insight into the different ways of tailoring the boundary object.
3
In the second stage of the analysis we conducted a member check (Schwartz-Shea and Yanow 2009) to see whether our ideas applied to the operational experiences of the officers. We conducted these interviews in the summer of 2012, two years after the initial observations. Our acquaintance with the officers allowed us in this stage to reflect on their experiences during the exercises in comparison with the actual operational responses in the two-year period, and ask them about the relative importance of the COP in these response operations. In these interviews we asked the officers to describe their work and experiences during operational responses of the last two years. This enabled us to confront and build on one of the challenges in our material, the fact that the observations were based on exercises instead of real incident response operations. By taking this step and checking the analysis and results with the officers, we felt more confident that the results reflected the actual practices of emergency response operations.
In the third stage of our data analysis we focused on possible temporal patterns in the development of the boundary object during the exercise. If the officers tailored the boundary object to suit their coordination needs, did they do this in specific ways and were there discernible patterns in the process of development over time? In combination with a progressively deepening literature review we gradually arrived at four coordination modes used to tailor the boundary object: selecting tasks, phasing action, standardizing information and transforming understanding. We recognized that these could be connected to the original forms of the boundary object in the work of Star and Griesemer (1989), and reframed them to capture processual moves. As such, our inductive theoretical reasoning was an iterative process of moving back and forth between empirical impressions and theoretical inferences (Alvesson and Kärreman 2007).
In this stage we continued moving back and forth between these theoretical insights and our empirical data. We analyzed the transcripts of the exercises to discover if specific patterns of discourse were present that resembled each coordination mode. A human/ machine augmented analysis with help of a lexical search in the program MaxQDA (Corbin and Strauss 2008), aided us in providing a word frequency analysis that suggested that each coordination mode had specific terms associated with it that were not associated with other patterns. We used these as markers of the mode, if we believed it represented the mode accurately enough. After this semi-automatic coding, we rechecked whether the codes actually reflected our original definition of the coordination mode and deleted those codes that did not reflect this original definition. A more elaborate description of the logic we used to code the discourse based on the four coordination modes can be
found in appendix 1. The average number of coded segments per exercise is 143, and ranges from 132 to 171. This resulted in the following list of key terms that signaled which of the coordination modes the officers were in during a specific unit of analysis:
Selecting tasks Phasing action Standardizing
information
Transforming understanding
Message Later Procedure Because
Meanwhile Take care Regulate Certain
Happened In a while Plan Means
Task First Distance Convenient
Created Arrange Alarming Think
Self Each other Deploy Sensible
Done Time Scaling up Why
Last Adjust Protocol Reason
Control Feedback GRIP Danger
Mono Consult Estimate
Bilateral Accounting
At this moment Safe
Pass on Understand
Priority Busy Ongoing
Table 3. Terms associated with each coordination mode
The third stage of analysis produced temporal sequences (timelines) of codings of coordination modes for the ten exercises. To facilitate identification of common sequential patterns across the exercises, we utilized formal modeling techniques in the fourth stage of our analysis. To detect patterns of action over time we performed a Markov chain analysis, which gives an indication of whether there are regularities in local sequential structure whereby acts are predictable from immediately preceding acts (Poole et al. 2000). The output of a Markov model is visualized in a transition matrix that presents the probability of the occurrence of sequences of coordination modes, relative to other sequences. This gives an overall 'picture' of the sequential structure in the data that
3
enabled us to determine statistically whether there were common recurrent structures. By identifying common recurrent sequences, Markov analysis points to features of interaction that can then be explored in further discourse analysis. The coded discussions among the emergency responders (coded into categories selecting tasks, phasing action, standardizing information, and transforming understanding) generated a sequence of activity types that are the input for the Markov analysis.
The validity of a Markov chain is assessed through testing the assumptions of order and stationarity based upon a log-linear analysis outlined in Poole et al. (2000), to which we refer the reader for a more detailed description of the statistical analysis. The first assumption of whether the process depicted by a Markov chain has a definite order is assessed by comparing the fit of models of successive order. We compared the fit of ‘higher’ order models (2nd order) with ‘lower’ order models (1st order), and a baseline model that indicates no local order in the interaction (0th order). In a first-order Markov chain, each successive action is reliably predictable from the preceding action, which implies an act-response format. In a second-order Markov chain, each successive action can be predicted based on the previous two actions, which implies an act-response- response format. All second order Markov chains can also be re-expressed as first order chains, which aids in interpretation of sequences. The second assumption is whether the same transition matrix exhibits stationarity, so the sequential probabilities hold for the entire sequence. This is tested by dividing the entire sequence into shorter segments, calculating their transition matrices and comparing these.
Our tests indicated that first and second order Markov chains represented the sequential dependencies. First order models fit five of the episodes: Exercises 1, 3, 4, 5, and 6. Second order models fit three of the episodes: Exercises 2, 7, and 10. Only two episodes, exercises 8 and 9, showed no significant sequential structure (0th order). The transition matrices are presented in appendix 2.
Markov analysis is useful because it tests whether there is structure in the overall episodes. To provide better resolution of this structure, we then utilized a visual mapping strategy (Langley 1999) by creating a timeline that visualized the transitions between the coordination modes based on the sequences of codings. This timeline depicts the sequence of coordination modes as it unfolds during the episodes and affords a more detailed understanding of the temporal sequencing. We moved back and forth between the Markov transition matrix and the line charts to look for and develop inferences about common sequences and the progressions of events (Langley and Truax 1994).
In the fifth stage we went back to our qualitative data of both the discourse strings from the exercises and interviews with the officers, to develop an interpretation of the critical transitions we identified in the previous analytical stage. The process analysis from stage four is suited to identify regularities in short sequences, while the discourse approach is able to lend meaning to the regularities, but also to make sense of significant, irregular patterns and longer strings of discourse. By analyzing and reflecting on the discourse in these critical patterns, we were able to develop theoretical inferences about how the boundary object is tailored to support or impede cross-boundary coordination.