3. Research methods
3.7. Steps Four, Five and Six: Constructing, Validation Cycle and Coding
The next three steps in the SPS model are 4. constructing and extending the theoretical lens, 5. confirming and validating data, and 6. selective coding. I began the data analysis in tandem with the data collection, as suggested in the SPS approach. The data from the first case study was compared to the insights gained from the literature on Agile software development and boundary objects. For example, literature discussing different perspectives on Agile software development was helpful when analysing the views of the methods held by the informants. Similarly, the boundary object literature helped in identifying the boundary objects and the boundaries these objects were bridging. However, the frameworks developed during the study are novel and the literature was applied to guide the data analysis, not as direct source for the coding or themes. The coding is illustrated in Table 14.
First Phase of Coding Second Phase of Coding Themes and Theoretical Lenses
Role and personal history
Individual roles: team leads, scrum masters, iteration managers, project managers, analysts, testers, designers, product owners Teams
Organisation background and structure
Different roles and stakeholders
Themes
Background and day to day operations of the and the overall Agile perspective of the organisation members Theory Boundary objects Agile methods Organisation culture Organisation change/organisation history Hiring practices Working from home\
Corporate culture and how Agile is applied in the organisation
First Agile encounter
Thoughts on Agile/What is Agile Agile training in organisation Agile process change
Personal Agile experience and perception on practice
Internal communication External communication Contractor communications Sales and marketing Risks/issues/concerns
Communication practices across different boundaries
Theme
Organisation of the stakeholder communication
Theory
Boundary objects enhancing collaboration
Agile methods enhancing collaboration
Stand-ups/Scrum of Scrums Pair programming
Retrospective meetings Other meetings
Applied Agile practices are part of the communication between stakeholders
Planning
Requirements and estimation Design
Product plans and designs as part of project initiation
Themes
Application of boundary objects in order to collaborate with stakeholders Theory
Boundary objects enhancing collaboration
Agile methods enhancing collaboration
Contracts/business cases Walls/Jira use
Chat tool/Flowdock use Prototypes
Wiki/Confluence use Testing tools or other tools
Application of artefacts in communication between stakeholders Infrastructure Deployment Regulations/Audits
Infrastructure and operations as part of the development process
Personal preference led me to forgo qualitative data-processing tools such as NVivo. Instead, I coded the textual data from the transcripts and observation notes in an Excel spreadsheet and performed the second phase of coding with a low-tech solution, Post-it notes (Strauss & Corbin 1990). During the first phase of data coding, I listened the interview taps, read the interview transcripts and noted taken during the interviews, and assigned a codes to the data based on the topic each paragraph, section or phrase was discussing. In the next phase of coding, I organised the data according to the larger themes emerging from the first phase of codes. These larger themes were finally organised into the major themes of the thesis: Agile perspectives, collaboration and boundary objects application from which the theory, discussed later in the sections 7.4 and 7.5 begun to emerge.
In addition to the coding in spreadsheet, I applied visual data analysis techniques such as visual mapping of the major events and stakeholder groups as well as writing up narratives of the events allowed me to trace application of the Agile methods and artefacts in each case (Langley 1999). I drew timelines of events that took place in each organisation, figures that captured different stakeholders and their relationships as well as process diagrams, which illustrated the use of objects, and Agile practices. These visual aids supported the thematically coded data by providing the linkages between the different themes.
Data validation and the sufficiency of data is a difficult topic in qualitative research. Myers and Newman (2007) state that a variety of voices is needed, Tan and Pan (2011) give an exact number for minimum interviews where as the positives authors such as Yin (2013) and Eisenheardt (1989) discuss triangulation and multiple data sources. However, the interpretive authors, such as Walsham (2006), give no definitive guidelines on when to finish the data collection. For my study, there were a few different ways to ensure sufficiency of data.
For the first case, the saturation was reached when I had interviewed someone from every tribe and almost every role in the organisation, excluding only the upper management. For the second case study, the process was simple: I interviewed every major stakeholder of the project, save two who were not available for interviews. In addition to the major stakeholders, I interviewed some people who were temporarily involved in the projects and had insights into the processes and tools. For the third case study, the program organisation was very complex. Interviewing someone from each role across all different streams was not feasible so I focused on the agile coaches and iterations managers who had good oversight on the daily work. In addition, I managed to discuss with few business stakeholders, business analysts and testers across the organisations. In the third case study, the observations supplemented the interview data and confirmed that I had understood correctly the daily program activities.