Step Three: Data Collection and initial Data Organising

The third step of the SPS is the data collection and data organising. This step went through several iterations. The data was collected in two ways: for the first two cases, the primary data collection method was semi-structured interviews; the third case study data is a combination of interview data and observations. The following section discusses these two data collection methods in details.

3.6.1. Data Gathering with Interviews

Semi-structured interviews were my main source of data collection. Snowball sampling (Biernacki & Waldorf 1981) – where interviewees directed me to the next person – was applied in all cases. For the actual interviewing process, I followed interviewing guidelines as described by Myers and Newman (2007). I briefly introduced my personal background, without going into too many details. I usually half-jokingly said that I was conducting the research in order to find good examples of Agile to counter my own experiences in waterfall projects, in order to set the scene to the interviewees. They could now use the technical jargon with ease and describe the events in their own terminology, as I shared the understanding of the language but described myself as an eager researcher, not an Agile authority of any kind. My attire was chosen to reflect the informal clothing of the organisations, usually with some quirky detail that the interviewees would expect from a researcher. Intentional use of ‘nerdy’ clothes or notebooks was sometimes a great icebreaker and conversation starter. Myer and Newman refer to this strategy as situating the researcher as an actor and minimising social dissonance, which was not difficult when I was already a member of the community.

The interviews themselves encompassed a wide variety of different people in different roles and each interview unfolded uniquely, but I did have a list of prewritten topics and questions, as summarised in Table 12. The purpose of the list was to ensure that each interview covered similar topic areas, but the list was often unnecessary as the interviewees discussed the majority of the topics without any probing.

Interview topic Purpose

Background information

Name and role, experience in organisation and industry

Relaxing the interviewee, building trust Software project methods

Management and development methods, experience in methods

How the interviewees see Agile methods

Software projects/products

What is the project/product, goals and expectations, challenges

How the interviewees see the purpose of their work

The team and stakeholders

Daily, weekly and rare communication, rules and processes

How the interviewees communicate

Tools and documentation

Communication tools and documents, when are they used, likes and dislikes, method support

How the interviewees apply tools

Table 12. Interview Topics and Their Purpose

For this research I drew from Walsham’s (2006) suggestions on how to capture interview and other data. I did record all the interviews and the tapes were later transcribed, but I did also take detailed notes during the interviews, where I also collected signs of unusual body language or other things that might be lost with only recordings. As Walsham suggests, my interviews were complemented with background research on secondary data. Every time I would interview people from new organisations, I would first conduct research on the organisations and familiarise myself with what was publicly available. The organisations, their pseudonyms, interview participants and the fieldwork details are summarised in Table 13.

Chapt. Organisation pseudonym

Industry Interview participants Fieldwork A case of…

4. Extreme Inc. Banking

and finance

Technology lead interviewed twice 6 team/delivery leads 4 developers/designers 3 business stakeholder/product manager Department lead Total: 16 interviews

Office visits Agile methods in collocated product development

5. Escapade eCommerce Project manager, interviewed three times

CFO interviewed two times 3 project Sponsors or other managers

4 testers or users Total: 12 interviews

Office visits Agile methods in virtual environment between a customer and a vendor Carmine IT consulting Manager, UX designer, Developer Total: 3 interviews 6. PrecautionCorp. Banking and finance 8 tech lead/iteration manager/business analysts 2 Agile coaches, both interviewed twice Lead Agile coach

4 program manager/business stakeholders,

3 other members of the organisation Total: 18 interviews Observations of several meetings, other office visits Agile methods in large offshore and local product development and process change program

Table 13. Summary of the Research Case Organisations

3.6.2. Data Gathering with Observations

The third case study was exceptional in two ways: I was allowed to follow the program for almost a year and I was also invited to observe the meetings of the program. The details of the arrangement are discussed in the case study description.

Observation of development projects and ethnographies does have a small but interesting part in the research of Agile development. Hummel (2014) claims that there have been sixteen ethnographic studies in the field of Agile

research. Even if the numbers are not large, ethnographies such as Sharp and Robinson’s (2004) and Chong’s (2005) studies on teams who practice Extreme Programming provide detailed accounts of the daily work of organisations not unlike my second case study subject, Extreme Inc.

For this thesis, I have made sure that the data from the field notes is authentic, plausible and critical (as discussed in influential IS ethnography by Schultze’s (2000)), but the sporadical nature of the observations is why I would not describe this as a full ethnographic study. However, the opportunity to observe multiple meetings with a variety of stakeholders could be considered as a complementary feature in my research access. The observations led me to understand the undertaking vastly better than only interviews would. I was also able to non-invasively observe people who would probably never have appeared on my radar through snowball sampling, due to their roles or personalities. These observations provided me with data that was crucial for understanding the complexities of the program and as a source of interview data that was not prompted by my questions.

3.6.3. Organising Data

Throughout the data-collection phase, the data was organised thematically according to how the objects were applied. Organising the data from the very beginning of data collection ensured I was aware of the broader themes and helped me when it came to conducting the interviews. The first cycle of thematic analysis, emerging from the first interviews of the first case study, focused on the very basic day-to-day use of the objects and the practices linked to their application e.g. what kinds of objects were used, when and in which situations.

The second cycle of analysis was conducted after the second case study had begun and focused on the collaborating partier rather than the objects themselves. The third thematic data analysis cycle was conducted after the third case study was completed. This analysis provided themes such as personal Agile experience, corporate culture, histories of the organisations, and infrastructure and operations emerged organically during the first case study data analysis. The new, emergent themes were incorporated in the later interviews with other case organisations. The new cycle of interviews and data analysis was coded against the new themes until no more higher-level themes emerged. This preliminary organisation of data was especially helpful when it came to the more detailed coding, described in the next step.