Qualitative studies are used to answer questions about the complex nature of
phenomena that occur in a natural setting and end with tentative answers or hypotheses (Leedy & Ormrod, 2001). In contrast, quantitative research enables researchers to count and measure statistics of two types: descriptive, which explains quantitative data in a summary form, and inferential, which draws meaningful and significant conclusions from quantitative data (Gillham, 2000). With agile method adaptation, very little knowledge or understanding exists, requiring in-depth investigation of the entire agile development approach and product development environment to identify and
understand agile adaptation factors. Qualitative study was more appropriate to investigate agile adaptation factors from a few successful agile development
environments having experience in adapting their agile methods and providing a basis for future learning of agile adaptation factors.
Qualitative study enables the researcher to study selected issues in-depth and to collect data which are detailed descriptions of situations, events, people, interactions, observed behaviours, and quotations from people about their experiences, attitudes, beliefs, and thoughts (Patton, 1990). Hence, qualitative studies provide understanding of events, situations, experiences, and actions of participants in organisations including the context within which the participants act, and processes by which events and actions take place (Myers & Avison, 2002). Quantitative methods require the use of standardised measures so that the varying perspectives and experiences from a great number of people can fit into a limited number of predetermined response categories (Patton, 1990). Quantitative methods would not have enabled the collection of detailed information to provide insights into individual context and reasons for making adaptation decisions. Qualitative method allowed the researcher to make an in-depth study of agile production labs
through observation and asking related questions, learning directly from the individuals in regards to agile adaptation factors. Quantitative study will be an appropriate way to learn about the general applicability of agile adaptation factors to agile software development community, once these factors are investigated from a few successful cases.
The selection of a qualitative approach for this research was based on the understanding that it was the most appropriate way to investigate agile adaptation, on which very little knowledge currently exists.
3.3.1 Qualitative focus
The themes of qualitative inquiry identified by Patton (1990), listed in Table 3, were applied to ensure the validity and rigor of this research. They ensured an in-depth investigation without any prejudice or manipulation of adaptation data collected from the case study sites. These themes enabled a credible research strategy to establish significant and compelling findings on factors that influence agile methods adaptation.
Table 3 Adopted qualitative inquiry themes
1. Naturalistic inquiry Observed the development teams; understood and documented their
development environment, adaptation and application of agile practices without any hindrance.
2. Inductive analysis This was an explanatory and exploratory study; the focus was to learn about
agile adaptation from the development teams without imposing any prior expectations; asked method users legitimate open questions allowing adaptation factors to emerge.
3. Holistic perspective Investigation took into account the entire setting for software development;
methods, practices, tips, techniques, tools and available resources for the teams. It enabled gathering data on multiple aspects of the agile approach.
4. Qualitative data An in-depth inquiry that produced thick description based on direct
quotations by method users; they gave their perspectives and experiences. Resulting description provides understanding on agile method adaptation without any judgment.
5. Personal contact and insight
Data collection done at the development lab; got in direct contact with the team members, their development environment and method application. Provided the necessary insights to understand agile adaptation.
6. Dynamic systems Expected the likelihood of unanticipated findings since the prevailing
software development environment is dynamic. Naturalistic inquiry enabled the researcher to investigate, describe and understand agile adaptation in such an environment.
7. Unique case orientation
Two case studies are successful software development houses with
international customers and competitors. Both have successfully adopted an agile approach; they provided rich information. The first level of inquiry
compared their method fragments and identified their adaptation factors.
8. Empathic neutrality This research provides understanding on agile adaptation factors in its
complexity (not advocating any agendas), learnt through association with development teams and by experiencing their method application. Gained insights to relevant data and took a unbiased stance towards emerging adaptation factors.
9. Design flexibility Adopted a flexible and iterative design approach; it enabled the pursuit of
new paths of discovery as the understanding of agile adaptation developed.
Source: (Patton, 1990)
3.3.2 Case study approach
There are several qualitative methods for research such as case study, ethnography study, grounded theory, and content analysis (Leedy & Ormrod, 2001). However, no research method is superior to others and the method selected to investigate an information systems phenomenon is based on current knowledge of the topic and the nature of the topic being investigated (Benbasat, Goldstein, & Mead, 1987).
Case study, ethnography study and grounded theory methods consisted of three kinds of data collection: in-depth, open-ended interviews; direct observation; and written
documents (Patton, 1990). This multi-source approach for data collection, known as triangulation, makes it possible to confirm if participants actually practice what they have said (Gillham, 2000) determining convergences and giving credibility to the patterns that emerge (Yin, 1994).
Ethnography study and grounded theory methods require generating theory through research data rather than testing or applying ideas formulated in advance, prior to data collection and analysis (Dey, 2004: Linclon & Guba, 1985). Case study research is driven by prior development of theoretical propositions to apply, test or generate a relevant theory providing a more scientific approach for the research (Yin, 1994). Grounded theory requires no literature review prior to research, no taping of interviews and no discussion of theory before being written (Glaser, 1998). Ethnography research also does not require a prior model, emphasising that no theoretical model can explain in advance the likely findings in a new contextual setting (Yin, 1994).
These requirements of grounded theory and ethnography study were seen as a major obstacle by the researcher for conducting this study. The researcher required prior readings on agile publication (books and academic literature) to have a good grasp of
agile methods and to identify research problems. Having a theory or conceptual framework to guide data collection and analysis was critical for this investigation, in order to ensure the researcher’s confidence to carry out this research and also for providing confidence in the findings.
The content analysis method focuses on investigating any recorded (books, websites, paintings and law) verbal, visual, or behavioural forms of communication and requires coding of material in terms of predetermined and precisely defined characteristics (Babbie, 2007). No formal documentation on agile adaptation decisions was kept by agile development teams, making it impractical to be solely dependent on this research method. However, an element of content analysis is part of the case study, ethnography study and grounded theory methods to analyse and code interview transcripts.
The case study method was determined to be the most appropriate and best suited qualitative method for this study, since the researcher investigated “how” and “why” research questions (Yin, 1994). In particularly, case study method is particularly suited to learn in-detail through an in-depth study of a few successful cases on a phenomenon on which very little knowledge exists (Dube & Pare, 2003; Patton, 1990). Hence, case studies are chosen for in-depth investigation of the agile product development
environment and development approach to learn about agile adaptation from a few successful software development organisations.
Ethnography study also enables an in-depth study of a contextual setting to understand the day-to-day practical functioning of individuals but emphasises longitudinal
involvement (more than a year) to observe and ask appropriate questions (Walter, 2006). However, a long term continuous involvement for observations at software production sites was seen as hindrance by development teams in meeting their short cycle delivery commitments. For this research, it took over a year to collect data through non continuous involvement. The grounded theory approach requires theory development to be repeated until the researcher reaches the data saturation point for theory creation from a new case. Grounded theory is more suitable for investigating a single aspect of phenomena and if the researcher has access to large pool of case studies to sample from as the study proceeds. Given the lack of a large number of local cases and the motivation for a holistic investigation of agile approaches to identify and
provide understanding about agile adaptation factors, the grounded theory approach was deemed not suitable for this study.
The case study method, similar to ethnography study and grounded theory enabled the collection and compilation of rich data and detailed descriptions of events, interactions, observed behaviours, and direct quotations from method users through direct access to the software production labs, allowing the researcher to see and experience agile method application (Donmoyer, 2000). This facilitated the identification of the relevant agile adaptation factors, which previously were not known (Leedy & Ormrod, 2002).
The case study, ethnography and grounded theory approaches tend to be idiosyncratic (Stake, 2000). This research used an idiographic (naturalistic context) rather than nomothetic (controlled environment) strategy to investigate and collect adaptation data (Fitzgerald & Howcroft, 1998; Lincoln & Guba, 2000). Idiographic research in
organisations required the researcher to be absorbed within the organisation’s culture for a significant period of time to understand particular events, enabling rich data to be collected. In the beginning, a week was spent at each case study site observing the application of agile methods and individual interactions in development environments as daily routines were performed.
Observation allowed a significant understanding of team cultures, application of agile method learning, various aspects of the methods (fragments) and their application, team environments, the team resources, individuals and make up of the teams, and the other stakeholders, which were further investigated with interviews.