1 Introduction to the study
4.10 The research process
The research question for this study relates to business analysis services, work practices and value propositions. Given the limited extant literature about this topic, research is
required into the experiences of business analysts in order to explore the nature of their work and the organisational contexts within which it had been undertaken. A similar research approach, based on observations and comments from highly knowledgeable and
experienced analysts, was adopted during an investigation into a core business analysis activity, requirements elicitation, (Hickey and Davis, 2004).
Figure 4.4 provides an overview of the research process adopted in this study. The stages shown in this figure are described in this section.
Quintain: BA community
Case: BA Manager Forum
Mini case: Expert BA
Data collection
Validation
Triangulation
Validation
Triangulation
Validation
Figure 4.4: The research process
Stage 1: Pilot study
The research project included a pilot study with the aim of evaluating: • The proposed research design.
• The proposed research question and objectives by discussing the experiences and reflections of business analysts regarding their career paths, roles performed, skills applied and overall contribution to projects.
The research question for the pilot study was:
‘How does business analysis contribute to the success of information system projects?’.
Stage 5: validity of research results
Validation informants: interviews and focus group
Stage 4: triangulation of findings
Service catalogue: discourse/content analysis
Job family description: discourse/content analysis
BA workshop group: content analysis
Stage 3: data analysis and theory development
Mini cases: template analysis
Stage 2: data collection
Mini cases: 1:1 semi-structured interviews
Stage 1: Pilot study
The aims of the pilot study were to investigate:
• The mechanisms for evaluating IS success and the processes for achieving such success.
• The standards and work practices applied in business analysis that contribute to the determinants of IS success.
• The contribution of business analysis to the realisation of business benefits.
An interpretivist case study approach was applied during the pilot study. Multiple case design using mini-cases (Stake, 2006) was adopted in order to collect data regarding experiences in undertaking business analysis and uncover patterns of business analysis work practices across different organisational contexts and projects. This aligned with the research design whereby the business analysis community formed the ‘quintain’ (Stake, 2006) and the research aim and question concerned business analysis in general rather than within one organisation. This research design allowed for cross-case analysis and was intended to improve the dependability of the findings.
The BAMF was the case investigated and three BAMF representatives, all BA specialists, were interviewed. The pilot study concentrated on the personal experiences and reflections of the senior business analysts across their organisations and projects. The BAMF case study, and the criteria for the selection of the BA specialists, are described in further detail in chapter five.
The three interviewees were BCS examiners in business analysis so possessed significant expertise and knowledge. This is a key element of the research design as it was vital to obtain rich insights into business analysis in order to address the research question. The context, content, process and outcomes model (Pettigrew and Whipp, 1991; Ward and Daniel, 2012) formed the conceptual framework to guide the research during the pilot study. This included the data collection interviews and the data analysis approach.
Template analysis (King, 2004b) was used to analyse the data collected during the pilot study interviews. A template was developed to code the data and identify emergent themes. The data analysis was structured according to the four dimensions of the conceptual
framework. The findings from the pilot study were categorised according to these dimensions, as follows:
• Context themes concerned the employing organisation and the interviewees’ career paths. For example, the impact of organisational attitudes to business analysis; qualifications held by analysts.
• Content themes concerned the business analyst role and the interviewees’ project experiences. These included lifecycle, role definition and stakeholder management. • Process themes concerned personal, technical and business skills.
• Outcome themes concerned the interviewees’ perceptions regarding business analysis and its contribution to business change projects.
The pilot study uncovered two key issues requiring further research:
• The attitude of employing organisations towards business analysis was raised as an issue, indicating that the level of recognition and awareness of business analysis may vary between organisations. The interviewees also commented on the difficulty encountered when defining the business analyst role. The issue of recognition and role clarity was identified as an area that required further investigation with regard to business analysis.
• All participants stated that business analysis made a significant contribution to successful IS project outcomes. However, the nature of this success was unspecific, raising questions over how the work practices and skills of business analysts
contribute to IS project success.
The pilot study supported the use of the case study method in investigating the research question. However, it also identified that the research question and objectives required further reflection. The revised research aim, question and objectives were developed following a subsequent, more detailed literature review and are stated in section 4.1 earlier in this chapter. These revisions resulted in changes to the interview questions used for data collection. The revised question set is explained in chapter five when discussing the case study and the data collection process.
The pilot study validated the research design. It also provided a basis for reflection and improvement prior to conducting the rest of the study.
The stages of the main study are discussed in the following sub-sections.
Stage 2: Data collection
This stage involved the investigation of the business analysis domain through the collection of primary data from seventeen senior business analysts. Semi-structured one-to-one
interviews were selected to collect data from each individual analyst. The interview questions were structured using the context, content, process and outcome framework outlined in the conceptual framework in chapter three. Some interviews were conducted online using
Microsoft® Skype, while others were conducted ‘face-to-face’ in a professional environment. Each interview was recorded and transcribed in order to enable qualitative analysis of the data collected.
Organisational confidentiality requires particular consideration when using case studies as they are ‘deeply embedded in rich empirical data’ (Eisenhardt and Graebner, 2007, p.25). Confidentiality is particularly relevant within the context of this study because business analysis work often concerns strategically important projects. Therefore, confidential information was not requested during the interviews instead the analysts were asked to discuss their personal experiences, knowledge and beliefs about business analysis practice.
Stage 3: Data analysis
The data analysis process is shown in Figure 4.5. Figure 4.5: The data analysis process for this research
The interview transcripts were analysed using template analysis (King, 2004b) and the results were recorded using NVivo. Template analysis provides a basis for coding the collected data and facilitated the identification of emergent themes. An iterative approach was applied to define and redefine the codes within the template in line with the four
modification types identified by King (2004b). The use of multiple BAMF mini-cases allowed for cross-case comparison, pattern identification and synthesis (Yin, 2013). The data
analysis involved a further iterative process whereby the emergent themes were identified, reflected upon and enrichened. This process uncovered research findings that addressed the research question and objectives and enabled the development of business analysis theory.
Inductive reasoning was applied to develop theory from the experiences described by the business analysts during their interviews. Inductive theory generation is commonly used in qualitative research (Sekaran and Bougie, 2009). As an interpretive research project, the
focus for this research was to understand ‘phenomena through accessing the meanings that participants assign to them’ (Orlikowski and Baroudi, 1991, p.5). This enabled the inductive generation of theory and development of propositions for further research into business analysis.
Eisenhardt (1989) suggests that theory building from case studies is particularly relevant where a phenomenon is relatively unknown and there is limited extant research and theory. The current research into the analysis of information systems does not, in the main,
recognise business analysis as a distinct domain of practice or identify the contribution such analysis might make to the success of IS projects. Therefore, an inductive study was
warranted (Eisenhardt and Graebner, 2007).
The findings from this stage resulted in the development of a Business Analysis Service Framework (BASF) that encompasses the three elements identified in the research objectives: the business analysis services and activities, the taxonomy of required techniques and skills, and the value proposition for each business analysis service.
Stage 4: triangulation of research results
This stage was concerned to establish the plausibility of the emergent theory through the triangulation of the findings. Data triangulation using multiple sources of evidence is important in case study research (Sekaran and Bougie, 2009; Yin, 2013) as it provides a means of extending the insights into the phenomenon under investigation and uncovering evidence in support of or in conflict with the findings from the case study interviews.
Evidence used in triangulation may also be obtained from different groups (Hartley, 2004) as this helps to confirm the original evidence and prevent against bias (Remenyi et al., 1998). Accordingly, a number of different data sources from different groups were used during the triangulation process for this study.
The use of different sources helps to increase the validity of research findings (Remenyi et al., 1998).Therefore, three sources of data were used to triangulate the initial findings: documentation provided by two BAMF member organisations; a standard provided by a professional body; group discussion outcomes collected during a business analysis
community workshop held at a BAMF member organisation. These are described in further detail as follows:
• Secondary data sources in the form of formal documents were provided by two internal business analysis functions: a service catalogue published for internal use within a major energy provider; a document published by the UK Government to
set out the skill requirements of the Business Analysis job family within the Digital, Data and Technology Profession. Discourse analysis was applied to investigate the underlying rationale for the documents (Dick, 2004) and content analysis to explore the constructs provided in the documents, the language used in defining those constructs and the patterns applied in the descriptions. These documents were used to triangulate the findings relating to research objectives one and two. • A standard skills framework offered by BCS, the Chartered Institute for IT,
provided a definition of business analysis skills and techniques. This framework was used to offer a direct comparison with the findings relating to research objective two.
• A group discussion exercise was undertaken to collect data during a business analysis community workshop. The results are in the form of documented group discussions, which were analysed using content analysis order to uncover meanings and patterns (Miles et al., 2013). The discussion results were used to triangulate the findings relating to research objective three.
This stage was intended to confirm and enhance the data analysis based upon the primary data sources, and extend the BASF developed to support business analysis practice. The triangulation process for each dimension of the conceptual framework is discussed further in chapters six and seven.
Stage 5: validation of emergent theory
The final stage involved the validation of the findings and the emergent theory. Discussions with two sets of validation informants were undertaken during this stage:
• The new BASF was discussed with selected individuals, of whom two were involved in stage 2: data collection, and two were new participants in this study. These individual discussions focused upon the content of the BASF and the relevance to contemporary business analysis practice and IS projects.
• The BASF was discussed with a focus group from an internal Business Analysis Practice. The focus group members represented three different IS roles: project manager, business systems analyst and business analyst. They each provided observations with regard to the BASF. These observations concerned the context of their IS project work and the relationships between the three IS roles
Hartley (2004) suggests that checking the research results with participants is an effective basis for improving the validity of the researcher’s findings. This combined approach of involving original study participants and a broader group offered a means of validating the BASF and increasing the potential for its adoption in different organisational settings.
4.11 Chapter summary
The aim and objectives for this study were to develop and validate a new service framework that would help clarify the business analyst role, and define business analysis work practices and the value propositions offered by business analysis. This chapter has reviewed the philosophical choices available to researchers from the ontological and epistemological perspectives, and has clarified the philosophical stance adopted by the researcher. This stance involved a relativist ontology and interpretivist epistemology.
The research method and techniques that may be adopted in order to conduct empirical research have also been discussed and the selected approach, the case study method, has been explained within the context of the research aim, question and objectives.
The available philosophies and research methods are summarised in Figure 4.6; the selected ontology, epistemology, research method and techniques are highlighted in this diagram.
Figure 4.6: Available research choices with selected approaches highlighted
An overview of the research process adopted for this study has also been provided in this chapter. This has included a description of the pilot study stage for this research and the rationale for revising the research proposition as a result of the pilot study. Chapter five
p d e c s Case study Samples Interviews Observation Documents Workshop Focus group
describes the BAMF case and the process to collect and analyse the data about the case. This includes further detail regarding the data collection and analysis during the pilot study in sub-sections 5.5.3 and 5.6.1. The remaining stages of the main study are further explained in chapters six, seven and eight.
5
BAMF case study, data collection and analysis
5.1 Rationale and structure of this chapter
The research design discussed in chapter four clarified that this study is based upon a relativist/interpretivist paradigm and that the case study method is the research approach. This chapter describes the selected BAMF case, the individual business analysts who form the embedded mini-cases within the BAMF, and the work conducted to collect and analyse the data from the business analysts.
The chapter is structured as follows:
• Section 5.2: the levels for this research; an explanation for adopting the ‘quintain’ concept.
• Section 5.3: the Business Analysis Manager Forum; a description of the rationale, structure and aims of the BAMF case.
• Section 5.4: the BA specialists; a description of the individuals interviewed as representatives of the BAMF, each of which is a ‘mini-case’.
• Section 5.5: the data collection interviews; an explanation of the approach adopted to conducting the interviews.
• Section 5.6: the data analysis process; an explanation of the research methods applied in order to analyse the collected data.
• Section 5.7: the triangulation process; an explanation of the approaches used to triangulate the findings from the data.
• Section 5.8: the validation process; an explanation of the process applied to the validation of the findings.
• Section 5.9: chapter summary; the key elements of the case study research.