Validity and generalisability

To ensure quality in the research findings, there are two approaches; validity and generalisability (Gibbs, 2007) are discussed below.

4.8.1 Validity

Validity means the accuracy of research findings. According to Maxwell (2002), validity has been debated amongst scholars in relation to the legitimacy of qualitative research study. It relates with the consistency of results, policies and programs or predication. If qualitative research does not comply with such consistency then the reliability of findings would be an

issue. Maxwell (2002) also suggested that validity pertains to data, conclusion and analysis, completed by a method with a particular context for a purpose.

There are several ways to deal with validity challenges, both in qualitative and quantitative research. Researchers using quantitative methods, in contrast to qualitative researchers, generally deal with expected and unexpected risks to the validity of findings. For instance, Maxwell (2005) argued that qualitative researchers rarely have the benefit of planned comparisons or strategies for sampling or statistical data manipulations. Consequently, researchers should rule out validity threats after research initiation by establishing alternative hypotheses for the evidence collected.

Two major risks to validity were identified (Maxwell, 2005) and they commonly relate to qualitative research techniques. These are:

• Researcher bias - this takes place when data is selected based on the researcher’s existing theory or research interests.

• Reactivity – this is the influence a researcher could have relating to the setting or individual studies.

Maxwell (2005) further argued that procedures or methods do not ensure validity, yet they are important to mitigate the potential risks associated with validity and increase the credibility of results. For the purpose of this study, the researcher used secondary data (literature), an online expert panel and case study interviews to support the validity of results.

4.8.2 Generalisability

Qualitative research does not usually allow a systematic generalisation to a wider populace, in contrary to quantitative or experimental studies. Generalisability is defined by Maxwell (2002) as the extent or a level to which one expands the account/finding to another person, time or setting beyond the actual account studied (Maxwell, 2002). According to Yin (1994), generalisability is often based on theoretical assumptions that lead to simplifying similar situations and the drawing of conclusions. It is recognised that sampling is important for a researcher to establish interfaces from actual facts based on person, event or activity observed at first instance against the other facts, event, situation and/or people at later times (Maxwell 2002). It is generally unrealistic to expect that a researcher would observe all aspects of a research study at a given time with the small setting with reference to the study.

Maxwell (2002) highlights two aspects of generalisability, as follows;

• Internal generalisability: This includes generalising within the setting, community, group or institution studied as part of research to the person, event or setting that were not directly included or involved.

• External generalisability: This includes generalising beyond the group, context or time that was not studied directly in research.

For the purpose of this thesis and research study, both types of generalisation were considered. The researcher is not claiming that the outcome of this research will absolutely apply to all cases discussed; however this study provides an opportunity for the reader to make judgements on the applicability of the findings. The researcher does believe that as the model was extensively tested using qualitative data analysis techniques (the expert panel and case study interviews), elements of the model will be useful for midsize businesses considering implementing ERP applications. As data was collected from experts around the world, it is assumed that the elements of the model or findings would be beneficial or applicable for midsize businesses in other countries as well. It is reported in the literature (Schofield, 2002) that case study analysis or a multiple case study approach could increase the generalisability of qualitative research. This research technique was used in the second phase of this study but the data was only collected in Australia, from the state of Victoria.

4.9 Chapter summary

This chapter provides an outline of the research methodology employed for this study. At the beginning of this chapter, several methodological approaches were discussed. For the purpose of this thesis, modified Delphi and case study approaches were used. More details on the associated expert panel and case study interviews were also provided. In addition, this chapter provided a brief review of the advantages and disadvantages of different approaches and the content analysis used to analyse data. The chapter also provided a discussion on the expert panel processes followed to select and conduct the study. In addition, an understanding of the selection of case study interviews and the process used to conduct interviews was provided. In the next chapter, the first data collection phase (the expert panel) is discussed in detail.

Chapter Five

5

Expert Panel

5.1 Introduction

This chapter focuses on the first data collection stage and provides details of the development and execution of an online expert panel. The first section outlines the expert panel approach used to select and engage with experts for this exercise. In the later part, I talk about the structure of discussion, the discussion key points and the patterns of discussion. Finally, I elaborate on the comments made by different experts on each topic and will discuss their relevance and impact on the revision of the ERP adoption model.