Reliability and validity of the data

According to Fellows and Liu (1997, p.135) ‘reliability concerns the consistency of measure …’ whereas ‘validity concerns how well a measure does measure the concept it is supposed to measure’. With qualitative research, where the data is obtained through conversations with participants, reliability can be problematic, as it is not seen as being objective as is the case with quantitative research. Would the participants respond in the same way to one researcher as another and would the response be the same at different times (Turner, 2005). A sample, by its very nature is only a representation, at a snapshot in time of a wider population; hence ‘the findings are valid for the subject data only’ (Fellows and Liu, 1997, p.136).

For this investigation it is important to explore the anecdotal evidence and to seek a widest possible range of opinions across a broad spectrum of topics. ‘By combining multiple observes, theories, methods and empirical materials, researchers can hope to overcome the weaknesses or intrinsic biases and the problems that come from a single method, single-observer and single-theory studies’ (Sachdeva, 2009, p.183). Mixed methods research provides an opportunity to draw on multiple data collection methods and quantitative and qualitative data analysis techniques bringing together several different perspectives on the evidence provided. The reliability of the study can be improved by triangulation in its various forms (Ihantola and Kihn, 2011). Triangulation is seen as an important part of the validation process to ensure the research is ‘fit for purpose’ by analyzing a research question from multiple perspectives.

Of interest to this research are internal and external validity in a quantitative research context and contextualized and, generalizability and transferability in a qualitative research context. Internal validity is concerned with ensuring that ‘any conclusions we draw are solidly based’ (Bechhofer and Paterson, 2000, p.18). In qualitative research, internal validity is achieved by ensuring that there is ‘logic between a piece of research and existing theory’ (Ihantola and Kihn,

2011, p.5). For the questionnaire, which generally follows a quantitative approach, validity is achieved by ensuring a good research design and

eliminating bias through the way the questions are worded, phrased or located within the questionnaire. The researcher has to also ensure that the data analysis techniques used and the way the data is interrogated is valid.

External validity is a key component of quantitative research. How does the data allow for more general and wide observations to be made? Ensuring that the sample size is appropriate and it is a random sample of the target

population will ensure external validity of the study.

In qualitative research, contextualised validity refers to the credibility of the responses provided and ensuring the responses accurately reflect the points being made. By interviewing a random sample of higher-level personnel and asking the right questions in a non-biased way helps ensure that the responses provided are valid. Recording the interviews and having the transcripts checked and approved by the higher-level personnel minimized the threats to the context of the data due to misrepresentation ranging from a ‘lack of descriptive validity of settings and events’ (Ihantola and Kihn, 2011, p.7).

Generalizability and transferability validity in qualitative research are important concepts and are concerned with ensuring the researcher has related the findings of the data collection methods, e.g. structured interviews, with theory and how new evidence ‘enhances our understanding of the research question’ (Ihantola and Kihn, 2011, p.8).

The suggestion that ‘academic study should link more closely with the workplace’ needs evidence to ensure its validity. Through a process of triangulation, defined by Biggam (2008, p.101) as occurring ‘when you use different sources of data to get a range of perspectives … and so achieve a more robust picture’, the reliability of the data and the process of gathering it increases. It serves to collaborate the data from different sources, thus minimizing bias and increasing the perceived quality of the research. The downside is that it is time-consuming and as Thurmond (2001, p.256) points out

can cause ‘disharmony based on investigator biases because of theoretical frameworks, and lack of understanding about why triangulation strategies were used’. A positivist and phenomenological approach will allow triangulation of the same phenomena to be made (Amaratunga et al., 2002). Providing

feedback loops enhances the qualitative-quantitative continuum. Data collected through a quantitative study follows the qualitative-quantitative continuum

approach if the research question has been determined through interviews. The qualitative foundations can enhance the project (Jha, 2008, p.50).

As part of the data gathering process, a questionnaire to final year built environment students at Anglia Ruskin University on how the students learn and apply knowledge in a workplace setting asks whether ‘professional practice is explained during their course’. The students’ response (82% full time and 72% part time) indicates clearly that professional practice is emphasised during their study, however they do not fully relate theoretical mode 1 knowledge to tacit mode 2 knowledge.

The response from higher-level personnel and final year students suggests that students would benefit from a period of WBL, but is this truly valid? Internal validity may have been satisfied, but what about external validity, where the researcher allows ‘valid generalisation to other times and places’ (Bechhofer and Paterson, 2000, p.18). To help ensure validity for qualitative research design, Jha (2008) has drawn up a list of criteria, based on the work of Guba and Lincoln (1982; 1989), Goetz and LeCompte (1984) and McMilan (1992), which the researcher can use to probe the validity of the methods employed. Within the investigation, the eleven criteria identified by Jha (2008, pp.122-125) as good practice for qualitative data collection have been implemented as follows:

1. Neutrality

The aim is to ensure objectivity in the data. All participants have to consent to be involved in the research and the Faulty Research Ethics Panel (FREP) at Anglia Ruskin University approves all methods of data collection prior to it

taking place. Objectivity is maintained in the research samples chosen as identified earlier in the chapter.