Data collection and types of evidence

While different authors have slightly divergent views on data collection methods or types of evidence for case study research, they all largely agree on the sources of data. Bassey (1999) outlines three major methods of collecting data: asking questions, observing events, and reading documents. These three methods are more commonly known as interviews, observation, and documentary data (Punch, 2009).

Yin (2009) takes things further with his suggestion for six types of evidence: documentation, archival records, interviews, direct observation, participant observation, and physical artefacts (although he acknowledges the last one has less relevance in most case studies). Based on the above discussion it appears that data collection methods or types of evidence for case studies can be loosely grouped into three broad categories:

A. Documentary evidence

These could be documents closely related to the participants of the study such as diaries, personal notes, biographies or autobiographies, letters or email correspondence and suchlike; or they could be less directly related documents such as studies or evaluations of similar cases and news articles. Yin (2009) proposes several strengths of documentary data, in that they are:

x stable, and can be viewed repeatedly;

x unobtrusive i.e. they are not created as part of the case study;

29

A significant weakness however, of documentary data is that it may not always be able to be taken at face value, so separating fact from opinion, from fiction or misleading documents, or recognising bias may be significant challenges for researchers.

B. Interviews or asking questions

Interviews can be conducted in a range of formats, from more structured or formal through to unstructured or open-ended interviews. The type of interview utilised by the researcher will depend on the type of research being conducted, and what type of data the researcher is trying to elicit from the participants. Punch (2009) offers a continuum model for interviews with structured interviews, through focussed or semi-structured interviews to unstructured interviews (Punch, 2009). Structured interviews are highly planned and standardised. Data from the interviews are put into pre-coded categories and the interviews typically do not go into much depth. At the other end of the spectrum, unstructured interviews do not consist of pre-planned, standardised questions, but rather general questions to initiate a discussion. Further questions are likely to be determined depending on how the interview is unfolding, and the interview is likely to be more in-depth. Focussed, or semi- structured interviews enable some pre-planning of questions and data categories to ensure sufficient data are captured. Semi-structured interviews allow participants to drive the direction of the interview to some extent, discuss ideas in more depth than a structured interview, and in a more organised and disciplined manner than an unstructured interview (Punch, 2009).

The type of interview or questioning technique used by the researcher will depend on the type of research being done, the depth of information being sought and the number of participants being interviewed. Interviews may be done individually with the interviewer or they may be conducted as group (or focus group) interviews. Each of these approaches has their strengths and limitations. Individual interviews are designed to elicit responses about what people think, or how they feel about specific phenomena. Advantages of using an individual approach are that an interviewer is focussed solely on the person being interviewed and is more likely to get responses that are not influenced by others. One on one interviews are also more likely to work better for individuals for whom an interactive group environment is a less comfortable situation. Conversely, while a group interview, may be a less comfortable environment for some people and runs the risk of not being able to explore individuals thoughts or feelings in more depth, a group dynamic can often stimulate discussion and encourage participants to voice their views or thoughts that they may not have done individually (Punch, 2009).

30 C. Observation

In a similar manner to interviews, observational data may come from different methods. Punch (2009) describes structured and unstructured approaches to observation. Unstructured or “naturalistic” observation is non-interventionist, where the observer does not manipulate or stimulate the behaviour being observed. A more structured approach has the observer as a participant in the situation, although the degree of active participation may vary on a continuum with a researcher in a more participant role, while being an observer; or mainly as an observer with less of a role as a participant. Wolcott (in Punch, 2009) notes that the researcher can range from an active participant, through a privileged observer to a limited observer, with each step having less active participation in the situation being observed. More active participation may allow the researcher a greater insight into what is happening, and to observe behaviour or events that they would not have otherwise been able to; however there is also a greater possibility that, through their participation, what they are observing may be influenced by their involvement (Punch, 2009). Yin (2009) describes a similar approach when he compares “direct observation”, where the researcher is merely observing phenomena in their natural setting – to participant observation, where the observer actively participates in events to some degree or other (Yin, 2009).

This study principally uses data gained from interviews with the participants. These are semi- structured interviews, designed to gather in-depth answers from the interviewees and allow them to more fully explore their thoughts, feelings and experiences in regard to the assessment methods. Due to the relatively large number of participants interviewed (23) and the need for some consistency of information from all the participants, the interviews need to have some structure and follow a similar format without restricting participants’ views. Having some structure to the interviews also enhances the reliability of the study as all participants views are sought on some key areas, meaning any conclusions drawn from these data are based on a larger sample. Similarly, this creates opportunities for some comparative analysis across the interviews, as again, there are a larger number of answers to similar questions to be able to compare.

There are also data in the form of documentation. There is documentation in the form of the assessment evidence portfolios to which all three participants; learners, employer/verifiers and assessors have contributed. This documentation underpins the information gained from interviews with participants. Observation has not formed part of the data for this case study as the researcher has not been present during the period of the case study. This study is seeking to discover the thoughts and perceptions of the three groups of participants on a changed assessment process.

31

Direct observation of the participants in this instance would not have yielded significantly different data, and would have been unrealistic from a logistical perspective.