Qualitative methods

The qualitative aspect of the project consists of a phenomenological approach. The project aims to explore the impact of the literacy and numeracy strategy, in particular the use of assessment, on teachers and pupils. A phenomenological approach is suitable as it describes “the meaning of the lived experiences for several individuals about a concept or phenomenon” (Creswell, 1998, p.51). The qualitative aspect of the research takes the form of document analysis of the literacy and numeracy strategy and assessment policy, interviews, a focus group and participant-produced drawings.

5.4.3.1 Document analysis

Ball argues that policy analysis is too complex for one theory and that what is needed is “a toolbox of diverse concepts and theories” (1994, p.14). He offers two conceptualisations of policy: policy as text; and policy as discourse (section 3.2.2). I will use these conceptualisations of policy to outline the research design of the policy analysis. Chapter 6 offers a document analysis of the policy elements regarding assessment in the draft plan, Better literacy and numeracy for children

and young people (DES, 2010) and in the published version, Literacy and numeracy for learning and life (DES, 2011). It examines the assumptions on assessment and

130 notes any differences. It examines submissions from a number of key stakeholders, representatives of which were interviewed as part of this doctoral study. These include the INTO, the NCCA and the Teaching Council. Throughout the document analysis reference will be made to Foucault’s archaeological and genealogical method and Bourdieu’s theory of habitus and capital.

5.4.3.2 Interview

Data for this doctoral study was gathered through interviews with primary school pupils and high profile interviewees. The primary school pupils were interviewed in a focus group setting, which is described in the next section. The high profile interviewees consist of policy makers who were involved to varying degrees in the development and implementation of Literacy and numeracy for

learning and life. Table 5 outlines the names and roles of these interviewees, as well

131

Table 5 List of high profile interviewees

Name Role Date of

interview

Interview duration Ms. Sheila

Nunan

General Secretary, INTO 8th May 2013 42 minutes

Mr. Tomás Ó Ruairc

Director, Teaching Council

9th May 2013 59 minutes

Dr. Peter Archer Director, ERC 14th June 2013 64 minutes Dr. Harold

Hislop

Chief Inspector, DES 11th July 2013 56 minutes

Mr. Edward Murtagh

Assistant Chief Inspector (retired), DES

22nd August 2013 55 minutes

Mr. Ruairí Quinn, TD

Minister for Education and Skills (2011-2014) 19th February 2014 33 minutes Dr. Sarah FitzPatrick

Deputy Chief Executive Officer (Primary

Education), NCCA

25th July 2014 90 minutes

Ms. Jan

O’Sullivan, TD

Minister for Education and Skills (2014-present)

27th March 2015 23 minutes

This section continues with an explanation of my approach to interviewing for this doctoral study. A qualitative interview “gives the informant the space to express meaning in his or her own words and to give direction to the interview process” (Brenner, 2006, p.357). This is contrasted to a quantitative approach, where the interviewer attempts to extract pre-existing knowledge and attitudes. In a qualitative stance the interviewer gives more of him or herself to the process and can be seen as fellow travellers on the road of discovery (Kvale and Brinkmann, 2009). The high-profile interviews and the focus groups with pupils are semi-structured and utilise open-ended questions. Brenner (2006) outlines four theoretical assumptions underlying interviews: cultural anthropology, cognitive anthropology, cognitive science and developmental psychology. My interview technique falls into the cultural anthropology camp as the goal is to “understand the shared experiences, practices, and beliefs that arise from shared cultural perspectives” (Brenner, 2006,

132 p.358). Due to the semi-structured nature of the interviews, the interviewer needs to be “knowledgeable about the interview topic, and to be familiar with the methodological options available, as well as to have an understanding of the conceptual issues producing knowledge through conversation” (Kvale and Brinkmann, 2009, p.16). Knight underscores the importance of an interview schedule as “the set of prompts itself amounts to a theory, explicit or tacit, of the subject being investigated” (2002, p.62). The interview schedule for the high profile interviews is included as Appendix 3.

Brenner explains that an inductive or deductive approach can be utilised (2006, p.360-361). My approach is deductive as I am working from a theoretical perspective about teaching, learning and assessment as outlined in Chapters 1, 2 and 3. Cohen and Manion identify four types of interviews: structured, unstructured, non-directive, and focused (1994, p.273). Mertens (2010) adds semi-structured to this list. I use semi-structured interviews as these are guided by the researcher’s literature but contain space for the participant to create new avenues of inquiry. I follow the interview procedures outlined by Cohen and Manion: decide the purpose of the research; translate general goals into detailed and specific objectives; prepare the interview schedule; decide the question format and response mode; construct the questions; set up and conduct the interviews; code and score the data; analyse and interpret data (1994, p.284-286).

The researcher is the instrument of data collection (Mertens, 2010; Eisner, 1998). As a researcher I am seeking to investigate the creation of the literacy and numeracy strategy; understand how the practice of teachers in assessment compares with the best practice as outlined in the research; and determine the impact on children as learners. The qualitative approach utilises dependability and

133 confirmability (Mertens, 2010; Creswell, 1998). This is achieved through triangulation of the data; prolonged engagement; member checks; and peer debriefing (Mertens, 2010). Informed consent is sought from all participants and the data collected is only used in the ways described to the participants (Knight, 2002, p.142). Consent was sought from the high profile interviewees for the use of their names and titles (Appendix 4). The high-profile interviews and focus groups with pupils were recorded on audiotape. The participants were given an opportunity to respond to, or elaborate on, points raised during the interviews at a later date.

Knight states that piloting should show “whether the prompts are useful and, perhaps more important, good pilot interviews can identify topics that need to be added to the list but which had been missed by the literature review” (2002, p.66). For the high-profile semi-structured interviews, the questions were piloted with the project supervisor in autumn 2012. The focus group approach with pupils was piloted with members of the researcher’s own school community in May 2013 and is described in the next section. Triangulation involves checking information that has been collected from different sources. In this project, triangulation entails checks between the interviews with two focus groups of pupils, elicitation on the meanings of the pupils’ drawings, and the results of the teacher survey. Rival explanations are explored to determine the convergence or non-convergence of data (Mertens, 2010, pp.258-259).

The data analysis of the interviews is theme-oriented and based on the literature review. Knight avers that qualitative data are “easier to categorise and make sense of if you know what you are looking for, and easier still if the research was designed explicitly to look for that and that alone” (2002, p.182). He outlines two stages of analysis: coding the data and reflecting upon interpretations of the data

134 (Knight, 2002, p.182). For Knight, data analysis of qualitative data is “a continuing, sensemaking toing and froing between the data, the categories, the emerging stories, and the literature” (2002, p.186). Creswell (2009) lists the following stages in qualitative data analysis which will be utilised in this research: a) organise and prepare the data for analysis; b) read through all the data; c) begin detailed analysis with a coding process; d) use coding to generate a description of the setting or people as well as the categories or themes for analysis; e) advance how the descriptions and themes will be represented in the qualitative narrative; and f) make an interpretation of the data (p.185-190).

5.4.3.3 Focus group interviews with primary school pupils: data collection and analysis

A focus group interview is utilised in this doctoral project to elicit the perspectives of pupils. Kamberelis and Dimitriadis state that “Focus groups have...allowed researchers to explore the nature and effects of ongoing social discourse in ways that are not possible through individual interviews or observations” (2005, p.902). Vaughn et al. give five reasons for utilising focus groups: i) they offer variety and versatility to both qualitative and quantitative research methods; ii) they are compatible with the qualitative research paradigm; iii) they offer opportunities for direct contact with subjects; iv) they offer distinctive advantages for data collection by encouraging interaction and support for participants; and v) they offer utility (1996, pp.14-20). These reasons match the underlying paradigm of this project as it utilises a mixed methods approach and seeks to understand pupils’ perspectives on assessment. Kamberelis and Dimitriadis

135 highlight three overlapping domains where focus groups predominantly occur: pedagogy, politics, and qualitative research practice (2005, p.888). This research is at the intersection of these three areas.

Scott comments that, usually, parents respond for children in surveys and questionnaires (2000, p.99). For her, “the best people to provide information on the child’s perspective, actions and attitudes are children themselves” (2000, p.99). Scott identifies a number of problems that may arise when conducting research with children. These include the language being used, the stage of cognitive ability, the fear of adult sanction, and issues of confidentiality and ethics (2000, p.100). However, she argues that by age 11, most children are fully able to articulate (2000, p.102). She outlines three aids to interviewing: routing, visual aids and prompts (2000, p.106). Routing is needed to ensure that children are asked appropriate questions. Visual aids are helpful if there are vocabulary problems. Prompts are essential when inadequate answers are given due to a lack of communication skills. The study includes visual aids through the use of three assessment tools that the pupils have encountered in primary school: spelling book, mathematics assessment book and standardised tests. Scott highlights the importance of clarity in instructions with young children. According to her, researchers have to “ensure that questions really do measure the desired concept; that the questions are unambiguous, and that children interpret the questions in the way the researcher intended” (2000, p.107).

Vaughn et al. cite Krueger (1988) in outlining how to summarise key ideas after the focus group interviews and before analysis begins:

1. Find the big ideas

136 3. Consider the context

4. Consider the consistency of responses (1996, p103).

The focus group approach was piloted in May 2013 in the researcher’s workplace. This pilot consisted of five sixth class pupils. The questions elicited detailed and thoughtful responses and the visual aids (three assessment tools) were an excellent stimulus for conversation. In gathering focus group data to be analysed in this dissertation, the study draws on convenience sampling of two schools, one urban and one rural (other than the researcher’s workplace). It consists of focus groups of five children in each school who are selected by random sampling once consent is received. Consent was received from both parent and pupil after information letters were distributed to all of the class (Appendices 5 and 6). The interviews took place in June 2014. One interview lasted approximately 30 minutes, whereas the other interview was 40 minutes approximately in duration. Bronfenbrenner (1979) describes how the subject of research is influenced and acted upon at various levels: microsystem (interactions in the immediate setting); Mesosystem (interactions between settings in which a person participates, e.g. home/school); exosystem (events that occur which affects what happens to the person without their participation); and macrosystem (social institutions). This approach is employed when analysing the data collected during the focus group interviews. The focus group interview also asks questions regarding national assessment policy (see interview protocol in Appendix 7).

Although there are limitations to this sample in that the results cannot be generalised beyond the given population pool (Mertens, 2010, p.325), it is hoped that by interviewing pupils in two schools comparisons can be made. Credibility is

137 addressed by engagement with the pupils through a variety of means including participant-produced drawings and interviews. Transferability is addressed through the use of multiple cases (Mertens, 2010, pp.256-262). The data is analysed in the same manner as the interview data.

5.4.3.4 Participant-produced drawings

Participant-produced drawings is a research technique that is employed in this dissertation to gather data with pupils. This method has become increasingly popular with researchers in the social sciences (Hogan and Pink, 2012; Ganesh, 2011). Hogan and Pink state that “it is pertinent to note that these approaches are part of a trend in social science and humanities research that focuses on the experiential, the sensory, and ways of knowing, being and remembering that cannot necessarily be articulated in words” (2012, p.232). Ganesh (2011) outlines how this method can be used as a descriptive or analytical tool in research. As a descriptive tool, the drawings can be utilised to elicit individuals’ understandings of a specific idea or construct. As an analytical tool, produced drawings can be used to compare an individual’s changes over time. The drawings produced by the pupils for this dissertation are used as a descriptive tool to elicit the pupils’ understandings of and feelings toward assessment in primary school. The drawings are also used in this study as an instrument to assist and develop contributions during the focus group. Ganesh (2011, p.223-4) describes an approach to make sense of drawings, which is utilised in this study:

138

 Axial coding – compare for cross-cutting concepts

 Image elicitation interviews – to confirm researcher’s interpretations of the drawings

Noth argues that “what pictures depict is very often already a symbolic sign whose interpretation requires cultural knowledge” (2011, p.304). Banks (2001) also argues that the social context in which the image is created is of vital importance when interpreting the image. Banks (2001) distinguishes between the internal narrative of the image and the external narrative. The internal narrative is related to the content of the image whereas the external narrative refers to the social context that produced the image. The internal narrative of the drawings is analysed using visual salience and pragmatics. The visual salience of a figure, its foregrounding, or its position in the centre, correspond to the importance in pictorial meaning. Visual pragmatics deals with the way pictures are used and the effects which they have on their viewers (Noth, 2011). The external narrative of the images are analysed through image-elicitation interviews with the pupils. By discussing the drawings with the participants, the researcher’s interpretations gain added validity (Ganesh, 2011). Ganesh states that when drawings are analysed in context, along with other sources of data such as author descriptions of the drawing or image-elicitation interviews, “the validity of the inferences that one can make from such data is enhanced, as the inferences will not rely solely on the researcher’s interpretations” (2011, p.238).

Ganesh states that participant-produced drawings have the following qualities: “they can be projective; they permit expression of feeling and imagery;

139 they allow for defining and redefining shared attitudes held by society; and they can be analysed using psychological, sociological, and cultural lenses with attention to the phenomena or concepts under study” (2011, p.238). In this study, the ten children were asked to complete drawings on assessment at the start of the focus group interviews. The pupils had received information letters about the nature of the focus group interview prior to the interviews taking place. The pupils were asked to draw their thoughts about assessment or how it made them feel. They were instructed to complete the activity in a short period of time (2 – 3 minutes). Some pupils asked whether they could write responses instead of drawing them and this was accepted. These drawings are analysed in Chapter 9.