According to Cohen Manion and Morrison (2000), data analysis involves ‘making sense of the data in terms of the participants’ definitions of the situation, noting patterns, themes, categories and regularities’ (p. 147). As Wellington (2000) anticipated, this project, like much qualitative research, produced a considerable amount of data owing to the centrality of talk and subsequent transcription. As
well, the 52-item questionnaire generated a lot of data that needed to be analysed in a way that complemented the analysis of interviews and observations.
Thematic analysis, as detailed by Braun and Clarke (2006), is consistent with a constructivist, grounded theory approach. They stress the non-linear character of qualitative data analysis and the need to embed analysis in the research process right from the start, describing analysis as a recursive process of moving back and forth between data and the analysis of text. A flexible approach to possible themes needs to be maintained throughout the phases of analysing and writing up.
Notwithstanding these qualifiers, they offer a six-phase model as guide for
thematic analysis (pp 87-93) (Table 3.4). In broad terms, the analytical procedures I undertook are reflected in this model.
Table 3.4: Stages of thematic analysis, based on Braun and Clarke (2006)
1. Familiarise yourself with your data
Transcription enabling close reading leading to interpretation 2. Generating initial codes Codes refer to ‘the most basic segment or element of the raw
data that can be assessed in a meaningful way’ (Boyatzis 1998, cited in Braun & Clarke, 2006, p. 88)
3. Searching for themes Refocuses the analysis at the broader level of themes. 4. Reviewing themes Checking that themes work in terms of coded data.
Is regrouping required? Are new themes emerging? 5. Defining and naming
themes
Ongoing analysis regarding and specific description of each theme.
6. Producing the report Final analysis and write up.
3.5.1 Analysis of questionnaire
As mentioned previously, the initial questionnaire enabled the collection of data related to each individual teacher’s beliefs about the relevance and self-reported frequency of application of identifiable Orff principles, processes and practices in their particular teaching context. To enable quantitative analysis in relation to themes that would be relevant to the research findings overall, I went back and forth between transcribed data and the questionnaire items. The initial allocation of initial codes for the transcribed data suggested several different possible thematic categorisations for the 52 items. I finally settled on following nine:
• Orff pedagogical principles • Materials and resources • Tonalities
• Structures
• Activity: Dance/Movement • Activity: Speaking/singing
• Activity: Playing instruments • Activity: Listening
• Activity: Reading and writing
The Likert ratings from each individual questionnaire were collated to provide a body of raw data (See Appendix N) that provided the following quantitative findings:
• Mean scores of each individual’s ratings for relevance and frequency of application of items within thematic categories. (See Appendix O)
• A ranked listing of the mean scores for relevance and frequency of application of all participants ratings for each item (See Appendix P)
• A ranked listing of the combined means scores of the nine thematic categories
(See Appendix Q)
The advantage of a mixed-methods approach to data collection is that a
questionnaire analysis can reveal statistical patterns of response that can be used to corroborate, illuminate or problematize findings that emerge from an analysis of qualitative data. This was realised in the present study, as Chapters 4 and 5 illustrate.
3.5.2 Analysis of interviews, observations and focus group discussion data
I transcribed data from the interviews and observations and follow-up interviews as soon as possible after the event. This embedded much of the raw data in my memory and enabled me to generate initial codings as I went along (See Appendix L for a list of initial codes). I also found myself ruminating upon emerging themes as I continued the data collection process. Appendix K, as well as being a record of the foci for the focus group discussion, also indicates the emergence of
preliminary themes. After several stages of recursive analysis using colour coding and segmenting (cutting and pasting) of data into a variety of categories, I decided to use the thematic categories derived from the questionnaire (mentioned above) as one lens through which to view relevant data. Appendix L documents, in a more tidy way than was the case, the way I coded and analysed other thematic patterns that emerged. Gradually, as a result of a process of sifting, grouping and regrouping, themes emerged as evident in the analysis of findings in Chapters 4 and 5.
3.5.3 Confirmability
In qualitative research, confirmability is related to triangulation. In order to confirm, one needs to look or examine from a variety of angles. This is not to suggest that there is one true story that needs to be dug out, but rather that a rich and informative description of a natural setting needs to have evidence or data from a variety of sources. In relation to a particular topic, confirmability relates to the extent to which the research uncovers or makes known most of what might be seen or perceived in the research setting that is of relevance to the research problem or questions.
In other terms, the intention of this research project was to produce a ‘story’ about the lived experience of the participants in relation to the research questions, which is trustworthy and exemplifies the following forms of validity as referred to by Cochran-Smith and Donnell (2006):
• Democratic validity: The methodology of this project implied an honouring of the perspectives of those involved because the way it systematically and
comprehensively sought access to the participants’ perceptions using a variety of data collection methods.
• Outcome validity: In the case of this project, the participants were given access to the findings (related to the raw data they provided) throughout the process. • Catalytic validity: Ongoing dialogue with participants regarding the data
transcription and the focus-group discussion supported the process of recursive analysis and led to a deepening of understanding of all participants through the sharing of knowledge. At the conclusion of the focus group discussion,
participants spoke very positively of their involvement in the project. They expressed the view that ongoing involvement in the group would support them in their professional lives.
• Dialogic validity: In this study the style of the semi-structured interviews and the commitment to a dialogical process of follow-up for checking and rechecking led to analysis and interrogation of solutions in a respectful and genuinely curious way.
3.5.4 Validity and reliability
According to Cohen, Manion and Morrison (2000), research validity is concerned with the extent to which a piece of research may be regarded as worthwhile. This, they argue, should be seen as a matter of degree rather than an absolute quality.
Lincoln & Guba (1985 and 1988 cited in Bresler & Stake, 2006) rejected the positivistic criteria of validity and reliability in favour of criteria compatible with the axioms of naturalistic research. They argued for a range of qualities as
constitutive of research validity and reliability (pp. 297-298).
A concern for credibility is displayed in the way the researcher, through the uniqueness of their understanding, seeks to build an account, which will be credible to their readers. This account is offered for substantiation or
modifications of existing understandings and/or generalisations about the topic. As indicated earlier, my assumption of the stance of connoisseur (Eisner, 2012) was premised on my years of experience as an Orff practitioner and mentor which enabled me to develop my understanding of the approach.
Transferability as a means of research evaluation asks to what extent does the
account produce knowledge, which resonates with the reader’s own understanding or situation? Can inferences from this piece of research be made in relationship to the situation of the reader? In this study, I have attempted to offer meaningful descriptions of people, settings and personal stories of success and frustration. These stories hopefully touch a chord in readers and enable them to associate these vicarious experiences with ones of their own, in a way that illuminates them in some way.
Dependability is the extent to which this account can be seen as ‘true’ to what
others with the same research interests might have seen or observed. According to Bresler and Stake (2006), ‘the researcher is greatly privileged in what to attend to but the extent to which an audience may or may not validate the account’ (p. 298) is yet to be seen. In submitting this thesis for critical examination, I am re-
entering a conversation with scholars and practitioners alike, which is reflected in the way I have approached my literature review and discussion.