Reliability in qualitative research refers to the fit between the researcher’s data and what actually occurs in the natural setting that is being researched; that is to say, the degree of accuracy, comprehensiveness of coverage and robustness. Reliability is a necessary pre-condition of the validity of a piece of research. Reliability concerns the question ‘whether the researcher would have made the same interpretations of data if they were gathered at a different time or in a different place?’, thus ensuring stability. Reliability is a synonym for consistency and applicability over time, over instruments and over respondents. In qualitative methodologies, reliability includes fidelity to real life, context- and situation-specificity, authenticity, comprehensiveness, honesty, depth of response and meaningfulness to the respondents (Cohen et al., 2007, pp.148-149).
Reliability of this study was ensured in many ways. Semi-structured interviews were supplemented with open-ended questions and document reviews to ensure robustness of data. All interviews were audio-recorded and they were listened to several times to ensure completeness, fidelity, meaningfulness and comprehensiveness of the transcripts, which were enriched by using field-notes. Interviews were held in the respondents’ natural settings, using the Sinhala language, and thus facilitating the flow of views. Situational questions were raised during interviews to further clarify participants’ responses. Policy documents, circulars and other materials were used to corroborate interview data and to understand the context broadly. Accordingly, interviews with policy-officials helped in exploring the actual processes of policy development and dissemination of PSI and CFSI, while school-participants’ data helped in comparing how
they are being implemented. Accordingly, the data helped in exploring what actually happens within PSI and CFSI in the research contexts.
Validity is an important key to effective research. In qualitative research, validity is addressed through honesty, depth, richness and scope of data of the respondents and the disinterestedness or objectivity of the researcher (Winter 2000, in Cohen et al., 2007, p.133). Validity is attached to respondents’ accounts or perspectives rather than to data or methods. Thus, in qualitative data collection, the intensive personal involvement and in-depth responses of individuals secure a sufficient level of validity and reliability. Maxwell (1992, in Cohen et al., 2007, p.134-135) suggests that ‘understanding’ is a more suitable term than validity in qualitative research and that this notion is explained by five kinds of validity: descriptive, interpretive, theoretical, evaluative validity and generalisability.
The validity of this study was ensured in the following ways: to ensure descriptive validity, the factual accuracy of participants’ accounts was supplemented and corroborated with documentary evidence and other participants’ data; however, these cross references may not be sufficient to triangulate the data. Interpretive validity was ensured through my substantial awareness of the policy context and the research context, pre-conceived meanings of my participants and myself, and my participants’ positions and my own. Theoretical validity was ensured by comparing the study findings (school case-study findings) with the propositions and policies. Similarities in the practice of SBM initiatives in different school case-study locations suggest the application of findings to other settings to a certain extent, thus ensuring generalisability or rather ‘transferability’ (Cohen, et al., 2007, p.137). This study, rather than being descriptive, has attempted to be evaluative and judgemental in exploring the strengths and weaknesses of the policies and practices of the two SBM initiatives. Thus, to a reasonable extent, it has ensured evaluative validity.
3.5.2 Methodological limitations
I interviewed selected key actors engaged in PSI or CFSI at policy and school levels. Two separate groups of officials from national and zone levels were engaged in these two initiatives, so it was not possible to interview them on both initiatives.
Moreover, owing to the limitations of time, capacity and the length of the study, it was not possible to interview other officials, teachers and parents. This is a limitation of this study. Even though I first proposed mixed methods to study the subject, considering the time limitations I chose qualitative approach and school case-studies, which are more illustrative than representative and have generated insights into the contexts. Hence, this is a multi-site study which provides rich data, thereby allowing the readers of this research to decide ‘whether transferability rather than generalisability of knowledge is claimed’ (Cohen, et al., 2007, p.137). However, collecting and analysing data on two initiatives and comparing their practices with their intended policies and between them, were complex processes. Nonetheless, I attempted to ‘do justice to the purpose’ (Creswell, 2009, p.214) to the best of my ability.
3.5.3 Ethics
In my professional capacity, I am well-known to all key informants at policy- level and to the DPA representatives. However, since they understood the purpose of my interviews and my ethical obligations on securing their confidentiality, they genuinely shared their responses including even criticisms of the policy-development process. I was not known to my school-level participants; however, I realised that when they knew my position at the MoE they paid special regard to me. To the best of my ability, I explained to them the purpose of the study and my intention to reveal what was actually happening at schools within PSI/CFSI. In addition, as guided by Cohen et al. (2007, pp.352-362), throughout the interviews I was careful about my language, treated my participants in a respectful, polite and friendly manner and maintained a good rapport with them, thus creating a free and friendly environment in which they could divulge their information. I hope that I was successful in this without any conflicts due to my professional and researcher roles.
I commenced my field-work following the ethical approval of the Sussex University and the approval of the MoE Secretary to collect field-data from the respective national organisations and schools. Moreover, I met heads of all organisations and schools to obtain their agreement to the data collection. Before commencing an interview with each participant, I explained the purpose of the study and assured them that their privacy, identity, anonymity and locations would be kept confidential.
Moreover, no financial or material benefit was offered and hence all participants contributed voluntarily, for which I am indebted to them.
3.4 Conclusions
Methodologically, I followed a qualitative research approach, with a case study approach to PSI and CFSI. The main method of data collection was interviews. Documentary reviews were used to corroborate interview data. I believe that I was taking an insider-outsider position, since I had certain involvements in PSI and CFSI at policy level but no involvements at implementation level. During my interviews, I attempted to treat my participants with great care and respect so as to minimise the impact of my position on the data that I was gathering. Thus, I believe that I was able to obtain open and genuine responses from my participants while keeping my research focus in my mind. All of them participated willingly and many of them revealed large amounts of information. I followed the ethical guidelines well in this study. As a result of these attempts, I believe, this study ensures validity and reliability to a greater extent and also draws certain transferable conclusions and recommendations within its own limitations. In this research, I believe that my ontological assumptions agreed with constructivism while epistemologically, I adopted the premise of interpretivism. In sum, this chapter attempted to disclose the researcher position and improve the transparency of what I actually did and how the conclusions were arrived at.
Chapter 4
Initiation of PSI and CFSI and their expected implications
4.1 Introduction
This chapter analyses data on the initiation and expected implications of PSI and CFSI. The second section provides a brief background to the research context in terms of country, education system and the province (Senrock Province). The third section introduces the background of SBM in Sri Lanka. The fourth section analyses Programme for School Improvement (PSI), addressing the Question (a) ‘Why was PSI initiated in Sri Lanka and what were its expected implications for policy and organisational practices?’ The fifth section then analyses the Child-Friendly Schools Initiative (CFSI) addressing Question-(c) ‘Why was CFSI introduced in Sri Lanka and what were its expected implications for policy and organisational practices?’ Data from key informants representing the national organisations and the province is combined with country-specific documentary evidence in these analyses. The sixth section concludes the chapter.