Validity and reliability are essential for all research, and care must be taken to ensure that the findings accurately describe the phenomenon being researched (Cohen, Manion &
Morrison, 2011). However, validity and reliability have traditionally been applied within a positivist paradigm, using the following criteria: internal validity, external validity, reliability, and objectivity (Lincoln & Guba, 1985; Merriam, 1988). Because these four positivistic criteria are not easily applied to qualitative research, alternative conceptions of trustworthiness are applied that include: credibility, dependability, confirmability, and transferability (Guba & Lincoln, 1989).
In qualitative research, credibility can be enhanced by employing techniques such as prolonged engagement, persistent observation, member checks, peer debriefing, negative case analysis, progressive subjectivity, and triangulation (Lincoln & Guba, 1985;
Onwuegbuzie & Leech, 2006; Teddlie & Tashakkori, 2009). Some of these steps have been described in the previous sections of this chapter and some are described below.
Prolonged engagement in a naturalistic research setting helps the researcher become more familiar with the setting, and to build trust and rapport with participants (Lincoln &
Guba, 1985). Persistent observation provides the opportunity for the researcher to apply detailed focus and to identify key elements of the study (Lincoln & Guba, 1985). As described in Section 4.4.3, both of these techniques were used in this study, as were member checks, peer debriefing, and progressive subjectivity.
Negative case analysis involves a continuous revision of the emerging themes until all data are accounted for within the theory (Kidder, 1981) and all outliers have been explained (Lincoln & Guba, 1985; Miles & Huberman, 1994). In this inquiry, negative case analysis was performed during the analysis phase. This technique was particularly helpful in digging deeper and refining the theorizing regarding changes in students‘
attitudes toward learning science, which appeared at first, to be incongruous. An important part of the negative case analysis involved looking at multiple sources of data, or triangulation.
In education research, triangulation involves the use of two or more data collection methods, sources of data, data collectors or times of data collection (Denzin, 1970).
While many tables have four legs, stability requires just three. A guild of three complementary plants - such as the Hopi ‗Three Sisters‘: corn, beans and squash - provides a stable cultivated ecology for growing food. A ship lost at sea can find its way using three beacons by a process called triangulation. In research, triangulation allows for stable (robust) findings and locates conclusions out of an ocean of data. Stable research is said to be reliable. But triangulation in every case described above is not a linear progression of improvement. In other words, two plus one does not represent the same incremental increase in stability as one plus one. For example, a table with one leg benefits little from adding one more leg, but greatly from adding a third. Corn and squash planted together do not thrive like they do when beans are added to fix nitrogen in the soil to feed them. And a lost ship is still lost with only two points for reference. In all of these cases, there is a tipping point of integrity reached by triads when symbiosis turns to synergy. The whole becomes greater than the sum of the parts, and the system punches above its weight. Three, it appears, really is a magic number (Dorough, 1973; Johnson, 2006).
In this study, triangulation involved multiple methods, sources and times of data collection. The multiple methods were described in Section 4.4, and included:
questionnaires, interviews and observations. The multiple sources were discussed in Section 4.7, and included: the students‘ self-reporting, the teacher reporting on students and my observations of students. My observations took place over 12 weeks in the classroom, a time period long enough (prolonged engagement) to provide time triangulation of data for the teacher and for certain notable students who are profiled in Chapters 6, 7 and 8. By using multiple methods, sources and times of data collection, the credibility and dependability of the conclusions of this study should have been enhanced (Miles & Huberman, 1994).
Triangulation was used to address the credibility of data relating to some pedagogical practices utilized during the intervention, as well as students‘ ecological literacy, scientific literacy and attitudes toward science. More specifically, these included students‘: science learning; feelings of care for the environment; actions for the environment; systems thinking skills; recognition of ecological limits and possibilities;
perspectives on science and technology; enjoyment of learning science in school. Themes emerged from the data regarding the areas of inquiry listed above and were then crosschecked against data gathered using a different method or from a different perspective. For example, students reported on their own enjoyment of learning in both the focus group interviews and on the questionnaires, but the teacher also commented on this from his perspective in the teacher interviews, as did I from my perspective during classroom observations. This process was repeated to greater or lesser extents for all major themes that emerged from the areas of inquiry.
While advances in trustworthiness occur between one and two, and two and three forms of data, subsequent improvements tail off thereafter. A more-the-merrier attitude turns to four‘s-a-crowd. That said, redundancy is not harmful in research or permaculture. If one plant in a guild succumbs to an insect pest or disease, or if one method is found to lack validity, then an extra component in the system proves critical. In fact, ecological validity in education research requires the consideration of as many characteristics and factors involved in the subject of study (Cohen et al., 2007). Brock-Utne (1996) promotes ecological validity when studying the adoption of new educational policies in actual classrooms. I submit that, when politics and scale are removed, that is essentially what I did in this case. In other words, I developed a new approach to teaching science, provided it to a teacher, and then attempted to chart what actually happened in his classroom.
However, ecological validity can run up against boundaries determined by ethical considerations such as anonymity and non-traceability (Cohen, et al. 2007). These considerations were paramount for this study, which took place in a small school in a small town in a small country.
Dependability in qualitative research means providing clear descriptions of how data was collected and the context in which it was collected (Lincoln & Guba, 1985). Merriam
the use of triangulation; establishing an audit trail; and, thorough descriptions of ―the assumptions and theory behind the study, their own position with respect to the group being studied, the basis for selecting participants and the social context‖ (p. 172).
As described above, triangulation was used extensively in this study, and much of this chapter serves as an audit trail by way of describing ―how data were collected, how categories were derived, and how decisions were made‖ (Merriam, 1988, p. 172).
Additionally, this chapter, along with Chapters 2 and 3, address the assumptions and theory behind the study, and Section 1.3 also provides a description on my role. Chapter 5 embeds those ideas in the intervention.
Confirmability has been suggested as an appropriate way to describe the objectivity of naturalistic research, where the researcher cannot be objective (Lincoln & Guba, 1985).
Like dependability, confirmability can be improved when a clear audit trail in provided.
This allows the reader to assess any potential influence of the researcher, and to determine for themselves the extent to which that may have affected the investigation.
In qualitative, naturalistic research, such as case study, findings are not easily transferred from one group to another. In other words, the external validity characteristic of quantitative research does not apply. Instead of the researcher taking responsibility to ensure the validity of the study, the onus shifts to the reader to determine the transferability of the findings to their own situation (Lincoln & Guba, 1985; Merriam, 1988). As discussed in Section 4.5, this is especially true of case study research (Larsson, 2009; Nisbet & Watt, 1984). However, in order for the reader to fully appreciate the findings, it is the responsibility of the researcher to provide ‗thick description‘ (Lincoln &
Guba, 1985) of the context of the inquiry and of the data. This thesis seeks to provide such description, although transferability is a recognized limitation of this study.
4.9 Ethical Considerations
Much of the integrity of both permaculture and education research relies on their commitment to ethics. The permaculture ethics – care for earth, care for people, share the surplus – are easy to rattle off. Not so for the ethics of education research, and particularly qualitative or interpretive research (Bourma, 1996; Hitchcock & Hughes,
1989; Merriam, 1988). Hitchcock and Hughes (1989) warn that ethical problems are directly related to using participant observation and interviews to collect data. These and other ethical problems are well recognized by the Centre for Science and Technology Education Research at the University of Waikato, where thorough guidelines and procedures have been established for all education research. The Centre has developed guidelines for students conducting research as well as a Human Research Ethics Committee that reviews all ethics applications. I followed all protocols set out by the Centre.
Access to participants was gained through permission from the school‘s principal and Board of Trustees. Participants and their guardians were given the opportunity to decline participation or withdraw at any stage. Guardians were given full information about the role of the children in the study and were provided with an informed consent form that allowed them to tick boxes for their child‘s participation in the following: questionnaires;
small group interviews; photos where the child cannot be identified; samples of school work reported anonymously. Different guardians gave different degrees of consent. For example, some guardians gave their consent for participation in the questionnaires but not focus group interviews, or photography, even when their child could not be recognized.
Confidentiality was ensured in the study by providing pseudonyms for all participants and locations in the reporting of all data. Data gathered from participants were kept in a locked cupboard, but participants were given the right of access to data gathered from them during the study. Additionally, they were given the right to decline participation or to withdraw from the study at any time without coercion.
4.10 Chapter Summary
This chapter has described the research methodology and methods for this inquiry. An interpretive methodology was selected for a naturalistic case study of a year 10 science class in New Zealand. Mixed methods included: student pre- and post-questionnaires, classroom observations, teacher interviews, and student focus group interviews. Of note, the pre- and post-questionnaires included concept mapping and SOLO Taxonomy exercises to assess systems thinking skills in students. As a qualitative inquiry, the quality
of both data generation and analysis was subjected to high standards of credibility, dependability, confirmability, and transferability (Guba & Lincoln, 1989).
Triangulation through the use of multiple data collection methods helped provide the credibility and trustworthiness of the data. Additionally, questionnaires and interviews were peer reviewed, and questionnaires were piloted, before their administration to address face and construct validity. As a case study, special attention was paid to avoiding researcher bias. Some steps taken to enhance the trustworthiness of this inquiry include:
extended engagement and observations in the field; leaving an audit trail; member checking; clarifying research bias; theoretical sampling; thick and rich description;
assessing rival explanations; negative case analysis; and, triangulation (Lincoln & Guba, 1985; Onwuegbuzie & Leech, 2006; Teddlie & Tashakkori, 2009).
Ethical considerations of this study aligned with the policies and protocols of the University of Waikato. An ethics application was reviewed and accepted by the Human Research Ethics Committee at the University, which included copies of letters to the school principal, the science teacher, and parents and guardians. Permission was given by parents and guardians for students‘ participation, and confidentiality was ensured by providing pseudonyms and retaining all data in a locked cupboard.
Data analysis for this study was thematic within case. It involved identifying primary patterns and nested patterns in the data, and discovering core themes. Triangulation was used for data from the mixed methods to confirm these themes across method and participant (teacher‘s perspective, students‘ perspective, researcher‘s observations). Most patterns were revealed through interview transcripts, from which point I pursued two tracks of analysis that I called vertical (within-case analysis) and horizontal (cross-case analysis). The horizontal dimension (breadth) represented how many participants held similar perceptions and attitudes. The vertical dimension (depth) represented how well other data on a particular participant supported their interview data. Vertical analysis of rich data on some students led to case stories of two students who appeared representative of others in the class. Additionally, negative case analysis revealed no significant contradictions in the data.
As introduced in Section 1.6 and emphasized in the subsequent chapters, the use of permaculture was not limited to the intervention design and execution. A permaculture perspective influenced all aspects of the inquiry including the selection of an appropriate methodology, robust methods, and a holistic approach to data analysis. The latter of these is evident throughout Chapters 6, 7 and 8 where a permaculture perspective is applied to the data seeking to recognize patterns within it. A more literal application of permaculture was used in the design of the intervention described next in Chapter 5.
CHAPTER FIVE - THE INTERVENTION
5.1 Chapter Outline
This chapter translates theory into practice. It describes how the theoretical framework constructed in Chapters 2 and 3 was interpreted into classroom learning experiences for students. The chapter does so in three steps. Section 5.2 highlights the key theoretical principles for the intervention, and describes how these were planned for in the intervention. Section 5.3 documents how I designed teaching and learning activities to suit the key principles, and inserted them into modified unit plans for the teacher. Section 5.4 describes which teaching and learning activities from the modified unit plans were used in the science class and to what extent. Table 5.1 summarizes all three sections by linking the key principles, the curriculum design, and what actually occurred in the classroom. Section 5.5 provides a chapter summary.
As noted in sections 3.3.2 and 3.4.1, specific permaculture principles (Holmgren, 2002;
Mollison, 1988; 1991) were not explicitly used in the design of the intervention. Instead, ecological design thinking was used that embodies the permaculture principles. However, in order to demystify the process and provide documentation from the literature, Section 9.2.1 includes many of Mollison‘s (1988; 1991) permaculture principles in the discussion of research sub-question 1: What are the characteristics of a permaculture approach to junior secondary science?