Mixed Methods Research Quality

Regarded as “… a vehicle for improving the quality of inferences that are potentially

Teddlie, 2008, p.110), Bazeley (2004) cautioned mixed methodology must not be taken for granted:

Mixed methods are inherently neither more nor less valid than specific approaches to research. As with any research, validity stems more from the appropriateness, thoroughness and effectiveness with which those methods are applied and the care given to thoughtful weighing of the evidence, than from the application of a particular set of rules or adherence to an established tradition. (p. 156)

The use of different data collection techniques is often rationalised with reference to triangulation (Bryman, 2006a). Originating in navigation (Cohen et al., 2007), it refers to a confirmatory technique for the validation of interpretation (Hammersley, 2008) and for seeking convergence or agreement of findings (Greene et al., 1989). Four types of triangulation were described by Denzin (1989): data (different times, social settings, strategies or participant groupings); investigator (more than one researcher); theoretical (alternative viewpoints); and methodological (within- or between-method data collection).

A concept of many colours, there has been criticism, especially within the qualitative research community, of the use of triangulation as a validation technique; a discussion that highlights varying conceptions of truth and discrepancy (Brannen, 1992; Bryman, 1992; Gorard & Taylor, 2004; Hammersley, 2008; Patton, 2002; Tobin & Begley, 2004). On the other hand, seen as a possible method of contributing complementary information, of adding richness, colour, and quality to a picture, the discussion of triangulation appears to attract little resistance (Denzin & Lincoln, 2003; Flick, 2006; Gorard & Taylor, 2004; Jick 1979). Triangulation of method, as in mixed methods research, facilitates a middle ground position

where “Areas of convergence increase confidence in findings. Areas of divergence open

windows to better understand the multifaceted, complex nature of a phenomenon” (Patton,

2002, p. 559).

Establishing quality within mixed methods research is not straight forward because while

there is some agreement that quality be viewed as a process not an outcome (O’Cathain,

2010), there is no consensus within the research community about actual quality criteria (Creswell & Plano Clark, 2007; Onwuegbuzie & Johnson, 2006). This state of affairs

reflects complexity and variation possible in mixed method research design including how techniques, data, analysis, and findings are integrated throughout the research process (Greene et al., 1989).

Options are available however. Bryman (2008) suggested three possibilities depending on the dominance of data collection and integration: convergent (the same standards are applied to qualitative and quantitative strands); separate (different criteria are used for qualitative and quantitative strands); and bespoke (new criteria are devised). The “Quality Legitimation

Model” of Onwuegbuzie and Leech (2005) integrated different types of validity to address threats to internal and external credibility. Tashakkori and Teddlie’s (2003) “integrative

model of inference quality” based upon design quality (suitability, adequacy, within-design consistency, analytic adequacy) and interpretive rigor (interpretive consistency, theoretical consistency, interpretive agreement, interpretive distinctiveness, and integrative efficacy) provided a third alternative.

There is consensus within traditional and modern paradigms that quality control is absolutely essential for research credibility. Consistent aspects include: integrity and fostering productive, respectful relationships; permeability, and continuity referring to robust quality process; usefulness and accessibility of the research product; and either explicitly expressed or implicit are responsibility, appropriateness and respect relating to the entire research endeavour. Together, all of the above factors form a montage of quality practice inextricably linked with, and situated within, ethical practice; a sum of parts that equals a pedagogy of

quality. Maybe Kvale’s (1996) notion of art was not so far off the mark.

3.10.3 The Current Study

Quality assurance aspects of the current study have been introduced throughout this chapter. Four key features that influenced quality control decisions were:

1. The study was philosophically situated within the pragmatic paradigm. 2. Data were collected using mixed techniques.

3. The study was conducted primarily in an educational setting with one of the participant groups being children.

4. The researcher was an experienced secondary teacher and educational psychologist with relevant expertise working with children with learning support needs and in the area of transition.

In this section, quality control measures undertaken in this study are presented with reference

to the aspects above and to Eisenhart and Howe’s (1992) general standards (introduced in §3.10.1) which “require that research studies be cogently developed, competently produced,

coherent with respect to previous work, important, ethical, and comprehensive” (p. 656). Thus, the margins between ethical behaviour, professional conduct, and quality assurance appear purposely blurred – each a vital ingredient, simultaneously influencing and shaping the other with the responsibility for quality placed squarely on the shoulders of the researcher; no reliance on mechanical or statistical indicators is mandated.

Previous chapters have articulated the thorough evolution of this research. The design was developed giving due priority to extant literature while considering and making explicit the professional experience and beliefs of the researcher, as recommended by Eisenhart and Howe (1992). Furthermore, at design and proposal stage the research was presented on numerous occasions to academic superiors, professional peers, and student fora for feedback. Also, feedback was sought in regard to the design, value, and applicability of the research to the educational community.

Positioning this research within the pragmatic paradigm and guided by a situational analysis framework was a judicious decision made with due regard to referential standards: fit for purpose (Biesta, 2010; Maxcy, 2003); inclusive of multiple perspectives (Annan, 2005; Greene & Hall, 2010); contextually responsive (Annan, 2005; Datta, 1997); link means to ends and outcome oriented (Johnson & Onwuegbuzie, 2004); and work toward bettering the human experience (Crotty, 1998; Rorty, 2002). The stated reasons for combining methods of sampling, enhancement, and completeness enabled quality judgements related to purpose (Bergman, 2008; Bryman, 2006b).

The present research design was sequential which afforded the opportunity to separate out quality control measures and enable quality checks to permeate the entire process of research. Scaled questionnaire items were checked for internal reliability, and other quality assurance

steps employed for both questionnaire and interview data collection included solid formatting and sequence, pilot testing, and careful attention to dissemination/collection and analysis details (Anderson, 1998). Following transcription of interviews, script accuracy was prioritised by the researcher and all were returned to participants to enable feedback or correction. This was regarded as a check for quality and also a respectful act.

The values and standards that guided this research were methodological and ethical; bound together and inseparable. A research journal was maintained and regular supervision meetings with academic supervisors and professional colleagues ensured awareness of the research act remained fresh as advocated in the literature (Fine & Sandstrom, 1988; Guba, 1981; Punch, 2002).

Eisenhart and Howe’s (1992) fourth standard charged the researcher with the responsibility of making the research pay by making it accessible to the practice community. They called this

“the “so what?” question” (p. 7). To this end, the ideas and process of the present research have already been presented to peers, school personnel, and professional colleagues in local and national forums via presentation and publication (Higgins, 2009a).

Attention has been drawn to the synergy between Eisenhart and Howe’s (1992) standards for quality, pragmatic philosophy, general ethical and quality ideals for educational research (Fine & Sandstrom, 1988; Lincoln, 1995; Mertens, 2010; Robinson & Lai, 2006), and the professional code of ethics for psychologists. These were reference points and benchmarks for quality in the present research which was undertaken seriously with the view of contributing to, and improving, educational practice and positive outcomes for children.

Bassey’s (1984) words sum up fittingly:

If studies … are carried out systematically and critically, if they are aimed at the improvement of education, if they are relatable, and if by publication of the findings they extend the boundaries of existing knowledge, then they are valid forms of education research. (p. 121)