R ESEARCH DESIGN

Like the mythology of the phoenix, mixed methods research has arisen out of the ashes of the paradigm wars to become the third methodological movement. The fields of applied social science and evaluation are among those which have shown the greatest popularity and uptake of mixed methods research designs (Cameron, 2009, p. 140).

While the history of pragmatism is long, mixed methods have a short history which can be traced to the early 1980s and has been described as a 'quiet' revolution due to its focus on resolving tensions between the qualitative and quantitative methodological movements (Cameron, 2009, p. 142; Tashakkori & Teddlie, 2003, p. 697). These tensions have been termed the 'paradigm wars' (Cameron, 2009, p. 140) when some researchers challenged 'rigid rules about paradigms and research methods' that initially rejected pragmatism (D. Morgan, 2007, p. 64-65). According to Feilzer, the paradigm wars also involved 'long-lasting, circular, and unremarkably unproductive debates discussing the advantages and disadvantages of quantitative versus qualitative research' (2010, p. 6). Because pragmatism 'does not require a particular method or methods mix and does not exclude others', (Feilzer, 2010, p. 13), it can be a specific justification for combining qualitative and quantitative methods (R. Johnson & Onwuegbuzie, 2004).

While I am using mixed methods, a higher priority is given to the qualitative in terms of data collection and analysis, because it is participants' perspectives in the HE context of DUU that I am primarily interested in, with regard to answering the research questions (Cresswell & Plano Clark,

2011). I also used, in a secondary role, my archival data about the project related to the school champions, most of which is quantitative. These data, once analysed, were merged with the analysed qualitative interview data to enhance the profiling of a selection of school champions in the form of a series of vignettes (Bryman, 2006). Merging is one way of mixing these data—and is a core

characteristic of mixed methods as it 'provide[s] a better understanding of research problems than either approach alone' (Cresswell & Plano Clark, 2011, pp. 5, 16). The purposes of the vignettes were: 'to clarif[y] the phenomenon, the activities, the setting and the issues' (Stake, 2010, p. 172); to verify, contextualise or clarify the selected school champions' recollections and/or perspectives (Mason, 2002, p. 108); and to illustrate the variety of their contexts and challenges. Because the vignettes integrated, into the discussion, findings from interpretation and analysis of the qualitative and quantitative data, this strategy, according to Cresswell and Plano Clark, 'shows a pragmatic stance' (2011, p. 280). My choice of vignettes, interpreted and analysed through the lens of the leadership and power conceptual framework, also illustrates the 'complexity, messiness, contradiction, [and] ambiguity [that were] intrinsic to the phenomenon' (i.e. DL at DUU) (Mason, 2002, p. 177).

Using both quantitative and qualitative methods in my study helps to offset their respective strengths and limitations because the data offer multiple ways, and provide more evidence for,

studying the research problems (Bryman, 2006; Jick, 1979). For example, mixed methods can enhance the credibility and validity of findings; that is, trustworthiness (Lincoln & Guba, 1985) or rigor, and lead to an enriched, more complete explanation of the research problem. This can occur in a number of ways, for example, when findings from one method corroborate findings from other methods so that lines of inquiry converge (also termed confirmation, substantiation, triangulation, validation, or verification);18 by helping uncover relationships between variables; and when findings from a dominant method are enhanced or elaborated by findings from another method (Bryman, 2006; Cresswell & Plano Clark, 2011; Eisenhardt, 1989; Guba, 1981; Jick, 1979; Teddlie & Tashakkori, 2009; Yin, 2003).

By carefully employing mixed methods in my study, I was seeking to enhance the

trustworthiness of my findings so they would provide strong support for my recommendations relating to the use of DL in the current HE context to implement change in teaching and learning. In the opinion of Papadimitriou, Ivankova and Hurtado, the '[h]igher education research is a context that is ripe for excellent mixed methods studies that address complex research problems that require explanation and exploration, as well as definitive answers' (2013, p. 151). A survey of the literature

18

Cresswell and Plano Clark comment that 'the terms to use in designing and conducting mixed methods study are far from settled', hence my listing of the common ones associated with 'checking on the quality of the data, the results, and the interpretation' (Cresswell & Plano Clark, 2011, p. 277, 210).

reveals that mixed methods approaches have been used with HE as the research context to carry out a variety of different studies, for example, comparing new faculty members' emotions in relation to their teaching compared to their research (Stupnisky, Pekrun, & Lichtenfeld, 2016); how HE teachers collaborate in teams to design educational innovations (Bron, Endedijk & Sleegers, 2015); emotional intelligence and teaching competencies in HE (Akhmetova, Kim & Harnish, 2014); the identity and experiences of marginalised groups in the academy (Griffin & Museus, 2011); PhD examination processes and outcomes in Australia (Holbrook & Bourke, 2004); student academic engagement in introductory science, technology, engineering and mathematics (STEM) courses (Gasiewski, Eagan, Garcia, Hurtado & Chang, 2012); and the capability and belonging of HE students from non-

traditional backgrounds (Burke, Bennett, Burgess, Gray & Southgate, 2016).

Mixed methods research designs have many different types, and the one I used was a case study of DL in an HE context (DUU) that I had been closely involved in for four years. Flyvberg argues that, 'the most advanced form of understanding is achieved when researchers place themselves within the context being studied ... only in this way can [they] understand the viewpoints and

behaviour that characterises the social actors' (2011, p. 310).

4.2.2

Case study

A case study was the logical choice because I considered the context of DUU was 'highly pertinent to [the] phenomenon of study' (Yin, 2003, p. 13); that is, the implementation of CRA using DL. It is also 'an established research design ... used extensively in a wide variety of disciplines, particularly in the social sciences' (Crowe et al., 2011, p. 1). According to Yin, a case study is 'an empirical inquiry investigating a phenomenon in its real-life context when the boundaries between phenomenon and context are not clearly evident' (1994, p. 13). The boundaries of the case I was studying were the 'phenomenon' of DL (an activity) at DUU (institution or place) in the years 2008 to 2011 (time). That is, the study was bounded by an activity in place and time (Cresswell, 2003; Stake, 1995). The HE context of DUU was complex, as illustrated in Chapters 2 and 3, and so too were the contexts of the actors in the implementation plan outlined in that latter chapter. Because I had worked in other HE institutions, I was aware that 'universities remain diverse institutions of schools and faculties each having distinct cultures and a major allegiance to a disciplinary or professional authority outside the university' (Anderson & Johnson, 2006, p. 7).

As well, I had been immersed within the DUU context during the years of the DL project, built up relationships with many of the actors, and developed initial understandings of these cultures and allegiances. I therefore had the advantage of not starting from scratch, and retrospectively, to comprehend what the DUU context was like during the project. However, I had no time to investigate the real-life scenario, the phenomenon of DL in depth (Gorard, 2012) while I was engaged in it as an

academic developer. Conducting a case study would allow me to study the phenomenon to develop a more holistic in-depth picture of the complexity, variation and richness of the contexts and the actors' perspectives, by obtaining 'detailed information using a variety of data collection procedures over a sustained period of time' (Cresswell, 2014, p. 42). As Flyvberg states 'depth—detail, richness and completeness [are] the main strengths of the case study' (2011, p. 314). The setting could provide me with 'a variety of relevant and interconnected data' such as interviews and my archival artefacts from the project plus 'different instances, facets and viewpoints' that possibly would form 'a microcosm of the research topic'—of DL in HE (Halliday, 2007, p. 34). Primarily this involved conducting

interviews about the retrospective perspectives of the school champions in their individual schools and faculties.

Thus my choice of a case study was 'defined by interest in an individual case, not by the methods of inquiry used' (Stake, 2005, p. 443). I was also reassured by Rowley's recommendation that 'case study research is also good for contemporary events when the relevant behaviour cannot be manipulated' (2002, p. 17). The contemporary event—using DL to implement CRA—is still

contemporary, as I demonstrated by my scan of 21 institutions (see 2.5). Because my case study case played a 'supporting role, facilitating ... understanding' of DL (Baxter & Jack, 2008, p. 549), I was conducting what Stake (1995) refers to as an instrumental case study. That is, the case was secondary to understanding a specific phenomenon—that of DL being used to implement CRA at DUU.

Despite the choice of a case study being the best, in my opinion, to do justice to the project, there are some limitations including: being difficult to summarise (Flyvberg, 2011, p. 313); there is too much highly complex data to interpret and analyse, hence the necessity to set aside sufficient time to do this (Crowe et al., 2011, p. 7); the complexity is difficult to represent simply (Hodkinson &

Hodkinson, 2001, p. 9); and regardless of the rigour of the research, (an account of a case study) is not completely objective because of the researcher's involvement in the creation, analysis and presentation of evidence (Hodkinson & Hodkinson, 2001, p. 10). I have acknowledged these challenges to being completely objective in section 3.2.3.

4.2.3

Four stage research design

Figure 4.1 represents my research design; that is, the overall strategy and the sequencing of qualitative and quantitative components in four numbered stages. Following the protocols for these figures advocated by Hesse-Biber, I have used QUAL to indicate that qualitative data gathering is the dominant approach, and quan to denote that quantitative data gathering is 'in the service of ... QUAL and assists in the interpretation of qualitative findings' (2010, p. 71). The first stage involved the collection and literal analysis of interview data with subsequent counting of themes and subthemes using NVivo10 (see 4.5.1 and 4.5.2). This was followed in stage two by rhetorical analysis (Billig,

1987) that moved beyond the literal, again with the counting of different themes and subthemes (see 4.5.3.2), but did not involve NVivo10.19 Quantitative archival data related to the school champions that I collected during the CRA project for reporting purposes were also selected and collated in this stage, based on how the data related to the research questions.

In stage three, this quantitative data plus the findings from the first two stages were integrated to inform the selection of a small sample (a distribution) of school champions' cases. This purposive sample was to illustrate representativeness of cases by 'maximum variation sampling', and to 'set up comparisons among different cases' from most typical to extreme or deviant (Teddlie & Yu, 2007, pp. 80-81). In stage four, this sample of school champions' cases was analysed using a conceptual

framework (see 4.5.3.3), which has the potential to be developed into a theory that could provide different insights into implementing institutional change other than in HE. Because my primary interest was the school champions' perspectives of DL at DUU, the research plan illustrates that their data is involved in all steps. As mentioned earlier, there is a dearth of studies of the perspectives of those at the coalface involved in DL.

This research design was my roadmap, reflecting the undergirding tenets of pragmatism because it gave me the freedom to 'take a pathway of pragmatic curiosity by exploring [my] research interests and [devise] the ... design ... that ... allow[ed] [me] to pursue [my] investigative curiosities' (Chenail, 2011, p. 1713). These tenets: focus on questions about what is useful and practical in relation to HE institutional change using DL; value the research questions more than method or paradigm; provide me as the researcher with flexibility to make research design choices about what methods to use; and support the use of both qualitative and quantitative methods in the same study (Tashakkori & Teddlie, 2003). The resulting complexity of the research design reflected some of 'the messiness of the social world' I was investigating, and referred to in the quote at the start of this chapter (Ashwin & Case, 2012).

19

Note that the labels as identity badges framework (see 4.5.3.1) was only used in Chapter 6 about the impact of the label, school champion. Hence I have not represented this on the diagram of the research design (Figure 4.1).