Design-Based Research Methods

Typically, design-based research uses mixed methods to conduct inquiry (Anderson & Shattuck, 2012). In the social sciences the use of mixed methods has become a common methodological approach (Bryman, 2012; Cresswell, 2008). It has previously been recognised that the paradigms of objectivist and subjectivist methodologies have been deemed incompatible, though overlap does exist between them (Bryman, 2012; Gay & Airasian, 2003). While debate has continued over the opposing methodologies, a shift towards combining the approaches of quantitative and qualitative research is also being strongly argued (Onwuegbuzie & Leech, 2005; Sale, Lohfeld, & Brazil, 2002). This has seen the use of mixed methods increasingly being viewed as an acceptable method of delivering quality research, particularly in the social sciences (Bryman, 2012; Creswell, 2008).

The use of mixed methods offers a comprehensive overview of the design principles to be considered for supporting student learning in popular music production. It does this through delivering multiple perspectives on the research questions. Because the research is situated in a higher education institution, there are many perspectives that can be drawn upon through the use of a mixed-methods approach. Student surveys provided a snapshot of attitudes surrounding the research questions. This was combined with the in-depth perspective on the design principles of on-demand learning resources elicited through student focus groups. Staff interviews contributed to the development of design principles for the research project, with staff involved in these interviews coming from a variety of backgrounds. While all staff interview participants have been involved in teaching within the BPM program, these staff also had an extensive history of involvement in the music

industry. The combination of these perspectives should contribute to the development of an objective study through incorporating multiple perspectives on the research questions. Wang and Hannafin (2005) suggested: “using a combination of methods, data from multiple sources increase the objectivity, validity, and applicability of the ongoing research” (p. 10).

The use of mixed methods in a design-based research framework has the potential to realise the aims of the research.

Design-based research uses multiple iterations to adapt and reflect on the pedagogical interventions used in the research (Anderson & Shattuck, 2012) and is regarded as a useful methodology for research surrounding “technology enhanced learning environments” (Wang & Hannafin, 2005, p. 5). The advantages of using multiple iterations for research into this area include the development, testing, and refinement of principles in an evolving area (Wang & Hannafin, 2005). However, one of the challenges of design-based research is incorporating the multiple iterations of the research cycle because of limitations on resources and researchers (Anderson & Shattuck, 2012; Kennedy-Clark, 2013). Consequently, one of the limitations of this methodology is that it is dependent on the continued support of the leadership and staff at the Queensland Conservatorium. The support for this project was favourable, with overall support for the expansion of blended and on-line models of education existing within the university.

Some limitations of design-based research have been anticipated, which is an important consideration for the development of a robust research design. Design-based research needs to account for “objectivity, reliability and validity” throughout the course of the project (The Design-Based Research Collective, 2003, p. 7). Researchers in the area often have to act as the “advocate and critic” and this can present challenges for remaining objective (The Design-Based Research Collective, 2003, p. 7). Barab and Squire (2004) also suggest this by asking, “What counts as reasonable and useful warrants for advancing assertions investigated through this type of research?” (p. 3). This criticism is a recurring one in the area, but one that should be counteracted through effective research design. The Design-Based Research Collective (2003) suggest that one way of doing this is through structuring the research design in such a way that it “triangulates multiple sources and kinds of data to connect intended and unintended outcomes to processes of enactment” (p. 7). Lincoln and Guba (1985) argued that the validity of research in the social sciences could be

measured by the trustworthiness of the study. They suggest four criteria can be used as a measure of the trustworthiness of the research.

1. Credibility: which is a measure of how believable the findings are. 2. Transferability: how transferable are the findings to other contexts. 3. Dependability: are the findings applicable at other times.

4. Confirmability: to what extent has the researcher’s opinions intruded on the findings.

These criteria are used to measure the trustworthiness of this research. Although this research project uses mixed methods, the study itself is aligned with qualitative social research. It has been argued that social research should be measured with different standards from those of quantitative researchers (Bryman, 2012). The credibility of the study can be interpreted through the resulting design principles, many of which aligned with other literature. The transferability of the study demonstrated that it could be applicable to other contexts of music education. This was also reiterated through the literature. The

dependability of the study is time and place dependent. The shifting focus of technology means that these design principles may become irrelevant as technology changes. The bias of the writer as a member of this community could never be completely eliminated, but all reasonable effort was made to remain impartial during the research.

In this research, triangulating the data from student surveys, student focus groups and academic staff interviews allowed a pragmatic approach to delivering the research outcomes. These perspectives were then refined into design principles to form the basis of the project before moving into the subsequent iterations. The credibility of this study can be measured through the finding’s applicability to the design of on-demand learning outside of formal education environments. During the research design process surveys were created, and then shared with academic supervisors to ensure the design of the surveys were appropriate to facilitate the aims of the study. The outcomes of the pilot study were also shared with academic staff during a seminar, where they were given an opportunity to validate the data collected.