Design-Based Research (DBR)

Design-Based Research is a systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and

implementation, based on collaboration between researchers and practitioners in real- world settings and leading to contextually-sensitive design principles and theories (Wang & Hannafin, 2005). This methodology has roots in the field of educational psychology. In proposing this methodology researchers e.g. Brown (1992) and Collins (1992) refer to the term ‘design experiments’ for studies of classroom-based

interventions where the purpose of the research is to actively participate in the design and implementation of innovation in order to test and develop instructional theories.

DBR projects have two major goals: i) the empirical investigation, and ii) development of high quality interventions to solve complex educational problems and the

accompanying set of well-articulated design principles (Linn, Davis & Bell 2004; Van den Akker, 1999). These principles show how the intervention works in practice, the effect of using the intervention, and an explanation of the working mechanisms. Design-based researchers carefully combine design and research activities in order to develop research-based solutions for complex problems in educational practice

(McKenney & Reeves, 2012; Van Den Akker et al., 2006). To support the dual goals of DBR, Barab and Squire (2004: 6) held that:

DBR requires more than simply showing a particular design works, but demands that the researcher generates evidence-based claims about learning that address

76

5.3.1 Justifications for DBR methodology

Numerous motives for DBR have been cited in the literature, many of them speak to the long-standing criticism that educational research has a weak link with practices (Design-based research collective 2003; Kelly, 2004; Reeves et al. 2005). In

combination with other social science research approaches, DBR has potential to help in the development of educational interventions, and to offer opportunities for learning during the research process (e.g. in this study teachers’ participation in the validation and implementation of the PD programme aimed to enhance their understanding of effective instructional approaches). In the drive for better understanding of teaching and learning, the belief that context matters leads to the conclusion that research paradigms that simply examine the learning process as isolated variable within laboratory settings will necessarily lead to an incomplete understanding of their relevance in more naturalistic settings (Barab &Squire, 2004; Barab & Leuhmann, 2003). For example, naturalistic settings such as classroom environments are complex hence required multiple methods that possess high degree of ecological validity, i.e the methods, materials and settings must approximate the real life situation under

investigation (Brewer, 2000).

The field of DBR was introduced with the expectation that researchers would systematically adjust various aspects of the designed intervention so that each

adjustment served as a type of experimentation that allowed the researchers to test and generate theory in naturalistic contexts (Barab & Leuhmann, 2003). For example, during prototyping approach (Chapter 6) the formative evaluation was designed to involve several levels such as experts and users appraisals, try-out with teachers and sudents, panel diacussions with experts and field implementation. The essence of having each of the mentioned levels was to improve the quality of prototypes before implementation in the target schools.

77

5.3.2 Characteristics of DBR

i. Pragmatic; ii. Grounded;

iii. Interactive, iterative, and flexible; iv. Integrative;

v. Contextual (McKenney & Reeve, 2012; Wang & Hannafin, 2005).

The following section illustrates each of these characteristics.

The first characteristic of DBR is that it ispragmatic, because its goals are to solve current real-world problems by designing interventions, as well as extending theories, and refining design principles (Design-Based Research Collective, 2003; Van den Akker & et al., 2006).

In traditional educational research, existing theories are usually tested through artificial treatments in controlled contexts (Reeves et al., 2008). People engaged in these

experimental approaches hope to be able to design instruction based on the principles that the theory and associated experimental results support (Edelson, 2002). However, in DBR the goal is not testing whether or not the theory works (Van Den Akker, 1999) rather both design and theory are mutually developed through the research process. Therefore, researchers use design to enact and refine theories continuously (Edelson, 2002) so that the theories “do the real work” in practice Cobb et al., (2003: 10) and eventually lead to substantial change in educational practice (Van Den Akker, 1999).

The second characteristic of DBR is that it is grounded in both theory and the real- world context (Wang & Hannafin, 2005) i.e. DBR uses theory, along with empirical findings, and local expertise as inputs to create interventions that solve real-world problems. Through research embedded in the intervention development process, DBR produces theoretical understanding as an output (McKenney & Reeves, 2012). In addition, DBR is conducted in real-world contexts replete with the complexities, dynamics, and limitations of authentic practice.

78

The third characteristic in terms of the research process is that DBR is interactive, iterative, and flexible.

DBR requires interactive collaboration among researchers and practitioners in order to refine the designed intervention. For example, in the current study the researcher worked with experts, biology teachers, and students at different stages of development of the intervention. Without such collaboration, interventions are unlikely to effect changes in the real world context (Design-based Research Collective, 2003; Reeves et al., 2005; Wang & Hannafin, 2005).

The insights and the interventions of DBR evolve over time through multiple iterations of investigation, development, testing, and refinement (MacKenney & Reeves, 2012). Furthermore, MacKenney & Reeves argue that within one research study, several sub- studies often take place, each with its own complete cycle of inquiry and chain of

reasoning. For example, the current study used cycles within three stages (Section 5.3.3) to portray the overall process of the study i.e. the preliminary analysis stage had two cycles of context analysis including field based investigation and a literature review; the design or prototyping stage had up to four cycles of prototypes (refer to Figures 6.1 and 6.2); and the field implementation and evaluation stage had two cycles, i.e.

implementation of the PD workshop, and implementation of new approaches in the experimental schools.

DBR procedures usually take a long period of time because theories and interventions tend to be continually developed and refined through an iterative design process from analysis to design to evaluation and re-design (Design-Based Research Collective, 2003; Wang & Hannafin, 2005). This on-going recursive nature of the design process also allows greater flexibility than traditional experimental approaches.

The fourth characteristic of DBR is that it is integrative. Researchers need to integrate a variety of research methods and approaches from both qualitative and quantitative research paradigms, depending on the needs of the research (Section 5.6).

The integrative use of multiple methods in the research process results in data from multiple sources, which serves to confirm and enhance the “credibility” of findings (Wang and Hannafin, 2005: 8), and shows the connection between the processes of enactment and outcomes. This aspect of methodology was explicitly implemented

79

when the intervention was enacted with experimental school teachers and students for a period of a school term during the third stage of the study. This stage provides an opportunity to use a quasi-experimental design (Section 5.4) with experimental and control groups, as well as pre-test and post-test comparisons for the measurement of teachers’ classroom practices, students' attitudes toward biology, and the teaching and learning methods.

The fifth characteristic of DBR is that it is contextualised because research results are “connected with both the design process through which results are generated and the setting where the research is conducted” (Wang & Hannafin, 2005: 11).

It is imperative that design-based researchers keep detailed records during the design research process concerning how the design outcomes (e.g. guidelines or principles) have worked or not, how the innovation has been improved, and what kind of changes have been made (Van Den Akker et al., 2006). Through this documentation, other researchers and designers who are interested in those findings can utilise them in relation to their own context and needs. In order to increase the “adaptability” of the findings in the new settings, guidance on how to apply those findings is also required (Wang & Hannafin, 2005: 12). The following summarises the three stages of the study.

5.3.3 An overview of the three stages of the study

The three main stages of this study are: preliminary analysis, design , and implementation and evaluation (Chapter 1 Figure 1.1).

Stage 1. The preliminary analysis entailed the problem identification, diagnosis, and initial recognition of design requirements. The researcher was involved in the activities such as conducting context analysis, field-based investigation, and a literature review. The preliminary analysis stage contributed to both practical and theoretical outputs. From a theoretical perspective this stage generated a clear understanding of the problem as well as specifications of long range goals, (i.e. the overall aim of the intervention, based on the analysis and exploration which have shaped an

understanding of both the problem at hand and teachers’ needs and wishes). In addition, tentative design requirements were determined and initial design guidelines were

80

generated based on contextual insights. Practically, this stage produced a descriptive and analytical understanding of the identified problems as manifested within a context.

Stage 2. The design and empirical testing stage generally takes place through prototyping approach (Section 6.3) where successive approximations of desired

solutions for the identified problems in the context are re-created. Each prototype of the research, i.e. the curriculum materials and PD programme was reviewed by experts, and teachers, tried out by teachers and students, and finally reviewed by experts. The process was iterative in nature which involves design, formative evaluation, analysis, and revision/refinement (Sections 6.3, Figure 6.1 and Section 6.4, Figure 6.2). Findings from each cycle through formative evaluation were used to improve the validity and practicality of prototypes.

The overall results from the design and empirical testing stage from practical perspectives included the intervention which was conceived and assembled with the most successive prototypes.

Stage 3. Implementation and evaluation of the PD programme. According to Smith and Ragan (1999), the purpose of evaluation of implementation of the PD programme is to determine the effectiveness of the revisions made during the prototyping stage and ascertain any problems that might arise in the administration of the intervention. From practical perspectives, the activities in the third stage lead to ideas for conclusions about how the intervention is to be implemented more widely. From a theoretical perspective the knowledge and pedagogical skills produced by the combined activities contribute to a broader theoretical understanding of the intervention.

The following section discusses the quasi-experimental design which guided research activities (e.g. student learning outcomes) during evaluation of the PD programme.