Chapter 3 Methodology
3.3 Data analysis
DBR provides the context to develop theoretical categories and frameworks (DiSessa and Cobb, 2004), and relies on systematic evaluation and analysis of the research problem, the design of the intervention and the methodologies and design procedures to develop warranted theory. While DBR characteristically sees a close collaboration between the researcher-designer and participants / practitioners, this collaboration needs to be suspended in order for the research-designer to generate knowledge that is neutral and objective (Pedgley, 2007) and open to validation (Sloane, 2006, p. 21). Additionally the suspension of the design-epistemology needs to occur, which Bourdieu (1984, 1991, 2000; 1992) describes as an epistemic break in analysis, where the primary vision of the
participants and designer-researcher as participant (in the objectification phase) makes way for the work of the analyst. He states:
“Since one cannot be content either with the primary vision or with the vision to which the work of objectification gives access, one can only strive to hold together,
104
so as to integrate them, both the point of view of the agents who are caught up in the object and the point of view on this point of view which the work of analysis enables one to reach by relating position-takings to the positions from which they are taken.” (Bourdieu, 2000, p. 189 emphasis in the original)
Following the design, enactment and evaluation phase, it is consequently necessary to employ an epistemic break and employ a structured analytical process associated with the critical, socio-cultural epistemology, in order to objectify the conceptions and processes of the object of research. The epistemic break requires dual processes. Firstly, the researcher needs to integrate the primary vision of participants and an objective, analytic bird’s eye- view or “quasi-divine viewpoint” (Bourdieu and Wacquant, 1992, p. 254) perspective. Secondly, the researcher needs to embody the impersonality of the analysis procedures, devoid of their own biases, assumptions and personal beliefs which may be achieved through reflexivity.
A bird’s eye-view position requires the analyst to take an objective position and view the data from interviews, participants created artifacts, reflections and formative evaluations, from an objective distance. Such an analysis reflects on the field, the objective relations between participants, their position takings within the field and the rules that govern these position-taking in order for the analysis to unveil reality hidden in interaction and shaped by objective relations. The analysis further reflects on the intervention, the enactment of the intervention and design procedure. As such, a bird’s eye-view analysis works to avoid the danger of merely restating in different words what the participants have already expressed, or to express meanings that participants could themselves have identified as products of conscious intention (Bourdieu and Wacquant, 1992).
A bird’s eye-view perspective is only one part of this analysis though, and Bourdieu urges researchers not to “stop short at the objectivist phase and the partial view of the ‘half- learned’ who … fail to bring into their analysis the primary vision” (Bourdieu, 2000, p. 189). Instead, as he frequently argues, the perspective of participants is an essential element of the analysis and cannot be discarded to “defend the monopoly” of scientific understanding
105
(Bourdieu, 2000, p. 189). It is therefore essential to include participants’ primary vision perspectives. The primary vision is assumed to be subjective and, as very often happens, paints the participant in the best possible light as they are under pressure to reduce the risk of public humiliation in the face of failure (Bourdieu, 1990).
Taking participants’ primary vision reflections at face-value and building theory around such realities, potentially restricts the validity of the research outcomes. Bourdieu however argues that it is essential to bring this primary vision into the analysis and acknowledge this “common knowledge … as an object to be understood” (Bourdieu, 2000, p. 189). Therefore, the researcher-analyst needs to integrate the efforts of participants as they construct their subjective representations of themselves and their experiences in the intervention, and the analysis using the analytic bird’s eye-view perspective (Bourdieu, 2000), coinciding with DBR’s emphasis on collaborating with the participant albeit with their subjectivities. This is not to say that a casual ‘accept-whatever-comes’ approach is appropriate. Such a hippy- approach to research would have disastrous effects on the validity of the work. Instead, every effort should be made to (gently) guide participants towards a more objective view, one that is less subjective, yet careful to avoid incurring the resistance to scientific
objectification that may ensue (Bourdieu, 2000).
The analytic process for DBR requires strategic and considered analysis of participants’ primary vision and the bird’s eye-view position of the researcher. Gravemeijer and Cobb (2006) suggest the use of retrospective and systematic analysis of the entire data set Concurring, McKenney, Nieveen and van den Akker (2006) add that a longitudinal analysis can make claims of credibility, validity and trustworthiness if all phases of the analysis process are rigorously and thoroughly recorded and documented, including the refinement of conjectures and inferences. However, potential challenges are inherent within a
retrospective analysis. Although a retrospective analysis ideally strives to account for all conjectures, implicit decisions and assumption-driven choices, given the magnitude of the data set, this may be a complex undertaking that may not adequately and comprehensively account for all changing conditions, contexts or participants’ experiences throughout the course of the project. More pertinently, the volume of data to analyse may limit the
106
description of the evolution of design principles and the impact this has on the refinement and redesign of each iteration.
Various solutions are used to overcome challenges to the retrospective analysis. Parallel analysis offers an alternate means with which to arrive at theoretical design principles and is employed after each iteration. Using parallel analysis, each iterative cycle and formative evaluated findings are analysed to refine and elaborate DPs, and to inform the redesign of subsequent iterations. This allows the researcher to map changing contexts or participants’ roles, skills and experiences, account for ways in which implementation problems were addressed, and include changing decisions regarding the design of the intervention itself. Other researchers can then theoretically follow the process from theoretical design principles, through analysis and evaluation, to the primary evidence in the form of
recordings, documents or artifacts. In addition to parallel analysis, the retrospective analysis of the entire data set is improved through the use of analytic tools and an analytic frame. The data analysis package, Nvivo11, is used to strengthen the retrospective analysis of the entire digital data set. Additionally, an analytic frame is developed to refine the
retrospective analysis.
The analytic frame for the study is informed by Edelson’s (2002, 2006) contention that the outcome of DBR should be domain theory, design frameworks and design methodologies. Hence, the analytic frame is conceptualised as developing theory through the analysis of the research problem to understand how transforming teachers’ dispositions may lead to changed practice; analysis of the ePlay MakerSpace inculcation process and enabling conditions for the transformation of teachers’ dispositions; and analysis of the design methodologies for the enactment of the research process.
3.3.1
Problem analysis to develop teacher change theory
Having paired the DBR approach with Bourdieu’s (1986, 1990, 2000) theory of practice, his guiding principles for research inform the problem analysis. Bourdieu’s research process consists of a) constructing the research object, b) analysing the field in which the object is107
situated, and c) employing reflexivity towards participant objectivation (Bourdieu and Wacquant, 1992). This process integrates the primary vision of participants and the bird’s eye-view/quasi-divine viewpoint of the researcher. These research principles frame the problem analysis to understand how teachers dispositions potentially changed as an
outcome of the inculcation processes and enabling conditions of the ePlay MakerSpace, and how this was translated to changes enacted after the sessions. Bourdieu’s research
principles frame the problem analysis as follows:
a) Analysing the position of the field vis-à-vis the field of power (Bourdieu and Wacquant, 1992), by analysing the context of the problem in relation to the greater field of power in which it is situated.
b) Mapping the objective structures of the relations between positions within the field and the position-takings (Bourdieu and Wacquant, 1992), mapping how participants operate within the constraints and structures of the institution, and how their actions impact the structures of the institution.
c) Analysing the individual and collective habitus of those participating in the field and their change journeys as a habitus-level analysis aimed at understanding the choices and decisions individual teachers make regarding their personal learning, regarding for instance their movement on the TCF.
The problem analysis develops theory regarding teaching change and is informed by the formative evaluation framework, participant artifacts, reflections and interviews.
3.3.2
Analysis of the ePlay MakerSpace
The design of an intervention or solution, such as the ePlay MakerSpace inculcation processes and enabling conditions, integrates a range of design elements and balances trade-offs to meet the design goals and needs (Edelson, 2002, 2006). The designer- researcher makes various decisions in the course of reaching a design solution, many of these being intuitive, based on tacit knowledge or experience, and difficult to quantify or even describe. These decisions however, have to be accounted for in order to develop
108
warranted theory, which van den Akker (1999) terms substantive design principles and Edelson (2006; 2002) terms design frameworks. Design frameworks need to reflect essential characteristics of an intervention (van den Akker, 1999, p. 9) and describe the characteristics that subsequent designs of the solutions should have in order to address particular needs in specific contexts. In this way substantive design principles can take the form of a
prescriptive design framework (Edelson, 2002, 2006) as a collection of coherent design guidelines for a particular class of design challenges.
In order to develop the design framework for the ePlay MakerSpace, Pedgley’s (2007) macro- and microscopic analysis is used to provide a comprehensive understanding of design decisions within the design process. Microscopic analysis concerns the structure of the decision-making process, the reasoning and modelling that was employed, and is conducted as part of the formative evaluation after each of the ePlay MakerSpace iterations. Macroscopic analysis takes a broader view across the different phases of the project, to identify opportunities or constraints across the project. The outcome of the macroscopic analysis is a prescriptive design framework for future designs of the ePlay MakerSpace.
3.3.3
Analysis of the research process and methodology
The analysis process should also develop design methodologies (Edelson, 2002, 2006) by analysing the research and design process and methodology. Analysing the design process is typically a complex undertaking, made all the more difficult when the researcher-analyst is also the designer. Edelson (2002, 2006) notes this complexity may stem from the open- endedness and reliance on creativity in the design process, making it especially challenging to characterise and explain. Similarly, Pedgley (2007) notes that the transient nature of the design activity makes it difficult to account for each decision and step, and increases the difficulty of analysis. Analysing the design process is further complicated by the designer’s back-and-forth use of processes of divergence and convergence.109
The analytic frame developed is employed for parallel analysis and reported in the formative evaluation of each iteration in Chapters 4 and 5, while Chapter 6 reports the retrospective analysis. The quality and validity of theories developed from the analysis processes depend on the quality of data produced. Measures to improve the quality of data and analysis are developed in the next section.