Quality of the Research

This study aims to iteratively develop, enact and refine the ePlay MakerSpace model to understand how the transformation of teachers’ dispositions may impact their

transformative integration of ETs. In conducting the research, I fulfil different roles, being insider-outsider, researcher-designer/designer-researcher, participant-observer and advocate-jury, in various degrees and often simultaneously. In light of these often

contradictory roles, as design-based researcher, it is important to guard academic rigour in order to produce high quality, objective and credible theory, while balancing “boldness and caution in a different way” (Walker, 2006, p. 13), and navigating the many ambiguities and unknowns within natural test-bed conditions.

In order for DBR to effectively contribute to the development of policies, programmes and changed practice, it needs to produce valid, legitimate results. Maxwell (1992) argues that research should be driven by the need to gain understanding, and different types of validity are derivatives of different kinds of understandings gained from inquiry (Maxwell, 1992). To this end, the “grounds for validating the trustworthiness of observations, interpretations and generalizations” (Maxwell, 1992, p. 280) are explicitly and deliberately foregrounded, using Maxwell’s (1992) typology, to guide and structure processes and outcomes, and avoid threats that may undermine the quality and validity of findings.

3.4.1

Descriptive validity

A primary aspect of validity is the factual accuracy with which data is collected, reported and accounted for. Since cultural forms find articulation in and through social action, social-

110

science researchers report and describe social action, behaviours and verbal and non-verbal interactions (Maxwell, 1992). Such reporting and description requires standard measures or instruments with which to conduct evaluations and collect or formulate descriptions to enhance the reliability of the research findings (Design-Based Research Collective, 2003). In this study, descriptive validity is achieved by accurately reporting data derived from interviews, questionnaires, participants created artifacts and reflections, and using

standardised formative evaluation measures. To increase inter-subjective agreement, where possible, interactions and behaviours are documented in photos and/or voice recordings. Recorded interviews are transcribed using standardised transcription symbols and cross- checked to ensure accuracy. After each session, photographs are uploaded to a secure drive and renamed using participants’ codes, while deleting photos containing the faces of those who did not provide permission. Teachers are also repeatedly given opportunities to refine and/or edit their created artifacts or reflections. Participants’ created artifacts, especially those shared from their Google Drives, are copied to my drive (so as not to lose access should they inadvertently delete these for example) and anonymised using teachers’ codes. Unless specifically indicated, no changes other than the grey text boxes with participants codes, are made to their created artifacts to ensure descriptive validity.

Descriptive validity also needs to be achieved in documenting the design process in order to provide critical evidence that may establish warrants for claims (Design-Based Research Collective, 2003). Systematic, thorough and transparent methods are chosen to document the process of design from thought-experiment to enactment and formative evaluation. Thought-experiments and ideas are thus captured and recorded in my design diary and on my phone using voice notes and my G/Drive. I also developed the habit of reading or writing difficult or challenging sections before going to bed at night, and deliberately concentrating on the particular challenge in the moments before falling asleep. Hours later I would wake and a crystallised understanding or deeply creative thought would have formed in my subconscious, which I immediately record for further interrogation.

111

Descriptive validity also needed to be achieved in the results obtained from the educators’ self-reported data. Self-reported data is generally viewed with some suspicion, since this type of data is notoriously biased and aimed at ‘painting the respondent in the best possible light’. It is frequently assumed that accepting such reports on face-value prejudices the data as being not-trustworthy or empirically flawed. In contrast, Csikszentmihalyi (1996, 1997) using Habermas (1973a, 1973b) argues that we are all prejudiced, whether as researchers or practitioners, and while this cannot be avoided, bringing scientific scepticism and reflection to the interpretation of data potentially erodes the effects of this prejudice on the reliability of findings. Moreover, Bourdieu (1984, 1991, 2000; 1992) insists in including the primary vision of the practitioner/participant and the designer-researcher to gain access to the work of objectification. However, as discussed, the epistemic break is employed to interrogate and analyse these findings, and the triangulation of multiple data sources (Design-Based Research Collective, 2003) to increase descriptive validity and develop interpretive and theoretical validity.

3.4.2

Interpretive validity

A researcher participating in design-based research looks at the constructed objects through socially constructed dispositions, produced and reproduced as part of their researcher habitus. Additionally, the design-based researcher fulfils simultaneous, often contradictory roles, requiring different dispositions at different times. In order to achieve interpretive validity, the researcher needs to suspend research dispositions towards searching for the meanings and understandings of those participating in the research endeavour. However, this may be more easily said than done since dispositions function below consciousness or thought, and are difficult to articulate or identify. Reflexivity (Bourdieu and Wacquant, 1992) is suggested as a tool to achieve this.

Different socially constructed dispositions construct the object of research in a certain light and predisposes the researcher to see it in certain ways and not others. Through critical reflexivity the researcher confronts their own underlying perceptions and assumptions and to bring to light their “presuppositions tacitly engaged in the view of the world” (Bourdieu,

112

2000, p. 221). This informs a critical analysis of possible dispositions that may create such perceptions, beliefs and practices. As an additional reflexive tool, the researcher systematic and deliberate charts each step during interpretation, recording the many themes, frames and codes that may emerge in the analysis phase to provide evidence of their thought processes for further reflexion.

Reflexivity and systematic charting of interpretation guides the collection, interpretation and analysis of data. Using reflexivity, I reflected on my past role as primary and high school teacher and emerging technology enthusiast, as well as lecturer of pre-service teachers and higher education lecturers, an insider in education. However, from the perspective of teachers participating in the ePlay MakerSpaces, I remain an outsider until my credibility is established and a trusting relationship developed. I also reflected on my gender – a woman working with women and men in a traditionally patriarchal context in which male-ness is often associated with power and influence, and female-ness with subservience and

powerless-ness. My role as researcher from a well-known university, and when interviewing teachers, as representative of the Cape Teaching and Leadership Institute of the Western Cape Education Department, may reduce traditional patriarchal power-differentials. Also, although I speak English and Afrikaans, I am not fluent in isiXhosa, the home language of many teachers from disadvantaged schools. Hence, I reflected my own assumptions regarding language and ETs, how ETs are generally shaped by normative English discourses and how difficult it would be for isiXhosa teachers to access terms or conditions I take for granted.

During analysis, I deliberately set aside the designer-epistemology and take on the critical, socio-cultural epistemology. The different epistemology enables a bird’s eye-view

perspective in order to discern patterns and understand relationships from the data – towards achieving theoretical validity. Coding themes emerged from this, which were used in the data analysis programme, Nvivo11, to add another layer of analysis of the data. Triangulation of multiple and different data sources offers a further means to achieve interpretive validity. As is frequently the case in DBR, large quantities of information rich,

113

salient and meaningful data is generated, and may lead to what van den Akker (1999, p. 11) terms an “overdose of uncertainty in interpretation”. Triangulation of methods and

instruments, sources and sites offers a means with which to overcome this (van den Akker, 1999). Repeated examination and scrupulous interrogation of data (Design-Based Research Collective, 2003) further enhances interpretive validity.

3.4.3

Theoretical validity

A theoretical understanding of data moves beyond a concrete description and

interpretation of physical and mental phenomena, and explicitly constructs theoretical abstraction as an outcome of the study (Maxwell, 1992). Theory in this sense refers either to physical events or mental constructs (Maxwell, 1992). Theoretical validity therefore moves beyond descriptive and interpretive validity, towards an explanation of the phenomena. Theory, Maxwell (1992) argues, should address two components: concepts and/or categories employed by the theory, and relationships that may exist, or development among these concepts. Thus, concepts need to be valid as well as the postulated

relationships between concepts. Theoretical validity relating to these concepts includes: content validity, the extent to which the design is based on state-of-the-art knowledge, and construct validity, how consistently the various design components are linked to each other (van den Akker, 1999, p. 10). Relating to relationships, content and construct validity

parallels internal or causal validity (Maxwell, 1992). Theoretical validity can also be

increased by considering alternative explanations and/or understandings of the phenomena under consideration and by incorporating emerging relationships between concepts

(Maxwell, 1992).

Content and construct validity in this study is achieved through the deliberate and systematic development of the analytic frame in this chapter, and applying this to the retrospective analysis process of the study. Bourdieu’s (1986, 1990, 2000) thinking tools of habitus, field and capital, guides the analysis of the context and participation, while

114

design and implementation. Findings are cross-referenced with the findings from previous studies as discussed in the literature review, considering various alternate theories and frameworks to the different aspects of the study. These discussions and cross-referencing elicits alternate explanations and understandings which are incorporated in the theory building. In developing the evaluation framework, specific emphasis was also placed on construct and content validity to develop internal consistency and interpretive validity, that informs the theoretical validity gained through the analysis process and limit threats to these.

Maxwell’s (1992) typology identifies potential threats to the nature of theoretical,

descriptive and interpretive validity in order to reduce or avoid these. This in turn increases internal consistency between the arguments put forward in this study, towards developing well-supported, warranted theory based on scrupulous readings, description and

interpretation of data, and abstraction towards theory building.