Chapter 3 The BRACElet Project
4.3 Research design for this study
4.3.2 Research problem and purpose
Design-based research concentrates on solving a complex real-world educational problem by designing and testing an innovative intervention or artifact, while contributing to design principles and theories in this process. The real-world problem addressed in this study (discussed in more detail in section 1.3.1), with the corresponding aims and research questions, is reiterated below.
4.3.2.1 Research problem
The results of introductory programming courses at educational institutions typically have a bimodal distribution, with a low pass rate and a high drop-out, as well as a high number of distinctions (Bornat et al., 2008; Corney, 2009; Robins, 2010). Unisa is no exception in this regard. Table 1.1 in Chapter 1 shows the pass rates, number and percentage of distinctions and dropout rates for the introductory programming module at Unisa, COS1511 from 2011 to 2014. The demands and nature of the ODL environment at Unisa, as discussed in Chapter 2, may affect the situation.
Chapter 2 shows that many of the difficulties novices experience, can be related to lack of viable mental models for programming, and that they need to master certain threshold concepts that will allow easier learning of new concepts as proposed by the LEM effect hypotheses (Robins, 2010). The importance of teaching novices program tracing to address misconceptions and to assist them in developing mental models for programming, was also pointed out in Chapter 2. Furthermore, research by the BRACElet project, discussed in Chapter 3, has shown that novices first have to master tracing, that is, reading code, before they develop the ability to explain and write code.
Memory tracing and parameter transfer are taught in the introductory programming course at Unisa, COS1511, by means of static variable diagrams in the COS1511 Study Guide (Halland, 2011). The researcher noticed that students do not understand the static diagrams in the study guide, but that face to face explanation of the static variable diagrams in consultations, help them to understand it. When marking examination scripts for the follow-up first-year module, it is also clear from the kind of errors
students make, that they lack knowledge and comprehension of the processes that occur in CPU memory during program execution. The knowledge gap is even more pronounced with regard to parameter transfer. This is supported by literature (Bruce-Lockhart & Norvell, 2007a; Milne & Rowe, 2002), and confirms the need for a better understanding of what happens in computer memory when a program executes.
This research is an attempt to address the lack of viable mental models in novice programmers and in doing so, to improve the pass rate of first-year programming students in COS1511 at Unisa. This was done by providing the COS1511 students with an interactive tutorial to teach them how to do memory tracing. The tutorial resembles a PV tool. It takes the form of a customized tutorial to teach students how to draw variable diagrams. The tutorial is based on the COS1511 Study Guide (Halland, 2011) since correspondence between the artifact and the related study material is a key factor in its acceptance as a learning tool (Naps, Röβling et al., 2003; Röβling, 2010). The tutorial was not intended as a tool to be used during program construction, but rather to teach students how to do manual tracing by themselves. This is in line with research that shows that students have to construct diagrams to trace code themself in order to make sense of the code (Thomas et al., 2004) and that engagement with visualization contributes to students’ program comprehension (Ben-Ari et al., 2011). However, the key features and the pedagogical benefits of visualizations are not clear (Urquiza- Fuentes & Vel
á
zquez-Iturbide, 2009b). Despite the general perception, research shows that visualization is not always pedagogically effective or used effectively (Shaffer, Cooper & Edwards 2007; Foutsitzis & Demetriadis 2008). A framework of guidelines from literature for using visualization to teach programming was compiled and used as a basis from which to develop the tutorial to teach students how to draw variable diagrams. This was then enhanced with lessons learnt during the development process and the evaluation of the tutorial itself and how students use it.4.3.2.2 Aim
Design-based research delivers dual outcomes with a practical intervention as well as a related theoretical contribution in the form of design principles. Correspondingly, this research also had the following two aims:
• A practical intervention with the tutorial to teach students to draw variable diagrams. The purpose was to enhance comprehension in an open distance learning (ODL) environment in such a manner that it assists students in mastering the art of programming to the extent that it impacts the pass rate for the module.
• A theoretical contribution in the form of a framework of guidelines for designing, developing and using visualization tools to teach programming.
4.3.2.3 Thesis and research questions
In the light of the research problem and the aim of this research, the following thesis was formulated for this research, as indicated in section 1.3.3:
A framework of guidelines for using and creating visualization tools to teach programming, synthesized from literature and tested and refined through practical application, can be employed to develop a PV tool to teach introductory programming students in an ODL environment, to trace programs in order to improve students’ programming skills.
To test this thesis the researcher investigated the following research questions:
1. What guidelines and principles for using and creating visualizations to teach programming, can be derived from literature?
2. Will a tutorial to teach tracing to visualize program execution and parameter transfer, based on the framework of guidelines, assist students in acquiring programming skills in an ODL environment so that it is noticeable in average marks?
3. How does students’ biographical properties, their positive user experience, their use of tracing and the extent and manner of tutorial use, affect their ability to acquire programming skills in an ODL environment?
4. What lessons have been learnt through the practical testing and refinement of the visualization tool?
Data collected to answer research questions are interpreted according to a philosophical paradigm or approach. The philosophical approach used in this research is motivated below.