Predictive Models and Selection 4.1 Introduction
4.3 Computerised Selection in Introductory Programming Courses
In a study conducted by Greyling (2000) in the Department of CS/IS at UPE, the skills measured in the selection battery were mathematical and language ability, spatial and planning ability as well as computer proficiency. A number of different computerised instruments were used in the investigation, namely the ACCUPLACER Computerised Placement Tests (ACC 1997), Computerised Logo System (Hunt 1998), Computerised Mazes System (Roos 1998) and Interactive Learner (Streicher 1998).
Greyling’s investigation (2000) concluded that three of the ACCUPLACER Computerised Placement Tests produced the most significant results for an introductory programming course and accounted for 63% of the variance in performance. These tests were namely the reading comprehension, arithmetic and elementary algebra tests.
Analyses conducted on the matriculation subject variables of Mathematics, English Language and the combined average of Science, Accountancy and Biology confirmed the finding of Calitz in an earlier study (1997). Greyling’s study (2000) also found that, despite the confirmation that black matriculation results were not good predictors of success in CS/IS introductory programming courses, the matriculation results complemented the results derived from the computerised selection battery. Consequently, research results showed that matriculation results used in conjunction with placement tests continue to play a role in selection and admission strategies at tertiary education institutions.
Obvious advantages were observed from both the administrative and student viewpoints, namely immediate scoring of tests for the former and, for the latter, a more positive attitude than that experienced in traditional pen-and-paper tests. The latter observation is supported by a more recent South African study that examined the effects of a changing society and technology on the way that learners interact with information in an educational environment (Miller 2003). From this more recent study it is evident that learners are motivated by the technology used in information transfer.
Greyling’s study (2000) determined that the ability to measure the elapsed time and computer proficiency contributed significantly to the predictive validity of the selection battery for a CS/IS introductory programming course. Greyling concluded further that the combination of the selection battery and matriculation subjects accounted for 62% of the variance in performance in an introductory programming course, a slight decrease (1%) on the variance observed when using the selection battery alone.
Greyling’s streaming model was implemented at UPE as from the beginning of 2001 based on the selection results. The model concentrated on the placement of students within alternative introductory programming courses, namely the customary introductory programming course and an extended introductory programming course (Greyling et al. 2002; Greyling et al. 2003). The extended introductory programming course covers the same material as the customary course but over a longer period of time.
The implication of different learning programmes for different groups of students as categorised by the computerised selection battery is that support mechanisms need to be in place in each of these different programmes. This is in agreement with the strategy adopted by the UPE Placement Task Team (Foxcroft et al. 1999).
The greatest risk to the implementation of alternative learning programmes for different groups of students was identified as being the fact that streaming could generate some opposition from students as well as parents. The expected opposition was due to the fact that it would in effect mean for some students that their degree
programme would be extended by a further year (Greyling et al. 2003). Due to the endeavour on the part of the Department of CS/IS at UPE to communicate the justification of the streaming process, most parents were positive and in fact thankful that such a support mechanism was in place.
The implementation of Greyling’s streaming model in 2001 provided an opportunity for further validation of the model. It was observed that the regression formula was successful in predicting the students’ final results (Greyling et al. 2003). This was apparent in the finding that the actual average mark (69%) differed only slightly from the predicted average mark (66.7%) for a customary introductory programming course.
Furthermore, the pass rate for first time students in the introductory programming course increased to 77% (Greyling et al. 2003). A pass rate of 68% was evident in the extended introductory programming course and the actual average mark (59.8%) exceeded the predicted average mark (49.5%) by 10.3%. The results observed in the extended introductory programming course were encouraging as the students enrolled in the extended introductory programming course were predicted as possible failures by Greyling’s streaming model.
The following limitations in Greyling’s investigation were however identified:
• limited success in implementation (Greyling et al. 2002; Greyling et al. 2003), especially with respect to the performance rates with the inclusion of repeating students within the student body enrolled for the introductory programming courses; and
• the need for educational support mechanisms in each of the alternative introductory programming courses.
Although the pass rates of the streamed students increased from previously recorded pass rates, the limited success in the implementation of Greyling’s streaming model was evident in that the actual pass rate of the extended introductory programming
course (68%) was considerably lower when compared with internal and external expectations of 75% (Department of Education 2001; UPE 2002).
Further, the presentation of alternative introductory programming courses by the Department of CS/IS at UPE initiated the requirement for appropriate technological support in the respective learning environments. Adherence to this requirement is in accordance with the strategy adopted by the UPE Placement Task Team (Foxcroft et al. 1999).
The need to further improve and maintain satisfactory individual and group performance rates (Greyling et al. 2002; UPE 2002; Greyling et al. 2003; Naudé et al. 2003) amongst novice programmers in an introductory programming course as well as the need for educational support mechanisms in introductory programming courses (Greyling et al. 2002) was instrumental in initiating the current investigation into the comparison of programming notations and associated development environments for an introductory programming course at tertiary level.
4.4
Conclusion
Limitations on resources, specifically financial, hardware and human, necessitate the implementation of strategies that effectively maximise the throughput rate of introductory programming course students. One category of strategy is the selection of students with the highest potential to succeed in an introductory programming course. The second type of strategy is to modify course presentation techniques. The latter type of strategy is the category into which the current investigation is classified.
Predicting the achievement potential of a novice programmer in an introductory programming course is essential to ensure a satisfactory level in both individual and group performance rates, as well as improve the personal motivation of introductory programming course students. This chapter provided an overview of the international and comparative national research in the selection of candidates for introductory programming courses.
Based on the results of research into the predictors of success in computer programming, the Department of CS/IS at UPE initially implemented a strategy of advisory selection in 1993. Subsequent research in the Department of CS/IS at UPE resulted in the implementation of an effective and efficient computerised selection and placement model at UPE in 2001.
The streaming model implemented at UPE provided the opportunity for students to register and complete degree programmes according to the level of their potential to succeed. Consequently, alternatively paced introductory programming courses were designed and presented and students were placed into the alternative introductory programming course streams based on their performance in a computerised placement test battery. The subjects of the current investigation, namely 2003 customary introductory programming course students in the Department of CS/IS at UPE, were subjected to the rigid implementation of this computerised selection and placement process.
Despite external and internal pressure to increase and maintain satisfactory group performance rates of students in introductory programming courses, CS/IS departments at tertiary educational institutions should strive to maintain the quality of their introductory programming course while adhering to the external and internal expectations. One strategy to ensure the maintenance of the quality of the introductory programming course is with the provision of educational technological support for novice programmers. This is the second type of strategy proposed as a solution to maximise throughput in an introductory programming course.
Chapter 5 discusses the design and implementation of the experimental technological support used in the current investigation. The chapter focuses on the design and implementation of B#, an educational technological support tool that assists in the modification of the introductory programming course presentation techniques.