The Evaluation of a Pedagogical-Program Development
Environment for Novice Programmers:
A Comparative Study
Dieter Vogts
Submitted in fulfilment of the requirements for the degree of Philosophiae Doctor in the Faculty of Science at the
Nelson Mandela Metropolitan University
January 2007
Promoter: Prof A. P. Calitz Co-promoter: Prof J. H. Greyling
To the three women in my life who have provided so much love, encouragement and support.
Acknowledgements
This thesis would not have been completed without the support, assistance, encouragement and guidance of a number of people over the last few years.
Professors André Calitz and Jéan Greyling, my promoters in the Department of Computer Science and Information Systems (CS&IS) at the Nelson Mandela Metropolitan University (NMMU), whose wisdom, enthusiasm and encouragement is greatly appreciated. Their input has been essential to the completion of this thesis.
Doctor Charmaine Cilliers, an exceptional colleague who provided much support, knowledge and resources to help further this research, even though it was not required. My other colleagues in the Department of CS&IS at the NMMU, especially Lester Cowley, Marinda Taljaard and Dixie for their support, knowledge and help in conducting the experiment.
Doug Christians, Shane de Jager and William Brander, three outstanding honours students at the CS&IS department at the NMMU who incrementally developed the building block of the current study, SimplifIDE.
Mr Danie Venter, lecturer in the Department of Mathematical Statistics at the NMMU, for expert advice and confirmation of the appropriateness of the quantitative statistical analysis techniques applied.
The 2005 introductory programming students for being such willing and co-operative participants in the current study.
My family, for helping out when it was needed and for providing support.
My wife, Shimoné, for love, support and encouragement during the long journey called “PhD”. It is a certainty it would not have been completed without her by my side.
Abstract
It is an acknowledged fact that many novice programmers experience difficulty in the process of learning to program. One of the contributing factors to this difficulty is the Program Development Environment (PDE).
Professional-PDEs are those developed specifically for professional programmers, but are often used by educational institutions in the instruction of programming. It has long been accepted that such environments are inappropriate in the instruction of programming due to unnecessary complexity and lack of support for novice programmers in the learning process. Numerous pedagogical-PDEs supporting the mechanics of programming have been developed in response to this. A review of literature, however, indicates that very limited empirical studies comparing pedagogical-PDEs and professional-PDEs have been conducted.
The current study investigates whether there are measurable benefits to using a pedagogical-PDE supporting the mechanics of programming in the instruction of programming instead of a professional-PDE. A comparative study of this nature requires a representative pedagogical-PDE and representative professional-PDE be compared with one another.
The first part of the current study determines a set of requirements that a pedagogical-PDE should adhere to based on literature. A set of representative features for a pedagogical-PDE is derived by examining the features of existing PDEs in conjunction with the set of requirements. Based on these features, a pedagogical-PDE, known as SimplifIDE, is developed that implements the representative set of features and that meets are the requirements for a pedagogical-PDE.
The second part of the current study is the specification and administration of an empirical experiment in which SimplifIDE and Borland© DelphiTM are compared with one another. A holistic approach in determining the differences between the PDEs is taken and three main areas are examined, namely academic performance, perceptions and programming behaviour.
Academic performance relates to the scores obtained for administered assessments and is used as a measure of understanding and knowledge at education institutions. The perceptions of novice programmers regarding the PDE used and activities performed are an indication of the motivation of novice programmers. Programming behaviour relates to the behaviour of a novice programmer during the act of programming.
A stratified approach is used in the experiment, whereby subjects are classified as high ability subjects or low ability subjects. The academic performance, perceptions and programming behaviour of the low ability and high ability subjects are analysed individually, as well as collectively.
The results of the experiment indicate that subjects of different ability levels benefit differently from the use of a pedagogical-PDE while learning to program. The main findings of the experiment are that:
• high ability subjects using the pedagogical-PDE improved significantly in terms of academic performance;
• low ability subjects using the pedagogical-PDE improved significantly in terms of programming behaviour;
• a significant improvement in programming behaviour does not correspond with a significant improvement in academic performance; and
• the perceptions of subjects did not differ significantly except at initial exposure to the pedagogical-PDE and after a long period of exposure.
Based on the empirical evidence from the current study, it can be stated that pedagogical-PDEs are beneficial to novice programmers learning to program. However, developers of pedagogical-PDEs need to take cognisance of the fact that not all novice programmers have the same ability levels, do not have the same requirements of a pedagogical-PDE and are affected differently by a pedagogical-PDE.
Keywords:
Novice Programmer, Introductory Programming, Mechanics of Programming, Technological Support
Table of Contents
List of Figures x
List of Tables xiii
Glossary of Terms xv
Chapter 1: Research Context
1.1 Background 1
1.2 Research Approaches 4
1.2.1 Student Selection and Streaming 4
1.2.2 Problem Solving 4
1.2.3 Concepts of Programming 5
1.2.4 Mechanics of Programming 5
1.2.5 Teaching Methodology 8
1.3 Relevance of the Study 9
1.4 Research within Department of CS&IS 11
1.5 Focus and Methodology of the Study 13
Chapter 2: The Process of Learning to Program
2.1 Introduction 19
2.2 Requirements for Meaningful Learning 21
2.3 Motivation 24
2.4 Learning to Program 26
2.4.1 Program Comprehension 29
2.4.2 Programming Languages 32
2.4.3 Program Development Environments 35
Chapter 3: Program Development Environments
3.1 Introduction 41
3.2 Requirements for Pedagogical-PDEs 42
3.2.1 Mechanics of Programming 43
3.2.2 Usability Issues 45
3.2.3 Requirements 49
3.3 Pedagogical-PDEs 52
3.3.1 Programming Notations and Associated Environments 53
3.3.2 Code Structure Editors 59
3.3.3 Novice Programmer PDEs 62
3.3.4 Novice Programmer Standalone Tools 69
3.4 Professional-PDEs 71
3.4.1 Graphical User Interface 72
3.4.2 Program Editing Features 74
3.4.3 Code Validity Features 75
3.4.4 Debugging Features 76
3.4.5 Help Features 77
3.5 Conclusions 78
Chapter 4: Design & Implementation of SimplifIDE
4.1 Introduction 83
4.2 Development of SimplifIDE 83
4.2.1 Extending Borland© DelphiTM 84
4.2.2 Feature Set of SimplifIDE 85
4.2.3 High-Level Design of SimplifIDE 87
4.3.1 User Rights & Administration 90
4.3.2 Event Logging 91
4.3.3 Simpler GUI 91
4.3.4 Wizards 92
4.3.5 Language Directed Editor 94
4.3.6 Code Structure Highlighting & Error Indicators 96
4.3.7 Variable Scope & Spelling 100
4.3.8 Compiler Error Messages & Visibility 100
4.4 Conclusions 101
Chapter 5: Investigative Research Methodology
5.1 Introduction 103
5.2 Investigative Study Learning Environment 105
5.2.1 Introductory Programming Course Structure at NMMU 105
5.2.2 Evidence of Student Learning 109
5.2.3 Historical Student Performance 109
5.3 Materials and Instruments 110
5.3.1 PDE Instruments 110
5.3.2 Investigative Study Material 111
5.3.2.1 Demographic Material 112
5.3.2.2 Formative Assessment Material 112
5.3.2.3 Training Material 114
5.3.2.4 Qualitative Assessment Material 114
5.3.2.5 Quantitative Behavioural Material 115
5.3.2.6 Quantitative Assessment Material 116
5.4 Procedure of Data Collection 119
5.5.1 Formulation of Hypotheses 123 5.5.2 Participant Population, Sample Size and Selection 124
5.5.3 Data Analysis 128
5.6 Risks to Investigative Study 133
5.6.1 Sample Size 133
5.6.2 Administering Practical Learning Activities 134
5.6.3 Home Usage of SimplifIDE by Students 134
5.6.4 Formal Assessment of Practical Performance in Laboratory Sessions
135
5.7 Conclusions 135
Chapter 6: Investigative Study Results
6.1 Introduction 138
6.2 Properties of the Sample Population 139
6.2.1 Sample Size and Attrition 139
6.2.2 Sample Stratification 140
6.2.3 Sample Demographics 143
6.3 Academic Performance Results 144
6.3.1 Academic Performance 146
6.3.2 Thoughput 148
6.3.3 Regression Formula for Final Course Score 150
6.4 Perception Results 151
6.4.1 Practical Learning Activity Attitudes 152
6.4.2 Practical Assessment Attitudes 154
6.5 Programming Behaviour Results 155
6.6 Summary of Results 160
Chapter 7: Interpretation of Results
7.1 Introduction 163
7.2 Constraints on Experiment 164
7.3 Academic Performance 164
7.3.1 Full Complement 165
7.3.2 Low Ability Subjects 166
7.3.3 High Ability Subjects 166
7.3.4 Academic Performance Implications 167
7.4 Perceptions 168
7.4.1 Full Complement 169
7.4.2 Low Ability Subjects 170
7.4.3 High Ability Subjects 171
7.4.4 Perception Implications 171
7.5 Programming Behaviour 173
7.5.1 Full Complement 173
7.5.2 Low Ability Subjects 175
7.5.3 High Ability Subjects 176
7.5.4 Implications of Programming Behaviour 176
7.6 Conclusions 177
Chapter 8: Conclusion of Investigation
8.1 Introduction 181
8.2 Summary of Study 182
8.3 Comparative Study Findings 184
8.4 Recommendations for Future Research 186
8.5 Conclusion 189
References
191Appendix A: Ethics Committee Application
A.1Appendix B: Practical Learning Materials
B.1Appendix C: Assessment Scripts
C.1Appendix D: Demographic Questionnaires and Results
D.1Appendix E: Null-Hypotheses and Alternate Hypotheses
E.1Appendix F: Perception Questionnaires and Results
F.1List of Figures
Figure 1.1: Typical Integrated Development Environment (Microsoft© Visual StudioTM 2005)
3 Figure 1.2: Programming Research Group Research Areas and Projects 12
Figure 1.3: Thesis Structure 15
Figure 2.1: Factors Influencing Learning to Program 19
Figure 2.2: Framework for Meaningful Learning 22
Figure 2.3: Problem and Programming Domain 27
Figure 2.4: Pennington’s Two Stage Mental Model for Program Comprehension 30 Figure 2.5: Indentation Highlighting Code Structure 33 Figure 2.6: Misleading Control Structure Indentation 34 Figure 2.7: Comments Highlighting Assumptions and Behaviour 34 Figure 2.8: Identifier Names Conveying Additional Meaning 35 Figure 2.9: IntelliSense Displaying Parameter Details 38 Figure 3.1: Derivation of a Comprehensive Feature Set for a Representative
Pedagogical-PDE
41 Figure 3.2: Syntactically correct, but semantically incorrect code 44 Figure 3.3: PDE for the Iconic Programming Language, B# Version 3 57
Figure 3.4: CS1-Sandbox 58
Figure 3.5: Cornell Program Synthesizer 60
Figure 3.6: ALICE 3D Program Development Environment 61 Figure 3.7: Screenshot of Mixed-Mode Editing in JPie 61
Figure 3.8: DrJava 63
Figure 3.9: Screenshot of the BlueJ Workbench 65
Figure 3.10: Screenshot of the Student Perspective in the Gild PDE 66
Figure 3.11: KenyaClipse 67
Figure 3.13: Compiler Feedback from Expresso 70
Figure 3.14: Visual StudioTM 2005 72
Figure 3.15: Borland© DelphiTM 7 Professional 73
Figure 3.16: Eclipse 73
Figure 4.1: High-Level View of SimplifIDE 88
Figure 4.2: Error Frequency Report 90
Figure 4.3: SimplifIDE GUI 91
Figure 4.4: New Project Wizard 92
Figure 4.5: Subprogram Wizard 93
Figure 4.6: Impossible to Achieve Code Indentation in Visual Basic .NET 95 Figure 4.7: Code Structure Highlighting, Code Structure Error Indicators and
Identifier Error Indicators
97
Figure 4.8: Example Annotation Techniques 98
Figure 4.9: Results of Code Structure Annotation Technique Questionnaires 99
Figure 4.10: Error Bubble 100
Figure 5.1: Learning Framework 106
Figure 5.2: Student Performance in Introductory Programming Course in 2002 110
Figure 5.3: Administration of Materials 120
Figure 5.4: Summary of Null-Hypotheses 125
Figure 5.5: Participant Selection Procedure 127
Figure 5.6: Recategorisation of Lickert Scale Questionnaire Responses 131
Figure 5.7: Diagram of Compile Event Pairs 132
Figure 6.1: Final Sample Distribution 139
Figure 6.2: Regression Equation for Final Score obtained in the Introductory
Programming Course (p < 0.05) 150
Figure 6.3: Compile Event Pair Proportions for Practical Assessment Task 3, Full Complement
157 Figure 6.4: Compile Event Pair Distributions for Practical Assessment Tasks
(High to Medium Risk Stratum)
Figure 6.5: Compile Event Pair Distributions for Practical Assessment Task 3 (Low Risk Stratum)
160 Figure 7.1: Paper-Based Form and Subprogram Wizard 169 Figure 7.2: PDE Effect on Academic Performance, Perception and Programming
Behaviour
180
Figure 8.1: Summary of Current Study 183
List of Tables
Table 2.1: Sources of Motivation 24
Table 2.2: Guidelines for Learning to Program 39
Table 3.1: Pedagogical-PDE Requirements 50
Table 3.2: Programming Notation Associated PDEs Feature List 59
Table 3.3: Code Structure Editors Feature List 62
Table 3.4: Novice Programmer PDEs Feature List 69
Table 3.5: Novice Programmer Standalone Tools Feature List 71
Table 3.6: Professional-PDE Feature List 78
Table 3.7: Typical Implementation of Features in PDEs 79 Table 3.8: Features of Pedagogical-PDEs and Professional-PDEs 80 Table 3.9: Feature Set of a Comprehensive Pedagogical-PDE 82
Table 4.1: Feature Set of SimplifIDE 86
Table 4.2: Keyword Triggers and Code Inserted 95
Table 4.3: Borland© DelphiTM & SimplifIDE Features 102
Table 5.1: Investigative Study Materials 111
Table 5.2: Captured Events 115
Table 5.3: Problems and Associated Goals of First Pen-and-Paper Test 116 Table 5.4: Problems and Associated Goals of Second Pen-and-Paper Test 117 Table 5.5: Problems and Associated Goals of Practical Test 118 Table 5.6: Problems and Associated Goals of Pen-and-Paper Examination 118 Table 5.7: Areas of Impact and a Subset of Independent Variables 122 Table 5.8: Investigative Study Materials and Associated Statistical Tests 137 Table 6.1: Correlation between Academic Performance Measures and
Stratification Variables (p < 0.01)
142 Table 6.2: Sample Derivation using Fine Grained Strata 142
Table 6.4: Descriptive Statistics and Results of Independent Sample t-test Categorised by Logical Strata
143 Table 6.5: Coding of Academic Performance Independent Variables 145 Table 6.6: Descriptive Statistics for Full Complement of Control and Treatment
Groups
146 Table 6.7: t-test Computed Values for Academic Performance Independent
Variables
147 Table 6.8: χ2-test Computed Values for Academic Performance Throughput 149 Table 6.9: p-Values of χ2
-test of Practical Learning Activity Questionnaire Responses (Summary of Table F.6)
152 Table 6.10: p-Values of χ2
-test of Practical Learning Activity Questionnaire Responses (Summary of Table F.7)
155
Table 6.11: t-test Computed Values for Programming Metrics of the Practical Assessment Tasks (Summary of Table G.1)
156 Table 6.12: χ2-test Computed Values for Compile Event Pairs of the Practical
Assessment Tasks (Summary of Table G.2)
Glossary of Terms
Code Completion
A PDE feature that allows the programmer to partially type the name of a function, variable, constant or type and press a special function key. A list of matching constructs is displayed, from which the programmer can choose one and continue coding. The listed constructs are context sensitive and only constructs that are valid within the current context are displayed.
Code Templates
A PDE feature that allows the programmer to type a code template abbreviation and then when a special key combination is pressed, the abbreviation is expanded into a portion of code.
IDE An Integrated Development Environment is a PDE that has tools integrated within a single environment.
Introductory Programming
Course
A course offered at an educational institution that teaches the fundamentals of programming.
Novice Programmer
A programmer with little previous exposure to computers or programming.
Parameter Tool Tips
A PDE feature that displays tool tips to indicate the number, type, order and name of parameters of a given function when the programmer types an opening brace following the name of an existing function. If there is more than one function with the same name, but different code signatures, the programmer is able to cycle through the parameters in the tool tip until the correct one is located.
PDE A Program Development Environment is the environment in which programs are developed, edited and tested. A PDE typically consists of tools that are use in conjunction to perform the tasks mentioned.
Tools include a code editor, file manager, debugger and documentation and need not be integrated within a single environment. If all the tools are integrated within a single environment, then the PDE is known as an IDE.
Pedagogical-PDE
A PDE specifically developed for use by novice programmers. The PDE is usually designed with educational goals in mind.
Professional-PDE/IDE
A PDE/IDE specifically developed for experienced programmers. Usually contains complex functionality to support experienced programmers.
Programming Notation
The notation that is used to describe a program can either be textual or visual. Textual programming notations are programming languages that use text to communicate the structure and logic of a program, such as C and C++. Visual programming notations use images and text to describe the structure and logic of a program.
Tool Tip Insight
A PDE feature that allows the programmer to obtain information regarding identifiers in program code by hovering the cursor over them. Information displayed includes the type or signature of the identifier, in the case of a variable or function name, or the type definition, in the case of a type. While debugging a program, the information displayed will be the actual value of the variable, in the case of a variable.
Chapter 1
Research Context
1.1 Background
Research on the topic of “learning to program” generally commence with the words “learning to program is difficult!”. The statement may appear to be a cliché, but in reality, learning to program is a complex and generally poorly understood task. It has been found that a small number of students are actually able to write simple programs that compile and run upon completion of first programming courses (McCracken et
al. 2001). Programming requires the use of a number of different skills and
procedures, many of which are used in parallel during programming exercises (Jenkins 2002). Learning to program is cognitively challenging as a novice programmer has to learn how to form structured solutions to problems, construct programs that implement these solutions and understand how programs are executed, while at the same time having to learn the rigid syntax and commands of the programming language, as well as the “softer” rules of writing code that conforms to various standards (Jenkins 2002).
Numerous aspects of programming add to the cognitive load experienced by novice programmers: for example the programming language, the program development environment (PDE) and the problem being solved1. Due to the memorisation problem2 short-term memory can only hold a given amount of information, some of which is lost if more information is placed in short-term memory. The information loss often occurs without the person even being aware of the loss (Studer et al. 1995). The memorisation problem is particularly acute in the beginning stages of learning to
1
(Ezell 1990; Garner 2000; Daly & Waldron 2004; Lane & VanLehn 2004)
2
The memorisation problem is the phenomena where a person is only able to hold a certain amount of information items in short-term memory. Any additional information items to be placed in short-term memory typically cause another information item in short-term memory to be displaced. Short-term memory has a very limited capacity and is able to retain seven plus or minus two items of information at a time (Miller 1956).
program, but as the novice programmer becomes more experienced, the memorisation problem becomes less noticeable (Studer et al. 1995).
Programming languages are not a natural manner of describing solutions to problems for many people and these programming languages typically have very strict rules governing their usage. General purpose programming languages, such as C++ and Java, while powerful and able to be used in solutions to complex problems, have not specifically been developed for use by novice programmers and are not appropriate for teaching programming (McIver 2000; Pane et al. 2002).
Program Development Environments (PDEs) are used in the creation, editing, running and testing of programs. As with programming languages, most professional Integrated Development Environments (IDEs3) have not been developed for use by novice programmers, but rather by experts4. The term PDE will be used to refer to both general PDEs and IDEs from this point forward. Professional-PDEs (Figure 1.1) typically provide numerous features and options that are often confusing to novice programmers and introduce a steep learning curve when learning to use the environment. Simply the creation of a “hello world” program can be quite daunting in some PDEs (Fincher 1999).
The PDE and the programming language used have an influence on one another. PDEs can make the mechanics of programming easier. For example, the syntax of the programming language can be made clearer and more readily apparent by implementing syntax highlighting. Likewise, PDEs can make the mechanics of programming more difficult, for example the program creation in some PDEs is a complex process. A number of different features for pedagogical-PDEs have been proposed and implemented (Chapter 3) to address the relationship between PDE and programming notation.
This investigation will address the issue of whether the use of a PDE that supports the mechanics of programming is beneficial to the process of learning to program and
3
For the purposes of this study, an IDE is an integrated development environment that provides comprehensive tools that cater for program creation, editing, debugging, tracing and integrated help. These tools are integrated and can communicate with each other as needed. Most professional programming packages have IDEs. A PDE has some, or all, of the tools of an IDE. An IDE is a PDE, but not vice versa.
4
hence the academic performance of novice programmers. With faster CPU speeds, it is possible to perform more real-time checking of source code and automate common programming tasks than was previously feasible. Feedback regarding syntax errors, structural errors and scope errors, amongst others, should be provided in real time rather than only at compile time using metaphors with which novice programmers are comfortable. The spelling and grammar check metaphors used by Microsoft© WordTM to indicate errors within a document are examples of such metaphors that could be used.
Figure 1.1: Typical Integrated Development Environment (Microsoft© Visual StudioTM 2005)
Section 1.2 discusses different research approaches that have been employed, both locally and internationally, in an effort to increase the academic performance of novice programmers learning to program. The relevance and purpose for conducting the current research is stated in Section 1.3 and the research conducted within the Department of CS&IS at the Nelson Mandela Metropolitan University is briefly described in Section 1.4. The focus and methodology of the investigation is outlined in Section 1.5, as well as the structure of the thesis.
1.2 Research Approaches
A number of different ways in which increasing the academic performance students in introductory programming courses can be attempted. The following different approaches have targeted specific stages of traditional introductory programming courses:
• prior to entering the course; • learning problem solving;
• learning programming concepts; and • learning the mechanics of programming.
The teaching methodology employed spans the entire introductory programming course and has also been the subject of research. Research approaches targeting the specific stages of learning to program, as well as the entire process encompassed by the teaching methodology, will each be discussed in turn in the following sub-sections. Specific emphasis will be placed on the mechanics of programming as it is focussed upon in the current research.
1.2.1 Student Selection and Streaming
At the one end of the introductory programming course, prior to entering, one approach is to determine the suitability (or aptitude) of students for programming. A student exhibiting an aptitude for programming can be accepted for the course, while a student not exhibiting an aptitude for programming can either not be accepted for the course or accepted for the course and remedial action performed. It has been found that existing mathematical and scientific skills are good predictors to success (Greyling 2000a; Byrne & Lyons 2001; Greyling et al. 2002). Research into other demographic factors contributing to success in introductory programming courses has been conducted, but it has been found that there are no dominant factors, such as learning style or gender, that contribute to success or failure (Byrne & Lyons 2001). 1.2.2 Problem Solving
Problem solving, the ability to break a problem into smaller parts, find solutions to each and to bring all the solutions together into a cohesive whole that solves the original problem, is one of the core skills that students are required to possess (McCracken et al. 2001). Many introductory programming courses commence with the topic of solving problems using strategies, such as problem decomposition, or
problem solving notations, such as flowcharts. A number of software tools have been developed to aid in problem solving, for example SOLVEIT (Deek 1999; Deek & McHugh 2000), RAPTOR (Carlisle et al. 2005) and Visual Logic (Crews & Murphy 2004).
1.2.3 Concepts of Programming
Concepts of programming deal with the constructs of programming (e.g. iteration, selective branching, the use of variables, classes, objects, polymorphism, etc.) and are generally independent of programming language (Kelleher & Pausch 2005). Knowledge of the concepts can largely be transferred from one programming language to another.
A number of different approaches have been proposed to aid in the acquisition of programming concepts knowledge. These approaches all try to hide the mechanics of programming in a particular language by providing higher-level views of solutions and concepts.
Mini-worlds have been used to introduce students to concepts of object-oriented programming, such as objects, classes and methods, without the need to write code defining these structures (Dann et al. 2003; Sanders & Dorn 2003). Mini-worlds typically consist of a small, constrained “world” in which objects have been placed. Objects in the mini-world can be interacted with and issued commands programmatically using a simple programming language.
Tools used for prototyping classes help to convey object oriented programming concepts, while hiding the mechanics of how to write class code, by providing students with an UML-like view of classes that have been defined and the relationships between them5. Objects can be instantiated from visible classes and placed in a workbench, while methods can be invoked by selecting the desired method from a menu and assigning values to parameters. The object state is visible for inspection as desired.
1.2.4 Mechanics of Programming
Mechanics of programming relates to how the concepts of programming are implemented in a specific programming language (syntax and semantics) and how a
5
program is written, run and debugged (Kelleher & Pausch 2005). Both the programming language (also known as a programming notation) and the PDE are part of the mechanics of programming and play an important role in teaching novice programmers to program. In order for novice programmers to start writing their own programs, novice programmers need to become familiar with both the programming language and the PDE. Most pedagogical-PDEs address problems novice programmers experience with programming languages in one manner or another: some change the programming language, while others provide alternate methods of program creation to avoid syntactical errors.
Professional-PDEs are created specifically with expert programmers in mind and as such, have complex features and functionality (Ziegler & Crews 1999; Kölling et al. 2003; Chatley & Timbul 2005). The multitude of features and options available are often confusing to novice programmers and introduce a steep learning curve when learning to use the environment. Simply the creation of a “hello world” program can be quiet daunting in some program development environments. In addition to this, removing syntax errors from program source code is often made difficult by cryptic messages from the compiler, such as “Project Homework.exe raised exception class
EInOutError with message ‘I/O Error 105’”, which are difficult for novice
programmers to comprehend. In fact, it has been stated that many novice programmers view writing programs as “fighting the compiler”, with some students going as far as changing degrees to avoid programming (Fincher 1999). A number of pedagogical-PDEs have been developed that either simplify the graphical user interface (GUI) of the PDE or provide better feedback to students regarding syntax errors (Allen et al. 2002; Hristova et al. 2003).
Programming languages that have syntax and semantics similar to natural languages, are easier for novice programmers to become accustomed too and incur less cognitive load (Myers 1999). Most general purpose programming languages, however, are not similar to natural languages or even appropriate for teaching programming (Pane et
al. 2002). Due to the memorisation problem, novice programmers have difficulty
remembering the exact syntax of a language and syntax errors can occur frequently (Studer et al. 1995). Even simple, easily fixed syntax errors can lead to frustration and
can be highly disruptive when learning to program (Ezell 1990). Visual, iconic and mini-languages attempt to circumvent the distractions caused by syntax.
Visual and Iconic programming languages allow students to build programs visually in the PDE, instead of textually, using a drag-and-drop metaphor (Ziegler & Crews 1999; Cilliers 2005). Programming constructs, such as loops and selection, are represented by individual icons, which are connected together to form complete programs. In this way, programs can be constructed that are syntactically correct at all times.
Mini-languages attempt to make the mechanics of writing programs easier by providing programming languages that are smaller, closer to natural languages or simpler to use for novice programmers than general-purpose programming languages (McCracken et al. 2001; Roberts 2001; de Pasquale III 2003). The rationale being that less confusion and cognitive strain will be incurred by small, well-defined and well thought out languages than large, abstract programming languages. If it is not possible to change the programming notation or as a support tool for textual programming notations to reduce syntactical errors then the use of language directed editors and structure editors is a viable approach.
Language directed editors and structure editors are PDE tools that allow students to create syntactically correct code in a textual programming notation (Feiler & Medina-Mora 1981; Neal 1986; Miller et al. 1994). Structure editors allow students to create syntactically correct code by creating abstract syntax tree structures of the program code. Nodes representing various programming constructs are added by selecting a programming construct from a list of constructs that are valid within the current context. These editors are typically cumbersome to use as backtracking and rearranging nodes is difficult (Kelleher & Pausch 2005).
Language directed editors, on the other hand, are text-based tools that allow syntactically correct code to be created while typing code. If a particular programming construct, such as an if-statement is entered, then the remainder of the code for the structure is inserted as well (Kelleher & Pausch 2005). Visual StudioTM is an example of a professional PDE that provides a language directed editor for a general purpose programming language, Visual BasicTM (Microsoft 2005).
Instead of hiding programming language details and rules from students, another approach has been that of programming tutors and coaches6. Tutors and coaches typically focus on small programming concepts and mechanics and require students to provide solutions to stated problems that illustrate or make use of the specified concepts or mechanics. Solutions are then evaluated and students are provided with feedback, such as syntax errors and possible sources of logic errors. Problems tend to be very small, typically no more than a few lines of code and feedback is mostly hard coded to specific problems. A more general approach would be to evaluate and critique the source code of general programs and not only tiny code samples.
Automated marking and assessment tools, while mostly being tools to assist educators with the load of having to assess potentially hundreds of programs on a regular basis, also have the potential to provide students with immediate assessment feedback of administered problems (Blumenstein et al. 2004). There are various levels at which programs can be assessed, from a very superficial comparison of input-output pairs to an in-depth evaluation of abstract syntax trees.
The evaluation of input-output pairs is simple to implement, but requires that solutions compile and provide output that matches strict presentation rules. This type of automated marking is not suitable for the examination and grading of program code where the logic and structure of programs are important. In order to perform this kind of evaluation, the structure of a program needs to be examined, usually in the form of a graph or abstract syntax tree. The evaluation of abstract syntax trees or graph representations of programs is an extremely complex problem to solve, particularly for syntactically incorrect programs (Naudé 2007). The successful implementation of tools for such evaluation can allow for highly intelligent and individualised feedback. The indication of syntactical errors, semantic errors, potential areas of misconception regarding programming constructs and logic errors are the types of information that can be provided to novice programmers.
1.2.5 Teaching Methodology
Teaching programming in such a way that students gain an in-depth understanding of the content, as opposed to superficial learning, is complex. Superficial learning is characterised by novice programmers memorising concepts of programming for
6
automatic future use. In-depth learning, on the other hand, requires the comprehension of the effects of programming concepts so that the concepts can be applied in the future implementation of novel solutions7. The educator’s training and teaching methodology employed, as well as the course curriculum, play vital roles in the successful acquisition of such knowledge by students.
Some of the approaches proposed for addressing teaching methodology have been: • educator training(Kushan 1994);
• the use of design patterns(Proulx 2000; Porter & Calder 2004); • approaches to teaching introductory programming courses8
; • student motivation (Jenkins 2001a, b);
• curriculum (Fincher 1999); • paradigm9
;
• assignments to widen understanding (Dann et al. 2003); and • assessments10
.
Numerous approaches to improving the academic performance of novice programmers learning to program have been attempted. The current study examines the use of a PDE that supports the mechanics of programming and the effect that it has on the process of learning to program. The relevance of the current study is discussed next.
1.3 Relevance of the Study
The educational background of students at South African universities has changed since 2000. The Department of Education adopted the Outcomes Based Education (OBE) paradigm and instituted its usage in primary and secondary schools throughout the country (DOE 2003; van Zyl 2003). This paradigm was selected as a means to address the challenges of developing a system of education that allows for equitable access, mobility and articulation through a National Qualifications Framework (NQF). It attempts to place assessment on a more valid and reliable footing than
7
(Mayer 1981; Lockard 1986; Kushan 1994; Studer et al. 1995; Wiedenbeck 1999; Dingle & Zander 2001; McCracken et al. 2001; Jenkins 2002)
8
(Bornat 1987; Barnes et al. 1997; Juliff 1997; Shackelford 1998; Stein 1998; Fincher 1999)
9
(Wiedenbeck 1999; Wiedenbeck et al. 1999; Roumani 2002; Cooper et al. 2003)
10
previous systems and provides a basis for setting standards, but cannot address all factors pertaining to quality education.
OBE attempts to provide a uniform level of education, however there are still many differences in the education received by pupils at various schools. Although the same curricula are followed, there are differences in the quality of education. This is due to various reasons, such as the training that a teacher has received and the teaching resources available to the teacher. There is currently a shortage of qualified maths and science teachers in South Africa (Mettler 2003). It is estimated that approximately 40% of all maths and science teachers in South Africa are under qualified to teach these subjects.
Learners have varying levels of exposure to technology and computers in particular. At some schools, it is not unheard of that pupils have never seen a television or a computer before. This, in addition to the fact that many tertiary institutions have lowered entrance requirements, has led to more under-prepared students entering tertiary education. The lowering of entrance requirements is a national and international phenomena (Jenkins 2001b).
At the Nelson Mandela Metropolitan University (NMMU)11 computer courses in the Department of Computer Science and Information Systems (CS&IS) are presented to students with differing educational backgrounds, some of which have come from underprivileged backgrounds similar to those mentioned. In the first year of the Computer Science curriculum, end-user computing and introductory programming courses are presented. The end-user computing courses teach basic computer literacy and proficiency in a number of typical computer applications, such as word processors and spreadsheets. The introductory programming courses teach problem solving and fundamental programming principles.
Failure rates, however, are high for these courses, which is not unforeseen considering that world wide, Computer Science courses (particularly programming courses) are notoriously difficult for novice programmers (Kushan 1994; Garner 2000; Mahmoud
11
The NMMU is the tertiary institution formed as a result of the merger between the University of Port Elizabeth, Port Elizabeth Technikon and Vista University. All the tertiary institutions fell within the same municipal area.
et al. 2004). A factor contributing to the failure rate is the cognitive load due to the
programming language and overly complex PDEs used.
The pass rates of courses are of concern to a number of parties: the country, the Department of Education (DoE), the university itself, the Department of CS&IS at the NMMU and the students themselves. Even though more students are enrolling, less are registering for computer science courses due to many potential students thinking that programming “is just too difficult” (Jenkins 2001a, b).
In an attempt to reduce the number of failures, thereby increasing throughput, research has been conducted locally at the Department of CS&IS and internationally into factors that affect the pass rates. Some of the international research conducted has already been covered in Section 1.2. The next section discusses some of the pertinent research conducted within the CS&IS department at the NMMU and context in which the current research takes place.
1.4 Research within Department of CS&IS
The Programming Research Groupof the Department of CS&IS at the NMMU has a number of current and past research projects that deal with improving the experience of learning to program at various years of instruction. The main aim of doing so is to improve the quality of student understanding and the overall improvement of throughput in programming courses.
The Programming Research Group has conducted a large amount of research. Figure 1.2 contains a break down of the various areas of research and projects therein. The bold-faced research project, Program Development Environment Support Tools, is the area in which the current study resides.
Section 1.2 detailed research approaches that have been employed in order to improve the academic performance of novice programmers learning to program. In a similar fashion, research within the Programming Research Group has focussed on the different stages of learning to program namely:
• prior to entering the course; • learning problem solving;
• learning programming concepts; and • learning the mechanics of programming.
The earliest research conducted by the Programming Research Group was in the selection and streaming of students based on their matriculation12 and entrance test results (Calitz 1997; Greyling 2000b). Using the matriculation results of students entering the NMMU, along with the results of an entrance test, results these students would obtain for the first year courses in the Department of CS&IS are predicted.
Prior to entering programming course Programming Concepts Programming Mechanics
Selection and streaming of students
Research Area Research Projects
Data Structure and Algorithm Animation Code Restructuring / CLOZE Tools
Iconic Programming Languages (B#)
Program Development Environment Support Tools
(current study)
Automated Assessment Tools
Figure 1.2: Programming Research Group Research Areas and Projects
Based on the predicted marks, students are either allowed to do a normal paced stream (six months) or a slower paced stream (one year) for either the end-user course or the introductory programming course or both. The two streams for each course cover the same content, but the slower paced stream has twice the amount of contact time and a much larger practical component built in. Results of this process have been favourable, but have not yielded sufficiently high pass rates for the introductory programming course (Greyling et al. 2002).
Algorithm animation allows various algorithms that introductory students need to comprehend, such as sorting algorithms, to be visually animated. This allows students to develop an understanding of the functioning of implemented algorithms (Yeh 2005). Different algorithms can operate on the same set of data and can be run in
12
Matriculation results are the final results obtained from secondary schools in South Africa. The results are a combination of a final year written examination and an accumulated mark gathered over the final two years in secondary schools.
parallel alongside one another so that students can explore differences between algorithms and how the order and nature of data can affect each. Simulations of data structures and important recursive algorithms that apply to them, such as binary trees with count, sum, pre-order traversal, amongst others, have been developed in the department as well (Klitsie 2005).
Research has further been conducted in iconic programming languages, a specific implementation of which was B# (Brown 2001; Thomas 2002; Yeh 2003). The use of iconic programming languages has been established as being of benefit to students. It promotes a better understanding of the control flow within a program and allows the student to be more concerned with creating solutions to problems than with the syntax required to implement it (Cilliers 2005). The use of B# is particularly useful for high risk students, i.e. whose predicted mark falls in the range 41%-50%.
At some stage, however, students are required to move to a textual programming language environment due to various factors, such as more advanced concepts being required to be taught and the usage of general purpose programming languages. At the moment, iconic programming languages are not employed in the commercial sector, as they are of limited use in large scale programming projects (Whitley 1996; Warren 2003). Visual representations of small programs are beneficial, but when programs become too large, the visual clutter creates a high cognitive load, negating the benefit of visualisation. So while iconic programming languages are beneficial, it is not practical or feasible for them to be the only vehicle of instruction for programming. The previous research naturally lead to the current study where PDEs are examined for ways in which the mechanics of programming can be made less cognitively challenging and whether such a PDE benefits novice programmers or not.
1.5 Focus and Methodology of the Study
Numerous people have stated that professional-PDEs are non-conducive to learning to program and some even state that they can actually be detrimental to novice programmers13. Most evidence for these stated views, however, is anecdotal in nature. There have been no empirical studies found, with the exception of one (de Pasquale
13
(Freund & Roberts 1996; Kölling 1999a, b; Ziegler & Crews 1999; McIver 2002; Donaldson 2003; Ko & Uttl 2003; Mueller & Hosking 2003; Murray et al. 2003; Storey et al. 2003; Warren 2003; Reis & Cartwright 2004)
III 2003), comparing a professional-IDE with a pedagogical-PDE to validate such claims. Unfortunately, the results of this study are questionable due to some of the data collection methods employed14.
Thus the main question this study attempts to answer is:
Q1: Do novice programmers in an introductory programming course perform better when using a pedagogical-PDE designed to support mechanics of programming rather than a professional-PDE?
For the purposes of this study, novice programmers are defined as students who have registered for the very first time, in their first year at the NMMU, for the introductory programming course (WRA101). This definition is similar to the definition of others, in which novice programmers are those computer users who typically have little or no experience in the domain of programming (Mayer 1981; Sutherland 1995; Cilliers 2005).
The introductory programming course is a six-month course taught in the first half of the academic year. The programming language used is Object Pascal, the programming language of Borland© DelphiTM. The PDE used is either Borland© DelphiTM 6 Professional Edition (the professional-PDE) or a modified version thereof called SimplifIDE (the pedagogical-PDE, see Chapter 4). All students are required to be proficient in basic programming using Object Pascal upon completion of the course.
The course commences with a problem-solving component in which the decomposition technique is taught. The remaining weeks cover basic concepts of imperative programming, namely: sequence, iteration, selection, variables, basic data types (integer, real, string and boolean), intermediate data types (arrays and text files), Input/Output and sub-programs (functions and procedures). Students are expected to write code that adheres to various style rules and to document code sufficiently. Students are expected to have a firm grasp of the mechanics of programming: writing, compiling, running and debugging programs. Students are assessed by a number of
14
The number of compiles and syntax errors of the pedagogical-PDE are captured, but subjects using the professional-PDE are asked to make an estimate for these values.
different assessments: weekly practical evaluations, one written and one practical semester test and a final written examination.
In order to answer Q1, it is necessary to answer the sub-questions:
Q1.1: What features should a pedagogical-PDE have to support novice programmers with the mechanics of programming in an introductory programming course?
Q1.2: Do novice programmers using a pedagogical-PDE supporting the mechanics of programming exhibit a positive measurable difference compared to those using a professional-PDE?
In order to answer Q1.1 it is necessary to examine how novice programmers think,
how they learn to program and also common problems they encounter while learning to program. This has been done in Chapter 2 through a critical analysis of literature. From this, a set of heuristics are derived that a pedagogical-PDE should meet in order to support learning to program.
Chapter 8:
Conclusion of Investigation Chapter 2:
The Process of Learning to Program Chapter 3: Program Development Environments Chapter 5: Investigative Research Methodology Chapter 6: Investigative Study Results Chapter 7: Interpretation of Results Chapter 1: Research Context Chapter 4:
Design & Implementation of SimplifIDE
Answering
Q1.1
Answering
Q1.2
The heuristics derived in Chapter 2 are refined into a set of requirements that PDEs supporting the mechanics of programming should meet in Chapter 3 (Figure 1.3). Existing professional-PDEs and pedagogical-PDEs are examined to determine features implemented and these are matched against the set of requirements. A critical analysis of literature and extant systems analysis was conducted and the results described in Chapter 3.
Most pedagogical-PDEs, however, do not implement many of the features in the feature set of an ideal pedagogical-PDE. To this end, a pedagogical-PDE, SimplifIDE, was developed and implemented. The development of this pedagogical-PDE is described in detail in Chapter 4, with special attention being paid to design decisions and the motivation for the inclusion of certain features.
In order to answer Q1.2, it is necessary to determine whether novice programmers
perform better using a pedagogical-PDE rather than a professional-PDE. This necessitates the conducting of an empirical study. There needs to be common ground between both PDEs and the same measures need to be obtainable in both PDEs. In order to conduct the study, students deemed to be novice programmers are partitioned into identical groups, namely a control group using the professional-PDE and a treatment group using the pedagogical-PDE. The selection method and experiment methodology are discussed in detail in Chapter 5. Each PDE must be instrumented to capture the same information from both experimental groups. All participants must have the same assessments administered.
Q1.2 can be refined as:
Q1.2.1: Do novice programmers using a pedagogical-PDE exhibit better
academic performance than those using a professional-PDE?
Q1.2.2: Do novice programmers using a pedagogical-PDE have more positive
perceptions than those using a professional-PDE?
Q1.2.3: Do novice programmers using a pedagogical-PDE exhibit better
Q1.2.1 seeks to determine if there is a difference in the grades of participants. The
grades obtained for assessments, as well as the final composite grade, are an indication of the knowledge that a novice programmer has acquired during the introductory programming course.
On a weekly basis, during practical sessions, participants will be required to complete an online questionnaire regarding the previous week’s practical tasks. Questions will be asked to determine a participant’s perceived learning and perceived difficulty experienced, amongst others (Appendix F). The answers will provide a measure of perceived performance, which Q1.2.2 examines.
Students are required to perform weekly practical learning activities and a single practical assessment. During these practical activities, programming events are continuously captured and require that both the professional-PDE and the pedagogical-PDE be instrumented to capture events while a participant is programming. Events captured will include:
• start and end time of a session;
• when a program was compiled or run and a copy of the source code at that point; and
• when help was accessed.
Analysing these events, it will be possible to determine programming behaviour metrics, such as the average number and time between compiles or runs, the average number and types of errors, the amount of change made to a program between compiles or runs and whether help was made use of or not. Q1.2.3 will make use of
these metrics to determine whether there is a difference between programming behaviour of participants in the experimental groups or not.
The PDE can also be used to instil good programming practices, such as secondary notation (code indentation) and code documentation. The pedagogical-PDE will be instrumented to facilitate these practices. Chapter 5 outlines the selection process, the experiment and the analysis of the data collected.
Data collected from the experiment will be analysed and the results reported in Chapter 6. In Chapter 7, the results are interpreted to answer the questions posed in this section. Based on the answers of the questions, it should be possible to determine
if professional-PDEs are actually as detrimental to novice programmers learning to program as researchers have indicated in the literature. Finally, the investigation is concluded in Chapter 8 with a summary of the entire study, the important findings and areas for future research.
Chapter 2
The Process of Learning to Program
2.1 Introduction
The manner in which novice programmers learn to program and factors that influence a successful outcome need to be considered when determining the features that a pedagogical-PDE supporting the mechanics of programming should implement (Q1.1
in Chapter 1). When learning to program, the novice programmer interacts with a number of entities. The interactions between each of the entities and the novice programmer are required when learning to program, but each interaction can become a barrier to the learning process.
Novice Programmer Programming Language Programming Development Environment Introductory Programming Course Barriers to Learning to Program symbiotic interaction (a) (b) (c) (d)
Figure 2.1: Factors Influencing Learning to Program
Figure 2.1 indicates some of the main entities that the novice programmer interacts with in a typical introductory programming course while learning to program. A novice programmer attends an introductory programming course, which can be
contact-based of distance-based, and is expected to create programs in a particular programming language within the prescribed PDE.
The interaction between the novice programmer and the introductory programming course (Figure 2.1(a)) is affected by the course content, presentation style and assessment methods, each of which can be a barrier inhibiting the process of learning to program. For example, incorrect matching between the novice programmer learning style and the presentation style of the lecturer can cause the novice programmer to become distracted or not absorb the information effectively.
Similarly, the interaction between the novice programmer and the programming language (Figure 2.1(b)) affects the learning process. For example, programming languages typically have a strict set of rules that need to be adhered to and it may not be possible to execute programs that deviate even slightly from these rules. This can lead to novice programmers concentrating excessively on “getting the rules right”, causing the novice programmer to lose sight of higher level issues regarding programming and regard programming as “fighting the compiler”.
PDEs are required to create, execute and debug programs on a computer. The interaction between the novice programmer and the PDE (Figure 2.1(c)) can become a barrier to the process of learning to program. Overly complex PDEs, with a multitude of options and convoluted set-up procedures can make the process of programming far more complex than need be for novice programmers.
The programming language and the PDE form a symbiotic system (Figure 2.1(d)) and are typically seen as one system by the novice programmer. The interaction between the PDE and the programming language can reduce or exaggerate any barriers already existing between the novice programmer and programming language (Figure 2.1(b)) or the novice programmer and the PDE (Figure 2.1(c)).
This study is particularly interested in the interaction between the novice programmer, programming language and the PDE (Figure 2.1(b-d)). The main focus is to determine whether the modification of the PDE can:
• alleviate or remove some of the barriers between the novice programmer and the PDE; and
• alleviate or remove some of the barriers between the novice programmer and the programming language via the symbiotic interaction between the programming language and the PDE, if it is assumed that the programming language itself cannot be changed.
The introductory programming course and the interaction between it and the novice programmer (Figure 2.1(a)) will not be focussed on specifically. In addition to the entities that the novice programmer is required to interact with, there is another factor that can influence the outcome of the process of learning to program, namely motivation.
Novice programmers may potentially have the intellectual capacity to program, but this, in itself, may not be enough to successfully learn to program. It has been proven that poorly motivated students, even though they may possess the intellectual capacity, may fail to learn to program if they are poorly motivated (Zimmerman 1995). Each of the interactions between the novice programmer and the introductory programming course, programming language and PDE should maintain or build the motivation of the novice programmer.
The remainder of this chapter examines what is required in order for in-depth learning to take place followed by how motivation is affected in an individual. Finally, the process of learning to program is discussed in light of factors that affect in-depth learning and motivation in an introductory programming course.
2.2 Requirements for Meaningful Learning
Learning can occur as two kinds of learning, superficial or in-depth1. Superficial learning is characterised by novice programmers memorising concepts of programming for automatic future use. In-depth learning, on the other hand, requires the comprehension of the effects of programming concepts so that the concepts can be applied in the future implementation of novel solutions. Programming often requires the application of existing knowledge to new situations, so it is important that the novice programmer understands programming at an in-depth level of learning, rather than superficially.
1
(Mayer 1981; Lockard 1986; Kushan 1994; Studer et al. 1995; Wiedenbeck 1999; Dingle & Zander 2001; McCracken et al. 2001; Jenkins 2002)
Meaningful learning, or in-depth learning, is the process by which a learner augments existing knowledge in long-term memory with new knowledge in short-term memory by forming connections (Bransford 1979). The existing, long-term knowledge, known as a schema, is extended in a process known as assimilation. Short-term memory is a temporary, limited capacity workspace for containing and manipulating information under attention. Long-term memory, or schema, is an organised information store, considered to be of unlimited capacity, which retains knowledge over a long period of time without conscious effort. According to Miller (1956), long-term memory for programming consists of syntactic and semantic knowledge.
Learning to program is highly technical information and in order for meaningful learning to occur, the following steps need to happen (Mayer 1981):
1) Reception: Incoming information enters the human cognitive system (indicated by the arrow labelled (a) in Figure 2.2). For this information to enter short-term memory, the learner needs to pay attention to it.
2) Availability: In order for the assimilation of the incoming information to occur, the learner must have appropriate prerequisite concepts in long-term memory (indicated by the box (b) in Figure 2.2).
3) Activation: The final step required for meaningful learning to occur involves the learner actively using the prerequisite concepts from long-term memory so that the new information can be connected to it and assimilated (indicated by the arrow labelled (c) in Figure 2.2).
Short Term Memory
Long Term Memory (b)
Stimulus Respon
(a)
(c)
se
Figure 2.2: Framework for Meaningful Learning
If any of the steps above are omitted, then meaningful learning will not occur. If such a case occurs, then the learner will be forced to learn each piece of new information as a separate piece of information by rote learning (superficial learning). While in-depth
learning and superficial learning both allow new information to be stored in long-term memory, when a learner is required to transfer information to a new situation, information learned using superficial learning can not be transferred (Mayer 1981). Instead, each new situation is treated as totally unrelated and requires a new learning process.
Short-term memory has a very limited capacity and is able to retain seven plus or minus two items of information at a time (Miller 1956). This phenomenon is known as the memorisation problem. The memorisation problem implies that any information that cannot be contained in short-term memory is discarded, often without the person being aware of the loss. In other words, if unnecessary information items occupy too much short-term memory then it is less likely that a successful transfer of the desired information items to long-term memory will occur via in-depth learning. For example, if the syntax of the programming language requires active concentration while the novice programmer is attempting to learn an algorithm, then the syntax will occupy a number of information item slots in short-term memory. This will leave less information item slots available in short-term memory for the algorithm to occupy, thereby increasing the chance that the algorithm will not be correctly assimilated into long-term memory via in-depth learning. This would result in the novice programmer being able to reproduce the algorithm, but not being able to apply it to new situations. It is possible to expand the capability of short-term memory using a process known as chunking. During chunking, a number of pieces of information are conceptually connected to form a unique item, occupying a single information slot in memory. Novice programmers are not able to create large chunks of information, but as they gain experience, larger chunks are formed from frequently observed patterns known as beacons (Curtis 1982). Programmer ability is indicated by the nature of the concepts in a chunk. For example, a novice programmer may only be able to identify beacons consisting of a couple of statements, such as “swap two variables” or “initialise variable”, while an expert may be able to identify much larger beacons such as “sort students in ascending order of class mark”.
Therefore, for in-depth learning to take place while learning to program, it is important to minimise the cognitive load placed on the novice programmer by
information not directly related to the current information to be learnt. In-depth learning can only occur under certain conditions, but even so, there is another factor, motivation, that plays a large role in the overall success or failure of a novice programmer in an introductory programming course.
Even should all the necessary requirements be in place for in-depth learning to occur, it is still not guaranteed that an individual will be successful in learning to program. An individual’s motivation can have a large influence on learning to program and the next section discusses this in more detail.
2.3 Motivation
Motivation is very personal and the exact sources thereof differ from person to person. There are a number of well-documented sources of motivation for individuals (Jenkins 2001b), as shown in Table 2.1.
Source Description
Extrinsic External influences, such as expected future reward. For example financial gain from employment.
Intrinsic Individual is a source of motivation to self. For example, finding a subject interesting.
Social Desire to please a third party, such as family, friends, sponsors or educators.
Achievement Desire to perform academically, even out-performing peers.
Null No clear motivating factors. For example, students who “fell” into a particular academic programme for no particular reason.
Table 2.1: Sources of Motivation
Of the sources of motivation mentioned in Table 2.1, the most influential is intrinsic motivation. If novice programmers are unmotivated, the chances of them succeeding in a course that is notorious for being difficult is greatly reduced (Ladd & Harcourt 2005). The extent to which motivation can be observed has been defined as (Biggs 1999; Jenkins 2001b):
Motivation = Expectancy × Value
Expectancy is the measure of how much the novice programmer expects to be able to succeed in their studies, while value is a measure of how much value, or worth, is
placed on the successful achievement of their studies. This is closely linked to the sources of motivation mentioned in Table 2.1, as both expectancy and value can be affected by different sources of motivation. If either expectancy or value drops to zero, then the novice programmer will not be motivated to succeed at all and will not engage in the educational environment.
The aim then, is to increase or maintain the expectancy to succeed and/or the value that students place on programming during the process of learning to program. Value is determined largely by external factors, such as expected future financial gain, achieving high marks, pleasing parents, and will not be discussed further. Students, however, must expect to be able to succeed in learning to program (Jenkins 2001b). The expectancy to succeed in an introductory programming course is affected by a number of different factors, one of which is perceived self-efficacy (Woszczynski & Guthrie 2003; Wiedenbeck 2005). Perceived self-efficacy is an estimation of ones' ability to successfully perform target behaviours. Individuals who judge themselves capable of performing certain tasks are found to attempt and successfully execute them (Byrne & Lyons 2001). A novice programmer’s self-efficacy beliefs come from four sources (Wiedenbeck 2005):
• personal experiences of task mastery;
• second-hand experiences, such as observing a peer performing a task; • verbal persuasion; and
• emotional arousal, consisting of monitoring of fatigue, stress and anxiety levels to assess self-efficacy.
The most influential source of self-efficacy is that of personal experience. In other words, the successful or unsuccessful mastery of tasks is the most direct and powerful factor influencing a novice programmer’s self-efficacy and hence their overall motivation. If novice programmers are unable to master tasks, such as getting a program to compile, these will have an impact on motivation levels and academic achievement (Gist et al. 1989; Horn et al. 1993).
In the early stages of skill development, efficacy beliefs have been shown to be malleable. This implies that during the beginning stages of learning to program (or any other skill), it is possible to affect self-efficacy beliefs positively or negatively. While a single failure to master a task will not affect self-efficacy negatively, fewer
