D e pa rt m e n t o f I n f o r m at i c s
P r o g r a m m i n g T h e o ry
Master Thesis
Novice Di
fficulties with
Language Constructs
Alexander Hoem Rosbach
Abstract
Programming is a difficult skill to learn, and programming courses have high dropout rates. In this thesis we study the problems that students have during their first introductory programming course at The University of Bergen. We inspect the solutions that they submit for the given assignments, and look at the frequency of the different kinds of mistakes in their work.
We present a problem taxonomy that we use to classify the mistakes found to be the most common, and conclude that a significant part of the problems are observable misconceptions. We introduce a web-based tool, Javis, that we have developed to aid the students with these kinds of problems.
Based on the experience and knowledge gained during this work we present a proposal of a grading by annotation scheme. This scheme is specif-ically designed to increase the quality of the feedback given to students on their submitted work and provide valuable feedback to the teachers regarding the problems that their students have.
Acknowledgements
Foremost, I want to thank my supervisor Anya Helene Bagge for providing the initial inspiration for the thesis, and contribution of ideas, support and advice during my work.
I would also like to thank May-Lill Bagge for her reviews and feedback of my work. Her non computer science perspective and input was a great help.
Contents
Contents i
List of Figures v
List of Tables vii
1 Introduction 1
1.1 Background and Motivation . . . 1
1.2 Research Questions . . . 2
1.3 Related Work . . . 3
1.4 Thesis Structure and Outline . . . 3
1.4.1 Outline . . . 4
2 Learning Programming 7 2.1 Programming Skills . . . 8
2.2 The Challenges . . . 9
2.3 Student Problem Taxonomy . . . 11
2.3.1 Concept-based problems . . . 12
2.3.2 Knowledge-based problems . . . 13
2.4 INF100 Course . . . 14
3 Study Design and Problem Taxonomy 17 3.1 Study Description . . . 17
3.2 Data Set . . . 18
3.3 Problem Taxonomy – Most common problems . . . 20
3.3.1 w r n g c o n d – Wrong condition . . . 20
3.3.2 b d ry c o n d– Boundary case condition . . . 21
3.3.3 l o o p c r e at e– Loop instantiation problem . . . 22
3.3.4 c m p l x c t r l s– Unnecessarily complicated . . . 24
3.3.5 c a l ln o – Missing method call(s) . . . 25
3.3.7 b a d a r i t– Erroneous arithmetic . . . 27 3.3.8 b o o l ac c u m– Accumulate boolean . . . 28 4 Result Analysis 31 4.1 Lazy students? . . . 31
4.2 Most common problems . . . 32
4.3 Parameter passing and references . . . 34
4.4 Conditions . . . 35
4.5 Loop constructs . . . 36
4.6 Method calls . . . 37
4.7 Concept-based problems . . . 37
4.8 Conclusion . . . 39
5 Tool Assistance – Javis 41 5.1 Requirements . . . 41
5.2 Implementation . . . 43
5.2.1 Java evaluation . . . 44
5.2.2 Abstract syntax tree . . . 45
5.3 Tool Design . . . 45
5.3.1 Design concepts . . . 46
5.3.2 Evaluation . . . 49
5.4 Visualisation Examples . . . 51
5.4.1 Step by step evaluation . . . 51
5.4.2 Nested scopes . . . 53
5.4.3 Parameter passing . . . 54
5.5 Related Work . . . 54
5.6 Conclusion . . . 56
6 Proposal: Grading by Annotation 57 6.1 Annotation Syntax . . . 58
6.2 Grading . . . 59
6.2.1 File annotations . . . 59
6.2.2 Meta annotations . . . 60
6.2.3 Problem annotations . . . 60
6.3 Collecting the Data . . . 63
7 Conclusion 67 7.1 Status Summary . . . 67
7.2 Future Work . . . 68
7.2.1 Javis . . . 68
A Student problems 71
B Annotation occurences 75
List of Figures
1.1 Graduated students relative to admitted students the
theo-retical starting year. . . 2
3.1 Inspected student solutions. . . 18
3.2 Decrease of submitted solutions throughout the semester. . . 19
3.3 Problem taxonomy . . . 20
4.1 The most common problems among the students. . . 33
4.2 Students who had parameter passing problems. . . 34
4.3 Students who had problems with conditional expressions. . 36
4.4 Domain model (UML class diag.) of semester assignment 2. 37 4.5 Students with problems from the concept-based and knowledge-based categories. . . 38
5.1 Javis with a Java program loaded. . . 44
5.2 Simplified domain model (UML class diag.) of Javis . . . 46
5.3 Scope stack example . . . 48
5.4 AST ofint number = 5 + 3; . . . 50
5.5 Example of step by step evaluation. . . 53
5.6 Nested scope example . . . 54
List of Tables
6.1 File annotations . . . 60
6.2 Meta annotations . . . 61
6.3 Variable problem annotations . . . 62
6.4 Conditional expression problem annotations . . . 62
6.5 Method call problem annotations . . . 63
6.6 Scope problem annotations . . . 64
Chapter 1
Introduction
In this chapter we explain the background and motivation for our work, present our research goals and give an outline of the thesis.
1.1
Background and Motivation
Learning to program is a difficult task that many students struggle with and fail to complete. Programming courses are generally regarded as difficult and many first year programming students perform much more poorly than hoped for [18] in these courses, resulting in high dropout rates [9, 24]. Our department offer two bachelor programmes, Computer Science1and
Computer Technology2, and compared to the number of students admitted
to the programmes, very few students graduate the scheduled graduation year (see Fig. 1.1).
In the capacity as a teaching assistant in the introductory programming course (INF1004) at the University of Bergen, the author has observed
the struggle of the students first hand. By assisting the students at lab sessions and grading their submitted solutions to assignments, the author has observed a multitude of problems and difficulties. This motivated us to investigate this more in depth, and we especially wanted to see if some problems were as widespread as they seemed to be, and if particular problems persisted throughout the semester.
1http://www.uib.no/studyprogramme/BAMN-DVIT 2http://www.uib.no/studyprogramme/BAMN-DTEK 3The University of Bergen
0 10 20 30 40 50 60 2008 2009 2010 2011 2012 42 31 57 42 47 8 8 16 16 5 Students
Graduated Admitted 3 years earlier
Figure 1.1:Number of students that graduated each year (2008-2012) relative to the number of students admitted to the study programmes the theoretical starting year (UoB3).
1.2
Research Questions
Our plan is to study the problems that students learning Java as their first programming language struggle with. We aim to identify and study mistakes, misconceptions and unsolved requirements in the solutions that the students submit to assignments given in the introductory programming course at the University of Bergen. Studying these problems, we then want to find out if:
• some problems are more common than others.
• particular problems, such as parameter passing and references is as common a problem as our experience suggest.
• problem occurrences can be related to misconceptions or missing knowledge.
Based on the experience and knowledge we gain in that study, we intend to develop a proposal of how to grade student assignments and collect data about problem occurrences, without increasing the workload or degrading the quality of the feedback to the students. This grading scheme can provide the teachers with valuable knowledge, as they then are able to address the most common misconceptions among the students.
Students often have incorrect conceptions of programming concepts [5, 16]. Our hypothesis is that a significant part of these misconceptions can be observed using the right techniques. Du Boulay et al. [10] suggests that how the programming language affect the computer should be supported with some kind of concrete tool which allows the interaction to be observed. We support this and believe that with an easy to use tool with proper visualisation of how programs actually work, step-by-step, the students themselves will be able to identify and understand these kinds of problems. We intend to develop such a tool.
1.3
Related Work
Lathinen et al. [14] analyses a survey of students and teachers opinions of which elements they consider difficult when learning programming. Parts of this survey is strongly related to what we look at in this thesis, though we have chosen other methods of gathering data and we have a more narrow focus. Robins et al. [24] reviews much research regarding learning and teaching of programming and present a discussion of the findings. Spohrer and Soloway [27] present an empirical study of the general belief of novice mistakes.
Johnson et al. [12] developed a bug classification scheme that relate bugs to program constructions and underlying misconceptions and Spohrer et al. [26] created a problem-dependent classification scheme based on Johnsons scheme. Our grading scheme (Chapter 6), and its taxonomy (Section 2.3), has similarities with both classification schemes, it relates problematic code segments both to the problem description (requirements) and the language constructs. We then discuss and connect problems with plausible underlying misconceptions.
1.4
Thesis Structure and Outline
To introduce the problem domain of this thesis we start with a discussion of the learning process of programming, where we look into the skills that the students must learn and the challenges that they will meet in the process. Based on that discussion and previous research regarding classification of student problems, we specify a student problem taxonomy that separates problems based on their underlying reasons. The two categories of problems that we look into are those that can be related to observable misconcep-tions, concept-based, and those that can be related to missing knowledge or
understanding, knowledge-based.
To answer the research questions of this thesis we inspected Java pro-grams developed by students, and annotated all the problems that we found. We then study the data generated by the annotations to identify the most common problems of the students, and classify them in our taxonomy. With the classification of the most common problems we aim to verify that a significant part of the students’ problems are concept-based, which we argue that the students can identify and understand themselves, with the proper assistance. We then look into particular problems and study the annotation data in detail.
To aid the students in identifying and understanding the concept-based problems, we have developed a visualisation tool which we present, and provide example executions of.
We then present a proposal of a grading by annotation scheme, that is based on the work in the other parts of this thesis.
1.4.1
Outline
A brief description of the contents of each chapter:
Chapter 2: Learning programming In this chapter we first discuss the skills that the students must master when learning programming, and the challenges they will encounter in the process. Then we specify a taxonomy that we have used to classify the most common problems of the students. We end the chapter with a description of the curriculum and organisation of the introductory programming course taught at our department.
Chapter 3: Study design and Problem taxonomy In this chapter we first give a description of the design of our study, and describe the data set used. Then we provide a thorough description of each of the problems we found were most common, and a list of plausible underlying reasons for them.
Chapter 4: Result analysis In this chapter we analyse the results of our study. First we describe the total amount of data we gathered, then we present and discuss our findings.
Chapter 5: Tool assistance – Javis In this chapter we present a tool that we are developing for students. We describe the requirements and intentions of the tool, and provide some examples of its use.
Chapter 6: Grading by annotation In this chapter we present a grading by annotation scheme that we have developed. We based it on the experience gained during the research for this thesis, and we describe how to use it, extend it and collect the data.
Chapter 7: Conclusion In this chapter we give a status summary of the work in this thesis, and we briefly explore the possibilities and future work of the grading by annotation scheme and Javis.
Chapter 2
Learning Programming
Learning programming is a complicated task that requires the student to acquire a lot of new and interconnected knowledge. This makes the teaching of programming a serious challenge for educators. With respect to the curriculum, different teaching methods, order of the topics and so on, they have many choices. Even the seemingly simple choice of which language to teach is a tough one:
• Should one teach programming using a dedicated learning language, or one in use in the industry?
• From which paradigm should one choose the language; functional, imperative or object-oriented?
• If object-oriented; should imperative (procedural) features or objects be taught first?
We gave this brief introduction into the teaching of introductory program-ming to inform the reader that the teaching methodologies of programprogram-ming are disputed. We do not consider this further in this thesis, as we take another perspective on the challenges of teaching and learning program-ming, instead we refer the interested reader to the extensive research in this area. For example Robins et al. [24] for a review and discussion on learning and teaching programming, Dale [8] for a survey on CS1 content practices, Schulte & Bennedsen [25] for a study of what is taught in CS1 and Bailie et al. [4] for a panel discussion of the object first approach.
In this chapter we discuss what goals the student should reach during the learning process and the difficulties the students encounter along the way. Then we present and discuss the problem taxonomy that we use in this thesis to classify the typical problems students have. Following that we
give a thorough description of the CS1 equivalent course, INF100, offered at our department.
2.1
Programming Skills
The end-goal of students learning programming are very different, and depends primarily on what they are planning to use the skill for. Some are interested in applying the skill within academia, some as developers, others learn out of curiosity. No matter what end-goal though, it is fair to say that the student should be able to express solutions for typical computer-solvable problems in the language he or she is learning.
First off, this requires that the student must learn the language itself, that is, the syntax and semantics of the necessary language features (constructs). By necessary features, we argue that it is not required that the student learns all the features of the language initially, as there are many (depending on the language) operators, keywords and constructs that are not immediately needed. Typically bitwise operators, exception-handling constructs, generics, polymorphism and inheritance are such more complicated and not initially needed features. Though some languages may force you to teach them early, e.g. Java forces exception-handling quite early.
In addition to learning the semantics of the language constructs, the student should also learn typical ways of applying them. In literature this is often referred to as schemas, or sometimes plans, depending on the author [24]. We only use the term schema, and we use it to refer to a certain way of combining language constructs to solve specific problems, i.e. a schema may be a way of combining language constructs to calculate the sum of the numbers in a list (array) or read and validate user input etc. These trivial schemas will give the student both a better conception of the construct semantics, and knowledge of actual usage.
The next basic skill is the ability to properly interpret a problem descrip-tion and generate a general nodescrip-tion of how to solve the problem. We do not however, recommend writing down an explicit solution in natural language and then translating that into the programming language. Spohrer and Soloway [27] argue that this may be a pitfall of possible misconceptions, as some natural language words are used as keywords for constructs in the programming language and do not necessarily have the expected semantics. Using this notion, or mental image, the student should then be able to express a procedural plan of how to apply a combination of schemas in a sequence that ultimately solves the problem. Depending on the size and complexity of the problem, it might be necessary to first divide it into smaller
problems.
Linn and Dalbey [15] propose that given a good learning environment the student should achieve the following “chain of cognitive accomplishments”:
1. Language features; understanding the semantics of the language structures.
2. Design skills; the ability to plan procedurally, apply schemas/plans, test and reformulate code.
3. Problem-solving skills; application of knowledge and strategies, in a way that is abstracted from the specific language being taught, i.e. able to apply it with other languages and settings.
It is possible to teach these skills separately without depending on the previous skill, e.g. teaching a schema for calculating the average of a list of numbers without using programming language examples. However we argue that the order given in this chain is important and should be followed by educators. This will provide the students with the means to experiment and observe that the schema works when applied using the respective constructs.
The ACT models by Anderson [2] suggest that abstract representations of knowledge cannot be learned directly, they can only be learned by practicing the operations on which they are based. The process of experimenting and observing consequences, learning-by-doing, is very important when learning programming and should be a primary focus. This will give the student a feel of accomplishment and improve the student’s conception of how it works.
Though we do recommend selecting subsets of constructs and follow the chain individually for them, e.g. selecting the if-statement, variables and user input calls, providing the student with schemas for these and skills to solve user input validation problems and actions based on the input. This way the student is able to achieve the full chain of knowledge for individual subsets quite early and gain experience with them.
2.2
The Challenges
In the process of learning programming the students will encounter many difficulties and challenges. Du Boulay [10] describes a list of five overlapping problem domains, that we describe in detail below:
2. The notional machine. How a language interacts with the computer. 3. Notation. Language syntax and semantics.
4. Structures. Possess and understand schemas.
5. Pragmatics. Planning, developing, testing, debugging etc.
General orientation (1) The understanding of what programs are, that they can be used to solve problems and what kinds of problems are solvable. Within the learning environment students are provided with problems to solve, these are implicitly solvable, and we argue that this is a challenge that may become more challenging later, outside the confines of the learning environment.
The notional machine (2) Du Boulay [10] describes this as:
An idealized, conceptual computer whose properties are implied by the constructs in the programming language employed.
That is, having a correct understanding of how the language constructs affect the computer, e.g. correct mental models of how an array is stored and accessed in memory, how references work etc. It is important to note that different languages have different models, though some are similar. This may be one of the most challenging parts of programming. The student may have a model that works in many cases, or even most, but have some cases where it breaks down and behaves unexpectedly. It is very difficult for the student to identify mistakes that are caused by this, since the entire conception of how the constructs work is tied to this model. It is improbable that the student even will suspect that the reason a section of the program does not work as intended is because the model that the student has observed to be correct in all previous cases really is incorrect.
Notation (3) The correct understanding of the syntax and semantics of the language. That is being able to express the language constructs syntactically correct and understanding the semantics of them. Any proper IDE1will provide the student with valuable assistance in the form of syntax highlighting, code formatting and active syntax error reporting. The error reporting removes some of the need for the student to interpret compile errors (which may seem very cryptic). The IDE reports errors by highlighting
the location and providing an error message for any syntax errors. Syntax highlighting (color and emphasis) and formatting will make the code easier for the student to read and differentiate between language keywords, literals, variables etc., making the whole task of writing syntactically correct programs a lot easier.
Spohrer and Soloway [27] conclude that misconceptions about language constructs does not seem to be as widespread as is generally believed. Though the semantics of the constructs may prove a bit challenging at times, the student should be provided with good explanations from the course literature and lectures.
Structures (4) Knowledge of schemas that can be applied to solve small sub-goals of a problem, when and where to apply them and how to combine them. Understanding a schema properly, i.e. being able to apply it for problems not entirely the same as those given as examples, may prove difficult, and combining or nesting them; even more difficult. It requires the student to understand that the schema is a general solution, and understand the inner-parts of it, and being able to mold it to the problem at hand.
Pragmatics (5) Planning, developing, testing, debugging etc. are skills that are developed with experience and will be a challenge throughout the process of learning programming.
We use some of these problem domains in Section 2.3 where we specify a problem taxonomy for student mistakes. Problems in the notional machine domain and parts of the problems in the structures domain are related to misconceptions that are important for the concept-based category. While problems in the notation domain and other problems in the structures domain are related to missing knowledge and fall into the knowledge-based category.
2.3
Student Problem Taxonomy
The mistakes of students are many and diverse, and though they on the sur-face may seem to be the same problem, e.g. by having the same consequence and appear in an equivalent location, there may be very different underlying reasons. The same can be said vice versa, some mistakes may have the same root cause but appear in different locations, or have different consequences. This makes the classification of student mistakes quite complicated and
require a lot of interpretation.
In this section we present the taxonomy scheme that we use in this thesis. We base our work on that of Spohrer and Soloway [27] who built a taxonomy for student problems where they classified mistakes into two categories, construct-based problems and plan composition problems. These categories they describe as not mutually exclusive or exhaustive. To determine which category problems should be placed in, they identified plausible underlying causes for the problems and used them as determinants.
We use some of the causes that Spohrer and Soloway identified, refine some of them and provide additional ones that we have identified, as determinants for our taxonomy. However, we have chosen other categories, concept-based problems and knowledge-based problems.
Brooks [6] describes the program comprehension process as expectation driven by creation, confirmation and refinement of hypotheses. This is what we attempt to capture with the concept-based problem category. Problems that can be identified and rooted out by creating hypotheses and confirm or refine them.
2.3.1
Concept-based problems
Typical concept-based problems are those that are related to incorrect schema applications or misconceptions of the notional machine. Problems we classify as concept-based are those where the student should be able to understand the mistake by observing the behaviour of the program. With a combination of reading, manually tracing the code and observing the results of executing the program, while experimenting with the data set, the student should be able to locate and correct these mistakes. Instances of these problems are expressions, statements or constructs in a program that are either unnecessary and does not affect the result of the program, or in some way prevent the program from working as it should.
I) Incorrect notional machine. An incorrect notional machine may have unexpected obscure consequences that may be challenging to notice and understand. Though difficult, they are identifiable by observing the behaviour of the program.
II) Natural language translation. The student may devise a solution to a problem in natural language. Translating from natural language to the programming language is not straight forward, and may cause problems for the student. Observing the behaviour of the program,
the student should be able to identify that it behaves differently than expected.
III) Misconception of construct semantics. The student may possess some conception of construct semantics that is incorrect, but seems to be correct based on previous experience. This may be an almost correct conception, that has some special case(s) it does not cover, or it may be based only on trivial cases and fail when more complicated parts of the semantics matter, e.g. evaluation order. These mistakes are observable, as the student has a correct intention, but failed to apply the construct due to some misconception.
IV) Specialisation problem. The student may find it challenging to apply the general schema correctly to a concrete problem. When customising the schema to the particular situation at hand, the student may deviate slightly from the correct application and create an incorrect solution. These mistakes should be observable as the student has a correct understanding of the schema and knows what is intended.
V) Test data. The data set the student is executing the program with may not be good enough to expose the problem, and if it was supplied with the assignment, the student has no reason to suspect that it is insufficient.
2.3.2
Knowledge-based problems
Knowledge-based problems are those situations where the student does not have the proper knowledge or understanding to solve a problem. This may be due to lack of schemas to apply, or an inadequate understanding of them (thus not able to apply them). Instances of these problems may be segments in a program that the student has either left completely blank or placed an empty skeleton of a schema. Other examples of instances may be a dynamic problem solved in a static way, with the consequence that they only work for the given example data set.
I) Interpretation of specification. Correct interpretation of the assignment specification may be challenging. Explicit requirements may be ne-glected or perceived incorrectly and implicit requirements may be missed.
II) Schema knowledge. The student may not possess the necessary schemas or adequate conceptions of the schemas to solve the problem.
III) Existing environment. Students may find it difficult to express their intention using the relevant existing parts of the program environment, e.g. identifying the correct variables and methods to use.
2.4
INF100 Course
INF100, our CS1-equivalent course, gives a thorough introduction to pro-gramming and is a mandatory course in both bachelor degrees offered at our department. The course, or an equivalent course, is a prerequisite for all other programming courses taught at our department. Though it is primarily aimed at computer science students, it is also offered and even recommended to students of other natural science degrees, and especially to Mathematics and Physics. This means that there is a lot of diversity in the students background, skill and motivation, and that the progress of the course must be adjusted accordingly. The primary motivation of the course is to teach students how to use a programming language to solve problems with a computer and doing so in a general sense. By that we mean that even though the course teaches one language, namely Java, it aims at doing it in such a way that the students learn the “programming way of thinking” and hence will be equipped with the knowledge to learn and use other languages as well.
Curriculum
We now provide a chronological list of the curriculum of INF100, which is reflected in both the lectures and the assignments given in the course.
1. Imperative language features. The first part, about one third, of the curriculum focuses on the basic concepts and constructs of imperative languages, e.g. variables, data types, expressions, control flow, arrays, input/output and assignment. The students will in addition be in contact with, and use, objects in this part, which is due to Java’s strong object-oriented nature. That means that a quick introduction to objects and data abstraction is required (and given) in this part, as well. 2. Objects. About one third into the curriculum the focus will shift
towards object-oriented-programming, where object communication, inheritance and polymorphism will be introduced. This is primarily a theoretical introduction and little to no focus will be given inheritance and polymorphism in the course assignments.
3. Algorithms. About half way into the curriculum algorithms, e.g. search-ing and sortsearch-ing, will be introduced, followed by abstract data types and dynamic data types, e.g. linked lists etc.
4. Recursion. Then recursion and recursive algorithms are presented, which also is given little to no attention in the assignments.
5. Exception handling. Towards the end exception handling is introduced both in the lectures and in assignments, followed by file- and stream handling.
6. GUI. Graphical user interfaces may be given a short introduction, if there is interest and available time for it.
Organisation
There is a heavy emphasis on learning by doing in the course, which is reflected by the many mandatory assignments and available lab sessions. We strongly encourage the students to attend at least one of the five lab ses-sions offered each week. If there is room, they may attend multiple sesses-sions. In these sessions it is envisaged that the students will work on their current course assignment, and it is encouraged that they co-operate and help each other. At each session a teaching assistant will assist the students with their problems, and preferably in an educational way. The assistants are all talented and experienced students who have come further in their studies, many at the end of their bachelor degree and some are working on their mas-ter thesis. There are however no special requirements to pedagogical skills or education, though the Psychological faculty offer them a course in pedagogy. In total there are ten assignments, of which seven are exercises and three are larger mandatory semester assignments. The exercises are only graded with a pass or fail, though the teaching assistant grading it will provide comments for the work. Of these, the student must pass at least five, but is required to submit something for all of them – even if it is an empty submission. The larger semester assignments are all graded with an individual score, where the total score of all three counts 30% towards the final grade in the course.
Chapter 3
Study Design and Problem
Taxonomy
The goal of this study is to identify and study the specific problems that students may encounter while learning programming, and the frequency of them. To achieve this we performed an empirical study by inspecting the source code written by the students for the assignments given in the INF100 course. We were interested in all kinds of problems and solutions, both syntactic and semantic.
In this chapter we first describe how we collected the data, and the size of the data set. Then we describe and classify some of the most frequent occurring problems using the taxonomy described in Section 2.3.
3.1
Study Description
The data we collected had to be represented in such a way that we would be able to analyse it in many different ways, without the need to inspect the programs multiple times. We found that by annotating any interesting expressions, statements or segments with an identifier, we could scan the files for these annotations afterwards and insert the information into a database. A database provided us with a powerful query language that we could use to retrieve the data and build interesting queries with. Each entry in the database contains a student identifier (anonymous), assignment, file, exact line where the problem occurs and the problem identifier. Storing the data in this way also enabled us to verify the results later, extract code snippets and make the data available for later research.
Before we began inspecting the student submissions we created an initial list of typical student problems, based on our previous educational work,
Ex1 Ex2 Ex3 Sem1 Ex4 Ex5 Sem2 Ex6 Ex7 Sem3 0 25 50 75 100 125 150 175 Inspected Others
Figure 3.1:Inspected student solutions.
and gave them each a unique identifier. However, during the process of annotating the source code we discovered other kinds of problems, that we had not initially foreseen, and also had to change some of the initial ones as they were either too general or too specific. See Appendix A for a complete list of the annotations used.
3.2
Data Set
We have a large number of submissions available from just one semester of the course INF100. In total 182 students enrolled for the course and there were 3 semester assignments and 7 exercises during the course. This sums up to a total of 1820 submitted solutions, if all the students had submitted a solution for all the assignments. However since a portion of the students dropped out during the course, and they were only required to pass 5 out of the 7 exercises, there was slightly fewer submissions, approximately 1400. This was still a large number of submissions, and we did not have the time and resources to inspect all of these. We had to limit the set of submissions to inspect (see Fig. 3.1) and we began by selecting a subset of the students, based on their performance on the semester assignments, of
Ex3 Sem1 Ex5 Sem2 Ex7 Sem3
0
20
40
60
80
100
Assignments
Students
Semester
Exercises
Figure 3.2:Decrease of submitted solutions throughout the semester.
which we would inspect the submitted assignments. Any student scoring strictly less than 25 out of 30 (∼ 85%) points on any of the three semester assignments would be added to the set. The reason why we chose this limit was that most of the submissions scoring higher than 25 points had very few mistakes, and they were mostly superficial.
This gave us a set of 79 students, which still was a too large amount of data for us to inspect. We then proceeded to limit the data set by selecting six assignments to inspect. The first two exercises were trivial introductions to programming, and had little value to us as they only involve simple arithmetics in a Hello World way. The three semester assignments were sure choices, and to close the time gap between them we chose the three exercises that were given before each semester assignment.
In Fig. 3.2 we present the number of students, from the subset we selected, who submitted solutions for the selected assignments. The decrease of students who submitted solutions for the semester assignments are all students who dropped out of the course, i.e. 20 students dropped out between semester assignment 1 and 3. For the exercises however, the decrease is mostly because the students are only required to submit solutions to five of them. Exercise 7 was especially affected by this.
Concept-based Knowledge-based
Loop instantiation problem Unnecessarily complicated
Wrong condition Boundary case condition
Missing method call Incorrect grouping Erroneous arithmetic Accumulate boolean C:I C:II C:III C:IV C:V
- Incorrect notional machine - Natural language translation
- Misconception of construct semantics - Specialisation problem - Test data K:I K:II K:III - Interpretation of specification - Schema knowledge - Existing environment C:II,III,V C:II,V C:I,III,IV C:II K:I,III K:I-III K:II,III C:V K:I,III K:I K:I K:I,III C:IV K:I,III Primary Secondary
Figure 3.3:Problem taxonomy
3.3
Problem Taxonomy – Most common problems
In this section we give a thorough description of the most frequent occurring problems and then we classify (see Fig. 3.3) them using the taxonomy described in Section 2.3. Together these problems account for more than half of all problem occurrences (see Section 4.7) and by classifying these we may be able to suggest actions that can be taken with respect to the teaching. For each problem we provide a general description, concrete examples and, if necessary, actual code samples. To classify the problems we have identified a list of plausible underlying causes for each problem and an argument is given for why those causes are plausible. Appendix A is a valuable resource throughout this section.
3.3.1
w r n g
c o n d
– Wrong condition
Any conditional expression that in some way is incorrect may be counted as a wrong condition problem. However the annotation is mutually exclusive
with those annotations that consider special cases of incorrect conditional expressions, e.g. 207c o n db d ry. Except for 207c o n db d ry, we did not identify any other
special cases of conditional expression problems that were common enough to deserve their own annotation. Though we do recommend adding anno-tations for any kind of conditional expression, e.g. tautology conditions, that seems to be common.
Occurrences of this problem are primarily incorrect conditional expres-sions in if-sentences, though there are a few occurrences in loop-constructs. Typical occurrences of this problem are conditions that are wrong because they use an incorrect operator, connective or method call. Concrete exam-ples of this problem are solutions that check for reference equality instead of object equality, or overcomplicated conditions that render the solution incorrect, etc.
We suggest these plausible underlying reasons:
1. Interpretation of specification. The students may find it difficult to extract the correct conditions from the specification. It requires them to understand the given specification in its entirety, and they may miss the implicit conditions.
2. Natural language translation. Not all logical expressions expressed in natural language can be directly mapped to the programming language. Conditions may change meaning and parts may be lost in translation.
3. Existing environment. The student may find it difficult to combine the relevant existing state of the program into a correct conditional expres-sion. This often leads to overcomplicated or conflicting expressions. The underlying reasons we identified for this problem are determinants from both the concept-based problem and the knowledge-based problem categories. Though the majority of the determinants are for the knowledge-based problems, we argue that it is uncertain what category is primary and secondary and choose both categories.
3.3.2
b d ry
c o n d
– Boundary case condition
Any conditional expression that evaluates incorrectly for some boundary case, either by going out of bounds or being too restrictive, e.g. incorrectly included or excluded boundary elements in a range. A concrete example of this is a condition that uses the inclusive less-than operator instead of the strictly less-than. This might result in an attempt at accessing an index
outside of an array, or a guard that accepts too little or too much of a number range. As a consequence the program might suffer of index-out-of-bounds errors for arrays. Listing 3.1 is a typical example of this kind of problem.
Listing 3.1: Boundary case sample 7
1 int[] numbers = new int[10];
2 for(int i = 0; i <= numbers . length ; i ++){
3 numbers [i] = i;
4 }
We suggest these plausible underlying reasons:
1. Specialisation problem. Identifying the correct boundary points may be challenging for the student when instantiating a schema.
2. Incorrect notional machine. The student may possess an incorrect con-ception of how the conditional expression relates to the environment, e.g. a model where the initial array-index is one-based instead of the actual zero-based.
3. Misconception of construct semantics. The student may have an incorrect understanding of how the language construct works, e.g. at what point the condition of a loop is validated or when the update statement of a for-loop is executed (or even what kinds of expressions that should be used as update statements). Du Boulay [10] notes that the automatic increment of the counter variable may be problematic.
4. Interpretation of specification. Students may have interpreted the pro-gram specification incorrectly, e.g. not perceived correctly if it should be an inclusive or exclusive condition.
These reasons are mostly determinants in the concept-based problem category, though (4) belongs to the knowledge-based problem category, and we argue that this problem should be classified primarily as a concept-based problem, and secondarily as a knowledge-based problem.
3.3.3
l o o p
c r e at e
– Loop instantiation problem
Any situation where the student has had trouble instantiating a loop-construct to solve a given task, is counted as a loop instantiation problem. Occurrences of this problem are situations where:
• The student has either not been able to instantiate a loop, or did not realise that one was needed, often leaving that part unsolved or given a hard coded solution.
• The student has only partly instantiated a loop, often the most trivial loop-construct schema.
• The student has instantiated a loop that is far away from a working solution, often with conditions and bodies that does not resemble a solution in any way.
Though situations where the student has solved the problem in a static manual way are not counted, these are counted as 206l o o pm a n. Listing 3.2 is an
example of a case where the student has either not been able to instanti-ate a loop or did not properly understand the semantics of the existing environment and failed to see why line 7 would not compile.
Listing 3.2: Missing loop 7
1 // Field variable representing shares in percentage
2 double[] shares = {30.0 ,25.0 ,10.0};
3 /**
4 * @return the remaining share in percentage .
5 */
6 public double getShareOfOther (){
7 double otherShare = 100 - shares ;
8 return otherShare ;
9 }
We suggest these plausible underlying reasons:
1. Schema knowledge. The student may not possess the necessary schemas to instantiate a loop construct to solve a particular problem, or may not understand how the schemas work and is not able to use them for any other cases than the default.
2. Existing environment. To combine the existing knowledge of the pro-gram and use the state of the environment when instantiating a loop construct may be very difficult. To realise which methods/functions and/or variables that should be utilised, requires a good overview of the program and a good conception of the semantics of them.
3. Interpretation of specification. In some cases the requirement that speci-fies the need for a loop-construct may be less obvious and the student may miss it.
All the underlying reasons we identified for this problem are determi-nants from the knowledge-based category, and we argue that it should be classified as a knowledge-based problem.
3.3.4
c m p l x
c t r l s
– Unnecessarily complicated
Instances of this problem are those situations where the student devise an overcomplicated solution that may or may not work. These code segments may be very difficult to understand, even for the student at the time of writing, and if it should prove to not work as intended it is very difficult to identify exactly what is incorrect. Many instances of this problem could in some situations, with a more thorough inspection and communication with the student, have been annotated with other more specific annotations. Typically these are complicated structures that may be any combination of the following (or other less frequent examples):
• Deeply nested blocks.
• Repetition of large segments of code.
• Conditional structures with individual branches for each possible case the student can imagine, even though they may not be relevant or should have the same result/consequence.
Listing 3.3 is an example of an unnecessarily repeated code segment that could easily be removed with some arithmetic or by performing the conditional difference calculation first. This implementation does not follow the specification completely either, it misses the case wheretime1is equal totime2.
Listing 3.3: Repeated code 7
1 if( time1 > time2 ){
2 int difference = time1 - time2 ;
3 /*
4 ...
5 Code segment that prints out the difference
6 converted to hours , minutes and seconds .
7 ...
8 */
9 }
10 else if( time1 < time2 ){
11 int difference = time2 - time1 ;
12 /*
13 ...
14 Exact same code segment .
15 ...
16 */
17 }
1. Schema knowledge. The student may not possess or understand the schemas necessary to instantiate sensible conditional structures, pre-vent repetition etc.
2. Existing environment. The student needs a good understanding of the environment, and its state, to be able to prevent the creation of unnecessary complicated structures. To prevent repetition the student needs to identify what separates the different cases, and how to exploit this knowledge to combine the segments.
Both the determinants we have identified and listed above are from the knowledge-based category, and we argue that it should be classified as a knowledge-based problem. However, in some cases there may be another underlying reason for this problem, namely if the student fears to change something that to some degree may fulfill the requirements.
3.3.5
n o
c a l l
– Missing method call(s)
Any situation where the student has omitted a necessary, or useful, method call is counted as a missing method call problem. In some situations the student has still managed to create a working solution by re-inventing the wheel, i.e. implementing the functionality of the method specifically for that situation, and in others the student may have left out that part of the solution entirely. When solving assignments given in a course, the student will in some cases be provided with methods (or classes) and/or information about any library methods that could be useful.
We have identified the following plausible underlying reasons:
1. Existing environment. The student needs a good overview of the existing program structure and the available library methods to be able to avoid this problem. Using this knowledge the student must be able to realise the possibility of calling a method in a given situation, and be able to identify which method.
2. Interpretation of specification. The student may not have perceived the specification correctly and in turn does not realise that there should be a certain method call in that situation.
3. Test data. The student may not be able to realise that there is a problem if the data set the student is executing the program with does not expose that the method(s) was not called.
Two of the determinants identified for this problem, (1) and (2), are for the knowledge-based category and the third is for the concept-based category. We argue that it is more likely that the student does not realise there is a missing method call due to insufficient test data than incorrect conception of the specification, and classify this as both a concept-based problem and a knowledge-based problem.
3.3.6
w r n g
g r p n g
– Incorrect grouping
Any situation where the student has incorrectly connected, or failed to con-nect, constructs and/or statements is counted as incorrect grouping problem. Typical examples of this are:
• Incorrectly connected, or failed to connect, multiple if-constructs (using the else-connective).
• Omitted to place those statements that should be the “else-case” in an else-block, i.e. placed in such a way that they do not depend on the if-construct.
• Placed the result action of a search schema in the body of the loop, resulting in that the action that should be performed on the result of the search, is performed each iteration instead.
• Placed a statement that also should depend on the condition of an if-structure, outside of that block.
Listing 3.4: else connective 7
1 if( player1Winner ){
2 System . out . println (‘‘ something funny’’);
3 }
4 if( player2Winner ){
5 System . out . println (‘‘ an angry message’’);
6 }
7 else {
8 System . out . println (‘‘ something sad’’);
9 }
We have identified the following plausible underlying reasons:
1. Misconception of construct semantics. The student may not properly understand how the if-construct connects using the else-statement, and consequently fails to connect them. This may be an understanding where sequential if-statements automatically connect (as if there was an else-statement connecting them) and that any else-statement at
the end of the sequence is the branch that is entered if none of the previous if-statement branches are entered.
2. Natural language translation. A plan of conditional actions may be expressed correctly in the natural language of the student, but in such a way that it does not directly map to the programming language. Listing 3.4 implements these requirements “If player1 wins print something funny, if player2 wins print something sad, otherwise print an angry message”. The student has implemented the requirements by directly mapping the natural language solution into the programming language and the angry message will only be printed if player2 does not win.
3. Interpretation of specification. It may be challenging for the student to perceive the specification as it is intended, especially regarding implicit requirements. In many cases the requirement that the calculated result of a program should not be printed if input validation fails, is often implicit and many students does not realise that.
4. Test data. The test data the student is using may not expose that there is a problem with the current structure, e.g. the data set might be so small that there is only one iteration of the search loop and the result action is only evaluated once.
The majority of the underlying reasons identified for this problem are determinants from the concept-based problem category, and we argue that this problem should be classified primarily as a concept-based problem and secondarily as a knowledge-based problem.
3.3.7
ba d
a r i t
– Erroneous arithmetic
Any arithmetic expression that is incorrect with respect to the given as-signment is counted as erroneous arithmetic. Common occurrences of this problem is when the student needs to calculate the difference between two numbers, and neglects that the result might be negative. The student may realise the problem but fail to find a correct solution, often statically multiplying by −1 or finding the absolute values of the individual terms instead of the entire expression. Other occurrences of this problem may be that the student fails to understand exactly what the numbers represent, what denominator they have and/or what they count.
1. Test data. The test data the student is using may not expose that there is a problem with the arithmetics, e.g. no case where calculating the difference will give a negative result.
2. Existing environment. The student may not possess the necessary understanding of the existing environment and fails to see the correct semantics of a number, e.g. incorrect denominator.
3. Interpretation of specification. The student may not have understood the specification correctly, e.g. choosing the wrong representation of a value.
4. Natural language translation. The experience the student has with arithmetics is likely purely statical and the dynamic behaviour of arithmetical expressions in a programming language may be chal-lenging and may make it difficult for the student to translate the expressions.
The underlying reasons identified for this problem are determinants from both the concept-based and knowledge-based problem categories, and we argue that it is uncertain if one of them is more dominant than the other, and choose to classify this problem as both a concept-based problem and a knowledge-based problem.
3.3.8
b o o l
ac c u m
– Accumulate boolean
Instances of this problem are situations where the student has neglected, or failed to, accumulate a result boolean when performing an operation on a list of elements. The operation that is performed on each element returns a boolean value that represents failure or success. This boolean value should be accumulated by the loop-construct iterating over the list of elements in a way that detects any failure, see Listing 3.5.
Listing 3.5: Boolean accumulate 3
1 boolean success = true;
2 for( Recipe r: getRecipes ()){
3 if(! r. printToFile ())
4 success = false;
5 // or the reduced way
6 success = success && r. printToFile ();
7 }
We have identified the following plausible underlying reasons:
1. Existing environment. The student may not realise that the operation performed on the elements returns a boolean value representing its success, and fails to realise that there should be an accumulation. 2. Interpretation of specification. The student may not have perceived that
the resulting boolean expression should consider the operations. 3. Specialisation problem. The student may not have been able to express
how to accumulate the boolean correctly.
The majority of the identified underlying reasons are from the knowledge-based category, (1) and (2), though we argue that (3) is more likely because the student is syntactically forced to have a boolean return expression in the method implementation. This should provide the student with a reason to investigate how that value should be calculated, which should limit the number of students having (1) and (2) as underlying reasons. We argue that it is primarily a concept-based problem and secondarily a knowledge-based problem.
Chapter 4
Result Analysis
In our study we inspected and annotated the source code of 349 solutions submitted by the 79 selected students, for the six selected assignments. A total of 2395 annotations across 920 files. Of these 1185 are meta annotations that describes if the files:
• do not compile ( 6c o m pe r r).
• contain methods that are not implemented ( 5u n s o lvp ro b ).
• are not implemented at all ( 7i m p ln ot).
• sufficiently solve the problem, or displays sufficient understanding of how ( 1pa s s).
• do not sufficiently solve the problem ( 2f a i l).
• are changed in a way that breaks the premise of the assignment ( 4c h n ge x e r).
We did not have strict requirements for the pass annotation ( 1pa s s), so any
file where the student was able to display how the problem should be solved to a satisfying degree, was annotated. Even some files that did not compile (42) was annotated with pass. The pass ( 1pa s s) and fail ( 2f a i l) annotations
are mutually exclusive. There were only 94 files annotated with fail, and the rest, 808, was annotated with pass.
There were 1210 problem annotations divided among 52 different an-notation kinds. Some of these anan-notations had very few occurrences and are not necessarily referred to, and if they are, it is in relation to some other problem that they are relevant for.
4.1
Lazy students?
While we studied the gathered data we noticed that as many as 49 of the 79 students had at least one file annotated with 5u n s o lvp ro b , missing implementation
of a method. In total 83 files was annotated with 5u n s o lvp ro b and in 17 files there
were multiple occurrences (i.e. multiple missing methods). In addition 14 students had files that were not implemented at all, 7i m p ln ot, in total 20 files.
We can only share our suspicions of why there were such a significant part of the submitted solutions that was not implemented. In our opinion there are two reasons that seems to be the most plausible.
1. The student may have speculated in how much of the assignment that were necessary to implement to achieve a passing grade.
2. The student did not possess the knowledge to be able to provide a solution for the missing parts.
If lazy students (1) was the reason why some of the submitted solutions were missing parts, we may have reasons to suspect that this may have been an underlying reason for other problems identified in the student submitted solutions as well. This especially applies to those situations where the solution was missing some part, e.g. 402c r e at el o o p (loop instantiation
problem). Though it may be possible that the student would have chosen to implement the skipped parts if the student had enough experience and knowledge such that the implementation would require less work and time.
We give this argument to caution the reader that the following analysis may be affected by this, and to make the reader aware that we did have this in mind when we performed the analysis.
4.2
Most common problems
In Fig. 4.1 we present a chart of the most common problems among the students, i.e. those problems that the most students were registered having. Conditions, overcomplicated solutions, loop-constructs and method calls are those problems that, by the data we collected, were the most common problems. In the chart there are also some annotations regarding unnecessary expressions, statements and placement:
• 101x t r a
va r, unnecessary variable.
• 104p r e
d e c l, unnecessarily pre-declared variable.
• 304b o o l
e q l, checking boolean expression for equality with boolean literal.
These three are only indicators that shows that the student may have some misconception of the semantics or the notional machine, and we have only gathered them under the idiom problem annotation for convenience. The
0 10 20 30 40 50 60 BdryCond WrngCond PreDecl XtraVar LoopCreate CmplxCtrls BoolEql NoCall 58 55 44 43 31 40 39 37 28 17 17 14 11 2 8 7 #students Repeated Once
Figure 4.1:The most common problems among the students.
students are quickly rid of the 304b o o le q l and 104d e c lp r e misconceptions,
most likely due to good feedback from the teaching assistants, as they are primarily only a “problem” in semester assignment 1 (and exercise 3 as well for 104d e c lp r e). The occurrences of 101x t r ava r are more thinly spread across the
assignments, though most frequent in exercise 5 and semester assignment 2. We argue that the most plausible reason for this “problem” is that it is some remnant of some plan or previous attempt to implement a solution, and that the student most likely has forgotten the existence of the variable. Conditional expressions is a concept that our data shows was a big challenge for many students. The most common kind of conditional expres-sion problem was incorrect boundary case conditions, 207c o n db d ry , which
as many as 58 students were registered having problems with. In addition 55 students were registered having trouble with conditions in general,
303
w r n g
c o n d , that is those that does not fall into a special category (like
boundary case). Most of the students (48) who had troubles with boundary case conditions also had troubles with conditions in general. Respectively 28 and 17 students were registered with 207c o n db d ry and 303w r n gc o n d across multiple
assignments.
We registered 40 students that created overcomplicated solutions, 202c m p l xc t r l s
, though only two students created such solutions in multiple assign-ments. In many cases it is plausible that the reason that these kinds of solutions are submitted is that the student fears to change something that
0 20 40 60 80 100 Ex3 Sem1 Ex5 Sem2 Ex7 Sem3 Percentage of students
Figure 4.2: Students who had parameter passing problems (w r n g a s s g n,va r sg l b l,
a r g
s c o p e,pa r a mf r m l).
seems to work, even though it might be clear that the code segment is very complicated and/or repetitive. We also suggest that it is plausible that this is a problem that students have with newly learned concepts, and that with experience they will learn to apply the concepts properly.
4.3
Parameter passing and references
Our experience in teaching suggested that the concepts of parameter passing and references would be a significant and common problem in the submitted student solutions. However this was not the case, and in fact very few students were registered as having difficulties with parameter passing, and none were registered having troubles with references. See Fig. 4.2 for the frequencies of parameter passing problems.
The teaching assistants reported that parameter passing was a hot topic in their lab sessions. If we consider this in relation to the low frequency of this problem in the submitted solutions, we may come the conclusion that the teaching assistants were successfully able to teach the students this concept. Though there is also the possibility of assistance from other students.
There is one interesting case though ( 121pa r a mpa s s ), 22 students seem
to have a small misconception regarding how parameter passing works. In some cases when they write method call statements, they declare an
unnecessary variable on the line above the method call with the same name as the formal parameter of the method. In the last two assignments the complexity of the domain model increased significantly, which we suggest is the reason why this problem only occurs in those assignments ( ) and why the frequency is high.
We suggest that a reason for the lack of reference problems and the low frequency of parameter passing problems might be related to the design of the given assignments. Situations where references could have been a problem in the assignments were all related to immutable objects like String, Integer etc.
4.4
Conditions
Problems related to conditional expressions, both in if-sentences and loop-constructs, was very common among the students, see Fig. 4.3. Semester assignment 3 was the most difficult assignment given during the course, and not only did it stress most of the previously learned topics, it also required the students to implement a total order relation-method (com-pareTo from the Comparable interface). Almost all, 93.2%, of the selected students who submitted a solution for this assignment had some trouble with conditional expressions. If we also take into account the problem of accumulating a boolean value ( 306ac c u mb o o l), there was only one student who
did not have problems. Exercise 3 and semester assignment 1 both had an introductory focus on if-sentences and loop-constructs, though the students were required to interpret, and understand, specifications for these struc-tures, and instantiate them in the implementation. In these assignments few students had difficulties and most were able to express correct conditional expressions. We argue that this may show that the students are able to identify situations where these structures are required and at least able to interpret the concrete specifications of the conditions. But when the specifications and conditions grow more complex in the later assignments the students experience difficulties.
A large part of the solutions submitted for exercise 5 and semester assignment 2 also had mistakes related to conditions, respectively 83.3% and 55.8%. The listed goals of these assignments are respectively Objects and methods and Classes and objects, but there are few identified problems related to these – which may suggest that the actual focus of the assignment was incorrect.
0 20 40 60 80 100 Ex3 Sem1 Ex5 Sem2 Ex7 Sem3 Percentage of students
Figure 4.3:Students who had problems with conditional expressions (b d ry c o n d,
x t r a
c o n d,u g lyi f ,w r n gc o n d,a l m s tc r r c t).
4.5
Loop constructs
Implementing programs that requires loop constructs is a challenge for most novices, and bugs are more common in relation with these constructs than with other common tasks like input, output, syntax/block structure and overall planning [26]. This is reflected in our results as well, during the course 82.2% of the selected students had troubles related to loop constructs, either with the condition or the instantiation of the construct. The assignments where the most students struggled with loop constructs was exercise 5 and semester assignment 2 and 3, .
Semester assignment 2 In this assignment 74% of the selected students that submitted a solution struggled with loop constructs. The focus of this assignment was Classes and objects but there was a frequent need for loop-constructs, and the nesting between the provided Java classes was quite deep for a novice. This may have caused the students to have difficulties when creating the mental model of how the classes and objects communicate. The given classes wereClient,Recipe,RecipePartandIngredient, see Fig. 4.4. ARecipeobject has a list ofRecipePartobjects, which in turn has a specific
Ingredientobject associated with it and theIngredientobject has an array of nutritions. In addition the main-method in the Clientclass has a list ofRecipeobjects. This provides many locations where loop-constructs are needed to traverse the data structure, and even some where the need is not
Recipe RecipePart +amount: double Ingredient +nutritions: double[] Client parts 0..* recipes 0..*
Figure 4.4:Domain model (UML class diag.) of semester assignment 2. specified directly, which may explain why the problem is more common in this assignment than others.
4.6
Method calls
During the course 60.1% of the selected students had misconceptions or difficulties with method calls. These problems were most common in semester assignment 3, where 59.3% of the selected students who submitted a solution had troubles. Most of the problems that the students had were situations where a necessary method call was missing ( 28c a l ln o ), which
46.8% of the selected students struggled with. In many of these cases the students had re-implemented the responsibility of the method by writing a copy of the method body (presumably unknowing) at the location where the method call should have been, and thus met the requirements of the assignment. In other cases the program did not perform the task of the method, and did not meet the requirements.
There was also other problems related to method calls, though quite rare. A few students struggled with correctly specifying method calls, where both syntax and choosing which expressions to pass as arguments ( 26s tat i cc a l l
and 29c a l lb a d) were problematic. Other cases were more subtle and did not
affect the solution in any way, e.g. repeated calls to observer methods ( 22x t r ac a l l,
23
u n u s d
va l - s and 24u n u s dva l - o).
4.7
Concept-based problems
If we disregard the annotations that signify unnecessary expressions, state-ments and placement, then the most common problems, both in occurrences and the number of students that had them, were those that we described in
