Identifying and addressing common programming misconceptions with variables (part 1)

(1)

Identifying and Addressing

Common Programming

Misconceptions with

Variables

(Part 1)

Danny Plass (s0021407)

(2)

Danny Plass-Oude Bos, PhD (s0021407)

in collaboration with Rifca Peters, MSc Onderzoek van Onderwijs (10 EC) Track: Computer Science

Master Science Education and Communication ELAN, Faculty of Behavioural Sciences

University of Twente

Commi ee

Nico van Diepen, University of Twente dr. Nellie Verhoef, University of Twente

Published by Universiteit Twente

(3)

Introduction

13 Background

17 Related Work

21 Methods

23 Results

31 Discussion

41 Conclusions

47 Bibliography

51 A. JavaScript Tests

55

(4)

(5)

List of Figures

1 Bimodal grade distribu onfor Java programming (havo 4 at one high school, 2010-2014) 13

2 Screen capture of the interac ve video, showing the code on the right with the current line increased in size, and the computer memory with its three variables on the left. 28

3 Screen capture of a ques on embedded in the interac ve video.

The question is: “What does the equals sign signify in: length = 4;”. The participant can select the answer from four options: (1)length

equals 4; (2) The computer will solve a mathematical equation; (3)

lengthwill get a new value, being 4; (4) 4 is added to the value inlength.

On submit, the video will indicate whether the selected answer was correct. 28

4 Distribu on of the amount of inconsistencies in the answers within par cipants. The questions were grouped in 8 sets of highly sim-ilar questions. Answers within sets were compared for inconsist-encies in misconceptions. 38

(6)

(7)

List of Tables

1 Variable assignment for primi ve types: 6 learning goals and their related misconcep ons. Misconceptions for one learning goal may affect other learning goals. For example, if the student thinks vari-ables can contain multiple values (goal 5 - container analogy), this will also affect goal 6: the student will think the new value will be added next to the value the variable already contained. 19

2 Pre-test ques ons mapped onto misconcep ons (design) Ques-tions were designed to trigger and identify the misconcepQues-tions lis-ted in Table 1. This table shows the code used in the questions, the correct answer for the values of the variables in the code after ex-ecution, and expected answers in relation to specific misconceptions. Answers in grey cells uniquely identify specific misconceptions. An-swers in light grey cells can also identify specific misconceptions, as soon as the alternative misconception option has been disproved for this participant. If answers for certain misconceptions are indis-tinguishable from the correct answer, they are not listed in this table. Aside from the Correct answer column, the only answers listed are those that could be signs of misconceptions. 26

3 Updated overview of the 6 learning goals and their related mis-concep ons for variable assignment for primi ve types. Added are the following misconceptions: M2b, M2c, M2d, S1b, O1, and O2.

32

(8)

(9)

Abstract

Programming is considered an important skill for the 21st century, but it is not an easy skill to teach or to learn. One of the most basic concepts in programming is that of variables. Students’ understand-ing can be obstructed by a wide range of incorrect ideas of how variables work. These misconceptions mostly originate from previ-ous experiences with math (demonstrated by 83% of our participant population), human interaction (34%), the container analogy (5%), and misunderstanding semantics (39%).

(10)

(11)

Samenvatting (Dutch)

Programmeren wordt gezien als een belangrijke vaardigheid voor de 21ste eeuw, maar het is geen eenvoudige vaardigheid om te doceren noch om te leren. Een van de meest basale concepten bij programmeren is variabelen. Leerlingen kunnen last hebben van een grote variatie aan misconcepten, verkeerde ideeën, over hoe variabelen werken. Deze misconcepties zijn meestal afgeleid van eerdere ervaringen met wiskunde (geobserveerd bij 83% van onze deelnemers), menselijke interactie (34%), de doos-analogie (5%) en semantiek (39%).

(12)

(13)

Introduction

Computer science is a relatively new course in secondary education in the Netherlands (Zwaneveld et al.,2009). An important aspect of this course is programming (Grgurina and Tolboom,2007), which demands a different way of thinking from students. As programming and the related proﬁciencycomputational thinkingare considered important skills for the 21st century (De Bakker and Land,2015; Koninklijke Nederlandse Akademie van Wetenschappen,2013; Mannila et al.,2014), it is important to know what issues are likely to block progress in learning these skills.

In the grades students get for programming courses there ap-pears to be a division between students who get it and students who don’t, as they often follow a bimodal distribution, where there are few ‘average students’ (see Figure1for an example grade distribu-tion of a programming test at a Dutch high school). This situadistribu-tion promotes the idea that perhaps some people simply cannot program,

causing students to give up on this course, sometimes advised to do “The camel hump” refers to the con-troversial, retracted paper by Dehnadi and Bornat: “The camel has two humps”

(2006,http://www.cs.mdx.ac.uk/

research/PhDArea/saeed/), which is a sharp expression of the frustration programming teachers can feel when they observe this division in their class. In this paper, they made the case that some students simply are not able to learn to program, and that these stu-dents can be identiﬁed with a simple test.

so by their own teachers. A more likely reason for this “camel hump” is that concepts build on each other throughout a course, leading to the Learning Edge Momentum (LEM) effect (Robins,2010): If certain concepts are not yet fully grasped, learning more complex concepts based on these more basic principles will be harder — and vice versa, if the basics are understood, future learning will be easier — leading

Figure 1:Bimodal grade distribu on

(14)

14 Identifying and Addressing Common Programming Misconceptions with Variables(Part 1)

to larger and larger differences within the student population. It is therefore important that students have a good understanding of the basics.

There are various incorrect ideas, or: misconceptions, students can hold that block their progress in learning how to program. For example concerning in what order program code is executed or what information is currently in computer memory (Kaczmarczyk et al., 2010;Taylor et al.,2014;Tew,2010). According to the LEM theory, being able to grasp the basics will have a cumulative effect (Robins, 2010). Therefore, for this study, we focus on one of the most basic concepts in imperative programming:assignment with variables of primitive types, with integer numbers speciﬁcally. Even such a simple topic is surrounded by a variety of potential misconceptions, as we will show in the next chapter. Many of the other programming concepts such asreference types, but alsofor loops, build on this concept, making it an important stepping stone.

Besides, one study showed that about one third of university students still did not fully grasp basic variables and assignment after an introductory programming course consisting of 20 lectures and 50 hours of tutorials and practical work (Ma et al.,2007).

How can teachers help students who hold programming miscon-ceptions around this basic concept of variables? AsHerman et al. (2008) describe: “[I]dentifying misconceptions alone is not enough to direct instructional reform. In addition, instructors must know why students [...] make the mistakes they do.”

Most research into programming misconceptions focuses on a relatively broad range of concepts, which does not allow for an in-depth investigation of the misconceptions related to one speciﬁc core concept (Such asBayman and Mayer,1983;Kaczmarczyk et al., 2010;Tew and Guzdial,2011). Other research does focus on assign-ment, but only describes the resulting behaviour instead of trying to determine the underlying ideas that are causing the incorrect predictions of code samples (Dehnadi,2006;Ma et al.,2011).

As a starting point, we provide an overview of common miscon-ceptions in relation to assignment with variables of primitive types, based on literature. These misconceptions have been categorized along four domains that seem to underlie these incorrect ideas. Following from this overview, we have developed a test to identify

The test comes in two variations, one used as a pre-test and one as a

post-test in this research. speciﬁc misconceptions with variables, as well as an interactive video

to transform the selected incorrect beliefs into correct ones. This video has been evaluated with students from both secondary edu-cation and applied university. There were signiﬁcantly more correct answers on the misconception post-test.

(15)

Introduction 15

these common misconceptions, we hope to empower teachers to help students — and students to help themselves — cross the divide between “those who get it” and “those who don’t”.

(16)

(17)

Background

Analysis of studies on misconceptions of programming and variables in particular reveals three main sources of misconceptions around assignment with variables of primitive values: math, human inter-action (or: anthropomorphism), and the container analogy (Bayman and Mayer,1983;Horstmann,2013;Pea et al.,1987;Ben-Ari,1998). There is also one catch-all category: misunderstanding of semantics (Ma et al.,2011).

Four misconception categories

Math Most students ﬁrst encounter variables in algebra: a = b + 4, where the letters stand for values. These letters are even called variables, but they behave a little differently from the variables in programming (Horstmann,2013;Ma et al.,2011). Algebraic ex-pressions describe relationships between variables, equations that need to be solved. From this domain, students bring miscon-ceptions such as that the equals-sign makes both sides equal (so a value can for example also be moved from the variable on the left of the equals-sign to the variable on the right), or that the

statement describes an equation to be solved by the computer In this report, we use JavaScript code to represent our code segments, as

this code is readable yet concise.var

indicates the declaration of a variable which can either be with or without an initial value. If a variable has no

value it isundefined. As you can see,

no speciﬁc type has been given. The type is determined dynamically based on the value the variable contains at that moment. In this report we only use integers.

(var a = 6; var b; a = b + 4;This is then expected to solve the equation for b:b = 2.) Also the idea that variables can only obtain a value once comes from this domain: within one math problem, anx cannot indicate the value 10 at the start, and then suddenly change into 20 near the end. Misconceptions stemming from algebra might be triggered speciﬁcally when using variable names that are also commonly used in math, such asxandyoraandb.

Human interaction From interaction with fellow human beings, we

(18)

under-18 Identifying and Addressing Common Programming Misconceptions with Variables(Part 1)

stand the intention behind their code, just like a human might infer (Pea et al.,1987). Misconceptions caused by this anthropomorph-ism are, for example, the idea that the name of the variable matters: that a variable with the namesmallestcannot possibly contain the largest number in the program, or thatsmallestautomatically con-tains the smallest number present in the program.

Container analogy Another potential source of misconceptions is

that of the container analogy: a variable is like a box (Ben-Ari,1998). The box has a label with the name of the variable on it so you can easily ﬁnd the box. And the box can contain something, the value of the variable. This analogy is often used to teach the concept of variables, as it provides a clear visual for students that is easy to understand. Unfortunately, this analogy does not work all the way. Say you have a box A and a box B. If you move the contents from box A to B, then box A is now empty. In the case of programming, however, box B will get a copy of the contents of box A. Another result from this analogy is that students may think a variable can contain multiple values, just like a box could contain multiple pieces of paper. Later on, when the student learns about reference types, the box analogy may become problematic again, as multiple boxes can contain (references to) the same object.

Semantics There are other consistent mistakes students make in

interpreting the semantics of assignment statements. This semantics category might be indicative of yet other domains causing miscon-ceptions in elementary programming. The following misconmiscon-ceptions

This “test for equality” is categorized under Semantics, with the idea that the meaning of the equals sign is misunderstood, but it could very well be seen as related to the Math category of misconceptions, as the idea of using the equals sign for equality obviously stems from this domain. The

idea that it would constitute atestfor

equality, however, would indicate a different way of thinking from simply math equations. This is why we decided on the current categorization.

are mostly fromMa et al.(2007), which is in turn based on the test by Dehnadi (Dehnadi,2006): (1) The variables are tested for equal-ity, which is true or false; (2) The receiving variable is on the right-hand side (Could be from the math analogy where side does not matter); (3) The values of two variables are swapped; (4) The new value isaddedto the previous value; (5) Results can only be stored in variables not mentioned in the computed expression, because it would create a conﬂict (Ex.a = a + 1). These misconceptions are not explicitly linked to previous experience in other domains, as was the case for the other three categories.

Six learning goals about variables and assignment

Combining these misconceptions with the right conceptions, we arrive at the six learning goals for our intervention, see Table1. Hestenes et al.(1992) caution: “[W]e would like to warn against a Hestenes et al.(1992) have developed

the FCI, a widely-used multiple choice test on the concept of Newtonian

force. Their test, also called aconcept

inventory, has functioned as a catalyst for numerous innovations in physics

(19)

(20)

(21)

Related Work

This report, describing part one of this research, focuses on the mis-conceptions students may hold concerning variables and assignment and identifying these misconceptions with a misconception test. Therefore, this Related Work section hones in on these two aspects.

Uncovering programming misconceptions

There has been a body of previous work on basic programming mis-conceptions in students. Common approaches to uncover miscon-ceptions have been to let students explain what they think happens in particular code segments (Bayman and Mayer,1983;Kurland and Pea,1985;Pea et al.,1987), by giving them small problems with code segments to solve while letting them think out loud and interviewing them on speciﬁc concepts (Kaczmarczyk et al.,2010), or by giving them open ended questions (Tew,2010).

While these approaches have led to various examples of miscon-ceptions and theories on misconception domains, there is of yet no comprehensive overview of programming concepts with the in-correct ideas underlying the mistakes students make with these concepts. There have been lists of misconceptions (Bayman and Mayer,1983;Dehnadi,2006;Kaczmarczyk et al.,2010), but in these cases misconceptions are the behavioural results, not the underlying misunderstandings that led to these results.

Table1is our attempt to provide an overview of misconceptions for variables and assignment based on literature, accompanied by the underlying domain of previous experience that might be causing these mistakes.

Assessing programming misconceptions

(22)

understanding of basic programming concepts that also contain questions concerning variables and assignment are the FCS1 byTew and Guzdial(2011), and the test byDehnadi(2006).

FCS1

An important tool for assessing the effectiveness of instruction is a concept inventory (CI), which is a type of multiple choice test (Taylor et al.,2014). Each distractor (incorrect answer) is an indicator for a particular misconception. To develop a concept inventory therefore requires thorough knowledge of the core concepts of the topic, and related misconceptions that might occur.

Although a lot of preliminary work to develop CIs for computer science topics has already been done, only two CIs are complete: one for digital logic (Herman et al.,2010), and one for CS1 (Tew and Guzdial,2011).Taylor et al.(2014) reports that as of 2014, neither of them is yet in general use.

The CI for CS1, the FCS1, contains questions concerning ment as part of their Fundamentals construct (on variables, assign-ment, and mathematical expressions), but only three of them. Be-sides, the distractors have not been constructed explicitly from a set of observed misconceptions, but were made up by the authors based on solid guidelines (Miller et al.,2009, pp. 194–217).

Dehnadi

In the end, we based our misconception test on the work byDehnadi (2006), as this is the only test that focuses on variables of primitive types and assignment. It consists of questions based on code frag-ments and a number of multiple choice answers. The multiple choice answers consisted of various permutations of numbers derived from the code fragment, and were mapped onto behavioural misconcep-tions.

(23)

Methods

The goal of the experiment is two-fold: (1) assessing existing mis-conceptions in students on the topic of variables in computer pro-gramming, and (2) evaluating an intervention designed to teach the correct concepts of variables in computer programming. Based on frequently reported misconceptions (see Table1) we created a pre-test designed to bring these incorrect ideas to light. After complet-ing the test, participants watched an interactive video instructcomplet-ing them about variables. Next, the participants ﬁlled out a post-test similar to the pre-test, to evaluate the learning effect of the inter-vention. The participants, materials, and procedure are described in further detail below.

Participants

The experiment was conducted among three student populations: (1) 33 fourth-year havo and vwo students of secondary education taking an introductory course in programming with Visual Basic, (2) 2 fourth-year vwo students of secondary education who had just had an introductory course in programming with JavaScript, and (3) 6 ﬁrst-year software engineering students of a local university of applied sciences who had had one full year of programming courses in Java. In total we obtained complete test data from 41 participants.

(24)

Materials

The materials consist of a pre-test designed to assess existing mis-conceptions, an interactive video to instruct correct concepts of variables and a post-test to evaluate the learning effect of the video intervention.

Pre-test and post-test

When constructing a test to identify speciﬁc misconceptions, it is important to establish: (1) the misconceptions it should be able to identify, (2) how it will identify these misconceptions, (3) that it can distinguish between the various misconceptions, and (4) that one careless answer will not determine the (then incorrect) outcome (partly from (Ma,2007)).

To detect these misconceptions we constructed questions sim-ilar to the questions used byMa(2007) which in turn is based on Dehnadi(2006). Their tests contained multiple choice questions, where the answer options consisted of all apparently logical altern-atives. Our tests consist of very similar questions, but then not as multiple choice but as open questions. This way students would not be guided by what we would think are logical options, but would be free to ﬁll out the forms in whatever way ﬁt best given their ideas of how variables and assignment works.

Dehnadi(2006) observed a misconception not related to assign-ment speciﬁcally, but that does affect students’ answers: that the statements may be interpreted independently of one another. To detect this misconception, two extra questions have been added to the test, as advised by Dehnadi himself. For example, observe the second-last code sample in Table2:var a = 10; var b = 20; a = b; b = a;. For the last statement, a student with this independent statements misconception would ignore the fact thataa should now contain the value ofb, but would instead still treataas containing its original value of10. In this way, the two assignment statements are treated as independent from each other (although they are still dependent on the initialization statements). As can be seen in the table, this misconception results in answers different from the other misconceptions.

As the last question of each questionnaire, we added an open question in which we asked the participant to describe what happens in example code with assignment, just asMa(2007) did.

(25)

Methods 25

see Table2(points 2 and 3). Another difference withMa(2007) is that many questions were designed to explicitly trigger certain mis-conception categories, for example by using variable names with meaning instead of justaandbto elicit human interaction-related misconceptions, usingxandyto trigger math-related misconcep-tions, as well as using undeﬁned variables on the right-hand side to trigger equation-related and semantic misconceptions.

The pieces of programming code were kept as short and simple as possible. This limits potential problems due to careless reading.

Some misconceptions are difﬁcult to distinguish from each other, such as S1 (Variables are tested for being equal. No effect on value.) and M3 (Variables are ﬁxed values or constants), and also M1 (Vari-ables are set to being equal, also from left to right) and S2 (The re-ceiving variable is on the right-hand side). In those cases, distinction is still possible, but indirectly. If the participant does not hold mis-conception S1 as determined for example byvar a = 7; var b = a;, then when there is doubt between M3 and S1 in the case ofvar a = 4; var b = 3; b = a;, M3 becomes the most probable option.

Each misconception is related to multiple questions, in order to prevent careless answers from signiﬁcantly affecting the tests results (point 4).

It is important to note that Table2contains ideal answers in this context. In practice there will be more variation. For example, in the case of uninitialized variables, any answer that shows that the participant understands that this variable is not initialized yet is acceptable: undefined, a default value of0, or even anError. How the compiler or interpreter deals with uninitialized variables was not officially one of the learning goals defined for this topic. The officially correct answer depends on the programming language. With JavaScript it isundefined. VisualBasic works with default values, which is0for integers. For local variables, Java yields an Errorwhen the program is compiled. Java also works with default values, which is0for integers, just like VisualBasic. When analysing the responses of participants it is therefore important to value their apparent understanding over the literal matching of answers.

Combinations of misconceptions are also possible, for example, the misunderstanding that the receiving variable is on the right hand side in combination with the container idea that values are not copied but moved. We did not list all combinations explicitly in this table, as it would become unwieldy.

(26)

26 Identifying and Addressing Common Programming Misconceptions with Variables(Part 1) Q uest ion (Ja va Scri pt ) Va ri abl e C orrect answer H1. V ari abl es ca nnot cont ain va lues t ha t a re i n confl ict wi th thei r na m e. H2. V ari abl es cont ain va lues tha

t were ne

ver ex pl ici tly assi gned. M1. V ari abl es are set to bei ng equa l, a lso from left to ri ght . M2. T he st atem ent is a n equa tion t o be sol

ved for t

he unknown va ri abl e. M3. V ari abl es are fi xed va lues or const ant s, assi gned a va lue onl y once. C 1. A va lue i s m ov ed i nst ea d of copi ed. C 2. V ari abl es ca n cont ain m ul tipl e va lues. S1. Va ri abl es/ va lues are t est ed for bei ng equa

l. (No

effect on va lue ) S2. T he recei vi ng va ri abl e i

s on t

he ri ght -ha nd si de. S3. T he va lues of the va ri abl es a re swa pped. S4. T he new va lue i s a dded t o the pre vi ous va lue. S5. Resul ts ca n onl

y be st

ored i n va ri abl es not m ent ioned i n t he ex pressi on on the ri ght . St atem ent s a re independent . (Not r elate d to assi gnm ent, b ut m ay affe ct re sults) var$tien; tien undefi ned 10 var$dozijn; dozi jn undefi ned 12 var$drie$=$5; dri e 5 undefi ned Error var$straatnaam$=$101; st ra atna am 101 undefi ned Error var$groot$=$10; kl ei n 20 undefi ned 10

10 or 20

20 undefi ned 20 20 10 10 20 var$klein$=$20; Error groot$=$klein; groot 20 undefi ned 20

10 or 20

10 20 10, 20 10 10 20 30 Error 20 var$Hugo$=$12; Hug o 12 undefi ned undefi ned 12 var$Tim$=$Hugo$+$3; Error Tim 15 undefi ned 15 undefi ned Error var$a$=$7; a 7 undefi ned 7 undefi ned undefi ned 7 var$b$=$a; b 7 7 undefi ned undefi ned 7 Na N var$a$=$4; a 4

3 or 4

4 undefi ned 4 4 3 3 4 var$b$=$3; b 4

3 or 4

3 4 3, 4 3 3 4 7 b$=$a; var$x$=$10; x 20

10 or 20

10 20 10, 20 10 10 20 30 var$y$=$20; y 20

10 or 20

20 undefi ned 20 20 10 10 20 x$=$y; var$x; x undefi ned -2 undefi ned undefi ned undefi ned var$y$=$8; y Na N 8 8 8, Na N 8 y$=$x$+$10; var$a$=$8; a Na N 8 8 8, Na N 8 var$b; b undefi ned 2 undefi ned undefi ned undefi ned a$=$b$*$4; var$a$=$10; a 12

8 or 10

Error 10 10, 12 10 22 10 a$=$a$+$2; Error var$i$=$1; i 2

0 or 1 or 2

Error 1 1, 2 1 3 1 i$=$i$+$1; Error var$a$=$6; a 6 undefi ned 6 6 var$b; b 7 7 undefi ned Na N b$=$a$+$1; var$x$=$8; x 8 undefi ned 8 undefi ned 8 var$y; y 8 8 undefi ned 8 Na N y$=$x; var$a$=$10; a 20

10 or 20 or 30

10 undefi ned 10, 20 10 30 30 20 var$b$=$20; b 20

10 or 20

20 undefi ned 20 20 10 20 20 var$c$=$30; c 20

10 or 30

30 20 30, 10, 20 30 20 60 10 a$=$b; c$=$a; var$a$=$10; a 20

10 or 20

10 undefi ned 10, 20 10 10 30 20 var$b$=$20; b 20

10 or 20

20 20 20, 10 20 20 50 10 a$=$b; b$=$a; De sc rib e w hat hap pe ns on the last li ne of c od e: var$a$=$9; var$b$=$3; a$=$b; The ol d va lue of va ri abl e a is repl aced wi th a cop

y of t

he va lue of va ri abl e b The va lue of va ri abl

e b i

s put ne xt t o t he alrea dy e xist ing va lue i

n va

ri abl e a The com put er test s whet her the va

lues of a

and b a

re equa l The ol d va lue of va ri abl

e b i

s

repl

aced b

y a

cop

y of t

he va lue of va ri abl e a The va lue of va ri abl e a is repl aced b y t he va

lue of va

ri abl e b a nd vi ce v ersa The va lue of va ri abl

e b i

s added t o t he alrea dy e xist ing va lue i n va ri abl e a Va ri abl es a a nd b are bei ng m ade equa l Va ri abl es a a nd b are st ated a s bei ng equa l Va ri abl e a alrea dy ha s a va lue, so t he assi gnm ent is ig nored The va lue of va ri abl

e b i

(27)

Methods 27

For the Visual Basic tests, these names have been renamed to their equivalent English words, to match what the students were used to from the code examples from class.

The post-test has been constructed by replacing variable names and values with analogous alternatives, and putting the questions in a different order. The open question to describe what happens in a line of code with variable assignment was kept at the end.

The pre- and post-tests can be found in AppendixA. JavaScript Testson page55.

Interactive video intervention

Based on the misconceptions listed in Table1, we formulated six learning goals (listed in the same table). These learning goals reﬂect the correct understanding of variables, and were used as a basis for the design of the interactive video animation. Video instructions provide high motivation and support long-term knowledge acquiring (Van der Meij and Van der Meij,2013).

The structure of the video adheres to the recommendations for teaching variables following the constructivist approach mentioned byKuittinen and Sajaniemi(2004), extended with the rest of our learning goals in order of complexity. We avoided unnecessary use of programming terminology and explicitly showed the effects of the current programming statement on the variables in computer memory. To make the video relatively language independent, we left out certain bits of information from the video itself, such as default values.

The video follows architect Nico and hacker Nellie. Nellie is tra-cing a few lines of codes to retrieve a hidden number, Nico’s pass-word. While reading the lines of code the values stored in and re-trieved from the computer memory are visualized, see Figure2. The whole video is in paper-cut animation style. The full scenario is available in AppendixB. Instructional Intervention Script (Dutch).

The video pauses ﬁve times. To continue, the student must answer a short question (see Figure3). This interaction was included to hold the students attention and highlight the learning objectives. The

interactive video was created with the free online tool Hapyak. http://corp.hapyak.com/

(28)

Figure 2:Screen capture of the

interac ve video, showing the code on the right with the current line increased in size, and the computer memory with its three variables on the left.

Figure 3:Screen capture of a ques on

embedded in the interac ve video.

The question is: “What does the equals sign signify in: length = 4;”. The parti-cipant can select the answer from four

options: (1)lengthequals 4; (2) The

computer will solve a mathematical

equation; (3)lengthwill get a new value,

being 4; (4) 4 is added to the value in length. On submit, the video will indic-ate whether the selected answer was correct.

on the interactive video, we did not provide additional exercises. Van der Meij and Van der Meij(2013) also offer a concrete ex-ample of a good design solution for a conceptual knowledge video: provide a visual simulation of the steps that are executed, with a voice-over to explain the reasoning behind each step. Our video fol-lows this recommendation by visually showing the effect of code on the variables and their values in memory. The voice-over provides context and explains verbally what is happening in each step, per-haps more in a procedural than conceptual manner.

Guideline 1: Provide easy access The title of our video is short but

(29)

Methods 29

part of a larger knowledge base, but offered by itself, Guideline 1 is not (yet) relevant.

Guideline 2: Use animation with narration Our video starts with a

quick recap of what variables are for, to then quickly move on to the main question: How do variables work? This question is addressed from a scenario in which a hacker looks through code line by line and tries to determine what the code does in order to ﬁnd a secret pass-word number. The code starts off simple but increases in complexity throughout the video. There is no ‘actual interface’ (Guideline 2.1), but the code shown is compilable, working programming code. The computer memory bank with the variables is a conceptual repres-entation. Human spoken voice explains what happens on the screen (Guideline 2.2), and action and voice are synchronized (Guideline 2.3).

Guideline 3: Enable functional interactivity The video follows a slow

but conversational pace (Guideline 3.1). The Hapyak video player is based on YouTube and provides similar functionality: play, pause, replay, and skipping to different moments by clicking on the timeline underneath (Guideline 3.2). In this timeline, Hapyak also shows when the questions will pop up, segmenting the video into logical parts.

Guideline 4: Preview the task As already mentioned, the video starts

off with the use of variables, to provide motivation, and then quickly moves on to its goal: explaining how variables work (Guideline 4.1). The video uses a conversational style (Guideline 4.2).

Guideline 6: Make tasks clear and simple Explanations are kept short

and simple, avoiding unnecessary terminology. Special attention has been paid to the sequencing of steps, so they start simple and increase in complexity (Guideline 6.1). The visualization of the variables in memory shows the relation between programming code (user actions) and the effect on the system (system reactions) (Guideline 6.2). The current line of code is highlighted by increasing its size, which also helps with its readability (Guideline 6.3). Variable names are highlighted in the same color as they are shown in the memory visualization.

Guideline 7: Keep videos short Another recommendation is to keep

(30)

a conscious decision, as the participants are meant to watch the whole video in one sitting. We do provide segmentation with the six multiple choice questions that pop up at logical points during the narration.

The resulting interactive video is available at:http://www.hapyak. com/embed?key=36d389f5308f403ea94e&project=20131.

Procedure

In the class

Each student sat at a computer individually. They received oral explanation of what they were expected to do: read the instructions thoroughly, fill out the first test, watch the video, and finally fill out the second test. It was emphasized that students were not allowed to talk to each other. Students were also not allowed to look up answers.

Further details and links were provided through the online learn-ing environment of the school, It’s Learnlearn-ing. This information was the same as the information on the website for the email parti-cipants. The only difference was their reward. In this case, the stu-dents who obtained a score of above 65% correct on the tests got exemption for a number of questions on a written test they would have to do later on (given full points).

During the experiment only procedural questions from the stu-dents were answered, for example when they had problems watch-ing the video.

Participants by email

The email participants received an invitation to participate in an experiment that would help themselves assess and improve their knowledge about variables. Besides, they would have a chance of winning cinema tickets.

The email contained a link to one of our websites. We had set up web pages for each of the target programming languages. The website for JavaScript can be seen here:http://www.dannyplass. nl/variabelen.

(31)

Results

In this report, which constitutes Part 1 of our research, we address Other important research questions, such as: “What is the effect of the inter-vention on each of the misconception categories?”, will be covered in Part 2 of the report.

the following questions:

1. What misconceptions did we observe in practice?

2. Did the interactive video intervention have an effect on the test scores of the misconception test?

What misconceptions did we observe in practice?

The question concerning what misconception we observed in prac-tice can be split up into the following subquestions: What additional misconceptions did we observe, aside from the misconceptions we predicted based on literature as listed in Table1? What misconcep-tions did wenotobserve in practice? To what extent are the different misconception categories identiﬁed by our pre-test questions? And did similar questions also result in similar answers?

Additional misconceptions

We observed a number of additional misconceptions we had not anticipated, but which did occur regularly. As such, we included these misconception in our analysis. An updated overview of Table1 with the newly added misconceptions can be found in Table3.

Math M2b: The statement is an equation to be solved. Variables con-tain ‘factors’ that are multiplied by or added to the original values of

the variables This misconception is a variation on misconception

(32)

(33)

Results 33

a;ﬁve participants tried to somehow solve the equation, arriving at solutions such as:a: -1andb: 1(by 4 participants), ora: 3/4and b: 4/3(by 1 participant).

These equations are not solved by putting the factors for both variables together inoneequation. Instead, the equation is solved for the variables one by one, forming separate equations for each. In

the case ofvar groot = 10; var klein = 20; groot = klein;, an grootmeans ‘big’ andkleinmeans ‘small’

answer given twice wasgroot: 2, klein: 0.5. Two timesgrootis equal toklein, and a half timekleinequals togroot, but together in one equation this would be2*groot = 0.5*klein, which would resolve to20 = 10, which is obviously incorrect and not what the participants intended. So, although the students are solving equa-tions, they are not approaching it in a consistent algebraic way, see also added misconception M2d.

It is interesting to note that participants are not always consistent with the misconceptions they display in particular types of ques-tions. For example, one participant started out not showing this particular misconception, but misconception M3, that variables are constants (question 6+7). Then for the next question (8+9) this par-ticipant moved on to the multiplication factors described above. For question 10+11, the approach changed again, this time to addition factors.

Math M2c: The statement is an equation to be solved. Variables can

contain expressions. With algebraic equations, variables are often

expressed in terms of the other variables in the original equation. One of the observed misconceptions was a clear expression of par-ticipants using this approach. For example, for the questionvar x; var y = 8; y = x + 10;the following answer was observed four times for the value ofy:x + 10, whilexwas resolved to either-2or no value.

Another example isvar Hugo = 12; var Tim = Hugo + 3;, whereTimwas correctly stated to contain the value15, but the value ofHugowas said to beTim - 3. This actually appears to be a combination of two misconceptions: M2c as just described, but also M2d: the equation is solved for the two variables separately). This misconception will be described next.

Math M2d: The statement is an equation to be solved. Affects variables

separately. First, the equation is solved for one variable, then

(34)

y = x + 10;, this misconception would result in simultaneously10 for variabley(assumingxcontains default value0) and-2forx.

Semantics S1b: Variables/values are tested for being equal. If false: Er-ror. The misunderstanding that the equality sign actually indicates a testof equality instead of an assignment statement has already been mentioned before in literature (such asMa et al.,2007). However, this existing literature did not detail how students would expect the computer to respond to such a test of equality. If the result of the statement would be simplytrueorfalse, like a normal test for equality would with==, the statement would simply have no effect, as the resulting value is not used.

Based on the answers students gave in our tests, we now know that most students who hold this misconception actually expect different behaviour from the program. “Test for equality” is perhaps a bit of a misnomer. Students seem to look at it more as a “statement of equality”. If this statement is correct, the program is executed. If

Apparently students have an intu-itive sense for the Java concept of

assertions. the statement is not correct, the program yields an Error.

Other O1: The value for one variable is computed, and the other is

simply set to be equal. To give a couple of examples: Forvar a =

These two misconceptions have been categorized as Other, as they do not seem to be just different interpret-ations of the logic of the code. They seem to disregard what the code is actually stating.

6; var b; b = a + 1;, four participants answered for bothaand bthe value7. In the case ofvar a = 8; var b; a = b * 4;, again four participants answered along this way of thinking, with a value of2for both variables. Incidentally, these second four participants were all different people from the four in the ﬁrst example. More on this internal inconsistency later on.

Other O2: When a variable value is undeﬁned, use a known value from

another variable. The ﬁnal unanticipated misconception we

ob-served multiple times is this: Givenvar x; var y = 8; y = x + 10;, as expected, many participants tried to solve the equation forx, resulting in-2(for 26 participants out of 41). But when then asked for the value ofy, some participants answered with the value18(3 participants). This value appears to be derived by simply adding the one other value mentioned in the code, even though it was not as-signed toxbut toy. We see a similar result withvar a = 8; var b; a = b * 4;where participants answer that the value ofais32.

Unobserved misconceptions

(35)

Results 35

Set Nr Question (JavaScript) Variable

Math Human inter. Container Semantics Other Total

1var$tien; tien 0 7 0 0 17 24

29var$dozijn; dozijn 0 11 0 0 5 16

15var$drie$=$5; drie 0 0 0 0 0 0

26var$straatnaam$=$101;$ straatnaam 0 1 0 0 0 1

2var$a$=$7; a 2 0 0 0 2 4

3var$b$=$a; b 4 0 0 0 1 5

24var$x$=$8; x 2 0 0 0 0 2

25var$y; y 0 0 0 0 0 0

y$=$x;

8var$groot$=$10; groot 12 0 0 5 1 18

9var$klein$=$20; klein 5 1 0 5 0 11

groot$=$klein;

10var$a$=$4; a 6 0 0 6 2 14

11var$b$=$3; b 11 0 0 8 3 22

b$=$a;

6var$x$=$10; x 13 0 2 7 1 23

7var$y$=$20; y 4 0 0 5 2 11

x$=$y;

4var$Hugo$=$12; Hugo 2 0 0 1 4 7

5var$Tim$=$Hugo$+$3; Tim 0 0 0 0 0 0

22var$a$=$6; a 0 0 0 0 5 5

23var$b; b 0 0 0 0 1 1

b$=$a$+$1;

13var$x; x 25 0 0 0 6 31

14var$y$=$8; y 6 0 0 0 7 13

y$=$x$+$10;

19var$a$=$8; a 4 0 0 0 7 11

20var$b; b 23 0 0 0 6 29

a$=$b$*$4;

21var$a$=$10; a 14 0 0 1 0 15

a$=$a$+$2;

12i$=$1; i 8 0 0 2 1 11

i$=$i$+$1;

16var$a$=$10; a 10 0 0 3 2 15

17var$b$=$20; b 4 0 0 1 3 8

18var$c$=$30; c 12 0 0 3 7 22

a$=$b; c$=$a;

27var$a$=$10; a 10 0 0 3 4 17

28var$b$=$20; b 2 0 0 3 12 17

a$=$b; b$=$a;

var$a$=$9; 12 0 0 4 5 21

var$b$=$3; a$=$b;

Total 191 20 2 57 104 374

Percentage 51% 5% 1% 15% 28% 100%

x = x

+ c

Independent

st

at

em

ent

s

Misconception categories

Describe what happens on the last line of code 30

Descri

be

H2

H1

x = y

? = x

+ c

x = ?

+ c

Table 4:Number of misconcep ons

(36)

or less, and not repeated within the answers of these participants) that it does not cross the error margin.

Human interaction Although we did observe misconception H2

(Variables contain values that make sense given its name, but were never explicitly assigned) inelevenparticipants, its related

miscon-Perhaps the variable names could have been made even more conﬂicting with its values, see also the other suggested improvements for the test in the Discussion.

ception H1 (Variables cannot contain values that are in conﬂict with its name) was hardly observed at all.

Container The misconceptions that could have been caused by the

container analogy have also rarely been observed. C1 (A value is moved instead of copied) never showed at all. C2 (Variables can contain multiple values) seemed to make an appearance in question 6+7, but was not repeated by the two participants in other answers.

Semantics We had expected for misconception S1 (Values are

tested for being equal) for the result to be true or false, as it would be for==. As a stand-alone statement this would have no further effect. This misconception was not observed at all (although it may have been difﬁcult to distinguish from M3 in all cases). We did ob-serve a lot of S1b, the same misconception, but then resulting in an Error when the test evaluates to false.

Misconception S3 (The values of the variables swap) has been sighted twice, but very inconsistently.

To what extent are the different misconception categories identiﬁed by

our pre-test questions?

Table4shows per question what misconception categories were identiﬁed from the answers of the participants. Indications for each misconception category have been identiﬁed, but the same does not hold for each individual misconception, as we saw earlier.

Triggering the highest number of misconceptions The questions that

triggered misconceptions in the most participants werevar x; var y = 8; y = x + 10;(34 out of 41 participants; 83%) and the

ana-The counts in Table4are for each

vari-able separately.Thiscount here is for

participants with a misconception for either one or both of these variables.

(37)

Results 37

The presence of misconception categories When looking at the

amount of participants who showed indications of each of the mis-conception categories, we ﬁnd the following percentages: Math: 83%, Human interaction 34%, Container 5%, Semantics 39%, and

Other 80%. This percentage is the number of participants showing These percentages are simply based on the presence of one or more answers indicating that particular category, so they do not take into account within participant consistency. Read on for more information about this aspect.

the misconception at least once to the total number of participants. Misconceptions from the Math domain have been observed the most. The Other category has also been highly present.

Within question set consistency Did similar questions indeed result

in similar answers? To investigate this, similar questions were paired

according to the Set column in Table4. Almost all question sets do It is important to note however, that although the distributions may be similar, this does not necessarily mean these misconceptions were also observed in the same participants, see also the next paragraph on Within participant consistency.

indeed show a similar distribution across the different misconcep-tion categories.

The two exceptions were H2 and Independent statements. We did indeed observe misconception H2, as can be seen in the ﬁrst two questions in the table. The very ﬁrst question also has a lot of

answers falling in the Other category, as many participants ﬁlled In the discussion section we propose a slight alteration to the test which might prevent this misinterpretation.

out the variable type (‘Integer’ or ‘Round number’) instead of an expected value.

The other question set that did not show consistent misconcep-tions was Independent statements. This set was added not as part of a learning goal, but as the Independent statements or Parallellism bug could affect interpretations of code with assignment statements. Where the answers tovar a = 10; var b = 20; var c = 30; a = b; c = a;showed mostly math-related misconceptions, the other question in the setvar a = 10; var b = 20; a = b; b = a;was answered with more variation, showing amongst others O1, O2, and Independent statements (also classiﬁed as Other).

Within participant consistency As noted before, we observed a high

inconsistency within the answers of participants. As mentioned, for example, when describing the added misconception O1, the four participants showing this misconception in one question were not the same as the four participants demonstrating this misconception in a different question.

This was enough cause to inspect the consistency of the answers within participants. For this, for each participant we marked for each of the 8 question sets (see the ﬁrst column in Table4) whether the answers indicated the same misconceptions.

(38)

Inconsistencies in the answers within participants

Number of inconsistencies for sets of similar questions

Per

cen

tag

e

of

par

ti

cipan

ts

Figure 4:Distribu on of the amount

of inconsistencies in the answers within par cipants.The questions were grouped in 8 sets of highly sim-ilar questions. Answers within sets were compared for inconsistencies in misconceptions.

Did the interactive video intervention have an effect

on the test scores of the misconception test?

Based on the amount of incorrect answers on the pre- and post-tests, we can determine whether the video intervention indeed had an effect in addressing any misconceptions at all. We had complete test data from 41 participants: 6 for Java, 2 for JavaScript, and 33 for VisualBasic. As one question was accidentally left out of the post-test, we could only use 29 of the 30 questions we had originally

The question accidentally left out of

the post-test was:var a = 5; a =

a + 10;, an assignment statement where the expression contains the variable used to store the resulting value in. The analogous question left out of the pre-test for this particular

analysis:var a = 10; a = a + 2;.

Its alternative was kept in the analysis,

which isvar i = 1; i = i + 1;for

the pre-test andvar i = 4; i = i +

1for the post-test.

developed for these tests. The answers from the participants were recoded to either correct or incorrect, which resulted in a total number of correct answers per test per participant.

Figure5shows the pre-test scores of each participant on the x-axis and the related post-test score on the y-x-axis. This plot already shows that for most participants, the post-test was an improvement over the pre-test. Most data points are above the diagonal line that marks the positions for neutral neutral test scores (the same score on the pre-test as on the post-test).

(39)

Results 39

Figure 5:Pre- and post-test scores

(40)

(41)

Discussion

Participant population

These results are mainly based on the data gathered from students from one school, taught by one teacher and one intern in one pro-gramming language. This group makes up 33 students out of the 41 participants in total. Different teachers may use different lesson materials and approaches, possibly leading to less, more, or simply different misconceptions. It is therefore important to follow-up this investigation with different groups, exposed to different teachers and materials, to further assess the effectiveness of the tests and intervention.

Effect of the intervention on test scores

We observed a signiﬁcant improvement in the misconception test scores. Part of this improvement may be simply due to the fact that the participants had recently done a very similar test. To reduce the number of interpretation mistakes in the ﬁrst test, it is recommen-ded to add some easy introductory questions to the beginning of the test, see also the section onSuggested improvements to the pre- and post-test at the end of this chapter.

(42)

Observing misconceptions

Integers as representatives of primitive types The code samples with

integers provided enough complexity to elicit an interesting set of misconceptions. The misconceptions with ﬂoating point numbers are expected to be similar. For the character type the operators used in our examples will have different effects (if they are even deﬁned for this type), so to test for misconceptions with characters a different question set will need to be constructed. A more commonly used primitive type is the boolean. This type may result in less misconcep-tions as students are likely to have less previous knowledge linking to true/false values. Although strings are treated in some languages as primitive types, in other common programming languages they are reference types. All in all, numbers are expected to be the most problematic in terms of misconceptions (see also Table3). This might make it didactically more sound to start teaching about variables and assignment with less ‘contaminated’ types such as booleans.

Selected misconceptions As the designs of our misconception tests

and our intervention were based on selected misconceptions, it is possible that there are misconceptions that students may hold but which are not uncovered with our tests nor addressed with our intervention.

When updating Table1(Learning goals and their related mis-conceptions) to Table3based on the answers of our participants, we already had to add to Learning goal 2: “A value is assigned to a variable, using the name of the variable followed by the ‘=’ symbol (name = value)” the subtle extension: “and the expression for the new value”, to explicitly point to the problems caused by misconcep-tions O1 and O2. In the interactive video this aspect had already been explicitly visualized, so these misconceptions may have already

Part 2 of our research report will discuss the effect of the intervention

on speciﬁc misconception categories. been addressed, even though they were not explicitly identiﬁed

be-forehand.

As long as new misconceptions seem to be sufﬁciently addressed by one of the previously-deﬁned learning goals (Table3), they are likely to be addressed by the interactive video.

Open answer questions versus multiple choice In her PhD thesis on

(43)

Discussion 43

physics concept inventory, over 1000 students were tested using short answer questions (Tew,2010).

The answers in the multiple choice tests available for the concept of assignment (Dehnadi,2006;Ma,2007) have not been rooted in such thorough prior research. Although Dehnadi did provide space to ﬁll out other values, students who are uncertain about their answer will probably go for one of the options already given, as the correct answer is most likely among them.

As already described in theMethods, we purposefully chose for short answer open questions. This has allowed us to observe answers very different from the options Dehnadi provided, such as those related to misconceptions M2b and M2c (seeResults).

On the other hand, the open question at the end of each mis-conception test — to describe what happens on a line of code with an assignment statement — might be more revealing as a multiple choice question, as often the descriptions of participants were not very distinctive. However, as Tew stated, this should be done very carefully. We should ﬁrst need to gather more answer alternatives through interviews, think-aloud protocols and open questions.

The presence of misconception categories Table2lists the total

num-ber of misconceptions per category and the corresponding percent-age of the total number of observed misconceptions. This number and percentage obviously depends on the number of questions that could trigger this category, which is, for example, bigger for the Math category than for Human interaction.

The percentages that indicate how many of the participants showed each of the misconception domains have not been correc-ted for within participant consistency. So it is possible that these numbers also contain the results of temporary lapses of concentra-tion or typographic errors.

Additionally, certain misconceptions may be difﬁcult to distinguish simply based on answers, such as M3 (variables as constants) and S1 (test of equality where the result (true, false) is simply discarded).

The answers have been recoded to observed misconception by one expert only. It would have been better to have multiple experts and aim for consensus.

Within question set consistency As already mentioned in theResults

(44)

Within participant consistency Participants themselves where

of-ten inconsisof-tent in misconceptions shown in their answers. Perhaps these unsettled participants were trying to get at least one answer correct by using different approaches on purpose, spreading the risk. This may have been stimulated by the promise of exemption for ofﬁcial test questions for students who scored well on the miscon-ception tests.

But there are various other possible reasons for this inconsist-ency. Students might actually hold multiple ideas of how constructs might function, each of which is triggered in other contexts (Halloun and Hestenes,1985). In the work ofHalloun and Hestenes(1985) students only realized their inconsistencies when they were asked to explicitly discuss their reasoning. It is known that novices do not reason from underlying principles as experts do, but focus more on literal features (Chi et al.,1981).

Alternatively, perhaps the students were simply trying to get a taste of what the results with another approach would be like. Par-ticipants may also have forgot what their way of reasoning was with that previous question that was similar, especially if there were not certain about their answer in that case either. Some participants changed their minds about what they expected the code to do, re-cognizable by moving to a different but consistent misconception in later answers. Besides, sometimes people just make mistakes in haste, either by interpreting the question incorrectly or by making a typographical error.

Such inconcistency is not uncommon nor speciﬁc to computer science.Halloun and Hestenes(1985), the developers of the FCI mentioned earlier, already report that only 6% of their participating students showed consistent beliefs (categorized in three theories) across similar tasks in their pre-test, and only 2% remained consist-ent on their post-test.

Conclusions

Assignment with variables of primitive types is one of the early con-cepts taught in programming classes, but even this basic principle can present a lot of problems to students. Previous experience from real life or other more abstract subjects can cause students to hold misconceptions as to how this assignment actually works.

Related work has uncovered a number of misconceptions, but most of these studies did not focus on assignment in particular. Those studies that did have this focus did not try to uncover the underlying incorrect ideas but only described the resulting beha-viour.

Based on literature, we have constructed an overview of miscon-ceptions, grouped in misconception categories, and learning goals with the correct alternative ideas which might function as an an-tidote. These correct conceptions have been used to develop an interactive video in paper cut style, to provide students and teach-ers with a simple intervention to address these misconceptions. To evaluate the effectiveness of this intervention, we also designed a misconception pre-test and an analogous post-test which consist of 30 short-answer questions concerning short samples of program-ming code with variable declarations and assignment statements. The evaluation has been conducted with 41 participants: 35 fourth-year havo and vwo students and 6 ﬁrst-fourth-year software engineering students of a local university of applied sciences.

This report contains the results of the ﬁrst half of the analysis, which describes the misconceptions observed with the pre-test. It also provides initial results of the intervention. A more thorough analysis of the effects of the intervention will be presented in part 2.

Observing misconceptions 83% of our participant population

(48)

80% of the participants showed misconceptions now categorized as Other. Follow-up research might be able to point to a more descript-ive category name.

A number of misconceptions concerning assignment were ob-served that we did not anticipate as such: M2b) Variables can con-tain factors to be multiplied by or added to the original variable values in order to solve the equation; M2c) Variables can contain ex-pressions; M2d) The equation is solved separately for each variable; and S1b) Variables are tested for being equal and when this is false, the program yields an Error. Two misconceptions which were ob-served multiple times, but which ignore part of the program are: O1) The value for one variable is computed, and the other is simply set to be equal; O2) When a variable is uninitialized, but used anyway, a known value from another variable is used instead.

Misconceptions not observed consistently (in more than 2 an-swers and not consistently within those participants) are: H1) Vari-ables cannot contain values in conﬂict with their name; all container misconceptions — C1) A value is moved instead of copied; C2) Vari-ables can contain multiple values — and S1) VariVari-ables are tested for being equal which results in a(n unused) value of true or false. To investigate whether these misconceptions are less common than expected or whether this observation purely the result of our partic-ular participant population, further research is recommended.

Within question set consistency and within participant consistency

Within the pre-test, most of the questions grouped for similarity were indeed answered similarly (with a similar distribution across the different misconception categories). However, these similar answers were not always given by the same participants. Of the 8 question sets, 88% of the participants were inconsistent at least once; 54% for at least 3 of the sets. Such high levels of within parti-cipant inconsistency correspond with the results of observations of student misconceptions in other subjects.

Effect of intervention on test scores We observed a positive effect

on the misconception test scores for this particular group of parti-cipants. Further research will be necessary to conﬁrm this effect on

In Part 2 of our research report, we will investigate its effects on the different

misconceptions in more detail. groups which may hold different misconceptions due to different

(49)

Conclusions 49

Recommendations for teachers and developers of

teaching materials

The materials and results from this research could be a valuable starting point for the development of teaching materials, for the teaching process and for assessment.

The way certain concepts are explained can have a large impact on the likelihood that the student picks up the learning goal correctly. By being aware of the possible misconceptions students can hold or form, developers of teaching materials have the option to avoid or to address them.

Similarly, for teachers it could be very helpful to keep in mind the overview of misconceptions and learning goals (correct ideas) in

Table3, especially for beginning teachers who do not yet have the And even before such basic con-cepts as assignment, it is import-ant that students understand the mechanical way that computers in-terpret our commands. One convin-cing and entertaining way to show this to students is by letting them instruct the peanut-butter-jelly-sandwich robot (the teacher). An example of how this could be done is

shown here by David Malan:https:

//youtu.be/cfCs7smUJbY?t=14m30s.

experience of having observed these misconceptions in practice. When you are more aware of possible problems students may have, it becomes possible to adjust your teaching in order to prevent these misconceptions, and if any still do occur they are easier to identify and address.

With the pre- and post-tests we developed it becomes possible to quickly assess students, or for students to quickly assess them-selves, for any presence of these selected misconceptions. If any mis-conceptions are detected, it becomes possible to offer extra learning materials, such as our interactive video.

Future work

The tests and intervention we propose in this work require further validation with other classes with other teachers and other lesson materials. Such a validation should also involve a control group to account effects besides the intervention itself.

This research focused on a very small part of the concepts intro-duced in programming. It could be valuable to further develop such misconception overviews, simple tests, and intervention materials for the other concepts as well, such as control structures, functions and objects.

A high number of participants showed misconceptions we tempor-arily categorized under the non-descriptive name ‘Other’. Follow-up research might be able to uncover some underlying domain or theme driving these misconceptions, and ﬁnd what exactly triggers them.

(50)

signiﬁc-50 Identifying and Addressing Common Programming Misconceptions with Variables(Part 1)

antly reduced. As a downside, this would make it more likely for a student to trigger a certain misconceptions due to a simple lapse of concentration. On the other hand, such lapses might occur less with a shorter test. As each question type is repeated at least twice for the aforementioned reason of excluding simple mistakes, it should be easy to reduce the amount of questions in the current test from 30 to 15 (see Table2for the question sets). Such a shorter version of the tests would require testing for validation, with other classes, teachers and materials, as mentioned before for the existing tests and intervention.

(51)

Bibliography

Bayman, P. and Mayer, R. E. (1983). A diagnosis of beginning pro-grammers’ misconceptions of basic programming statements. Communications of the ACM, 26(9):677–679.

Ben-Ari, M. (1998). Constructivism in computer science education. InProceedings of the twenty-ninth SIGCSE technical symposium on Computer science education - SIGCSE ’98. ACM Press.

Chi, M. T., Feltovich, P. J., and Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive science, 5(2):121–152.

De Bakker, L. and Land, A. (2015). Informele wetenschapseducatie (Informal science education). In Dam, V., Bakker, D., and Dijkstra, editors,Wetenschapscommunicatie. Een Kennisbasis. Boom, Nether-lands. To appear.

Dehnadi, S. (2006). Testing programming aptitude. InProceedings of the 18th Annual Workshop of the Psychology of Programming Interest Group, pages 22–37.

Grgurina, N. and Tolboom, J. (2007). The Dutch secondary school informatics curriculum–another ‘polder model’, broad in scope, but not too deep. InProceedings Joint Conference of IFIP/WG3. 1 and WG3, volume 5.

Halloun, I. A. and Hestenes, D. (1985). Common sense concepts about motion.American journal of physics, 53(11):1056–1065.

Herman, G. L., Kaczmarczyk, L., Loui, M. C., and Zilles, C. (2008). Proof by incomplete enumeration and other logical misconcep-tions. InProceedings of the Fourth international Workshop on Com-puting Education Research, pages 59–70. ACM.

Identifying and addressing common programming misconceptions with variables (part 1)