Integers as representatives of primitive types The code samples with
integers provided enough complexity to elicit an interesting set of misconceptions. The misconceptions with floating point numbers are expected to be similar. For the character type the operators used in our examples will have different effects (if they are even defined for this type), so to test for misconceptions with characters a different question set will need to be constructed. A more commonly used primitive type is the boolean. This type may result in less misconcep- tions as students are likely to have less previous knowledge linking to true/false values. Although strings are treated in some languages as primitive types, in other common programming languages they are reference types. All in all, numbers are expected to be the most problematic in terms of misconceptions (see also Table3). This might make it didactically more sound to start teaching about variables and assignment with less ‘contaminated’ types such as booleans.
Selected misconceptions As the designs of our misconception tests
and our intervention were based on selected misconceptions, it is possible that there are misconceptions that students may hold but which are not uncovered with our tests nor addressed with our intervention.
When updating Table1(Learning goals and their related mis- conceptions) to Table3based on the answers of our participants, we already had to add to Learning goal 2: “A value is assigned to a variable, using the name of the variable followed by the ‘=’ symbol (name = value)” the subtle extension: “and the expression for the new value”, to explicitly point to the problems caused by misconcep- tions O1 and O2. In the interactive video this aspect had already been explicitly visualized, so these misconceptions may have already
Part 2 of our research report will discuss the effect of the intervention
on specific misconception categories. been addressed, even though they were not explicitly identified be-
forehand.
As long as new misconceptions seem to be sufficiently addressed by one of the previously-defined learning goals (Table3), they are likely to be addressed by the interactive video.
Open answer questions versus multiple choice In her PhD thesis on
assessing introductory computing concepts, Tew states: “[Multiple choice questions], when constructed carefully, can provide the same information about conceptual knowledge as short answer or open response questions with significant advantages in test administra- tion and scoring”, where she provides the development of the FCI, a concept inventory for introductory physics, as an example. For this
Discussion 43
physics concept inventory, over 1000 students were tested using short answer questions (Tew,2010).
The answers in the multiple choice tests available for the concept of assignment (Dehnadi,2006;Ma,2007) have not been rooted in such thorough prior research. Although Dehnadi did provide space to fill out other values, students who are uncertain about their answer will probably go for one of the options already given, as the correct answer is most likely among them.
As already described in theMethods, we purposefully chose for short answer open questions. This has allowed us to observe answers very different from the options Dehnadi provided, such as those related to misconceptions M2b and M2c (seeResults).
On the other hand, the open question at the end of each mis- conception test — to describe what happens on a line of code with an assignment statement — might be more revealing as a multiple choice question, as often the descriptions of participants were not very distinctive. However, as Tew stated, this should be done very carefully. We should first need to gather more answer alternatives through interviews, think-aloud protocols and open questions.
The presence of misconception categories Table2lists the total num-
ber of misconceptions per category and the corresponding percent- age of the total number of observed misconceptions. This number and percentage obviously depends on the number of questions that could trigger this category, which is, for example, bigger for the Math category than for Human interaction.
The percentages that indicate how many of the participants showed each of the misconception domains have not been correc- ted for within participant consistency. So it is possible that these numbers also contain the results of temporary lapses of concentra- tion or typographic errors.
Additionally, certain misconceptions may be difficult to distinguish simply based on answers, such as M3 (variables as constants) and S1 (test of equality where the result (true, false) is simply discarded).
The answers have been recoded to observed misconception by one expert only. It would have been better to have multiple experts and aim for consensus.
Within question set consistency As already mentioned in theResults
chapter, H2 was not answered consistently, probably because par- ticipants were yet uncertain of the type of answers asked of them. See SectionSuggested improvements to the pre- and post-test at the end of this chapter for a suggestion to improve this in follow-up experiments.
44 Identifying and Addressing Common Programming Misconceptions with Variables(Part 1)
Within participant consistency Participants themselves where of-
ten inconsistent in misconceptions shown in their answers. Perhaps these unsettled participants were trying to get at least one answer correct by using different approaches on purpose, spreading the risk. This may have been stimulated by the promise of exemption for official test questions for students who scored well on the miscon- ception tests.
But there are various other possible reasons for this inconsist- ency. Students might actually hold multiple ideas of how constructs might function, each of which is triggered in other contexts (Halloun and Hestenes,1985). In the work ofHalloun and Hestenes(1985) students only realized their inconsistencies when they were asked to explicitly discuss their reasoning. It is known that novices do not reason from underlying principles as experts do, but focus more on literal features (Chi et al.,1981).
Alternatively, perhaps the students were simply trying to get a taste of what the results with another approach would be like. Par- ticipants may also have forgot what their way of reasoning was with that previous question that was similar, especially if there were not certain about their answer in that case either. Some participants changed their minds about what they expected the code to do, re- cognizable by moving to a different but consistent misconception in later answers. Besides, sometimes people just make mistakes in haste, either by interpreting the question incorrectly or by making a typographical error.
Such inconcistency is not uncommon nor specific to computer science.Halloun and Hestenes(1985), the developers of the FCI mentioned earlier, already report that only 6% of their participating students showed consistent beliefs (categorized in three theories) across similar tasks in their pre-test, and only 2% remained consist- ent on their post-test.