Participants selected debugging tactics by considering anticipated risks as well as short- and long-term rewards. As summarized in Figure 4.6, we described the following risks and rewards that participants anticipated when deciding to use a debugging tactic: (1) time
investment, (2) expected immediate value, (3) future value, (4) future work, (5) manual effort, (6) mental effort, (7) risk of failure, and 8) power.
Figure 4.6: Codes and examples of activities in selecting debugging tactics that fall under the “anticipating future risks and rewards” theme
Time investment.
Participants considered the potential time necessary to apply certain strategies. For example, Participant 12 described having to spend time taking out a piece of paper as a barrier to them taking notes on paper as a strategy: “For me, I don't want to have to spend the time to
get a paper and draw it. You know, go back and forth. I know it's very useful, but if for easy problems, I'm just like, well what if I just printed out this and let's see where it takes me.”
Other participants also raised concerns about underestimating the amount of time they needed. According to Participant 4, he does not expect to spend an hour fixing a small error and “if it takes an excessive amount of time, at some point [he will] just stop.”Alternatively, the potential to save time also motivated participants to pursue certain strategies. Participant 9, for example, described adding comments as a strategy that would save them time in the future:
“So, commenting is basically abstracting away from the code and just saying what it does. So instead of rereading, ‘Oh okay. What does this line do? What does this line do?’ I just look at comments and be like, ‘Oh, yeah. Okay. So, this block does this. [...] And when I look back that will save me time.’”
Manual and Mental Effort.
In addition to time, participants worried about the manual and mental effort needed to apply certain strategies. Participants mentioned the tediousness involved when applying the debugger. For example, Participant 6 stated, “[Y]ou have to figure out where exactly it would be useful, the breakpoint, and stepping through. And it's just kind of tedious.” Participant 1
described similar sentiments about the extra work involved with debuggers: “I don’t know. I feel it’s just extra steps. You have to figure out what line you want to do it and then step through it.”
In other environments, such as MIPS for assembly, print statements were the culprit rather than debuggers. Participant 9 described the overhead involved with setting up print statements for debugging in assembly: “It's [...] too much of a hassle, because instead of Python and Java, where you can just print, you need to load into a register. [...] Basically, the sycalls are just a pain.”
In addition to the setup costs involved with applying certain strategies, the effort involved with processing large amounts of data also made certain strategies less appealing. Participant 11 described the inconvenience of using the debugger to investigate large for-loops due to having to repeatedly run through many iterations of the loop: “It doesn't run all the
solutions at once. It runs only one at a time. So that makes it a bit more inconvenient [...] which is why for for-loop statements typically I don't really rely on breakpoints.”
Participant 6 highlighted the mental effort involved in keeping track of changes when applying the debugger to large volumes of data: “I remember when I would use it just stepping through and seeing if every variable would change how I wanted it to and it's just a lot to keep track of.”Meanwhile, for smaller datasets, Participant 2 described favoring print statements: “And then the print statements [...] I use those if it's small data. If it's just one or two, three lines that you can print out and kind of see what issue is happening where.”
Another barrier that participants described involved learning how to use strategies they were not familiar with. For Participant 1, the effort involved learning a new process when they could fall back on habits. They stated, “I'm just used to typing things and printing things but debuggers is like a whole [other process] Again, I'm lazy. It’s a whole other process. Something that I’m not familiar with.”
Risk of Failure.
When faced with potential risks such as failure, participants described using strategies that they normally would not use if not forced to. For example, when asked whether they have had to use debuggers throughout their academic career, Participant 5 answered, “Not really. Unless it was required for an assignment, I didn't really use it.” Participant 1 gave a similar answer and highlighted the risk of affecting their grades as the primary factor for why they
“If I had to debug code that was already written and it was by my own free will, then no. Because I'm lazy. So, I'm not going to falter, I would just give up. But if it was say for a grade, right? Or, for some other external reason, and I had to do it [..] and I was stuck, then I would pull out the debugger.”
While participants used the strategy that their classes required, this did not mean they incorporated these strategies into their debugging habits. In fact, Participant 9 described how being forced to learn a strategy actually make them not want to use it outside of class, because they ended up not seeing the value after learning the new strategy:
“All right, so I think our first assignment [...] was literally jump right in Eclipse to use the debugger. [The Professor] told us the very first day to go into the debugger. I feel that just being an assignment kind of maybe made me not want to use the debugger. [...] Because I can see how the breakpoints can be useful: you stop and see what's going on at that point. But I think with print statements I can get the exact information I want.”
Participants also described other elements of their social environment, such as the time pressures and deadlines involved with completing their assignments, as reasons for using certain strategies. For example, Participant 10 described using debuggers during lab exams: “I did debuggers on some of [Professor Y’s] lab exams simply because I thought I had little time. And then I was like, maybe I could implement a breakpoint to see how that works.”
We also saw the effects of time pressures during our own study when participants remarked how the study setting felt so much like a lab exam that they felt compelled to not use strategies like searching online. For instance, Participant 7 remarked how they would use StackOverflow “normally, but here [they didn’t] know if it was first allowed to use
Long term value.
In addition to anticipating the costs of using certain strategies, participants also
described evaluating the longer-term benefits of using strategies as deciding factors for picking them in the first place.Participant 4 described learning and using a strategy only if they received a return on investment in future debugging sessions: “It’s almost like if it’s not worth learning then I’m not going to. I would rather learn something else worth learning or fix the problem outright if I’m never going to look at this ever again.” Participant 8 expressed the same
sentiments after learning to use a debugger in class: “I just threw out everything I learned. I’m not using this debugger ever in my life.”
Participant 5 described refraining themselves from using certain strategies, because they felt they could learn more: “For [Informatics & Computer Science] 31, 32 or whatever, I didn't really ask for help because […] I just felt I could probably learn more by solving it myself.”
Expected immediate value.
Other participants expressed expecting immediate value when using certain strategies as reasons they chose to use them. For example, Participant 9, as a visual learner, described how they expected value from sketching their code on paper: “Pointers are literally the words, but if you draw it out like with arrows, it helps me a lot to see they're actually pointing to
something.” As another example, Participant 11 described expecting to see a detailed view of their program’s internal state whenever they opened the IDE’s debugger:
“So in Java it's very convenient because it opens a separate menu and in that separate menu it goes to the code line-by-line. And also it shows the arrays. Each different object is a different window. So, it shows you in depth
Participant 2 discussed how they expect certain strategies to behave, such as how they help format the program output they produce and how considering this they prefer the one that produces neater output: “the print statement, [...] unless you format it [...] [in] a very neat way, you can dump data onto the screen and rather than formatting it’s definitely way better to just use the debugger where they give you the information.”
Power.
Participants also anticipated gaining power, the sense of control and freedom to
manipulate the code or environment, when using certain strategies. For instance, Participant 3 described possessing a sense of control when taking notes on paper: “Maybe it's just my
personal preference, but when I write the stuff out I feel I actually do feel I'm in more control.” When describing sketching on paper while debugging, Participant 9 also described similar feelings: “You're free to do anything on the paper. You can draw whatever. I don't really see how anything else can give you this freedom—just writing anywhere.”
Participants also described gaining a sense of control when using both print statements and debuggers. When asked why they preferred using print statements rather than debuggers, Participant 5 described having a sense of control: “I think there's probably more sense of control when you're using a print statement.” Similarly, Participant 9 favored print statements due to the sense of freedom and control that it afforded:
“Because [the] debugger lists all the variables, so you have to pick out exactly which one you want. [For example,] ‘Oh, I want to see this variable.’ But print statements is more concisely. It has exactly the information that I want. It’s more catered and well, you can custom say what you want.”
Section Summary
Participants described their experiences making enrichment decisions that were based on their prior experiences, their present environment, and their immediate and distant future. They would use their prior experience and present environment to determine how suitable certain strategies may be. These elements would factor into their perceptions of future risks and value associated with using a strategy, and their expectations ultimately lead to some decision.
Chapter Summary
This chapter explored how novice developers describe their decision-making process for picking debugging tactics by answering two questions: (1) What debugging tactics do they use? and (2) What activities are involved in selecting a debugging tactic to use?
In answering the first question, we identified three categories of debugging tactics. Participants would run their code to generate new sources of information in the form of program output or views that displayed internal state, such as the debugging views. Moreover, this strategy involved modifying code either through print statements or custom code to test certain program behaviors. The second type of debugging tactic involved seeking help from online resources as well as from other people. This strategy described how novice developers filter for information in both computing and social environments by formulating queries that can narrow down their searches. This strategy saw instances where strategies branched out from the IDE, which many debugging tactics derive from. Lastly, we found that novice developers would take notes on paper to improve the usefulness of the information they gather in other mediums, such as the IDE. Again, this strategy demonstrated how some strategies cross over to other environments.
For our second research question, our findings identified three activities that novice developers do to select debugging tactics. They leveraged past experiences, information they
decisions. When leveraging their past experience, they considered previous encounters with strategies and their impressions from those experiences to decide whether or not they should use them. Novice developers also took into account more recent events like their current progress on the debugging task, as well as aspects of their surrounding and computing
environments to motivate their decisions. Lastly, novice developers acted in anticipation of the future by deriving estimates for how valuable certain strategies may be or how risky, and then determining whether they should pursue them. Thus, the decision-making activities we identified in this chapter revolved around elements of the past, present, and future.
Chapter 5: Discussion and Implications
This chapter discusses how our results relate to previous studies describing what
debugging tactics novice programmers use and what factors may affect their selection. While we find that novice developers use a variety of tactics, all of which already detailed in the literature, we highlight the various environments in which novices execute them. Furthermore, we discuss the significance of understanding how novice programmers debug within different settings and environments and how they make decisions across time throughout this process. We conclude by describing the implications of our findings for educators and future research to better understand novice programmers’ debugging processes.