It should be obvious that careful thought will find most errors without the debugger even going near the computer. One particular thought process is induction, where you move from the particulars of a situation to the whole. That is, start with the clues (the symptoms of the error and possibly the results of one or more test cases) and look for relationships among the clues. The induction process is illustrated in Figure 8.1.
FIGURE 8.1 The Inductive Debugging Process. 160 The Art of Software Testing
C08 08/17/2011 1:8:14 Page 161
The steps are as follows:
1. Locate the pertinent data. A major mistake debuggers make is failing to take account of all available data or symptoms about the problem. Therefore, the first step is the enumeration of all you know about what the program did correctly and what it did incorrectly—the symptoms that led you to believe there was an error. Additional valuable clues are provided by similar, but different, test cases that do not cause the symptoms to appear.
2. Organize the data. Remember that induction implies that you’re processing from the particulars to the general, so the second step is to structure the pertinent data to let you observe the patterns. Of particular importance is the search for contradictions, events such as the error occurs only when the customer has no outstanding balance in his or her margin account.
You can use a form such as the one shown in Figure 8.2 to struc- ture the available data. In the ‘‘what’’ boxes list the general symptoms; in the ‘‘where’’ boxes describe where the symptoms were observed; in the ‘‘when’’ boxes list anything you know about the times when the symptoms occurred; and in the ‘‘to what extent’’ boxes describe the scope and magnitude of the symptoms. Notice the ‘‘is’’ and ‘‘is not’’
FIGURE 8.2 A Method for Structuring the Clues.
columns: In them describe the contradictions that may eventually lead to a hypothesis about the error.
3. Devise a hypothesis. Next, study the relationships among the clues, and devise, using the patterns that might be visible in the structure of the clues, one or more hypotheses about the cause of the error. If you can’t devise a theory, more data are needed, perhaps from new test cases. If multiple theories seem possible, select the more probable one first. 4. Prove the hypothesis. A major mistake at this point, given the pres-
sures under which debugging usually is performed, is to skip this step and jump to conclusions to fix the problem. Resist this urge, for it is vital to prove the reasonableness of the hypothesis before you proceed. If you skip this step, you’ll probably succeed in correcting only the problem symptom, not the problem itself. Prove the hypoth- esis by comparing it to the original clues or data, making sure that this hypothesiscompletely explains the existence of the clues. If it does not, the hypothesis is invalid, the hypothesis is incomplete, or multiple errors are present.
5. Fix the problem. You can proceed with fixing the problem once you complete the previous steps. By taking the time to fully work through each step, you can feel confident that your fix will correct the bug. Remember though, that you still need to perform some type of re- gression testing to ensure your bug fix didn’t create problems in other program areas. As the application grows larger, so does the likelihood that your fix will cause problems elsewhere.
As a simple example, assume that an apparent error has been reported in the examination grading program described in Chapter 4. The apparent error is that the median grade seems incorrect in some, but not all, instances. In a particular test case, 51 students were graded. The mean score was correctly printed as 73.2, but the median printed was 26 instead of the expected value of 82. By examining the results of this test case and a few other test cases, the clues are organized as shown in Figure 8.3.
The next step is to derive a hypothesis about the error by looking for patterns and contradictions. One contradiction we see is that the error seems to occur only in test cases that use anodd number of students. This might be a coincidence, but it seems significant, since you compute a median differently for sets of odd and even numbers. There’s another strange pattern: In some test cases, the calculated median always is less 162 The Art of Software Testing
C08 08/17/2011 1:8:15 Page 163
than or equal to the number of students (26 51 and 1 1). One possible avenue at this point is to run the 51-student test case again, giving the stu- dents different grades from before to see how this affects the median calcu- lation. If we do so, the median is still 26, so the ‘‘to what extent! is not’’ box could be filled in with, ‘‘The median seems to be independent of the actual grades.’’ Although this result provides a valuable clue, we might have been able to surmise the error without it. From available data, the calculated median appears to equal half of the number of students, rounded up to the next integer. In other words, if you think of the grades as being stored in a sorted table, the program is printing the entry number of the middle student rather than his or her grade. Hence, we have a firm hypothesis about the precise nature of the error. Next, we prove the hypothesis by examining the code or by running a few extra test cases.