Why Review Programs?

When developers regularly produce programs that run correctly the first time, ask them how they do it. You will find that they are proud of their work. They carefully review each program before they compile or test it. They don’t want anyone to see their product until they are sure it is something they are proud of. If you want qual-ity products, you must personally review and rework them until you are satisfied with their quality.

In my early PSP research, I wrote 72 Pascal and C++ programs. The relative times it took me to fix defects are shown in Table 9.1. On average, it took eight times longer to find and fix a Pascal defect during testing than it did in the code re-view. The average code review time to find and fix a defect in both Pascal and C++

was between one and two minutes. The average time to find and fix a defect in unit testing was longer for both languages and was even longer after unit testing. This fix time, however, was made up of a lot of fixes that took only a few minutes and a few that took much longer.

My data for these programs also show that the fix time is essentially the same for most defect types. The principal exception was typographical defects, which took about five times as long to find and fix in unit testing as in the code

review. This was true for both the Pascal and C++ programs. Figure 8.1 (see p. 138) shows similar data for a Xerox TSP team. This figure has an exponential scale, so although finding and fixing defects took only a few minutes in code reviews, they averaged over 20 hours each in system testing.

Until they learn how to do effective code reviews, many developers write pro-grams as fast as they can and immediately compile and test them. The compiler then leads them from one problem to the next. When the program finally compiles, they are so relieved that they try to run it. However, when programs have compile defects they almost certainly have test defects. If the first test doesn’t find defects, the developers will likely declare victory and ship. Then all of the program’s re-maining defects must still be found, either in system testing or by the user. That is when finding defects becomes truly expensive. Reviewing a program is like re-viewing a draft paper. Professional writers will tell you that the secret of good writ-ing is rewritwrit-ing. Writwrit-ing good programs is much the same. Few programmers can

9.2 Why Review Programs? 165

TABLE 9.1 RELATIVE DEFECT FIX TIMES

Products Relative Code Relative Unit Test Relative Post-Unit Review Fix Time Fix Time Test Fix Time

47 Pascal programs 1 8 16

25 C++ programs 1 12 60

0 5 10 15 20 25 30 35 40 45 50

1 2 3 4 5 6 7 8 9 10

Program Number Compile and Test Time as % of Development Time

Upper 1/4 Average Lower 1/4

FIGURE 9.1 COMPILE AND TEST TIME RANGE (810 DEVELOPERS)

write a perfect program on the first try. Many waste a lot of time compiling and testing a defect-prone first draft. Even after all this work, they still produce pro-grams with numerous patches and many remaining defects.

Many developers, when they start PSP training, spend more than one-third of their time compiling and testing. Figure 9.1 shows the average and the upper and lower quartiles for the percentage of development time a group of 810 experienced software developers spent writing ten PSP exercise programs. On Program 1, one-quarter of the developers spent over 43% of their time compiling and testing, and three-quarters of them spent at least 23% of their time compiling and testing. The average compile and test time for these developers was 34%. By the end of PSP training, they were doing design and code reviews, and their average compile and test percentage dropped from 34% to 12%, or only about one-third of the time they had spent at the beginning of the course.

As shown in Figure 9.2, the improvement in compile time was even more dramatic. At the beginning of the course, average compile time was about 13% of development, and it dropped to about 2% by the end of the course. This improve-ment not only saved the developers time, but also made their developimprove-ment process more predictable and produced higher-quality products. Developers often argue that the compiler finds defects faster than they could and that they spend the saved compile and test time doing the reviews. As shown in Figure 9.3, however, these same 810 developers spent about 8% of their time in reviews on Program 10, while their compiling time was cut by 11%. The reduced test time and improved prod-uct quality are a free bonus.

Program Number Compile Time as % of Development Time

Upper 1/4 Average Lower 1/4

0 2 4 6 8 10 12 14 16 18

1 2 3 4 5 6 7 8 9 10

FIGURE 9.2 COMPILE TIME RANGE (810 DEVELOPERS)

Although the correlation between compile and test defects is not very good for groups, Figure 9.4 shows that it is often quite good for individuals. Furthermore, Figure 9.5 shows that when developers find a lot of defects during compiling, they usually will have a lot of defects during testing. The dark bars in Figure 9.5 show

9.2 Why Review Programs? 167

Program Number Code Review Time as % of Development Time

Upper 1/4 Average Lower 1/4

0 2 4 6 8 10 12 14 16 18

1 2 3 4 5 6 7 8 9 10

FIGURE 9.3 CODE REVIEW TIME RANGE (810 DEVELOPERS)

Compile Defects

Test Defects

0 1 2 3 4 5 6 7 8 9

0 5 10 15 20

FIGURE 9.4 COMPILE VERSUS TEST DEFECTS, STUDENT 1 (r = 0.808)

the test defects for programs that had 30 or more compile defects per KLOC, and the light bars are for those with 10 defects or less per KLOC during compiling.

By examining your own PSP data, you will see that it is fastest and cheapest to find and fix problems before you design the wrong function or implement an er-roneous design. A little time spent reviewing a program can save much more than the review time during compiling and testing. More importantly, when everyone on a development team does thorough design and code reviews, integration and system testing time is cut by a factor of five to ten or more. That is why TSP teams typically deliver on or ahead of schedule.