The last thing to realize about program debugging is that, in addition to its value in removing an error from the program, it can have another valuable effect: It can tell us something about the nature of software errors, something we still know too little about. Information about the nature of software errors can provide valuable feedback in terms of improving future design, coding, and testing processes.
Every programmer and programming organization could improve immensely by performing a detailed analysis of the detected errors, or at least a subset of them. It is a difficult and time- consuming task, for it implies much more than a superficial grouping such as “x percent of the errors are logic-design errors,” or “x percent of the errors occur in IF statements.” A careful analysis might include the following studies:
• Where was the error made? This question is the most difficult one to answer, because it requires a backward search through the documentation and history of the project, but it also is the most valuable question. It requires that you pinpoint the original source and time of the error. For example, the original source of the error might be an ambiguous
statement in a specification, a correction to a prior error, or a misunderstanding of an enduser requirement.
• Who made the error? Wouldn’t it be useful to discover that 60 percent of the design errors were created by one of the 10 analysts, or that programmer X makes three times as many mistakes as the other programmers? (Not for the purposes of punishment but for the purposes of education.)
• What was done incorrectly? It is not sufficient to determine when and by whom each error was made; the missing link is a determination of exactly why the error occurred. Was it caused by someone’s inability to write clearly? Someone’s lack of education in the programming language? A typing mistake? An invalid assumption? A failure to consider valid input?
• How could the error have been prevented? What can be done differently in the next project to prevent this type of error? The answer to this question constitutes much of the valuable feedback or learning for which we are searching.
• Why wasn’t the error detected earlier? If the error is detected during a test phase, you should study why the error was not detected during earlier testing phases, code
inspections, and design reviews.
• How could the error have been detected earlier? The answer to this is another piece of valuable feedback. How can the review and testing processes be improved to find this type of error earlier in future projects? Providing that we are not analyzing an error found by an end user (that is, the error was found by a test case), we should realize that something valuable has happened: We have written a successful test case. Why was this test case successful? Can we learn something from it that will result in additional successful test cases, either for this program or for future programs?
Again, this analysis process is difficult, but the answers discovered can be invaluable in
improving subsequent programming efforts. It is alarming that the vast majority of programmers and programming organizations do not employ it.
Chapter 8: Extreme Testing
Overview
In the 1990s a new software development methodology termed Extreme Programming (XP) was born. A project manager named Kent Beck is credited with conceiving the lightweight, agile development process, first testing it while working on a project at Daimler-Chrysler in 1996. Although several other agile software development processes have since been created, XP is by far the most popular. In fact, numerous open-source tools exist to support it, which verifies XP’s popularity among developers and project managers.
XP was likely developed to support the adoption of programming languages such as Java, Visual Basic, and C#. These object-based languages allow developers to create large, complex applications much more quickly than with traditional languages such as C, C++, FORTRAN, or COBOL. Developing with these languages often requires building general-purpose libraries to support your efforts. Methods for common tasks such as printing, sorting, networking, and statistical analysis are not standard components. Languages such as C# and Java ship with full- featured application programming interfaces (APIs) that eliminate or reduce the need for custom library creation.
However, with the benefits of rapid application development languages come liabilities. Although developers were creating applications much more quickly, the quality was not guaranteed. If the application worked, it often failed to meet the program specification. The purpose of the XP development methodology is to create quality programs in short time frames. Classical software processes still work, but often take too much time, which equates to lost income in the competitive arena of software development.
The XP model relies heavily on unit and acceptance testing of modules. In general, you must run unit tests for every incremental code change, no matter how small, to ensure that the code base still meets its specification. In fact, testing is of such importance in XP that the process requires that you create the unit (module) and acceptance tests first, then create your code base. This form of testing is called, appropriately, Extreme Testing (XT).