Learning Objectives:
After studying this chapter, you should be able to:
• Define different types of faults and how to classify them.
• Define the purpose of testing.
• Describe unit testing and integration testing and understand the differences between them.
• Describe several different testing strategies and understand their differences.
• Describe the purpose of test planning.
• Apply several techniques for determining when to stop testing.
Summary:
This chapter explores several aspects of testing programs. A distinction is made between conventional testing approaches and the cleanroom method. A variety of testing strategies are presented. The chapter also presents definitions and categories of software problems and discusses how orthogonal defect
classification can make data collection and analysis more effective. The difference between unit testing and integration testing is explained. The chapter also describes the need for a testing life -cycle and describes how automated test tools and techniques can be integrated into it.
Testing is not the first place where fault-finding occurs; requirements and design reviews help to ferret out problems early in development. But testing is focused on finding faults, and there are many ways to make testing efforts more efficient and effective. It is important to understand the difference between a fault (a problem in the requirements, design, code, documentation or test cases) and a failure (a problem in the func tioning of the system). Testing looks for faults, sometimes by forcing code to fail and then seeking the root cause. Unit testing is the development activity that exercises each component separately;
integration testing puts components together in an org anized way to help isolate faults as the combined components are tested together.
Testing is both an individual and a group activity. Once a component is
written, it can be inspected by some or all of the development team to look for faults that were not apparent to the person who wrote it. The research
literature clearly shows that inspections are very effective at finding faults early in the development process. But it is equally clear that other techniques find faults that inspections often miss. So it is important for team members to work with the team in an egoless way, using the many available methods, to find faults as early as possible during development.
The goal of testing is to find faults, not to prove correctness. Indeed, the absence of faults does not guarantee correctness. There are many manual and automated techniques to help find faults in code, as well as testing tools to show how much has been tested and when to stop testing.
Exercises:
1. Examine faults from code that you have written. For each fault, identify the type of fault (as in Section 8.1, Types of Faults) and classify the fault using a defect classification. Provide the details of the defect classification used. (You may use the IBM defect classification presented
in Table 8.1 or the one from HP illustrated in Figure 8.1.) Describe any difficulties encountered in classifying the faults.
2. Based on the faults identified in the previous question, which type of fault occurred most frequently? How might you change your software
development approach to eliminate or reduce the occurrence of this type of fault?
3. Describe the differences between unit and integration testing. Give the goals for each type of testing and describe when and how each should occur.
4. Describe the differences between object-oriented and traditional testing.
5. Choose a piece of code and write test cases for the code to satisfy the requirements of statement testing. Write the test cases for all -uses testing. Which testing strategy is stronger? Which strategy requi res more test cases?
Answer Guidelines:
1. Chapter 8 describes many different types of faults. The purpose of this exercise is to give you a better understanding of how these descriptions can be used to identify and understand faults in actual code. It wil l also help you to understand the difficulties in classifying defects.
Sometimes it is difficult to classify faults, especially when the defect classification is not orthogonal. The chapter presents two defect classifications. You may use one of th ese classifications or any other, reasonable classification. Be sure your answer clearly describes the defect classification you are using. When answering this question first decide which type of fault you have found. Then, determine if the fault is one of omission or commission. Finally, based on the fault type and your classification of omission or commission, use your defect
classification to classify the fault. Did you have difficulties in determining the fault type? Did any of your faults seem to fit in multiple categories?
2. Answers to this question will vary depending on your fault profile. Your fault profile can be used to identify areas of improvement for yourself.
Based on your profile, which type of fault had the highest frequency?
Would any of the techniques described in this chapter or previous chapters be useful in helping you to reduce the frequency of this type of fault?
Which type of fault occurs least frequently? Have you done anything in the past to prevent this type of fault from o ccurring?
3. The main purpose of unit testing is to make sure that the component is functioning properly. The component is tested in isolation to make sure that the inputs produce the expected outputs. The main purpose of
integration testing is to verify t hat the system components work together as specified by the design. Integration testing occurs after unit
testing. Sections 8.3 and 8.4 describe unit and integration testing in greater detail.
4. Section 8.5 addresses the difference between testing object-oriented systems and traditional systems. Most of the techniques used for traditional testing also apply to object -oriented systems. Object-oriented programs have special characteristics that need several
additional steps. Some of the characteristics that must be considered with OO programs that may not be included with traditional testing techniques are: missing objects, unnecessary classes, missing or unnecessary associations, or incorrect placement of associations or attributes. Test case adequacy must also be considered more carefully with OO systems. As Perry and Kaiser (1990) found, when a subclass is added or modified, the inherited methods from the ancestor superclasses must be retested. As noted by Graham (1996a), objects tend to be small
and low in complexity. However, the complexity often is pushed to the interfaces among components. This shift of complexity means that unit testing may be easier with OO systems, but integration testing must be more extensive.
5. Section 8.3 describes the different types of test strategies for test thoroughness. Statement testing and all -uses testing are two of the
options described. With statement testing, every statement is executed at least once in a test case. With all -uses testing, the test set includes at least one path from every definition to every use that can be reached by the definition. In general, all -uses is stronger and requires more test cases than statement testing.