Unit Testing

Unit testing concentrates on testing the code from the inside, exercising the code logic. This activity is directly linked with coding and happens at the same time. Hence, the intention is to direct the testing search to those portions of the code that are most likely to contain faults. As each unit is constructed, it is tested to ensure that it (1) does everything that its specification claims and (2) does not do anything it should not do. Conducting these unit tests is particularly productive because the visibility into the code under test is at its maximum. The degree of visibility is related directly to the testability (the ease with which the code under test can be tested) (Clements and Northrop,2001).

In Unit Testing the main goal is to ensure this portion is working properly. A product comprises several units, which, at this point, should be tested individually. This activity enables to find, and then correct, errors at a fine grained level, which can reduce the error propagation.

Once you have the tests for a class up and running, the next step is to hook up with other methods

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

and with other services in order to examine the interactions among methods and classes inside a component, i.e. inside a unit.

Taking the SPL framework defined in (Clements and Northrop,2001), unit testing is per-formed in Core Asset development, as soon as the core assets are being developed and their quality must be assured before instantiated into derived products.

Unit testing will be performed in Product Development in cases when a new feature that not belong to the Core Asset base yet is to be implemented. In this case, Unit tests are performed in Product Development phase, but the flow of information, tasks and activities are the same we detail in this section. As our main intention is to provide a Core Asset Base which comprises the largest amount of assets to be further reused by diverse products, we advocate that such ’new’

features should be integrated to the Core Asset Base, and then instantiated in a product, even in case where specific features will work in only one product. The intention here is to have a common room that ease assets management, as previously mentioned in this Chapter.

The whole unit testing process workflow is here presented in a detailed way. Along with the workflow, the activities, artifacts and roles to be included into RiPLE unit testing are presented as well, in order to exploit the reuse potential given by the commonalities and variabilities concerns and avoid time wasting as well. It is worthwhile to mention that we do not introduce new terminologies for testing, but rather tailor existing ones (IEEE,1998), (McGregor,2001b) to our context.

Process Workflow

The Unit Testing process is composed of four activities: Planning, Design, Execution and Reporting. Figure4.6shows the process workflow. All RiPLE-TE test levels, both in CAD and PD use this same workfloe. Although they are presented as sequentially initiated, this is an iterative and incremental activity, with feedback connections enabling refinements. Next, each activity will be discussed in detail.

Unit Test Planning

Planning accomplishes the initial activity of the RiPLE-TE Unit testing process. In this phase, Test Manager and Test Architect define which will be covered by the unit test and to which extent. Moreover, the planning is responsible for resources estimate, coverage criteria, hardware and schedule, besides information about which features will or not be tested, which variation point and/or variants will be considered in that test cycle. The risks inherent in these strategic decisions should be also considered.

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

For every component a test plan is devised. Despite the high cost to implement and maintain this artifact, the main reason of having taken such a decision isto have an appropriate strategy to ensure traceability of the artifacts built, thus considering a long life cycle for the component.

Test planning should involve at least the following steps:

1. Analyze input artifacts - The first step concerns to analyze the input artifacts in order to define, taking the Master Plan as a guide, what will be tested along the cycle. If any other cycle has previously been executed, the reports should be also considered.

2. Define and prepare the test environment - This step focuses on tailoring what was prior defined in the Master Test Plan, regarding testing tools and other tools to support test activities, applying it to the context of unit testing. If any tool was not previously described in the Master Plan, it is highly recommendable to have it updated.

3. Select features to be tested in each cycle - Features and variation points to be tested in every cycle must be identified, the ones which will not be tested should also be pointed out. Moreover, critical parts, as defined in the risk analysis, prior conducted, should also be considered. This identification is based on requirements and use cases analysis and priorization performed during RiPLE-RE. The SPL risk document should be considered during this step.

4. Define the schedule - The schedule to perform the unit tests should be described to make stakeholders aware of timing. Timing previously alloted in the Master Plan should be respected.

5. Define testing techniques - Identify white-box and black-box techniques and/or strategies to be applied in unit testing.

6. Define strategy to test the integration of classes - Identify a strategy which can be suitable to test the integration of classes and methods. An strategy suitable for the RiPLE-TE context is the concept of object clusters. A cluster consists of related classes that may work together to support a required functionality for the complete system (Burnstein,2002). To integrate methods ans classes using the cluster approach the tester could select clusters of classes that work together to support simple functions as the first to be integrated. Then these are combined to form higher-level, or more complex, clusters that perform multiple related functions, until the system as a whole is assembled.

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

7. Define the coverage criteria - At this point, it is necessary to establish which coverage criteria will be applied in the project and the rates associated.

8. Define the test input domain - Determine the test input domain, in order to address the coverage criteria defined previously.

9. Define the acceptance (Pass/Fail rate) criteria - Specify the stop criteria to be used during the component unit test cycle (e.g. 98% passed).

10. Summarize the information in a test plan - All the information from the previous steps are then comprised in a test plan. A template for planning should be used is encouraged.

11. Review and approve the test plan - The test plan developer or manager schedules a review meeting with the major players, reviews the plan in details to ensure it is complete and workable, and obtains approval to proceed. These would encompass incorrect, incomplete, missing and inappropriate information.

As input to this activity, the SPL Architecture Plan must be visited as well as the Project Plan and the Component Source Code as well. The former assembles the architectural deci-sions on the variabilities and commonalities of the components to be tested, which can directly impacts on the strategies adopted in this activity. The second one will be useful to define the schedule for unit testing and establish the relationship among activities, coverage criteria and available resources. Finally, unit tests take place at the source code level, hence access to the code should also be helpful for the planning definitions. The Unit Test Plan is presented as the output of this activity.

Unit Test Assets Design

The second activity covers the design of test assets, mainly Test Cases, Test Suites, Test Pro-ceduresand Test Scripts. Test Architect, Test Designer and Tester work jointly in this activity.

When creating test cases, the inputs, steps and expected results should be considered. The relationship among test cases is identified to group related tests in test suites. It is interesting to devote effort in producing reusable assets at this pont, so that next turns can benefit from it. All assets should be recorded in a test repository for use in future releases.

The following steps have to be followed when designing test assets:

1. Create tests to evaluate the components methods - Based on the previous activity, a test case is generated for each method within the component in order to evaluate if it works

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

according to the specification. It is strongly recommended to build scripts in order to automatically execute the unit tests. Architecture information should be considered during this step, e.g. Component Class Diagrams are important, since it helps the tester to verify if the specification is in conformance with the component code.

2. Create tests to evaluate the classes integration - After creating test cases for each method, the integration among the component’s classes can be performed using an integration strategy, in order to evaluate this integration the test cases are created. The test cases will evaluate the component’s methods based on their specification (e.g. class diagrams), risks performed during planning step.

3. Group test cases in test suites - Group the tests created in the previous step according to a common objective, e.g. group a set of test or scripts in order to evaluate a specific feature.

Unit Test Execution

After producing the required test assets, the activity should be the execution of the test cases, activity commissioned to Testers. This activity is basically stated under the purpose of finding errors in the code.

Aiming at reducing the test execution effort, test automation is a useful strategy. It is important that the tests scripts development and maintainance costs should be lower than manual tests execution. Sometimes a change might impact on the test script as a whole and force that a new script should be developed from scratch. Whenever test automation scripts have been posed as necessary into the scenario in use, testers are requested to use them.

Figure 4.7 Unit Test execution steps

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

In order to give detailed information on the fine-grained tasks to be executed inside a component until to ensure it accomplishes what is required, we designed a sample illustration, Figure4.7, which presents the test execution steps to be performed inside a component.

As an example, we consider a Component, named α, which contains the classes 1, 2 and 3. Each of these classes contains n methods, as can be seen in the top right of the figure. For illustrative purposes, we are using the nonincremental testing approach, but the incremental approach could also be adopted. The first step (1) is to test every method, individually. Next (2) test the integration of the methods contained in the class. In both cases, if a method or class depends on a not coded yet, we encourage the use of test harness. Later (3) all the classes which compose the component must be integrated and tested. We must consider the acceptance criteria, which has been previously defined in Test Planning phase.

An important artifact generated along this activity is the CR, which associates defects found in test execution within the developed code.

For each error found by a tester, it should be analyzed and a CR is associated. The steps needed to find the defect are recorded. The CR is then assigned to the development team responsible for the functionality under test, and must be further analyzed, in order to fix the error. This artifact serves as a bridge between development and test teams. CRs management is handled by RiPLE-EM process. CR is one important means of measurement, as for measuring the productivity and effectiveness of a test team, as for measuring the quality of the development team.

Furthermore, the acceptance criteria established during the planning phase serves to indicate the moment to finish the test execution. After finishing the execution of the specified set of tests, the test result (in which the state of test case result changes according to the range: pass, fail, block or invalid, as can be seen in Figure4.8) should be reported in a form of a Partial Report.

In short, the steps to be followed in such a task are:

1. Perform tests for each method - The first step is to test each method, individually.

2. Perform tests of the integration of methods contained in a class - Secondly, test the integration of the methods contained in the class.

3. Perform tests of the integration of classes contained in a component - Later, all the classes which compose the component must be integrated and tested.

4. Associate a CR to errors found - As soon as an error is found, the tester must associate the error with a Change Request (CR).

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

Figure 4.8 State transition diagram of test case result

Unit Test Reporting

During test execution defects are found and reported using change requests, which will be further used when assembling the test report. This artifact has some objectives: first, the recorded information is important to calibrate the unit test process as a whole. After testing a component, it will provide information about the coverage criteria, common defects and their respective fixes, tested features, tested variants regarding to a variation point, process bottlenecks and gaps.

Last, it will provide information to perform another unit test cycle or maybe a regression test cycle, whenever the component is not completely tested at first time, in terms of acceptance criteria. Test Manager is responsible for conducting the reporting.

The reporting artifact is usually based on a template, that should be available in a repository so that every test team member has access to it. RiPLE-TE provides such a template document as well.

The steps to be followed during Test Reporting are:

1. Assemble information on the cycle execution - Information provided by the test execution cycle is to be included in this report, to further guide additional cycles, regarding to workforce, personnel allocation, defects found rate, and so on.

2. Record the issues found - Record the main issues found during the cycle execution.

3. Check for test completion and provide feedback - Whenever the test objective, in terms of coverage and acceptance criteria, defined during the planning phase is not achieved, a feedback should be performed in order to calibrate the execution for the next test cycle.

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT