Integration Testing

Unit testing is an essential quality control activity, as showed in the previous section, but we must deal with testing in a situation where different units of works are combined into a workflow.

Once we have tests for a unit up and running, the next step is to hook up with other units and with other services. Examining the interaction between components, possibly when they run in their target environment, is the stuff of integration testing.

As an example, let A and B be two components, SA and SB be their respective suites of units tests, TAB be the test case from SA that cause A to call B and let TBSA be the set of test cases from SB which cause A to pass through a path in which B is called. The integration testing is the junction of TAB and TBSA.

Hence, after verifying individually the units, these need to be integrated to compose modules, sub-systems, SPL reference architecture and futher instantiate it to assemble specific product architectures. The test in which units are joined is known as Integration Test.

Integration Testing in CAD aims to test the interaction about the SPL common components and the reference architecture as well. As input of this testing level, we should consider the following assets: unit tested components, feature dependency diagram and architectural views (behavioral and structural).

Process Workflow

In the context of the RiPLE-TE, integration testing assembles four activities, like in unit testing process: Planning, Design, Execution and Reporting. Each is following detailed.

Integration Test Planning

Planning is the first activity to be performed. Test Manager and Test Architect define which will be covered by the integration test and to which extent. Following, the steps are described.

1. Analyze input artifacts - The first step concerns to analyze every input artifact in order to define, using the Master Plan as a guide, what will be tested along the cycle. If any other cycle has previously been executed, the reports should be also considered.

2. Define and prepare the test environment - This step focuses on tailoring what was prior defined in the Master Test Plan, regarding testing tools and other tools to support test activities, applying it to the context of integration testing. If, e.g. any tool is not described in the Master Plan, it must be updated.

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

3. Define White-box and Black-box Techniques - Define white-box and black-box techniques to be used in the project.

4. Define strategies for test the integration of components and modules - Identify a strategy to test integration of components. The concept of object clusters can also be applied in the RiPLE-TE integration testing. In this context, a priorization strategy should be established in order to analyze the relevance of the components which implement the most common or complex variation points.

5. Define the coverage criteria - Identify features and variation points to be and not to be tested, as well as identify critical paths and relationships. The coverage criteria define how much functionality of the application code expanding the application architecture will be covered with test suites. The criteria can be useful when the test architect, when looking for the structural view and the dependency diagram, can capture some critical components and interactions. By looking the behavioral view he can capture software functionalities that he is interested to test.

6. Define the test input domain - Determine the test input domain, in order to address the coverage criteria defined previously.

7. Define the Pass/Fail rate criteria - Specify the stop criteria to be used during the integration test cycle.

8. Summarize the information in a test plan - All the information from the previous steps are thus comprised in a test plan. A template for planning should be used.

9. Review and approve the test plan - The test plan developers should schedule a review meeting with the major players to review the plan in detail to ensure it is complete and workable, and obtains approval to proceed. These would evaluate issues such as incorrect, incomplete, missing and inappropriate information. The plan will only be ready for use after the approval granted.

Integration Test Assets Design

After planning, assets design takes place. Test Architect, Test Designer and Tester are commis-sioned to this activity. The following steps are the object of this integration activity.

1. Create test scenarios - Based upon the interactions among components, as documented through the architecture scenarios available, integration test scenarios that have to be

4.4. RIPLE-TE IN CORE ASSET DEVELOPMENT

executed to test a specific feature should be created. These include scenario outcomes (the expected results). Indeed, in some situations, not all test scenarios may be documented in the same detail, as a template would require.

2. Create tests to evaluate interactions among components - Based upon the test scenarios, test cases are generated for critical paths and interactions among components in order to evaluate if it works according to the specification. Architecture information should be considered during this step. For example, Architectural Views (component class diagrams, modules class diagrams, sequence diagrams, and so on) are important, since they aid testers to verify the specification against code. Whereas the behavioral view provides functional informations regarding the reference architecture, the structural view provides structural information.

3. Create tests to evaluate interactions among modules - After creating test cases for the critical module’s interactions, the integration among modules is performed using an integration strategy. In order to evaluate the integration, test cases are created. Test cases should focus on evaluating interactions among modules based on their specification (e.g.

architectural views).

4. Create test suites - Group the tests created in the previous step according to a common objective. E.g: Group a set of test or scripts in order to evaluate a specific feature.

5. Verify test coverage - Verify if the test cases created are covering the desired paths.

Integration Test Execution

Next, Testers are responsible for executing the tests previously created. Some are the steps included in this activity. These are:

1. Perform tests for critical component’s interactions - The first step is to test critical component’s interactions, individually.

2. Perform tests of the integration of modules modules - Secondly, test the integration of the methods contained in the reference architecture.

3. Associate a CR to errors found - Whenever an error is found, the tester must associate the error with a CR.

This activity can be conducted both manually and automatically, following what is described in the test plan, which meets major organizational needs.