Increasing Model Quality through Model Validation

For increasing the quality of models, we present a model validation approach. Since tests are going to be generated from models, increasing the quality of the models also increases the quality of the generated tests. During the val- idation stage, models are check for consistency, correctness, and complete- ness. With consistency we mean that the models are consistent with respect to each other. For example, that information in one diagram does not con- tradict the information specified in another diagram. With correctness we mean well-formedness, i.e., that models conform to the UML standard. For example, whether the target item is a valid UML item, or whether a diagram contains valid elements within it. We notice that this check is left to the UML tool that we are using, since most modern UML tools will not allow users to break the UML meta-model. However, some tools might allow users to specify models that are not well-formed. With completeness, we mean

that the models are complete with respect to the transformation and test generation process. For example, we check that a model contains certain diagrams and that these diagrams contain the right elements with right val- ues and stereotypes. More information regarding model validation can be found in [35]. In general, the concepts are related to checking that our UML models conforms to certain guidelines, the UML standard, and that all the necessary data is present in the models. The validation step is a prerequisite for proceeding to test generation because with low quality models only low quality tests can be generated.

Figure 2.11: Example of a model validation.

The validation process allows us to check, for instance that all leaf re- quirements are traced to different model elements and that no requirement is overlooked. It also allows us to check that all the data fields are properly filled in. Figure 2.11 depicts the result of validating a requirement model

in MagicDraw, according to our specified rules. The Validation Results box shows all the OCL constraints that has been violated during the validation process. In this example, MagicDraw detected that requirement 6.1 has no id. All the requirements highlighted in red are not traced to any model elements. This means that those requirements will not be propagated fur- ther to any tests. Before proceeding to the next phase of the process, all violations need to be corrected and models re-validated.

2.2.3 Requirements Traceability Across the MATERA Pro- cess

Requirements traceability is essential to the MATERA process. The activ- ity is preformed throughout the whole testing phase in order to assure that all requirements have been tested. To accomplish this requirements need to be traceable both to and from tests. Gotel and Finkelstein define re- quirements traceability as ”the ability to follow the life of a requirement, in both forwards and backwards direction, i.e., from its origins, through its development and specification, to its subsequent deployment and use” [36]. One of the main purposes of tracing requirements to models is for an- alyzing which parts of the specification ”implement” different requirements. This will allow later on propagating these requirements from models to tests. Another reason for tracing requirements is that if a requirement changes, it is essential to know how this change is reflected in the models. To be able to verify what requirements that have been tested, we trace requirements from models to test runs and back to models as illustrated in Figure2.12.

As shown previously, we create requirements diagrams that describe how requirements are decomposed into a tree-like structure. In our research we employ a specific relationship between requirements and other model elements. The relationships between requirements and models elements are specified on several levels. Non-leaf requirements are linked to models, e.g. state machine models. An exceptional situation is in the case of top-level requirements which are linked to use cases in the use case diagram. The leaf requirements diagram are linked to other model elements to which they apply, e.g. transitions in a state machine or classes in a class diagram. This is done to ensure traceability of requirements to test cases. When the state machine is later executed, if a transitions is successfully executed we consider the requirement tested.

These links are useful for evaluating whether all the modeled require- ments have been reflected in the models. Further, by tracing requirements to model elements we can trace requirements, which were left uncovered during testing, back to system models again. This facilitates the process of identifying which of the requirements that have been left untested.


                      

      

    

     
        ! "    
   

Figure 2.12: Tracing requirements throughout the MBT process.

model elements, the models undergo a transformation phase. During this stage the UML models are transformed into a output format suitable for test generation. This is because UML models, as such, can not be used for test generation.

2.3

Validation

In this section we describe how each contribution has been validated. We present tool support and empirical evaluation of the presented research ap- plied in a case study.