model-checker [Visser et al., 2004] to generate all the instances that satisfy the given predicates. The implementation of generator methods can be quite expensive for complex data models; furthermore, UDITA cannot be adopted to reuse existing test inputs to test new data requirements.
8.9
Model-Based Slicing Approaches
In this dissertation, we introduced an algorithm that relies upon model slicing to improve the perfor- mance of constraints solving. Other approaches that rely upon model slicing to improve the perfor- mance of constraint solvers exists, but they focus mostly on the slicing of static models for satisfia- bility purposes (i.e. to verify whether it is possible to create instances of the class diagram without violating any constraint [Shaikh et al., 2010, Balaban and Maraee, 2013]). The generated slices typ- ically contain a subset of the elements belonging to the static models. The approach proposed in this dissertation instead performs slicing on an instance of a class diagram; furthermore, the pro- posed approach does not simply verify satisfiability but also supports the incremental augmentation of incomplete model instances.
Kato [Uzuncaova and Khurshid, 2008] is a tool that incrementally builds a solution for an Alloy model by grouping the predicates in two formulas (i.e. slices). A solution for the Alloy model is generated by solving the base slice first, and then by conjoining the solution with the predicates in the other slice. This approach has also been used to speed up the generation of test cases for testing product lines [Uzuncaova et al., 2010]. Kato deals with performance issues that depend on the presence of several constraints in a same Alloy model but it does not deal with the scalability problems related to the generation of large collections of elements. The approach proposed in this dissertation is thus complementary to Kato.
Chapter 9
Tool Suite Description
A tool suite was developed to automate the approaches that we propose in this dissertation.
The models and constraints developed for this project (as described in Chapter 3) using RSA are exported in the UML 2.2 (.uml) file format; UML 2.2 files are in the XML format and are supported by various UML tools.
We developed our tools in Java using the Eclipse Modelling Tools package. The implementations described below make use of the Eclipse Modelling Framework (EMF) and the following Eclipse Modelling Development Tools: UML2 and OCL [Eclipse Foundation, 2016]. The mentioned Eclipse development tools are used to load the UML 2.2 files and support our development efforts.
Section 9.1 describes the initial implementation of the Input Validation and Oracle tool used for the empirical study of Chapter 4. Sections 9.2 and 9.3 describe the core tools that were used for the rest of the empirical studies of this dissertation.
Additional tooling related to search-based test generation (Chapter 6) and test data generation by constraints solving (Chapter 7) was implemented for the research of this dissertation.
9.1
Initial Input Validation and Oracle Tool
Fig. 9.1 illustrates the architecture of the initial tool that was created to implement the test validation and oracle approach. Unlike the tool (which replaced this one) presented in Section 9.2, this initial version of the tool used hard-coded data parsers. As hard-coded parsers require frequent updates due to frequently evolving data requirements changes, parsers that make use of modelling annotations (i.e. UML stereotypes) to drive model instantiation were later developed.
The tool takes as an input the model and constraints of the system together with the input, config- uration and output files of the test case that needs to be validated. The tool then processes these inputs and generates logs that capture the results. The model and constraints can be in any format that can be imported into the EMF environment as an Ecore model.
At the end of the process, for each system instantiation (i.e. test case evaluation), Result Logs are generated to record each failing constraint and the execution time for each execution of the tool (i.e.
Chapter 9. Tool Suite Description Model Instance Result Logs Input Data Validator Oracle* Java Code Generator Java Classes Model Instan<ator File Parsers Data Loading Transmission Data System Configura<on System Output Logs*
Input/Output Structure
Data Model
Input/Output Constraints
Figure 9.1. Tool architecture for the automation of test validation and oracle. Initial version using hard-coded file parsers to instantiate the Model Instance.
*The system functions as an Input Data Validator only, if the System Output Logs are not
provided.
Transmission Data System Configura3on System Output Logs*
Model Instance Result Logs Input Data Validator Oracle* Model Instan3ator Generic File Parsers Data Loading Input/Output Structure Data Model Input/Output Constraints
Figure 9.2. Tool architecture for the automation of test validation and oracles.
*The system functions as an Input Data Validator only, if the System Output Logs are not
provided.
the times for model instantiation, test validation and oracle are recorded).
For this implementation, IBM RSA was used to create the UML class diagrams and the OCL constraints. The model and constraints were exported from RSA in the UML 2.2 (.uml) file format. These UML 2.2 files could then be imported into the EMF environment as an Ecore model. The Java Code Generatoris a component in the EMF that takes as input the Ecore model and produces Java Classesthat correspond to the model. The Java Classes directly support model instantiation and the OCL calls.