Up to now, the Test Creation Framework (TCF) was treated as a black box that is applied by the test developer in order to derive tests for a specific service based on the “Structured
Service Customer Service Analyst
Service Developer Test Developer
Define informal
requirements requirementsStructure
Attend SQG meetings Discuss minor misunderstandings Attend project meetings Validate project <<include>> Maintain project status Define and
4.3 Framework Architecture and Components
Requirements” document compiled by the service analyst in cooperation with the service customer. The proposed TCF is based on an architecture that is presented in the following Figure 4.5.
Figure 4.5: Test Creation Framework architecture
The workflow of the methodology within the TCF architecture is started by the test developer who has access to the Test Framework User Terminal (TFUT) and to the Test Modules Environment (TME).
Test Framework User Terminal
With the help of the TFUT (see Figure 4.5), the test developer can actually plan the testing process. By using the terminal website, the test developer can manage all of the projects
Test Framework User Terminal (TFUT)
Test Modules Environment
(TME) Test Data Pool
(TDP) Test Modules Repository (TMR) Automatic Composition Engine (ACE) Test Configuration Unit (TCU) Test Case Derivation Unit (TCDU)
Test Suite Generator (TSG)
Test Code Generator (TCG) Test Suite Builder (TSB)
Test Execution Environment (TEE) Service Test Description Behaviour Models Abstract Test Suite Executable Test Suite Test Developer Test Report
he is currently working on. He also sees the current progress of the service developer and can compare the outcomes. Basically, the TFUT is the entrance point for the testing where he is also able to define instances of the so-called Service Test Description (STD) based on the “Structured Requirements” document. As soon as an instance is established, the test developer is able to start the testing process.
Test Modules Environment
The TME (see Figure 4.5) enables a test developer to create, modify or erase so-called reusable test modules through a graphical user interface. Furthermore, it allows the test developer to get access to the relating test data templates and data structures that are connected to the appropriate reusable test modules. To store the relevant data, the TME uses two different databases, the Test Modules Repository (TMR) (see Figure 4.5) as well as the Test Data Pool (TDP) (see Figure 4.5). The reusable test modules including further related metadata is stored within the TMR, whereas the test data templates that are related to the reusable test modules and instances of these are stored within the TDP. When a test developer defines new reusable test modules, it is important to define the metadata which is needed to specify the test module so that a process can automatically select the test module. In this approach, the reusable test modules are modelled by means of a Statecharts-based notation and cover typical service characteristics such as sequences of multimedia protocols like SIP or RTP (Real-Time Transport Protocol) and other important protocols, such as HTTP. The test modules usually define a protocol-specific behaviour of a certain use case, e.g. the sending of an instant message by using the SIP protocol, and cover both standard behaviour as well as possible alternative behaviour like
4.3 Framework Architecture and Components
timeouts. To sum it up, the test modules define the standard compliant behaviour of a certain use case.
Service Test Description
The STD (see Figure 4.5) is a novel type of specification or rather service description language that comprises elements of test specifications and service specifications. Furthermore, it contains architectural definitions describing the participating roles involved in the consumption of a value-added telecommunication service and their relationships as well as dynamic behavioural definitions specifying use-case related requirements. In the compilation phase, the test developer has to follow a guideline to define a STD for a service. The specification of the behaviour definitions will be done with an applied pi-calculus approach. Within the methodology, this is the only task being carried out by a human, the subsequent process performs automatically. One positive aspect of the STD besides others mentioned in section 5.3 is that the specified requirements within the STD can be directly mapped to the definitions in the “Structured Requirements” document.
Automatic Composition Engine
The Automatic Composition Engine (ACE) (see Figure 4.5) gets as input the STD after the test developer has defined it completely. The main task of the ACE is the generation of behaviour models, which are complete formal models based on Statecharts notation which describe the desired possible behaviour of the specified value-added telecommunication service. As a first step, the ACE parses the architectural definitions from the STD and forwards them to the Test Configuration Unit (TCU) (see Figure 4.5).
Afterwards, the ACE continues parsing the behavioural perspective of the STD and identifies the participating entities (or roles) (see section 5.2.2) within the specified requirements to select the suitable reusable test modules from the Test Modules Repository (TMR) via the service interface of the TME. Afterwards, the ACE connects to the Test Data Pool (TDP) in order to read the corresponding variables that are related to the selected reusable test modules. Then, new variables are instantiated and created for the instances of the reusable test modules and these are parameterised from the inputs of the STD. In the next step, the ACE starts with the composition of the reusable test module instances. Each reusable test module has interfaces which are linked to the existing states within the underlying Statecharts notation of a reusable test module. If two reusable test module instances have to be combined, the originating reusable test module instance and the destination reusable test module instance are connected with a new transition between them. The whole process is thoroughly described in section 6.4.
Test Configuration Unit
The TCU (see Figure 4.5) receives the architectural definitions from the ACE and thereupon extracts the relevant information for the Test Code Generator. Relevant information is for instance the SUT addressability and the participating test components. It is relevant for the TCU to identify which protocol is applied in order to deliver a test adapter configuration to the Test Suite Builder.
Test Case Derivation Unit
The behaviour model delivered from the ACE is the input for the Test Case Derivation Unit (TCDU) (see Figure 4.5). It contains a test case finder which uses an algorithm and
4.3 Framework Architecture and Components
follows selected coverage criteria to enable the derivation of abstract test cases from the behaviour model. Depending on the coverage criteria, the amount of derived test cases differs significantly. The output of the TCDU is an abstract test suite which includes abstract test cases for each behaviour model.
Test Suite Generator
The Test Suite Generator (TSG) (see Figure 4.5) creates a valid TTCN-3 test suite that can be imported into a TTCN-3 test execution environment. To achieve this, the abstract test cases have to be translated into TTCN-3 test cases by means of the Test Code Generator. The Test Suite Builder will enhance the TTCN-3 code with specific test modules and includes also the configuration of TTCN-3 codecs and adapters. Furthermore, the Test Suite Builder includes the TTCN-3 compilation as well as the Java compilation process in order to generate an Executable Test Suite (ETS).
Test Execution Environment
The final step of the framework’s underlying methodology takes place within the Test Execution Environment (TEE). The generated ETS can be executed due to the control part of the main TTCN-3 module. Of course, the TEE has to be installed into the service provider’s test environment in advance in order to be able to address the deployed service.
Test Report
The test report (see Figure 4.5) is the document the test developer and all the other members of the SQG receive after the test execution took place. Based on the results, the test developer has to maintain the project status according to the evaluation of the specified requirements.
The framework components will be further described and introduced in the following chapters. Chapter 5 discusses the structure of the Service Test Description (STD) as well as the underlying concepts. The concept of the test modules, the Automatic Composition Engine (ACE) algorithms to compose them based on the STD as well as the generation of the behaviour model will be described in chapter 6. The other test-specific aspects, such as test case derivation from the behaviour model, the transformation from abstract test cases to TTCN-3 test cases and the test execution itself against the SUT will be part of chapter 7.
4.4 Conclusion
This chapter has introduced a novel methodology for functional testing of value-added services considering current development life cycles in service provider environments.
First, several preconditions have been introduced as well as the new tasks of the roles participating in the service development and service testing process. Both the test developer and the service developer have to improve their social skills in order to get in contact with the service customer and support him during the development and testing phases. Finally, a new role has been defined, the service analyst, who is acting as the communication link between the service customer and the service and test developers (see section 4.1).
Furthermore, section 4.2 has proposed an overall methodology that allows a better involvement of the service customer within the development and testing process. Here, also the establishment of the novel Service Quality Group (SQG) with all the tasks for
4.4 Conclusion
the participating roles (service analyst, test developer and service developer) has been discussed.
Finally, section 4.3 has described one of the most relevant aspects of this research, the architecture of the proposed Test Creation Framework (TFC) with all its components.
The following chapter deals with the introduced STD and will include related work done in the field of specifying the functionality of services, especially telecommunication services.
5 Novel Service Test Specification and
Related Specifications
A well-defined specification of a value-added service is the basis from which functional tests can be derived. In literature, a number of service description languages and specification languages are presented and developed, mainly with the endeavour to automatically build the source code of the services and deploy them within service provider environments. To our knowledge, there is no existing service description for the purpose of functional testing that has been specifically defined for value-added communication services. To close this gap, this chapter will introduce a new kind of service description language, the Service Test Description. Firstly, section 5.1 will introduce existing service description languages and specification languages that can be applied to specify the functionality of value-added services. In section 5.2, the requirements on a service description language for the TCF described in chapter 4 will be documented and the Service Test Description will be presented. The existing specification and description languages and the new Service Test Description will be compared against each other in section 5.3.