Embedded Systems
4.5 Related Work
4.5.1 Property of a Signal
Property of a signal, called SigF in this thesis, has been introduced in hybrid Sequence Charts notation [GKS99] used for describing the behavior of hybrid systems. The language is based on Message Sequence Charts [ITU96, ITU99], including some concepts of timing diagrams [ABH+97]. Already there, the signal has been partitioned according to its characteristics and constraints put on it. This technique will be recalled in Chapter 6 to illustrate some of the de- veloped concepts.
[GW07] indicates that the descriptive approach to the signals reveals some advantages over the commonly-used constructive approach, especially in the context of SUT behavior evaluation. Here, an obvious link to the test evaluation in MiLEST exists.
Further on, the consortium developing the Testing and Test Control Notation (TTCN-3) for embedded systems [Tem08] incorporates the paradigm of SigF into the ongoing research on the test assessment functions.
Finally, the timing relations between signals classified in [GHS+07] contribute to the develop- ment of temporal expressions within the test specification proposed in MiLEST.
4.5.2 Test Patterns
Referring to the test patterns, [Bin99] describes object-oriented test strategy patterns. They
handle several strategies to derive test cases for testing object-oriented software. However, this definition cannot be adapted in this thesis as object orientation is out of its scope. [Din08] elaborates a set of methods and patterns to design and implement efficient performance tests. These are also only related to the test patterns presented in this thesis. In [Neu04] Real-Time
Communication patterns are used in the form of time relations among communication opera- tions. They describe real-time requirements related to delay, throughput, periodic events, and jitter. These patterns are applicable to the telecommunication systems mainly.
[Neu04] also presents a detailed survey through test patterns regarding a number of criteria. According to his classification the proposed patterns are categorized as functional in the context of the test goals, as test design patterns in the sense of the test development and as component level considering the scope of testing.
[TYZ+03] argue that developers often specify embedded systems using scenarios and a typical medium-size system has hundreds of thousands of scenarios. Each scenario requires at least one test case, which in turn requires individual development and debugging. Verification patterns
(VP) are proposed to address this problem. The VP approach classifies system scenarios into patterns. For each scenario pattern (SP), the test engineer can develop a template to test all the scenarios that belong to the same pattern. This means that the engineer can reuse the test tem- plates to verify a large number of system scenarios with minimum incremental cost and effort. Each scenario has preconditions (causes), postconditions (effects), and optional timing con- straints. A SP [TYZ+03] is defined as a specific temporal template or cause-and-effect relation representing a collection of requirements with similar structures. A VP [TYZ+03] is a prede- fined mechanism that can verify a group of behavioral requirements that describe similar sce- narios. A GUI-based specification tool to facilitate the scenario specification is available. In this work the focus is put on the test specification patterns, which mainly relate to the test behavior similarly to [TYZ+03]. However, both discrete and continuous signals are handled, while [TYZ+03] addresses only scenarios describing discrete behavior.
4.6 Summary
In this chapter the second set of the research questions given in the introduction to this thesis has been addressed. In particular, a new way for handling the discrete and continuous signals at
the same time, based on the SigFs, has been provided. By that, a first sketch of the test frame- work realizing this concept has been given, indicating the design decisions in terms of test gen-
eration and test evaluation.
The main technical intention of this chapter was to introduce a new way of signal description by application of the SigFs. These have been discussed in Section 4.1. Further on, based on the previous assumptions, Section 4.2 introduced the classified means for signal generation and signal detection. The challenges and limitations of the realization have been discussed.
In Section 4.3, test patterns have been investigated, with a particular emphasis on the hierarchi- cal architecture of the proposed test system. The MiLEST test patterns have been attached to this thesis as a table in Appendix B. These will be carefully reviewed in Chapter 5 and applied to a number of case studies in Chapter 6.
Furthermore, Section 4.4 elaborated on the proposed test specification process and its develop- ment phases starting from requirements analysis until the test execution. Finally, the related work on SigFs and test patterns has been described in Section 4.5.
The detailed discussion on the test development process artifacts will be continued in Chapter 5.
“I dream, I test my dreams against my beliefs, I dare to take risks, and I execute my vision to make those dreams come true.“
- Walt Disney The upcoming chapter is related to the previous one since the considerations on the new test paradigm introduced there, are continued here. In particular, a test framework is provided in
order to automate the creation of concrete test systems. This is possible by application of test patterns that are organized into a hierarchy on different abstraction levels.
The fundamental approach to the signal features (SigFs), their classification, generation, and detection mechanisms enable to synthesize the entire architecture of the test system. This is done in Section 5.1 in order to exploit the SigF for testing purpose. Different abstraction levels for both test specification (TSpec) and test data generation (TDGen) are provided. The full con- solidation of the architecture levels can be additionally found in Appendix C. Their realization is possible applying the Model-in-the-Loop for Embedded System Test (MiLEST) method pro- posed in this thesis. In the TSpec, also the test evaluation is modeled as Section 5.2 emphasizes. The advantages of designing the architecture as such are revealed especially at the validation function (VF), feature generation, and feature detection levels, where the real test stimuli crea- tion and test assessment of the SUT behavior takes place. Furthermore, the link to the TDGen is established by application of the automatic transformations, which in consequence, supports the automatic generation of the test signals. The details of these activities are given in Section 5.3. The obtained test patterns are classified and categorized. In contrast to the related work, the numerous patterns proposed herewith are very granular and represent different abstraction lev- els while specifying the tests. This gives the possibility to navigate through the test system eas- ily and understand its contents from several viewpoints immediately.
In Section 5.4, a description of signal variants generation is discussed, combination strategies for test case construction are reviewed and variants sequencing at different levels is considered. This contributes to the concept of automatic and systematic test data generation representing a comprehensive advantage over the existing solutions.
Section 5.5 introduces the concept of reactive testing. There, also the test control is investi- gated. The concept of online test data manipulation is introduced that contributes to test time reduction at the end. Afterwards, Section 5.6 underlines the progress on integration level test- ing. In Section 5.7, the execution of the resulting test model is considered. Also, the importance of the test report is shortly mentioned.
Finally, Section 5.8 gives insights into the related work in the context of the test evaluation process based on the SigF, other transformation approaches and the ongoing work of the author of this thesis towards some extensions affected by the paradigm presented here. The summary in Section 5.9 completes this chapter.