Features Classification

This thesis is driven by the practicability factor. Thus, the classification of SigF is done follow- ing the realization algorithms, instead of any other theoretical approach.

This trend is motivated by the facts that MiLEST is based on the already existing modeling platform and its implications contribute to the reasoning about features. Moreover, a principal idea of this work is to show the feasibility of the proposed solution relating to the running case studies. Thus, the implementation behind the conceptual reasoning is in the primary focus. The fundamental task of signal processing, in the context of the approach proposed in this work, is to include the concept of SigF. Hence, the core problems of signal generation and sig-

nal evaluation are limited to the generation of an appropriate SigF or a combination of SigFs

over a predefined signal, on the one hand; and evaluation of an extracted SigF from a signal, on the other hand. Therefore, the activities of performing those practices are sometimes denomi- nated as feature generation and feature extraction, respectively. Feature extraction is a mecha-

nism for reducing the information about signal evaluation. This enables the test assessment be abstracted from the large sequences of values that signals represent. Feature extraction is also

called feature detection in this thesis.

Generation of a feature characterizing a signal translates to the generation of a specific signal, which contains the particular properties. The concept of generating the signals relates to the mechanisms which serve for their extraction, and thus evaluation. The features extraction per- spective is used for their classification. In fact, SigFs could be categorized applying the genera- tion viewpoint too, but it is of more value to use the other perspective. This kind of practice simplifies the process of understanding the entire test specification. Moreover, it is motivated by the fact, that the specification and evaluation part, including feature extraction, must be de- signed by an engineer, whereas the signal generation part is done fully automatically based on the test specification model. Thus, the starting point is to get familiar with the mechanisms of feature extraction, in fact. An additional classification would cause only an abstract overhead for the end-user.

Nevertheless, before SigFs will be categorized in detail, a brief overview on the scheme of fea- ture generation will be given. Firstly, a default signal shape is defined for every SigF (cf. Figure 4.3). Then, the range of permitted values for the signal is defined. Further on, a minimal dura- tion time of the feature is provided, in case needed. Otherwise, a default duration time is set. Finally, feature specifics are introduced in terms of the so-called generation information. For

example, a step generation includes a size of the step as shown in Figure 4.3, whereas an in- crease generation includes the shape of the increase, a slope, initial and final values. Additional

parameters that need to be taken into account while feature generation relate to the evaluation

mechanism for a particular feature. They must be set following the values of the same parame- ters that have been applied in the extraction part. A simple example is a step, for which the du- ration of constant signal appearing before the step, must be set. Otherwise, the feature detection mechanism could not work properly. Then, generating the step, the duration of the generated constant signal, must be set on the minimal value specified within the extraction so as to be de- tectable at all. kT generated signals: x(kT) y(kT) y(kT) duration perm itt ed va lu e ra nge

signal as a sequence of similar shapes (increase, decrease)

shaped signal signal of any shape

Generation information: step size Parameter:

constant duration

Figure 4.3: Signal-Features Generation – a few instances.

To sum up, a generic pattern for signal generation is always the same – a feature is generated

over a selected signal and the parameters are adjusted according to a predefined algorithm (cf. Figure 4.4); however, some feature specifics must be included for an actual generation of every single SigF. The details concerning the abstract considerations on that subject and their realiza- tion in MiLEST are described in the next section.

Signal Generation Feature Generation Parameters Sweep Generation Information

Feature Specific Parameters

signal x(kT) signal y(kT)

Figure 4.4: Signal-Features Generation – a Generic Pattern.

A similar approach is used for the signal evaluation (cf. Figure 4.5). Firstly, a signal is prepared

is extracted to be finally compared with the reference value. A verdict is set based on the ap- plied arbitration mechanism. The details w.r.t. extraction of the concrete SigFs are given in Sec- tion 4.2, whereas the patterns classification leading to the test architecture and the arbitration mechanism are elaborated in the upcoming chapter.

The time step size employed in the process of signal-features extraction is a critical factor in establishing its success. The choice of the time step size is dependent on the different rates of

response that the system exhibits. If it is chosen too small, it may result in a lack of sensitivity to changes: too large – it may produce incorrect inferences. Decreasing the time step may help in differentiating between discontinuities, abrupt changes, and continuous effects. On the other hand, if the time step is too small when applied to a variable with a relatively slowly decreasing slope, it appears that the signal does not change for a period of time; therefore, it is reported to be normal or to have reached steady state. In reality it is decreasing, and reporting it as normal may result in premature elimination of true faults [Mos97].

InOut Bus pass (1) fail (0) Signal Evaluation Signal Preprocessing Feature Extraction Comparison & Arbitration Ref

Figure 4.5: Signal-Features Evaluation – a Generic Pattern.

The concept of feature extraction and features classification were already given in the previous work of the author [ZSM06]. Then, the types of features discussed in this thesis and the sub- stantial parts of the evaluation implementation are directly adopted from [MP07, ZMS07a]. Extracting SigF from a signal can be generally seen as a transformation of SUT signals to so- called feature signals (not to be confused with signal feature, also called SigF or simply feature in the following). The concrete values of an extracted feature signal represent the considered

SigF at every time step. The feature signal is then compared with the reference data according

to a specific, SigF related algorithm.

The scope of the online evaluation is reduced to all the past time steps until the actual one. Many features are not immediately identifiable, though. Taking an example of a maximum, it can only be detected at least one time step after it takes place. Some features are only identifi- able with a delay, that might be known in advance (determinate) or not (indeterminate). This phenomenon is revealed in the naming convention for SigFs applied in this thesis.

The feature extraction realization determines the two aspects according to which the features can be classified. In Figure 4.6, one example for every combination is drawn, including the ac- tual signal and the feature signal (i.e., result of the feature extraction). For triggered features an

Vertically, the classification in Figure 4.6 addresses the SigF identification trigger. Time- independent (also called non-triggered) features are identifiable at every time step, while trig- gered features are only available at certain time steps. In Figure 4.6, the valid evaluation time steps of the triggered features are colored in light green.

Horizontally, Figure 4.6 presents the identification delay, differentiating between no delay (immediately identifiable), determinate delay, and indeterminate delay. Immediately identifi- able features are a special case of the features identifiable with determinate delay, but their de- lay equals zero. The signal value and the searched time step of a given signal are the immedi- ately identifiable features in Figure 4.6, cases – a and b. In the latter, the trigger signal activates the comparison mechanism, whereas, the feature signal represents the simulation time. If a ver-

dict for this check is being set, actually only three time steps deliver a verdict, for every trigger rise. Every conceivable causal feature can be classified under this aspect, i.e., all causal filter types, moving transforms, slope checks, cumulated values, etc.

When the identification of SigF occurs with a determinate delay (cf. Figure 4.6, cases c and d), the feature signal is delayed too. It reports about features in the past that could not be identified

immediately. A prominent example is detecting a local maximum, for which a constant delay is necessary. The size of the delay varies depending on the maximum detection algorithm used. When the delay is constant and known, the time step when a certain feature value was observed can be determined. However, the signal evaluation is retarded. This fact is particularly impor- tant when considering relations between features in the upcoming sections. Other features iden- tifiable with determinate delay are impulse detection algorithms or non-causal filters.

The features that cannot be identified immediately or with a determinate delay after the actual observation are exemplified in Figure 4.6f. There the rise time of a step response of a control loop is extracted. This feature is clearly triggered and the delay is indeterminate because it de- pends on the time when the actual loop will respond. Assuming that the step time is the obser- vation starting point, the delay is then computed as the difference between the trigger rise and the observation time. This situation is indicated by the reset signal, in Figure 4.6f – the step time). The actual rise time (i.e., feature signal in this context) must be extracted not later than

when the feature is triggered. In the figure, the rise time is available very early, but the test sys- tem gets this information when the feature is triggered. Other triggered features identifiable with indeterminate delay are, e.g., any other step response characteristic values, a system re- sponse delay, or the pattern complete step. Generally, most features based on the detection of two or more asynchronous events are of this type.

Finally, Figure 4.6e presents the maximal delay to date. This feature measures the delay be- tween the actual SUT signal and a reference signal when the reference outputs a rising edge. Then, it compares the gathered value with the highest delay to date as soon as possible. When exactly this will happen is unknown in advance, though. Finally, the highest value is saved for the next simulation step. This feature is defined for every time step, although value changes are triggered. As expected, the feature signal is a stair step signal that can only increase. An im- plementation of this kind of feature extraction is not considered in this work since only a few features of this type could be identified so far and all of them were either describable using the formalism for triggered features or were of low practical interest.

a) Signal value

x

b) Time when signal = x

c) Detect maximum

d) Signal value when maximum

f) Step response rise time Ti m e- in dep en de nt T ri gge re d

Immediately identifiable Identifiable with determinate delay

Identifiable with indeterminate delay

kT kT kT kT kT kT kT kT kT kT kT kT kT kT signal feature signal trigger feature signal feature reset signal feature trigger signal trigger feature reference feature signal e) Maximal delay to date kT kT kT

Figure 4.6: Signal-Features Classification based on the Feature Identification Mechanism. The classification of different feature types given in Figure 4.6 is comprehensive as far as the output signals of a casual system are evaluated. Using the presented mechanisms, SigFs that are observable in the past up to the current time step, can be identified and assessed.

Under these circumstances, the classification of feature types is completed. Though, the in- stances representing those types may lead to inconsistencies, as Gödel’s theorem [Fra05] would imply. Indeed, the different types of SigFs may often be implemented using different mecha- nisms depending on the current needs of the test system (cf. Figure 5.10).

The realization of the test evaluation enables to run it online, i.e., during the execution of the SUT. This implies that the feature extraction algorithm is run cyclically in SL. Hence, verdicts are computed at every time step on the fly.

In the next three sections, the three types of SigFs are explained in detail. The scheme of de- scription is always the same. Firstly, the definitions are given. Then, a few generic examples are

provided, on the basis of which feature generation principles follow. They are listed in tables. Furthermore, the implementation examples are described in depth. In particular, algorithms for feature extraction are reviewed and the mechanisms for feature generation are discussed. The separation into different feature types is motivated mainly by the fact that several features are not available at every time step and not all features can be extracted causally. For the time steps when the feature is not available, a none verdict is set.