Object Classifier
This section evaluates the performance of the proposed Behaviour-based Object Classifier in outdoor surveillance videos. An outline of the experimental methodol- ogy adopted to evaluate the performance of the proposed behaviour patterns and the BFC is detailed. During the following section, the benefits and adaptability given by the inclusion of different degrees of freedom in the classification process is studied in the quantitative evaluation of the experimental results. Besides, in an effort to evaluate the performance of the proposed behaviour patterns, a comparison between the proposed Behaviour-based Object Classifier and a classifier relying on
3Individual behaviour features were trajectory dependent due to their calculation within a 2D-
projected space, omitting their dependence on the depth in the image within the object represen- tation vector.
appearance features is presented.
5.3.1
Quantitative Performance Evaluation
In this chapter, the Behaviour-based Object Classifier is proposed for automatic object classification for outdoor surveillance videos. The stream of spatio-temporal information conveyed by the Motion Analysis and Object Extraction Component is exhaustively analysed to generate a set of behavioural patterns which enable modelling semantic objects according to their behaviour as seen by humans. The framework includes the analysis of the moving objects’ temporal evolution to allevi- ate the external factor effects over the objects’ spatio-temporal information, object representation based on behaviour models and automatic object classification based on fuzzy logic. The use of fuzzy logic in the BFC presents an advantage against state of the art binary classifiers, due to the inclusion of the uncertainty of the classifier in the decision-making process. Different degrees of uncertainty were considered, pro- viding two implementations of the BFC using Type-1 and Type-2 Fuzzy Logic (refer to Section 5.2.3). Fuzzy classifiers provide not only a membership label for each detected moving object but also a membership degree/interval exhibiting the relia- bility on the membership label. Table 5.1 shows the classification results obtained considering uniquely the membership label procured by the BFC and omitting the uncertainty degree.
Table 5.1: Performance of the proposed BFC according to the membership labels uniquely
Behavioural Semantic True True False False
Fuzzy Classifier Class Positive Negative Positive Negative
(FC) (%) (%) (%) (%)
Type-1 BFC VEHICLE 36.63 93.62 6.38 63.36
PERSON 93.62 36.63 63.36 6.38
Type-2 BFC VEHICLE 79.40 51.06 48.94 20.60
PERSON 51.06 79.40 20.60 48.94
Type-1 and Type-2 Fuzzy Classifiers (T1FC and T2FC, respectively) represent the uncertainty of the classification with either a membership value (in the case of T1FC) or a membership interval (for T2FC). In Table 5.1, the flexibility provided by fuzzy logic is omitted to study the performance obtained by sharp binary classi- fiers in a behaviour-based object classifier. In order to evaluate the approximation provided by the different levels of uncertainty included in the BFCs, a retrieval sys- tem has been applied. Figure 5.6 reveals an improvement of the retrieval results when using T2FC. For the semantic concept Person, Figure 5.6 show on average 35% precision for the T2FC, which outperforms by 10% the results obtained by
Figure 5.6: Retrieval performance based on the membership degree, µdecision, ob-
tained from the BFC
the T1FC. Whilst, for the semantic concept Vehicle, T2FC achieves 100% precision under 30% recall, its precision decreases with the recall, obtaining a 45% precision at 50% recall. Moreover, T1FC achieves 100% precision under 10% recall, while its average performance for the semantic concept Vehicle is 55%. In conclusion, T2FC generally outperforms T1FC for both concepts.
The appearance of fuzzy logic was due to the need for expressing uncertainty, allowing a higher degree of freedom in decision-making processes. Consequently, fuzzy classifiers address more adaptive and robust techniques to represent real world scenarios with a higher accuracy, compared with the binary classifiers used in the state of the art. In the proposed approach, objects are represented by behaviour features. These attributes can be easily understood by humans, but lack meaning for an automatic classifier. The use of fuzzy logic in the BFC was motivated by the need for a flexible approach which enabled the modelling of human inference rules and the estimation of their mathematical equivalents. In Figure 5.7, the proposed Type-1 and Type-2 BFCs are evaluated against a binary classifier based on SVMs. As noted from the results, the classifiers based on fuzzy logic outperform the binary classifier by 40% for the semantic concept Vehicle and 20% for the semantic concept
Figure 5.7: Performance comparison between binary and fuzzy classifiers based on behaviour patterns
Person, proving a more adaptive approach.
5.3.2
Comparison between behaviour and appearance-based
object classifiers
Psychological studies demonstrated that the human inference procedure for clas- sification relies on motion and behaviour patterns rather than on appearance fea- tures. An exhaustive evaluation of appearance-based features was presented in [74]. In order to evaluate the performance of the behaviour features presented in Section 5.2.2, a comparison between the proposed Appearance-based Object Classifier (refer to Chapter 4) and the proposed BFC is shown in Figure 5.8. The presented results show an average 55% precision for the semantic concept Vehicle for the T1FC which outperforms on average the appearance-based object classifier by 10%. While, T2FC outperforms the appearance-based classifier and the T1FC by over 30% with 30% recall. For the semantic concept Person, T2FC exceeds the other approaches with an average of 20%.
Figure 5.8: Performance comparison between the proposed behavioural classifier and a classifier based on appearance features
semantic class within the ground truth, generating a less accurate model. On the other hand, the results obtained by the BFC for the semantic concept Vehicle are limited due to the speed-restricted road appearing in the surveillance video dataset, reducing the discriminative effect of the velocity feature.