In this section, we describe the principles of modelling the hyperellipsoidal one-class SVM proposed in [132] for multivariate data vectors. The quadric optimiza-tion problem of modeling the hyperellipsoidal SVM classifier has been converted to the linear optimization problem in [100] by fixing the center of mapped data vectors in the feature space at the origin. The geometry of hyperellipsoidal one-class SVM-based approach is shown in Figure 6.1. The general process of model-ing the hyperellipsoidal SVM classifier for multivariate data vectors is addressed below.
Assume that m data vectors {xi²<d, i = 1, . . . , m} of d variables in the input space are mapped into the feature space using some non-linear mapping func-tion φ. The hyperellipsoidal SVM aims at enclosing a majority of mapped data vectors φ(xi) in the feature space by fitting a hyperellipsoid centered at the ori-gin with a minimum effective radius R. Thus, the optimization problem in this hyperellipsoidal SVM classifier is represented as:
R²<,ξ²<minm R2+υm1 Xm i=1
ξi (6.1)
Origin
Figure 6.1: Geometry of the hyper-ellipsoidal formulation of one-class SVM [100]
subject to : φ(xi)Σ−1φ(xi)T ≤ R2+ ξi, ξi≥ 0, i = 1, 2, . . . m where υ ² (0, 1) is a parameter that controls the fraction of mapped data vectors that can be outliers, and slack variables {ξi: i = 1, 2, . . . m} are denoted to allow some of mapped data vectors to lay outside the hyperellipsoid. Σ−1 is the inverse of the covariance matrix Σ of mapped data vectors, which is computed as follows:
Σ = 1 Using Mercer Kernels [128], the inner products of mapped data vectors in the feature space can be computed in the input data space without needing any knowledge about the non-linear function φ. Let K²<m×mbe the kernel matrix of the original data vectors. Similarly, mapped data vectors can be centered in the feature space by subtracting the mean. Then the centered kernel matrix Kccan be obtained in terms of the kernel matrix K using Kc= K − 1mK − K1m+ 1mK1m, where 1mis the m × m matrix with all its values equal to m1.
The eigen structures of Kc is denoted by Kc= AΩAT, where Ω is a diagonal matrix with positive eigenvalues as the diagonal elements, and A is the eigenvec-tor matrix corresponding to the positive eigenvalues [40]. Hence the covariance matrix Σ can be denoted as Σ = (Ω−12Aφ(x)T)(mΩ)(Ω−12Aφ(x)T))T, where X is the data vectors in feature space. By calculating the pseudo inverse Σ+, we
can approximate Σ−1 as Σ−1 = Σ+ = mXTAΩ−2ATX [132]. Consequently, Equation 6.1 will become as follows:
R²<,ξ²<minm R2+υm1 where Kci is the ith column of the kernel matrix Kc. Using similar Lagrange function and deviations as explained in Chapter 5, finally the dual formulation of hyper-ellipsoidal SVM will become a linear optimization problem represented as:
α²<minm −
The data vectors with αi = 0 will fall inside the hyperellipsoid and will be considered as normal. The data vectors with 0 < αi < υm1 will reside on the surface of the hyperellipsoid. Their distances to the hyperellipsoidal center in-dicate the minimum effective radius R, which can be obtained by calculating R2= k√
mΩ−1ATKcik2 for any margin support vectors. Those data vectors with α = υm1 whose distances to the origin are larger than R of the hyperellipsoid are considered as outliers.
6.3.1 Hyper-Ellipsoid SVM VS. Hyper-Sphere SVM
We have described hyper-ellipsoid SVM and hyper-sphere SVM in this chapter and Chapter 5, respectively. Both hyper-ellipsoid SVM and hyper-sphere SVM are used to model the normal behavior of given data vectors. A significant dif-ference between the two SVMs is that they use different distance measures to determine the similarity of data vectors and further model the normal behavior of data vectors. More specifically, hyper-sphere SVM uses Euclidean distance (ED) while hyper-ellipsoid SVM uses Mahalanobis distance (MD). The two dis-tance measures are both commonly used to measure the similarity between any two data vectors [118]. Euclidean distance does not consider the correlation be-tween attributes and calculates the distance in terms of individual attribute.
On the contrary, Mahalanobis distance considers the correlation between at-tributes and calculates the distance by combining all atat-tributes together. This correlation between attributes can be represented by covariance matrix, where variance of a variable itself and covariance between any two variables are in-cluded. Formally, given multivariate data vectors x = (x1, x2, x3, ..., xN) with mean µ = (µ1, µ2, µ3, ..., µN) and covariance matrix Σ, the Mahalanobis distance of these data vectors is defined as:
M D(x) = q
(x − µ)Σ−1(x − µ)T (6.5)
If the covariance matrix Σ is the identity matrix I, where all diagonal elements are set to 1, Mahalanobis distance reduces to the Euclidean distance for two data vectors x and y and is represented as:
EM (x, y) = vu utXN
i=1
(xi− yi)2 (6.6)
Compared to Euclidean distance, Mahalanobis distance, which takes into ac-count both distance from the center of mass and the direction, has a better understanding of multivariate data structure as Euclidean distance is blind to attribute correlation and assumes all data vectors have equal distance from the center of mass. Moreover, Mahalanobis distance is scale-invariant meaning that it is independent on the scale of data attributes, while Euclidean distance is ex-tremely sensitive to the scale of data attributes. However, the computational and memory complexity of Mahalanobis distance is much higher than Euclidean distance due to computation of covariance matrix. Euclidean distance is simple to calculate. Table 6.1 clearly shows the comparison between the two distance measures.
Consequently, using Euclidean distance for as similarity measure would gen-erate a sphere at 2-D data space, where data vectors are equally distributed around the center of mass. Using Mahalanobis distance as similarity measure would generate an ellipse at 2-D data space, where data vectors are distributed in directional linear trend [45], which indicates the correlation between variables.
The two different shapes actually define the normal behavior of data vectors.
We here introduce an example to represent the results of outliers using the two shapes. Figure 6.2 illustrate the normal behaviors of data vectors modelled by hyper-ellipsoid SVM and hyper-sphere SVM using corresponding distance mea-sures, respectively. As seen from Figure 6.2, outliers detected by the sphere may not be considered as outliers by the ellipse (point B); whereas, data vectors that are not declared as outliers may be considered as outliers by the ellipse (point A).
Classifiers Distance measure Characteristics Shape
Hyper-ellipsoid Mahalanobis - Considers attribute correlation Ellipse
SVM distance - Scale-invariant
- High complexity
Hyper-sphere Euclidean - Ignores attribute correlation Sphere
SVM distance - Scale-sensitive
- Low complexity
Table 6.1: Comparison between hyper-ellipsoid SVM and hyper-sphere SVM
Feature 1
Feature2
Origin
Point A
Point B
Figure 6.2: Similarity measure between Mahalanobis distance and Euclidean dis-tance. Using Mahalanobis distance generates an ellipse while using Euclidean distance generates a sphere at 2-D space.
Resource-Constraint WSNs
Therefore, using an appropriate shape and its corresponding distance measure to model normal behavior of data vectors is significantly important for accurate out-lier detection. The choice between Euclidean distance and Mahalanobis distance depends on data characteristics and requirements of applications.
In this chapter, we use hyper-ellipsoid SVM to model the normal behavior of sensor data, which is based on the fact that sensor data are often correlated as also shown by experimental results in Chapter 3. We would further reduce the computational complexity of modelling hyper-ellipsoid SVM for resource-constraint WSNs.