Interaction Model

Currently, the most successful way to explore the synchronization phenomena is Kuramoto Model [94, 95] (cf. Section 3.2). However, in order to introduce the Kuramoto model into subspace clustering, it is reconsidered in a different way.

1. Local Interaction Fashion. To exploit the hidden clusters or pat- terns in arbitrarily oriented subspaces, the local structure of data should be investigated. Therefore, it focuses on the dynamics of objects in a local way.

2. Weighted Interaction. In high dimensional space, the correlations in the dimensions are often specific to data locality, which means some objects are correlated with respect to a given set of dimensions and others are correlated with different dimensions. Thus, the coupling strengths of interactions of objects in relevant or irrelevant dimensions should be considered with different weights.

In the following, the interaction model will be reformulated based on the above two criteria.

6.3 The Algorithm ORSC 101 P Covariance Σx ε P ε Mahalanobis distance Euclidian distance ε

Figure 6.2: -Range search with different distance functions. Local Interaction

To formalize the local interaction for each object, the intuitive way is to consider its ε-neighborhood according to Equation 4.2.

However, suchε-neighborhood search can not fit the goal well since it does not consider the local data distribution. To look for objects which are close in the local subspace cluster structure, therefore, the Mahalanobis distance instead of Euclidean distance is used to determine similar objects.

Definition 6.1 (ε-Neighborhood with mahalanobis distance)

Given a ε ∈ R and x ∈ D, the ε-neighborhood of an object x with Maha- lanobis distance, Nm(x), is defined as:

N_εm(x) = {y∈ D|p(y−x)·Σ−1

x ·(y−x)T} (6.1)

where Σx is the covariance matrix of ε-neighborhood of x. Since the Ma-

halanobis distance considers the local data distribution, it can better search similar objects considering the local cluster structure and is also less sensitive to noise. Fig. 6.2 illustrates the similar objects determination of an object

According to Definition 6.1, the Kuramoto model is extended in a local fashion, where each object interacts with its ε-Neighborhood with maha- lanobis distance during time revolution. Moreover, since without additional attributes of each object, all objects are assumed to be the same frequency

ω, which well fits the condition of Kuramoto model. Here, each dimension of an object is viewed as a phase oscillator and its original value represents the initial phase.

Formally, let x ∈ Rd _{be an object in the data set D and x}

i be the i-th

dimension of the data object x. N_εm(x) is the ε-neighborhood of object x. According to Eq.(3.1), the dynamics of each dimension xi of the object x

with a local interaction is further written as:

dxi dt =ω+ K |Nm ε (x)| X y∈Nm ε (x) sin(yi−xi) (6.2) Letdt = ∆t, then: xi(t+ ∆t) =xi(t) + ∆t·ω+ ∆t·K |Nm ε (x(t))| · X y(t)∈Nm ε (x(t)) sin(yi(t)−xi(t)) (6.3)

Since the term ∆t·ω is the same for each dimension of all objects and thus can be ignored. Let C = ∆t·K, the dynamics of each dimension xi of an

object x over time is written as:

xi(t+ ∆t) = xi(t) + C |Nm ε (x(t))| · X y(t)∈Nm ε (x(t)) sin(yi(t)−xi(t)) (6.4)

Weighted Interaction Determination

For most existing interaction models, e.g. [94], [11], [3], the coupling strength of the object interactions is constant. However, it is not appropriate for sub- space clustering since clusters exist in different subspaces. Thus, to ensure all

6.3 The Algorithm ORSC 103

cluster objects can synchronize in corresponding subspaces, the strength of interactions is considered followed by the local data structure of the objects. For an objectx, it is expected that the interactions along the main directions of the local cluster structure (relevant and potentially correlated dimensions) are imposed much higher weights while those in irrelevant dimensions have lower weights. Therefore, to determine the main directions of the local cluster structure, the PCA is used to decompose the covariance matrix Σ of objects

Nm

ε (x), which is denoted by Σ = V EVT. The orthogonal Matrix V called

eigenvector matrix and the diagonal matrixE called eigenvalue matrix. The eigenvectors represent the principal directions of these similar objects and the eigenvalues represent the variance along these directions. The eigenvalues are normalized into the interval (0, 1) by dividing each eigenvalue λi with the

sum of all eigenvalues. Then, the normalized eigenvalues are viewed as the in- teraction weights along with the corresponding principal directions. Finally, the difference vector between two objects is projected onto these orthogonal eigenvectors and the difference with corresponding normalized eigenvalues is coupled. Formally, the weighted interaction between two objects are defined as follows.

Definition 6.2 (Weighted Interaction) Let x ∈ Rd be an object

in the data set D. Nm

ε (x(t)) is the ε-Neighborhood of the object x and

y ∈ Nm

ε (x(t)). v~1, ..., ~vd and λ1, ..., λd are the eigenvectors and eigenvalues

by PCA decomposition of the covariance matrix of Nm

ε (x(t)). The weighted

interaction between the object y ∈ Nm

ε (x(t) and the object x, donated by

P εε v1(λ1) Q1 v1 (λ1) v2(λ2) P Q Q2 Q

Figure 6.3: Weighted interaction between two objects.

W I(y./x) =

d

X

k=1

λi·sin(proj(∆(y, x), ~vk)) (6.5)

where ∆(y, x) = y − x means the difference vector between y and x,

proj(∆(y, x), ~vi) means the projection of vector ∆(x, y) onto v~i. Since the

eigenvectors v~i are unit vectors, therefore,

proj(∆(y, x), ~vi) = (∆(y, x)v~i)·v~i (6.6)

where means the inner product.

To illustrate the weighted interaction, Fig. 6.3 gives an example with a 2- dimensional data set. Given an objectP, first, theε-neighborhood of object

P with Mahalanobis distance are obtained. Then the covariance matrix of these objects is decomposed by PCA and the eigenvectors (v~1, v~2) and eigenvalues (λ1, λ2) are obtained respectively. For each interaction with object P, e.g. Q ./ P interaction, the difference vector −→QP is projected to the first direction v~1 with

−−→

Q1P and the second direction v~2, denoted by

−−→

Q2P. The interaction between objects Q and P is finally determined with

λ1·sin(

−−→

Q1P) +λ2·sin(

−−→

Q2P).

6.3 The Algorithm ORSC 105 xi(t+ ∆t) = xi(t) + 1 |Nm ε (x(t))| · X y(t)∈Nm ε (x(t)) (6.7) · d X k=1 λk·sin(proj(i)(∆(x(t), y(t)), ~vk))

where proj(i) means the i-th dimension of project vector. The object x at time step t = 0 : x(0)(x1(0);· · · ;xd(0)) represents the initial state of the

object. The xi(t+ ∆t) describes the renewal state value of i-th dimension of

object xat time point (t+ ∆t).

To determine the termination of the dynamic process, a synchronization order parameter r is defined as measuring the degree of synchronization of objects.

Definition 6.3 (Synchronization order parameter) The synchro-

nization order parameter r characterizing the degree of synchronization is defined as the average movements of objects over time:

r = 1 N N X i=1 1 |Nε(x)| X y∈Nε(x) W I(y−x) (6.8)

The value ofrdecreases as more and more objects synchronize over time. The process towards synchronization terminates when r converges, which indicates there is no further change of objects.