Implicit Subclass Division Based Discriminant Analysis

An implicit subclass division utilises the subclass information without enlarging the class labels in a fining and coarsening scheme, and is realised in the discriminant analysis directly. The discriminant analysis based algorithms classify the samples with a projection of the original data into a reduced subspace with an optimised separability by simultaneously max- imising the between-class distance and minimising the within-class distance. Accordingly, a general criterion [167] of separation for most discriminant analysis algorithms is defined as

Table 3.1 Comparison of KNN-LDA and LDA based inter-day hand motion recognition with inadequate training data with/without normalisation processing

Subjects With Normalisation Without Normalisation

LDA (%) KNN-LDA (%) LDA (%) KNN-LDA (%)

1 97.08±2.41 99.31±0.52 98.23±0.94 98.82±0.59 2 75.98±6.87 77.65±6.64 88.21±2.14 88.86±2.34 3 82.92±6.30 84.87±6.65 94.88±2.52 95.10±2.42 Mean 85.33 87.28 93.77 94.26 S= |ω T SBω | |ωT_S Wω | (3.16) where SW and SB are the scatter matrices of within-class distance and between-class

distance respectively, while ω represents the direction for projection. The discriminant analysis aims to find an optimal projection direction by maximising S. Conventionally the within-class scatter matrix is defined as

S_W = 1 N C

∑

i=1 Ni

∑

j=1 (xi j− µi)(xi j− µi)T (3.17)

where N is the total number of samples, C is the number of classes, Niis the number of

samples that belong to class i, xi j is the j-th sample within the class i, and µiis the mean

centre of class i.

Different from the previous KNN-LDA solution, an integration of subclass division into the projection determination is adopted instead of using two independent subclass division and linear discriminant stages. The subclass discriminant analysis (SDA) algorithm proposed by Zhu et al. [168] is the first discriminant analysis considering the distance between subclasses instead of the distance between classes. The idea is adopted in combination with the nearest neighbour based division criterion to find the most convenient division of each class into multiple subclasses in an exhaustive scheme. In this thesis, the between-class scatter matrix is defined as

S_B= C

∑

i=1 ci

∑

p=1 C

∑

j=i+1 cj

∑

q=1 nip N_i njq N_j(µip− µjq)(µip− µjq) T (3.18) where C is the number of classes, ciis the number of subclasses within the class i, Niand

Nj are the number of samples belonging to class i and j respectively, nip is the number of

class i. The distance between subclasses is utilised instead of the distance between classes to extract the distribution information of subclasses attributed to the overlapping classes in long-term use.

Conventionally, once the scatter matrices are acquired, the projection direction is found to linearly separate the pooled data following the generalised eigenvalue decomposition as

SBW = SWW Λ (3.19)

where W is the matrix whose columns are formed by the right eigenvectors and Λ is the diagonal matrix whose diagonal elements are corresponding eigenvalues. The first k columns of W with the greatest magnitude of eigenvalues are selected to form the projection matrix Wk for a k + 1-class classification problem.

Based on the definition in 3.18, the further calculation of projection direction requires an determined subclass division number of each class. The strategy for seeking optimal subclass divisions with the leave-one-out test is adopted on the pooled dataset for both training and testing with only one sample excluded to achieve the global optimum. In this thesis, the selection of separate training and testing samples is adopted to address the inter-day and long-term use instead of solely considering the intra-day use. Specifically, the training samples Utrain from each class are first divided into subclasses according to their

inter-sample distances. The nearest neighbour method is adopted in the distance sorting, which measures the Euclidean distance Dp,q between the two samples xip and xiq within the

class i to determine their subclass category. Then SDA classifiers using different subclass division choices are compared with the recognition accuracy on the testing data Utest. The

one with the highest accuracy is finally selected for further validation on the subclass division number Γ. The testing algorithm is summarised as Shown in Algorithm 1.

The comparison of SDA and LDA based solutions on inter-day use is depicted in Fig. 3.8. The data of 3 experienced subjects performing 9 motions in consecutive 7 days are adopted. The classifiers are trained by only 2 day’s data and tested on the rest days’ data using the same group of TDAR features, which means the recognition is totally conducted on the unseen data captured in novel days. The number of subclasses is set equal among all classes in opposition to the previous flexible settings in GNG. The average recognition error rate decreases with slight improvement for all the 3 subjects utilising the proposed SDA instead of LDA. However, the performance improvement on subject 1 can be ignored in comparison with the other 2 subjects, which is possibly attributed to either the less variant sEMG patterns exerted by the subject among multiple days or the less flexibility of the adopted subclass division scheme. It is also worth noting that the optimal subclass division is subject dependent with a respectively selected subclass division number of 2, 4 and 7 for

Algorithm 1 SDA algorithm with subclass division tested on inter-day recognition Dataset: Utrain, Utest

Output: Γ

Initialisation: T , R = {r1, r2, · · · , rT}

Calculate the within-class scatter matrix SW of Utrain using 3.17

for i=1 to C do

Extract the data from Utrain belonging to class i, denoted as Xi= {xi1, xi2, · · · , xini}

Let Ui1= /0, Ui2= /0

Construct the distance matrix D, whose element is calculated by Dp,q= ∥xip− xiq∥

Let diag(D) = +∞

Retrieve the index of the largest distance element (a1, b1) = arg max(l,m)Dl,m

U_i1= U_i1∪ {x_ia1}, U_i2= U_i2∪ {x_ib1} for j=2 to ⌈ni/2⌉ do

a_j= arg min_ajD_aj−1,aj,bj= arg minbjDbj−1,bj

U_i1= U_i1∪ {x_iaj}, U_i2= U_i2∪ {x_ibj} Let D_(bj−1,t)= D_(t,bj−1)= D_(aj−1,t)= D_(t,aj−1)= +∞, ∀1≤t≤ni end for U_i= U_i1∪U_i2 end for for k=1 to T do

Divide the sorted samples in Uievenly into k subsets, ∀1≤k≤T

Calculate the between-class scatter matrix SBincluding all Uiusing 3.18

Calculate the generalised eigenvalue decomposition SBW = SWW Λ

Let W = [w1, w2, · · · , ws],Λ = Diag(λ1, λ2, · · · , λs) with sorted eigenvalues

λ1≥ λ ≥ · · · ≥ λs

Let WC−1= [w1, w2, · · · , wC−1]

Project training data Utrainand testing data Utest to subspace of WC−1

Predict label for testing data according to the decision boundary Calculate the recognition accuracy rk

end for

each subject. Thus it is favoured that an optimisation of the subclass division should be conducted on new users to ensure its effectiveness, yet the process of which remains time consuming and to be further improved.

Subject 1 Subject 2 Subject 3

0 2 4 6 8 10 12 14

Recognition Error Rate %

LDA SDA

Fig. 3.8 Comparison of inter-day recognition error rate between SDA and LDA methods for 3 subjects

To further validate the feasibility of SDA in inter-day hand motion recognition with inadequate data for the implicit subclass division, a test on another dataset comprising 6 subjects, who are inexperienced at sEMG based hand motion recognition for prosthetic control, performing 13 hand motions in 10 days is utilised, where the training is conducted respectively with 1 day’s data in Fig. 3.9 and 2 days’ data in Fig. 3.10 whose sEMG are captured at distinct trials and testing on the rest days’ data respectively. An improvement of recognition accuracy across the subjects can be seen for the situation with inadequate training of 1 day, and 9 days’ totally unseen data for testing, which supports the implicit incorporation of subclass division when inadequate training data are provided.

A detailed numerical comparison of the recognition accuracy and corresponding numbers of testing samples is shown in Table 3.2. It is seen that the recognition accuracy increases by a large extent of around 10% when a new day’s data is included for the training which aligns with the intuition. Regardless of the enriched training of 2 days’ data, the total samples are still inadequate when compared to the 8 days’ unseen data for prediction. When adequate training data are available across multiple days and subjects, instead of the conventional

S1 S2 S3 S4 S5 S6 0 10 20 30 40 50 60 70 Recognition Accuracy % SDA LDA

Fig. 3.9 Comparison of inter-day recognition accuracy between SDA and LDA methods for 6 subjects with 1 day for training

S1 S2 S3 S4 S5 S6 0 10 20 30 40 50 60 70 80 Recognition Accuracy % SDA LDA

Fig. 3.10 Comparison of inter-day recognition accuracy between SDA and LDA methods for 6 subjects with 2 days for training

pattern recognition approaches, the improved feasibility of deep learning approaches over discriminant analysis is explored in the following sections.

Table 3.2 Comparison of SDA and LDA based inter-day hand motion recognition accuracy with inadequate training data of 1 day and 2 days

Subjects Training on 1 Day Training on 2 Days

LDA (%) SDA (%) LDA (%) SDA (%)

1 40.11±3.79 40.54±3.46 53.29±10.50 53.12±10.51 2 52.11±9.06 52.53±9.28 62.08±3.50 62.08±3.70 3 62.46±6.91 62.72±6.76 70.78±3.25 70.85±3.18 4 53.83±3.89 54.26±3.88 62.48±3.32 62.56±3.31 5 49.17±8.82 49.43±8.83 60.93±4.45 61.24±4.62 6 62.27±5.37 62.58±5.29 71.03±3.28 71.47±3.18 Mean 53.32 53.67 63.43 63.55

In document Surface electromyography driven hand motion recognition for long-term rehabilitation use (Page 73-79)