Classifying the five imagined words for fixed time separated data

rated data using time-domain features

For each subject, one block (40 trials) from each word was recorded where the word imagination was 4 second (see Chapter 5, section 5.1.3).

Table 6.9 shows the classification of the five imagined words using the proposed DTW and two TD feature sets using four classifiers. For all subjects, the proposed DTW features using the LDA classifier significantly outperformed the other feature sets. Table 6.10 presents pairwise t-tests results of using the proposed DTW in comparison to the other feature sets.

Table 6.10 Pairwise t-test between results of the proposed DTW features and the other feature sets in classifying the five imagined words (fixed time sepa- rated data);Xmeans significant, × means not significant. The values inside the parenthesis are p values

Compared Features Classification Algorithms

SVM NB RF LDA

DTW and MaxCC _{X(0.0001) X(0.0005) X(0.0001) X(0.000004)} DTW and Statistics _{X(0.003) X(0.0007)} × _{X(0.00000005)}

The results in Table 6.9 were then compared with the classification of imagined words using mouse-click separated data. Table 6.11 presents the classification of the five words using 40 trials for each word for mouse click separated data. In contrast to classifying speech and non-speech results, the fixed-time separated data results were significantly higher than the mouse-click separated data results (see sections 6.6.1 and 6.6.3). This could be due to several factors, including the imagination length, the overlap between the imagination task and the motor execution task using mouse clicks. More discussion of the results is in Section 6.7.

Table 6.11 10-fold cross-validation classification accuracy (%) to classify the five imagined words using the proposed DTW and four classifiers (mouse click separated data);using 40 EEG trials for each word; t-tests compare the average classification of mouse click separated words with the average classification of

fixed time separated words for each classifier

Subject SVM NB RF LDA 1 49.37 43.68 58.82 70.87 2 47.18 38.68 49.16 53.24 3 55.74 36.18 40.16 58.37 4 57.26 30.05 43.08 65.32 5 84.45 76.37 81.39 87.50 6 41.66 27.61 29.08 46.68 7 35.68 26.08 50.79 49.82 8 44.68 29.63 40.68 50.24 9 65.82 38.24 47.79 65.32 10 63.34 49.18 53.21 69.82 AVE 54.52 39.57 49.42 61.72 SD 13.83 14.49 13.86 12.25 T-test × × _{X(0.04) X(0.015)}

Toward understanding merits of the proposed DTW-based features

In the proposed DTW features, the variation in the distances between the two warped signals represents the features distribution. Figure 6.4 shows the distribution of distances for the word “up” in comparison to other words for subject 9. Subject 9 had the best classification accuracy (95.48%) using LDA. Moreover, “up” has the best

6.6 Analysis and Results 117

accuracy of the five words (100%), as can be seen in the confusion matrix in Table 6.12. The distances between trials for word “up” did not completely overlap with the distances with the other words, as in Figure 6.4.

Subject 10 had the lowest classification accuracy (54.77%) using LDA. The word

“select” had the lowest classification (25.00%) among the five words (Table 6.12). As

seen in Figure 6.5, an overlap exists when comparing the distances of the trials of the word “select” and the trials for the other words. Most of the misclassifications were between “select” and “right”. This was because the average distance between these two words was very small and overlapped with the distances of the word “select” trials.

Table 6.12 Confusion matrix for classifying the five imagined words using DTW- based features and LDA classifier for subject 9

Word Left Right Up Down Select

Left 100.00 0.00 0.00 0.00 0.00

Right 0.00 100.00 0.00 0.00 0.00

Up 0.00 0.00 100.00 0.00 0.00

Down 0.00 5.00 7.50 85.00 2.50

Select 0.00 2.50 0.00 2.50 95.00

Table 6.13 Confusion matrix for classifying the five imagined words using DTW- based features and LDA classifier for subject 10

Word Left Right Up Down Select Left 50.00 37.50 2.50 0.00 10.00 Right 2.50 90.00 5.00 0.00 2.50

Up 5.00 32.50 60.00 2.50 0.00

Down 2.50 2.50 32.50 55.00 7.50

U vs U U vs L U vs R U vs D U vs S Compared Words 1000 1500 2000 2500 3000 3500 4000 4500 5000 Average Distance

Fig. 6.4 Distribution of average DTW distances between trials of the word “up” for subject 9 and DTW distances between the word “select” and other words. (The letters represent the first letter from each word).

S vs S S vs L S vs R S vs U U vs D Compared Words 1000 1500 2000 2500 3000 3500 4000 Average Distance

Fig. 6.5 Distribution of average DTW distances between trials of the word “se- lect” for subject 10 and DTW distances between the word “select” and other words. (The letters represent the first letter from each word).

6.6 Analysis and Results 119

6.6.5

Classifying the five imagined words for mouse click sep-

arated data using DDTW feature sets

Section 6.4.4 explains derivative dynamic time warping (DDTW), which was a proposed modification of classical DTW. Kumagai et al. (2017) claimed that DDTW solves some problems during signal warping. In the present study, DDTW was used to classify the five imagined words using mouse-click separated data applying the steps used for the proposed DTW. Table 6.14 lists the average classification accuracy across subjects of classifying the five imagined words and the pair-wise t-test results in comparison to the proposed DTW framework.

The results show that the proposed DTW statistically outperformed DDTW with the SVM and LDA classifiers. However, the difference is not significant in comparison to the RF and NB classifiers. This finding could be justified based on the differences between DTW and DDTW and the technical differences between the classifiers. DDTW works on the derivative of the signal, which amplifies the noise more than the original signal (Jauberteau and Jauberteau, 2009). Moreover, LDA and SVM are known to be the best classifiers for brain computer interface studies. However, their results can be easily affected by noise (Müller et al., 2004). The NB classifier is similar to the LDA classifier in terms of assuming Gaussian distribution of the data. However, NB assumes the features are independent (Misaki et al., 2010). Consequently, the results are stable when using either the derivative or the original signal. The RF classification algorithm performs the decision based on a subset of the features. Typically, this makes it successful and stable with noise and a small training size (Lotte et al., 2018).

Table 6.14 Average classifications results (across subjects) (%) of classifying the five imagined words for mouse click separated data using DDTW feature sets; pairwise t-tests to compare the classification accuracies resulted from using the proposed DTW and DDTW for each classifier;Xmeans significant, × means not significant. The values inside the parenthesis are p values

Method SVM NB RF LDA

DTW 38.37 35.57 48.83 52.50

DDTW 21.93 34.23 51.00 43.77

T-test _X(0.007) × × _X(0.0001)