The MI Classification Results for Each Classifier Testing Dif-

In this study, three classifiers are used to identify the MI EEG signals where the two training feature sets (Fs1 and Fs2) are used separately to train each classifier and the testing feature sets are used to evaluate the general efficacy of each classifier.

Table 5.5: The classification results of each classifier based on the eleven-feature set (Fs1) in dataset IVa.

Subject

MLP LS-SVM LR

Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%)

aa 98.31 100 99.15 98.31 100 99.15 98.31 100 99.15 al 100 100 100 100 100 100 100 100 100 av 100 100 100 100 96.61 98.31 100 100 100 aw 100 100 100 100 100 100 96.61 100 98.31 ay 98.31 100 99.15 98.31 100 99.15 98.31 100 99.15 Average 99.32 100 99.66 99.32 99.32 99.32 98.65 100 99.32

Table 5.6: The classification results of each classifier based on the two-features set (Fs2) in dataset IVa.

Subject

MLP LS-SVM LR

Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%)

aa 98.31 100 99.15 98.31 100 99.15 98.31 100 99.15 al 100 100 100 100 100 100 100 100 100 av 100 100 100 100 100 100 100 100 100 aw 100 100 100 100 100 100 98.31 100 99.15 ay 98.31 100 99.15 98.31 100 99.15 98.31 100 99.15 Average 99.32 100 99.66 99.32 100 99.66 98.99 100 99.49

the eleven-feature set (Fs1) and the two-feature set (Fs2) are presented in Table 5.5 and Table 5.6, respectively, for each subject in dataset IVa. In both Table 5.5 and 5.6, the results of each classifier are shown in terms of sensitivity (Se), specificity (Sp), and overall Accuracy(Acc).

It can be seen from Table 5.5 and 5.6 that the MLP classifier achieves the same performance in both feature sets. It produces the accuracy of 99.15% for Subject aa,

100% for subjects al, av and aw, and 99.15% for Subject ay for both feature sets in dataset IVa. The average classification accuracy, sensitivity, and specificity of the MLP classifier for both Fs1 and Fs2 feature sets are 99.66%, 99.32%, and 100%, respectively.

For the same dataset, the LS-SVM for the eleven-feature set (Fs1) obtains 99.15%, 100%, 98.31%, 100%, and 99.15% classification accuracy for subjects aa, al, av, aw, ay, respectively, whilst those values are 99.15%, 100%, 100%, 100%, and 99.15%, re- spectively, for the two-feature set Fs2. As shown in Table 5.5 and 5.6, The overall clas- sification accuracy, sensitivity, and specificity of the LS-SVM classifier for the feature set Fs1 are 99.32%, 99.32%, and 99.32%, respectively, while these values are 99.66%, 99.32%, 100%, respectively, for the feature set Fs2.The results show that the LS-SVM classifier for the feature set Fs2 performed slightly better in terms of Acc and Sp on dataset IVa.

On the other hand, it is observed from Table 5.5 and 5.6 that the LR classifier achieves the classification accuracy of 99.15%, 100%, 100%, 98.31%, and 99.15% for subjectsaa, al, av, aw, ay, respectively, for the eleven-feature set Fs1; whereas these values are 99.15%, 100%, 100%, 99.15%, and 99.15%, respectively, for the two-feature set Fs2.The average accuracy, sensitivity, and specificity of this classifier are 99.32%, 98.65%, and 100%, respectively, for the feature set Fs1; where the values are 99.49%, 98.99%, and 100%, respectively, for the feature set Fs2. The results show that this clas- sifier for the feature set Fs2 performed slightly better in terms ofAccandSe on dataset IVa.

From Table 5.5, it is also seen that the highest average classification accuracy is achieved for the MLP classifier for the feature set Fs1, which is 99.66% as compared to the LS-SVM and LR classifiers. While for the feature set Fs2 (Table 5.6), both MLP and LS-SVM achieves the overall accuracy of 99.66%.

Table 5.7: The classification results of each classifier based on the eleven-feature set (Fs1) and the two-feature set (Fs2) in dataset IVb.

Feature set

MLP LS-SVM LR

Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%) Se(%) Sp(%) Acc(%)

Fs1 100 100 100 100 98.31 99.15 100 100 100

Fs2 100 100 100 100 100 100 100 100 100

Table 5.7 presents the classification results of each classifier for the eleven-feature set Fs1 comparing with the results of the two-features set Fs2 for dataset IVb. As shown in Table 5.7, both MLP and LR classifiers obtain the same performance in both feature sets. They achieve an accuracy of 100%, the sensitivity of 100% and the specificity of 100% for dataset IVb. On the other hand, The LS-SVM classifier for the feature set Fs2 performed better in terms ofAccand Sp on dataset IVb as compared to the results for the feature set Fs1. It produces the accuracy of 99.15% the sensitivity of 100%, and specificity of 98.31%for the feature set Fs1, while those values are 100%, 100%, and 100%, respectively, for the feature set Fs2.

Figure 5.5 illustrates the total training and testing execution time for each classi- fier based on both feature set Fs1 and Fs2. The training execution time is total time of training phase of the proposed method including time for normalisation, temporal pat- tern extraction, cross-covariance sequence computation, statistical features extraction, and time for training the classifier. The testing execution time includes all processing times in testing part of the proposed method including time for normalisation, cross- covariance sequence computation, statistical features extraction, and time for testing the classifier. From Figure 5.5, it can be seen that the total training and testing execution time for each classifier with feature set Fs2 is much less than the total training and testing execution time of same classifier with feature set Fs1. These results show the efficiency of the feature selection method in reducing the execution time.

From the classification results and execution time results, one can see that the two features: Maximum, and Mode selected by Best-First+CfsSubsetEval method; are the best features to characterise the distribution of MI tasks signal. In addition, they provide shorter execution time as compared to the eleven-feature set. Thus, the feature set Fs2 is considered as the best feature set for MI task classification process in this study. It is worth to mention that it is not possible to provide a comparison report with existing methods [22, 23, 24, 25, 6, 26, 27, 28, 29, 30] in terms of computational complexity since the computational complexity of the existing methods are not reported in their paper.

Figure 5.6 presents ROC areas for the MLP, LS-SVM, and LR classifiers for feature set Fs2, separately for all subjects in both dataset and their overall ROC area as well. The area of the ROC curve is used as a measure for assessing the classifier performance (e.g., a higher value of the area indicates better performance of the classifier). As seen in Figure 5.6, each of the three classifiers achieves high ROC area close to 1 for each subject and the MLP classifier produces slightly higher overall ROC area than the other two classifiers. From the experimental results for both data sets, it is obvious that the proposed feature extraction approach based on PCA and CCOV is capable of extracting robust features for MI tasks data and both MLP and LS-SVM classifiers have the ability to accurately classify MI tasks in BCI application.