Spectral Collaborative Representation based Classification for hand gestures recognition on electromyography signals

(1)

ContentslistsavailableatScienceDirect

Biomedical

Signal

Processing

and

Control

j ou rn a l h o m e p a g e :w w w . e l s e v i e r . c o m / l o c a t e / b s p c

Spectral

Collaborative

Representation

based

Classiﬁcation

for

hand

gestures

recognition

on

electromyography

signals

Ali

Boyali

∗

,

Naohisa

Hashimoto

NationalInstituteofAdvancedIndustrialScienceandTechnology,1-1-1Umezono,Tsukuba,305-8568Ibaraki,Japan

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received30June2015

Receivedinrevisedform24August2015 Accepted1September2015

Availableonline16September2015 MSC:

00-01 99-00 Keywords: EMGgesture

Continuousgesturerecognition Spectralrepresentation Gesturetrainingmatrix MYOarmband

a

b

s

t

r

a

c

t

Theclassiﬁcationofthebio-signalhasbeenusedforvariouspurposesintheliteratureastheyareversatile indiagnosisofanomalies,improvementofoverallhealthandsportperformanceandcreatingintuitive humancomputerinterfaces.However,automaticidentiﬁcationofthesignalpatternsonastreaming real-timesignalrequiresaseriesofcomplexprocedures.Aplethoraofheuristicmethods,suchas neu-ralnetworksandfuzzysystems,havebeenproposedasasolution.Thesemethodsstipulatecertain conditions,suchaspreconditioningthesignals,manualfeatureselectionandlargenumberoftraining samples.

Inthisstudy,weintroduceanovelvariantandapplicationoftheCollaborativeRepresentationbased Classification(CRC)inspectraldomainforrecognitionofhandgesturesusingrawsurface electromyo-graphy(EMG)signals.TheCRCbasedmethodsdonotrequirelargenumberoftrainingsamplesforan efficientpatternclassification.Additionally,wepresentatrainingprocedureinwhichahighendsubspace clusteringmethodisemployedforclusteringtherepresentativesamplesintotheircorrespondingclass labels.Thereby,theneedforfeatureextractionandspottingpatternsmanuallyonthetrainingsamples isobviated.

Wepresentedtheintuitiveuseofspectralfeaturesviacirculantmatrices.TheproposedSpectral Col-laborativeRepresentationbasedClassiﬁcation(SCRC)isabletorecognizegestureswithhigherlevelsof accuracyforafairlyrichgesturesetcomparedtotheavailablemethods.Theworstrecognitionresult whichisthebestintheliteratureisobtainedas97.3%amongthefoursetsoftheexperimentsforeach handgestures.Therecognitionresultsarereportedwithasubstantialnumberofexperimentsandlabeling computation.

1. Introduction

Electromyographic (EMG) signal based applications beyond clinicalandrehabilitationpurposeshavebeenextensivelystudied intheliterature.TheclassificationofEMGsignalsinthe kinesiolog-icalfieldhasbeeninvestigatedtoanalyzeandimprovesport[1,2], art[3]andoccupational[4]performance.HumanComputer Inter-faceresearchfieldisanothernichewheretheEMGsignalshave beenutilizedincluding,butnotlimitedtocontrolofexoskeletons, roboticprostheticarmsandhands.

ThesuccessoftheEMGsignalclassiﬁcationhighlydependson threestageswhicharepre-processing,featureselectionandthe classiﬁcation.Inthepre-processingstage,noiseandartifactsare removedfromthesignals.In[5],theauthorsproposeatimelagged RecurrentNeuralNetworks(RNN)toeliminatethenoiseonthe

∗Correspondingauthor.Tel.:+81298588008. E-mailaddress:[email protected](A.Boyali).

EMGsignals.Wavelettransforms[6],higherorderstatistics[7,6]

andempiricalmodedecomposition[8]methodhavealsoshownto beeffectiveforremovingartifactsandnoisefromtheEMGsignals. The successof the EMG signalclassiﬁcation highly depends onthreestages:pre-processing,manualfeatureselectionandthe classiﬁcation.Inthepre-processingstage,noiseandartifactsare removedfromthesignals.Theauthorsin[5]proposeatimelagged RecurrentNeural Networks(RNN)toeliminate thenoiseonthe EMGsignals.Wavelettransforms[6,9],higherorderstatistics[7,6]

andempiricalmodedecomposition[8]methodhavealsoshown tobeeffectivemethodsforremovingartifactsandnoisefromthe bio-signals[10].

Featureextractiondealswithextractingdiscriminating infor-mationhiddeninthedatawhichrequireexperienceinthefield andtediousinvestigationofallpossiblefeatures.Statisticalfeatures suchasmean,varianceandzero-crossingandderivativeshavebeen predominantly examinedand usedinthebio-signalstudies.No matterhow purethesignalandhow accuratetheclassification methodis,withoutdiscriminatingfeaturesitisdifficulttoobtain

http://dx.doi.org/10.1016/j.bspc.2015.09.001

(2)

reliableclassiﬁcationresultsforanysignal.Inrecentyears,with theintroductionoftheauto-encodersandnewnetwork architec-tures[11],thedormantneuralnetwork hasbeenrevivedinthe burgeoningdeeplearningresearches.Thepremiseofthisﬁeldis theself-learningneuralnetworkswhichlearnthefeaturesinan unsupervisedmannerfromthebigdata.

Heuristicmethods and variants of classicalneural networks havebeenextensivelyusedinclassificationofthestochastic bio-signals[12–16].Theauthorsin[12]useaclassicalneuralnetwork andstatisticalfeaturesforclassificationoftheEMGsignalsobtained onfromthefacemuscles tosteer apower wheelchair.A feed-forwarderrorbackpropagationandwaveletneuralnetworkswere trainedin[13]witharelativelysufficientnumberofthetraining sampleswithrespecttothatin[12].Inthestudies[14,15]a com-binationoffuzzyandneuralnetworkmethodswasusedtoclassify EMGsignals.

Dependingonthenumberof signalpatternstobeclassified, variouskindsoffeaturesandmethodsforpre-processingand fil-tering,theaccuracyoftheclassificationmethodsrangesbetween 88and99.75%[17,10].Thecommonpointsofthelistedmethodsare thelargenumberofprocedurestoobtainasubstantialrecognition accuracyandthedifficultyintheimplementation.Ontheother hand,neuralnetworkcanonlyresultinagenerativemodel pro-videdthatasufficientnumberoftrainingsamplesarefedthrough. TheEMGsignalmeasuredonskinsurfaceisnotameresignalbut ratherthesuperpositionoftheMotorUnitActionPotential(MUAP) ofthemanytinymusclefibers[18,19].Duetothestochasticnature ofMUAPs,theEMGsignalpatternmightexhibitintraand inter-personalvariationsforthesamemuscleactivity.Thesevariations maketheEMGpatternclassificationanintricatetaskforcreating robustclassifiers.

In concert with the advancements in pervasive computing technologies,thewearablegadgetsinwhich highgradesensors embeddedhavebeenreleasedtothemarket.ThalmicLabs’MYO armband[20]isoneofthestateoftheartsensorswhichconsists ofeightsurfaceEMGsensors.Thesensorkitcontainsaninertial measurementunitaswell.Inevitably,accessibilitytosuchkindof deviceswithanaffordablepricetaggarneredpublicattentionboth fromacademicandnon-academicenvironments.

In this article, we introduce a signal pattern classification methodinspectraldomainandtrainingprocedurestoclassifythe forearmEMGsignals.Thesignalsareobtainedby8-channelMYO armbandreal-time.Theclassificationmethodisbasedonthe Col-laborativeRepresentationbased Classification(CRC) [21] which competeswiththeSparseRepresentationbasedClassification(SRC)

[22]withthesameaccuracylevelsbutmuchfastercomputation times.ThedrawbackoftheCRCcomparedtotheSRCisthatthe CRCrequirestheobservedandtrainingsignalpatternstobeofequal length.ThisrestrictstheuseofCRCmethodsforgestureand pos-turerecognitioninwhichthedurationofthegesturesmightvary foreachrepetition.Ontheotherhand,theSRCmethodsrelyon therepresentationfidelity,whereastheCRCdatafidelity. There-fore,theCRCmethodsleadlowerrecognitionaccuracylevels,if thedatadonotexhibitfidelity[23].AstheEMGsignalsarethe productsofcomplexstochasticprocesses,theperformanceofthe CRCmethodsislowaswedetailcomparisonsintheproceeding sections.

TheSpectralCollaborativeRepresentationbasedClassiﬁcation (SCRC)proposed inthis paper overcomesthesedrawbacks and yieldshighrecognitionaccuracyforafairlyrichhandgestureset. Sincetheobservedandtrainingsignalpatternsmusthaveequal lengths,aspecialtrainingschemeisadaptedtobuildatraining dic-tionary.Therefore,theneedforspottingsignalpatternsandmanual featureselectionisautomaticallyeliminatedandtheboundaryof therepresentativecolumnsisimplicitlyembeddedinthetraining matrix.

Thecontributionsofthisstudytotheliteratureare:

•Thespectral content; complexconjugate eigenvaluepairs are obtained from 1D vectors by converting the observed signal patterntoatrajectoryoracirculantmatrixtocapturethe repre-sentativefeaturesoftheEMGsignals.

•Thegestureandposturerecognitionisperformedina continu-ousmannereliminatingtherequirementsforspottingorpicking thesignalpatternsonthestreamingsignalsduetotheproposed trainingschemewhichimplicitlyembedsthesignalboundaries ontherepresentativecolumns.

•Thetrainingphaseiseasytoimplement;thustheendusercan obtainatrainingdictionaryonthespot.Theﬂexibilityinbuilding atrainingdictionarypavesthewayfortheuseoftheprocedures andmethods introducedheretoimplementdifferent applica-tions,suchascontrolofabionicprosthesis.

•Thenumberofhandgesturesarethehighestamongthesimilar studies.Ourworstrecognitionresultisover97%witha substan-tialnumberofgesturelabelingcomputation.

Therestofthepaperisorganizedasfollows.InSection2,abrief reviewoftheCRCmethodisgivenandthenthecirculantmatrix approachisdetailed.InSection3,aftergivingthetechnicaldetails oftheMYOarmband,trainingphaseforthegesturesandSCRCis elaborated.Theexperimentandsimulationresultsarediscussed inSection4.Thepaperendswithconclusionandfutureworksin Section5.

2. SpectralCollaborativeRepresentationbased Classiﬁcation

2.1. Collaborativerepresentationbasedclassiﬁcation

Spectralmethodshavebeenusedfordecadesinvarietyof sci-encedisciplineswhererandomsignalorstochasticprocessesare involved.

Althoughrandomnessinthesystemsmightrenderanalysisof thefeaturesdifﬁcultintimedomain,thespectralfeaturessuchas eigenvaluesandfrequenciesmightrevealvaluableinformationof theunderlyingprocesses[24].Fourieranalysisisthewidelyused spectral analysismethodfor this purpose. In this study,asthe EMGsignalsshowrandomness,weexploitthespectralanalysisby employingthecirculantmatrixstructureforeigenvalue decompo-sition.

Bothofthemethods,theSRCandCRCaddressﬁndingthelinear representationcoefﬁcientsvectorxfortheobservedpatterny=Ax

whereA=[A1,A2,...,An]∈Rmxnisthedictionarymatrix.The

rep-resentativesamplesarestackedascolumnvectorsinthedictionary. Intheobjectivefunctionofthesolution(Eq.(1))differentvector normsareutilizeddependingonthemethodsandrequirements. ˆ

x=argmin

x y−Axp+xq (1)

IntheSRCsolution1regularizationisused(p=2,q=1),whereas

thenormspandqbecome1or2intheCRCmethodsdependingon therequirementssuchrobustnessoftheclassiﬁer.Iftheobjective functionissolvedbyRegularizedLeastSquare(RLS)inwhich2

normusedforbothtermsoftheobjectivefunctionthesolution turnsouttheridgeregression.

WeusedtheleastsquareversionoftheCRCmethodwhichis dubbedasCRCRLS bytheauthorsin [23].Theridgeregression solutionofEq.(1)isobtainedas ˆx=PywhereP=(AT_A₊_I₎−1

AT

andistheregularizationparameter.Oncethesolutionvector ˆx

isobtainedthelabeloftheobservedsignaliscomputed evaluat-ingtheminimumrepresentationresidualsrigiveninEq.(2)where ıi:Rn→Rnistheselectionoperatorthatselectsthecoefﬁcientsof

(3)

ithclasswhilekeepingothercoefﬁcientszerointhesolutionvector ˆ

x. min

i ri(y)=y−Aıi(ˆx)2 (2)

TheSRCmethodsyieldgoodclassiﬁcationresultsaslongasthe dictionarymatrixisover-completewhereastheCRCdonotrequire over-completedictionaries.Inadditionwiththelowernumberof dictionaryrepresentatives,theregressionoperatorPisonly com-putedandstoredonceandiscodedoverthenewobservedpatterns forthesolutionvectorx.Thisconveniencemakestheclassiﬁcation methodperformfasterthantheSRCinrealtimeapplications.

Infact,newapproacheshavebeenintroducedintheliterature suchasBlockSparseorStructuredSparse1 solvers[25,26]and

BlockSRCmethods performmuch betterthan theplainSRCin termsofthecomputationaltimewithoutcompromisingthe accu-racylevels[27].However,asthe1 solversaremostlybasedon

iterativealgorithms,thecomputationtimesarenotsatisfactoryon themobiledevicessuchasphonesandtablets[28].

2.2. Circulantmatricesandspectralcollaborationbased classiﬁcation

Circulantmatricesappearnaturallyinmanysystemsof equa-tionsuchasthatofdynamicalsystems,convolutionandvibration theories[24,29,30].Acirculantmatrixisconstructedfromavector byshiftingitselementsinclockwiseorcounterclockwisedirections circularly.Dependingonthedirectionoftheirdiagonals(rightor leftcirculants)theytaketheformofToeplitzandHankel matri-cesrespectivelywhichhavewiderangeofapplicationareasfrom compressingsensing,ﬁlterdesign,singularspectrumanalysisto inverseeigenvalueproblems[31–34].

Inthisrespect,assumingavectora=[

v

0,

v

1,...,

v

n],theright circulantC=circ(a)∈Rnxn_becomes

Ca=

⎛

⎜

⎝

v

0

v

1

v

2 ···

v

n−1

v

_n₋1

v

0

v

1 ···

v

n−2

v

_n₋2

v

n−1

v

0 ···

v

n−3 . . . ... ... . .. ...

v

1

v

2

v

3 ···

v

0

⎞

⎟

⎠

(3)

Any circulant can be diagonalized by a unitary or Discrete FourierTransformation(DFT)matrixduetotheircyclicstructure. Thisfactorizationisalsoa similaritytransformation. Ifanytwo matricesAandBaresimilar,oneofthethemcanbeexpressedas

B=T−1_AT_where_T_is_an_invertible_matrix._If_T_is_a_DFT_matrix_F n(Eqs.

(5)and(4)),thematrixAbecomesadiagonalmatrix,theentriesof whicharetheeigenvaluesofthecirculanttransformed.

=FH

nCFn (4)

Inthisequationdiag()={1,2,...,n}andthematrix

oper-ator(o)H_is_the_Hermitian_transpose.

Fn= √1 n

⎛

⎜

⎝

1 1 1 _·_·_· 1 1 W W2 _·_·_· _WN−1 1 W2 _W4 _·_·_· _W2_N₋₂ . . . ... . .. ... 1 WN−1 _W2(N−2) _·_·_· _W(N−1)(N−1)

⎞

⎟

⎠

(5)

The columns of the DFT matrix Fn with the entries

W=exp−2i/n thus become the eigenvectors of the circulant matrix. Since all the eigenvectors of the decomposed circular

matrices withthesame sizeare thesame,theresultant eigen-valuesconstituteadiscriminativesubspace.Thus,thesesubspaces whichareshiftinvariantcanbeusedasadictionarymatrixinthe representation basedclassiﬁcations.Thetime complexityof the eigenvaluedecompositionwithanumberofoperationO(nlogn) islowerthantheconventionaldecompositionmethods;therefore, eigenvaluedecompositionofthecirculantsisfasterthanthe con-ventionaleigenvaluedecompositionalgorithms[32].

IntheSCRC,webuildatrainingdictionaryusingthe eigenval-uesoftheobservedsignalasthefeatures.Since,complexnumbers areinvolved,theHermitiantransposeofthevectorsisusedinthe innerproductspace.Inthecomplexdomain,thesolutionvector ˆx

isobtainedbytheequation ˆx=Pcxwhere

Pc=(AHA+I)−1AH (6)

ThetrainingmatrixisnormalizedandcenteredintheCRC meth-odswhiletheobservedsignalyisonlycentered.Itisimportantto notethatsincethespectralfeaturesareusedintheFourierdomain, Hermitianconjugatetransposeisusedforallinnerproduct opera-tions.

3. SCRCapplicationinEMGsignalclassiﬁcation

3.1. MYOArmbandandGestures

MYOarmbandwhichhasbeenrecentlyreleasedtothemarket byThalmicLabsrevolutionizedtheEMGbasedstudiesbecauseof affordablepricetagandeasydatacollection.Thesensorarmband hasa8-channelEMGsensorgroupalongwithanInertial Measure-mentUnit(IMU)whichreportslinearandrotationalaccelerationas wellastherotationanglesaround3-axes.Thesensorreadingsare transferredoveralowpowerblue-toothadaptertothecomputer (Fig.1).

TheSoftwareDevelopmentKit(SDK)allowsthedevelopersto accesstheEMGsignalsandmotionparametersonthewornarm. AsthemeasuredEMGsignalsdependonthesensorlocationon themuscles,theMYOarmbandapplicationsrequireaspecial cali-brationgestureeverytimewhenthearmbandisputonthearmor takenoff.Inthisway,theapplicationcalibratesthesensorlocations dependingonthesensordirectionandrotationofthearmband. Inourexperimentswecollectedtheexperimentaldatawithout givingabreakandtakingoffwithoutneedingre-calibratingthe device.Thearmbandlocationintheexperimentsis closetothe

(4)

Fig.2.MYOhandgestures,1–ﬁst,2–handrelax,3–ﬁngerspread,4–wavein,5–waveout,6–doubletap.

elbow.Palmarislongus,ﬂezorcarpiradialis,brachioradialis,and ﬂexorcarpiulnarismusclesontheanterior,extensorcarpiulranis, extensordigitorum,extensorcarpiradialisbrevis,extensorcarpi radialislongusmusclesontheposteriorformarmarelocatedunder thearmbandsensors.

TheIMUunitreportsthemotionrelatedparametersata fre-quencyof50Hz,whereasthefrequencyfortheEMGsignalis200Hz asgivenonthecompanywebsite[20].TheSDKprovidesagesture objectbywhichﬁvedifferenthandgesturesarerecognized.The samehandgesturesetwhichiscomposedofhandﬁst,wavein, waveout,handspreadanddoubletapisusedtotesttheSCRC per-formance(Fig.2).Ifnoneofthemisperformed,thehandisassigned tothehandrelaxgesture.

ThelocationoftheEMGsensorsisimportantforcertaintypes ofgesturesasdifferenthandandfingermovementsarecontrolled bythedifferentmusclegroupsontheforearm.Thearmband con-figurationisputonclosetotheelbowwherethediscriminantEMG signalscanbecapturedforhandmovements.Inordertoclassify individualfingergestures,theEMGsensorsmustbeplacedonthe relatedmusclegroupsresponsibleforthefingers.

3.2. Training

Ingesturerecognitionstudies,oneoftheopenproblemsis spot-tingthegesturesonthestreamingsignals.Agesturehasstartand endpointswhichcanbespottedonacontinuouslymonitored sig-nal;howevergesturesmightdifferamongthepeopleevenforthe sameperformer. Alongwiththesevariations,thetimelengthof thedifferentgesturesinasetmightalsoshowvariations.The spot-tingproblemmanifestsitselfinbuildingatrainingdictionaryfor therepresentationbasedclassiﬁcationmethods.Thisproblemcan besolvedbyusingaswitchthatcanmarkthebeginningandend pointsofthegesturesintherealworldandatriggerorswitchevent inthesoftwareapplications.

Theauthorsin[35]collectgesturesbydrawingintheairusingan IRpenwhichistrackedbyanIRcameraofWiimote.Theperformer pressesandreleasestheswitchoftheIRpenatthebeginningand theend ofthegestures respectively.Spotting canalsobedone visuallybyanalyzingthetrajectoryorthevisuallydiscriminating featureswhichrequirestediouseffortforfairlyrichgesturesets.

Inthisstudy,sincetheEMGsignalsconsistofMUAPspikes,the gesturepatternscannotbespottedvisuallyontheﬁguresofthe signal.Therefore,weemploysubspaceclusteringmethodsforthe repeatedcouplegestures.Thesubspaceclusteringmethodsfallin unsupervised.

Wetestedthisapproachforourpreviousstudies[28,36].The reasonistwofoldtoadaptthisapproach.Theﬁrstisthedrawback oftheconstantslidingwindowwhich isveryimportantforthe CRCmethods.Thesignalsarecapturedbyaslidingwindowboth inthereal-timeapplicationandthetrainingphaseforbuildinga representativedictionary.Thesecondisﬁndingrepresentativesfor everygesturestateinanytimewindowwhichaffecttherecognition accuracy.

Asthetrackedbodypart,inourcase,thehandmightswitch betweengestureswhilebeingtrackedandinthetransitionregions gestures overlap. Overlapping gesture regions degenerate the robustnessand accuracy of therecognition.However,subspace clusteringmethodscanseparatetheoverlappingregionsintotheir correspondinggestureclasses.Atleasttwogesturesarenecessary forclustering.

Thestateofartsubspaceclusteringmethodsexploitthe self-similarity property of the signals to be clustered. The most prominentofthesealgorithmsaretheSparseandLowRank Sub-spaceClustering(SSC, LRR)methodswhich makeuseof1 and

nuclearnormintheirobjectivefunctions[37,38].

WeuseavariantofSSC,theOrderedSubspaceClustering(OSC) methodwhich puts an additionalpenalty in theself-similarity objectivefunction for sequentialdata [39].The additionalterm intheobjectivefunctionenforcestheneighboringrepresentative samplestobesamebypenalizingthedifferencesoftwoneighbors. Wecollectedgesturecouplesforalloftheclassesrepresentedin thetrainingdictionary.Thehandperformstwogesturesrepeatedly forthegesturecouples.Oneofthegesturesineachofthecouplesis handrelaxstateinwhichhandstaysatarelaxedposition.Thisstate isassumedtobegesturemateoftheeachgesture.Asaconcrete explanation,letusassumethatahandperformsthehandwave ingestureandreturnstothehandrelaxpositionrepeatedlyasin demonstratedinFig.3.

Aconstantlengthslidingwindowwithanincrementof samp-lingintervalcapturesthecontentontherecordedEMGsignal.The lengthoftheslidingwindowinourapplicationcorrespondsto100

(5)

Fig.3.Wavein–handrelax–waveinrepetition.

samples(halfsecondat200Hz).Themeasurementstakenbythe slidingwindowareputintotheunclustereddictionarymatrix.The subspaceclusteringmethodisusedinthissteptodistillthepure representativesintotheirrespectiveclasses.Asampleclustering resultisgiveninFig.4forthedoubletapgestureonasingle chan-nelEMGsignals.Asseen,theperiod oftheonegesturecycleis approximately0.5scorrespondingto100timesamples.

Infact,althoughitisnotseenontheﬁgure,therearefourhand statesinrepetitionsofthegesturecouples;twostatichand pos-tureswherethehandstaysatthehandrelaxandterminalwavedin positions,andthedynamichandgesturesthroughthesepostures. However,sincethenumberofsubspacesarechosenastwoforthe gesturecouples,theposturestatesareembeddedintotheclusters automaticallyandeveryobservablehandstatesarerepresentedin theseclusters.Thetrainingdictionarycontains10gestureclassesin thiscaseandwemapeveryreturnstatestothehandrelaxposition fromanyhandgesturearemappedtotheﬁvehandgestures. 4. Simulationresults

Wecollectedﬁvegesturecoupleexperiments,onefortraining andtherestfortesting.Inaddition,weperformanarbitraryhand gesturesequenceinwhichthehandvisitsarbitrarygesturestates. The simulations are repeated using the conventional CRCRLS for comparison of recognitionperformance ofthe CRCRLS and SCRC.Thedurationoftheeachhandgesturecoupleexperimentsis

Fig.5. Wavein–handrelaxgesturesonEMGdatawithCRC.

Fig.6.Wavein–handrelaxgesturesonEMGdatawithSCRC.

approximately6scorrespondingto1200sampleddata.Asliding windowwitha lengthof100samples(0.5s)andone sampling timeincrementisusedtocomputethelabelofthegestureatthe samplingtime.Therefore,ontheﬁguresgiveninthissection,at least1200labelingresultsaregiven.Thelabelsareshownasabin onthebarﬁgures.

AlthoughtheconventionalCRCRLSyieldsthesamerecognition accuracyforsomeoftheexperimentaldata,itcannotdiscriminate thehandwaveinandfistgestureslabelingthewaveingestureas thefistgestureinalltheexperiments(Fig.5)withsomenumberof misclassificationwiththehandspreadgesture.However,theworst recognitionresultforthehandwaveingesturesamongthefourtest setsis99.36%fortheSCRCmethod.

Fig.6showstheworstresultofthewavein–handrelaxtest experiment.ThebottomﬁgureisthegesturelabeloftheMYO ges-turerecognitionmodule.SincetheSRCrecognizes10gestures,the numberoflabelsaredifferentthantheMYOgesturemodulewhich labelsthegesturesas1–handrelax,2–ﬁst,3–wavein,4–wave

(6)

Fig.7. Recognitionresultsforﬁstandhandrelaxexperiment,spectralCRCandMYO.

Fig.8.Recognitionresultsfordoubletapandhandrelax,spectralCRCandMYO.

out,5–handspreadand6–doubletap.Thenumberoflabeling computationinthefigureis1219andtheSCRCmisclassifiesthe signalpatternsasthefistgestureeighttimes.

AsseeninFig.6,themisclassifiedgesturesseemtooccuratthe transitionstates(afterthe200thsampleinthefigure)inwhich handgoesfromthehandrelaxstatestowardthehandfist.This isduetothefactthatwecomputethelabeloftheeach instan-taneoushandpositionatafrequencyof200Hzandthelengthof theslidingwindowis100samples.Thereare400representative samplesforeachclassinthetrainingdictionaryimplyingonlyfour representativesoftheeachinstantaneoushandposition.

Therecognitionaccuracytheﬁstandthedouble-tapgesturesis 100%forallthetestexperiments(Figs.7and8).Theﬁstanddouble tapgesturesarewellseparatedfromthehandrelaxstate.InFig.8

onlytworipplesareseenaround600and1100thsampleswhere thealgorithmschangethelabelfromtheintendedgesture.Inother wordsatthesesamplingtimes,thestateofthehandislabeledeither asthehandrelaxordoubletapwhilethehandisintendedtostay attheotherstate.Thesetwoinstancesarenotmisclassification.As seeninthefigure,thehandperformsthreedoubletapgesturesin asecondwhichisaquickhandmovementandthefingerstapeach otherinthisexperiments.

TheresultsofthefingersspreadrecognitionisgiveninFig.9.The worstrecognitionaccuracyinthestudyamongalltheexperiments isobtainedforthishandgestureas97.3%.Thealgorithmgivessome misclassificationsforfingerspreadsand thewaveoutcoupleas thereareoverlappingwristcontractionandflexionstatesforthe twohandmovementsthatthehandmightvisitinvoluntarily.

The handwave out gesturerecognitionresults are given in

Fig.10withoutmappingthehandrelaxreturngesturetothehand relaxstate.Inalltheﬁgures,themappedhandrelaxstateislabeled asone.Inthefollowingﬁgurethelabelsixshowsthatthehand is moving towardto thehandrelaxposition. Theworst recog-nitionaccuracy outoffourtest experimentsis98.34% for1274 labelingcomputationinthewaveoutexperiment.Theresultsfor theCRCRLSisgiveninFig.11,inwhichtheaccuracyislessthan 85%.

(7)

Fig.10.Recognitionresultsforwaveoutandhandrelax,spectralCRCandMYO.

Fig.11. RecognitionresultsforwaveoutwithCRCRLS.

Astheﬁnalexperimentwegivetheresultsofanarbitrary ges-turesequence.Inthis experiment,unliketheotherexperiments wherethehandonlyperformstwohandgestures,different ges-turesareperformedarbitrarily(Fig.12).Therecognitionaccuracy is98.47%forthisexperiment.

Fig.12.Recognitionresultsforrandomhandgesturesequence,spectralCRCand MYO,1–handrelax,2–ﬁst,4–wavein,6–waveout,8–handspread.

Inthissection,wediscussedthesimulationresultsoftheSCRC methodandcomparedwiththeconventionalCRC.Thelabelsare computedforeachofthesamplingtime200computationsfora second.Evenwiththisresolution,therecognitionaccuracy com-peteswiththeproposedcomplexproceduresintheliteraturewith ahighernumberofhandgesturesandpostures.Itisnotnecessary tocomputethelabelsatasuchhighrates.Asweimplementeda real-timehandpostureandgesturerecognitionapplicationusing the CRCmethod and LeapMotion sensor for steering a power wheelchair,20Hz is sufﬁcient forsmooth, continuousand high accuracyrecognitionandsteering.

5. Conclusionandfutureworks

In this study we introduced a novel application of the CRC methodsinFourierdomainandelaboratedthemethodologiesand experimentalresultsfor thehandgesturerecognitionusingthe EMGmeasurementsonaforearm.Theaccuracylevelsforthefairly richgesturesetispromisingfortherawEMGsignals;therefore, theycanbefurtherimprovedbyanalyzingthesignalnoiseand introducingappropriateﬁlteringmethods.TheuseofFourier fea-turesisexplainedintuitivelyusingthecirculantapproachwhich revealsshiftinvariantfeaturesinanelegantway.Theapproach detailedinthetrainingphasewhichweusedfordetectingbraking maneuversofamobilityrobotﬁrsttime[40]leadstoadictionary withahighpowerofdiscrimination.Theembeddingofthegesture boundariesintotherepresentativestatesisimplicitlyrealizedby thesubspaceclusteringmethods.

Themethodsandprocedureswedetailed inthispaperarea smallportionofaprojectbywhichamulti-modalintuitiveHCIhave beendevelopedfortheelderlyorseverelyhandicappedpeople.The motivationoftheprojectistoenablethesepeopletosteerarobotic wheelchairwithoutexcessivecognitiveorphysicalefforts.Onthe otherhandthedevelopedsystemswillbeemployedforaugmented andvirtualrealityenvironmentstoestablishtrainingtheatersfor peoplewhoareprescribedpowerwheelchairsﬁrsttime.

Acknowledgments

ThestudyissupportedbytheJapanSocietyforthePromotionof Science(JSPS)fellowshipprogramandtheKAKENHIGrant(Grant Number15F13739).

References

[1]J.P.Clarys,J.Cabri,Electromyographyandthestudyofsportsmovements:a review,J.SportsSci.11(5)(1993)379–448.

[2]N.Masso,F.Rey,D.Romero,G.Gual,L.Costa,A.German,Surface electromyo-graphyapplicationsinthesport,ApuntsMed.L’Esport45(2010)127–136. [3]G.Shan,P.Visentin,EMGApplicationsinStudiesofArts,INTECHOpenAccess

Publisher,2012.

[4]J.P.Clarys,Electromyographyinsportsandoccupationalsettings:anupdateof itslimitsandpossibilities,Ergonomics43(10)(2000)1750–1762.

[5]S.N.Kale,S.V.Dudul,IntelligentnoiseremovalfromEMGsignalusingfocused time-laggedrecurrentneuralnetwork,Appl.Comput.Intell.SoftComput.2009 (2009)1.

[6]M.Hussain,M.Reaz,M.I.Ibrahimy,N.Mastorakis,V.Mladenov,Z.Bojkovic,D. Simian,S.Kartalopoulos,A.Varonides,C.Udriste,etal.,SEMGsignalprocessing andanalysisusingwavelettransformandhigherorderstatisticstocharacterize muscleforce,in:WSEASInternationalConference.Proceedings.Mathematics andComputersinScienceandEngineering,No.12,WSEAS,2008.

[7]K.Yana,H.Marushima,H.Mine,N.Takeuchi,Bispectralanalysisofﬁltered impulseprocesseswithapplicationstotheanalysisofbioelectricphenomena, in:Workshop on Higher-Order Spectral Analysis, 1989, IEEE, 1989, pp. 140–145.

[8]A.O.Andrade,S.Nasuto,P.Kyberd,C.M.Sweeney-Reed,F.VanKanijn,EMG sig-nalﬁlteringbasedonempiricalmodedecomposition,Biomed.SignalProcess. Control1(1)(2006)44–55.

[9]M.Taghizadeh-Sarabi,M.R.Daliri,K.S.Niksirat,Decodingobjectsofbasic cat-egoriesfromelectroencephalographicsignalsusingwavelettransformand supportvectormachines,BrainTopogr.28(1)(2014)33–46.

(8)

[10]R.H.Chowdhury,M.B.Reaz,M.A.B.M.Ali,A.A.Bakar,K.Chellappan,T.G.Chang, Surfaceelectromyographysignalprocessingandclassiﬁcationtechniques, Sen-sors13(9)(2013)12431–12466.

[11]P. Baldi, Autoencoders, unsupervised learning, and deep architectures, Unsupervised and Transfer Learning Challenges in Machine Learning 7 (2012)43.

[12]L.Wei,H.Hu,K.Yuan,Useofforeheadbio-signalsforcontrollinganintelligent wheelchair,in:IEEEInternationalConferenceonRoboticsandBiomimetics, 2008.ROBIO2008,IEEE,2009,pp.108–113.

[13]A.Subasi,M.Yilmaz,H.R.Ozcalik,ClassiﬁcationofEMGsignalsusingwavelet neuralnetwork,J.Neurosci.Methods156(1)(2006)360–367.

[14]A.DelBoca,D.C.Park,Myoelectricsignalrecognitionusingfuzzyclusteringand artiﬁcialneuralnetworksinrealtime,in:1994IEEEInternationalConference onNeuralNetworks,1994.IEEEWorldCongressonComputationalIntelligence, vol.5,IEEE,1994,pp.3098–3103.

[15]M.Khezri,M.Jahed,Aneuro-fuzzyinferencesystemforSEMG-based identi-ﬁcationofhandmotioncommands,IEEETrans.Ind.Electron.58(5)(2011) 1952–1960.

[16]G.R.Naik,D.K.Kumar,M.Palaniswami,Multirunicaandsurfaceemgbased signalprocessingsystemforrecognisinghandgestures,in:8thIEEE Interna-tionalConferenceonComputerandInformationTechnology,2008.CIT2008, IEEE,2008,pp.700–705.

[17]M.R.Ahsan,M.I.Ibrahimy,O.O.Khalifa,etal.,EMGsignalclassiﬁcationfor humancomputerinteraction:areview,Eur.J.Sci.Res.33(3)(2009)480–501. [18]P.Konrad,TheABCofEMG:APracticalIntroductiontoKinesiological

Elec-tromyography,Version1.0,Noraxon,2005.

[19]I.Rodríguez-Carre ˜no,L.Gila-Useros,A.Malanda-Trigueros,Motorunitaction potentialduration:measurementandsigniﬁcance,Adv.Clin.Neurophysiol. (2012)133.

[20]ThalmicLabs,MyoUserGuide,2015https://developer.thalmic.com/docs/api reference/platform/getting-started.html.

[21]L.Zhang,M.Yang,X.Feng,Sparserepresentationorcollaborative representa-tion:whichhelpsfacerecognition?in:2011IEEEInternationalConferenceon ComputerVision(ICCV),IEEE,2011,pp.471–478.

[22]J.Wright,A.Y.Yang,A.Ganesh,S.S.Sastry,Y.Ma,Robustfacerecognition viasparserepresentationIEEETrans.PatternAnal.Mach.Intell.31(2)(2009) 210–227.

[23]L.Zhang,M.Yang,X.Feng,Y.Ma,D.Zhang,Collaborativerepresentationbased classiﬁcationforfacerecognition,arXiv:1204.2358.

[24]D.G. Manolakis, V.K. Ingle, S.M. Kogon, Statistical and Adaptive Signal Processing:SpectralEstimation,SignalModeling,AdaptiveFiltering,andArray Processing,vol.46,ArtechHouseNorwood,2005.

[25]Z.Zhang,B.D.Rao,Recoveryofblocksparsesignalsusingtheframeworkofblock sparseBayesianlearning,in:2012IEEEInternationalConferenceonAcoustics, SpeechandSignalProcessing(ICASSP),IEEE,2012,pp.3345–3348.

[26]J.Huang,T.Zhang,D.Metaxas,Learningwithstructuredsparsity,J.Mach.Learn. Res.12(2011)3371–3412.

[27]A.Boyali,N.Hashimoto,Block-sparserepresentationclassiﬁcationbased ges-turerecognitionapproachforaroboticwheelchair,in:IntelligentVehicles SymposiumProceedings,2014IEEE,IEEE,2014,pp.1133–1138.

[28]A.Boyali,N.Hashimoto,O.Matsumoto,Asignalpatternrecognitionapproach formobiledevicesanditsapplicationtobrakingstateclassiﬁcationonrobotic mobilitydevices,Robot.Auton.Syst.(2015),Elsevier.

[29]L.Harvill,AppliedMatrixAlgebra,Xlibris,2011.

[30]H.Karner,J.Schneid,C.W.Ueberhuber,Spectraldecompositionofrealcirculant matrices,LinearAlgebraAppl.367(2003)301–311.

[31]D.Bini,Toeplitzmatrices,algorithmsandapplications,ERCIMNews22(1995). [32]J.F.Henriques,R.Caseiro,P.Martins,J.Batista,High-speedtrackingwith ker-nelizedcorrelationﬁlters,IEEETrans.PatternAnal.Mach.Intell.37(3)(2015) 583–596.

[33]H.Hassani,Singularspectrumanalysis:methodologyandcomparison,J.Data Sci.5(2007)239–257.

[34]Y.Saad,NumericalMethodsforLargeEigenvalueProblems,vol.158,SIAM, 1992.

[35]A.Boyali,M.Kavakli,Arobustgesturerecognitionalgorithmbasedonsparse representation,randomprojectionsandcompressedsensing,in:20127thIEEE ConferenceonIndustrialElectronicsandApplications(ICIEA),IEEE,2012,pp. 243–249.

[36]A.Boyali,N.Hashimoto,O.Matsumato,Handposturecontrolofarobotic wheelchairusingaleapmotionsensorandblocksparserepresentationbased classiﬁcation,in:SMART2014,theThirdInternationalConferenceonSmart Systems,DevicesandTechnologies,2014,pp.20–25.

[37]E. Elhamifar, R. Vidal, Sparse subspace clustering: algorithm, theory, and applications, IEEE Trans.Pattern Anal. Mach. Intell. 35 (11) (2013) 2765–2781.

[38]G.Liu,Z.Lin,S.Yan,J.Sun,Y.Yu,Y.Ma,Robustrecoveryofsubspacestructures bylow-rankrepresentation,IEEETrans.PatternAnal.Mach.Intell.35(1)(2013) 171–184.

[39]S. Tierney, J. Gao, Y. Guo, Subspace clustering for sequential data, in: Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2014, pp.1019–1026.

[40]A.Boyali,N.Hashimoto,O.Matsumoto,Paradigmshiftincontinuoussignal patternclassiﬁcation:mobilerideassistancesystemfortwo-wheeledmobility robots,arXiv:1506.04810.