ContentslistsavailableatScienceDirect
Journal
of
Neuroscience
Methods
j ou rn a l h o m epa g e :w w w . e l s e v i e r . c o m / l o c a t e / j n e u m e t h
Computational
Neuroscience
EEG
artifact
elimination
by
extraction
of
ICA-component
features
using
image
processing
algorithms
T.
Radüntz
a,∗,
J.
Scouten
a,
O.
Hochmuth
b,
B.
Meffert
baFederalInstituteforOccupationalSafetyandHealth,MentalHealthandCognitiveCapacity,Nöldnerstr.40-42,10317Berlin,Germany bHumboldt-UniversitätzuBerlin,DepartmentofComputerScience,RudowerChaussee25,12489Berlin,Germany
h
i
g
h
l
i
g
h
t
s
•Machine-drivenEEGartifactremoval throughautomatedselectionofICsis proposed.
•Feature vectors extracted from IC viaimageprocessingalgorithmsare used.
•LDAclassificationidentifiesrange fil-teraspowerfulfeature(accuracyrate 88%).
•The method does not depend on directrecordingofartifactsignals. •Themethodisnotlimitedtoaspecific
numberofartifacts.
g
r
a
p
h
i
c
a
l
a
b
s
t
r
a
c
t
a
r
t
i
c
l
e
i
n
f
o
Articlehistory: Received17July2014
Receivedinrevisedform26January2015 Accepted28January2015
Availableonline7February2015 MSC: 68-04 68U10 68U99 68T10 92C50 92C55 Keywords:
Independentcomponentanalysis ICA
EEG
Artifactelimination Imageprocessing Localbinarypatterns Rangefilter Geometricfeatures
a
b
s
t
r
a
c
t
Artifactrejectionis acentralissue whendealing withelectroencephalogram recordings. Although independentcomponentanalysis(ICA)separatesdatainlinearlyindependentcomponents(IC),the classificationofthesecomponentsasartifactorEEGsignalstillrequiresvisualinspectionbyexperts.
Inthispaper,weachieveautomatedartifacteliminationusinglineardiscriminantanalysis(LDA)for classificationoffeaturevectorsextractedfromICAcomponentsviaimageprocessingalgorithms.
Wecomparetheperformanceofthisautomatedclassifiertovisualclassificationbyexpertsandidentify rangefilteringasafeatureextractionmethodwithgreatpotentialforautomatedICartifactrecognition (accuracyrate88%).Weobtainalmostthesamelevelofrecognitionperformanceforgeometricfeatures andlocalbinarypattern(LBP)features.
Comparedtotheexistingautomatedsolutionstheproposedmethodhastwomainadvantages:First, itdoesnotdependondirectrecordingofartifactsignals,whichthen,e.g.havetobesubtractedfromthe contaminatedEEG.Second,itisnotlimitedtoaspecificnumberortypeofartifact.
Insummary,thepresentmethodisanautomatic,reliable,real-timecapableandpracticaltoolthat reducesthetimeintensivemanualselectionofICsforartifactremoval.Theresultsareverypromising despitetherelativelysmallchannelresolutionof25electrodes.
©2015TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).
∗ Correspondingauthor.Tel.:+4930515484418.
E-mailaddress:[email protected](T.Radüntz). URL:http://www.baua.de(T.Radüntz).
http://dx.doi.org/10.1016/j.jneumeth.2015.01.030
1. Introduction
Theelectroencephalogram(EEG)asamulti-channelsignalof neuronalbrainactivitycanreflectbrainstateslinkedtothemental conditionofaperson.Duetoitstemporalresolution,itisan excel-lentandwidelyusedtechniqueforinvestigatinghumanbrain func-tioning.AmajorproblemthoughisthecontaminationoftheEEG signalbyvariousphysiologicalandnon-biologicalartifactssuchas eyemovements,blinks,muscleactivity,heartbeat,highelectrode impedance,linenoise,andinterferencefromelectricdevices.
ThediscardingofentireEEGsegmentsduetonoiseisawidely appliedmethodinresearchsettingsandresultsinthelossof exper-imentaldata.Thisbecomes especiallyproblematicifonlya few epochsareavailable andartifactssuchasblinksor movements occur toofrequently. Moreover, this approach is inappropriate when working withthe continuous EEG and non-event-locked activity(Vanhataloetal.,2004),long-rangetemporalcorrelations (Linkenkaer-Hansenetal.,2001),real-timebrain–computer inter-face(BCI)applications,andonlinementalstatemonitoring(Jung etal.,2000).Otherproposedmethodsforartifactrejectionarebased onregressioninthetimeorfrequencydomain(Kenemansetal., 1991).Theyconcentratemainlyonremovingocularartifacts(Jung etal.,2000),maythemselvesintroducenewartifactsintotheEEG recording(WeertsandLang,1973;OsterandStern,1980;Peters, 1967),andareunsuitable forreal-timeapplications(Jungetal., 2000).AreviewofBCI-systemartifactreductiontechniquesisgiven byFatourechietal.(2007).
Apromisingmethodwhichhasestablisheditselfasan impor-tant part of EEG analysis is the application of independent componentanalysis(ICA)fordatadecomposition(Jungetal.,2000; Makeigetal.,1996)andseparationofneuronalactivityfrom arti-facts(Fitzgibbonetal.,2007;Romeroetal.,2008).Theideacentral tothismethodisthattheEEGsignalisamixtureoflinearly indepen-dentsourcecomponents(IC)thatcanbeseparatedbyICA,visually examined,andclassifiedasartifactorEEGsignalcomponents.Once theartifactcomponentshavebeenidentified,theycanberemoved andtheremainingEEGsignalcomponentscanbeprojectedbackto theoriginaltimedomain.Thisprocedureyieldsthereconstruction ofanartifact-freeEEG.
TheexaminationandclassificationoftheICsistime-consuming andrequirestheratertohaveexperienceandknowledgefor deci-sionmaking.Viola etal. (2009)and Mognonetal.(2010) both workedonmitigatingthisdifficultybydevelopinganEEGLAB plug-inforfindingartifactICs.Thefirstreliesonauser-definedtemplate whilethesecondiscompletelyautomaticbutlimitedtofour arti-facts(Jungetal.,2000;Mognonetal.,2010).
However,EEGdatacontainanarrayofartifactswithunknown properties.Hence,thereisafurtherneedtodevelopfully auto-matedmachineclassification,e.g.oftheindependentcomponents, forautomaticartifacteliminationthatcandealwithallkindsof artifacts.
Theideaunderlyingourmethodisinspiredbytheobservation ofexpertsduringvisualclassification.Forthisprocesstheyvisually inspectthe2DscalpmapprojectionsoftheICs,calledtopoplots. Basedonthetopoplots’imagepatternandontheexperts’ know-how,theydecideiftheICisanartifactoranEEGsignalcomponent. Computervisionaimsatimitatingtheabilitiesofhumanvisionby electronicallyperceivingandunderstandinganimageforfurther decisionmaking.Hence,weintroduceinthispaperanewmethod forartifact rejectionbased onmachine classificationoffeatures derivedfromtopoplotimagesbyapplyingimageprocessing algo-rithmsforimprovingperformance.Imagefeaturescanbeextracted throughrangefiltering,localbinarypatterns(LBP)thatare orig-inally used as texture features, and geometric approaches like Gaussiancurvature.Thesemethodsarecommonlyusedinthe con-textof 2Dobject recognitionand include information fromthe entireimage.Combinedwithaclassificationmethodsuchaslinear discriminantanalysis(Moghaddametal.,2000),theycanenhance the recognitionperformance of automated artifact elimination. Thisgroundbreakingmethoddoesnotdependondirectrecording ofartifactsignals,neitheritislimitedtoaspecificnumberortypeof artifact.Anadditionaladvantageofthisnovelmethodisits adapt-abilitytoanarbitrarynumberofEEGchannelconfigurations.This isduetothenatureofthetopoplots,whicharegeneratedby inter-polatingtheICAmixingmatrixcolumnsontoafixed-sizegrid(in ourcase:51×63).Thisinterpolationstepenablestheprojection ofdifferentnumbersofEEGchannelsanddifferentEEGchannel positionsonthesamegrid,preciselybygeneratingimagesof iden-ticaldimension.Thispermitstheclassificationofimagesderived fromICscomposedofvaryingcolumnlengthsofthemixingmatrix, independentofnotonlythenumberbutalsothepositionofthe channelsutilized.Re-trainingtheclassifierforanyfurther investi-gationsusinganyothernumberofchannelsoranyotherchannel positionsisthereforeredundant.Thiscanberecognizedbylooking atFig.1.ItclearlyillustratesthepatternsimilarityofICimagesper artifactdespitetheirdifferentEEGchannelconfigurations. 2. Materialsandexperiments
ThedataacquisitiontookplaceintheshieldedlaboftheFederal InstituteforOccupationalSafetyandHealthinBerlin.The exper-imentwasfullycarriedoutwitheach subjectinasingleday.It
Fig.1.ExamplesofsimilarICartifactpatternsdespitethedifferentEEGchannelconfigurations.Firstrow:ICimagesfroma25channelconfiguration;secondrow:ICimages froma30channelconfiguration.
Fig.2.Processingpipeline.
consistedoftwocomponents:atrainingphaseandthemain exper-iment.Duringthetrainingphasesubjectswerefamiliarizedwith thecognitivetasks,whichwereidenticaltothemainexperiment butwithshorterduration.AnEEGsignalwasnotrecordedatthis stage.Themainexperimentstartedafterashortbreaksubsequent tothetrainingphase.TheEEGwascapturedduringthemain exper-imentby25electrodesarrangedaccordingtothe10–20system. Data wererecorded withreference toCz and ata sample rate of500Hz.Forsignalrecordingweusedanamplifierfrom Brain-ProductsGmbHandtheirBrainRecordersoftware.Thetaskswere presentedinacounterbalancedorderandwerecontrolledremotely bymeansofaremotedesktopconnection,anintercommunication system,andavideomonitoringsystem.
Thesampleconsistsof57peopleinpaidworkandshowshigh variability with respect to cognitive capacity and age. Table 1 describesthesamplesetused.
Thevariouscognitivetaskswererealizedthroughthe imple-mentation of a task battery in the E-Prime application suite (PsychologySoftwareTools,Pittsburgh,PA).Thebatteryconsists ofninetasks withdiversecomplexity anddifficultythatreflect thefollowingexecutivefunctions:shiftingbetweentasks,updating andmonitoringworkingmemoryrepresentations,andinhibitionof dominantresponses.TheimplementedtasksarelistedinTable2 andexplainedingeneralinWikipedia(2014a,b,c,d)andUnsworth etal.(2005).Furthermore,we recordedtheEEGduringa short periodofrelaxationatthebeginningandattheendofthemain experiment.
Table1
Sampleset.
Age Female Male Total
30–39 7 7 14 40–49 12 13 25 50–59 9 2 11 60–62 3 4 7 Total 31 26 57 3. Methods
WeimplementedaMATLABtoolboxconsistingofthreemain
modulesforpre-processing,featuregeneration,andclassification.
To train the classifier, we used visualratings of the topoplots
fromtwoexperts.Thealgorithmthenautomaticallyclassifiedthe
ICs, based either onthe new generated image features or the
post-processedtopoplots.Artifactcomponentsarediscardedfrom
subsequentprocessing.ComponentsclassifiedasEEGcomponents
areprojectedback,andinthiswayreconstructanartifact-freeEEG
signal.Fig.2summarizesthisprocessingpipelineandisdescribed
in-depthinthefollowing. 3.1. Pre-processing
AllrecordedEEGsignaldatawereimportedintoMATLAB (Math-Works,R2012b).Inourcase,the25recordedEEGchannelswere savedinonefilepertaskandperperson.Thesignalswere mul-tiplied with a Hamming window function and filtered with a bandpassfilter(order100)between0.5and40Hz.Subsequently, independentcomponentanalysis(Infomaxalgorithm)wasapplied tothe25-channelEEGsignal(Makeigetal.,1996;Delormeand Makeig,2004):
W·x=u (1)
withxisthesignalofscalpEEGchannels,Wtheunmixingmatrix anduthesources(ICs).
Inthatway,ourmulti-channeldataweredecomposedinto25 ICs.Eachofthe25-elementcolumnvectorsofthe25×25ICA mix-ingmatrix W−1 wasinterpolatedonto a 51×63 gridusingthe inversedistancemethodofSandwell(1987).Theyformthe25 two-dimensionalscalpprojectionmaps,herereferredtoastopoplots. ThefirstplotontheleftofFig.3(a)illustratestheinterpolationona fine,rectangularCartesiangrid.Examplesofsuchimageswithout thebackgroundaredepictedinFig.5,whereasetoftopographic mapsofthescalpdatafieldarepresentedina2Dcircularview.The 25computedICsarecharacterizedbytheiractivations(thesources’
Table2
Taskbatteryreflectingexecutivefunctions.
Task 0-Back 2-Back Sternberg SerialSternberg Stroop
Duration(min) 5 5 10 10 5
Task SwitchPAR SwitchNUM SwitchXXX AOSPAN Relaxation(start,end)
Duration(min) 5 5 10 20 3+3
timecourse),theiractivitypowerspectrumcalculatedwithFFT,
andtheir2Dscalpcomponentmaps(interpolatedfromthecolumns
ofthemixingmatrix).
Alltopoplotsin combination withtheirpowerspectrawere
visuallyinspectedandratedbytwoexpertsaseitheranartifactor
EEGsignalcomponent(Fig.5).Altogether,15625topoplotswere
labeled,46%assignalvs.54%asartifact.Eachoftheexpertsrated halfofthetopoplots.Theselabelswereusedlaterforsystem train-ingandclassification.
Becauseofthetime-consumingprocedureandthelarge num-beroftopoplotstoberated,itwasimpossibletofindmorewilling expertsforlabeling.Thisfactillustratesoncemorethenecessity ofanautomatedsolution.However,wecomputethecorrelationof ourexperts’labelingonasub-sampleof5500topoplotsthatwere ratedbyboth.Theyachievedanagreementof92.8%.
3.2. Featuregeneration
Thefeaturegenerationmoduleconsistsofalgorithmsforimage featureextraction.Furthermore,itcomprisespost-processing algo-rithms that can be applied to the generated features but also directlytothetopoplotimages.Post-processingalgorithmsaimto reducecomputationaleffortbasedonimagedownsamplingand vectorization.Inthisway,theycontributetoenhancing classifica-tionperformance.
3.2.1. Imagefeatureextraction
Imagefeatureextractiongeneratesfeaturesthat canbe gen-erallycategorizedinthreegroups:texturefeatures,gradientand rangefeatures,andgeometricfeatures.
3.2.1.1. Localbinarypattern(LBP). LBPisapowerfulfeaturefor2D textureclassificationfirst described in1994 (Ojala etal., 1994) andrefinedlaterinOjalaetal.(1996).TheLBPfeaturevaluesare
computedbybuildingthedifferencesbetweeneachpixelandits neighborsP∈{8,16,24}withinaprescribedradiusR.Ifthecenter pixelisgreaterthanitsneighborwenotea1,elsea0,sothatthe relationtotheneighborsisbinarycoded.InthecaseofP=8 neigh-bors,weobtainaneight-digitbinarynumberthatcanbeconverted intoadecimalnumber.Thisconstitutesthenewvalueofthecentral pixel.
Arotation-invariantalternativeoftheLBPmethodisdescribed byOjalaetal.(2002).Here,thedigitsofthebinarynumberare shifteduntilaminimalvalueiscreated.Anothervariantisthe uni-formLBP,wherethresholdingisusedtoreplacerarefeaturevalues withasinglevalue.Acombination ofboth resultsin arotation invariant,uniformLBPfeature.
Huangetal.introducedthe3DLBPmethod,whichcomputes fourdistinctLBPlayers(Huangetal.,2006).Herecodingisdone notonlyforwhetherthecentralpixelislessorgreaterthanits neighborbutalsoforthevalueofthedifferencesthemselves.Ina fourbitwordthesignofthedifferenceistrackedinthefirstbit anditsvalueintheremainingthreebits.IneachLBPlayer(sign layer,firstbitlayer,secondbitlayer,thirdbitlayer)wesetthe correspondingdecimalvalueatthepositionofthecentralpixel.
WeimplementedandtestedallLBPvariantsdescribedhereon thetopoplotimagesandpresentedtheresultsinSection4.Some examplesoftheLBPfeatureimagesarepresentedinFig.3(a)and (b).
3.2.1.2. Gradientimages,rangefilterandLaplacian. Wealso evalu-atedtherecognitionperformanceofseveralgradientfeatures.The horizontalSobeloperator(HSO)isobtainedbyspatialconvolution withthefiltermask
HSO=
⎛
⎝
−1−2 00 12 −1 0 1⎞
⎠
(2)Fig.3. Examplesoffeatureimagesgeneratedfromthetopoplotimage(topleft)usingimageprocessingalgorithmsintroducedinthemaintext:(a,b)texturefeaturesand (c)gradient,range,andgeometricfeatures.
anddetectsverticaledges.Tocomputethehorizontalderivative, typicallya2×1kernelisappliedthatcalculatesthedifferenceof twopixels.Thelargehorizontalgradientoperator(LHG)detects horizontalrange differences over a greater horizontaldistance. Heseltineetal. testeda varietyof imageprocessingtechniques forfacerecognitionandidentifiedtheLHGoperatorasthemost effective surface representation and distance metric for use in applicationareassuchassecurity,surveillance,anddata compres-sion(Heseltineetal.,2004).Inconsiderationofthis,wedecidedto usetheirrecommendedsizeof5andcomputetheLHGviathefilter mask
LHG=
−1 0 0 0 1 (3)We also tested the range filter, which takes the difference betweenthemaximalandminimalvalueforeachtopoplotimage’s pixelina3×3neighborhood(BaileyandHodgson,1985).In addi-tion,weappliedtheLaplacianoperatorasasecondorderfeature.
Fig.3(c)illustratesexamplesforgradientimages,rangefiltering andtheLaplacianoperator.
3.2.1.3. Gaussiancurvature. TheGaussiancurvaturebelongstothe geometricfeaturesandisdefinedasfollows.ThecurvatureKofa pointonasurfaceistheproductoftheprincipalcurvaturesk1and
k2: K=k1·k2= 1 r1· 1 r2 (4) withmaincurvatureradiusr1andr2.
WeimplementedtheGaussiancurvatureofthetopoplot sur-facepointsasafurtherfeatureandtesteditsclassificationaccuracy (Horn,1984).AnexampleisshowninFig.3(c).
3.2.2. Post-processing
Eachfeatureimageconsistsof51×63=3213pixels.Wecropped theinnerareacontainingthescalpmapanddiscardedtheframe, thusobtaining 1489pixels(approx. 46%of theoriginal feature image).Wecontinueddownsamplingtheimagebyselectingeither eachsecondpixelinhorizontalandverticaldirection(4:1),each third(9:1),oreachfourth(16:1)pixel.Inthisway,wegenerated featureimageswith372pixels for the4:1alternative (approx. 11.58%oftheoriginalfeatureimage),164pixelsforthe9:1 alterna-tive(approx.5.10%oftheoriginalfeatureimage)and94pixelsfor the16:1alternative(approx.2.93%oftheoriginalfeatureimage). Themainpurposeofdownsamplingwastoinvestigatehowmuch pixelinformationisstillneededforachievingreasonablygood clas-sificationresultsandtopossiblyreducecomputationaleffortfor theclassification.Forthesakeofconveniencewetransformedthe imagematrixinafeaturevectorforfurtherclassification.
3.3. Classification
Weusedlineardiscriminantanalysis(LDA)asaneasily compre-hensibleandreproduciblemethodforclassification.LDA,awidely usedmethodforfacerecognition(Kongetal.,2015),imageretrieval (SwetsandWeng,1996),textclassification(Yeetal.,2004),etc.was appliedtoallcomputedfeatures.LDAminimizesthewithin-class varianceSWwhilemaximizingthebetween-classvarianceSB by
computationoftheprojectionmatrixWLDA.
SW= C
i=1 li n=1 (xin−mi)·(xin−mi)T (5) SB= C i=1 li·(mi−m)·(mi−m)T (6)withCisthenumberofclasses,mithemeanofclassiandlithe
numberofsamplesinclassi.HencetheratioofSBtoSWhastobe
maximizedformaximalseparabilityoftheclasses: WLDA=argmaxW
WT·S B·W
WT·SW·W
(7)
Incontrasttoprincipalcomponentsanalysis(PCA),LDAaimsto maximizetheclass-discriminatoryinformation.Hence,itprovides class separability and a decision boundarybetween theclasses bycomputingtheaxes’directionsthatmaximizingtheseparation betweenthem.Euclideandistanceisusedtoclassifythedata.
Awell-knownchallengeofLDAformanyapplicationsisthat itrequiresthewithin-classcovariancematrixtobenonsingular, whichisnotalwaysgiven.Inordertoovercomethepossible singu-larityofSW,severalapproacheshavebeenproposed(Swetsand
Weng,1996; Dai and Yuen, 2003; Kong et al., 2015)including computationofthepseudo-inverseLDA(RaudysandDuin,1998; SkurichinaandDuin,1996).Thelatterapproachwasusedforthis workwheneverthecomputationoftheinversefailed.
Altogether,thedataconsistof625sets(subjects×tasks)with25 topoplotsineachset.Werandomlyselected60%ofthesetsforthe trainingandtheremaining40%fortesting.Hence,foreachfeature generationmethodappliedweused9375imagefeaturevectors (375sets)fortrainingtheclassifierand6250imagefeaturevectors (250sets)fortestingit.Inadditiontothefeaturesgenerated,we alsotrainedandclassifiedthetopoplotsthemselves.
4. Results
Weevaluatedourartifactremovalalgorithmbymeansofaclose inspectionoftheclassificationresults,thesignal-to-noiseratioand theperformance time.While thefirst twoare standard forthe assessmentofperformance,theperformancetimingaccountsfor thereal-timefeasibilityofthealgorithm.
4.1. Classificationresults
Determinationoftheminimum numberof eigenvectorswas doneforeachfeatureandsamplingrate.Themainpurposewas toreducecomputationaleffort.Theidentificationofthisminimum wasdoneempiricallyusingthereceiveroperatingcharacteristic (ROC)curve(Fig.4)whichillustratesquitewelltheperformanceof abinaryclassifiersystemasitsdiscriminationthresholdisvaried. Hence,itprovidesawaytoselectpossiblyoptimalmodelsandto discardsuboptimalones.BasedonourROCanalysisofdiagnostic decisionmaking,acut-offof70%ofthetotalnumberof eigenvec-tors,sortedindescendingorderbythesizeoftheireigenvalues,was setasthreshold.Theentireclassificationprocedurewithrandom selection,trainingandtestingwascrossvalidatedbyexecutingit 50timesforeachfeature.Resultswereaveragedandthevariance wascalculated(Table3).
Tofurthervalidateourresults, wecomparedtherecognition ratesfromtheLDAwith70%ofthetotalnumberofeigenvectors (Table3)withtheclassificationresultsfromanLDAclassifierwith alleigenvectors.Theextremelysmalldifferences(intherangeof 0–0.7%)intherecognitionratesbetweenthetwosystemssuggest theuseofthe70%-classifierinfavorofaminimalcomputingtime, whichisalwaysadvantageouswhendealingwiththedevelopment ofreal-timesystems.
Therecognitionratesforfeatureimagesvarybetween73%for thefourthlayerof3DLBPand88%forrangefilter.Downsampling ofthefeatureimagesdoesnothavealargeimpactonmostofthe features’accuracyrates.LookingatTable3wenoticeasexpected that recognitionratesdecline differently. Theydecline leastfor thetopoplots,therangefilterandtheGaussiancurvature.They
Fig.4. Dependenceofrecognitionratesonamountofeigenvectorsusedforeachofthe15features(subplots)andthefoursamplingrates(blue1:1,green4:1,red9:1,cyan 16:1);theverticallineindicatesthecut-offby70%oftotalnumberofeigenvectors(eigenvectorsaresortedbythesizeoftheireigenvalues).
declinemostfortheuniform,rotationinvariantLBPcomputedfor 16neighborsandfortherotationinvariantLBPcomputedforeight neighbors,bothsufferingfromdownsamplingontheorderof4–8%. ThisappearsreasonablewhenlookingatFig.3(b).
Additionally,byapplyingdimensionalityreduction,wenotonly triedtoreducethecomputationalcostsofclassificationbutalso tested for over-fitting. In general,dimensionalityreduction can helpminimizetheerrorinparameterestimationduetoredundant information(thecurseofdimensionality,alsoknownastheHughes effectinmachinelearning).
Forimprovingclassificationwealsomixeddifferentkindsof fea-tures(e.g.3DLBP(1st/2nd)andLaplacian,LaplacientandGaussian curvature,rangefilterandgradients,etc.)buttheresultsdidnot outperformthe88%recognitionrateofrangefilteralone.
On a finalnote, weconsideredbenchmarking ouralgorithm withanexistingremovalmethodimplementedinEEGLAB.Two
ofthebest-knownEEGLABimplementationsforartifactremoval are ADJUST (Mognonet al., 2010) and CORRMAP (Viola et al., 2009).ADJUST distinguishesbetween stereotypedartifacts (e.g. oculareyemovements,blinks,heartbeats)andnon-stereotyped artifacts(e.g.movements,externalsources).Theauthorsnotein their tutorial(Buiatti and Mognon, 2014) that ADJUST aims to automaticallyidentifyandremovecomponentsofthestereotyped artifacts, while the non-stereotyped artifacts are problematic: “ADJUST doesnot attemptto remove these artifactsand relies ona suitable pre-processing forremoving them beforetheICA decomposition”(BuiattiandMognon,2014).Theyadvisetheusers to “remove from the continuous data all segments containing paroxysmal artifacts” (Buiatti and Mognon, 2014), after filter-ing. Due to this pre-processing step of segments’ removal we deemed ADJUST as inappropriate for benchmarking. We then switchedovertoCORRMAP,ranitinautomaticmode,andmanually
Table3
Recognitionratesoffeaturestested:mean(%)over50cross-validatedresultsandvariance(inparentheses).
Sampling 1:1 4:1 9:1 16:1 Eigenvectors 1042 260 114 65 Features Topoplots 86.6(0.16) 86.6(0.17) 86.5(0.17) 86.7(0.19) 3DLBP(1st) 78.4(0.20) 78.0(0.16) 78.1(0.16) 77.0(0.25) 3DLBP(2nd) 85.1(0.14) 84.9(0.16) 85.3(0.17) 84.6(0.16) 3DLBP(3rd) 80.3(0.12) 80.1(0.11) 80.4(0.12) 79.8(0.11) 3DLBP(4th) 74.5(0.21) 74.0(0.18) 74.2(0.22) 73.5(0.22)
LBProtationinvariant,8neighbors 79.3(0.25) 76.4(0.23) 74.0(0.14) 71.5(0.28) LBPuniform,8neighbors 78.6(0.16) 78.7(0.18) 78.4(0.15) 77.8(0.17) LBPuniform,16neighbors 79.3(0.21) 79.4(0.22) 79.0(0.19) 78.3(0.25) LBPuniform,rotationinvariant,8neighbors 78.3(0.20) 78.0(0.22) 78.0(0.20) 77.1(0.22) LBPuniform,rotationinvariant,16neighbors 80.1(0.15) 79.1(0.14) 79.1(0.19) 77.6(0.27)
LHG 87.9(0.10) 87.7(0.11) 87.5(0.14) 87.6(0.09)
HSO 87.8(0.11) 87.7(0.13) 87.5(0.10) 87.4(0.12)
Rangefilter 88.0(0.10) 88.1(0.09) 88.0(0.11) 87.8(0.11)
Laplacian 87.9(0.11) 87.6(0.10) 87.0(0.12) 86.8(0.14)
Gaussiancurvature 86.3(0.13) 86.2(0.18) 86.0(0.16) 86.1(0.15) Recognitionratesgreaterthan88%inbold.
Fig.5.ExampleofacompleteICsetof25topoplotswithoneexpert’slabelingprintedonthetopofeachtopoplot.Divergingmachineratingsarecircledingray.
providedartifacttemplatesasrequestedbythesystem.Wetested
250ICsandachieveda62%agreementwiththeexperts’ratings.
TheCORRMAPresultsarenotsurprisingduetoCORRMAP’sfocus
oneye blinksandlateraleyemovements,asmentioned bythe
authors(Violaetal.,2009).CORRMAPsimilartoADJUSTis
engi-neeredtocapturein particularstereotypedartifactslikeblinks, eyemovementsand/orheartbeats.Theiracknowledged lackof abilitytocapturenon-stereotypedartifactsisachallengethatwe accept.
4.2. Signal-to-noiseratio
After automated classification, the components classified as artifact were discarded from the subsequent process. The remaining ones, classified as signal components, were back projected. Theresult is themachine de-noised signal.A repre-sentative complete IC set of 25 topoplots is shown in Fig. 5. The expert’s labeling is printed above each topoplot. Diverg-ing machine ratings are circled in gray, e.g. the image in the third row,third column was classifiedby themachine as arti-fact. For the sake of completeness, we should mention that our experts used the activity power spectrum as additional information fortheclassification procedure. Anexample ofthe original band pass filtered signal of electrode Fp1, the expert de-noised signal, and the signal de-noised automatically using range filtering (sampling 9:1) is shown for one subject in Fig.6.
Weusedthesignal-to-noiseratio(SNR)asacriterionto charac-terizethequalityandsimilarityoftheartifactrejectionprocedure whencomparingtheexperts’visualclassificationwiththeimage patternapproach.SNRvalueswerecomputedonthebasisofthe definitioninStrutz(2000). SNR=10·log10
2 x 2 e(dB) (8) with2
xisthevarianceofthesignalande2thevarianceofthenoise.
Forzeromeansignalsasisthecasehere,thisresultsin
SNR=10·log10
N (i=1)x2i
N (i=1)(si−xi) 2 (9)
withNisthenumberofsamplepoints,xithenoisereducedsignal
attimei,andsithebandpassfilteredsignalattimei.
Forthenoisesignalinthedenominator,weemployedthesame valuefortheexpertde-noisedSNREXPandthemachinede-noised
SNRM. Hence,we used as residual noise for both SNR
calcula-tionsthedifferencebetweenbandpassfilteredsignalandexperts’ noisereducedsignal.Underthisassumptiontheperformanceof bothapproachescanbecompared.Thesignalsofeachtaskwere appendedoneafteranother,resultinginonelongsignalforeach person,andtheSNREXPandSNRMwerecomputedforeach
elec-trode.Resultswereaveragedoverthesubjects.
Fig.6.BandpassfilteredsignalofelectrodeFp1(red)withapronouncedeye-blink artifact.Expertde-noisedsignal(blue)andautomaticallyde-noisedsignal(green) byrangefiltering(sampling9:1)withouttheartifact.
Table4
Performancetimesfortheprocessingpipeline.
Pre-processing Bandpassfilter 0.164s
ICA 4.731s
Featuregeneration Topoplot 0.121s
Rangefilter 0.002s
Featurepostprocessing 0.00007s
Classification Training(offline) 1:1(worstcase) 1.7s
16:1(bestcase) 0.025s Testing(online) 1:1(worstcase) 0.03s
16:1(bestcase) 0.002s
Fig.7. SNRmaps(groupaverage)forexpertde-noisingandforthemachinebased de-noisingoffourselectedfeatures(sampling9:1).
TheSNRrangeisverylow,from−9.8to9.8dB,wellwithinthe
rangefoundintheliterature.SNRsare,asexpected,lowerinthe
frontalareas(Goldenholzetal.,2009;MishraandSingla,2013)that
arecontaminatedbyeyeartifacts.ThedifferencebetweenSNREXP
andSNRMisalmostzeroindicatingthatweachievedsimilar
qual-itybetweenexpertbasedandmachinebasedde-noising.SNRscalp mapsforexperts’noisereducedsignalandforde-noisingoffour selectedimagefeatures are shownin Fig.7.They illustratethe similarityofthetime-consuming,manualclassification for arti-factrejectionandtheautomatic,machinede-noisingforallEEG channelsaveragedoversubjectsandtasks.
4.3. Performancetime
We tested our systemonan Intel Core i5-3320M Processor (2.6GHz)with8GBDDR3-SDRAM(2×4GB).Asminimuminput signalforperformancetimetestingofthealgorithmwechosea short25-channel-EEGsignalof94.34s(500Hzsamplerate).The selectionwasmadewithreferencetothefactthattheICA compu-tationformsthebottleneckforthesystem’stimingperformance and depends on thesignal length.According theEEGLAB Wiki tutorial(DelormeandMakeig,2014)findingNstablecomponents requiresmorethankN2datasamplepoints(Ndenotesthenumber
ofEEGchannels,N2 isthenumberofweightsintheICA
unmix-ingmatrix,andkisamultiplierthatincreasesasthenumberof channelsincreases).Hence,inourcasethepre-processingmodule needsapprox.5sforfilteringandperformingICA,whilethe gen-erationofonefeaturetakesonly2.7ms.Systemtraining canbe doneofflineandactuallydoesnotneedtobeconsideredforthe evaluationofreal-timeperformance.However,forthe9375image featurevectorsused,thecomputation timewasquitesmalland variedbetween1.7s(1:1image,worstcase)and0.025sforthe downsampledimage(16:1image,bestcase).Classificationtesting
ofoneimagefeaturevectorneedsin bestcase only0.002sand inworstcase0.03s.Table4outlinestheperformancetimesand demonstratesthatthesystemisreal-timecapable.
Asasidenote,wecallattentiontohardware-level program-ming,whichcanenhancetimingperformanceofthepre-processing module,inparticularbyconsiderationofparallelisminherentin hardwaredesign.
5. Conclusions
Inthispaperwedescribedanovelapproachforrobust, auto-matedEEGartifactrejectioninspiredbycomputervision.Itisbased onmachineclassificationofICsasartifactorEEGsignalvia fea-turesgainedwithimageprocessingalgorithms.Weimplemented, tested,andvalidatedseveralimagefeatures,e.g.rangefilter,local binarypatterns,andgeometricfeatures(Fig.3).Linear discrimi-nantanalysiswasappliedforclassification.Accuracyratesaswell assignal-to-noiseratioswerecalculatedforassessingthefeatures’ achievementpotential.
Existing methods rely either on a user-defined template or arelimited toa fewartifacts(Violaet al.,2009;Mognonetal., 2010; Jung et al., 2000). Practitioners on the other hand well knowthatreal-worldEEGdatacontainanarrayofartifactswith unknownproperties.Ourgoalthereforewastodevelopan auto-matedmethodcapableofdistinguishingbetweenthepureEEG signal and all types of artifacts. One could suggest a simpler approachthatusesjustthecolumnsofthemixingmatrixinsteadof theirinterpolationontoafixed-sizegrid(topoplots).Tojustifyour method,weconductedclassificationtestsutilizingonlythemixing matrixcolumnsandreceivedrecognitionratesof87.7%.Indeed,the resultdoesnotoutperformthe88%recognitionratesofrange fil-terbutsupportsourideaofusingtheinterpolatedimages,which inherentlyutilizethecolumnsofthemixingmatrixtotrainour classifier.Themajordifferencebetweenbothproceduresisthatour approachisuniversal.Thismeans:itisindependentofthenumber ofusedEEGchannels,itperformsforanyEEGchannelpositions,and itdoesnotneedtoberetrainedanymore.Inthispaperwewantto highlightalltheseadditionaladvantagesarisingfromourmethod’s adaptabilitytoanarbitrarynumberofEEGchannelconfigurations thatconstituteitsuniversality.Itisworthpointingoutthatexperts generallyproceedduringvisualclassificationofthetopoplots with-outtakingintoaccountthenumberofEEGchannelsused.Thisfact supportsourapproachofusingthe2Dscalpmapprojectionsof theICsinsteadofthemixingmatrixcolumns.Inshort,thespatial patternoftheimagecontainsinformationcrucialfortheexperts’ decisions.
Currently,weareworkingonevaluatingourmethodbytesting itwithEEGdatafromanotherlab.ThisEEGdatahasadifferent EEGchannelsetup,itisconductedwithanotherEEGregistration system,anditsexperimentalsettingandtasksdifferfromoursas well.Nevertheless,theclassifiertrainedwithourdatawillnotin anywayberetrainedonthenewEEGdata.Specifically,thismeans
thatneitherthelargenumberoftopoplotsnorthetime-consuming procedurefor expertratingsis requiredfor thepurposes of re-training theclassifier.The resultswillbepresented in a future paper.
To conclude, we achieved robust, automatic, and real-time recognitionperformancewithabestrecognitionrateof88%for rangefiltering,whichisverygoodcomparedwiththecurrently existingmethods.Furthermore,theproposedmethodisnotlimited tospecifictypesofartifactsandcanbeappliedonlineoverdifferent experimentsandsubjects.Itisknownthatmanualclassificationby expertscanslightlydifferbetweeneachother(Winkleretal.,2011). Hence,thedifferencesoftheSNRsbetweenexpertandmachine de-noisingareextraordinarilygood.However,formoreaccurate classificationresultsan averageratingbetweenmore than two expertsshouldbeconsideredforsuccessfulsystemtraining. To reducecomputationaleffortdownsamplingcanbeappliedwithout lossofclassificationaccuracy.
Insummary,thepresentedmethodisanautomatic,reliable, real-time capable, and practical tool that reduces the time-intensivemanualselectionofICsforartifactremoval.Theresults arevery promising despitethe relativelysmall channel resolu-tionof25electrodes.Weexpectanimprovementofthealgorithm by combining the image features with frequency information andapplyingnon-linearclassifiers(e.g.supportvectormachine). This will be donein future work and presented in a separate paper.
Acknowledgments
WewouldliketothankDrSergeiSchapkin,DrPatrickGajewski, andProfMichaelFalkensteinforselectionofthebattery’stasks. WewouldliketothankMrLudgerBlankefortechnicalsupport duringthetimingtestsforthetasks.Inaddition,wewouldlike tothankMsXenijaWeißbecker-Klaus,MrRobertSonnenberg,Dr SergeiSchapkin,andMsMarionExnerforgeneraltasktestingand forconductingthelaboratoryexperiments.Furthermore,wewould liketothankMsMarionExnerfordailyoperationalsupportandDr GabrieleFreudeforhervaluablesuggestions.
MoreinformationabouttheprojectwhereourEEGdatawere acquiredcanbefoundunderhttp://www.baua.de/de/Forschung/ Forschungsprojekte/f2312.html?nn=2799254.
References
Bailey D, Hodgson R. Range filters: local intensity subrange fil-ters and their properties. Image Vis Comput 1985;3(3):99–110,
http://dx.doi.org/10.1016/0262-8856(85)90058-7, URL
http://www.sciencedirect.com/science/article/pii/0262885685900587
BuiattiM, Mognon A.ADJUST: an automatic EEG artifact detector basedon the joint use of spatial and temporal features – a tutorial; 2014, URL
http://www.unicog.org/pm/uploads/MEG/TutorialADJUST11.pdf [accessed 19.09.14].
DaiD,YuenP.Regularizeddiscriminantanalysisanditsapplicationtoface recogni-tion.PatternRecognit2003;36(3):845–7.
DelormeA,MakeigS.EEGLAB:anopensourcetoolboxforanalysisofsingle-trial EEGdynamics.JNeurosciMethods2004;134:9–21.
DelormeA, MakeigS.EEGLABwiki;July2014, URLhttp://sccn.ucsd.edu/wiki/ EEGLAB#TheEEGLABTutorialOutline[accessed10.09.14].
FatourechiM,BashashatiA,WardR,BirchG.EMGandEOGartifactsinbrain com-puterinterfacesystems:asurvey.ClinNeurophysiol2007;118(3):480–94.
Fitzgibbon S, Powers D, Pope K, Clark C. Removal of EEG noise and arti-factusingblindsourceseparation.JClinNeurophysiol2007;24(3):232–43,
http://dx.doi.org/10.1097/WNP.0b013e3180556926.
GoldenholzD,AhlforsS,HämäläinenM,SharonD,IshitobiM,VainaL,etal. Mappingthesignal-to-noiseratiosofcorticalsourcesin magnetoencephalog-raphyandelectroencephalography.HumBrain Mapp 2009;30(4):1077–86,
http://dx.doi.org/10.1002/hbm.20571.
HeseltineT,PearsN,AustinJ.Three-dimensionalfacerecognition:afishersurface approach.In:CampilhoA,KamelM,editors.Imageanalysisandrecognition,vol. 3212oflecturenotesincomputerscience.SpringerBerlinHeidelberg;2004.p. 684–91,http://dx.doi.org/10.1007/978-3-540-30126-483[accessed1.09.14].
Horn B. Extended Gaussian images. Proc IEEE 1984;72(12):1671–86,
http://dx.doi.org/10.1109/PROC.1984.13073.
HuangY,WangY,TanT.Combiningstatisticsofgeometricalandcorrelativefeatures for3Dfacerecognition.In:ProceedingsofBritishmachinevisionconference; 2006.p.879–88.
JungT,MakeigS,HumphriesC,LeeT,MckeownM,IraguiV,etal.Removing electroencephalographicartifactsbyblindsourceseparation.Psychophysiology 2000;37(2):163–78.
Kenemans J, Molenaar P, Verbaten M, Slangen J. Removal of the ocular artifactfromtheEEG:acomparisonoftimeandfrequencydomain meth-ods with simulated and real data. Psychophysiology 1991;28(1):114–21,
http://dx.doi.org/10.1111/j.1469-8986.1991.tb03397.x.
KongH,LiX,WangJ-G,TeohEK,KambhamettuC.Discriminantlow-dimensional subspaceanalysisforfacerecognitionwithsmallnumberoftrainingsamples. In:ClocksinWF,FitzgibbonAW,TorrPHS,editors.BMVC.BritishMachineVision Association;2005, ISBN1-901725-29-4, http://www.bibsonomy.org/bibtex/ 2ed5a880a804cbb1335a14747db4609cf/dblp, http://www.bmva.org/bmvc/ 2005/papers/paper-57-165.html, http://dblp.uni-trier.de/db/conf/bmvc/bmvc 2005.html#KongLWTK05
Linkenkaer-Hansen K, Nikouline V, Palva J, Ilmoniemi R. Long-range tem-poral correlations and scaling behavior in human brain oscillations. J Neurosci 2001;21(4):1370–7,arXiv:http://www.jneurosci.org/content/21/4/ 1370.full.pdf+html.URLhttp://www.jneurosci.org/content/21/4/1370.abstract
MakeigS,BellA,JungT,SejnowskiTJ.Independentcomponentanalysisof electroen-cephalographicdata.In:Advancesinneuralinformationprocessingsystems. MITPress;1996.p.145–51.
MishraP,SinglaS.ArtifactremovalfrombiosignalusingfixedpointICA algo-rithmforpre-processinginbiometricrecognition.MeasSciRev2013;13:7–11,
http://dx.doi.org/10.2478/msr-2013-0001.
MoghaddamB,JebaraT,PentlandA.Bayesianfacerecognition.PatternRecognit 2000;33:1771–82.
MognonA,JovicichJ,BruzzoneL,BuiattiM.ADJUST:anautomaticEEGartifact detec-torbasedonthejointuseofspatialandtemporalfeatures.Psychophysiology 2010:1–12.
Ojala T, Pietikainen M, Harwood D. A comparative study of texture meas-ures with classification based onfeatured distributions. PatternRecognit 1996;29(1):51–9.
OjalaT,PietikainenM,MaenpaaT.Multiresolutiongray-scaleandrotationinvariant textureclassificationwithlocalbinarypatterns.IEEETransPatternAnalMach Intell2002;24(7):971–87,http://dx.doi.org/10.1109/TPAMI.2002.1017623. Ojala T,Pietikainen M,Harwood D.Performance evaluation oftexture
meas-ureswith classification basedon kullbackdiscrimination ofdistributions. In:Proceedingsofthe12thIAPRinternationalconferenceonpattern recog-nition, computer vision and image processing, vol. 1; 1994. p. 582–5,
http://dx.doi.org/10.1109/ICPR.1994.576366.
OsterP,SternJ.Measurementofeyemovementelectrooculography.In:MatinI, VenablesP,editors.Techniquesinpsychophysiology.Chichester,UK:Wiley; 1980.p.275–309.
PetersJ.Surfaceelectricalfieldsgeneratedbyeyemovements.AmJEEGTechnol 1967;7(2):27–40.
RaudysS,DuinR.ExpectedclassificationerroroftheFisherlinearclassifierwith pseudo-inversecovariancematrix.PatternRecognitLett1998;19(5–6):385–92,
http://dx.doi.org/10.1016/S0167-8655(98)00016-6.
RomeroS,Ma nanasM,BarbanojM.Acomparative studyofautomatic tech-niquesforocularartifactreductioninspontaneousEEGsignalsbasedonclinical target variables:a simulation case.Comput Biol Med 2008;38(3):348–60,
http://dx.doi.org/10.1016/j.compbiomed.2007.12.001.
Sandwell D. Biharmonic spline interpolation of GEOS-3 and SEAS, altimeter data. Geophys Res Lett 1987;14(2):139–42,
http://dx.doi.org/10.1029/GL014i002p00139[accessed26.11.14].
SkurichinaM,DuinRPW.Stabilizingclassifiersforverysmallsamplesizes.In: Proceedingsofthe13thinternationalconferenceonpatternrecognition,vol. 2;1996.p.891–6,http://dx.doi.org/10.1109/ICPR.1996.547204.
StrutzT.Bilddatenkompression.Braunschweig:Vieweg;2000.
Swets D, Weng J. Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 1996;18(8):831–6,
http://dx.doi.org/10.1109/34.531802.
Unsworth N, Heitz R, Schrock J, Engle R. An automated version of the operation span task. Behav Res Methods 2005;37(3):498–505,
http://dx.doi.org/10.3758/BF03192720, URL http://webs.wofford.edu/boppkl/ coursefiles/Thesis/articles/UnsworthHeitz2005autoospan.pdf [accessed 02.04.14].
VanhataloS,PalvaJ,HolmesM,MillerJ,VoipioJ,KailaK.Infraslowoscillations mod-ulateexcitabilityandinterictalepilepticactivityinthehumancortexduring sleep.ProcNatlAcadSciUSA2004;101(14):5053–7.
ViolaF,ThorneJ,EdmondsB,SchneiderT,EicheleT,DebenerS.Semi-automatic identificationofindependentcomponentsrepresentingEEGartifact.Clin Neu-rophysiol2009;120(5):868–77,http://dx.doi.org/10.1016/j.clinph.2009.01.015. Weerts T, Lang P. The effects of eye fixation and stimulus and response location on the contingent negative variation (CNV). Biol Psychol 1973;1(1):1–19, http://dx.doi.org/10.1016/0301-0511(73)90010-0, URL
http://www.sciencedirect.com/science/article/pii/0301051173900100
Wikipedia. n-Back; August 2014, URL http://en.wikipedia.org/wiki/N-back
Wikipedia. Task switching (psychology); July 2014, URL http://en.wikipedia. org/wiki/Taskswitching(psychology)[accessed01.09.14].
Wikipedia. Stroop effect; August 2014, URL http://en.wikipedia.org/wiki/ Stroopeffect[accessed01.09.14].
Wikipedia. Saul sternberg; April 2014, URL http://en.wikipedia.org/wiki/ SaulSternberg[accessed01.09.14].
WinklerI,HaufeS,TangermannM.Automaticclassificationofartifactual ICA-componentsforartifactremovalin EEGsignals.BehavBrain Funct2011.,
http://dx.doi.org/10.1186/1744-9081-7-30.
YeJ,JanardanR,ParkC,ParkH.Anoptimizationcriterionforgeneralized discrim-inantanalysisonundersampledproblems.IEEETransPatternAnalMachIntell 2004;26:982–94.