A multi-stage algorithm for blind source separation

(1)

ContentslistsavailableatScienceDirect

Optik

j ou rn a l h o m e p a g e :w w w . e l s e v i e r . d e / i j l e o

A

multi-stage

algorithm

for

blind

source

separation

夽

Xian-feng

Xu

a,∗

_,

_Chen-dong

_Duan

a

_,

_Lai-jun

_Liu

b

_,

_Xiao-jun

_Yang

c

a_School_of_Electronic_&_Control_Engineering,_Chang’an_University,_Xi’an,_Shaanxi_710064,_China b_School_of_Highway,_Chang’an_University,_Xi’an,_Shaanxi_710064,_China

c_School_of_Information_Engineering,_Chang’an_University,_Xi’an,_Shaanxi_710064,_China

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received24September2015 Accepted28December2015 Keywords:

Blindsourceseparation(BSS) Multi-stagealgorithm(MSA) Symmetriccostfunction Triplyiterativestrategy(TIS)

a

b

s

t

r

a

c

t

Intheexistingclassicandrepresentativesecondorderstatisticsbasedmethodsforblindsource separa-tion

(BSS), the mixingmatrix is usuallytransformed intoanunknown unitary matrix after whitening procedure.Inordertoderivetheunitarymatrix,anovelleastsquarebasedsymmetricalcostfunction withrespecttoonecolumnoftheunknownunitarymatrixisproposed.Thecostfunctionisbasedon theorthogonalitybetweeneachtwocolumnsoftheunitarymatrix.Anewtriplyiterativestrategy(TIS) followingthegradientdescentideaisdevelopedtoseektheminimumpointofthetri-quadraticcost functionbyalternatelyestimatingoneofthethreeindependentvariablesparametersubsets.Afterthe convergenceofthecostfunction,thecolumnoftheunitarymatrixcorrespondingtothesourcesignal withthemaximumPower-Likecanbeobtained.Witheachcolumnbeinggotbyutilizingthesystemic multi-stagealgorithm(MSA),theunitarymatrixcanbeestimatedandthenthesourcesignalscanbe retrieved.Simulationresultsillustratethat,comparedwiththeclassicSOBImethodwhichsolvesthe unitarymatrixusingsuccessiveGivensrotations,MSApossessesbetterseparationperformance,lower computationalcomplexity,andthuscouldaccuratelyretrievethesourcesignalsblindly.

1. Introduction

Thetechniqueonlyutilizingthereceivingsignalsbyanarrayof sensorsandthestatisticalpropertyofthesourcesignalstoestimate thechannelparametersandtoretrievethesourcesignals,without anyotherpriorknowledgeonthem,isnamedasblindsource sep-aration(BSS),alsonamedasblindsignalseparation(BSS).Inthe pasttwodecades,BSShasdevelopedrapidlyandhasfoundwide application inwireless communication, radar signalprocessing, imagesignalprocessing,speechsignalprocessing,biomedical sig-nalprocessing,seismicwavedetection, etc.BSShasbeena hot researchtopicinsignalprocessingﬁeld[1–6].

Jointdiagonalization(JD)isoneofthemostefﬁcienttoolsforBSS [1,4,5].IntheseJDalgorithms,itisdesiredtoseekamatrixwhichis theestimationofthemixing(demixing)matrix,usuallycalledthe “jointdiagonalizer”,todiagonalizesimultaneouslyasetofsquare

夽 ThisworkwassupportedinpartbytheNationalNaturalScienceFoundation ofChina(GrantNo.61201407,No.61473047),inpartbyChinaPostdoctoral Sci-enceFoundation(GrantNo.2013M542309),andinpartbytheSpecialFundfor BasicScientiﬁcResearchofCentralColleges,Chang’anUniversity(GrantNo. 0009-2014G1321038).

∗ Correspondingauthor.Tel.:+8602982337230. E-mailaddress:[email protected](X.-f.Xu).

targetmatrices.Oncethemixing(demixing)isderived,itcanbe usedtoretrievethesourcesignalsdirectlyuptothesources’scaling andpermutationindeterminacies.TheBSSisthusrealized.Ofthese algorithms,Cardosohasproposedablindidentificationalgorithm by joint approximate diagonalization of eigen-matrices (JADE) basedonfour-ordercumulantmatrices[4]andhasintroduceda second-orderblindidentification(SOBI)algorithm[5].Bothofthese twoalgorithmsarewidelyconsideredasthepioneering contribu-tionstooff-linemethodsonBSS.Of them,SOBIis aclassicand representativealgorithmwhichisbasedonsecondorderstatisticof receivedsignalstosolveBSSproblem.Thealgorithmconsistsoftwo steps.Thefirststepistowhitenthereceivingdatabythederived whiteningmatrixtotransformtheunknownmixingmatrixtoan unknownunitarymatrix.Inthesecondstep,theorthogonaljoint diagonalization(OJD)isachievedandtheunknownunitarymatrix isthusestimated,throughthesuccessiveGivensrotations.Itshould benotedthat,althoughthereisananalyticalsolutioninonesingle Givensrotation,onlyatwo-dimensionedmatrixcouldbesolved. Therefore,theestimationofamulti-dimensionedunitarymatrix needsmanysweeps.Anewalgorithmnamedmulti-stagealgorithm (MSA)isproposedinthispapertosolvetheunitarymatrix.Inevery stage,asymmetricalcostfunctionbasedonleastsquarescriterion issolvedtoderivedonecolumnoftheunitarymatrix.Thenevery columnisgotsystematically.Insuchway,thewholeunitarymatrix

http://dx.doi.org/10.1016/j.ijleo.2015.12.128

(2)

isestimatedﬁnally.Simulationresultsillustratethatthe perfor-manceofthisalgorithmonestimatingthemixingmatrixisbetter thanSOBIandcouldretrievedthesourcesignalsmoreprecisely.

2. Signalmodel

Considera lineararrayconsistingMsensors,Nnarrow-band sourcesignals arriving at thearray in different directions, the dimension-Mreceivingsignalsvectoris

x(t)=As(t)+n(t). (1)

Here,A∈CM×N_(M_≥_N),_s(t)₌_[s₁_(t),_._._.,_s_N_(t)]T_,_x(t)₌_[x 1(t),...,

xM(t)]T,n(t)=[n1(t),...,nM(t)]T,representthemixingmatrix,the

sourcesignals, theobserving signalsandthenoises.In BSS,the mixingmatrix A isassumed unknownand thesources s(t) not observable.TheproblemisﬁrstlytoestimatethemixingmatrixA

givenonlyTsnapshots{x(t)}T_t=1oftheobservingsignalsx(t)=[x1(t),

...,xM(t)]Tundersomeassumptions[6],andthentoretrievethe

sourcesignalsnapshots{s(t)}Tt=1 withtheestimatedA.Basedon

theassumptionsandTobservingsignalssnapshots{x(t)}Tt=1,aset

ofKtargetmatrices{Ck_,_k₌_1,_._._.,_K_},_the_{intercorrelation}_matrices

of{x(t)}Tt=1withdifferenttimeshiftsk,(k /= 0,k=1,...,K−1),

couldbeestablished:

Rx(0)=E{x(t)xH(t)}=Adiag[01,...,0N]AH+n2I (2) Rx(k)=E{x(t)xH(t+k)}=Adiag[1k,...,Nk]AH, (k /=0) (3)

It isobvious that,omitting theﬂuencyof noise, each target matrixinthissethassuchdiagonalizablestructurethat:

Ck₌_ADk_AH_, _k₌₁_,_._._.,_K. ₍₄₎

AllDk_,_k₌_1,_._._.,_K_are_diagonal_matrices._First_to_jointly

diagonal-izethetargetmatrixsetresultingwiththeestimationofA,thento retrievethesourcesignalsaccordingtheestimatedAtorealizeBSS. Consideringthelimitationofsnapshotnumberandtheexistence ofnoise,thediagonalizablestructureshownin(4)isapproximate ratherthanexact.

3. Themulti-stagealgorithmbasedonsymmetricalcost function

Thedetailsofthemulti-stagealgorithm(MSA)basedon sym-metricalcostfunctionaredepictedasfollows.

3.1. Whitening

Whiteningisanefﬁcientpre-processingmethodforblindsource separation.Applyingindependentcomponentanalysistragedyto whiteningdataofteneasilyresultmoreefﬁcientalgorithmwith morequicklyconvergentrapidthantooriginaldatadirectly.Also, forthecasethesensorsbeingmorethansources,whiteningreduces thenumberofparameterstobeestimatedbyreducingthe dimen-sionofthemixingmatrixandfurthermorereducethecomputation complexity.Andthewhiteningoperationcouldsuppressnoisein somesense.Thewhiteningmatrixcouldbeobtainedinthe follow-ingway[5]:

Theintercorrelationmatricesofthereceivingsignals{x(t)}Tt=1

with zero time shifts could be eigenvalue decomposed as:

Rx(0)=UUH.HereU=[u1,u2,...,uM] denotestheeigenvector

matrix.=diag[1,2,...,M]denotestheeigenvaluematrixand

alltheentriesaresorted indescendingorder.ObservingEq. (2) wecandetectthatthenoisevariancecouldbedenotedas2

n =

(1/(M−N))

M_i=N+1i.LetU=[u1,u2,...,uM]ands=diag[1−

2

n,2−_n2,...,N−n2].Thewhiteningmatrixcouldbedeﬁnedas

P=−1/2s UHs.Andthewhitenedreceivingdatacouldbedenoted as:

y(t)=Px(t)=P[As(t)+n(t)]=Qs(t)+Pn(t). (5) HereQ=PA.Accordingtothestatementabove,itiseasilyto concludethatQ=PAisaunitarymatrix.Thefollowingproblemto besolvedis,withthewhitenedreceivingdatay(t),howtoderive

QandfurthermoretoderivethemixingmatrixAwithQandP.On thisbase,thesourcesignalsarefurtherretrieved.

Moreover,afteronerobustwhiteningmethodwasproposedin [7],amorerobustoneutilizingseveralnon-zerotimeshifts inter-correlationmatricesofreceivingsignalswereproposedwhichhas gainedgoodperformance[8].Itisobviouslythat,betterwhitening methodcouldtransformthemixingmatrixintotheunitarymatrix moreprecisely.Andthiswillimprovethewholeperformanceof thealgorithm.

3.2. Thesymmetricalcostfunction

For thepurpose toget Qwiththe whitenedreceivingdata, a scientiﬁc and reasonablecost function shouldbeestablished. Similarly,theintercorrelationmatricesof{y(t)}Tt=1 withdifferent

non-zerotime shifts p,(p /=0, p=1,...,P) couldbedenoted

as a setof eigen-matriceswith diagonalizable structure traits:

Ry(p)=E{y(t)yH₍_t₊_p₎_}₌_Q_diag[p 1,...,

p

N]QH.Here p=1,...,

P;p=ptandp=1,...,P;p=ptisthetimeshiftstepsize.For

thesimplicity,Ry(p)aredenotedasRp_y.

WedeﬁnePLi=

Pp=1piqiqiH,(i=1,...,N).Ofwhich,qi

rep-resentstheithcolumnofQ.Andfurtherdeﬁnepli=

PLi

2 F.Itis

obviouslythatplireﬂectstheeffectoftheithsourcesignaltothe

receivedsignaly(t).Inthecommoncasewhenthesignal chan-nelisuniformity,thesourcesignaliwithstrongerpowerusually correspondslagerpli.ThuspliiscalledPower-Like.Sortallpliin

descendingorder.Andthearrangedsubscriptsaredenotedas{1, ...,N}.

The least square cost function could be established as J1=

P

p=1

Rpy−pqqH

2

F.ThentheqmakesJ1theleastisthe

estima-tionof1thcolumnofQ.Itisinaccordancewiththe1thsource signalwiththelargestPower-Likepl₁.Itiseasytoﬁndthat,J1is

aquarticfunctionwithrespecttoq,withhighcomputationload, anddifﬁculttodetecttheminimumpoint.For thesimplicityof computation,animprovedsymmetricalcostfunctionisproposed asfollows: J(a,1_,_._._.,P_,_b) = P

p=1

_Rp y−pabH

2 F+ P

p=1

_Rp y−pbaH

2 F s.t.a 2 =1;

b

2=1 (6)

HerebothaandbplaythesameroleinJasqinJ1.Thereason

forutilizing_aand_btodenote_qrespectivelyisonlytodegenerate J1,thequarticfunctionwithrespecttoq,toJ,thebi-quadriccost

functionwithrespecttoaandbrespectively,forthepurposeof easilycomputing.TheparametersinJcouldnaturallybedivided intothreeindependentvariablesparametersubsets,a,1_,_._._.,P

andb.Fixtwoofthem,thecostfunctionJin(5)fallsintoacommon quadriccostfunctionwithrespecttothethirdsubsetofparameters. ThuswecallthecostfunctionJin(5)asatri-quadriccostfunction.

(3)

3.3. Thetriplyiterativestrategy

Inthissubsection,agradientdescentbasedtriplyiterative strat-egy(TIS)isproposed,toestimatea,1_,_._._.,P_and_b_{alternatively}

andseektheminimumpointofJ.EachiterativestepinTISconsists ofthreesub-steps.Ineachsub-step,twosubsetsofparametersare fixed,tominimizethecostfunctionJwithrespecttothethirdsubset ofparameters.Atthebeginning,randomlychooseonenormalized vectorastheinitialvalueof_a(0)and_b(0).Wetakethekthiterative stepasanexample.ThedetailsofTIScouldbedescribedasfollows: Thefirstsub-step,fixa(k−1)andb(k−1),minimizethecost functionJ(a(k−1),1_(k),_._._.,P_(k),_b(k₋₁₎₎_with_respect_to1_(k),

...,P_(k)._{Differentiate}_J(_a(k₋_1),1_(k),_._._.,P_(k),_b(k₋₁₎₎ _with

respecttop_(k),_(p₌_1,_._._.,_P)_{respectively.}_And_let_the_differentia

bezero.Wecanderive: p₍_k)₌ aHRypb+bHRpya

bHbaH_a₊_aH_abH_b, (p=1,...,P). (7)

Thesecondsub-step,ﬁxb(k−1)and1_(k),_._._.,P_(k),_solve_a(k)

tominimizethecostfunctionJ(a(k),1_(k),_._._.,P_(k),_b(k₋_1)).

Dif-ferentiateJ(a(k),1_(k),_._._.,P_(k),_b(k₋₁₎₎_with_respect_to_a(k)._And

letthedifferentiabezero.Wecanderive:

a(k)= (

P p=1p ∗ (k)Rpy)b(k−1)+(

Pp=1p(k)RpHy )b(k−1) 2bH(k−1)(

P_p=1p(k)p∗₍_k))b(k₋₁₎ . (8) Thethirdsub-step,similarly,ﬁxa(k)and1_(k),_._._.,P_(k),_solve b(k)tominimizethecostfunctionJ(a(k),1_(k),_._._.,P_(k),_b(k))._We

canderive: b(k)= (

P p=1p ∗ (k)Rpy)a(k)+(

Pp=1p(k)RpHy )a(k) 2aH₍_k)(

P p=1p(k)p∗(k))a(k) . (9)

Alternativelyrepeatedtheabovethreesub-stepsuntilthe algo-rithmisconvergent.Simulationresultsshowthatthedifference betweentheﬁnallyderivedaandbistiny.Bothofthemcouldbe acceptedastheestimationofonecolumnofQ.

3.4. Themulti-stagealgorithm

Forclarity,wedenotetheoriginalR1

y,...,RPyasR1y(0),...,RPy(0). Thesystematicmulti-stagealgorithm(MSA)couldbedescribedas follows:

Theﬁrststage,utilizetheTISmentionedinlastsubsectionto solvethecostfunction(5).Afterconvergence,q1,theestimation

ofthe1thcolumnofQ,couldbeobtained.Itisinaccordancewith the1thsourcesignalwiththelargestPower-Likevalue.Itis inter-estingthatthroughTIS,notonlyonecolumnofQbutalso1_,_._._.,

P_are_derived._Thus _it_is_possible_to_minus_the_{contributions}_of

the1thsourcesignaltoallRpy(0)resultingwiththenewtarget matrices:Rpy(1)=Rpy(0)−pq1qH1,(p=1,...,P).

Thesecondstage,replaceRpyin(5)withRpy(1).UtilizetheTISto solvethecostfunctionagain.Similarly,q2,theestimationofthe

2thcolumnof_Q,couldbeobtained.Itisinaccordancewiththe 2thsourcesignalwiththesecondlargestPower-Likevalue.

Keepontheoperationssimilarwiththeonesstatedintheﬁrst andthesecondstagesuntilqN,theestimationoftheNthcolumn

ofQ,isobtained.

Inthisway,afterNstages,theestimationoftheunitarymatrix

Qisderivedas: ˜Q=[q1,...,qN].Combingwiththewhitening

matrixP,theestimationofthemixingmatrixAcouldbeobtained as ˜A=PH_Q˜.Thereafter,thesourcesignalscouldberetrievedas ˜

s(t)= ˜A†x(t).Herethesuperscript“_†”denotesthepseudoinverse of a matrix [9]. Furthermore, the retrieved source signals are arrangedindescendingorderofPower-Like.Thismeansthat,in

Fig.1. ThecurvesofmeanCRLversustheiterativetimesin100independenttimes. thecommoncasewhenthesignalchannelisuniformity,wecould usetheMSAtoretrieveonlyonesourcesignalswiththelargest Power-LikeorsomesourcesignalswithsomelargerPower-Like insteadofretrievingallsourcesignals.Thisisobviouslyofgreat importantandpracticalsigniﬁcance.

4. Simulations

4.1. Experiment1

Inthisexperiment,assumethat5sensorsreceive4 indepen-dent sourcesignals with zeromean values. These foursources are s1(t)=sign[cos(310t)], s2(t)=sin[600t+6cos(120t)],

s3(t)=sin(180t),s4(t)=sin(60t)sin(600t).ThemixingmatrixA

aregeneratedrandomly.ThesignaltonoiseratioisSNR=15dB. Thenumberoftheintercorrelationmatriceswithdifferenttime shiftsisP=5.

First,wedeﬁnethecolumnrejectionlevel(CRL)andthe col-umniterativeerror(CIE)asfollowstoshowtheconvergenceofthe algorithm

CRL=20log10(

max(

qHnPA

)−1

). (10)

CIE=10log10

q(k)−q(k−1)

2

F. (11)

Here_q(k)represents_a(k)inthekthiterativestep.

ThecurvesofmeanCRLandCIEversustheiterativetimesin 100independenttimesare showninFigs.1 and2respectively. The MSA consistsof4 stages because N=4. Thus both theCRL curvesandCIEcurvesconsistof4curves.Fig.2showsthe con-vergenceineverystage.Fig.2notonlyshowstheconvergenceof thealgorithmbutalsoshowsthattheiterativestepsneededfor

(4)

Fig.3. ThecurvesofmeanGRLversusSNRsin500independenttrialsofMSAand SOBI.

convergenceineverystepisfew(nomorethan10steps).Andthere islittleerrorbetweentheconvergentvalueandthetruevalue. ThismeansthateverycolumnofQcouldbeestimatedprecisely. Besides,inallindependentexperiment,theinitialvalues_a(0)and

b(0)aregeneratedrandomly.Infact,thebettertheinitialvalues, thelesstheiterativestepsneededbyCRLand CIEconvergence. Simulationresultsshowthat,ineverystage,afeasiblemethodto deriveagoodinitialvalueis,letustakethenthstageasanexample withoutlossofgenerality,tomaketheeigenvaluedecomposition ofR=

P_p=1Rpy(n−1)RpHy (n−1)andchoosetheeigenvector cor-respondingtothelargesteigenvalueastheinitialvaluesa(0)and

b(0).

Then,onthesameconditions,wecomparetheMSAandSOBI throughtheglobalrejectionlevel(GRL)[1,4–6]andthe computa-tiontimeindifferentSNRs.GRLisdeﬁnedas

GRL=10log10 1 2N

⎧

⎨

⎩

N

i=1

⎛

⎝

N j=1

( ˜A−1_A) ij

max_k

( ˜A−1_A)_ik

−1

⎞

⎠

+ N

j=1

_N

i=1

( ˜A−1_A) ij

max_k

( ˜A−1_A)_kj

−1

⎫

_⎬

⎭

. (12)

Fig.3showsthecurvesofGRLversusSNRsin500independent trialsofthetwoalgorithms.ItillustratesthattheGRLofMSAis 9dBbetterthanSOBI.Fig.4showsthecurvesofcomputationtime versusSNRsin500independenttrialsofthetwoalgorithms.It illustratesthatthecomplexityofMSAislowerthanthatofSOBI. AndthemeanconvergencetimeneededbyMSAisabout0.128s

Fig.4.ThecurvesofmeancomputationtimeversusSNRsin500independenttrials ofMSAandSOBI.

Fig.5.(a)Theoriginalsourcesignals.(b)Thereceivedsignals.(c)Theretrieved sourcesignalsbyMSA.(d)TheretrievedsourcesignalsbySOBI.

(5)

whilethemeanconvergencetimeneededbySOBIisabout0.253s. WhenwesynthesizeFigs.3and 4,wecouldﬁndthattheMSA possessesbetterperformanceandshortercomputationtimethan SOBI.

4.2. Experiment2

Four speech signals (shown in Fig. 5(a)) being picked by a 5_×4mixingmatrix_Aresultswithfiveobservingsignals(shown inFig.5(b)).Aisgenerated randomly. SNR=15dB. Thenumber ofeigen-matricesisP=5.ApplyMSA andSOBItoreceiveddata respectively.ThesourcesignalsestimatedbyMSAareshownin Fig.5(c)andestimatedbySOBIareshowninFig.5(d).Wecould findthat compared withthe classical SOBI, theproposed MSA couldretrievethesourcesignalsmoreprecisely.Thisverifiesthe effectivenessoftheproposedalgorithm.

5. Conclusions

AnMSAforblindsourceseparationisproposedinthispaper. DifferentwiththeclassicSOBImethodusingGivensrotationsto derivetheunitarymatrix,MSAseeksonecolumnoftheunitary throughsolvingasymmetricaltri-quadriccostfunction.Agradient descentbasedTISisproposed.TISaimstoseekthreeindependent

coefﬁcients subsets alternatively to solve the tri-quadric cost function. Simulation results show that, compared with classic SOBI,MSApossesseslesscomputationtimeandbetterseparation performance,thuscouldachieveblindsourceseparationefﬁciently.

References

[1]G.Chabriel,M.Kleinsteuber,E.Moreau,etal.,Jointmatricesdecompositionsand blindsourceseparation:asurveyofmethods,identiﬁcation,andapplications, IEEESignalProcess.Mag.31(3)(2014)34–43.

[2]J.Bobin,J.Rapin,A.Larue,etal.,Sparsityandadaptivityfortheblindseparationof partiallycorrelatedsources,IEEETrans.SignalProcess.63(5)(2015)1199–1213. [3]M.Li,Y.Liu,F.Chen,etal.,Includingsignalintensityincreasestheperformance ofblindsourceseparationonbrainimagingdata,IEEETrans.Med.Imaging34 (2)(2015)551–563.

[4]J.F.Cardoso,A.Souloumiac,Blindbeamformingfornon-Gaussiansignals,IEE Proc.F140(6)(1993)362–370.

[5]A.Belouchrani,K.Abed-Meraim,J.F.Cardoso,etal.,Ablindsourceseparation techniqueusingsecond-orderstatistics,IEEETrans.SignalProcess.45(2)(1997) 434–444.

[6]Z.-z.Luo,M.Cao,AnalgorithmtoﬁlterartifactsinEEGbasedonblindsource separationofmaximumsignalnoiseratio,ActaElectron.Sin.39(12)(2011) 2926–2931(inChinese).

[7]H.-F.Chen,X.-R.Cao,H.-T.Fang,etal.,Nonlinearadaptiveblindwhiteningfor MIMOchannels,IEEETrans.SignalProcess.53(8)(2005)2635–2647. [8]D.-Z.Feng,W.-X.Zheng,A.Cichocki,Matrix-groupalgorithmviaimproved

whiteningprocessforextractingstatisticallyindependentsourcesfromarray signals,IEEETrans.SignalProcess.55(3)(2007)962–977.

[9]G.H.Golub,C.F.V.Loan,MatrixComputation,secondedition,JohnsHopkins Uni-versityPress,Baltimore,MD,1989,pp.70–75.