va an of tr al tim sy w Le Le Si Th sim sy th pr un an ad m Th th ad be Li co tim el es th sy fil to co En Sw En (e-Te na Co krr Abstract—Ad arious fields, in nd System Ide f Adaptive
ansform-doma lgorithms to a
me Demonst ynthesized with
as developed t east Mean Sq east Mean Squ ign-Sign adapt he results hav mulation lengt Index Terms ystem identifica
In recent ye heory, metho rocessing. The nderlying the ndtheoretical p daptive filters many, choices
he basicappli he classical "re
daptive system eing applied in ine Enhancem ommon imple
me-domain an Adaptive filt lectrical noise stimate of a de he adaptive filt ystem contain lter is a filter t o an optimizin Manuscript rece oordinating the rev
T. Sreekanth Ra ngineering, Bleki weden (e-mail: ra B. Prathyusha i ngineering and T -mail: prathyusha P. Nagarjuna R echnology and agarjunreddyp@g
K Raja Rajesw ollege of Eng [email protected]
The A
daptive filters ncluding Noise entification. Th filters util ain based adadjust filter co ration of a hin Matlab and to the end user quare, Leaky L
uare, Recursiv tive algorithm ve been prese th and applicat s—Adaptive n ation, channel
I. INTR ears, a great p ods and app eobjective is area of adapti point of view. s and provide available by ication structu eal-world"app ms are discuss n various field ment and Sy ementation o nd transform-d ters are norma
e cancelling esired signal. I ter works as a ing actuators that self-adjus g algorithm. B eived April 28, 2
view process for aot is with the Dep
inge Institute of ao.sreekanth.sp@ is with Departm Technology, Hyd a.signalprocessing Reddy is with De
Science, gmail.vom).
ari is with the D gineering, Visak m).
Applica
T. Sreek are widely e Cancellation,he most comm lizes the t aptive filters oefficients. In t adaptive algo
d a Graphical U r with different
Least Mean S ve Least Squa s have been st nted for some tions. noise cancelle simulator. RODUCTION progress has plications of to introduce ve filtering fro . It reviews the es an indicatio way of adap ures are introd plications link sed. Adaptive ds, including N ystem Identifi of Adaptive f domain adapti ally defined fo where the f In control app controller con and amplifier stsits transfer Because of the 2012; revised Ma
this paper was M partment of Signa f Technology, SE
gmail.com). ment of ECE, M
derabad,Andhra [email protected]). epartment of EEE
Warangal 506
Department of EC khapatnam, Ind
ations a
kanth Rao, B
being applie Line Enhance mon implement time-domain
employing va this paper, the rithms has User Interface
t applications. Square, Norma are, Sign-Error tudied and ap e cases for sp
er, adaptive
been made in f adaptive s
the basic con om a philosop e concepts of l on of some o ptation algorit duced and som ked to these. L
e filters are w Noise Cancella
ication. The filters utilize ive filters. or problems su
filter output i lications, how ntrolling a dyn rs etc. An ada function acco e complexity o ay 30, 2012; The Ms. Yang.
al Processing, Sch E- 371 79, Karls
Malla Reddy Coll Pradesh,India -5
E, Kakatiya Insti 015, India (e
CE, Andhra Uni dia 530003 (e
and Sim
in M
B. Prathyusha ed in ement tation and arious e real been (GUI) Also, alized r and pplied. pecific filter, n the signal ncepts phical linear of the thms. me of Linear widely ation, most s the uch as is an wever, namic aptive ording of the Editor hool of skrona, lege of 500014 tute of e-mail: versity e-mail: optim that perfo non-colle are r oper are n empl valu Th a fou Leas (RLS varia the r Matl wind a use Fig. selec simu to se pane assigmulatio
MATLA
a, P. Nagarju
mizing algorit perform dig ormance based -adaptive filt ectively form t required since ration (for inst not known in a loy an adaptiv es of the filter
Fig
Fig
he block diagr undation for pa st Mean Squa S), etc. The able filter extr results can be lab GUI. A G dows containin er to perform i
2 the user can ct the approp ulation results elect in which els. The panel gn the input v
n of Ad
AB
una Reddy, an
thms, most ad gital signal p
d on the input er has static the transfer fu
some parame tance, the pro advance. In th ve filter, whic r coefficients.
g. 1. General form
. 2. GUI template
ram, shown in articular adap ares (LMS) a idea behind racts an estim
observed usin GUI is a grap
ng controls, c interactive tas n select the alg priate applicat . The simulato each has been consists of ed values for the
daptive
nd K. Raja R
aptive filters a processing a t signal. By w c filter coef unction). Adap eters of the de operties of som hese situations
ch uses feedba
m of adaptive filte
e: simulator mode
n the following tive filter real and Recursive
the block dia mate of the de
ng Simulation hical display called compon sks. In this par gorithm which
tion and can or has six adap n distinguished dit boxes in wh e variables of
e Algori
Rajeswari
are digital filte and adapt th way of contrast fficients (whi ptive coefficien
sired processi me noise sign it is common ack to refine t
er
el
g Fig.1, serves izations, such e Least Squar
agram is that sired signal. A n template call
in one or mo nents that enab rticular templa h he requires, c
acquire desir ptive algorithm d with individu hich the user c
algorithm. There are different types of inputs (here stationary and Non-stationary) and three different types of applications. There are two pop-up menus, from which the user can select the type of input and application. The user can select any application with any desired input signal. There are five axes in which the user can observe five different output wave forms as Input signal, estimated signal, Learning curve, Co-efficient curve, Error signal. The user can also assign the simulation length. One can estimate the signal performance based on these different algorithms with different applications.
II.ALGORITHMS FOR ADAPTIVE FILTERS
The algorithm is the procedure used to adjust the adaptive filter coefficients in order to minimize a prescribed criterion. The algorithm is determined by defining the search method (or minimization algorithm), the objective function and the error signal nature. The choice of the algorithm determines several crucial aspects of the overall adaptive process, such as existence of sub-optimal solutions, biased optimal solution, and computational complexity.
A. The Leaky Mean Square (LMS) Algorithm
The LMS is a search algorithm in which a simplification of the gradient vector computation is made possible by appropriately modifying the objective function. The LMS algorithm, as well as others related to it is widely used in various applications of adaptive filtering to its computational simplicity. The convergence characteristics of the LMS algorithm are examined in order to establish a range for the convergence factor that will guarantee stability. The convergence speed of the LMS is shown to be dependent on the Eigen value spread of the input signal correlation matrix. The main features that attracted the use of the LMS algorithm are low computational complexity, proof of convergence in stationary environment, unbiased convergence in the mean to the Wiener solution, and stable behavior when implemented with finite-precision arithmetic. The optimal (Wiener) solution is given by
W = R r (1)
where = ( ) ( ) & = E[d(k) X(k)], assuming that d(k) and X(k) are jointly wide –sense stationary. The update equation for the least mean square (LMS) is given by
w(k
1) = w(k) μe(k)X(k) (2)Here e(k)X(k) is the estimate of the gradient of ε (k) and
µ is known as the step size and it is computed as[1]
0 < µ < ( ) | ( )| (3)
A simple and effective algorithm that does not require any ensemble averages to be known. For wide-sense stationary processes, the LMS algorithm converges in the mean if the step size is positive and no larger than 2/ , and it converges in the mean-square if the step size is positive and no larger than 2/ ( ) [1].
B. The Leaky LMS (LLMS) Algorithm
The main difference between LLMS and common LMS algorithm is the introduction of γ into the autocorrelation matrix of the input signal X(k). One of the main situations in this algorithm is when the autocorrelation matrix of the filter input signal is ill conditioned, which means it has Eigen values close to zero. From the point of formula one can observe the following differences.
The update equation for the LLMS algorithm is given by [1]
w(k 1) = (1 μ )w(k) e(k)X(k) (4)
Here γ is the leakage coefficient which has limits
0 < γ <<1 (5) Through proper choice of positive leakage parameter Ɣ, excess parameter drift can be avoided. Though LLMS has very low implementation complexity, it applies filter tap leakage indiscriminately thus biasing W(k) and increasing mean square error (MSE).The leaky LMS algorithm is useful in overcoming the problems that occur when the autocorrelation matrix of the input process is singular [1].
C. Normalized LMS (NLMS) Algorithm
To increase the speed of the LMS algorithm without using the estimates of the input signal correlation matrix, a variable convergence factor is a natural solution. The normalized LMS algorithm usually converges faster than the LMS algorithm since; it utilizes a variable convergence factor aiming at the minimization of the instantaneous output error. The updating equation of the LMS algorithm can employ a variable convergence factor μ in order to improve the convergence rate. In this case the updating formula is expressed as [1]
w(k 1) = w(k) 2μ e(k)X(k) (6)
where μ must be chosen with the objective of achieving a faster convergence. A possible strategy is to reduce the instantaneous squared error as much as possible. The motivation behind this strategy is that the instantaneous squared error is a good and simple estimate of the mean square error (MSE).
μ =XT( )X( ) (7)
Using the variable convergence factor, the updating equation for the LMS algorithm is given by [4]
w(k 1) = w(k) e(k)X(k)
XT(k)X(k) (8)
Using a fixed convergence factor μ is introduced in the updating formula in order to control the misadjustment, since all the derivations are based on the instantaneous values of the squared errors and not on the MSE. Also a parameter ε should be included in order to avoid large step sizes when
( ) ( ) becomes small. The coefficient updating
sim co D m er m eq su si in fo fo kn sp al va m st E. re eq in al by X( al w th ca us co us w ch to re LM co w(k 1) The resulting mplifies the oefficients con
D. Recursive L
The RLS al minimizes least rror between le minimizes weig
quation [1]
ε(n) =
Least-square um of the squ
gnal and the m ncoming signa or the least sq orm resulting
nown to pursu pread of the i lgorithm has e arying enviro more complex ationary proce
. Sign Error
Sign-error al educe the c quivalent to t nversely propo lgorithm, the L y applying the
X(k) and d(k)
lgorithm is [1] w(k
where sgn
The simplific he step size is ase, the coeff sing p+1 dat ompared to th ses a noisy es when replacing hange in magn
( ) and does o the form μ = equired. The u
MS algorithm onvergence ra
= w(k) β
ε
g algorithm is selection of t nverge [1].
Least Square
lgorithm utili t square error east squares an ghted least squ
= λ |e(i)
es algorithms uares of the model filter ou als are receive quares problem
in the RLS ue fast converg
input signal c excellent perf onments. RLS than the LMS esses [1].
r Algorithm
lgorithm is on complexity. T
the LMS algo ortional to the
LMS coeffici e sign operato are real valu ]
1) = w(k)
{e(k)} = 0
cation in the S chosen to be ficient update
ta shifts inst he LMS algo stimate of the g e(k) with the nitude of the c
not alter the = 2 , only shi update mecha m. The steady ate decreases.
βe(k)X(k) XT(k)X(k)
s called the N the step size
(RLS) Algorit
izes least squ r. Strictly spea nd mean squar uare error at tim
)|
aim at the m difference be utput. When n ed at every iter m can be com algorithms. gence even wh correlation ma formance whe S algorithm i S adaptive filt
ne way of the The Sign-er orithm with a e magnitude o ent update eq or to the error ued processes
μsgn{e(k)}X
1 ; e(k) 0 ; e(k) =
1 ; e(k) <
Sign-error algo a power of tw
equation ma tead of p+1
rithm, the Sig gradient. Th e sign of the er
correction tha direction [5]. ifting and addi anism is degra
y state error
NLMS algorith to ensure tha
thm
uares approac aking it minim res. RLS algo me ‘n’ given b
minimization o etween the de new samples o ration, the sol mputed in recu This algorith hen the Eigen atrix is large. en working in is computatio ter, for
wide-e simplificatio rror algorithm
a step size th of the error. In quation is mod
e(k).Assumin , the signal
X(k) 0 0 0 orithm comes wo, µ=2 . In ay be implem
multiplies. W gn-error algo e change obse rror is that onl at is used to u
If μ is constr ition operation aded, compar will increase (9) hm. It at the ch, it mizes rithm by the (10) of the esired of the lution ursive hm is value This n time onally sense ons to m is hat is n this dified ng the error (11) (12) when n this mented When rithm erved ly the update rained ns are red to e, the F. Si redu LMS to th the L the X(k) the s Whe Un conv The noise weig cova algor A.No On filter Nois conc corru 2, th corru corre refer prim B.Ad Th adap The in fig a ha into adap Δ an filter acco
Sign – Sign A
ign-Sign algor uce the compl S algorithm w e magnitude o LMS coefficie sign operat .Assuming th signal error alg
w(k 1)
ere sgn{e(k
nlike any othe verges is indep
sign-sign alg e variance, in ght misalignm ariance matrix
rithm.
III. APPLIC
Noise Cancella
ne of the most rs is noise ca se Canceller (A cept of ANC, a
upted by addit he method u upted signal a elated in some rence input is mary input to o
Fig. 3. Schema
daptive Line E
he ALE was ptive signal pro structure of an g. 4. After dig armonic signa two channels. ptive filter. Th nd then acts a r. The adapt ording to the A
Algorithm
rithm is anothe lexity. This al with a step size
of the error as w ent update equ
or to both e X(k) and d(
gorithm is [1] ) = w(k) μ
k)} = 1 0 1
er algorithm, w pendent of th gorithm conve nitial and toler ment, and the x [5]. This algo
CATIONS FOR A
tion
t common pra ancellation. T ANC) is to rem
an alternative tive noise or in ses a primar and a referenc e unknown wa adaptively fil obtain the sign
atic diagram of ad
Enhancement
introduced b ocessing as a n ALE or adap gitalizing, the i al plus broadb . One is treate he other goes t
as a reference tive filter w Adaptive algor
er way of the s lgorithm is eq e that is invers
well as data. In uation is modif
the error
(k) are real va
μsgn{e(k)}sgn
; e(k) 0
; e(k) = 0 ; e(k) < 0
whether or not he magnitude ergence time rable magnitu e Eigen value orithm is faste
ADAPTIVE ALG
ctical applicat The Applicati
move noise fro method of es nterference [5 ry input d[n] ce input x[n] c ay with the pri ltered and sub nal estimate [6
daptive noise canc
(ALE)
by Widrow e spectral estim ptive notch fil input signal x[
band backgrou ed as the expec through a dec e input signal will adjust its
rithm.
simplification quivalent to t sely proportion
n this algorith fied by applyi
e(k) and da alued processe
n{X(k)} (1
(1
t, sign-sign LM of the step siz depend on t des of the me es of the inp r than sign LM
GORITHMS
tions of adapti on of Adapti om a signal. T stimating sign
] as shown in containing t containing no imary noise. T btracted from t 6].
celler (ANC)
et al for use mation techniqu lter is illustrat
[n] composed und noise; go cted signal to correlation del to the adapti s own weigh
ou no
C
m ap us re ad su ch Th m id fil ad si es cr
di th er re al ar va =1
Fig. 4. Sch
After the opt utput y[n] of oise, and mean
C.System Ident
System Iden modeling of a pplications de sed to provid eplacement to daptive filter a upplies the de hannel is dyna he schematic model. The d
dentification a lter is noise f dditive noise gnal-processin stimate of the riterion of perf
Fig. 5.
The followin ifferent applic
(i) co-efficie hree axes are (i rror signal. In esults. Choosi
lgorithm, i.e., re obtained for alues are as fo 1000.
hematic diagram o
timum process the ALE syst nwhile enhanc
tification
ntification is a process fo
aling with ide de a linear m o unknown c
are driven by esired respons amic in nature
block diagra distinguishing application is free and the d
that is uncorr ng application e desired resp formance [4].
Schematic diagr
IV. SIMULA ng are the s cations. The to ent curve (ii) iii) input signa n this we ha ing different for LMS Alg r the applicatio ollows: µ =0.0
of adaptive line e
sing of the ada em will suppr ce the harmon
experimental or a channel. entification, an model that re channel. The
same input. T se for the ada , the model w am shown in characteristi that the inpu desired respon
related with th ns, the goal i ponse accordi
ram of system ide
ATION RESULT imulation res op two axes ar learning curv al (iv) estimate ave shown di
applications gorithm. Fig. 6
on Noise Canc 02, order =10
enhancer (ALE)
aptive iteration ress the wide nic signal [2].
l approach to . In the clas n Adaptive fil epresents the channel and The channel o aptive filter. I ill be time var n fig. 5, give ic of the sy ut of the ada nse is corrupte he input sign s to obtain a ing to the ado
entification
TS
sults for the re
ve. The rema ed/ output sign ifferent simul
for one parti 6 shows the re celler. The var , simulation le
n, the band
o the ss of lter is
best d the output
If the rying. es the
ystem aptive ed by al. In good opted
three aining nal (v) lation icular esults riable ength
A.LM
F
B.LL
Fi ALE γ= 1
Fi
C.NL
Fi the o Syste learn
Fig.
MS Algorithm
Fig. 6. Output wa
LMS Algorithm
ig. 7 shows th E. The variable & simulation
ig. 7. Output wav
NLMS Algorithm
ig. 8 shows NL order is set t em Identifica ning curve. Th
8. Output wavef
m with Noise C
aveforms: LMS a applica
m with ALE as
he results are e values are as n length =1000
eforms: LLMS al
m with System
LMS is selecte to 25. A Stat ation and the he simulation
forms: NLMS alg applica
Cancellation as
lgorithm with no ation
s Application
e obtained for s follows: µ = 0.
lgorithm with AL
m Identificatio
ed to work and tionary signal e output is ta
length is chos
gorithm with syste ation
s Application
ise canceller as
r the applicati =0.02, order =1
LE as application
n as Applicati
d β is set to 1 a l is selected f
aken to be t sen as 1000.
em identification ion 10,
ion
and for the
D.RLS Algorithm with ANC as Application
Fig. 9. Output waveforms: RLS algorithm with ANC as application
Fig. 9 shows RLS is selected to work and γ is set to 1 and the order is set to10. A Non-Stationary signal is selected for ANC and the output is taken to be the learning curve. The simulation length is chosen as 1000.
E.Sign Error Algorithm with ALE as Application
Fig.10 shows Sign-Error is selected to work and µ is set to 0.001 and the order is set to 10. A Non-Stationary signal is selected for ALE and the output is taken to be the learning curve. The simulation length is chosen as 1000.
Fig. 7. Output waveforms: sign LMS algorithm with ALC as application
F.Sign – Sign Error Algorithm with System Identification as Application
Fig.11 shows Sign-Sign Error is selected to work and µ is set to 0.001 and the order is set to 10. A Non-Stationary signal is selected for system identification and the output is taken to be the learning curve. The simulation length is chosen as 1000.
Fig. 8. Output waveforms: sign-sign error algorithm with system identification as application
V.CONCLUSION
The implementation of various algorithms successfully achieved system identification and noise cancellation. The effect of varying different parameters in the algorithm is also observed.
A performance analysis of adaptive filter algorithms based on adaptive weight update equation is presented in stationary and non-stationary environments. It should be noted that all results presented here for the different algorithms with different applications are demonstrated and the end user can select an appropriate parameter for the desired application.
REFERENCES
[1] M. H. Hayes, “Statistical Digital Signal Processing and Modeling,” 2nd ed., John Wiley and Sons, Inc., Wiley India, 2002.
[2] B. Widrow, J. R. Glover, Jr, J. M. McCool, J. Kaunitz, C. S. Williams, R. H. Hearn, J. R. Zeidler, E. D. Jr, R. C. Goodlin, "Adaptive noise cancelling: Principles and applications," in Proceedings of the IEEE , vol. 63, no.12, pp. 1692-1716.
[3] A. Poulariksa and Z. M. Ramadan, “Adaptive filtering primer with Matlab,” CRC Press, 2006.
[4] S. Haykin, “Adaptive Filter Theory,” fourth edition, Pearson Education. ISBN978-81-317-0869-9.
[5] D. G. Manolakis, V. K. Ingle, and S. M. Kogon, “Statistical and Adaptive Signal Processing: Spectral Estimation,” Signal Modeling, Adaptive Filtering and Array Processing, McGraw-Hill Companies, Inc., Boston, 2000.
[6] M. S. E. Abadi and J. H. Husoy, “Mean –Square Performance of Adaptive Filter Algorithms in Non stationary Environments,” International Journal of Signal Processing, vol. 88, no. 8, pp. 2008-2018 ISSN: 01651684, 2008.
Sreekanth Rao Tangellapally born in Warangal, India in 1987. He received the Bachelor degree in Electronics and Instrumentation Engineering from Kakatiya University, Warangal, in 2008. The M.S. in Electrical Engineering, with emphasis on Signal Processing at Blekinge Institute of Technology, Karlskrona, Sweden (at present) and M.Tech. Degree from Andhra University, Visakhapatnam, India, in 2011. He has received academic awards and
publications in reputable IEEE Conferences and Journals. He is author of the
textbook Fingerprint Recognition System published by VDM Verlag Dr.
Müller e.K. His areas of interests are Image Processing and Bio-medical
Signal Processing.
B Prathyusha born in Warangal, India in 1987. She received the Bachelor degree in Electronics and Communication Engineering from Jawaharlal Technological University (JNTU) in 2009. She is currently undergoing a Master’s program in Systems & Signal Processing at Malla Reddy College of Engineering & Technology, Hyderabad. She has received academic awards and publications in
reputable International Conferences and Journals. Her
areas of interests are Speech Signal Processing and Bio-medical Signal Processing.
Nagarjuna Reddy completed his Bachelor degree in Electrical & Electronics Engineering from University college of Engineering, JNTU, Hyderabad and Master’s Program in High Voltage Engineering from University college of Engineering, JNTU, Kakinada He is currently pursuing his research in JNTU, Hyderabad. His areas of interests are Gas insulated Substations, MATLAB applications in Electrical and Electronics.
