FAST EXACT AFFINE PROJECTION ALGORITHM USING DISPLACEMENT STRUCTURE
THEORY
Manolis C. Tsakiris and Patrick A. Naylor
Dept. of Electrical and Electronic Engineering, Imperial College London
Communications and Signal Processing Group
manolis.tsakiris08, [email protected]
ABSTRACT
This paper exploits the displacement structure of the coefficient matrix of the linear system of equations pertinent to the Affine Projection Algorithm (APA), to obtain the exact solution in a way faster than any other existing exact method. The main emphasis of the paper is to present the concepts of displacement structure theory and how these are applied to the APA context.
Index Terms— Adaptive Filters, Affine Projection Algorithm, Displacement Structure Theory, Choleski Factor
1. INTRODUCTION
The affine projection algorithm (APA) [1] was proposed in order to improve the convergence speed of the Normalized-Least-Mean-Squares (NLMS) algorithm for colored input signals.
Part of the computational complexity of APA comes from the requirement to solve a linear system of equations (LSOE) with coef-ficient matrix being a sample covariance matrix of the input signal. The cost of solving this LSOE determines the overall cost of APA in applications where the adaptive filter length is not much larger than the projection order, such as in adaptive beamforming [2]. In applications where this is not true, such as in echo cancellation [3], approximate fast versions of APA (FAPs) can be used, where the crit-ical issue is again to solve a LSOE with the same coefficient matrix as in the exact APA.
Over the past few years, many efforts have been made, espe-cially in the context of FAPs, in order to solve this LSOE in a fast and reliable way. The majority of the proposed methods, e.g. [3], [4], [5], [6] and [7], achieve relatively low complexity, albeit at the disadvantage of leading to approximate solutions, while few only studies have been concerned with obtaining the exact solution, e.g. [8] and [9].
In this study, the special structure of the sample covariance ma-trix is fully exploited by invoking the theory ofdisplacement struc-ture and the concept ofdisplacement rank, in order to solve the LSOE in a fast and exact manner. A numerical linear algebra al-gorithm for time-variant structured matrices [10] is applied into the APA context resulting in the fastest existing exact implementation of APA to the best knowledge of the authors. This is done by prop-agating from iteration to iteration the Choleski factor of the sample covariance matrix and solving two triangular LSOE.
Although the presentation is done in the context of the exact APA, the core of the proposed algorithm can equally well be applied to the context of FAPs. The notation used is standard with vectors and matrices denoted with boldface lower and upper case respec-tively and with the indexing of the columns and rows of a matrix starting from0except if explicitely otherwise stated.
1.1. The Affine Projection Algorithm
In the standard system identification scenario, the APA can be de-scribed as follows. Consider a collection of scalar measurements
{d(i)}that arise from the model
d(i) =uiwo+v(i), i
≥0 (1)
wherewois anM
×1column vector representing the unknown finite impulse response of the system to be identified, ui is the1×M regressor vector that captures the input data
ui= [u(i), u(i−1),· · ·, u(i−M+ 1)] (2) andv(i)accounts for the measurement noise. At timeithe APA delivers an estimatewiof the unknownwovia
wi=wi−1+µU∗ibi (3)
wherebiis the solution of
Ribi=ei (4) with Ri=UiU∗ i+!IK (5) Ui=!uT i,uTi−1,· · ·,uTi−K+1 "T (6) being theK×Mregressors matrix,eithe vector estimation error
ei=di−Uiwi−1 (7)
anddi= [d(i), d(i−1),· · ·, d(i−K+ 1)]TtheK×1 measure-ments vector. The parameterKdenotes the APA projection order, whileµis a step-size that controls convergence,!is a small positive regularization parameter that enforces the positive-definiteness of
RiandIKis theK×Kidentity matrix.
1.2. Fast Affine Projection Algorithms
The fast affine projection algorithms (FAPs) are approximations of the standard APA and are well suited for echo cancellation, where usuallyM $K. An approximation adopted in this context is that
eihas the following structure:
ei≈ # e(i) (1−µ)ei−1 $ (8)
whereei−1denotes the upperK−1elements ofei−1[3]. Another key approximation of FAPs is that their recursions employ an alter-native weight vector, whose computation is much faster than for the weight vector of the standard APA.
In order for this weight vector to be updated from iteration to iteration the solution of a LSOE of the form (4) is required, with the right side now satisfying property (8).
1.3. Background in Solving Linear Systems of Equations
A standard approach towards solving a LSOERb=e,RbeingK×
K symmetric positive-definite, is via the Choleski decomposition
R= LL∗, whereLis the unique lower triangular factor ofRwith positive diagonal elemens. The cost isO(K33)m1([12], p.144).
IfRis additionally Toeplitz, then the solution can be obtained via the Levisnon algorithm atO(4K2)m([12], p.197).
Alternatively, a sequence of approximate solutions can be found usually at a smaller cost via the so-callediterative methods, typical examples of which are the Conjugate-Gradient and the Gauss-Seidel methods [12].
1.4. Existing Approaches Towards Solving the APA and FAP Linear System of Equations
Returning to the LSOE (4) that arises in the APA and FAP con-texts (with the approximation (8) for the latter), it is observed that the coefficient matrixRiis symmetric positive-definite and
conse-quently the standard method to solve it is via Choleski decompo-sition, which as mentioned in subsection 1.3, requiresO(K3
3 )m. Several methods have been proposed in order to obtain the exact so-lution or an approximate one faster thanO(K3
3 )m. Although this work is concerned with obtaining the exact solution, it is significant to first briefly refer to the approximate methods.
To begin with, in [3] the inverse coefficient matrix is estimated using a sliding window fast RLS and an approximate solution is ob-tained atO(20N)m. In [4]Riis assumed to be Toeplitz and solu-tion via the Levinson algorithm is implied (O(4K2)m). In [5] and [6] an approximate solution is iteratively obtained via the conjugate-gradient algorithm inO(2K2)mand via the Gauss-Seidel algorithm inO(N2/p)m(pis an integer) respectively, after some significant simplification taking place by settingµ = 1in (8). Finally, in [9] two approximate methods are proposed: the first assumes thatRi
can be regarded as constant overKsampling intervals and the sec-ond assumes as [4] thatRiis Toeplitz.
Less work has been done in obtaining the exact solution of (4). A method that most efficiently exploits so far in the literature the struc-ture ofRiis [8], where the exact solution is obtained atO(4K2)m by using the matrix-inversion-lemma in a clever way. An exact ap-proach that targets better robustness than [8] is also proposed in [9], whose cost is however proportional toK3. No other significant con-tribution has come to the attention of the authors, as far as obtaining the exact solution of (4) is concerned.
1.5. The Contribution
Displacement structure theory [13] from numerical linear algebra can be applied in a novel way to fully exploit the structure ofRi
and solve the LSOE atO(3K2)m, a lower computational complex-ity than that of [8]. An indirect reference to the lowdisplacement
1In this paper the computational complexity of an operation is measured in terms of the order of required multiplications (m).
rank ofRihas been made in [3], where it was in passing mentioned that the so-calledGeneralized Levinson algorithm[11] can be used to solve (4) atO(7K2)m. In fact, a numerically better algorithm than the generalized Levinson, the so-calledGeneralized Schur al-gorithm[14] can be used to solve the LSOE atO(7K2)m, although in the very recent [7] direct inversion requiringO(K3)mis referred to.
In this work it is attempted to fill the gap between the APA lit-erature and the numerical linear algebra litlit-erature, thus opening new perspectives for even faster and more reliable APA implementations. This is done by observing at the first place that the displacement rank of the matrixRiis constant in time and equal to 2, regardless of the stationarity or non-stationarity of the input process. In the sequel, it is shown how the Choleski factor ofRican be efficiently propa-gated from iteration to iteration by applying the results of [10] into the APA context. The result is a clearly motivated and presented im-plementation of the standard regularized APA, which requires only O(3K2)mto obtain the exact solution of (4). Note that this rithm is even faster than solving the system via the Levinson algo-rithm, under the assumption thatRiis Toeplitz. This is not surpris-ing since 1) accordsurpris-ing to the development in [11] thedistanceofRi
from a Toeplitz matrix of the same size is zero and 2) the proposed algorithm does not compute the Choleski factor ofRidirectly, albeit
indirectly by updating the Choleski factor ofRi−1 and thus using only the minimum amount of computation.
2. DISPLACEMENT STRUCTURE
LetRibe a time-varyingK×Kpositive-definite matrix with lower-triangular Choleski decomposition
Ri=LiL∗i (9)
whereLiis the unique lower-triangular Choleski factor ofRiwith positive diagonal elements, and define the matrix∇ZRias
∇ZRi
∆
=Ri−ZRi−1Z∗ (10) where Zis some sparse lower-triangular displacement matrix. If rank(∇ZRi) = r(i) < K, thenRi is said to have displacement
structure with respect to the displacement defined byZ. In this work it is assumed thatr(i) =rfor everyi.
Since∇ZRi is Hermitian, its eigenvalues are all real and it is
assumed that it hasppositive andqnegative eigenvalues withp+q= r. Moreover, consider the eigen-decomposition
∇ZRi=GiΛiG∗i (11)
whereGicontains in its columns the eigenvectors of∇ZRiandΛi
is a diagonal matrix with the corresponding eigenvalues in its diago-nal elements. Since the eigen-decomposition (11) is not unique, the eigenvectors of∇ZRican be ordered in such a way, so that the firstp
columns ofGicontain the eigenvectors which correspond to the pos-itive eigenvalues of∇ZRi, the nextqcolumns contain the
eigenvec-tors that correspond to the negative eigenvalues, and the remaining (K−r)columns contain the eignevectors which correspond to the zero eigenvalues of∇ZRi. By expanding its right side (11) becomes
∇ZRi= K%−1
k=0
λi,kgi,kg∗
i,k (12)
wheregi,kis thekthcolumn ofGi. Given the ordering of the
the right side of (12) will be zero and consequently ∇ZRi= r−1 % k=0 λi,kgi,kg∗ i,k. (13)
By defining the scaled eigenvectors as
gi,k ∆
=&|λi,k|gi,k, k= 0,1,· · ·, r−1 (14)
(13) can be rewritten as ∇ZRi= p−1 % k=0 gi,kg∗ i,k− r−1 % k=p gi,kg∗ i,k (15)
which can be expressed more compactly in matrix form as
∇ZRi=GiJG∗i =Ri−ZRi−1Z∗ (16) whereGiis theK×rgenerator matrixdefined as
Gi∆
=' gi,0 · · · gi,p−1 gi,p · · · gi,r−1 ( (17) andJis ther×rsignature matrixdefined as2
J=Ip⊕(−Iq)=∆ # Ip 0p ×q 0q×p −Iq $ (19) where⊕denotes thedirect sumoperator. Equation (16) will be ref-ered to as thedisplacement equation.
3. CHOLESKI FACTOR PROPAGATION
The displacement equation (16) has an extremely important impli-cation: the Choleski factorLi−1 of Ri−1 can be updated to the Choleski factorLiofRiand most importantly this can be done at O(rK2)m[10]. In the rest of this section the general theory of how to obtainLifromLi−1is presented.
Towards this end, a key lemma is invoked, known asHyperbolic Basis Rotation Lemma:
Hyperbolic Basis Rotation LemmaConsider twon×m(n≤m) matricesAandB. IfAˆJA∗=BˆJB∗is of full rank, for somem×m signature matrixˆJ=Ip⊕(−Iq),p+q =m, then there exists an m×mˆJ-unitary matrixH(HˆJH∗= ˆJ) such thatA=BH. A proof can be found at [10], while a more elegant proof can be found at ([15], p.608). Moreover, it is noted that the transformation
His highly non-unique since it can be shown that any other tranfor-mationHC, whereCisˆJ-unitary, has the same effect, i.e. that of mappingBtoA.
Now, from (16)
Ri=ZRi−1Z∗+GiJG∗i (20)
which can be rewritten as
' Li 0K ×r ( # IK 0K ×r 0r×K J $ # L∗ i 0r×K $ = ' ZL i−1 Gi ( # IK 0K ×r 0r×K J $ # L∗ i−1Z∗ G∗ i $ . (21) 2IfAisp×pandBisq×q, then A⊕B∆=diag{A,B}= # A 0p×q 0q×p B $ (18)
Since the left side of equation (21) equals to Ri, which being positive-definite is also full-rank, equation (21) fits exactly to the statement of the Hyperbolic Basis Rotation Lemma with
A=' Li 0K×r ( (22) B=' ZLi−1 Gi ( (23) and ˆ J= (IK⊕J) = # IK 0K ×r 0r×K J $ . (24)
Consequently, there exists a(IK⊕J)-unitary matrixHisuch that
' ZLi
−1 Gi (Hi=' Li 0K×r (. (25)
Equation (25) clearly reveals that knowlegde ofLi−1andGiis suf-ficient for the computation ofLi. It remains to carefully design the transformationHiwhich will yieldLi.
Towards this end, note first thatHishould result in a zero right-most K×r block when applied to the matrix ' ZLi−1 Gi (. Moreover,Hishould be designed so as to yield a lower-triangular leftmostK×Kblock with positive diagonal elements. It is shown in the sequel that ifHisatisfies these two properties, then the result-ing leftmostK×Kblock is necessarilyLi.
Schematically and assuming for simplicityK = 3andr = 2,
Himust be designed so as to map' ZLi−1 Gi (to a matrix of the following form:
indef initepositive positive0 00 00 00 indef inite indef inite positive 0 0
(26)
In other words,
' ZLi
−1 Gi (Hi=' Xi 0K×r ( (27)
whereXiis lower-triangular with positive diagonal elements. Now, taking the squared(IK⊕J)-norm3 of both sides of equation (27)
leads to ' ZLi −1 Gi (Hi # IK 0K ×r 0r×K J $ H∗ i # L∗ i−1 G∗ i $ = ' Xi 0K ×r ( # IK 0K ×r 0r×K J $ # X∗ i 0r×K $ (28) which in view of the(IK⊕J)-unitarity ofHi4becomes
ZLi−1L∗i−1Z∗+GiJG∗i =XiX∗i (29)
which can be rewritten as
ZRi−1Z∗+GiJG∗i =XiX∗i. (30)
By combining equations (30) and (20) the result is
XiX∗
i =Ri (31)
which is the lower-triangular Choleski decomposition ofRi, since
Xiis by design lower-triangular with positive diagonal elements. By
the uniqueness of the Choleski decomposition it is concluded that
Xi=Li. (32)
In the next section the theory of this section is applied in the APA context.
3The squaredˆJ-norm of a column-vectorxis the sign-indefinite quantity #x#ˆ2J ∆ =x∗ˆJx. 4H i # IK 0K×r 0r×K J $ H∗ i = # IK 0K×r 0r×K J $
4. ALGORITHM DEVELOPMENT
Returning to the APA context, consider theRimatrix of equation (5) with its Choleski decomposition given by equation (9). Then by using the lower-shift matrix
Z= # 0 1×(K−1) 01×1 IK−1 0(K−1)×1 $ (33) it is seen that multiplication ofRi−1from the left byZand from the right byZ∗amounts to shifting it downward along the main diagonal by one position while setting the first column and the first row equal to zero. Consequently, the displacement equation becomes
Ri−ZRi−1Z∗= uiu∗ i +! uiu∗i−1 · · · uiu∗i−K+1 ui−1u∗i ui−1u∗i−1+! · · · ui−1u∗i−K+1 .. . ... · · · ... ui−K+1u∗i ui−K+1u∗i−1 · · · ui−K+1u∗i−K+1+! − 0 0 · · · 0 0 ui−1u∗i−1+! · · · ui−1u∗i−K+1 0 ui−2u∗i−1 · · · ui−2u∗i−K+1 .. . ... · · · ... 0 ui−K+1u∗i−1 · · · ui−K+1u∗i−K+1+! = uiu∗ i+! uiu∗i−1 · · · uiu∗i−K+1 ui−1u∗i 0 · · · 0 .. . ... · · · ... ui−K+1u∗i 0 · · · 0 (34)
with the right side being a rank-2 matrix that can be factored as in (16) with Gi= / )ui)2+! 0 ui−1u∗i √ $ui$2+! ui−1u∗i √ $ui$2+! ui−2u∗i √ $ui$2+! ui−2u∗i √ $ui$2+! .. . ... ui−K+1u∗i √ $ui$2+! ui−K+1u∗i √ $ui$2+! (35) and J= # 1 0 0 −1 $ . (36)
Now assume that the Choleski factor ofRi−1, i.e.Li−1is available. According to section 3, in order to obtainLi, the matrix
# ZLi −1 0 12 3 K×K Gi 0123 K×2 $ (37) must be(IK⊕J)-transformed, withJnow explicitely given by equa-tion (36), to a matrix of the form
# Xi 0123 K×K 0 0123 K×2 $ (38) whereXiis lower-triangular with positive diagonal entries. ThenXi
will be the Choleski factor ofRi, i.e.Li.
Details are now given on how to design an appropriate transfor-mationHi, which performs the mapping (37)Hi
→(38). Towards this end, considerHi as a sequence ofKelementary(IK⊕J)-unitary
transformations{Hi,j}K−1
j=0 successively applied to the matrix (37), i.e.Hi=Hi,0Hi,1· · ·Hi,K−1.
Denote byli,j the non-zero part of thejthcolumn ofLi and
expand the Choleski decomposition ofRias follows
Ri=LiL∗i =li,0l∗i,0+ K%−1 j=1 # 0j ×1 li,j $ ' 0 1×j l∗i,j (. (39)
It is important to note that thejthterm of the above sum is aK×K matrix with its firstjcolumns and rows equal to zero.
Now, set in (37)ZLi−1 =Xi,0andGi=Gi,0and consider the matrix
Bi,0=' Xi,0 Gi,0 ( (40)
which eventually must be transformed into the form (38). Observe that the first row ofXi,0is equal to zero and moreover the second en-try of the first row ofGi,0is also zero, while its first entry is positive. Consequently,Hi,0can be selected as
Hi,0 = # IK 0K ×2 02×K Qi,0 $ P0↔K=P0↔K (41)
whereQi,0 = I2 and P0↔K denotes the orthogonal permutation
matrix which permutes columns0andK. In this way,
Bi,1=Bi,0Hi,0=Bi,0P0↔K= #
xi,0 01×(Xi,K−1) 01×2 1 Gi,1
$
(42) where it is also mentioned for future reference that the first row of
Xi,1equals to zero, sinceXi,1is the lower-right(K−1)×(K−1) block of the matrixZLi−1. Now, by taking the squared(IK⊕J
)-norm of both sides of equation (42), noting that)Bi,0Hi,0)2(IK⊕J)=
Riand invoking equation (39), the result is
li,0l∗i,0+ K%−1 j=1 # 0j×1 li,j $ ' 01 ×j l∗i,j (= xi,0x∗i,0+ # 0 1×(K−1) Xi,1 $' 0(K−1)×1 X∗i,1 ( + + # 0 1×2 Gi,1 $ J' 02×1 G∗i,1 ( (43) from which it is evident that
xi,0=li,0 (44)
and hence the first column ofLihas been found. Now, again from (43) it is seen that Ri−li,0l∗i,0= # 0 1×1 01×(K−1) 0(K−1)×1 Xi,1X∗i,1+Gi,1JG∗i,1 $ . (45) IfLiis partitioned in the following block form
Li= # li,0 01×Li,(K−1) 1 $ (46) it is readily found that
Ri−li,0l∗i,0=LiL∗i−li,0l∗i,0=
# 0
1×1 01×(K−1)
0(K−1)×1 Li,1L∗i,1
$
. (47) Combining equations (45) and (47) it is seen that
Hi,1can now be determined. Denote the first row ofGi,1by
gi,1=' vi,1 vi,1 ( (49)
wherevi,1andvi,1are scalars. Since the first row ofXi,1is zero, it is infered from equation (48) that
gi,1Jg∗
i,1>0 (50) sincegi,1Jg∗
i,1 equals to the squared magnitude of the upper-left 1×1entry ofLi,1, which is seen from equation (46) to be equal to the second diagonal element ofLi, which by definition is positive.
Moreover, it is deduced from inequality (50) that
|vi,1|>|vi,1|. (51) It can easily be checked that the matrix5
Qi,1= / 1 |vi,1|2− |vi,1|2 # v∗ i,1 −vi,1 −v∗i,1 vi,1 $ (52) isJ-unitary and also that
' v i,1 vi,1 (Qi,1= ' g i,1Jg∗i,1 0 ( . (53)
The previous analysis suggests thatHi,1can be designed as
Hi,1= # IK 0K ×2 02×K Qi,1 $ P1↔K (54)
whereP1↔Kpermutes columns1andK. As a result, Bi,2=Bi,1Hi,1=
# xi,0 0xi,1×1 1 02×(K−2) 02×2 Xi,2 Gi,2 $ (55) where the first row ofXi,2 is equal to zero, sinceXi,2 consists of the (K −2) rightmost columns of Xi,1. By taking the squared (IK⊕J)-norm of both sides it is verified by using similar arguments as before thatxi,1 equals toli,1, which is the non-zero part of the second column ofLi. By proceeding in a similar fashion forj =
2,3,· · ·,(K−1)it can be shown that
Bi,K=Bi,0Hi,0Hi,1· · ·Hi,K−1=' Li 0K×2 (. (56)
5. THE PROPOSED ALGORITHM
The proposed algorithm of this paper can now be stated:
Displacement-APA (DAPA)Select a filter orderM, a projection or-derK, a positive regularization parameter!, a positive step-sizeµ, setw−1=0M×1,L−1=√!IK,G−1 =0K×2,J=diag{1,−1}
and iterate fori≥0: 1. Compute√ 1 $ui$2+! and set Gi= / )ui)2+! 0 ui−1u∗i √ $ui$2+! ui−1u∗i √ $ui$2+! ui−2u∗i √ $ui$2+! ui−2u∗i √ $ui$2+! .. . ... ui−K+1u∗i √ $ui$2+! ui−K+1u∗i √ $ui$2+! . (57)
5The matrix (52) represents anelementary hyperbolic rotationand is a generalization of the unitary Givens elementary rotation matrix.
2. UsingZof equation (33) form the matrix
Bi,0=' ZLi−1 Gi (. (58)
3. Iterate forj= 0,1· · ·,(K−1): (a) Form the scalars
vi,j=Bi,j(j, K) (59)
vi,j=Bi,j(j, K+ 1) (60)
whereBi,j(j, K)andBi,j(j, K+ 1)denote the(j, K) and(j, K+ 1)respectively entries ofBi,j.
(b) Form the2×2J-unitary transformation
Qi,j=/ 1 |vi,j|2− |vi,j|2 # v∗ i,j −vi,j −vi,j∗ vi,j $ . (61) (c) Form the(IK⊕J)-unitary(K+ 2)×(K+ 2)
trans-formation Hi,j = # IK 0K ×2 02×K Qi,j $ Pj↔K
wherePj↔K is the permutation matrix that permutes
columnsjandK.
(d) ApplyHi,jtoBi,jand obtainBi,j+1
Bi,j+1=Bi,jHi,j. (62)
4. Obtain the Choleski factor of(UiU∗i +!IK)as the leftmost
K×Kblock ofBi,K
Li=Bi,K(0 :K−1,0 :K−1). (63)
5. Solve the two triangular systemsLici=eiandL∗ibi=ci.
6. Update the weight vector
wi=wi−1+µU∗ibi. (64)
Note that alwaysQi,0=I2since the second entry of the first row of the generator matrixGiis always equal to zero. This is implicitely stated in the algorithm formulation sincevi,0= 0.
6. COMPUTATIONAL COMPLEXITY
In this section it is shown that the proposed algorithm requires O(3K2)multiplications in order to compute the vectorbi.
To begin with, the computationally intensive operations per-formed by the proposed algorithm towards computing Li are the elementary hyperbolic rotations (EHRs) of the rows of{Gi,j}K−1
j=0 , where
Gi,j=Bi,j(j:K−1, K:K+ 1). (65) Each EHR is performed via a vector-matrix multiplication of size (1×2)(2×2)and hence requires 4 multiplications.
Now, at each iterationjthe aforementioned EHR is performed (K−j)times. The total number of EHRs performed is therefore
!4K−1
j=1 (K−j)
"
where it has been taken into consideration that no EHRs take place for j = 0. This leads to the order of total EHRs performed beingO5K2
2
6
and consequently, the order of total required multiplications isO(2K2).
Now, as can be seen from step 5 of the proposed algorithm,Li
is used to form two triangular systems of equations. These can be solved directly by forward and backward substitution atO(K2
2 )m each. The cost required for solving these two systems is therefore O(K2)multiplications.
50 100 150 200 250 300 350 400 450 500 −30 −20 −10 0 10 MSD (a) dB iteration index
Displacement APA (DAPA) standard APA using MATLAB solver
50 100 150 200 250 300 350 400 450 500 −350 −300 −250 −200 k(i) (b) dB iteration index error convergence to limit of numerical precision
Fig. 1. Comparison between a standard APA implementation using the MATLAB solver for the LSOE and the proposed implementa-tion using the propagated Choleski decomposiimplementa-tion (DAPA). Plot (a) shows the MSD of the two algorithms and plot (b) shows the Eu-clidean norm of the difference of the two computed solutions of the LSOE of each iteration.
7. SIMULATIONS
In this section, DAPA is compared to a standard APA implemen-tation, which solves the LSOE (4) using the MATLAB solver linsolve(Ri,ei,opts), where the fields SYM and POSDEF of the
structureoptshave been set totrue, while the others are set tofalse. The two implementations are compared in a system identifica-tion scenario, wherewois randomly generated and of unit norm. The
input signal is zero-mean, unit-variance white noise filtered through the systemH(z) = (1−0.9z−1)−1. The measurement noise is such so that the SNR at the output of the unknown system is30dB. The algorithmic parameters are set to the standard valuesM = 16, K = 8,µ = 1and!= 10−5. The results are averaged over 100 independent trials.
The Mean Square Deviation (MSD) for the two implementations is depicted at the top of figure 1, from which it is clear that they coincide. The fact that the two implementations are theoretically and practically equivalent is further verified by the extremely small values of the quantity
k(i) =77bi,(M AT LAB SOLV ER)−bi,DAP A
7
72 (66)
which is depicted at plot (b) of Figure 1, showing thatk(i) = 0 within the numerical limits of the computations.
8. CONCLUSIONS
The displacement structure theory for time-variant matrices has been applied to the APA context resulting in the fastest existing exact APA implementation to the best knowledge of the authors. The technique employed to solve the involved linear system of equations can also be used to derive a FAP algorithm.
Acknowledgment
The authors are thankful to Charalampos Nakos at the National Technical University of Athens for his comments on the manuscript.
9. REFERENCES
[1] K. Ozeki and T. Umeda, “An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its prop-erties,” Electron. Commun. Jpn., 1984, vol 67–A, no. 5, pp. 19–27.
[2] Y.R. Zheng and R.A. Goubran, “Adaptive beamforming using affine projection algorithms,” inProc. IEEE Int. Conf. Signal Process., August 2000, vol. 3, pp. 1929-1932.
[3] Steven L. Gay and Sanjeev Tavanthia, “The fast affine pro-jection algorithm,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1995, vol. 5, pp. 3023–3026. [4] S. Oh, D. Linebarger, B. Priest and B. Raghothaman, “A fast
affine projection algorithm for an acoustic echo cancellation using a fixed-point DSP processor,” inProc. IEEE ICASSP, 1997, vol. 5, pp. 4121–4124.
[5] H. Ding, “A stable fast affine projection adaptation algorithm suitable for low-cost processors,” inProc. IEEE ICASSP, 2000, Instabul, Turkey, pp. I-360–363.
[6] F. Albu, J. Kadlec, N. Coleman and A. Fagan, “The Gauss-Seidel fast affine projection algorithm,” InProc. IEEE Signal Process. Systems Workshop, October 2002, San Diegeo, CA, pp. 109–114.
[7] Yuriy V. Zakharov, “Low complexity implementation of the affine projection algorithm,” IEEE signal process. letters, pp. 1–4, April 2008.
[8] Q. G. Liu, B. Champagne and K. C. Ho, “On the use of a mod-ified fast affine projection algorithm in subbands for acous-tic echo cancellation,” inProc. IEEE Digital Signal Process. Workshop, 1996, pp. 354–357.
[9] Heping Ding, “Fast affine projection adaptation algorithms with stable and robust symmetric linear system solvers,”IEEE trans. on signal process., vol. 55, NO. 5, pp. 1730–1740, May 2007.
[10] Ali H. Sayed, Hanoch Lev-Ari and Thomas Kailath, “Time-variant displacement structure and triangular arrays,” IEEE Trans. on Signal Process., vol. 42, NO. 5 issue 8, pp. 1052– 1062, May 1994.
[11] Ph. Delsarte, Y. Genin and Y. Kamp, “On the mathematical foundations of the generalized Levinson algorithm,” inProc. IEEE ICASSP, May 1982, vol. 7, pp. 1717–1720.
[12] G. H. Golub and C. F. Van Loan, Matrix Computations, The John Hopkins University Press, Baltimore, MD, third edition, 1996.
[13] Thomas Kailath and Ali H. Sayed, “Displacement Structure: Theory and Applications,” SIAM review, vol. 37, NO. 3, pp. 297–386, September 1995.
[14] T. Kailath and A. H. Sayed, Fast Reliable Algorithms for Ma-trices with Structure, SIAM, 1999.
