A New Formalism of the Sliding Window Recursive Least Squares Algorithm and Its Fast Version

Full text

(1)IEICE TRANS. FUNDAMENTALS, VOL.E94–A, NO.6 JUNE 2011. 1394. PAPER. A New Formalism of the Sliding Window Recursive Least Squares Algorithm and Its Fast Version Kiyoshi NISHIYAMA†a) , Member. SUMMARY A new compact form of the sliding window recursive least squares (SWRLS) algorithm, the I-SWRLS algorithm, is derived using an indefinite matrix. The resultant algorithm has a form similar to that of the traditional recursive least squares (RLS) algorithm, and is more computationally efficient than the conventional SWRLS algorithm including two Riccati equations. Furthermore, a computationally reduced version of the ISWRLS algorithm is developed utilizing a shift property of the correlation matrix of input data. The resulting fast algorithm reduces the computational complexity from O(N 2 ) to O(N) per iteration when the filter length (tap number) is N, but retains the same tracking performance as the original algorithm. This fast algorithm is much easier to implement than the existing SWC FTF algorithms. key words: recursive least squares algorithm, sliding window, forgetting factor, fast algorithm, system identification, adaptive filter. 1.. Introduction. The recursive least squares (RLS) algorithm has been widely used in adaptive filtering, system identification, and adaptive control [1]–[4]. Typically the RLS algorithm is adopted as an adaptive algorithm for finite impulse response (FIR) filters whose coefficients are linearly combined with N input samples to approximate the desired output signal. The most popular adaptive scheme for the FIR filters is the least mean square (LMS) algorithm. Although the LMS algorithm is very simple, with a computational complexity of O(N), and is relatively robust to numerical errors, it suffers from very slow convergence, especially when applied to highly correlated excitation signals such as speech. In contrast, the RLS algorithm provides very fast convergence regardless of the characteristics of input signals. However, the RLS algorithm requires a computational complexity of O(N 2 ) per time step. This complexity often proves fatal to practical application of the RLS algorithm since acoustic systems require at least hundreds of filter coefficients. To reduce the computational burden, various fast versions of the RLS algorithm have been developed [5]–[10]. Furthermore, introduction of a forgetting factor to the RLS and fast RLS (FRLS) algorithms improved the tracking behavior in time-varying environments. Nevertheless, the tracking rate is generally slower than the initial convergence rate since the steady-state filter gain has to be much smaller than the initial gain [10]. This phenomenon is essentially Manuscript received October 18, 2010. Manuscript revised January 27, 2011. † The author is with the Department of Electrical Engineering and Computer Science, Faculty of Engineering, Iwate University, Morioka-shi, 020-8551 Japan. a) E-mail: [email protected] DOI: 10.1587/transfun.E94.A.1394. caused by the exponential forgetting window of input data whose size continues to increase with time. To overcome this tracking degradation, the sliding window RLS (SWRLS) algorithm and its fast version, called the SWC FTF algorithm, have been developed using the sample-correlation matrix of input data with finite sliding window [8]–[10]. The SWRLS algorithm is somewhat computationally inefficient and has no correspondence in form to the well-known RLS algorithm. The SWC FTF algorithm displays numerical deficiencies (frequent instability and strong dependency on the forgetting factor and filter length) and has a complicated initialization procedure [8]. Although a more stable SWC FTF algorithm has been derived by embedding the sliding window covariance (SWC) problem into a two-experiment prewindowed problem [11], the feasible lower bound of the forgetting factor is much close to one, thus restricting the tracking capability. In this paper, a new compact form of the SWRLS algorithm is derived using the 2 × 2 indefinite matrix W s with diagonal elements 1 and −ρLs where ρ is the forgetting factor and L s a sliding window length. This new formalism, designated the indefinite matrix-based SWRLS (I-SWRLS) algorithm, has a one-to-one correspondence with the traditional RLS algorithm. Although the computational complexity of the proposed algorithm is still of O(N 2 ), it is more computationally efficient than the conventional SWRLS algorithm with two Riccati equations. Furthermore, for real-time applications, its fast version, the indefinite matrix-based sliding window FRLS (I-SWFRLS) algorithm, is developed using a shift property of the correlation matrix of input data. The I-SWFRLS algorithm, which has a computational complexity of O(N) per iteration, is much easier to implement in programming than the existing SWC FTF algorithms. Especially, implementation of the stable SWC FTF algorithm is troublesome due to the two-experiment prewindowed problem [11]. The remainder of this paper is organized as follows. Section 2 overviews the traditional RLS algorithm, and Sect. 3 outlines the conventional SWRLS algorithm. In Sect. 4 we derive the I-SWRLS algorithm using some useful propositions. Based on the results of Sect. 4, a fast version of the I-SWRLS algorithm, the I-SWFRLS algorithm, is developed in Sect. 5. In Sect. 6 several simulations justify the I-SWRLS and I-SWFRLS algorithms. Finally, we present our conclusions in Sect. 7.. c 2011 The Institute of Electronics, Information and Communication Engineers Copyright .

(2) NISHIYAMA: A NEW FORMALISM OF THE SLIDING WINDOW RECURSIVE LEAST SQUARES ALGORITHM AND ITS FAST VERSION. 1395. the computational view of point. 2.. The RLS Algorithm and Its Limitation 3.. The most popular linear system model is that with an FIR structure. This restriction is imposed in order to simplify the estimation problem and to reduce the computational load in real-time applications. The FIR system with noise is expressed by yk = Uk xk + vk ,. k≥0. (1). Use of the sliding data window discards old data to reduce the influence of the past on the estimate, and proves to have a better tracking property than the RLS algorithm. Differentiating the exponentially weighted squares error function windowed by a finite length L s :. where yk is the observed output signal at time index k, xk corresponds to the finite impulse response or filter coefficients, vk is the observation noise, and Uk = [uk , uk−1 , . . . , uk−N+1 ]. (2). consisting of successive inputs {uk } is called the observation matrix. Here N denotes the filter length (tap number). Given a sequence of inputs {uk } and measured outputs {yk }, the optimal estimate of xk is obtained by minimizing the following exponentially windowed criterion: Jk =. k . ρk−i (yi − Ui xk )2. J˜k =. where ρ is a weight factor, called the forgetting factor, which can treat the most recent data as more important than the past data, and the expression of the forgetting effect is very suitable to a recursive manner of adaptive algorithms. This cost function is minimized by setting the gradient of Jk equal to 0, i.e., ∂Jk /∂xk = 0, and then the optimal estimate x̂k satisfies Rk x̂k = γk. (4). Rk =. ρk−i UTi Ui , γk =. i=0. ρk−i UTi yi .. −1. R̃k =. in which x̂k and Pk are initialized with x̂−1 = 0 and P−1 = ε0 I (ε0 > 0), respectively, and the forgetting factor is set to 0 < ρ ≤ 1 [1]. Here I is the identity matrix. Unfortunately, the initialization have a continuously decreasing effect in the estimation. Periodical reinitializations of the RLS algorithm constitutes a remedy for such estimation degradation. However, the requirement of an initialization for each data block causes inconvenience from. ρk−i UTi Ui , γ̃k =. k . ρk−i UTi yi .. (12). i=k−L s +1 −1. Defining P̃k = R̃k provides an alternative expression to the inverse matrix on the right-hand side of (11): −1. P̃k = ( P̆k − ρLs UTk−Ls Uk−Ls )−1. (13). in which P̆k is expressed with P̃k−1 as −1. P̆k = (ρ P̃k−1 + UTk Uk )−1 .. (14). Applying the matrix inversion lemma to (13) and (14), respectively, we obtain a recursive manner of (11) as [SWRLS Algorithm] [8], [9] x̂k = x̂k−1 + P̃k (UTk ỹk − ρLs UTk−Ls ỹk,Ls ) ⎛ ⎞ P̃k−1 UTk Uk P̃k−1 ⎟⎟⎟ 1 ⎜⎜⎜ P̆k = ⎝⎜ P̃k−1 − ⎠⎟ ρ ρ + Uk P̃k−1 UTk. (6). (7) (8) (9). k i=k−L s +1. Solving (4), we have. [RLS Algorithm] x̂k = x̂k−1 + kk (yk − Uk x̂k−1 ) kk = Pk−1 UTk (ρ + Uk Pk−1 UTk )−1 Pk = (Pk−1 − kk Uk Pk−1 )/ρ. (11). where. (5). which can also be solved with Pk = R−1 k in a time-recursive manner, resulting in the RLS algorithm as. (10). x̂k = R̃k γ̃k. i=0. x̂k = R−1 k γk. ρk−i (yi − Ui xk )2. with respect to xk and then setting the result to zero, we find an optimal estimate of xk which minimizes J˜k as. where k . k i=k−L s +1. (3). i=0. k . A Conventional SWRLS Algorithm. P̃k = P̆k +. P̆k UTk−Ls Uk−Ls P̆k 1/ρLs − Uk−Ls P̆k UTk−Ls. (15) (16) (17). where ỹk = yk − Uk x̂k−1 ỹk,Ls = yk−Ls − Uk−Ls x̂k−1 .. (18). Also x̂k and P̃k are initialized with x̂−1 = 0 and P̃−1 = ε0 I (ε0 > 0), respectively, and the forgetting factor is set to 0 < ρ ≤ 1. 4.. Derivation of a New Compact Form of the SWRLS Algorithm Using an Indefinite Matrix. Now, recalling the following relationship: −1. P̃k = R̃k. (19).

(3) IEICE TRANS. FUNDAMENTALS, VOL.E94–A, NO.6 JUNE 2011. 1396. we can rewrite (11) as x̂k = P̃k γ̃k. (20). in which P̃k and γ̃k can be expressed, respectively, as L s −1 −1 Ls P̃k = (P−1 k − ρ Pk−L s ) , γ̃k = γk − ρ γk−L s. (21). Similarly, we can obtain the recursive updating equation of γ̃k . [Proposition 2] A solution of the sliding window least squares problem x̂k = P̃k γ̃k is recursively given by x̂k = x̂k−1 + P̃k−1 CTk R−1 e,k (y̆k − Ck x̂k−1 ).. where P−1 k =. k . ρk−i UTi Ui , γk =. i=0. Note that. −1 P̃k. k . ρk−i UTi yi .. (22). i=0. = P−1 k and γ̃k = γk hold for k < L s .. Interestingly, it is possible to recursively update P̃k and γ̃k with indefinite matrices as follows. [Proposition 1] The inverse autocorrelation matrix P̃k and crosscorrelation vector γ̃k for sliding-windowed input data are recursively calculated as P̃k = ( P̃k−1 − P̃k−1 CTk R−1 e,k Ck P̃k−1 )/ρ. (23). CTk W s y̆k. (24). γ̃k = ργ̃k−1 +. (Proof) According to Proposition 1, we can obtain a recursive algorithm to solve the sliding window least squares problem as follows: x̂k = P̃k γ̃k = (ρ−1 P̃k−1 − ρ−1 P̃k−1 CTk R−1 e,k Ck P̃k−1 ) ×(ργ̃k−1 + CTk W s y̆k ) = ρ−1 P̃k−1 − ρ−1 P̃k−1 CTk −1 −1 ×(Ck ρ−1 P̃k−1 CTk + W −1 s ) Ck ρ P̃k−1 × ργ̃k−1 + CTk W s y̆k = P̃k−1 γ̃k−1 −1 −ρ−1 P̃k−1 CTk (ρ−1 Ck P̃k−1 CTk + W −1 s ) ×Ck P̃k−1 γ̃k−1 +ρ−1 P̃k−1 CTk W s y̆k −1 −ρ−2 P̃k−1 CTk (ρ−1 Ck P̃k−1 CTk + W −1 s ) ×Ck P̃k−1 CTk W s y̆k = x̂k−1 − P̃k−1 CTk R−1 e,k Ck x̂k−1. where Re,k = R s + Ck P̃k−1 CTk , R s = ρW −1 s Ws =. 1 0 0 −ρLs yk yk−Ls. y̆k =. Uk Uk−Ls. , Ck =. .. +ρ−1 P̃k−1 CTk W s y̆k T −ρ−1 P̃k−1 CTk R−1 e,k Ck P̃k−1 Ck W s y̆k. (25). = x̂k−1 − P̃k−1 CTk R−1 e,k Ck x̂k−1. (Proof) Recalling P−1 k. =. ρP−1 k−1. T +ρ−1 P̃k−1 CTk R−1 e,k (Re,k − Ck P̃k−1 Ck )W s y̆k. +. UTk Uk. = x̂k−1 − P̃k−1 CTk R−1 e,k Ck x̂k−1. (26). +ρ−1 P̃k−1 CTk R−1 e,k R s W s y̆k. we can rewrite the first equation of (21) as. = x̂k−1 + P̃k−1 CTk R−1 e,k (y̆k − Ck x̂k−1 ).. −1. L s −1 P̃k = P−1 k − ρ Pk−L s. L s −1 = ρ(P−1 k−1 − ρ Pk−L s −1 ). +UTk Uk − ρLs UTk−Ls Uk−Ls −1. = ρ P̃k−1 UTk. UTk−Ls. 1 0 0 −ρLs. −1. = ρ P̃k−1 + CTk W s Ck .. P̃k = ( P̃k−1 − K̃ k Ck P̃k−1 )/ρ. Uk Uk−Ls. P̃k = ( P̃k−1 −. (28). where Re,k = R s + Ck P̃k−1 CTk , R s = ρW −1 s .. (34). where (27). Then, applying the matrix inversion lemma to (27) yields P̃k−1 CTk R−1 e,k Ck P̃k−1 )/ρ. (31). From Propositions 1 and 2, we can newly derive a compact form of the SWRLS algorithm as follows: [I-SWRLS Algorithm] (Indefinite Matrix-Based SWRLS Algorithm) x̂k = x̂k−1 + K̃ k (y̆k − Ck x̂k−1 ) (32) T −1 K̃ k = P̃k−1 Ck Re,k (33). T = (ρP−1 k−1 + U k U k ) T −ρLs (ρP−1 k−L s −1 + Uk−L s U k−L s ). +. (30). (29). Re,k = R s + Ck P̃k−1 CTk Rs =. ρ 0 0 −ρ−Ls +1. y̆k =. yk yk−Ls. , 0<ρ≤1. , Ck =. Uk Uk−Ls. x̂−1 = 0, P̃−1 = ε0 I, ε0 > 0.. (35).

(4) NISHIYAMA: A NEW FORMALISM OF THE SLIDING WINDOW RECURSIVE LEAST SQUARES ALGORITHM AND ITS FAST VERSION. 1397 Table 1 Computational complexity of the I-SWRLS algorithm; Ck P̃k−1 is given by the transpose of the resulting P̃k−1 CTk , and scalar devisions such as 1/ρ is not counted. Computation. Multiplications. x̂k = x̂k−1 + K̃k (y̆k − Ck x̂k−1 ) K̃ k = P̃k−1 CTk R−1 e,k. 4N 2N 2 + 4N + 6. P̃k = ( P̃k−1 − K̃ k Ck P̃k−1 )/ρ. 3N 2. This form of the I-SWRLS algorithm, which is very compact and easy to comprehend, is very similar to that of the well-known RLS algorithm with the forgetting factor ρ. Additionally, this formalism leads to a computational reduction. Namely, the introduction of the indefinite matrix W s reduces the computational complexity from 7N 2 multiplications for the SWRLS to 5N 2 multiplications for the I-SWRLS, where complexity of order less than two is neglected. Table 1 shows the number of multiplications for each computation in the the I-SWRLS algorithm where ( P̃k−1 − K̃ k Ck P̃k−1 )/ρ is treated as ( P̃k−1 − K̃ k Ck P̃k−1 ) × 1ρ . 5.. = K̃ k W −1 s .. (37). Hence, we have K̃ k = K k W s .. Fortunately, the auxiliary gain matrix K k = P̃k CTk has an alternative expression with a computational requirement of O(N). [Proposition 4] The auxiliary gain matrix K k is updated with a computational complexity of O(N) as K k = mk − Dk μk. (38). where Dk = [ Dk−1 − mk W s ηk ][1 − μk W s ηk ]−1 mk μk. 0 K k−1. =. 1 1 S k Ak + Ck Dk−1 +. ηk = ck−N. A Fast Version of the I-SWRLS Algorithm. (39). [cTk + ATk CTk−1 ]. (40) (41). and Although the I-SWRLS algorithm possesses high tracking property, its computational complexity per iteration is of O(N 2 ). In this section, this drawback is solved in a way similar to deriving the fast H∞ filter [13], [14]. 5.1 Preliminaries In derivation of a fast version of the I-SWRLS algorithm, the following proposition plays a very important role. [Proposition 3] The filter gain K̃ k = P̃k−1 CTk R−1 e,k can be expressed with the auxiliary gain matrix K k = P̃k CTk as K̃ k = K k W s , W s =. 1 0 0 −ρLs. .. (36). (Proof) Recalling the last equation of (27), we can arrange the auxiliary gain matrix K k = P̃k CTk as −1 K k = P̃k CTk = ρ P̃k−1 + CTk W s Ck −1 CTk = ρ−1 P̃k−1 CTk −1 T −1 −ρ−1 P̃k−1 CTk W −1 s + Ck ρ P̃k−1 Ck ×Ck ρ−1 P̃k−1 CTk = ρ−1 P̃k−1 CTk −1 T −1 −ρ−1 P̃k−1 CTk W −1 s + ρ Ck P̃k−1 Ck T −1 −1 × (W −1 s + ρ Ck P̃k−1 Ck ) − W s = ρ−1 P̃k−1 CTk T −1 −1 −1 × W −1 Ws s + ρ Ck P̃k−1 Ck T −1 −1 = ρ P̃k−1 Ck I + ρ W s Ck P̃k−1 CTk −1 = ρ−1 P̃k−1 CTk R s + Ck P̃k−1 CTk −1 R s = ρ−1 P̃k−1 CTk R−1 e,k R s. Ak = Ak−1 − K k−1W s [ck + Ck−1 Ak−1 ] S k = ρS k−1 + [cTk + ATk CTk−1 ]W s [ck + Ck−1 Ak−1 ].. (42) (43). The recursions are initialized with K −1 = 0, A−1 = 0, S −1 = 1/ε0 , and D−1 = 0, respectively. (Proof) This proposition is proved in the same way as used in the derivation of the fast H∞ filter [13], [14]. Note that Hk in [13], [14] is replaced with Uk . 5.2 Derivation of a Fast I-SWRLS Algorithm From Propositions 3 and 4, we can immediately obtain a fast version of the I-SWRLS algorithm with O(N), which is referred to as the I-SWFRLS algorithm, as follows: [I-SWFRLS Algorithm] (Indefinite Matrix-Based SWFRLS Algorithm) (44) x̂k = x̂k−1 + K̃ k (y̆k − Ck x̂k−1 ) K̃ k = K k W s , K k = mk − Dk μk (45) Dk = [ Dk−1 − mk W s ηk ][1 − μk W s ηk ]−1 ηk = ck−N + Ck Dk−1 1 mk 0 1 = + [cTk + ATk CTk−1 ] μk K k−1 S k Ak S k = ρS k−1 + eTk W s ẽk ek = ck + Ck−1 Ak Ak = Ak−1 − K k−1W s ẽk ẽk = ck + Ck−1 Ak−1 where y̆k , W s , and Ck are given, respectively, by. (46).

(5) IEICE TRANS. FUNDAMENTALS, VOL.E94–A, NO.6 JUNE 2011. 1398 Table 2 Computational complexity of the I-SWFRLS algorithm; ATk CTk−1 is given by the transpose of the resulting Ck−1 Ak , and scalar devisions such as 1/S k is not counted. Computation. Multiplications. x̂k = x̂k−1 + K̃k (y̆k − Ck x̂k−1 ) K̃k = K k W s K k = mk − Dk μk Dk = [Dk−1 − mk W s ηk ][1 − μk W s ηk ]−1 ηk = ck−N + Ck Dk−1 mk 0 1 = + S1 [cTk + ATk CTk−1 ] k μk K k−1 Ak S k = ρS k−1 + eTk W s ẽk ek = ck + Ck−1 Ak Ak = Ak−1 − K k−1 W s ẽ ẽk = ck + Ck−1 Ak−1. y̆k =. yk yk−Ls. Ck =. Uk Uk−Ls. , Ws =. 4N 2N 2N 3N + 6 2N. N−1 hk (i)uk−i + vk , k = 1, 2, . . . , L yk =. (48). i=0. 2N + 2 5 2N 2N + 2 2N. 1 0 0 −ρLs (47). and ck is the first column of Ck and ck−N is the last column of Ck+1 . Also, x̂k , K k , Dk , S k , and Ak are initialized with x̂−1 = 0, K −1 = 0, D−1 =0, S −1 = 1/ε0 , ε0 > 0 and A−1 =0, respectively, and 0 < ρ ≤ 1. Note that this fast algorithm reduces to the fast Kalman filter as L s tends to infinity (ρLs → 0) when ρ < 1. This I-SWFRLS algorithm can be performed with 21N multiplications per iteration, whereas the normalized SWC FTF algorithm, which is a numerically stable version of the SWC FTF, requires 22N multiplications and 4 square roots [8]. The more stable SWC FTF algorithm [11] becomes considerably complicated due to the two-stage strategy, and includes a number of tuning parameters for stabilizing the algorithm. Table 2 shows the number of multiplications for each computation in the the I-SWFRLS algorithm. Also, the I-SWFRLS algorithm can be regarded as a fast technique for computing the solution of the set of equations R̃k x̂k = γ̃k for successive Toeplitz matrices R̃k , and might be more efficient than the Kumar’s fast algorithm for solving a Toeplitz system [15]. Indeed, the Kumar’s algorithm has a computational complexity of 5N×(log2 (2N−1))2 per each k for k > N which is greater in order than that of the I-SWFRLS algorithm, 21N. In arithmetic operations, the I-SWFRLS algorithm is superior to the Kumar’s algorithm for N ≥ 3 because when 5N × √(log2 (2N − 1))2 = 21N, N is nearly equal to 3 due to N = (2 21/5 + 1)/2. 6.. echo path whose impulse response {h0 (i)} at the initial time consists of {0.0, 0.008, -0.012, 0.064, 0.013, -0.052, -0.007, 0.039, 0.011, 0.0, -0.002, -0.009, -0.016, -0.013, -0.001, 0.004, 0.015, 0.013, 0.007, 0.0, -0.001, -0.002, -0.001, 0.0} for i < 24 and zero otherwise (24 ≤ i < N). The observed echo is given by. Simulation. The performance of the I-SWRLS and I-SWFRLS algorithms derived in this paper is evaluated for identification of an FIR system such as used in echo cancellers. All calculations are performed using the MATLAB (64-bit floating point arithmetic), except for the last example. As an unknown system to be identified, we consider an. where vk is a stationary, white Gaussian noise with zero mean and standard deviation σv = 1.0 × 10−4 , N denotes the length of the adjustable impulse response (tap number), and L stands for the length of observation data. Note that the linear model of the echo yk has the time-varying observation matrix Uk = [ uk , uk−1 , . . . , uk−(N−1) ] with a shifting property such that Uk+1 = [uk+1 , Uk (1), Uk (2), . . . , Uk (N − 1)], provided that xk = [ hk (0), hk (1), . . . , hk (N − 1) ]T . The input signal {uk } is generated by the following autoregressive (AR) model: uk = α1 uk−1 + α2 uk−2 + wk. (49). where α1 = 0.7, α2 = 0.2, and wk is a stationary, white Gaussian noise with zero mean and standard deviation σw = 0.04. In the above-mentioned initial setting, we have assumed that the underlying system is basically time-invariant. However, the motivation of the sliding window RLS algorithms is identification of time-variant systems. So, we consider the scenario that the impulse response of the echo path is suddenly shifted from the initial state to another one such as hk (i + 15) = hk−1 (i) at time k = 2000 and then remains constant after that time. Here the length of the impulse response (tap number) is set to N = 128, as used in echo cancellers of cellular phones. Figure 1(a) shows the tracking result of the changed impulse response by the I-SWRLS algorithm ((32)–(34)), along with that of the conventional RLS algorithm ((7)– (9)). Figure 1(b) shows a comparison of the I-SWFRLS algorithm ((44)–(46)) with the conventional FRLS algorithm (an exponentially weighted version of the fast Kalman filter [5]). Here, the window size and the forgetting factor are empirically determined as L s = 400 and ρ = 0.999 for this example. Also, all the algorithms are initialized as x̂−1 = 0, P−1 = P̃−1 = ε0 I, and ε0 = 100.0. As a measure of the estimation accuracy, we employ the squared norm of the tap error vector: e2tap (k) = (xk − x̂k )T (xk − x̂k ).. (50). From the results in Fig. 1, it is obvious that the I-SWRLS and I-SWFRLS algorithms successfully track the changed impulse response despite its sudden switch, whereas the performances of the conventional RLS and FRLS algorithms is not enough to track it quickly. The Fig. 2 shows the performance and stability of the stabilized SWC FTF algorithm with error feedback [11] for.

(6) NISHIYAMA: A NEW FORMALISM OF THE SLIDING WINDOW RECURSIVE LEAST SQUARES ALGORITHM AND ITS FAST VERSION. 1399. Fig. 1 Tracking performance of the I-SWRLS and I-SWFRLS algorithms; (a) comparison of the I-SWRLS with the RLS, (b) comparison of the I-SWFRLS with the conventional FRLS (FKF), where N = 128, L s = 400, ρ = 0.999, and ε0 = 100.0.. Fig. 3 Dependency of performance of the I-SWFRLS algorithm on the tuning parameters; (a) dependency on the window size L s when ρ = 0.999, (b) dependency on the forgetting factor ρ when L s = 400, where N = 128 and ε0 = 100.0.. Fig. 2 Performance and stability of the stabilized SWC FTF algorithm with error feedback for various values of a forgetting factor ρ where N = 96 and L s = 400.. Fig. 4 An AR signal of order 16, which is input to the I-SWFRLS algorithm.. various values of a forgetting factor ρ where N = 96 and L s = 400. When the forgetting factor is set to the same value (ρ = 0.999) as used in Fig. 1, the stabilized SWC FTF algorithm suddenly diverges once it converges, and didn’t work well for tap numbers more than 96. Generally, the stabilized FTF algorithm as well as the original one is numerically unstable and its stability is very sensitive to values of ρ and N. This numerical property of the FTF algorithms causes such sudden divergence of the stabilized SWC FTF algorithm. On the other hand, the dependency of performance of the I-SWFRLS algorithm on the window size L s and forgetting factor ρ is depicted in Fig. 3. When ρ = 0.99 and L s = 400 are selected, the I-SWFRLS algorithm provides the best performance for this example. Generally, decreasing ρ and L s accelerates the tracking rate of the slidingwindow RLS algorithms, but risks their numerical instability. So, for effective use of the I-SWFRLS algorithm, ρ and L s should be carefully determined for each application. Additionally, the I-SWFRLS algorithm with N = 128 and L s = 2000 was performed in 32-bit floating-point and fixed-point arithmetics for an AR signal of order 16 with data length L = 16000 as shown in Fig. 4, which is implemented by C programming language. Here the impulse. response is suddenly shifted from the initial state to another one such as hk (i + 15) = hk−1 (i) at time k = 6000. Note that the Q format in the fixed-point arithmetic is set to Q7.24, which is consistently used for every arithmetic operations. From these realistic examples, it is seen that the I-SWFRLS algorithm can provide high tracking performance for the more practical situation implemented in a single precision. 7.. Conclusions. A compact form of the SWRLS algorithm, the I-SWRLS algorithm, has been derived using the 2 × 2 indefinite diagonal matrix W s with diagonal elements 1 and −ρLs . Interestingly, the I-SWRLS algorithm has a form very similar to that of the RLS algorithm, and is more computationally efficient than the conventional SWRLS algorithm with two Riccati equations although the computational complexity is still of O(N 2 ). Furthermore, a computationally reduced version of the I-SWRLS algorithm, the I-SWFRLS algorithm, has been successfully developed using a shift property of the correlation matrix of input data. The I-SWFRLS algorithm possesses a computational complexity of O(N) per iteration, and could be much easier to implement on DSPs than the existing SWC FTF algorithms. This is because the.

(7) IEICE TRANS. FUNDAMENTALS, VOL.E94–A, NO.6 JUNE 2011. 1400. [10]. [11]. [12] [13]. [14]. [15]. [16] Fig. 5 Performance of the I-SWFRLS algorithm in 32-bit floating-point and fixed-point arithmetics; (a) tap errors in 32-bit floating-point arithmetic, (b) tap errors in 32-bit fixed-point arithmetic, where N = 128, L s = 2000, ρ = 0.999, and ε0 = 100.0.. existing SWC FTF algorithms necessarily uses two FTF algorithms as the SWRLS algorithm needs two Riccati equations. Hence, their implementation must become troublesome. Also, the I-SWFRLS algorithm coincides with the fast Kalman filter algorithm in the limit of L s = ∞ when ρ < 1. This implies that the I-SWFRLS algorithm can be regarded as a sliding window version of the fast Kalman filter. Additionally, the I-SWFRLS algorithm will be available for time-varying spectrum estimation since the algorithm includes the forward and backward linear prediction of input signal [16]. Another application of the I-SWFRLS algorithm, including speech recognition and speech coding, will be one of the future works. References [1] S. Haykin, Adaptive Filter Theory, 3rd ed., Prentice-Hall, 1996. [2] G. Carayannis, D. Manolakis, and N. Kalouptsidis, “A unified view of parametric processing algorithms for prewindowed signals,” Signal Process., vol.10, pp.335–368, 1986. [3] A.H. Sayed and T. Kailath, “A state-space approach to adaptive RLS filtering,” IEEE Signal Process. Mag., vol.11, no.3, pp.18–60, 1994. [4] G. Glentis, K. Berberidis, and S. Theodoridis, “Efficient least squares adaptive algorithms for FIR transversal filtering,” IEEE Signal Process. Mag., vol.16, no.4, pp.13–41, 1999. [5] L. Ljung, M. Morf and D. Falconer, “Fast calculation of gain matrices for recursive estimation schemes,” Int. J. Control, vol.27, no.1, pp.1–19, 1978. [6] J.M. Cioffi and T. Kailath, “Fast, recursive-least-squares transversal filters for adaptive filtering,” IEEE Trans. Acoust. Speech Signal Process., vol.ASSP-32, no.2, pp.304–337, 1984. [7] G.V. Moustakides and S. Teodoridis, “Fast Newton transversal filters — A new class of adaptive estimation algorithms,” IEEE Trans. Signal Process., vol.39, no.10, pp.2184–2193, 1991. [8] J.M. Cioffi and T. Kailath, “Windowed fast transversal filters adaptive algorithms with normalization,” IEEE Trans. Acoust. Speech Signal Process., vol.ASSP-33, no.3, pp.607–625, 1985. [9] B.-Y. Choi and Z. Bien, “Sliding windowed weighted recursive. least-squares method for parameter estimation,” Electron. Lett., vol.25, no.20, pp.1381–1382, 1989. H. Liu and Z. He, “A sliding-exponential window RLS adaptive filtering algorithm: Properties and applications,” Signal Process., vol.45, pp.357–368, 1995. D.T.M. Slock and T. Kailath, “A modular prewindowing framework for covariance FTF RLS algorithms,” Signal Process., vol.28, 1, pp.47–61, 1992. B. Hassibi, A.H. Sayed, and T. Kailath, Indefinite-Quadratic Estimation and Control, 1st ed., SIAM, 1999. K. Nishiyama, “Derivation of a fast algorithm of modified H∞ filters,” Proc. IEEE International Conference on Industrial Electronics, Control and Instrumentation, RBC-II, pp.462–467, 2000. K. Nishiyama, “An H∞ optimization and its fast algorithm for timevariant system identification,” IEEE Trans. Signal Process., vol.52, no.5, pp.1335–1342, 2004. R. Kumar, “A fast algorithm for solving a Toeplitz system of equations,” IEEE Trans. Acoust. Speech Signal Process., vol.ASSP-33, no.1, pp.254–267, 1985. T. Katsumata, K. Nishiyama, and K. Satoh, “Error analysis and numerical stabilization of the fast H∞ filter,” IEICE Trans. Fundamentals, vol.E93-A, no.6, pp.1153–1162, June 2010.. Kiyoshi Nishiyama was born in Tokyo, Japan in 1957. He received the M.E. degree in electrical engineering from Chiba University in 1985 and the degree of Dr. Eng. from Tokyo Institute of Technology in 1991. He joined the Department of Computer and Information Sciences, Iwate University in 1998. He is currently a Professor. His research interests are in estimation, optimization, and learning theory..

(8)

No results found