Reduced search space multiple shift maximum element sequential matrix
diagonalisation algorithm
J Corr
∗, K Thompson
∗, S Weiss
∗, I K Proudler
†, J G McWhirter
‡∗Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow, Scotland †School of Electronic, Electrical and Systems Engineering, Loughborough University, Loughborough, UK
‡School of Engineering, Cardiff University, Cardiff, Wales, UK
{jamie.corr,keith.thompson,stephan.weiss}@strath.ac.uk
Keywords: Multiple Shift Maximum Element; Sequen-tial Matrix Diagonalisation; Polynomial Matrix Eigenvalue Decomposition.
Abstract
The Multiple Shift Maximum Element Sequential Matrix Diagonalisation (MSME-SMD) algorithm is a powerful but costly method for performing approximate polynomial eigen-value decomposition (PEVD) for space-time covariance-type matrices encountered in e.g. broadband array processing. This paper discusses a newly developed search method that restricts the order growth within the MSME-SMD algorithm. In addi-tion to enhanced control of the polynomial degree of the paraunitary and parahermitian factors in this decomposition, the new search method is also computationally less demanding as fewer elements are searched compared to the original while the excellent diagonalisation of MSME-SMD is maintained.
1. Introduction
To accurately model the delay and multipath properties of broadband array processing systems explicit lag elements must be used rather than the phase shifts employed in narrowband systems. Using delays rather than phase shifts generates a space-time covariance matrix,R[τ], which captures both spa-tial and temporal aspects of the signals. Taking thez-transform of the space-time covariance matrix,R(z)•—◦R[τ], gener-ates the polynomial cross-spectral density (CSD) matrix. The structure of the CSD matrix produced can be seen as a matrix with polynomial elements or as a polynomial that has matrices as its coefficients. The CSD matrix, R(z), is parahermitian, i.e. R(z) = ˜R(z), where the parahermitian operation,{·}˜ , consists of a complex conjugate transpose and time rever-sal, such that R˜(z) = RH(z−1). The traditional eigenvalue
decomposition (EVD) which is used in many narrowband sig-nal processing problems is not directly applicable to the poly-nomial CSD matrix. The polypoly-nomial EVD (PEVD) [1] can be seen as an extension of the EVD to the polynomial matrix case. The PEVD can be used to factorise the parahermitian,
R(z), into
R(z)≈Q˜(z)D(z)Q(z) , (1)
whereQ(z)is paraunitary such thatQ(z) ˜Q(z) = ˜Q(z)Q(z) =
I and D(z) is a diagonal polynomial matrix. Although an exact decomposition in (1) cannot be guaranteed [1], [2] sug-gests the approximation can be accurate for sufficiently high orders ofQ(z). Recently a wide variety of applications for the PEVD have arisen in areas such as broadband angle of arrival estimation [3], filter bank-based channel coding [4], subband coding [5], and the design of broadband precoding and equalisation of MIMO systems [6]. The polynomial sub-space decomposition techniques, including [4,6,3], require an accurate PEVD with low order paraunitary matrices to reduce the computational cost of the application. The decomposition in (1) can be calculated through a variety of different PEVD algorithms [1,7–10]. Since the PEVD is only unique to within multiplication by all-pass filters each algorithm may produce a different decomposition and each has its own merits. This paper considers the recently developed sequential matrix diag-onalisation (SMD) family of algorithms [8,9]. Compared to other PEVD algorithms, such as the SBR2 algorithm, [1] the SMD methods produce decompositions exhibiting a greater degree of diagonalisation (thus greater accuracy) but are more computationally costly to implement. The original SMD algo-rithm [8] has been shown to give good results and the lowest order paraunitary matrices. The results of the multiple shift maximum element (MSME) SMD [9] are better in terms of convergence however the multiple shifts cause the order of the paraunitary matrices to grow faster. The aim of this paper is to restrict the order growth seen in MSME-SMD whilst maintaining a similar level of diagonalisation performance. A further benefit is that computational cost is also reduced due to less data being processed. Sec. 2. reviews the current state-of-the-art sequential matrix diagonalisation algorithms. Sec. 3. analyses the worst case polynomial order growth for the SMD and MSME-SMD algorithms. Sec. 4. then highlights the ben-efits of the restricted search in the MSME-SMD algorithm. Simulation results are presented in Sec. 5. to compare the different PEVD methods and conclusions are given in Sec. 6.
2. PEVD Algorithms based on Sequential Matrix Diago-nalisation
SMD and MSME-SMD algorithms in detail.
The SMD family of algorithms has an initialisation step where all off-diagonal energy of the zero lag is transferred onto the diagonal via an EVD,
S(0)[0] =Q(0)R[0]Q(0)H . (2)
The modal matrixQ(0)(z) is then applied to all lags in the parahermitian matrix,
S(0)(z) =Q(0)R(z)Q(0)H . (3)
At each iteration a generic PEVD algorithm consists of three main steps, first a search is carried out to determine which row(s) and column(s) are to be brought onto the zero lag. This search step is algorithm dependent and will be discussed in more detail below. Next the selected row(s) and column(s) are shifted onto the zero lag by means of a paraunitary shift oper-ation,
S(i)′(z) =Λ(i)(z)S(i−1)(z) ˜Λ(i)(z), i= 1. . . I . (4)
Each PEVD iteration is then completed by bringing the off-diagonal energy at lag zero onto the off-diagonal, which is done by calculating and applying the modal matrix for the EVD of the zero lag to all lags of the parahermitian matrix,
S(i)(z) =Q(i)S(i)′(z)Q(i)H . (5)
All SMD algorithms stop when either a set number of itera-tions,I, have been carried out or the search step returns a value which is below a predefined threshold. Upon completion the approximately diagonal parahermitian matrix isS(I)(z), and the product of the individual delay and EVD operations is used to construct the paraunitary matrix for the decomposition i.e.
Q(I)(z) =G(I)(z). . .G(1)(z)G(0)(z) , (6)
where the unitary matrices,G(i)(z), are constructed from the delay and EVD operations from each of theIsteps,
G(i)(z) =Q(i)Λ(i)(z) . (7)
2.1 Sequential Matrix Diagonalisation
To determine which elements are brought onto the zero lag the original SMD algorithm uses a search based on column norms within the parahermitian matrix
{k(i), τ(i)}= arg max
k,τ kˆs
(i−1)
k [τ]k2, i= 1. . . I , (8)
whereˆs(ki−1)[τ]is a modified column vector which contains all elements excluding the on-diagonal entry. Once the column with the largest norm is found it is brought onto the zero lag using the delay matrix
Λ(i)(z) = diag{1 . . . 1
| {z }
k(i)−1
z−τ(i) 1 . . . 1
| {z }
M−k(i)
} (9)
where the parameters k(i) and τ(i) are the column and lag
indices obtained in (8).
b
b
(a)
search(b)
space
search space
search space
search space
search space
searc
h
space
[image:2.595.316.562.93.258.2](c)
(d)
Figure 1. View of a5×5parahermitian matrix during theith iteration,
not showing the lag dimension: starting from the top2×2matrix containing
the maximum off-diagonal element in (a), (b) shows an example of an ele-ment resistant to permutations, the third and fourth stages of the set of reduced search space strategy are shown in (b) and (d).
2.2 Multiple Shift Maximum Element SMD
For the multiple shift maximum element search, step (8) is modified to use thel∞rather than thel2norm,
{k(i), τ(i)
k(i)}= arg max k,τ kˆs
(i−1)
k [τ]k∞, i= 1. . . I , (10)
where the symbols have the same meanings as (8). The maxi-mum element search is carried out a total ofM−1times during each iteration in an attempt to maximise the amount of energy brought onto the zero lag
The parahermitian symmetry limits the maximum number of elements that can be moved onto the zero lag without affecting previous choices toM −1. Even then some choices result in fewer elements being able to be moved. To ensure that the full quota ofM −1maximum elements are transferred onto the zero lag and that the shifts do not adversely affect one another, a set of reduced search spaces is required [9]. The masks used to reduce the search spaces are shown in Fig. 1 for the case whereM = 5.
Prior to Fig. 1 (a) the first, global, maximum element is found (without any restrictions) and permuted into the top left
2×2sub-matrix. Any element found in the search space iden-tified in Fig. 1 (a) can be brought onto the zero lag and per-muted into the top left3×3sub-matrix with out affecting the initial global maximum. If the second element was found in the position of elementb in Fig. 1 (b) it would not be possi-ble to permute it into the upper left 3×3 sub-matrix. As a result we would have to proceed to Fig. 1 (d) meaning a total of only 3 elements could be found. Using the MSME-SMD search method, following the search space in Fig. 1 (a) and a permutation to bring a second element (from the search space in Fig. 1 (a)) into the upper left 3×3 sub-matrix, Fig.1 (c) is obtained. The search, shift and permute process is repeated using the mask in Fig. 1 (c), however this time elementbfrom Fig. 1 (b) could be chosen. Finally the fourth or(M −1)-th element is found using the mask in Fig. 1 (d).
required, they are only used here to help illustrate the search and mask process, without permutations the search spaces sim-ply become split up and are in different locations from Fig. 1. After the(M −1)maximum elements have been located they are then transferred onto the zero lag using the delay matrix
Λ(i)(z) =diagnz−τ1(i) z−τ2(i) . . . z−τM(i)
o
. (11)
3. Polynomial Order Growth
This section analyses the worst case polynomial order growth for both the SMD and MSME-SMD algorithms. It is likely that the actual polynomial matrix growth will be less than the worst case at each iteration. The analysis below assumes we have a parahermitian matrix,S(i)(z), at thei-th iteration with a size ofM ×M×2L+ 1, i.e. the maximum lag in either direction is|L|.
The growth in order of the parahermitian, S(i)(z), and paraunitary,Q(i)(z), matrices is determined by the order of the shift matrix,Λ(i)(z). To help analyse the problem we define the largest possible shift as ∆max and the highest possible
order for the shift matrixΛmax.
In the case of the SMD algorithm the growth is simply deter-mined by the magnitude ofτ(i)found in (8) which can have a
maximum value ofL, therefore∆max =L. With∆max =L
the maximum SMD shift matrix length, Λmax = L. When
Λ(i)(z)of orderΛmaxis applied toS(i)(z)andQ(i)(z)their
order will increase by2Λmaxor in this case2L. The
parame-terΛmaxis doubled because it is used to advance/delay a
col-umn and delay/advance a row onto the zero lag which grows the polynomial order in both directions.
For the MSME-SMD algorithm each of the(M −1)shifts can potentially interact such that the maximum shift length,
∆max is⌈((M −1)L)/2⌉. The multiple shift algorithm can
both delay and advance elements onto the zerolag in a sin-gle iteration usingΛ(i)(z)therefore the maximum shift matrix order,Λmax, is2⌈((M−1)L)/2⌉. As with SMD, whenΛ(i)(z)
of orderΛmaxis applied to the polynomial matrices their order
will increase by2Λmax. For the MSME-SMD algorithm the
worst case polynomial order growth is4⌈((M−1)L)/2⌉. Even with reasonably small values ofMthe multiple shift algorithm can result in a significant increase in the worst case polynomial order growth.
The growth in polynomial order can be curtailed using appropriate parahermitian [11] and paraunitary [12,13] trim functions. Both methods are permitted to remove up to a pre-defined threshold of energy,µ, from the outer lags of the poly-nomial matrices. For a parahermitian matrix the trim is done symmetrically taking advantage of its parahermitian nature. The parahermitian property is also preserved by the trim func-tion. In the case of paraunitary matrices the trim function is applied to both sides of Q(z) asymmetrically because the outer lags of a paraunitary matrix will have different energies. The paraunitary property is replaced by near-paraunitarity after the trim function is applied but the extent of this can be minimised by using the row-shift corrected trim from [13].
Table 1. Summary of worst case polynomial order growth for the different SMD variants .
SMD MSME RS-MSME
∆max L ⌈((M−1)L)/2⌉ L
Λmax L 2⌈((M −1)L)/2⌉ 2L
Ord. Growth 2L 4⌈((M −1)L)/2⌉ 4L
To reduce computational costs of the PEVD algorithms the parahermitian truncation can be carried out at the end of every iteration, the resulting maximum total loss in energy afterI
iterations isI×µP H. As the paraunitary matrix is only ever
calculated when the PEVD is complete the trim function is only applied once and so the resulting energy loss has a maxi-mum ofµP U.
4. Restricted Search MSME-SMD
In the restricted search (RS) MSME-SMD we impose an extra condition on the search spaces in Fig. 1 to control the polyno-mial order growth inS(i)(z)andQ(i)(z). Rather than allowing every search to select elements from any lag, we restrict it to elements closer to the zero lag than the global maximum, found during the first search of each iteration. The new approach still uses (10) but now once the first search of thei-th iteration finds a maximum element onτk(i()i), the lag parameter,τ, in (10) is restricted such that|τ| ≤ |τk(i()i)|for the remaining searches in thei-th iteration. Using this method the worst case maximum shift, ∆max, is L, the maximum order for the shift matrix,
Λmax, is2Land the polynomial order growth is4L. For
com-parison the maximum shift, shift matrix order and polynomial order growth are summarised in Tab. 1 for all three SMD vari-ants. The worst case scenario sees the RS-MSME-SMD order grow twice as fast as SMD but this is significantly lower than the original MSME-SMD, especially when the matrix width
M is increased.
Ultimately limiting the search space to lower lags will result in missing some elements and slow the algorithm’s con-vergence slightly but these missed elements are likely to be found by searches during future iterations. The reduced search space will benefit the real time performance in two ways; first the searches during one iteration where the restriction,
|τ| ≤ |τk(i()i)|, is applied will be on fewer elements and second the slower growth in parahermitian matrix means searches and matrix operations in future iterations will be over fewer elements.
5. Results
To illustrate the performance of the different PEVD algorithms we first present the performance metrics, followed by the sim-ulation set up. Finally the results are presented and the perfor-mance of the PEVD algorithms is analysed.
5.1 Performance Metrics
test will measure diagonalisation performance: the remaining off-diagonal energy afteriiterations normalised by the energy in the initial parahermitian matrix,R[τ],
Enorm(i) =
P
τ
PM k=1kˆs
(i)
k [τ]k
2 2
P
τkR[τ]k2F
, (12)
wherek · kF represents the Frobenius norm and the vector,
ˆ
s(ki)[τ], in the numerator is the same as that used in (8) which contains all but the on diagonal elements.
The main objective of the search space restriction discussed in this paper is to limit the order growth in the polynomial matrix. With this in mind the order of the parahermitian and paraunitary matrices are recorded after each iteration.
An added benefit of the reduction in parahermitian matrix order is a reduction in the computational cost of calculating the PEVD. Here we use execution time as a measure of the com-putational complexity of the PEVD algorithms implemented in Matlab 2014a with the following system specification: Ubuntu 14.04 on a workstation with IntelR XeonR E5-1607V2 3.00
GHz x 4 cores and 8 GB RAM.
5.2 Simulation Set Up
The results were obtained using an ensemble of103
paraher-mitian matrices produced using the source model from [14] where the source model is not majorised and has an average dynamic range of approximately30dB. The source model is randomised so that the parahermitian matrices produced are unique for each instantiation. The parahermitian matrix,R(z), is R(z) ∈ C6×6 with the initial number of lags set to 119.
Each of the PEVD algorithms was run for200iterations with the performance metrics recorded after each iteration. The sim-ulations are first run usingµP H =µP U = 0, i.e. only
remov-ing zero filled lags, then repeated over the same ensemble for
µP H =µP U = 10−6.
5.3 Algorithm Convergence
Fig. 2 shows the reduction in off-diagonal energy vs. algo-rithm iterations for the SMD algoalgo-rithm and the two versions of MSME-SMD. Despite the reduced search space we can see for the example in Fig. 2 both MSME-SMD algorithms transfer a similar amount of energy at each iteration and follow an almost identical convergence curve. The amount of energy transferred by both MSME-SMD algorithms is also significantly higher than the SMD method.
As discussed in Sec. 4. the original MSME-SMD transfers marginally more energy per iteration than the new RS-MSME-SMD algorithm in Fig. 2 however these are still significantly better than convergence of the SMD approach.
5.4 Paraunitary/Parahermitian Matrix Order
This section investigates one of the main goals of the RS-MSME-SMD algorithm which is to reduce the growth in poly-nomial order of the parahermitian and paraunitary matrices.
0 20 40 60 80 100 120 140 160 180 200
−16 −14 −12 −10 −8 −6 −4
iteration indexi
5
lo
g10
E
{
E
(
i
)
n
o
rm
}
/
[d
B
]
SMD MSME-SMD RS-MSME-SMD
[image:4.595.315.561.98.263.2]µP H= 10−6
Figure 2. Diagonalisation vs. algorithm iterations for the SMD algorithm and the two MSME-SMD varients.
102 103
−16 −14 −12 −10 −8 −6 −4
parahermitian matrix order
5
lo
g10
E
{
E
(
i
)
n
o
rm
}
/
[d
B
]
MSME-SMD RS-MSME-SMD SMD
[image:4.595.314.561.312.481.2]µP H= 10−6
Figure 3. Reduction in off-diagonal energy vs. growth in parahermitian matrix order.
Figs. 3 & 4 show the order growth of the parahermitian and paraunitary matrices for each of the selected PEVD algo-rithms. Generally the SMD method produces parahermitian and paraunitary matrices of lower order than any of the other PEVD methods [10]. Here we can see in both Fig. 3 & 4 that the matrices produced by RS-MSME-SMD are significantly shorter than their MSME-SMD equivalent and are a similar level to SMD. Even when a truncation algorithm such as those described in [11] and [13] are applied to the parahermitian and paraunitary matrices the reduced search method still out-performs the original MSME-SMD as shown in Fig. 3 & 4 although it does lose out slightly to SMD.
5.5 Real Time Execution
101 102 103 −16
−14 −12 −10 −8 −6 −4
paraunitary matrix order
5
lo
g10
E
{
E
(
i
)
n
o
rm
}
/
[d
B
]
MSME-SMD RS-MSME-SMD SMD
[image:5.595.46.291.97.267.2]µP U= 10−6
Figure 4. Reduction in off-diagonal energy vs. growth in paraunitary matrix order.
5 10 15 20 25 30 35
−16 −15 −14 −13 −12 −11 −10 −9 −8 −7 −6 −5
mean execution timeE{t}/ [s]
5
lo
g10
E
{
E
(
i
)
n
o
rm
}
/
[d
B
]
SMD MSME-SMD RS-MSME-SMD
µP H= 10−6
Figure 5. real time convergence of PEVD algorithms, diagonalisation mea-sure vs. mean execution time.
is more efficient than the original MSME search, in fact the new method takes on average around 10 seconds less than its predecessor to complete 200 iterations. When the parahermi-tain truncation methods are included both MSME-SMD vari-ants obtain a significant performance improvement, whereas the same change in SMD has a lesser effect. The performance benefits of the reduced search MSME-SMD are not as obvious when the parahermitian truncation is used but it still performs better than the original MSME-SMD.
6. Conclusion
Through analysis of the polynomial order growth of the SMD and MSME-SMD algorithms we have proposed a new search method which can significantly reduce the polynomial order growth of the MSME-SMD algorithm. Results indicate almost no degradation in energy transfer between the existing and pro-posed method. In addition the experiments have shown that the new method leads to a significant reduction in polynomial matrix order growth even when truncation methods are used. The reduced search spaces and resulting lower order
paraher-mitian matrices also result in an improved real time conver-gence. When the parahermitian and paraunitary matrices are truncated the benefits of the new search method are reduced. In general the restriction of the search space slows the growth of both paraunitary and parahermitian matrices which leads to a faster execution time with minimal impact on the algorithm convergence.
Acknowledgement
This work was supported by the Engineering and Phys-ical Sciences Research Council (EPSRC) Grant number EP/K014307/1 and the MOD University Defence Research Collaboration in Signal Processing.
References
[1] J.G. McWhirter, P.D. Baxter, T. Cooper, S. Redif, J. Foster. An EVD Algorithm for Para-Hermitian Polynomial Matrices.IEEE Trans. SP, 55(5):2158–2169, May 2007.
[2] S. Icart, P. Comon. Some Properties of Laurent Polynomial Matrices. In Conf. Math. Signal Proc., Birmingham, UK, Dec. 2012.
[3] M. Alrmah, S. Weiss, S. Lambotharan. An Extension of the MUSIC Algorithm to Broadband Scenarios using Polyno-mial Eigenvalue Decomposition. In EUSIPCO, pp. 629–633, Barcelona, Spain, Aug. 2011.
[4] S. Weiss, S. Redif, T. Cooper, C. Liu, P.D. Baxter, J.G. McWhirter. Paraunitary Oversampled Filter Bank Design for Channel Coding.J. Applied SP, 2006.
[5] S. Redif, J. McWhirter, S. Weiss. Design of FIR Paraunitary Filter Banks for Subband Coding using a Polynomial Eigenvalue Decomposition.IEEE Trans. SP, 59(11):5253–5264, Nov. 2011. [6] C.H. Ta, S. Weiss. A Design of Precoding and Equalisation for Broadband MIMO Systems. In15th Int. Conf. DSP, pp. 571– 574, Cardiff, UK, July 2007.
[7] A. Tkacenko. Approximate Eigenvalue Decomposition of Para-Hermitian Systems through Successive FIR Paraunitary Trans-formations. In IEEE ICASSP, pp. 4074–4077, Dallas, TX, Mar. 2010.
[8] S. Redif, S. Weiss, J. McWhirter. Sequential Matrix Diagonaliza-tion Algorithms for Polynomial EVD of Parahermitian Matrices.
IEEE Trans. SP, 63(1):81–89, Jan. 2015.
[9] J. Corr, K. Thompson, S. Weiss, J. McWhirter, S. Redif, I. Proudler. Multiple Shift Maximum Element Sequential Matrix Diagonalisation for Parahermitian Matrices. In IEEE SSP, pp. 312–315, Gold Coast, Australia, June 2014.
[10] J. Corr, K. Thompson, S. Weiss, I. Proudler, and J. McWhirter. Causality-Constrained Multiple Shift Sequential Matrix Diago-nalisation for Parahermitian Matrices InEUSIPCO, Lisbon, Por-tugal, September 2014.
[11] J. Foster, J.G. McWhirter, J. Chambers. Limiting The Order of Polynomial Matrices within the SBR2 Algorithm. InIMA Math. Signal Proc., Cirencester, UK, Dec. 2006.
[12] C.H. Ta, S. Weiss. Shortening the Order of Paraunitary Matrices in SBR2 Algorithm. InInt. Conf. Inf. Comm. & SP, pp. 1-5, Singapore, Dec. 2007.
[image:5.595.45.292.315.484.2]
