3 Second-order description of complex random vectors
Result 3.7. There is the majorization preordering
3.5 Independent component analysis
0 0.2 0.4 0.6 0.8 1
0 0.2 0.4 0.6 0.8 1
PFA
P D
0 0.2 0.4 0.6 0.8 1
0 0.2 0.4 0.6 0.8 1
PFA
P D
(a) (b)
Figure 3.5 Receiver operating characteristics of the GLRT detector. In (a), the SNR is fixed at 0 dB. From northwest to southeast, the curves correspond to phase tracking-error variance of 0.7, 0.95, 1.2, and 1.5. In (b), the phase tracking-error variance is fixed at 1. From northwest to southeast, the curves correspond to SNR of 5 dB, 0 dB, and−5 dB. In all cases, the number of samples was M= 1000.
3.5 Independent component analysis
An interesting application of the invariance property of the circularity coefficients is independent component analysis (ICA). In ICA, we observe a linear mixture y of independent complex components (sources) x, as described by
y= Mx. (3.100)
We will make a few simplifying assumptions in this section. The dimensions of y and x are assumed to be equal, and the mixing matrix M is assumed to be non-singular. The objective is to blindly recover the sources x from the observations y, without knowledge of M, using a linear transformation M#. This transformation M# can be regarded as a blind inverse of M, which is usually called a separating matrix.
Note that, since the model (3.100) is linear, it is unnecessary to consider widely linear transformations.
ICA seeks to determine independent components. Arbitrary scaling of x, i.e., multi-plication by a diagonal matrix, and reordering the components of x, i.e., multimulti-plication by a permutation matrix, preserves the independence of its components. The product of a diagonal and a permutation matrix is a monomial matrix, which has exactly one nonzero entry in each column and row. Hence, we can determine M#up to multiplication with a monomial matrix.
Standard ICA requires the use of higher-order statistical information, and the blind recovery of x cannot work if more than one source xiis Gaussian. If only second-order information is available, the best possible solution is to decorrelate the components, rather than to make them independent. This is done by determining the principal com-ponents UHy using the EVD Ryy= EyyH= UΛUH. However, the restriction to unitary
82 Second-order description of complex random vectors
ξξξ∗
Ryy K
Rxx K
ω x y
ξξξ A−1xx M Ayy
ω∗ A∗yy
y∗ M∗
x∗ A−∗xx
Figure 3.6 Two-channel model for complex ICA. The vertical arrows are labeled with the cross-covariance matrix between the upper and lower lines (i.e., the complementary covariance).
rather than general linear transformations wastes a considerable degree of freedom in designing the blind inverse M#.
In this section, we demonstrate that, in the complex case, it can be possible to determine M# using second-order information only. This was first shown byDeLathauwer and DeMoor (2002) and independently discovered byEriksson and Koivunen (2006). The key insight in our demonstration is that the independence of the components of x means that, up to simple scaling and permutation, x is already given in canonical coordinates.
The idea is then to exploit the invariance of circularity coefficients of x under the linear mixing transformation M.
The assumption of independent components x implies that the covariance matrix Rx x
and the complementary covariance matrix Rx x are both diagonal. It is therefore easy to compute canonical coordinates between x and x∗, denoted by = Ax xx. In the strong uncorrelating transform Ax x = FHx xR−1/2x x , R−1/2x x is a diagonal scaling matrix, and FHx xis a permutation matrix that rearranges the canonical coordinates such that ξ1corresponds to the largest circularity coefficient k1,ξ2to the second largest coefficient k2, and so on.
This makes the strong uncorrelating transform Ax xmonomial. As a consequence, also has independent components.
The mixture y has covariance matrix Ryy = MRx xMH and complementary covari-ance matrix Ryy= MRx xMT. The canonical coordinates of y and y∗are computed as
= Ayyy= FHyyR−1/2yy y, and∗= A∗yyy∗. The strong uncorrelating transform Ayy is determined as explained in Section3.2.2.
Figure3.6shows the connection between the different coordinate systems. The impor-tant observation is that and are both in canonical coordinates with the same circularity coefficients ki. In the next paragraph, we will show that and are related as = D
by a diagonal matrix D with diagonal entries±1, provided that all circularity coefficients are distinct. Since has independent components, so does . Hence, we have a solution to the ICA problem.
Result 3.9. The strong uncorrelating transform Ayyis a separating matrix for the com-plex linear ICA problem if all circularity coefficients are distinct.
The only thing left to show is that D= AyyMA−1x x is indeed diagonal with diagonal elements±1. Since and are both in canonical coordinates with the same diagonal
3.5 Independent component analysis 83
canonical correlation matrix K, we find E H= whenever ki = kj. Therefore, D is diagonal and unitary if all circularity coefficients are distinct. Since K is real, the corresponding diagonal entries of all nonzero circularity coefficients are actually ±1. On the other hand, components with identical circularity coefficient cannot be separated.
Example 3.5. Consider a source x= [x1, x2]T. The first component x1 is the signal-space representation of a QPSK signal with amplitude 2 and phase offset π/8, i.e., x1∈ {±2ejπ/8, ±2jejπ/8}. The second component x2, independent of x1, is the signal-space representation of a BPSK signal with amplitude 1 and phase offset π/4, i.e., x1∈ {±ejπ/4}. Hence,
In order to take x into canonical coordinates, we use the strong uncorrelating transform Ax x = FHx xR−1/2x x = monomial. The circularity coefficients are k1= 1 and k2= 0. Note that the circularity coefficients carry no information about the amplitude or phase of the two signals.
Now consider the linear mixture y= Mx with M=
In order to take y into canonical coordinates, we use the strong uncorrelating transform (rounded to four decimals)
Ayy= FHyyR−1/2yy =
0.7071 − 2.1213j 0.7071 + 0.7071j
−0.7045 − 0.0605j 0.3825 − 0.3220j
84 Second-order description of complex random vectors
is monomial, and
AyyMA−1x x =
0.7071 − 0.7071j 0 0 0.7650 − 0.6440j
is diagonal and unitary.
One final comment is in order. The technique presented in this section enables the blind separation of mixtures using second-order statistics only, under two crucial assumptions.
First, the sources must be complex and uncorrelated with distinct circularity coefficients.
Second, y must be a linear mixture. If y does not satisfy the linear model (3.100), the objective of ICA is to find components that are as independent as possible. The degree of independence is measured by a contrast function such as mutual information or negentropy. It is important to realize that the strong uncorrelating transform Ayy
is not guaranteed to optimize any contrast function. Finding maximally independent components in the nonlinear case requires the use of higher-order statistics.4
Notes
1 Much of the material presented in this chapter has been drawn fromSchreier and Scharf (2003a) andSchreier et al. (2005).
2 The circularity coefficients and the strong uncorrelating transform were introduced byEriksson and Koivunen (2006). The facts that the circularity coefficients are canonical correlations between x and x∗, and that the strong uncorrelating transform takes x into canonical coordinates were shown bySchreier et al. (2006) andSchreier (2008a). These two papers are cIEEE, and portions of them are reused with permission. More mathematical background on the Takagi factorization can be found inHorn and Johnson (1985).
3 The degree of impropriety and bounds in terms of eigenvalues were developed bySchreier (2008a).
4 Our discussion of independent component analysis barely scratches the surface of this rich topic.
A readable introductory paper to ICA isComon (1994). The proof of complex second-order ICA in Section3.5was first presented bySchreier et al. (2009). This paper is cIEEE, and portions are reused with permission.