Foundations for measuring multivariate association between two complex random vectors

The correlation coefficient between two scalar real zero-mean random variables u and v is defined as

ρuv= √ Euv Eu²√

Ev² = √ Ruv

Ruu

√R_vv. (4.1)

The correlation coefficient is a convenient measure for how closely u andv are related.

It satisfies−1 ≤ ρuv≤ 1. If ρuv= 0, then u and v are uncorrelated. If |ρuv| = 1, then u is a linear function ofv, or vice versa, with probability 1:

u= Ruv

R_vvv. (4.2)

4.1 Measuring multivariate association 87

0 1 2

0 1 2

(b)

0 1 2

0 1 2

(a)

Figure 4.1 Scatter plots of 100 sample pairs of u andv with different ρuv.

In general, for|ρuv| ≤ 1,

ˆ u = Ruv

R_vvv (4.3)

is a linear minimum mean-squared error (LMMSE) estimate of u fromv, and the sign ofρuvis the sign of the slope of the LMMSE estimate. An expression for the MSE is

E| ˆu − u|²= Ruu− R_u²_v

R_vv = Ruu(1− ρu²v). (4.4) Example 4.1. Consider a mean, unit-variance, real random variable u and a zero-mean, unit-variance, real random variable n, uncorrelated with u. Now letv = au + bn for given real a and b. We find

ρuv =√ Ruv

Ruu

√R_vv = a

√a²+ b².

Figure 4.1depicts 100 sample pairs of u andv, for uniform u and Gaussian n. Plot (a) shows the case a= 0.8, b = 0.1, which results in ρuv= 0.9923. Plot (b) shows a = 0.8, b = 0.4, which results in ρuv= 0.8944. The line is the LMMSE estimate v = au (or, equivalently, ˆu = aˆ ⁻¹v).

How may we define a correlation coefficient between a pair of complex random vectors to measure their multivariate association? We shall consider the extension from real to complex quantities and the extension from scalars to vectors separately, and then combine our findings.

4.1.1 Rotational, reflectional, and total correlations for complex scalars

Consider a pair of scalar complex zero-mean random variables x and y. As a straight-forward extension of the real case, let us define the complex correlation coefficient

ρx y = E x y^∗

E|x|²

E|y|² = Rx y

√Rx x

Ryy

, (4.5)

88 Correlation analysis

0 1 2

0 1 2

(a) 0 1 2

0 1 2

(b)

Figure 4.2 Sample pairs of two complex random variables x and y withρx y= exp(jπ/2) and

|Rx y|/Ryy= 1.2. Plot (a) depicts samples of x and (b) samples of y, in the complex plane. For corresponding samples we use the same symbol.

which satisfies 0≤ |ρx y| ≤ 1. The LMMSE estimate of x from y is ˆx(y)= Rx y

Ryy

y= |Rx y| Ryy

e^{j
Rx y}y, (4.6)

which achieves the minimum error

E| ˆx(y) − x|²= Rx x−|Rx y|² Ryy

= Rx x(1− |ρx y|²). (4.7) Hence, if|ρx y| = 1, ˆx(y) is a perfect estimate of x from y. Figure4.2depicts five sample pairs of two complex random variables x and y withρx y = exp(jπ/2), in the complex plane. Plot (a) shows samples of x and (b) the corresponding samples of y. We observe that (b) is simply a scaled and rotated version of (a). The amplitude is scaled by the factor|Rx y|/Ryy, preserving the aspect ratio, and the rotation angle is
Rx y =
ρx y.

Now what about complementary correlations? Instead of estimating x as a linear func-tion of y, we may employ the conjugate linear minimum mean-squared error (CLMMSE) estimator

ˆx(y^∗)= E x y

E|y|²y^∗= R
x y

Ryy

y^∗= |
Rx y| Ryy

e<sup>j

R</sup>x yy^∗. (4.8) The corresponding correlation coefficient is

ρx y = R
x y

√Rx x

Ryy

, (4.9)

with 0≤ |
ρx y| ≤ 1, and the CLMMSE is E| ˆx(y^∗)− x|²= Rx x−|
Rx y|²

Ryy = Rx x(1− |
ρx y|²). (4.10) Hence, if |
ρx y| = 1, ˆx(y^∗) is a perfect estimate of x from y^∗. Figure4.3depicts five sample pairs of two complex random variables x and y with ρ
x y = exp(jπ/2), in the complex plane. Plot (a) shows samples of x and (b) the corresponding samples of y.

4.1 Measuring multivariate association 89

0 1 2

0 1 2

(a)

0 1 2

0 1 2

(b)

Figure 4.3 Sample pairs of two complex random variables x and y withρ
x y= exp(jπ/2) and|R
x y|/Ryy= 1.2. Plot (a) depicts samples of x and (b) samples of y, in the complex plane.

Samples of y correspond to amplified and reflected samples of x. The reflection axis is the dashed line, which is given by
R
x y/2 =
ρ
x y/2 = π/4.

We observe that (b) is a scaled and reflected version of (a). The amplitude is scaled by the factor|
Rx y|/Ryy, preserving the aspect ratio. Since
x =

Rx y−
y, we have, with probability 1:

x −¹₂

Rx y = −

y − ¹₂

Rx y

. (4.11)

Thus, the reflection axis is

Rx y/2 =

ρx y/2, which is the dashed line in Fig.4.3.

Depending on whether rotation or reflection better models the relationship between x and y,|ρx y| or |
ρx y| will be greater. We note the ease with which the best possible reflection axis is determined as half the angle of the complementary correlation
Rx y

(or half the angle of the correlation coefficientρ
x y). This would be significantly more cumbersome in real-valued notation.

Of course, data might exhibit a combination of rotational and reflectional correlation, motivating use of a widely linear minimum mean-squared error (WLMMSE) estimator

ˆx(y, y^∗)= αy + βy^∗, (4.12)

where α and β are chosen to minimize E| ˆx(y, y^∗)− x|². We will be discussing WLMMSE estimation in much detail in Section 5.4. At this point, we content our-selves with stating that the solution is

ˆx(y, y^∗)= ˆx(y)− ˆx[ ˆy^∗(y)]+ ˆx(y^∗)− ˆx[ ˆy(y^∗)]

1− |
ρyy|²

=(Rx yRyy−
Rx yR
^∗_yy)y+ (
Rx yRyy− Rx yR
yy)y^∗

R²_yy− |
Ryy|² , (4.13) where
Ryy= Ey²is the complementary variance of y,|
ρyy|²= |
Ryy|²/R²yyis the degree of impropriety of y, ˆy^∗(y)= [
R^∗_yy/Ryy]y is the CLMMSE estimate of y^∗ from y, and

ˆy(y^∗)= [
Ryy/Ryy]y^∗is the CLMMSE estimate of y from y^∗.

90 Correlation analysis

Through the connection

E| ˆx(y, y^∗)− x|²= Rx x(1− ¯ρ²x y), (4.14) we obtain the corresponding squared correlation coefficient

ρ¯x y² =|ρx y|²+ |
ρx y|²− 2 Re[ρx yρ
^∗x yρ
yy] 1− |
ρyy|²

=(|Rx y|²+ |
Rx y|²)Ryy− 2 Re(Rx yR
^∗_{x y}R
yy)

Rx x(R²_yy− |
Ryy|²) , (4.15) with 0≤ ¯ρx y² ≤ 1. We note that the correlation coefficient ¯ρx y, unlikeρx y = ρyx^∗ and

ρx y =
ρyx, is not symmetric in x and y: in general, ¯ρx y² = ¯ρyx² .

The correlation coefficient ¯ρx yis bounded in terms of the coefficientsρx yandρ
x yas max(|ρx y|², |
ρx y|²)≤ ¯ρx y² ≤ min(|ρx y|²+ |
ρx y|², 1). (4.16) The lower bound holds because a WLMMSE estimator subsumes both the LMMSE and CLMMSE estimators, so we must have WLMMSE≤ LMMSE and WLMMSE ≤ CLMMSE. However, there is no general ordering of LMMSE and CLMMSE, which we write as LMMSE CLMMSE.

A common scenario in which the lower bound is attained is when y is maximally improper, i.e., Ryy = |
Ryy| ⇔ |
ρyy|²= 1, which yields a zero denominator in (4.15).

This means that, with probability 1, y^∗ = e^jαy for some constantα, and Rx y = e^jαR
x y. In this case, y and y^∗carry exactly the same information about x. Therefore, WLMMSE estimation is unnecessary, and can be replaced with either LMMSE or CLMMSE esti-mation. In the maximally improper case, ¯ρx y² = |ρx y|²= |
ρx y|². Two other examples of attaining the lower bound in (4.16) are either
Rx y = 0 and
Ryy= 0 (i.e.,
ρx y = 0 andρ
yy = 0), which leads to ¯ρ²x y = |ρx y|², or Rx y = 0 and
Ryy = 0 (i.e., ρx y = 0 and

ρyy= 0), which yields ¯ρx y² = |
ρx y|²; cf. (4.15).

The upper bound ¯ρ²x y = |ρx y|²+ |
ρx y|²is attained when the WLMMSE estimator is the sum of the LMMSE and CLMMSE estimators: ˆx(y, y^∗)= ˆx(y) + ˆx(y^∗). In this case, y and y^∗carry completely complementary information about x. This is possible only for uncorrelated y and y^∗, that is, a proper y. It is easy to see that
Ryy = 0 ⇔ |
ρyy|²= 0 in (4.15) leads to ¯ρ²x y = |ρx y|²+ |
ρx y|². The following example gives two scenarios in which the lower and upper bounds are attained.

Example 4.2. Attaining the lower bound. Consider a complex random variable y= e^jα(u+ n),

where u is a real random variable andα is a fixed constant. Further, assume that n is a real random variable, uncorrelated with u, and Rnn = Ruu. Let x = Re (e^jαu)= cos(α)u. The LMMSE estimator ˆx(y) = ¹₂cos(α)e^−jαy and the CLMMSE estimator ˆx(y^∗)= ¹₂cos(α)e^jαy^∗ both perform equally well. However, they both extract the same information from y because y is maximally improper. Hence, a WLMMSE estimator has

4.1 Measuring multivariate association 91

no performance advantage over an LMMSE or CLMMSE estimator:|ρx y|²= |
ρx y|²= ρ¯²x y =¹₂.

Attaining the upper bound. Consider a proper complex random variable y and let x be its real part. The LMMSE estimator of x from y is ˆx(y)= ¹₂y, the CLMMSE estimator is ˆx(y^∗)=¹₂y^∗, and the WLMMSE estimator ˆx(y, y^∗)= ¹₂y+¹₂y^∗produces a perfect estimate of x. Here, rotational and reflectional models are equally appropriate.

Each tells only half the story, but they complement each other perfectly. It is easy to see that |ρx y|²= |
ρx y|²=¹₂ and ¯ρx y² = |ρx y|²+ |
ρx y|²= 1. (This also shows that ρ¯²x y = ¯ρyx² since it is obviously impossible to perfectly reconstruct y as a widely linear function of x.)

The following definition sums up the main findings of this section, and will be used to classify correlation coefficients throughout this chapter.

Definition 4.1. A correlation coefficient that measures how well a

In document Statistical Signal Processing of Complex-Valued Data.pdf (Page 108-113)