When dealing with a set of two or more random variables, it is sometimes convenient to group those random variables into a vector and use vector and matrix notation. A vector whose components are random variables is called a random vector. In this section a random vector is denoted by a bold upper case symbol such as9
where the Xi are random variables, and the realization of the random vector (i.e., the
value that it takes on) is denoted by a bold italic lower case symbol
The notation is thus consistent with that used in the earlier parts of the text. To keep the section brief, we discuss random vectors in the context of continuous random variables, although a similar development could be done for purely discrete random variables if needed.
5.7.1 Cumulative distribution and density functions
Let us define the notation X≤x to mean
(5.63)
9 Throughout this text, vectors are represented by column vectors and the superscript T denotes
vector or matrix transpose.
The cumulative distribution function (CDF) for the random vector is then defined as (5.64) As in the one- and two-dimensional cases, this function is monotonic and continuous from the right in each of the n arguments. It equals or approaches 0 as any of the arguments Xi approach minus infinity and equals or approaches one when all of the Xi
The PDF for the random vector (which is in reality a joint PDF among all of its components) is given by
(5.65) The PDF for a random vector has the probabilistic interpretation
(5.66) where denotes the event
(5.67) and where the ∆i are small increments in the components of the random vector.
The probability that X is any region of the n-dimensional space is given by the multidimensional integral
(5.68) In particular, if R is the entire n-dimensional space then
(5.69)
5.7.2 Expectation and moments
If G is any (scalar, vector, matrix) quantity that depends on X, then the expectation of
G(X) can be defined as
(5.70) If G is a vector or matrix, (5.70) is interpreted as the application of the expectation operation to every component of the vector or element of the matrix. Thus, the result of taking the expectation of a vector or matrix is another vector or matrix of the same size.
The quantities usually of most interest for a random vector are the first and second moments. The mean vector is defined by taking G(X)=X:
(5.71) The mean vector is a vector of constants
The correlation matrix is defined by
(5.73) Note that the term XXT is an outer product rather than an inner product of vectors; thus the result is a matrix, not a scalar. This matrix has the form
The off-diagonal terms represent the correlations between all pairs of vector components, and the diagonal terms are the second moments of the components. The mean vector and correlation matrix thus provide a complete description of the random vector using moments up to second order.
The covariance matrix is the matrix of second central moments of the random vector and is defined by
(5.74) This matrix has the form
The covariance and correlation matrices are related as
(5.75) (The proof of this is similar to the proof of (4.15).) For the case of a two-dimensional random vector this equation has the form
This is just the matrix embodiment of the relations (4.15) and (5.28), so it is not anything new.
As in the case of a single random variable, it is ofter easier to compute the covariance matrix using (5.75) than from the definition (5.74). The following example illustrates this computation.
Example 5.11: The two jointly-distributed random variables described in Example 5.6 are taken to be components of a random vector
The mean vector and correlation matrix for this random vector are given by
and
where the numerical values of the moments are taken from Example 5.6. The covariance matrix can then be computed from (5.75) as
The elements , c, and correspond to the numerical values computed in the earlier Example (5.6).
5.7.3 Multivariate Gaussian density function
The multidimensional or multivariate Gaussian density function for an n-dimensional random vector X is defined by
(5.76)
In the 2-dimensional case mX and CX are given by
where ρ is the correlation coefficient defined by (5.29). In this case the explicit inverse of the matrix can be found as
while the determinant is given by
Substituting these last two equations in (5.76) yields the bivariate density function for two jointly Gaussian random variables given as equation (5.35).
In Section 5.4 it is shown that the marginal and conditional densities derived from the bivariate density are all Gaussian. This property also extends to the multivariate Gaussian
density with n>2. In particular, any set of components of the random vector has a jointly Gaussian PDF, and any set of components conditioned on any other set is also jointly Gaussian. The multivariate Gaussian density is especially important for problems in signal processing and communications. A more extensive treatment including the case of complex random variables can be found in [3].
5.7.4 Transformations of random vectors Transformations of moments
Translations and linear transformations have easily formulable effects on the first and second order moments of random vectors because the order of the moments is preserved. The results of nonlinear transformations are not easy to formulate because first and second order moments of the new random variable generally require higher order moments of the original random variable. Our discussion here is thus restricted to translations and linear transformations.
When a random vector is translated by adding a constant, this has the effect of adding the constant to the mean. Consider
(5.77) where b is a constant vector. Then taking the expectation yields
or
(5.78)
It will be seen shortly, that while a translation has an effect on the correlation matrix, it has no effect on the covariance matrix because the covariance matrix is defined by subtracting the mean.
Consider now a linear transformation of the form
(5.79) where A is an m×n matrix.10 The mean of Y is easily found as
or
The correlation matrix of Y can be found from the definition
or
(5.81) It can easily be shown that the covariance matrix satisfies an identical relation (see Prob. 5.37). Note that the matrix A in this transformation does not need to be in-vertible or does not even need to be a square matrix. In other words, the linear transformation may result in a compression or expansion of dimensionality. The results are valid in any case.
The complete set of first and second order moments under a combination of linear transformation and translation is listed for convenience in Table 5.4. Notice that the transformation Y=AX+b
mean vector mY= AmX+b
correlation matrix
covariance matrix CY=ACXAτ