White Noise

In practice, a normalized function, known as the cross correlation function, which is also be abbreviated as CCF, ρuy(l) = γuy(l) γuu(0) γyy(0) (6.33) is used.

Recall that we have suggested the use of time averages to estimate statistical averages. We will now state when such an estimation is valid. A random signal u(n) is said to be ergodic if all the statistical averages can be determined from a single realization with probability 1. That is, for an ergodic signal, time averages obtained from a single realization are equal to the statistical averages. For ergodic processes, the estimates of the statistical properties approach the actual values when a sufficiently large number of samples are taken while evaluating the summation. In the rest of this book, we will assume that we have a sufficient number of samples.

Unless otherwise stated, we will also assume that the noise process being studied is of zero mean. This helps simplify the calculations. In case the process under study does not obey this condition, we will subtract the mean, so that it becomes a zero mean process. The first zero mean process that we will study is white noise.

6.3.2

White Noise

The discrete time white noise sequence {ξ(k)} is a set of independent identically distributed (iid ) values belonging to a stationary stochastic process, with the following properties. The mean of white noise is zero. That is,

µξ= 0 (6.34)

Because ξ(k) is independent, its ACF is an impulse function:

γξξ(k) = σ2ξδ(k) = 

σ2_ξ k = 0

0 otherwise (6.35)

where σ2

ξ is the variance of{ξ(k)}. On taking the Z-transform, we obtain

Γξξ(z) = σ2ξ (6.36)

We obtain the Fourier transform of{γξξ(k)}, known as the power density spectrum

170 6. Identification

The reason why Γ(ejω) and ξ are called the power density spectrum and white noise, respectively, will be explained in Sec. 6.5.2.

Because white noise is uncorrelated with itself, it is easy to see that it is uncorrelated with any other sequence. For example, for an arbitrary sequence u different from ξ, we obtain

γuξ(k) = 0, ∀k (6.38)

Note that the above equation is true for k = 0 as well.

White noise, as defined in this section, is an idealization and it is difficult to create it. We approximate it with random number sequences. Even though not iid, these sequences satisfy Eq. 6.34–6.35. In Matlab, white noise is approximated using the command randn.

The concept of white noise is indispensable in system identification. Most noise sequences of interest can be modelled as filtered white noise. Moreover, because white noise is uncorrelated with every other sequence, we can simplify calculations quite a bit, as we will see in the rest of this chapter. We present a small example that demonstrates how white noise, combined with the concepts of ACF and CCF, can be used in estimating the model parameters of a simple system.

Example 6.6 Determine a procedure to find the model parameter a in the LTI system given by

y(n)− ay(n − 1) = ξ(n) (6.39)

where ξ is white and y is stationary.

Because y is stationary, by applying the expectation operation to Eq. 6.39, we see that y is of zero mean. Multiplying Eq. 6.39 by ξ(n−k) and taking the expectation, we obtain

γyξ(k)− aγyξ(k− 1) = γξξ(k) (6.40)

where we have made use of the definition of CCF, given in Eq. 6.29. In the rest of this chapter, we will carry out such calculations mechanically, without any further explanation. By evaluating Eq. 6.40 for k = 0, we obtain

γyξ(0) = γξξ(0) = σ2ξ (6.41)

where we have used the fact that

γyξ(−k) = 0, ∀k > 0 (6.42)

which is nothing but the causality condition of Eq. 6.32. Next, we substitute k = 1 in Eq. 6.40 to obtain γyξ(1)− aγyξ(0) = 0, where we have used Eq. 6.36. Using Eq. 6.41, we obtain

6.3. Covariance 171 One may be tempted to think that the above equation can be used to determine the model parameter a. Unfortunately, however, the white noise ξ is usually not measured and hence it will not be possible to calculate γyξ(k), or even its estimate,

ryξ(k). Nevertheless, we now explain how to make use of the above relation. Multiplying Eq. 6.39 by y(n− k) and, as before, taking the expectation,

γyy(k)− aγyy(k− 1) = γyξ(−k) (6.43)

By evaluating this for k = 1, we obtain

γyy(1)− aγyy(0) = 0 (6.44)

where we have used Eq. 6.42. We are now in a position to calculate the model parameter a. From the above equation, we obtain

a = γyy(1)

γyy(0)

(6.45)

Using the above calculations, we can also get an idea of the behaviour of ACF for AR(1) processes. By evaluating Eq. 6.43 for k = 0, we obtain

γyy(0)− aγyy(1) = σξ2 (6.46)

Solving Eq. 6.44 and Eq. 6.46 simultaneously, we obtain

γyy(0) =

σ_ξ2

1− a2 (6.47)

Using Eq. 6.42, we obtain also from Eq. 6.43 the following recursive relations: γyy(k) = aγyy(k− 1) = a2γyy(k− 2) = · · · = akryy(0) (6.48) Substituting the expression for γyy(0) from Eq. 6.47, we obtain

γyy(k) =

σ2

ξ 1− a2a

k _(6.49)

This equation shows that if |a| < 1, the ACF decays monotonically when a > 0 and with oscillations when a < 0.

Calculating model parameters in this manner is known as the theoretical prediction. In reality, however, we work with actual data and hence replace all the quantities of γ obtained above with their estimates, r. Thus, a may be determined from experimental data, through ACF calculations.

The above example shows how the ACF naturally gets to be used in parameter estimation problems and how the idealization of white noise plays a role in it.

We will next show that the ACF can be used to detect the periodicity of the underlying process.

172 6. Identification