Continuous random variables
4.2 Expectation of a single random variable
i=1
{xi}
=
∑
∞i=1P(X = xi) = 0, since, as argued above, each term is zero.
4.2 Expectation of a single random variable
For a discrete random variable X with probability mass function p, we computed expec-tations using the law of the unconscious statistician (LOTUS)
E[g(X)] =
∑
i
g(xi) p(xi).
Analogously, for a continuous random variable X with density f , we have E[g(X)] =
∞
−∞g(x) f (x)dx. (4.3)
In particular, taking g(x) = x yields
E[X] = ∞
−∞x f(x)dx.
We derive these formulas later in this section. For now, we illustrate LOTUS with several examples.
Example 4.7. If X is a uniform[a,b] random variable, find E[X], E[X2], and var(X).
Solution. To find E[X], write
E[X] = which is simply the numerical average of a and b.
To compute the second moment, write E[X2] = ∞
Example 4.8 (quantizer noise). An analog-to-digital converter or quantizer with reso-lution or step size∆ volts rounds its input to the nearest multiple of ∆ volts as shown in Figure 4.8. If the input is a random voltage Vinand the output is denoted by Vout, then the performance of the device is characterized by its mean squared error,E[|Vin−Vout|2]. In gen-eral it is difficult to compute this quantity. However, since the converter is just rounding to the nearest multiple of∆ volts, the error always lies between ±∆/2. Hence, in many cases it is assumed that the error Vin− Vout is approximated by a uniform[−∆/2,∆/2] random variable [18]. In this case, evaluate the converter’s performance.
/2 Vin
Vout
Figure 4.8. Input–output relationship of an analog-to-digital converter or quantizer with resolution ∆.
Solution. If X ∼ uniform[−∆/2,∆/2], then the uniform approximation allows us to write
E[|Vin−Vout|2] ≈ E[X2]
= var(X), since X has zero mean,
= (∆/2 − (−∆/2))2
12 , by the preceding example,
= ∆2 12.
Example 4.9. If X is an exponential random variable with parameterλ = 1, find all moments of X .
Solution. We need to compute
E[Xn] = ∞
0 xne−xdx.
Use integration by parts (see Note 4 for a refresher) with u= xn and dv= e−xdx. Then du= nxn−1dx, v= −e−x, and
E[Xn] = −xne−x'' ''∞
0
+ n ∞
0
xn−1e−xdx.
Using the fact that xne−x= 0 both for x = 0 and for x → ∞, we have E[Xn] = n ∞
0
xn−1e−xdx = nE[Xn−1], (4.4) Taking n= 1 yields E[X] = E[X0] = E[1] = 1. Taking n = 2 yields E[X2] = 2 · 1, and n = 3 yieldsE[X3] = 3 · 2 · 1. The general result is that E[Xn] = n!.
Observe that ∞
0
xne−xdx = ∞
0
x(n+1)−1e−xdx = Γ(n + 1).
Hence, the preceding example shows thatΓ(n + 1) = n!. In Problem 14(a) you will gener-alize the calculations leading to (4.4) to show thatΓ(p + 1) = p · Γ(p) for p > 0.
Example 4.10. Find the mean and variance of an exp(λ) random variable.
Since this last integral is the nth moment of the exp(1) random variable, which is n! by the last example, it follows that
E[Xn] = n!
Example 4.11. Let X be a continuous random variable with standard Gaussian density f ∼ N(0,1). Compute E[Xn] for all n ≥ 1.
Solution. Write
E[Xn] = ∞
−∞xnf(x)dx, where f(x) = exp(−x2/2)/√
2π. Since f is an even function of x, the above integrand is odd for n odd. Hence, all the odd moments are zero. For n≥ 2, write
E[Xn] = ∞
Integration by parts shows that this last integral is equal to
−xn−1e−x2/2'''∞
−∞+ (n − 1) ∞
−∞xn−2e−x2/2dx.
Since e−x2/2decays faster than any power of x, the first term is zero. Thus, E[Xn] = (n − 1) ∞
At this point, it is convenient to introduce the double factorial notation,
which is seen to be zero once we recognize the integral as having the form of the mean of an N(0,1) random variable Y.
To computevar(X), write
Making the same change of variable as before, we obtain E[(X − m)2] = σ2 ∞
−∞y2e−y2/2
√2π dy.
Now recognize this integral as having the form of the second moment of an N(0,1) random variable Y . By the previous example,E[Y2] = 1. Hence, E[(X − m)2] =σ2.
Example 4.13 (infinite expectation). Pareto densitiesbhave been used to model packet delay, files sizes, and other Internet characteristics. Let X have the Pareto density f(x) = 1/x2, x≥ 1. Find E[X].
Example 4.14. Determine E[X] if X has a Cauchy density with parameterλ= 1.
Solution. This is a trick question. Recall that as noted following Example 2.24, for signed discrete random variables,
if at least one of the sums is finite. The analogous formula for continuous random variables is
assuming at least one of the integrals is finite. Otherwise we say thatE[X] is undefined.
Since f is the Cauchy(1) density,
x f(x) = x · 1/π 1+ x2. Since this integrand has anti-derivative
1
bAdditional Pareto densities are considered in Problems 2, 23, and 26 and in Problem 59 in Chapter 5.
Derivation of LOTUS
Let X be a continuous random variable with density f . We first show that if g is a real-valued function taking finitely many distinct values yj∈ IR, then
E[g(X)] = ∞
−∞g(x) f (x)dx. (4.6)
To begin, observe that
P(Y = yj) = P(g(X) = yj) =
{x:g(x)=yj}f(x)dx.
Then write
E[Y] =
∑
j
yjP(Y = yj)
=
∑
j
yj
{x:g(x)=yj}f(x)dx
=
∑
j
{x:g(x)=yj}yjf(x)dx
=
∑
j
{x:g(x)=yj}g(x) f (x)dx
= ∞
−∞g(x) f (x)dx,
since the last sum of integrals is just a special way of integrating over all values of x.
We would like to apply (4.6) for more arbitrary functions g. However, if g(X) is not a discrete random variable, its expectation has not yet been defined! This raises the question of how to define the expectation of an arbitrary random variable. The approach is to approx-imate X by a sequence of discrete random variables (for which expectation was defined in Chapter 2) and then defineE[X] to be the limit of the expectations of the approximations.
To be more precise about this, consider the sequence of functions qnsketched in Figure 4.9.
___1 2n n
n x
q ( )n x
Figure 4.9. Finite-step quantizer qn(x) for approximating arbitrary random variables by discrete random variables.
The number of steps is n2n. In the figure, n= 2 and so there are eight steps.
Since each qntakes only finitely many distinct values, qn(X) is a discrete random variable for whichE[qn(X)] is defined. Since qn(X) → X, we then define5E[X] := limn→∞E[qn(X)].
Now suppose X is a continuous random variable. Since qn(X) is a discrete random variable, (4.6) applies, and we can write
E[X] := lim
n→∞E[qn(X)] = lim
n→∞
∞
−∞qn(x) f (x)dx.
Now bring the limit inside the integral,6and then use the fact that qn(x) → x. This yields E[X] = ∞
−∞lim
n→∞qn(x) f (x)dx = ∞
−∞x f(x)dx.
The same technique can be used to show that (4.6) holds even if g takes more than finitely many values. Write
E[g(X)] := lim
n→∞E[qn(g(X))]
= limn→∞ ∞
−∞qn(g(x)) f (x)dx
= ∞
−∞lim
n→∞qn(g(x)) f (x)dx
= ∞
−∞g(x) f (x)dx.