M ATLAB Write M ATLAB scripts to solve Problems 23 and 24.

2.4: Expectation

32. Compute E[X] if P(X = 2) = 1/3 and P(X = 5) = 2/3. 33. If X ∼ geometric0(1/2), compute E[I(2,6)(X)].

34. If X is Poisson(λ), compute E[1/(X + 1)].

35. A random variable X has mean 2 and variance 7. Find E[X2_].

36. If X has mean m and varianceσ2_{, and if Y := cX, find the variance of Y.}

37. Compute E[(X +Y)3_{] if X ∼ Bernoulli(p) and Y ∼ Bernoulli(q) are independent.}

38. Let X be a random variable with mean m and varianceσ2_{. Find the constant c that best}

approximates the random variable X in the sense that c minimizes the mean-squared

errorE[(X − c)2_].

39. The general shape of (x/a)r _{in Figure 2.11 is correct for r> 1. Find out how Fig-}

ure 2.11 changes for 0< r ≤ 1 by sketching (x/a)r and I_[a,∞)(x) for r = 1/2 and

r= 1.

40. Let X ∼ Poisson(3/4). Compute both sides of the Markov inequality, P(X ≥ 2) ≤ E[X]

41. Let X ∼ Poisson(3/4). Compute both sides of the Chebyshev inequality, P(X ≥ 2) ≤ E[X2]

4 .

42. Let X and Y be two random variables with means mX and mY and variancesσX2and

σ2

Y. LetρXY denote their correlation coefficient. Show thatcov(X,Y) =σXσYρXY.

Show thatcov(X,X) = var(X).

43. Let X and Y be two random variables with means mX and mY, variances σX2 and

σ2

Y, and correlation coefficientρ. Suppose X cannot be observed, but we are able to

measure Y . We wish to estimate X by using the quantity aY , where a is a suitable constant. Assuming mX = mY = 0, find the constant a that minimizes the mean-

squared errorE[(X − aY)2_{]. Your answer should depend onσ}

X,σY, andρ.

44. Show by counterexample that being uncorrelated does not imply independence. Hint: LetP(X = ±1) = P(X = ±2) = 1/4, and put Y := |X|. Show that E[XY] = E[X]E[Y], butP(X = 1,Y = 1) = P(X = 1)P(Y = 1).

45. Suppose that Y := X1+ ··· + XM, where the Xkare i.i.d. geometric1(p) random vari-

ables. FindE[Y2_].

46. Betting on fair games. Let X ∼ Bernoulli(p). For example, we could let X = 1 model the result of a coin toss being heads. Or we could let X = 1 model your winning the lottery. In general, a bettor wagers a stake of s dollars that X= 1 with a bookmaker who agrees to pay d dollars to the bettor if X= 1 occurs; if X = 0, the stake s is kept by the bookmaker. Thus, the net income of the bettor is

Y := dX − s(1 − X),

since if X = 1, the bettor receives Y = d dollars, and if X = 0, the bettor receives

Y = −s dollars; i.e., loses s dollars. Of course the net income to the bookmaker is

−Y. If the wager is fair to both the bettor and the bookmaker, then we should have E[Y] = 0. In other words, on average, the net income to either party is zero. Show that a fair wager requires that d/s = (1 − p)/p.

47. Odds. Let X∼ Bernoulli(p). We say that the (fair) odds against X = 1 are n2to n1(written n2: n1) if n2and n1are positive integers satisfying n2/n1= (1 − p)/p.

Typically, n2and n1are chosen to have no common factors. Conversely, we say that

the odds for X = 1 are n1to n2if n1/n2= p/(1 − p). Consider a state lottery game

in which players wager one dollar that they can correctly guess a randomly selected three-digit number in the range 000–999. The state offers a payoff of $500 for a correct guess.

(a) What is the probability of correctly guessing the number? (b) What are the (fair) odds against guessing correctly?

(c) The odds against actually offered by the state are determined by the ratio of the payoff divided by the stake, in this case, 500 :1. Is the game fair to the bettor? If not, what should the payoff be to make it fair? (See the preceding problem for the notion of “fair.”)

_{48. These results are used in Examples 2.24 and 2.25. Show that the sum} Cp := ∞

∑

diverges for 0< p ≤ 1, but is finite for p > 1. Hint: For 0 < p ≤ 1, use the inequality

 k+1 k 1 tpdt ≤  k+1 k 1 kpdt = 1 kp,

and for p> 1, use the inequality

 k+1 k 1 tpdt ≥  k+1 k 1 (k + 1)pdt = 1 (k + 1)p.


_{49. For C}

p as defined in Problem 48, if P(X = k) = C−1p /kp for some p> 1, then X

is called a zeta or Zipf random variable. Show thatE[Xn] < ∞ for n < p − 1, and E[Xn_{] = ∞ for n ≥ p − 1.}

50. Let X be a discrete random variable taking finitely many distinct values x1,...,xn.

Let pi:= P(X = xi) be the corresponding probability mass function. Consider the

function

g(x) := −logP(X = x).

Observe that g(xi) = −log pi. The entropy of X is defined by

H(X) := E[g(X)] = n

∑

∑

If all outcomes are equally likely, i.e., pi= 1/n, find H(X). If X is a constant random

variable, i.e., pj= 1 for some j and pi= 0 for i = j, find H(X).

_{51. Jensen’s inequality. Recall that a real-valued function g defined on an interval I is}

convex if for all x,y ∈ I and all 0 ≤λ ≤ 1,

gλx+ (1 −λ)y ≤ λg(x) + (1 −λ)g(y).

Let g be a convex function, and let X be a discrete random variable taking finitely many values, say n values, all in I. Derive Jensen’s inequality,

E[g(X)] ≥ g(E[X]).

Hint: Use induction on n.


_{52. Derive Lyapunov’s inequality,}

E[|Z|α]1/α ≤ E[|Z|β]1/β, 1 ≤α<β < ∞.

Hint: Apply Jensen’s inequality to the convex function g(x) = xβ/αand the random variable X= |Z|α.

_{53. A discrete random variable is said to be nonnegative, denoted by X ≥ 0, if P(X ≥}

0) = 1; i.e., if

∑

I_[0,∞)(xi)P(X = xi) = 1.

(a) Show that for a nonnegative random variable, if xk< 0 for some k, then P(X =

xk) = 0.

(b) Show that for a nonnegative random variable,E[X] ≥ 0.

(c) If X and Y are discrete random variables, we write X≥ Y if X −Y ≥ 0. Show that if X≥ Y, then E[X] ≥ E[Y]; i.e., expectation is monotone.

Exam preparation

You may use the following suggestions to prepare a study sheet, including formulas men- tioned that you have trouble remembering. You may also want to ask your instructor for additional suggestions.

2.1. Probabilities involving random variables.Know how to do basic probability calcu- lations involving a random variable given as an explicit function on a sample space.

2.2. Discrete random variables.Be able to do simple calculations with probability mass functions, especially the uniform and the Poisson.

2.3. Multiple random variables. Recall that X and Y are independent if P(X ∈ A,Y ∈

B) = P(X ∈ A)P(Y ∈ B) for all sets A and B. However, for discrete random variables,

all we need to check is whether or notP(X = xi,Y = yj) = P(X = xi)P(Y = yj), or, in

terms of pmfs, whether or not pXY(xi,yj) = pX(xi) pY(yj) for all xiand yj. Remember

that the marginals pX and pY are computed using (2.8) and (2.9), respectively. Be

able to solve problems with intersections and unions of events involving independent random variables. Know how the geometric1(p) random variable arises.

2.4. Expectation. Important formulas include LOTUS (2.14), linearity of expectation (2.15), the definition of variance (2.16) as well as the variance formula (2.17), and expectation of functions of products of independent random variables (2.23). For sequences of uncorrelated random variables, the variance of the sum is the sum of the variances (2.28). Know the difference between uncorrelated and independent. A list of common pmfs and their means and variances can be found inside the front cover. The Poisson(λ) random variable arises so often that it is worth remembering, even if you are allowed to bring a formula sheet to the exam, that its mean and variance are bothλand that by the variance formula, its second moment isλ+λ2. Similarly, the mean p and variance p(1 − p) of the Bernoulli(p) are also worth remembering. Your instructor may suggest others to memorize. Know the Markov inequality (for nonnegative random variables only) (2.18) and the Chebyshev inequality (for any random variable) (2.21) and also (2.22).

A discrete random variable is completely characterized by its pmf, which is the collection of numbers pX(xi). In many problems we do not know the pmf. However,

the next best things to know are the mean and variance; they can be used to bound probabilities as in the Markov and Chebyshev inequalities, and they can be used for approximation and estimation as in Problems 38 and 43.

Work any review problems assigned by your instructor. If you finish them, re-work your homework assignments.

More about discrete random variables