The binomial theorem says that

n

∑

Derive this result for nonnegative a and b with a+ b > 0 by using the fact that the binomial(n, p) probabilities sum to one. Hint: Take p = a/(a + b).

11. A certain digital communication link has bit-error probability p. In a transmission of n bits, find the probability that k bits are received incorrectly, assuming bit errors occur independently.

12. A new school has M classrooms. For i = 1,...,M, let nidenote the number of seats

in the ith classroom. Suppose that the number of students in the ith classroom is binomial(ni, p) and independent. Let Y denote the total number of students in the

school. FindP(Y = k).

13. Let X1,...,Xnbe i.i.d. withP(Xi= 1) = 1− p and P(Xi= 2) = p. If Y := X1+···+Xn,

findP(Y = k) for all k.

14. Ten-bit codewords are transmitted over a noisy channel. Bits are flipped indepen- dently with probability p. If no more than two bits of a codeword are flipped, the codeword can be correctly decoded. Find the probability that a codeword cannot be correctly decoded.

15. Make a table comparing both sides of the Poisson approximation of binomial proba- bilities, _ n k  pk(1 − p)n−k ≈ (np) k_e−np k! , n large, p small,

for k= 0,1,2,3,4,5 if n = 150 and p = 1/100. Hint: If MATLABis available, the binomial probability can be written

nchoosek(n,k) ∗ p^k ∗ (1 − p)^(n − k) and the Poisson probability can be written

(n ∗ p)^k ∗ exp(−n ∗ p)/factorial(k).

3.3: The weak law of large numbers

16. Show that E[Mn] = m. Also show that for any constant c, var(cX) = c2var(X).

17. Student heights range from 120 to 220 cm. To estimate the average height, determine how many students’ heights should be measured to make the sample mean within 0.25 cm of the true mean height with probability at least 0.9. Assume measurements are uncorrelated and have varianceσ2= 1. What if you only want to be within 1 cm of the true mean height with probability at least 0.9?

18. Let Z1,Z2,... be i.i.d. random variables, and for any set B ⊂ IR, put Xi:= IB(Zi).

(a) FindE[Xi] and var(Xi).

(b) Show that the Xiare uncorrelated.

Observe that Mn = 1 n n

∑

∑

counts the fraction of times Zilies in B. By the weak law of large numbers, for large

n this fraction should be close toP(Zi∈ B).

19. With regard to the preceding problem, put p := P(Zi∈ B). If p is very small, and n

is not large enough, it is likely that Mn= 0, which is useless as an estimate of p. If

p= 1/1000, and n = 100, find P(M100= 0).

20. Let Xibe a sequence of random variables, and put Mn:= (1/n)∑ni=1Xi. Assume that

each Xihas mean m. Show that it is not always true that for everyε> 0,

lim

n→∞P(|Mn− m| ≥ε) = 0.

Hint: Let Z be a nonconstant random variable and take Xi:= Z for i = 1,2,.... To be

specific, try Z∼ Bernoulli(1/2) andε= 1/4.

_{21. Let X}

1,X2,... be uncorrelated random variables with common mean m and common

variance σ2. Letεn be a sequence of positive numbers withεn→ 0. With Mn:=

(1/n)∑n

i=1Xi, give sufficient conditions onεnsuch that

3.4: Conditional probability

22. If Z = X +Y as in the Poisson channel Example 3.18, find E[X|Z = j].

23. Let X and Y be integer-valued random variables. Suppose that conditioned on X = i,

Y∼ binomial(n, pi), where 0 < pi< 1. Evaluate P(Y < 2|X = i).

24. Let X and Y be integer-valued random variables. Suppose that conditioned on Y = j,

X∼ Poisson(λj). Evaluate P(X > 2|Y = j).

25. Let X and Y be independent random variables. Show that pX|Y(xi|yj) = pX(xi) and

pY|X(yj|xi) = pY(yj).

26. Let X and Y be independent with X ∼ geometric0(p) and Y ∼ geometric0(q). Put T := X −Y, and find P(T = n) for all n.

27. When a binary optical communication system transmits a 1, the receiver output is a Poisson(µ) random variable. When a 2 is transmitted, the receiver output is a Poisson(ν) random variable. Given that the receiver output is equal to 2, find the conditional probability that a 1 was sent. Assume messages are equally likely. 28. In a binary communication system, when a 0 is sent, the receiver outputs a ran-

dom variable Y that is geometric0(p). When a 1 is sent, the receiver output Y ∼

geometric0(q), where q = p. Given that the receiver outputs Y = k, find the condi-

tional probability that the message sent was a 1. Assume messages are equally likely. 29. Apple crates are supposed to contain only red apples, but occasionally a few green apples are found. Assume that the number of red apples and the number of green ap- ples are independent Poisson random variables with parametersρandγ, respectively. Given that a crate contains a total of k apples, find the conditional probability that none of the apples is green.

30. Let X ∼ Poisson(λ), and suppose that given X = n, Y ∼ Bernoulli(1/(n + 1)). Find P(X = n|Y = 1).

31. Let X ∼ Poisson(λ), and suppose that given X = n, Y ∼ binomial(n, p). Find P(X =

n|Y = k) for n ≥ k.

32. Let X and Y be independent binomial(n, p) random variables. Find the conditional probability of X> k given that max(X,Y) > k if n = 100, p = 0.01, and k = 1. An-

swer: 0.576.

33. Let X ∼ geometric0(p) and Y ∼ geometric0(q), and assume X and Y are independent.

(a) FindP(XY = 4).

(b) Put Z := X +Y and find pZ( j) for all j using the discrete convolution formula

(3.17). Treat the cases p= q and p = q separately.

34. Let X and Y be independent random variables, each taking the values 0,1,2,3 with equal probability. Put Z := X +Y and find pZ( j) for all j. Hint: Use the discrete

35. Let X ∼ Bernoulli(p), and suppose that given X = i, Y is conditionally Poisson(λi),

whereλ1>λ0. Express the likelihood-ratio test P(Y = j|X = 1) P(Y = j|X = 0) ≥

P(X = 0) P(X = 1) in as simple a form as possible.

36. Let X ∼ Bernoulli(p), and suppose that given X = i, Y is conditionally geometric0(qi),

where q1< q0. Express the likelihood-ratio test

P(Y = j|X = 1) P(Y = j|X = 0) ≥

P(X = 0) P(X = 1) in as simple a form as possible.

_{37. Show that if P(X = x}

i|Y = yj) = h(xi) for all j and some function h, then X and Y are

independent.

3.5: Conditional expectation

38. Let X and Y be jointly discrete, integer-valued random variables with joint pmf

pXY(i, j) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ 3j−1e−3 j! , i = 1, j ≥ 0, 46 j−1_e−6 j! , i = 2, j ≥ 0, 0, otherwise. ComputeE[Y|X = i], E[Y], and E[X|Y = j].

39. Let X and Y be as in Example 3.15. Find E[Y], E[XY], E[Y2_{], and var(Y).}

40. Let X and Y be as in Example 3.16. Find E[Y|X = 1], E[Y|X = 0], E[Y], E[Y2_{], and}

var(Y).

41. Let X ∼ Bernoulli(2/3), and suppose that given X = i, Y ∼ Poisson3(i + 1). Find E[(X + 1)Y2_].

42. Let X ∼ Poisson(λ), and suppose that given X = n, Y ∼ Bernoulli(1/(n + 1)). Find E[XY].

43. Let X ∼ geometric1(p), and suppose that given X = n, Y ∼ Pascal(n,q). Find E[XY].

44. Let X and Y be integer-valued random variables, with Y being positive. Suppose that given Y = k, X is conditionally Poisson with parameter k. If Y has mean m and variance r, findE[X2].

45. Let X and Y be independent random variables, with X ∼ binomial(n, p), and let Y ∼ binomial(m, p). Put V := X +Y. Find the pmf of V. Find P(V = 10|X = 4) (assume

n≥ 4 and m ≥ 6).

Exam preparation

You may use the following suggestions to prepare a study sheet, including formulas men- tioned that you have trouble remembering. You may also want to ask your instructor for additional suggestions.

3.1. Probability generating functions. Important formulas include the definition (3.1), the factorization property for pgfs of sums of independent random variables (3.2), and the probability formula (3.4). The factorial moment formula (3.5) is most useful in its special cases

G_X(z)|z=1 = E[X] and GX(z)|z=1 = E[X2] − E[X].

The pgfs of common discrete random variables can be found inside the front cover.

3.2. The binomial random variable. The binomial(n, p) random variable arises as the

sum of n i.i.d. Bernoulli(p) random variables. The binomial(n, p) pmf, mean, vari- ance, and pgf can be found inside the front cover. It is sometimes convenient to remember how to generate and use Pascal’s triangle for computing the binomial co- efficientn_k= n!/[k!(n − k)!].

3.3. The weak law of large numbers. Understand what it means ifP(|Mn− m| ≥ε) is

small.

3.4. Conditional probability.I often tell my students that the three most important things in probability are:

(i) the law of total probability (3.12); (ii) the substitution law (3.13); and

(iii) independence for “dropping the conditioning” as in (3.14).

3.5. Conditional expectation.I again tell my students that the three most important things in probability are:

(i) the law of total probability (for expectations) (3.23); (ii) the substitution law (3.22); and

(iii) independence for “dropping the conditioning.”

If the conditional pmf of Y given X is listed in the table inside the front cover (this table includes moments), thenE[Y|X = i] or E[Y2_{|X = i] can often be found by in-}

spection. This is a very useful skill.

Work any review problems assigned by your instructor. If you finish them, re-work your homework assignments.