Conditional Probability

Some Probabilistic Concepts and Results

2.2 Conditional Probability

A s s T s H s

B s T s H s

{

∈

}

{

∈^S^S

}

; contains more ’ than

; any in precedes every in

’ , . Compute the probabilities P(A), P(B).

2.1.6 Suppose that the events A_j, j= 1, 2, . . . are such that P A_j

( )

^∞

∑

∞ 1

< .

Use Deﬁnition 1 in Chapter 1 and Theorem 2 in this chapter in order to show that P(A¯ )= 0.

2.1.7 Consider the events Aj, j= 1, 2, . . . and use Deﬁnition 1 in Chapter 1 and Theorem 2 herein in order to show that

P A P A P A P A

n n

( )

^≤^{lim inf}_→∞

( )

^≤^lim_→∞^sup

( )

^≤

( )

2.2 Conditional Probability

In this section, we shall introduce the concepts of conditional probability and stochastic independence. Before the formal deﬁnition of conditional probabil-ity is given, we shall attempt to provide some intuitive motivation for it. To this end, consider a balanced die and suppose that the sides bearing the numbers 1, 4 and 6 are painted red, whereas the remaining three sides are painted black.

The die is rolled once and we are asked for the probability that the upward side is the one bearing the number 6. Assuming the uniform probability function, the answer is, clearly, ¹

6. Next, suppose that the die is rolled once as before and all that we can observe is the color of the upward side but not the number on it (for example, we may be observing the die from a considerable distance, so that the color is visible but not the numbers on the sides). The same question as above is asked, namely, what is the probability that the number on the uppermost side is 6. Again, by assuming the uniform probabil-ity function, the answer now is ¹

3. This latter probability is called the condi-tional probability of the number 6 turning up, given the information that the uppermost side was painted red. Letting B stand for the event that number 6 appears and A for the event that the uppermost side is red, the

above-2.2 Conditional Probability 21

mentioned conditional probability is denoted by P(B|A), and we observe that this is equal to the quotient P(A∩ B)/P(A). Or suppose that, for the purposes of a certain study, we observe two-children families in a certain locality, and record the gender of the children. A sample space for this experiment is the following:S = {bb, bg, gb, gg}, where b stands for boy and g for girl, and bg, for example, indicates that the boy is older than the girl. Suppose further (although this is not exactly correct) that: P({bb})= P({bg}) = P({gb}) = P({gg})

= ¹₄, and deﬁne the events A and B as follows: A= “children of one gender” = {bb, gg}, B = “at least one boy” = {bb, bg, gb}. Then P(A|B) = P(A ∩ B)/

P(B)= ¹₃.

From these and other examples, one is led to the following deﬁnition of conditional probability.

Let A be an event such that P(A)> 0. Then the conditional probability, given A, is the (set) function denoted by P(·|A) and deﬁned for every event B as follows:

P B A P A B

( )

⁼

⁽

P A

_{( )}

^∩

⁾

P(B|A) is called the conditional probability of B, given A.

The set function P(·|A) is actually a probability function. To see this, it sufﬁces to prove the P(·|A) satisﬁes (P1)–(P3). We have: P(B|A)ⱖ 0 for every event B, clearly. Next,

P S A P S A

The conditional probability can be used in expressing the probability of the intersection of a ﬁnite number of events.

(Multiplicative Theorem) Let Aj, j= 1, 2, . . . , n, be events such that

2.4 Combinatorial Results 23 (The proof of this theorem is left as an exercise; see Exercise 2.2.4.)

REMARK 3 The value of the above formula lies in the fact that, in general, it is easier to calculate the conditional probabilities on the right-hand side. This point is illustrated by the following simple example.

An urn contains 10 identical balls (except for color) of which ﬁve are black, three are red and two are white. Four balls are drawn without replacement.

Find the probability that the ﬁrst ball is black, the second red, the third white and the fourth black.

Let A1 be the event that the ﬁrst ball is black, A2 be the event that the second ball is red, A3 be the event that the third ball is white and A4 be the event that the fourth ball is black. Then

P A A A A

and by using the uniform probability function, we have

P A P A A P A A A Thus the required probability is equal to ¹

42 0.0238.

Now let Aj, j= 1, 2, . . . , be events such that Ai∩ Aj= ∅, i ≠ j, and ΣjAj= S. Such a collection of events is called a partition of S. The partition is finite or (denumerably) infinite, accordingly, as the events Aj are finitely or (denumerably) infinitely many. For any event, we clearly have:

B B A_j

provided P(A_j)> 0, for all j. Thus we have the following theorem.

(Total Probability Theorem) Let {A_j, j = 1, 2, . . . } be a partition of S with P(A_j) > 0, all j. Then for B∈ A, we have P(B) = ΣjP(B|A_j)P(A_j).

This formula gives a way of evaluating P(B) in terms of P(B|A_j) and P(A_j), j = 1, 2, . . . . Under the condition that P(B) > 0, the above formula

2.2 Conditional Probability 23

EXAMPLE 1

THEOREM 4

can be “reversed” to provide an expression for P(A_j|B), j = 1, 2, . . . . In

REMARK 4 It is important that one checks to be sure that the collection {A_j, j≥ 1} forms a partition of S, as only then are the above theorems true.

The following simple example serves as an illustration of Theorems 4 and 5.

A multiple choice test question lists ﬁve alternative answers, of which only one is correct. If a student has done the homework, then he/she is certain to identify the correct answer; otherwise he/she chooses an answer at random.

Let p denote the probability of the event A that the student does the home-work and let B be the event that he/she answers the question correctly. Find the expression of the conditional probability P(A|B) in terms of p.

By noting that A and A^c form a partition of the appropriate sample space, an application of Theorems 4 and 5 gives

P A B P B A P A

For example, for p = 0.7, 0.5, 0.3, we ﬁnd, respectively, that P(A|B) is approximately equal to: 0.92, 0.83 and 0.68.

Of course, there is no reason to restrict ourselves to one partition of S only. We may consider, for example, two partitions {A_i, i= 1, 2, . . .} {Bj, j= 1,

2.4 Combinatorial Results 25

is a partition of S. In fact,

A_i∩B_j A_i B_j i j i j

( )

^∩

(

^′^∩ ^′

)

⁼^{∅ if}

( )

^, ^{≠ ′ ′}

( )

and

A_i B_j A B A S

i j

i j

j i

i i

(

∩

)

⁼

(

^∩

)

⁼ ⁼

∑ ∑ ∑ ∑

The expression P(Ai∩ Bj) is called the joint probability of Ai and Bj. On the other hand, from

Ai Ai Bj B A B

j i j

∑ (

∩

)

^and ⁼

∑ (

^∩

)

we get

P A_i P A_i B_j P A B P B

i j j

( )

⁼

^∑ (

^∩

)

⁼

^∑ ( ) ^{( )}

j^,

provided P(Bj)> 0, j = 1, 2, . . . , and

P B_j P A_i B_j P B A P A

j i i

( )

⁼

^∑ (

^∩

)

⁼

^∑ ( ) ^{( )}

i^,

provided P(A_i) > 0, i = 1, 2, . . . . The probabilities P(Ai), P(B_j) are called marginal probabilities. We have analogous expressions for the case of more than two partitions of S.

Exercises

2.2.1 If P(A|B)> P(A), then show that P(B|A) > P(B) (P(A)P(B) > 0).

2.2.2 Show that:

i) P(A^c|B)= 1 − P(A|B);

ii) P(A∪ B|C) = P(A|C) + P(B|C) − P(A ∩ B|C).

Also show, by means of counterexamples, that the following equations need not be true:

iii) P(A|B^c)= 1 − P(A|B);

iv) P(C|A+ B) = P(C|A) + P(C|B).

2.2.3 If A∩ B = ∅ and P(A + B) > 0, express the probabilities P(A|A + B) and P(B|A+ B) in terms of P(A) and P(B).

2.2.4 Use induction to prove Theorem 3.

2.2.5 Suppose that a multiple choice test lists n alternative answers of which only one is correct. Let p, A and B be deﬁned as in Example 2 and ﬁnd P_n(A|B)

Exercises 25

in terms of n and p. Next show that if p is ﬁxed but different from 0 and 1, then P_n(A|B) increases as n increases. Does this result seem reasonable?

2.2.6 If A_j, j= 1, 2, 3 are any events in S, show that {A1, A^c₁∩ A2, A^c₁∩ A^c2∩ A₃, (A₁∪ A2∪ A3)^c} is a partition of S.

2.2.7 Let {A_j, j= 1, . . . , 5} be a partition of S and suppose that P(Aj)= j/15 and P(A|A_j) = (5 − j)/15, j = 1, . . . , 5. Compute the probabilities P(Aj|A), j= 1, . . . , 5.

2.2.8 A girl’s club has on its membership rolls the names of 50 girls with the following descriptions:

20 blondes, 15 with blue eyes and 5 with brown eyes;

25 brunettes, 5 with blue eyes and 20 with brown eyes;

5 redheads, 1 with blue eyes and 4 with green eyes.

If one arranges a blind date with a club member, what is the probability that:

i) The girl is blonde?

ii) The girl is blonde, if it was only revealed that she has blue eyes?

2.2.9 Suppose that the probability that both of a pair of twins are boys is 0.30 and that the probability that they are both girls is 0.26. Given that the probabil-ity of a child being a boy is 0.52, what is the probabilprobabil-ity that:

i) The second twin is a boy, given that the ﬁrst is a boy?

ii) The second twin is a girl, given that the ﬁrst is a girl?

2.2.10 Three machines I, II and III manufacture 30%, 30% and 40%, tively, of the total output of certain items. Of them, 4%, 3% and 2%, respec-tively, are defective. One item is drawn at random, tested and found to be defective. What is the probability that the item was manufactured by each one of the machines I, II and III?

2.2.11 A shipment of 20 TV tubes contains 16 good tubes and 4 defective tubes. Three tubes are chosen at random and tested successively. What is the probability that:

i) The third tube is good, if the ﬁrst two were found to be good?

ii) The third tube is defective, if one of the other two was found to be good and the other one was found to be defective?

2.2.12 Suppose that a test for diagnosing a certain heart disease is 95%

accurate when applied to both those who have the disease and those who do not. If it is known that 5 of 1,000 in a certain population have the disease in question, compute the probability that a patient actually has the disease if the test indicates that he does. (Interpret the answer by intuitive reasoning.) 2.2.13 Consider two urns U_j, j= 1, 2, such that urn Uj contains m_j white balls and n_j black balls. A ball is drawn at random from each one of the two urns and

2.4 Combinatorial Results 27

is placed into a third urn. Then a ball is drawn at random from the third urn.

Compute the probability that the ball is black.

2.2.14 Consider the urns of Exercise 2.2.13. A balanced die is rolled and if an even number appears, a ball, chosen at random from U₁, is transferred to urn U₂. If an odd number appears, a ball, chosen at random from urn U₂, is transferred to urn U₁. What is the probability that, after the above experiment is performed twice, the number of white balls in the urn U₂ remains the same?

2.2.15 Consider three urns U_j, j= 1, 2, 3 such that urn Uj contains m_jwhite balls and n_j black balls. A ball, chosen at random, is transferred from urn U₁ to urn U₂(color unnoticed), and then a ball, chosen at random, is transferred from urn U₂ to urn U₃ (color unnoticed). Finally, a ball is drawn at random from urn U₃. What is the probability that the ball is white?

2.2.16 Consider the urns of Exercise 2.2.15. One urn is chosen at random and one ball is drawn from it also at random. If the ball drawn was white, what is the probability that the urn chosen was urn U₁ or U₂?

2.2.17 Consider six urns U_j,j= 1, . . . , 6 such that urn Uj contains m_j (≥ 2) white balls and n_j (≥ 2) black balls. A balanced die is tossed once and if the number j appears on the die, two balls are selected at random from urn U_j. Compute the probability that one ball is white and one ball is black.

2.2.18 Consider k urns U_j, j= 1, . . . , k each of which contain m white balls and n black balls. A ball is drawn at random from urn U₁ and is placed in urn U₂. Then a ball is drawn at random from urn U₂ and is placed in urn U₃ etc.

Finally, a ball is chosen at random from urn U_k−1 and is placed in urn U_k. A ball is then drawn at random from urn U_k. Compute the probability that this last ball is black.

2.3 Independence

For any events A, B with P(A)> 0, we deﬁned P(B|A) = P(A ∩ B)/P(A). Now P(B|A) may be >P(B), <P(B), or = P(B). As an illustration, consider an urn containing 10 balls, seven of which are red, the remaining three being black.

Except for color, the balls are identical. Suppose that two balls are drawn successively and without replacement. Then (assuming throughout the uni-form probability function) the conditional probability that the second ball is red, given that the ﬁrst ball was red, is ⁶

9, whereas the conditional probability that the second ball is red, given that the ﬁrst was black, is ⁷

9. Without any knowledge regarding the ﬁrst ball, the probability that the second ball is red is

10. On the other hand, if the balls are drawn with replacement, the probability that the second ball is red, given that the ﬁrst ball was red, is ⁷

10. This probabil-ity is the same even if the ﬁrst ball was black. In other words, knowledge of the event which occurred in the ﬁrst drawing provides no additional information in

2.3 Independence 27

calculating the probability of the event that the second ball is red. Events like these are said to be independent.

As another example, revisit the two-children families example considered earlier, and deﬁne the events A and B as follows: A = “children of both genders,” B= “older child is a boy.” Then P(A) = P(B) = P(B|A) = ¹₂. Again knowledge of the event A provides no additional information in calculating the probability of the event B. Thus A and B are independent.

More generally, let A, B be events with P(A)> 0. Then if P(B|A) = P(B), we say that the even B is (statistically or stochastically or in the probability sense) independent of the event A. If P(B) is also > 0, then it is easily seen that A is also independent of B. In fact,

P A B P A B P B

P B A P A P B

P B P A P B P A

( )

⁼

⁽ _{( )}

^∩

⁾

⁼

( ) ^{( )}

( )

⁼

( ) ( )

( )

⁼

( )

That is, if P(A), P(B)> 0, and one of the events is independent of the other, then this second event is also independent of the ﬁrst. Thus, independence is a symmetric relation, and we may simply say that A and B are independent. In this case P(A ∩ B) = P(A)P(B) and we may take this relationship as the deﬁnition of independence of A and B. That is,

The events A, B are said to be (statistically or stochastically or in the probabil-ity sense) independent if P(A∩ B) = P(A)P(B).

Notice that this relationship is true even if one or both of P(A), P(B)= 0.

As was pointed out in connection with the examples discussed above, independence of two events simply means that knowledge of the occurrence of one of them helps in no way in re-evaluating the probability that the other event happens. This is true for any two independent events A and B, as follows from the equation P(A|B) = P(A), provided P(B) > 0, or P(B|A) = P(B), provided P(A)> 0. Events which are intuitively independent arise, for exam-ple, in connection with the descriptive experiments of successively drawing balls with replacement from the same urn with always the same content, or drawing cards with replacement from the same deck of playing cards, or repeatedly tossing the same or different coins, etc.

What actually happens in practice is to consider events which are inde-pendent in the intuitive sense, and then deﬁne the probability function P appropriately to reﬂect this independence.

The deﬁnition of independence generalizes to any ﬁnite number of events.

Thus:

The events A_j, j= 1, 2, . . . , n are said to be (mutually or completely) indepen-dent if the following relationships hold:

P A_j A_j P A_j P A_j

k k

1∩ ⋅ ⋅ ⋅ ∩ 1

( )

⁼

( )

^{⋅ ⋅ ⋅}

( )

for any k= 2, . . . , n and j1, . . . , j_k= 1, 2, . . . , n such that 1 ≤ j1< j2< · · · < jk≤ n. These events are said to be pairwise independent if P(A_i∩ Aj)= P(Ai)P(A_j) for all i≠ j.

DEFINITION 3

DEFINITION 4

2.4 Combinatorial Results 29

It follows that, if the events Aj, j= 1, 2, . . . , n are mutually independent, then they are pairwise independent. The converse need not be true, as the example below illustrates. Also there are

n n n all necessary. For example, for n= 3 we will have:

P A A A P A P A P A

That these four relations are necessary for the characterization of indepen-dence of A1, A2, A3 is illustrated by the following examples:

THEOREM 6

The following result, regarding independence of events, is often used by many authors without any reference to it. It is the theorem below.

If the events A₁, . . . , A_n are independent, so are the events A′1, . . . , A′n, where A′j is either A_j or A^c_j, j= 1, . . . , n.

PROOF The proof is done by (a double) induction. For n = 2, we have to show that P(A′1 ∩ A′2)= P(A′1)P(A′2). Indeed, let A′1 = A1 and A′2= A^c2. Then

Next, assume the assertion to be true for k events and show it to be true for k+ 1 events. That is, we suppose that P(A′1∩ · · · ∩ A′k)= P(A′1 ) · · · P(A′k),

This relationship is established also by induction as follows: If A′1= A^c1 and A′i= Ai, i= 2, . . . , k, then

2.4 Combinatorial Results 31 by the induction hypothesis

P of k. Thus, we have shown that

P A

(

¹′ ∩ ⋅ ⋅ ⋅ ∩ ′ ∩A_k A_k⁺¹

)

⁼P A

( )

¹^′ ^{⋅ ⋅ ⋅}P A P A

( ) ( )

_k^′ _k⁺¹^.

Finally, under the assumption that

P A

(

¹′ ∩ ⋅ ⋅ ⋅ ∩ ′A_k

)

⁼P A

( )

¹^′ ^{⋅ ⋅ ⋅}P A

( )

_k^′ ^,

take A′k+1= A^ck+1, and show that

P A

(

¹′ ∩ ⋅ ⋅ ⋅ ∩ ′ ∩A_k A_k^c⁺¹

)

⁼P A

( )

¹^′ ^{⋅ ⋅ ⋅}P A P A

( )

_k^′

( )

_k^c⁺¹^.

2.3 Independence 31

In fact,

by the induction hypothesis and what was last proved

This completes the proof of the theorem. ▲

Now, for j= 1, 2, let Ej be an experiment having the sample space Sj. One may look at the pair (E1,E2) of experiments, and then the question arises as to what is the appropriate sample space for this composite or compound experi-ment, also denoted by E1× E2. If S stands for this sample space, then, clearly, S = S1× S2= {(s1, s₂); s₁∈ S1, s₂∈ S2}. The corresponding events are, of course, subsets of S. The notion of independence also carries over to experiments.

Thus, we say that the experiments E1 and E2 are independent if P(B₁∩ B2)= P(B₁)P(B₂) for all events B₁ associated with E1 alone, and all events B₂ associ-atedE2 alone.

What actually happens in practice is to start out with two experiments E1, E2 which are intuitively independent, such as the descriptive experiments (also mentioned above) of successively drawing balls with replacement from the same urn with always the same content, or drawing cards with replacement from the same deck of playing cards, or repeatedly tossing the same or differ-ent coins etc., and have the corresponding probability spaces (S1, class of events, P₁) and (S2, class of events, P₂), and then deﬁne the probability func-tion P, in terms of P₁ and P₂, on the class of events in the space S1× S2 so that it reﬂects the intuitive independence.

The above deﬁnitions generalize in a straightforward manner to any ﬁnite number of experiments. Thus, if Ej, j= 1, 2, . . . , n, are n experiments with corresponding sample spaces Sj and probability functions P_j on the respective classes of events, then the compound experiment

E E

(

₁, , ₂,⋅ ⋅ ⋅ Eⁿ

)

⁼E E₁^× ₂^{× ⋅ ⋅ ⋅ ×}Eⁿ has sample space S, where

S S= ¹× ⋅ ⋅ ⋅ ×Sⁿ =

{ (

^s¹^,⋅ ⋅ ⋅^,^sⁿ

)

^;^s^j^∈S^j^,^j⁼¹^, ^2,⋅ ⋅ ⋅^,ⁿ

}

The class of events are subsets of S, and the experiments are said to be independent if for all events B_j associated with experiment Ej alone, j = 1, 2, . . . , n, it holds

2.4 Combinatorial ResultsExercises 3333

P B

(

₁∩ ⋅ ⋅ ⋅ ∩B_n

)

⁼P B

( )

₁ ^{⋅ ⋅ ⋅}P B

( )

_n ^.

Again, the probability function P is deﬁned, in terms of P_j, j= 1, 2, . . . , n, on the class of events in S so that to reﬂect the intuitive independence of the experimentsEj, j= 1, 2, . . . , n.

In closing this section, we mention that events and experiments which are not independent are said to be dependent.

Exercises

2.3.1 If A and B are disjoint events, then show that A and B are independent if and only if at least one of P(A), P(B) is zero.

2.3.2 Show that if the event A is independent of itself, then P(A)= 0 or 1.

2.3.3 If A, B are independent, A, C are independent and B∩ C = ∅, then A, B+ C are independent. Show, by means of a counterexample, that the conclu-sion need not be true if B∩ C ≠ ∅.

2.3.4 For each j = 1, . . . , n, suppose that the events A1, . . . , Am, Bj are independent and that Bi∩ Bj= ∅, i ≠ j. Then show that the events A1, . . . , Am, Σⁿj=1Bj are independent.

2.3.5 If Aj, j= 1, . . . , n are independent events, show that

P A_j P A

j n

j c

j n

= =

⎛

⎝⎜

⎞

⎠⎟= −

^∏ ( )

1 1

U

1 ^.

2.3.6 Jim takes the written and road driver’s license tests repeatedly until he passes them. Given that the probability that he passes the written test is 0.9 and the road test is 0.6 and that tests are independent of each other, what is the probability that he will pass both tests on his nth attempt? (Assume that the road test cannot be taken unless he passes the written test, and that once he passes the written test he does not have to take it again, no matter whether he passes or fails his next road test. Also, the written and the road tests are considered distinct attempts.)

2.3.7 The probability that a missile ﬁred against a target is not intercepted by an antimissile missile is ²

3. Given that the missile has not been intercepted, the probability of a successful hit is ³

4. If four missiles are ﬁred independently, what is the probability that:

i) All will successfully hit the target?

ii) At least one will do so?

How many missiles should be ﬁred, so that:

In document A Course in Mathematical Statistics 0125993153 (Page 42-55)

Some Probabilistic Concepts and Results

2.2 Conditional Probability

{

}

{

}

( )

∑

( )

( )

( )

( )

2.2 Conditional Probability

( )

(

( )

)

( )

(

)

( )

( )

(

)

(

)

∑ ∑ ∑ ∑

∑ (

)

∑ (

)

( )

∑ (

)

∑ ( ) ( )

( )

∑ (

)

∑ ( ) ( )

Exercises

2.3 Independence

( )

( ( )

)

( ) ( )

( )

( ) ( )

( )

( )

( )

( )

( )

(

)

( )

( ) ( )

(

)

( )

( )

(

)

( )

( )

( )

(

)

{ (

)

}

(

)

( )

( )

Exercises

∏ ( )

U

⁽

_{( )}

⁾

^∑ (

^∑ ( ) ^{( )}

^∑ (

^∑ ( ) ^{( )}

⁽ _{( )}

⁾

( ) ^{( )}

^∏ ( )