ENGI 4421 Probability and Statistics Faculty of Engineering and Applied Science Problem Set 4 Solutions Conditional probability, Bayes' theorem

ENGI 4421 Probability and Statistics Faculty of Engineering and Applied Science

Problem Set 4 Solutions

Conditional probability, Bayes' theorem

1. One urn contains five red balls and five green balls. A second urn contains eight red balls and one green ball. A ball is chosen randomly from the first urn and placed in the second urn. Then a ball is chosen randomly from the second urn and placed in the first urn.

(a) What is the probability that a red ball is selected from the first urn and a red ball is selected from the second urn?

Let R₁ = (red ball selected from the first urn),

2

R = (red ball selected from the second urn),

1

G = (green ball selected from the first urn), (= R ) and ₁

2

G = (green ball selected from the second urn), (= R ) , then 2

1 1 1 2 1 5 5 9 P , P 1 P , P | 10 10 10 R G R R R         

        (because after event R₁ has

occurred there are now nine red and one green balls in the second urn).

2 1 8 P | 10 R G   

  , (because after event G₁ has occurred there are now eight red and two green balls in the second urn).

1 2 2 1 1 9 5 P P | P 10 10 R R R R R                 1 2 P R R_  _ 9 .45 20

(b) What is the [unconditional] probability that a red ball is selected from the second urn?

2 1 2 1 2

P _{ }R  P_R R _P_R R _ (total probability law)

2 1 1 2 1 1

PR |R  P R PR |R  P R

 _{ } _{ } _{ } _{ } (general multiplication law)

9 8 5 9 8 20 10 10 20 20       17 = .85 20

1 (b) (continued)

Note that P R _{ 2} is different from 8/9 = .888... (the probability of drawing a red ball from the second urn if no ball were added from the first urn).

(c) At the conclusion of the selection process, what is the probability that the numbers of red and green balls in each urn are identical to the initial numbers?

The desired event E = (numbers before and after are identical) can be expressed as

1 2 1 2 E  R R G G 1 2 2 1 1 9 5 9 P P | P 10 10 20 R R R R R          

      [which is part (a) above]

1 2 2 1 1 2 5 2 P P | P 10 10 20 G G G G G                 So P

 

9 2 20 20 E    P E

 

11 .55 20

A tree diagram may assist in the solution to this question:

Now consider the generalization to the case where there are m balls initially in urn 1, a of which are red balls, and n balls initially in urn 2, b of which are red balls, together with m > 1, n > 1, 0 < a < m and 0 < b < n (so that each urn initially contains at least one ball of each colour).

1 (d) Show that the [unconditional] probability that a red ball is selected from the second urn during this process is, in general, different from the probability of drawing a red ball directly from the second urn [when the first urn is left alone].

Let A = red ball drawn from urn 1 and B = red ball drawn from urn 2, then

 





P , P 1 P | , P | 1 1 a m a A A m m b b B A B A n n     _{ }  _{ }  _{ }  

 

1 P , P 1 1 a b m a b AB AB m n m n  _{ }     _{ } 

  [multiplication law of probability]

 

 









P P P 1 1 ab a mb ab a mb B AB AB m n m n          _{ }  

  [total probability law]

Let C = red ball drawn from the original contents of urn 2 (that is, when no balls have been added from urn 1), then

 

 

P C b P B

n

  in general.

(e) Find a necessary and sufficient condition on the values of a and b for the two unconditional probabilities in part (d) above to be equal.

   













P P 1 1 a mb b B C a mb n bm n m n n          an bmn bm bmn an bm

      and none of a, b, m, n can be zero 

b a n  m

that is, the initial proportions of balls in each urn that are red is the same. (Conversely, it is easy to show that b a P

   

B P C

2. A test for unacceptable metal fatigue in a girder is not perfect. The test is positive, (that is, the test suggests that unacceptable metal fatigue is present), 95% of the time when unacceptable metal fatigue really is present. The test is positive 10% of the time when there really isn’t any unacceptable metal fatigue. It is known that 5% of all girders at a particular site have developed unacceptable metal fatigue. If the next test result is positive, then what are the odds that that girder really does have unacceptable metal fatigue?

Let A = “The girder has unacceptable metal fatigue”

and B = “The test is positive (suggesting that unacceptable metal fatigue is present)” The following probabilities are provided in the question:

P[A] = .05 P[B | A] = .95 P[B | ~A] = .10 We are required to find P[A | B].

Using Bayes’ theorem directly,







  

P



|

  

P P | P | P P | P .95 B A A A B B A A B A A         _{ } _{ }  .05 .95  .05 .10 .95    5 1 5 10 3   

2 (continued)

Therefore P[A | B] = 1/3 , r_{A B|} 1/ 2 and the odds of A | B are 2 :1 against

Another way to think of this problem is as follows.

Suppose that there are exactly 10,000 girders in the population.

Then 500 girders (5% of the 10,000) are defective. Of these 500 girders,

95% (475) will test positive (correctly) and there will be only 25 false negatives. 10% of the good girders will test positive (incorrectly).

That is, 950 of the 9,500 good girders will be false positives.

The ratio of true positives to false positives is then 475:950, or 1:2 on = 2:1 against. The corresponding odds of a positive test result being correct are then 2:1 against. There is also an Excel file available for this question.

3. A factory has the following information about the quality control process on its production line:

If an item is defective, then there is a 99.9% chance that the quality control process will reject it [which, with the benefit of hindsight, is the correct decision]. If an item is good, then there is a 2% chance that the quality control process will reject it [a “false positive”, which, with the benefit of hindsight, is not the correct decision]. It is known that 0.1% of all items on the production line are defective.

Given that the quality control process has just rejected an item, find the probability that that decision is correct [i.e. that the item really is defective].

Let D = item is truly defective and R = item is rejected, then P[R | D] = .999 P[~R | D] = .001

P[R | ~D] = .02 P[~R | ~D] = .98 P[D] = .001 P[~D] = .999 We are required to find P[D | R]. By Bayes’ theorem,





P



 





P







P | P P P D R R D D R R R D R D       



  



  

P | P P | P P | P R D D R D D R D D           .999 .001 1 .999 .001 .02 .999 1 20         P



D R|



  1 .0476 21 or

A tree diagram can be used to solve this question:

and





.000999 P | .020979 D R   1 21

The Excel spreadsheet

BayeTree.xlsx can be used

to confirm the calculation of these probabilities.

Question 3 Additional Note:

At first sight, it may seem strange that there is less than a 5% chance that a randomly chosen rejected item is actually defective. This is largely due to the fact that so few items in the population are defective, (only 0.1%).

Suppose that the population consists of exactly 1,000,000 items. Then 999,000 items are good and only 1,000 items are defective.

The proportion of good items that will be rejected [incorrectly] is only 2%. However, 2% of 999,000 is still a large number, 19,980.

19,980 items are good and will be rejected by the quality control process.

Of the 1,000 defective items, 999 will be rejected and only 1 will slip through as a false negative. Therefore, of the 1,000,000 items, 20,979 will be rejected, of which 19,980 are good and only 999 are defective.

An item chosen randomly from the rejected group is therefore much less likely to be defective (true positive) than to be good (false positive), (with odds of 999:19980 on = 20:1 against).

4. Alice, Bob and Carol take part in a game where each of them, in turn, rolls a fair die until one of them rolls a six and wins the game. Alice rolls first. If she doesn’t roll a six, then Bob rolls next. If he doesn’t roll a six, then Carol has her first chance to win. If none of them roll a six on the first attempt, then Alice gets a second roll and so on.

(a) Find the probability that Carol wins the game in the first round.

Let A = Alice wins the game, B = Bob wins the game , C = Carol wins the game, Ai = Alice rolls a six in round i, Bi = Bob rolls a six in round i and

Ci = Carol rolls a six in round i , then

Carol cannot win at all unless Alice and Bob both fail to win in round 1.





1 1 1 1 1 1 1 1 1

P Carol wins in round 1  P_A B C _  P_{ } A P_B |A _P_C |B A _ But the rolls are all independent

1 1 1 1 1 1 5 5 1 25 P P P P .1157 6 6 6 216 A B C A B C          _{ }  _{ } _{ } _{ }      Therefore





P Carol wins in round 1  25 216

4 (b) Find the probability that the game lasts for more than two complete rounds.

The desired event is A B C A B C_{1 1 1 2 2 2} (no-one rolls a six on the first two attempts).

1 1 1 2 2 2 6 5 P .3349 6 A B C A B C      _{ }     _{ } 15 625_{46 656}

(c) Find the probability that Alice wins.

P[Alice wins] = P[Alice wins immediately] + P[Alice wins in round 2] + P[Alice wins in round 3] + ...

 

1 1 1 1 2 1 1 1 2 2 2 3 P A  P_{ } A P_A B C A _P_A B C A B C A _



2 3



3 6 9 3 1 5 1 5 1 5 1 1 5 1 where 6 6 6 6 6 6 6 6 r r r r 6           _{ }  _{ }  _{ }       _{ }        

This is an absolutely convergent geometric series, with

3 1 5 125 , 6 6 216 a r _{ }   

 

1 216 P .3956 1 6 216 125 a A r         36 91 OR

P[Alice wins] = P[Alice wins immediately]

+ P[no-one wins in round 1 and Alice wins eventually]

 

1 1 1 1 1 1 1 1

 

P A P A PA B C A P A PA B C  P A   _{ } _{ }  _{ } _{ }

 

1 5 3

 

125

 

1 P P 1 P 6 6 216 6 A   A   A   _{ }   _  _     

 

1 216 P 6 91 A     36 91

4 (d) Find the probability that Carol wins, given that Alice doesn’t roll a six on her first roll.

Given that Alice doesn’t roll a six on her first roll, the first turn now belongs to Bob. P[Carol wins | ~A1] = P[Carol wins in round 1] + P[Carol wins in round 2]

+ P[Carol wins in round 3] + ...

1 1 1 1 1 2 2 2 1 1 2 2 2 3 3 3 P_C A| _  P_B C _P_B C A B C _P_B C A B C A B C _



2 3



4 7 10 5 1 5 1 5 1 5 1 5 1 6 6 6 6 6 6 6 6 36 r r r          _{ }  _{ }  _{ }             3 5 where 6 r     

This is an absolutely convergent geometric series, with

3 5 5 125 , 36 6 216 a r _{ }    1 5 216 P | .3297 1 36 216 125 a C A r    _{ }        30 91

Question 4 Additional Note:

The probabilities of victory for each of Alice, Bob and Carol are all the sums of related geometric series. It is easy to show that P

 

36, P

 

30, P

 

25

91 91 91

A  B  C  .

These probabilities are in the ratio 36 : 30 : 25, with the advantage to the person rolling first.

5. A network of pumping stations is connected as shown.

The reliability of each station is independent of the others, except for stations C and D, for which P



D C|



 P



C D|



 .9 and P_D C| _  P_C D| _  .5.

Water passes through each of the other stations 90% of the time.

Find the probability that water flows through the network from point X to point Y.

Directly from the question, we have P

     

A P B P E .9

The multiplication law  P

 

CD  P

  

C P D C|



 P

  

D P C D|



 

 

 

 

9 9 P P P P 10 C 10 D C D p      Also P_CD_  P

 

C P_D C| _  P _{ }D P_C D| _







1 P |





1



P | p D C p C D      _{  }





1 5 5 1 5 5 6 5 10p 10 p p p p p 6           Therefore P P P |



1





1 P |



1 1 1 6 2 12 CD C D C p D C                       





1 11 P 1 P 1 12 12 C D C D    _{ }   

The subsystem CD is in series with stations A, B and E. Therefore the probability that the entire system works is

    

  

9 9 11 9

P P P P .668 25

10 10 12 10

A  B  C D  E      2 673  4 000

The odds are slightly better than 2:1 on that water will flow through the network.

Additional note:

For two pumping stations in parallel, suppose, more generally, that









P D C|  P C D|  a and P_D C| _  P_C D| _  b



0  b a 1



Then P

 

CD  P

  

C P D C|



 P

  

D P C D|



 P

 

C  a P

 

D a  P

 

C  P

 

D  p and P_CD_  P

 

C P_D C| _  P_{ } D P_C D| _  p



1a

 

 1p b





1



P

   

P





1 b a b p b p C D a b          

5. (continued)

















1 1 1 1 1 1 1 a b b b a p a b a b a b                





 

1





1





1 P P P | 1 1 1 1 a b a b ab CD C D C p b a b a b                        _{   }

The probability that the parallel subsystem CD works is therefore





1 1





1





P 1 P 1 1 1 a b a b ab a b ab C D C D a b a b              _{ }        









2





P 1 b a C D a b    

For this question,



.5



5 .9, .5 1 .9 .5 6 a b  p    and













.5 2 .9 .5 1.1 55 11 P 1 .9 .5 1 .4 60 12 C D          A special case is b 1 a



P_D C| _  1 P



D C|





, for which P





1 2 a C D  

An Excel file to demonstrate these results is at

6. In a criminal trial, a prosecutor states that only one in every ten million innocent men have a DNA profile that matches the known DNA profile of the guilty person. The defendent’s DNA profile matches that of the guilty person. The prosecutor goes on to state that the chance that the defendent is innocent is therefore only one in ten million, so unlikely that the jury must find the defendent guilty.

What is the major flaw in this argument?

From the information in the question, P[DNA match | innocent] = 10–7

However, the probability that the jury needs to know is P[innocent | DNA match] ,

which could be a very different number.

The two probabilities are related by Bayes’ theorem.

Let I = suspect is innocent, G = suspect is guilty = ~I and M = DNA match. Then







  

P I

 



  

P



|

  

P I



  

P | P | P P | P P | P M I I M M I M M I I M G G       

[A similar confusion of probabilities actually did lead to several miscarriages of justice in England in the 1990’s. It is now known as the “prosecutor’s fallacy”. One of the most severe examples is reported in the President’s Address (Mathematical Association), in the article “Taking perspective”, Barry Lewis, Mathematical Gazette, vol. 87 #510, pp. 418-431, (2003 November). The example begins near the foot of page 422.]

7. This is a continuation of Problem Set 2 Question 7. Use the labels shown there for events and decisions. The following exact probabilities are known:

P[D] = .05 P[N | D] = .99 P[N | G] = .10

Use a separate tree diagram (or use Bayes’ theorem) to calculate (a) P[D | N]

The tree diagram is





P



 



P | P D N D N N  .0495 .0495 .0950   495 1445   99 289

or, directly from Bayes’ theorem,







  

P



|

  

P



  

P | P | P P | P .99 .05 .99 .05 .10 .95 .0495 495 .0495 .0950 1445 N D D D N N D D N G G               99 289









P D N|  99  .3426 to 4 d.p. 289

7 (b) P[D | Y]





P

 

 

.0005 5 P | P .0005 .8550 8555 DY D Y Y      1 1711









P D Y|  1  .000584 to 3 s.f. 1711 (c) P[N]

P[N] is the denominator in the Bayes’ theorem expression in part (a) above. Therefore,

 

P N  289  .1445 2 000

(d) Place the exact values of the following probabilities, (expressed as fractions in their lowest terms), at the appropriate locations on the original decision tree:

P[D | N] P[G | N] P[D | Y] P[G | Y] P[D] P[G] P[N] P[Y] P[D | N] = 99/289 = .3425... P[G | N] = 1 – P[D | N] = 190/289 P[D | Y] = 1/1711 = .0005... P[G | Y] = 1 – P[D | Y] = 1710/1711 P[D] = 1/20 = .05 P[G] = 1 – P[D] = 19/20 = .95 P[N] = 289/2000 = .1445 P[Y] = 1 – P[N] = 1711/2000 = .8555

When the result from quality control (N or Y) is known, then that result updates our assessment of the probabilities of D and G from the prior (or “unconditional”) values to the appropriate posterior (conditional) probabilities.

Question 7(d) (continued)

Thus, on the lower four final branches of the decision tree, the prior values of P[D] and P[G] are placed. However, on the upper eight branches, the prior values are replaced by the appropriate posterior values, to generate this diagram.

The calculations for this question are also available in this Excel file.

We shall develop this example further, in a future problem set.