Conditional probability and independence

Conditional probability and independence deal with how to compute probabilities and the meaning of probability. They are directly related to the description of Bayesian probability that we alluded to in Section 4.3.1.

4.4.1 Conditional probability

The occurrence of one event might change the likelihood of another. For example you may be asked about the likelihood that a day was cloudy when you know it rained during that day. In such a case, you would say that the likelihood is 100%. If you are asked about the likelihood of rain given that it was cloudy during the day, your answer will be less than 100%.

Example 4.27. Suppose that in a population, 0.1% have a certain disease. A diag-nostic test is available, but it is correct in only 80% of the cases—; diagnosing the disease when a person is actually infected. The other 20% show false positives. Now choose a person from the population randomly and consider the following events:

E, an individual carries the disease and F , an individual’s diagnostic test result is positive. Let P (E|F ) denote the probability of E given F . Then from the data

P (E) = 0.001 , P (E|F ) = 0.8 .

From this, we conclude that before having the test result, the probability of E is unlikely. Once we have the test result, the probability that the person is infected has

increased several folds. ut

Here is another example of how conditional probabilities are calculated.

Example 4.28. For reasons one might guess, identical drugs are more expensive in the U.S. than in Canada. Some states that border Canada decided to offer their resi-dents the option to buy drugs from Canada. As you might expect, the drug companies oppose this effort voraciously. They claim that drugs from Canada may be tainted—

more so than drugs bought in the U.S. The data for this example are imagined. Yet, it can serve as a model to resolve the drug manufacturers claim.

You buy 25 pills, manufactured by a single company, from Canada and from the U.S. Some of the pills are tainted according to the following data:

Not tainted Tainted Total

Canada 11 4 15

US 7 3 10

Total 18 7 25

Select a pill randomly for analysis and let E be the event that the chosen pill is from Canada and F the event that the chosen pill is tainted. From the table above we conclude:

P (E) = 15

25 = 0.60 , P (F ) = 7

25 = 0.28 , P (E and F ) = 4

25 = 0.16 . Now suppose that the analysis revealed that the pill is tainted. How likely is it that the pill came from Canada? Again, based on the table

P (E|F ) = 4

7 ≈ 0.571 .

Conditional probability and independence 115 This is smaller than the original P (E) because there is a lower percentage of tainted pills from Canada than from the U.S. The same conditional probability can also be calculated according to:

P (E|F ) = 4 7 = 4

25÷ 7

25 =P (E and F ) P (F ) .

In other words, P (E|F ) is the ratio of the probability that both events occur divided by the probability of the conditioning event F .

Figure 4.9 Conditional probability.

The idea of conditional probability can be represented with a Venn diagram (Figure 4.9). We know that the outcome was F . The likelihood that E also occurred is the size of E and F relative to the size of F . ut We thus arrive at the definition of:

Conditional probability Let E and F be two events with P (F ) > 0. Then the conditional probability of E given that F has occurred is

P (E|F ) = P (E and F ) P (F ) .

Example 4.29. The seed banks in two prairie areas, labeled A and B, were studied.

The data consist of the relative number of seeds of three major grass species; call them a, b and c. The table below shows the fraction of the seeds from each species and area.

Species

Area a b c Total

A 0.40 0.21 0.09 0.70 B 0.10 0.09 0.11 0.30 Total 0.50 0.30 0.20 1.00

Tables such as this are called joint probability tables. Examples from the table: 70% of all seeds were from area A, 50% of the seeds came from species a. Denote the following events: E, a selected seed is from A and F , a selected seed is from a. Now select a

seed at random and identify it. It turns out to be from a. What is the probability that the seed was collected from A?

P (E|F ) = P (E and F ) P (F ) =0.40

0.50 = 0.80 .

In other words, 80% of the seeds from species a came from area A. Here P (E|F ) = 0.80 > P (E) = 0.70 .

Furthermore,

P (F|E) =P (E and F )

P (E) = 0.40

0.70= 0.571 > 0.5 = P (F ) .

It is not always the case that conditional probability improves chances that an event will occur. For example, let C be the event that the selected seed came from b. Then

P (E|C) = P (E and C)

P (C) = 0.21

0.30= 0.70 = P (E) .

In other words, P (E|C) = P (E). That is, if we are told that the seed belongs to a, the likelihood that it came from A remains unchanged. ut Throughout the preceding (and future) discussion of probability, we always interpret it in frequentist terms: “If we repeat the experiment many times, then the probability is obtained from the ratio of the number of times an event occurred to the total number of repetitions.”

4.4.2 Independence

If the occurrence of one event does not change the probability that another event will occur, we say that the events are independent.

Independent events (first definition) Events E and F are said to be indepen-dent if

P (E|F ) = P (E) .

If E and F are not independent then we say that they are dependent.

Similarly, if P (F|E) = P (F ) then E and F are said to be independent. Independence implies the following:

P (not E|F ) = P (not E) , P (E|not F ) = P (E) , P (not E|not F ) = P (not E) . Another way to define independent events is:

Independent events (second definition) The events E and F are independent if and only if

P (E and F ) = P (E)P (F ) .

Conditional probability and independence 117 This identity is called the multiplicative rule. The “if and only if” statement above implies that if E and F are independent, then the multiplicative rule is true and if the multiplicative rule is true, then the events are independent.

Example 4.30. Let E be the event that a statistics class begins on time and F the event that an ornithology class begins on time. The professors of both classes are unaware of each other’s behavior. We therefore assume that E and F are independent.

Suppose that P (E) = 0.9 and P (F ) = 0.6. Then

P (E and F ) = P (both classes begin on time)

= P (E)P (F )

= 0.9× 0.6 = 0.54 . Also

P (not E and not F ) = P (neither class begins on time)

= P (not E)P (not F )

= 0.1× 0.4 = 0.04 .

The probability that exactly one of the two classes begins on time is P (exactly one class begins on time) = 1− (0.54 + 0.04)

= 0.42 .

u t Independence applies to more than two events. If E1, E2, . . . , En are independent then

P (E1and E₂and . . . and E_n) = P (E1)P (E2)∙ ∙ ∙ P (Eⁿ) . (4.3) Note that the relations are not if and only if. In other words, if (4.3) is true, it does not necessarily mean that the events are independent. We say that n events are independent if and only if (4.3) is true and all possible pairs of events are independent and all possible triplets are independent and so on. To see this, consider the following example.

Example 4.31. In Figure 4.10 the proportion of the size of the rectangles A, B and C to the rectangular space S reflects their probability and the inset darker rectangle represents A∩ B ∩ C. Given that

P (A) = P (B) = P (C) = 1 6

Figure 4.10 Seemingly independent events that are not.

we have

P (A∩ B) = P (A ∩ C) = P (B ∩ C) = P (A ∩ B ∩ C) = 1 36 . Also

P (A) P (B) = P (A) P (C) = P (B) P (C) = 1 36 , so the events appear to be independent. However,

P (A∩ B ∩ C) 6= P (A) P (B) P (C) = 1 216 .

In other words, pairs of events are independent, but their triplet is not. Therefore,

the events A, B and C are not independent. ut