Probability and random variables
4.5 Algebra with probabilities
As we have seen, we combine probabilities in different ways for dependent and indepen-dent events. When the number of outcomes is small, we can enumerate all outcomes and compute probabilities. This is not possible when the number of outcomes is large.
Thus, we need to have some (when possible) rules about how to deal with addition, subtraction and in general, how to combine events and obtain their probabilities.
4.5.1 Sampling with and without replacement
Sampling with replacement refers to drawing a sample from a population and then putting the sample units back in the population before drawing another sample. Sam-pling without replacement refers to removing the sample from the population after it is drawn. When the population is small, sampling without replacement may change probabilities significantly. In other words, for a small population, sampling without replacement introduces noticeable dependency among the probabilities of events. Here is an example with a small population.
Example 4.32. Last semester, there were 35 students in my statistics class, 20 females and 15 males. Of the females, 15 had blond hair. Of the males, 10 were blond. Consider the following experiment: select a student at random with replace-ment and record gender and hair color. Let B1denote the event that the first chosen student is a male blond, B2 the second chosen student is a male blond and B3 the third chosen student is a male blond. Then
P (B3) = 10
35= 0.28 571
regardless of whether B1 or B2occurred. Next, sample without replacement. Then P (B3|B1and B2) =10− 2
35− 2 = 0.24 242 . Also
P (B3|not B1and not B2) = 10
35− 2 = 0.30 303 .
Thus, probabilities may be noticeably different, depending on whether we sample with
or without replacement. ut
Next, consider a large population.
Algebra with probabilities 119 Example 4.33. The Minnesota Vikings and the Green Bay Packers are two football teams with a long history of rivalry. Of the 10 000 people that show up to a game between these teams, 2 500 are Packers fans. The rest are Vikings fans. Choose 3 fans without replacement and define the following events: E1, the first choice is a Packers fan, E2, the second choice is a Packers fan and E3, the third choice is a Packers fan.
Then
P (E3|E1and E2) = 2 498
9 998 = 0.24 985 and
P (E3|not E1and not E3) =2 500
9 998 = 0.25 005 .
Thus, for all practical purposes, E1, E2and E3 are independent. ut A rule of thumb If at most 5% of the population is sampled without replacement,
then we may consider the sample as if it is with replacement.
4.5.2 Addition
As we already saw in Section 4.3.4, adding dependent events is like adding sets that are not disjoint. Consequently, the probabilities of adding two events when they are dependent or independent are different. We need to subtract the common outcomes of the events. For two events, we have the
Addition rule For any two events, E and F ,
P (E or F ) = P (E) + P (F )− P (E and F ) .
In words, the probability that the events E or F occur equals the sum of the proba-bilities that each event occurs, minus the probability that both E and F occur.
Example 4.34. Of the students in an Ecology class, 60% took statistics, 40% took calculus and 25% took both. Select a student randomly. What is the probability that the student took at least one of these two courses? Let E be the event that the selected student took statistics, F that the selected student took calculus and G that the selected student took at least one of the courses. Then
P (E) = 0.6 , P (F ) = 0.4 , P (E and F ) = 0.25 . Therefore,
P (G) = P (E or F )
= P (E) + P (F )− P (E and F )
= 0.60 + 0.40− 0.25
= 0.75 .
Now let H be the event that the selected student took none of the courses. Then P (H) = P (not (E or F ))
= 1− P (E or F )
= 0.25 .
Let I be the event that the selected student took exactly one of the courses. Then P (I) = P (E or F )− P (E and F )
= 0.75− 0.25
= 0.50 .
u t 4.5.3 Multiplication
Recall that for conditional probability with P (F ) > 0, we have P (E|F ) = P (E and F )
P (F ) . Therefore,
P (E and F ) = P (E|F )P (F ) . (4.4) When E and F are independent, P (E|F ) = P (E) and the last equation reduces to
P (E and F ) = P (E)P (F ) . (4.5)
Multiplication rule If two events E and F are dependent, then (4.4) holds. If the events are independent, then (4.5) holds.
Example 4.35. You are told that in a certain area, 70% of the birds are song birds.
Of these, 20% are sparrows. Also, 30% of the birds in the area are hummingbirds and of these, 40% are Calliope hummingbirds. Suppose that your best guess of what bird you will be seeing next as you walk in the forest is that it will be a random individual among the birds in the area. What is the probability that you will see a sparrow next?
First, we write the data in a convenient format:
% of birds Of these Songbirds 70% 20% sparrows Hummingbirds 30% 40% Calliope
Let D be the event that a song bird is observed and E a sparrow is observed. From the data
P (D) = 0.70 , P (E|D) = 0.20 . Therefore,
P (D and E) = P (E|D) × P (D) = 0.70 × 0.20 = 0.14 . ut
4.5.4 Counting rules
So far, we have dealt with a small number of events. Drawing tree diagrams and computing probabilities of events with and without replacement was relatively easy.
When the number of outcomes is large, computing all possible outcomes becomes impossible. We have to be a bit more clever in calculating probabilities. We deal with these issues next.
Algebra with probabilities 121
Multiplication
Multiplication applies to sampling with replacement. When we have two experiments, the first with n1 possible outcomes and the second with n2 possible outcomes, then the total number of outcomes, n, is
n = n1× n2.
Example 4.36. Ten single males and 12 single females are invited to a party. In how many ways can they be paired? Identify each male as Mi, i = 1, . . . , 10 and each female as Fi, i = 1, . . . , 12. Here n1= 10 and n2= 12. Now M1 can be paired in 12 different ways (with F1, . . . , F12). M2 can be paired in 12 different ways and so on up to M10. Therefore, the number of possible pairs is
n = 10× 12 = 120 .
Next suppose that 3 of the males are from England and 4 of the females are from France. Let C be the event that the male member of a pair is from England and the female from France. Assume that pairs are formed randomly. Then
P (C) = (number of pairs in C)
n = 3× 4
120 = 0.10 .
u t When we combine k experiments, each with ni outcomes, the number of possible outcomes, n, is
n = n1× n2× ∙ ∙ ∙ × nk.
Example 4.37. In a group of suspected terrorists, 10 speak English fluently, 15 are combat trained and 12 can fly planes. In how many ways can the group divide itself into subgroups of three? Here
n = 10× 15 × 12 = 1 800 .
Suppose that 3 of the English speaking suspected terrorists, 4 of the combat trained and 3 of those who can fly planes are from Saudi Arabia. Triplets are chosen randomly.
What is the probability that all members of a triplet are from Saudi Arabia?
P (all are from Saudi Arabia) = 3× 4 × 3
1 800 = 0.02 .
u t
Permutations
In the previous section, we chose members of a pair or a triplet with replacement.
When the order of choosing is important and when it is done without replacement, the rule for finding the number of possible events is different.
Example 4.38. Twelve students apply for summer field work that requires 5 different tasks: trapping (s1), mist-netting (s2), collecting vegetation data (s3), entering data (s4) and analyzing data (s5). All students are equally skilled at these tasks. How many different teams can be formed? We start with n = 12 students. For s1 we can
choose one student out of 12. Therefore, there are 12 possibilities. For a particular choice of a student for task s1, there are 11 students to choose from for task s2 and so on. Therefore, the number of different teams that can be chosen is
n (n− 1) ∙ ∙ ∙ (n − 4) = 12 × 11 × 10 × 9 × 8
= 95 040 . ut
To generalize the example, we consider n objects. They are to be arranged into an ordered subset of k objects. Thus, for the first slot in k we can choose one of the n objects in n different ways. For the second slot in k we can choose one of the objects in n− 1 ways. Therefore the first and the second slots in k can be chosen in n(n − 1) ways and so on. For the last kth slot, we have n− (k − 1) objects to choose from. We thus have the following definition:
Permutation The number of permutations of k objects selected randomly from a population of n objects, denoted by Pn,k is
Pn,k= n (n− 1) (n − 2) ∙ ∙ ∙ (n − k + 1) . (4.6)
Instead of writing the permutations explicitly, we use a short hand notation, called factorial.
Factorials
n! := n× (n − 1) × ∙ ∙ ∙ × 2 × 1 , 0! := 1 .
Therefore, we can write equation (4.6) as Pn,k:= n!
(n− k)! . (4.7)
To see this, expand the numerator and denominator and cancel equal elements.
Equation (4.7) is called the permutations equation. It gives the number of permu-tations (ways to arrange) of k objects taken from a population of size n, when the order of selecting the objects is important.
Example 4.39. In a random mating experiment, there are 10 females available for mating. A male mates with 6 of them. The mating order is important because the male’s viability deteriorates with more matings. In how many ways can that male mate with 6 females?
P10,6= 10!
(10− 6)! = 151 200 .
Now suppose that another male chooses the 6 females in the same order. In other words, both males chose the same permutation out of 151 200. We will then conclude that it is highly unlikely that the choice of mating order is random. ut Often, we wish to produce the permutations themselves, instead of counting the num-ber of permutations. Here is how we do it in R.
Algebra with probabilities 123 Example 4.40. We wish to produce a list of all permutations of the last five letters.
First, we create a vector of these letters and print it:
> x <- letters[22 : 26]
> nqd(array(x)) v w x y z
The function nqd() prints an array with no quotes and no dimension names. Its code is
nqd <- function(x) print(noquote(no.dimnames(x))) Next, we create a list of all permutations of the letters.
> library(combinat)
> px <- unlist(permn(x))
permn() returns a list of all permutations. We collapse the list into a vector with unlist(). Here are the first two permutations:
> nqd(array(px[1 : 10])) v w x y z v w x z y
To display all permutations compactly, we cut the matrix pmx into two and then column bind them with space between them. Finally, we print the results:
> pmx <- matrix(px, ncol=4, byrow=TRUE)
> pmx <- cbind(pmx[1 : 6, ], ' ', pmx[7 : 12, ], + ' ', pmx[13 : 18, ], ' ', pmx[19 : 24, ])
> nqd(pmx)
w x y z w y z x z y x w x z y w
w x z y w y x z y z x w z x y w
w z x y y w x z y x z w z x w y
z w x y y w z x y x w z x z w y
z w y x y z w x x y w z x w z y
w z y x z y w x x y z w x w y z ut
Combinations
Recall that in the case of permutations, order was important. When order is not impor-tant we deal with combinations. It should be clear that the number of possibilities when order is not important is smaller than otherwise.
Example 4.41. Consider five genes: A, B, C, D and E. In how many ways can you arrange three of these genes when their order is important?
P5,3= 5!
(5− 3)! = 60 .
If order is not important, the first gene in the arrangement can be in one of three positions (first, second or third), the second gene in one of two remaining positions and the last in the one remaining position. This means that we count fewer choices because
different orders represent the same choice if the same three genes were selected. How many fewer choices? As many as the number of permutations of the three genes. In other words, by a factor of 3!. Therefore, when order is not important, the number of distinct arrangements of three out of five genes is
5!
Note that the number of permutations (np) = the number of combinations (nc)× 3!.
To list the combinations, do
> x <- LETTERS[1 : 5] Thus we have the following definition:
Combination An unordered subset of k objects chosen from among n objects is called a combination. The number of such combinations is computed with
Cn,k :=
Example 4.42. In Example 4.39, we had 10 females and a male was to mate with 6 of them. We found that the male could mate with the 6 females in
P10,6= 10!
(10− 6)! = 151 200
different ways when order was important. Suppose that the male’s viability does not deteriorate with more matings. Then the order of mating is not important. The number of possible matings with 6 females out of 10 now becomes
C10,6 =
This represents a large number of choices. If another male mates with the same combination of females, we conclude that the choice of mates is not random.
Suppose that after the mating, the male shows a particular preference for 2 of the females. Now the male is presented with 8 females and is going to choose 4 of them
Algebra with probabilities 125 randomly. What is the probability that his 2 preferred females will be included in the male’s choice? Let E be the event that both preferred females are chosen. Assume that all possible choices (4 out of 8) are equally likely. Then
P (E) = number of outcomes in E
number of ways to select 4 out of 8 females
= Let us contrast permutations and combinations with and without replacement using R.
Example 4.43. The genome of an organism is carried in its DNA. Genes code for RNA, which in turn codes for amino acids. Genes that code for amino acids are composed of codons. When strung together (in a specific order), these amino acids form a protein. In RNA, each codon sub-unit consists of three of the following four nucleotide bases: adenine (A), cytosine (C), guanine (G) and uracil (U).
Imagine a brine with billions of A, C, G and U. In how many ways can a codon be arranged? The first slot of the sequence of three can be filled with four differ-ent nucleotides, the second with four and the third with four. Therefore, we have 43= 64 different codon combinations. These 64 permutations are called the RNA Codon Table. Here is how we make the table in R:
1 nucleotide <- c('U', 'C', 'A', 'G')
2 library(gtools)
3 RNA.codons <- permutations(4, 3, v = nucleotide,
4 repeats.allowed = TRUE)
5 RNA.table <- data.frame(RNA.codons[ 1 : 16, ], ' ')
6 for(i in 2 : 4){
To use the function permutations(), we first load the package gtools with a call to library() (line 2). In lines 3 and 4 we call permutations() with the appropriate arguments v (a vector) and allowing repetition (repeats.allowed). We then create the RNA table as a data.frame() and print it as.matrix(). The output from this script is
A C C C C C G C C U C C
A C G C C G G C G U C G
A C U C C U G C U U C U
A G A C G A G G A U G A
A G C C G C G G C U G C
A G G C G G G G G U G G
A G U C G U G G U U G U
A U A C U A G U A U U A
A U C C U C G U C U U C
A U G C U G G U G U U G
A U U C U U G U U U U U
Next, suppose that the brine is limited in the supply of A, C, G and U. Then change all 16 to 6 in the script above and change the call to permutations() to
RNA.codons <- permutations(4, 3, v = nucleotide, repeats.allowed=FALSE)
Now you get fewer permutations (because replacement is not allowed):
A C G C A G G A C U A C
A C U C A U G A U U A G
A G C C G A G C A U C A
A G U C G U G C U U C G
A U C C U A G U A U G A
A U G C U G G U C U G C
Order is important here because every codon sequence produces a different amino acid. Suppose that identical amino acids were produced with the same three nucleo-tides, regardless of their sequence with unlimited supply of nucleotides. Then the script
1 nucleotide <- c('U', 'C', 'A', 'G')
2 library(gtools)
3 RNA.codons <- combinations(4, 3, v = nucleotide,
4 repeats.allowed = TRUE)
5 RNA.codons <- data.frame(RNA.codons[1 : 10, ], ' ',
6 RNA.codons[11 : 20, ])
7 nqd(as.matrix(RNA.codons))
produces
A A A C C C
A A C C C G
A A G C C U
A A U C G G
A C C C G U
A C G C U U
A C U G G G
Random variables 127
A G G G G U
A G U G U U
A U U U U U
and if the supply of the nucleotides is limited (combination is without replacement), then change the call to combinations()
RNA.codons <- combinations(4, 3, v = nucleotide, repeats.allowed = FALSE)
and you you get the possible combinations A C G
A C U A G U C G U
Observe that sampling with and without replacement for permutations and
combina-tions result in a different number of outcomes! ut