There are many probability problems, especially those concerned with gambling, that can ultimately be reduced to questions about cardinalities of various sets. We saw several examples in Section 1.3. Those examples were simple, and they were chosen so that it was easy to determine the cardinalities of the required sets. However, in more complicated problems, it is extremely helpful to have some systematic methods for finding cardinalities of sets. Combinatorics is the study of systematic counting methods, which we will be using to find the cardinalities of various sets that arise in probability. The four kinds of counting problems we discuss are:
(i) ordered sampling with replacement; (ii) ordered sampling without replacement; (iii) unordered sampling without replacement; and (iv) unordered sampling with replacement.
Of these, the first two are rather straightforward, and the last two are somewhat complicated.
Ordered sampling with replacement
Before stating the problem, we begin with some examples to illustrate the concepts to be used.
Example 1.28. Let A, B, and C be finite sets. How many triples are there of the form
Solution. Since there are |A| choices for a, |B| choices for b, and |C| choices for c, the
total number of triples is|A| · |B| · |C|.
Similar reasoning shows that for k finite sets A1,...,Ak, there are|A1|···|Ak| k-tuples of
the form(a1,...,ak) where each ai∈ Ai.
Example 1.29. Suppose that to send an Internet packet from the east coast of the United
States to the west coast, a packet must go through a major east-coast city (Boston, New York, Washington, D.C., or Atlanta), a major mid-west city (Chicago, St. Louis, or New Orleans), and a major west-coast city (San Francisco or Los Angeles). How many possible routes are there?
Solution. Since there are four east-coast cities, three mid-west cities, and two west-coast
cities, there are 4· 3 · 2 = 24 possible routes.
Example 1.30 (ordered sampling with replacement). From a deck of n cards, we draw k cards with replacement; i.e., we draw each card, make a note of it, put the card back in the
deck and re-shuffle the deck before choosing the next card. How many different sequences of k cards can be drawn in this way?
Solution. Each time we draw a card, there are n possibilities. Hence, the number of
possible sequences is
n· n···n
k times
= nk.
Ordered sampling without replacement
In Example 1.28, we formed triples(a,b,c) where no matter which a ∈ A we chose, it did not affect which elements we were allowed to choose from the sets B or C. We next consider the construction of k-tuples in which our choice for the each entry affects the choices available for the remaining entries.
Example 1.31. From a deck of 52 cards, we draw a hand of 5 cards without replace-
ment. How many hands can be drawn in this way?
Solution. There are 52 cards for the first draw, 51 cards for the second draw, and so on.
Hence, there are
52· 51 · 50 · 49 · 48 = 311875200. different hands
Example 1.32 (ordered sampling without replacement). A computer virus erases files
from a disk drive in random order. If there are n files on the disk, in how many different orders can k≤ n files be erased from the drive?
Solution. There are n choices for the first file to be erased, n − 1 for the second, and so
on. Hence, there are
n(n − 1)···(n − [k − 1]) = n!
(n − k)! different orders in which files can be erased from the disk.
Example 1.33. Let A be a finite set of n elements. How may k-tuples (a1,...,ak) of
distinct entries ai∈ A can be formed?
Solution. There are n choices for a1, but only n−1 choices for a2since repeated entries
are not allowed. Similarly, there are only n− 2 choices for a3, and so on. This is the same
argument used in the previous example. Hence, there are n!/(n − k)! k-tuples with distinct elements of A.
Given a set A, we let Akdenote the set of all k-tuples(a1,...,ak) where each ai∈ A. We
denote by Ak∗the subset of all k-tuples with distinct entries. If|A| = n, then |Ak| = |A|k= nk, and|Ak∗| = n!/(n − k)!.
Example 1.34 (the birthday problem). In a group of k people, what is the probability
that two or more people have the same birthday?
Solution. The first step in the solution is to specify the sample space Ω and the proba-
bilityP. Let D := {1,...,365} denote the days of the year, and let Ω := {(d1,...,dk) : di∈ D}
denote the set of all possible sequences of k birthdays. Then|Ω| = |D|k. Assuming all sequences are equally likely, we takeP(E) := |E|/|Ω| for arbitrary events E ⊂ Ω.
Let Q denote the set of sequences (d1,...,dk) that have at least one pair of repeated
entries. For example, if k= 9, one of the sequences in Q would be (364,17,201,17,51,171,51,33,51).
Notice that 17 appears twice and 51 appears 3 times. The set Q is complicated. On the other hand, consider Qc, which is the set of sequences(d1,...,dk) that have no repeated entries.
Then |Qc| = |D|! (|D| − k)!, and P(Qc) = |Qc| |Ω| = |D|! |D|k(|D| − k)!,
where|D| = 365. A plot of P(Q) = 1 − P(Qc) as a function of k is shown in Figure 1.15. As the dashed line indicates, for k≥ 23, the probability of two more more people having the same birthday is greater than 1/2.
0 5 10 15 20 25 30 35 40 45 50 55 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Figure 1.15. A plot of P(Q) as a function of k. For k ≥ 23, the probability of two or more people having the same
birthday is greater than 1/2.
Unordered sampling without replacement
Before stating the problem, we begin with a simple example to illustrate the concept to be used.
Example 1.35. Let A = {1,2,3,4,5}. Then A3 contains 53= 125 triples. The set of
triples with distinct entries, A3
∗, contains 5!/2! = 60 triples. We can write A3∗as the disjoint
union
A3∗ = G123∪ G124∪ G125∪ G134∪ G135
∪ G145∪ G234∪ G235∪ G245∪ G345,
where for distinct i, j,k,
Gi jk := {(i, j,k),(i,k, j),( j,i,k),( j,k,i),(k,i, j),(k, j,i)}.
Each triple in Gi jkis a rearrangement, or permutation, of the same three elements.
The above decomposition works in general. Write Ak∗as the union of disjoint sets,
Ak∗ = G, (1.32)
where each subset G consists of k-tuples that contain the same elements. In general, for a
k-tuple built from k distinct elements, there are k choices for the first entry, k−1 choices for
the second entry, and so on. Hence, there are k! k-tuples that can be built. In other words, each G in (1.32) has|G| = k!. It follows from (1.32) that
|Ak
and so the number of different subsets G is |Ak ∗| k! = n! k!(n − k)!.
The standard notation for the above right-hand side is n k := n! k!(n − k)!
and is read “n choose k.” In MATLAB,nk= nchoosek(n,k). The symbolnkis also called the binomial coefficient because it arises in the binomial theorem, which is discussed in Chapter 3.
Example 1.36 (unordered sampling without replacement). In many card games, we are
dealt a hand of k cards, but the order in which the cards are dealt is not important. From a deck of n cards, how many k-card hands are possible?
Solution. First think about ordered hands corresponding to k-tuples with distinct en-
tries. The set of all such hands corresponds to Ak∗. Now group together k-tuples composed of the same elements into sets G as in (1.32). All the ordered k-tuples in a particular G represent rearrangements of a single hand. So it is really the number of different sets G that corresponds to the number of unordered hands. Thus, the number of k-card hands isnk.
Example 1.37. A new computer chip has n pins that must be tested with all patterns in
which k of the pins are set high and the rest low. How many test patterns must be checked?
Solution. This is exactly analogous to dealing k-card hands from a deck of n cards. The
cards you are dealt tell you which pins to set high. Hence, there arenkpatterns that must be tested.
Example 1.38. A 12-person jury is to be selected from a group of 20 potential jurors.
How many different juries are possible?
Solution. There are
20 12 = 20! 12! 8! = 125970 different juries.
Example 1.39. A 12-person jury is to be selected from a group of 20 potential jurors of
which 11 are men and nine are women. How many 12-person juries are there with five men and seven women?
Solution. There are11 5
ways to choose the five men, and there are97ways to choose the seven women. Hence, there are
11 5 9 7 = 11! 5! 6!· 9! 7! 2! = 16632 possible juries with five men and seven women.
Example 1.40. An urn contains 11 green balls and nine red balls. If 12 balls are chosen
at random, what is the probability of choosing exactly five green balls and seven red balls?
Solution. Since balls are chosen at random, the desired probability is
number of ways to choose five green balls and seven red balls number of ways to choose 12 balls .
In the numerator, the five green balls must be chosen from the 11 available green balls, and the seven red balls must be chosen from the nine available red balls. In the denominator, the total of 5+ 7 = 12 balls must be chosen from the 11 + 9 = 20 available balls. So the required probability is 11 5 9 7 20 12 = 16632 125970 ≈ 0.132.
Example 1.41. Consider a collection of N items, of which d are defective (and N − d
work properly). Suppose we test n≤ N items at random. Show that the probability that k of the n tested items are defective is
d k N− d n− k N n . (1.34)
Solution. Since items are chosen at random, the desired probability is
number of ways to choose k defective and n− k working items number of ways to choose n items .
In the numerator, the k defective items are chosen from the total of d defective ones, and the
n− k working items are chosen from the total of N − d ones that work. In the denominator,
the n items to be tested are chosen from the total of N items. Hence, the desired numerator isdkNn−k−d, and the desired denominator isNn.
Example 1.42 (lottery). In some state lottery games, a player chooses n distinct num-
bers from the set{1,...,N}. At the lottery drawing, balls numbered from 1 to N are mixed, and n balls withdrawn. What is the probability that k of the n balls drawn match the player’s choices?
Solution. Let D denote the subset of n numbers chosen by the player. Then {1,...,N} = D∪ Dc. We need to find the probability that the lottery drawing chooses k numbers from D
and n− k numbers from Dc. Since|D| = n, this probability is n k N− n n− k N n .
Notice that this is just (1.34) with d= n. In other words, we regard the numbers chosen by the player as “defective,” and we are finding the probability that the lottery drawing chooses
k defective and n− k nondefective numbers.
Example 1.43 (binomial probabilities). A certain coin has probability p of turning up
heads. If the coin is tossed n times, what is the probability that k of the n tosses result in heads? Assume tosses are independent.
Solution. Let Hi denote the event that the ith toss is heads. We call i the toss index,
which takes values 1,...,n. A typical sequence of n tosses would be
H1∩ H2c∩ H3∩ ··· ∩ Hn−1∩ Hnc,
where Hc
i is the event that the ith toss is tails. The probability that n tosses result in k heads
and n− k tails is P H1∩ ··· ∩ Hn ,
where Hiis either Hior Hic, and the union is over all such intersections for which Hi= Hi
occurs k times and Hi= Hicoccurs n− k times. Since this is a disjoint union,
P H1∩ ··· ∩ Hn =
∑
P H1∩ ··· ∩ Hn . By independence, P H1∩ ··· ∩ Hn = P H1 ···P Hn = pk(1 − p)n−kis the same for every term in the sum. The number of terms in the sum is the number of ways of selecting k out of n toss indexes to assign to heads. Since this number isnk, the probability that k of n tosses result in heads is
n k
pk(1 − p)n−k.
Example 1.44 (bridge). In bridge, 52 cards are dealt to four players; hence, each player
has 13 cards. The order in which the cards are dealt is not important, just the final 13 cards each player ends up with. How many different bridge games can be dealt?
Solution. There are52 13
ways to choose the 13 cards of the first player. Now there are only 52− 13 = 39 cards left. Hence, there are3913ways to choose the 13 cards for the second player. Similarly, there are2613ways to choose the second player’s cards, and 13
13
= 1 way to choose the fourth player’s cards. It follows that there are 52 13 39 13 26 13 13 13 = 52! 13! 39!· 39! 13! 26!· 26! 13! 13!· 13! 13! 0! = (13!)52!4 ≈ 5.36 × 1028
games that can be dealt.
Example 1.45. Traditionally, computers use binary arithmetic, and store n-bit words
composed of zeros and ones. The new m–Computer uses m-ary arithmetic, and stores n- symbol words in which the symbols (m-ary digits) come from the set {0,1,...,m − 1}. How many n-symbol words are there with k0zeros, k1ones, k2twos, . . . , and km−1copies
of symbol m− 1, where k0+ k1+ k2+ ··· + km−1= n?
Solution. To answer this question, we build a typical n-symbol word of the required
form as follows. We begin with an empty word, ( , ,..., )
n empty positions
. From these n available positions, there arekn
0
ways to select positions to put the k0zeros.
For example, if k0= 3, we might have
( ,0, ,0, ,..., ,0)
n− 3 empty positions
.
Now there are only n− k0 empty positions. From these, there are
n−k0
k1
ways to select positions to put the k1ones. For example, if k1= 1, we might have
( ,0,1,0, ,..., ,0)
n− 4 empty positions
.
Now there are only n− k0− k1empty positions. From these, there are
n−k
0−k1
k2
ways to select positions to put the k2 twos. Continuing in this way, we find that the number of n-symbol words with the required numbers of zeros, ones, twos, etc., is
n k0 n− k0 k1 n− k0− k1 k2 ··· n− k0− k1− ··· − km−2 km−1 , which expands to n! k0!(n − k0)!· (n − k0)! k1!(n − k0− k1)!· (n − k0− k1)! k2!(n − k0− k1− k2)!··· ··· (n − k0− k1− ··· − km−2)! km−1!(n − k0− k1− ··· − km−1)!.
Canceling common factors and noting that(n − k0− k1− ··· − km−1)! = 0! = 1, we obtain
n! k0! k1!···km−1!
We call n k0,...,km−1 := n! k0! k1!···km−1! the multinomial coefficient. When m= 2,
n k0,k1 = n k0,n − k0 = n! k0!(n − k0)! = n k0 becomes the binomial coefficient.
Unordered sampling with replacement
Before stating the problem, we begin with a simple example to illustrate the concepts involved.
Example 1.46. An automated snack machine dispenses apples, bananas, and carrots.
For a fixed price, the customer gets five items from among the three possible choices. For example, a customer could choose one apple, two bananas, and two carrots. To record the customer’s choices electronically, 7-bit sequences are used. For example, the sequence (0,1,0,0,1,0,0) means one apple, two bananas, and two carrots. The first group of zeros tells how many apples, the second group of zeros tells how many bananas, and the third group of zeros tells how many carrots. The ones are used to separate the groups of zeros. As another example,(0,0,0,1,0,1,0) means three apples, one banana, and one carrot. How many customer choices are there?
Solution. The question is equivalent to asking how many 7-bit sequences there are with
five zeros and two ones. From Example 1.45, the answer is57,2=75=72.
Example 1.47 (unordered sampling with replacement). Suppose k numbers are drawn
with replacement from the set A= {1,2,...,n}. How many different sets of k numbers can be obtained in this way?
Solution. Think of the numbers 1,...,n as different kinds of fruit as in the previous
example. To count the different ways of drawing k “fruits,” we use the bit-sequence method. The bit sequences will have n− 1 ones as separators, and the total number of zeros must be
k. So the sequences have a total of N := n − 1 + k bits. How many ways can we choose n− 1 positions out of N in which to place the separators? The answer is
N n− 1 = n− 1 + k n− 1 = k+ n − 1 k . Just as we partitioned Ak
∗in (1.32), we can partition Akusing Ak = G,
where each G contains all k-tuples with the same elements. Unfortunately, different Gs may contain different numbers of k-tuples. For example, if n= 3 and k = 3, one of the sets G would be
while another G would be
{(1,2,2),(2,1,2),(2,2,1)}.
How many different sets G are there? Although we cannot find the answer by using an equation like (1.33), we see from the above analysis that there arek+n−1k sets G.