Combinatorial analysis - Introduction to CEE 3030 page 1

By Gilberto E. Urroz, January 2003

Conditional probability. Conditional probability is the probability associated with an event, say A, given the occurrence of a related event, say B. For example, when throwing a fair die you may be interested in determining what is the probability of getting a 3 given that the number selected is odd. In this example of conditional probability, the event of interest is A =

“getting a 3”, and the condition is B = “the number is odd”. The notation for conditional probability is P(A|B) interpreted as “the probability of A given B” or “the probability of the occurrence of A given that B has occurred.” The corresponding definition is

) .

Suppose that events A and B belong to a sample space S. The probabilities P(A∩B) and P(B) are calculated by the classical approach, i.e., P(A∩B) = hA∩B/n, and P(B) = hB/n, where n is the total number of outcomes in S, and hA∩B and hB are the number of outcomes in events A∩B and B, respectively. Thus, the conditional probability can also be written as

Notice that this latter result is similar to the classical definition of probability if we think of B as a “reduced” sample space, i.e., the condition (B) becomes the new sample space in the definition of conditional probability.

Example 1. Highway closing under snow conditions. Suppose we are interested in determining the probability that a high-elevation highway will remain open under snow conditions. Our records indicate that for the 3-month winter period (n = 90 days), snow in amounts significant enough to affect the traffic conditions in the highway of interest is recorded during 30 days in a typical year. Also, records show that during 15 days in the winter period snow conditions produce closure of the highway. If we define the events B = “significant snow observed in the winter months” and A = “highway closure”, the data indicates that

P(B) = P(“significant snow observed in the winter months”) = 30/90 = 1/3,

Conditional probability – Page 2

Theorems on conditional probability. The following are two important theorems related to conditional probability:

(a) For any three events A1, A2, and A3 the following relationship holds true:

P(A₁∩A₂∩A₃) = P(A₁)P(A₂|A₁)P(A₃|A₁∩A₂).

Example 2. Defective computer chips. Suppose you are in the process of fixing a computer by replacing three identical computer chips and you have a container with 20 computer chips from which to select the replacements. The chips are selected at random.

Unbeknownst to you, 5 of the computer chips in the container are defective. What is the probability that you would select three defective chips for your computer repair?

Solution. Let A1, A2, A3 be the events that you select a defective computer chip in the 1^st, 2^nd, and 3^rd picks out of the container. Thus, you are interested in calculating

P(A1∩A2∩A3) = P(A1)P(A2|A1)P(A3|A1∩A2)

In this formula, P(A1) is the probability that a defective chip is selected in the first trial.

Since there are 5 defective chips out of 20 chips, P(A1) = 5/20 = ¼ = 0.25. P(A2|A1) is the probability that the second chip is defective given that the first was defective. Now, if the first chip was defective then there remain 4 defective chips out of 19 chips in the

container, thus, P(A₂|A₁) = 4/19. Finally, P(A₃|A₁∩A₂) is the probability that you get a defective chip in the third pick given that first two chips were defective. If chips 1 and 2 were defective, there remain 3 defective chips out of 18 in the container, thus,

P(A3|A1∩A2) = 3/18 = 1/6. Finally,

P(A₁∩A2∩A3) = P(A₁)P(A₂|A₁)P(A₃|A₁∩A2) = (1/4)(4/19)(1/6) = (1x4x1)/(4x19x6) = 1/114 = 0.00877

(b) If an event A must result in one of the mutually exclusive events A1, A2, …, An, then P(A) = P(A1)P(A|A1) + P(A2)P(A|A2) + … + P(An)P(A|An)

The event A and its relation to the mutually exclusive events A1, A2, …, An, is illustrated in the following Venn diagram:

Notice the equation for P(A) is equivalent to

P(A) = P(A∩A1) + P(A∩A2) + … + P(A∩An).

Example 3. Irrigation methods. While conducting a study on the effects of different irrigation methods on a given crop, you define the following events:

• A₁ = sprinkler irrigation

• A2 = steady furrow irrigation

• A₃ = surge furrow irrigation

• A4 = drip irrigation

Based on your evaluation of 50 experimental plots, you find that 20 plots use sprinkler irrigation, 15 plots use steady furrow irrigation, 8 plots use surge furrow irrigation, and 7 plots use drip irrigation to irrigate the same type of crop. Thus,

P(A1) = 20/50 = 2/5, P(A2) = 15/50 = 3/10, P(A3) = 8/50, and P(A4) = 7/50.

You also find that the crop is successful if using sprinkler irrigation 85% of the time, if using steady furrow irrigation 90% of the time, if using surge furrow irrigation 70% of the time, and if using drip irrigation 60% of the time. Thus, if event A represents “a successful crop”, then we have that

P(A|A1) = 0.85, P(A|A2) = 0.90, P(A|A3) = 0.70, and P(A|A4) = 0.60.

Thus, the probability of a successful crop (event A) in the experimental plots that use four different irrigation systems is

P(A) = P(A1)P(A|A1) + P(A2)P(A|A2) + P(A3)P(A|A3) + P(A4)P(A|A4) = (2/5)x0.85 + (3/10)x0.90 + (8/50)x0.70 + (7/50)x0.60 = 0.806

Example 4. Highway traveling. To reach Grenoble (France) from Turin (Italy) one can follow either of two routes. The first connects Turin and Grenoble, whereas the second passes through Chambery (France), i.e., the second route is Turin-Chambery-Grenoble. During extreme weather conditions in winter, travel between Turin and Grenoble is not always possible because some parts of the highway may not be open to traffic.

Define the following events:

• A = the highway from Turin to Grenoble is open

• B = the highway from Turin to Chambery is open

• C = the highway from Chambery to Grenoble is open

Conditional probability – Page 4

In anticipation of driving from Turin to Grenoble, a traveler listens to the next day’s weather forecast. If snow is forecast for the next day over the southern Alps, one can assume (on the basis of past records) that P(A) = 0.6, P(B) = 0.7, P(C) = 0.4, P(C|A) = 0.5, and P(A|B∩C) = 0.4.

(a) What is the probability that the traveler will be able to reach Grenoble from Turin?

(b) What is the probability the traveler will be able to drive from Turin to Grenoble by way of Chambery?

Solution. Given P(A) = 0.6, P(B) = 0.7, P(C) = 0.4, P(C|A) = 0.5, and P(A|B and C) = 0.4, we need to find:

(a) P(A ∪ (B ∩ C) ) = P(A) + P(B ∩ C), since A and (B ∩ C) are mutually exclusive (b) P(B ∩ C)

(c) Is P(A) > P(B ∩ C) or vice versa?

First, we solve (b), i.e., find P(B ∩ C) using the data given P(C|A) = P(A ∩ C)/P(A),

by definition. From which

P(A ∩ C) = P(A)⋅P(C|A) = 0.6x0.5 = 0.3 Next, use the following equation

P(C) = P(C∩A) + P(C∩B) = P(A)*P(C|A) + P(B)*P(C|B), resulting in

0.4 = 0.6 x 0.5 + 0.7 x P(C|B), from which

P(C|B) = (0.4-0.6 x 0.5)/0.7 = 1/7 Then,

P(B ∩ C) = P(B) P(C|B) = 0.7 x (1/7) = 0.1.

Thus, (b) P(B ∩ C) = 0.1

(a) P(A ∪ (B ∩ C)) = P(A) + P(B ∩ C) = 0.6 + 0.1 = 0.7

(c) Since, P(A) > P(B ∩ C) [0.6 > 0.1], to maximize the probability of getting to Grenoble, use path A

Independent events. From the definition of conditional probability, namely, P(A|B) = P(A∩B)/P(B), it follows that

P(A∩B) = P(B)⋅ P(A|B).

This statement is valid for any two events A and B.

If events A and B are independent, the occurrence of B has no effect on the occurrence of A and P(A|B) = P(A). Replacing this result in the equation above we find that, for independent events A and B,

P(A∩B) = P(B)⋅ P(A).

It turns out that the inverse statement is also true, i.e., if P(A∩B) = P(B)⋅ P(A), then events A and B are independent. These two statements can be summarized by saying that events A and B are independent, if and only if, P(A∩B) = P(B)⋅ P(A).

Example 5. Highway construction. Highway construction in a remote area is dependent on the availability of construction workers in the area and on the weather conditions. Suppose that event A represents “availability of construction workers” and event B represents “favorable weather conditions”, and that these events are independent of each other. Previous data from the area indicates that P(A) = 0.8 and P(B) = 0.75. Suppose that we need to determine the probability of “availability of construction workers or favorable weather conditions”, i.e., P(A∪B). We will use the formula

P(A∪B) = P(A) + P(B) - P(A∩B),

and since events A and B are independent, we can also write, P(A∩B) = P(A)P(B) = 0.8x0.75 = 0.60 Thus,

P(A∪B) = P(A) + P(B) - P(A)P(B) = 0.8 + 0.75 - 0.8x0.75 = 0.95.

Notice that the probability P(A∩B) = P(“availability of construction workers AND favorable weather conditions”) is only 60%, i.e., construction will be possible only 60% of the time.

The result P(A∪B) = 0.95 simply indicates that one of the two conditions for highway

constructions (either, availability of construction workers OR favorable weather conditions, but not necessarily both) are found 95% of the time. From the point of view of predicting the ability to carry out construction activities the probability P(A∩B) = 0.60 is more important.

Example 6. Wave heights in a lake. In the process of re-designing a harbor in a lake, data is collected on wind velocity in the area as well as water temperature to check what effect these two variables have on wave height in the harbor. Of interest for the designer are the conditions A = “strong wind velocity” (registered when wind velocity is larger than 15 mph) and B = “warm waters” (registered when water temperature is larger than 70^oF). Records indicate that P(A) = 0.350, P(B) = 0.150, and P(A∩B) = 0.052. Are the events A and B, i.e., “strong wind velocity” and “warm waters”, independent?

Solution. To check for independence, we need to check that P(A∩B) = P(A)P(B). We know that P(A∩B) = 0.052, and we find that P(A)P(B) = 0.0525. We notice that P(A∩B) ≈ P(A)P(B), and we may conclude that the events A and B are indeed independent.

Bayes’ theorem or Bayes’ rule. Suppose that A1, A2, …, An, are mutually exclusive events whose union is the sample space, S (i.e., one of the events must occur). Then, if A is any

Conditional probability – Page 6

for k = 1, 2, …, n. This formula allows us to find the probabilities of the different events, A1, A2, …, An, that can cause event A to occur. This fact explains the alternative name given to Bayes’ rule: the theorem on the probability of causes.

Example 7. Irrigation methods (part 2). The events A1, A2, A3, and A4 for an irrigation study were defined in Example 3 as follows: A₁ = sprinkler irrigation, A₂ = steady furrow irrigation, A₃

= surge furrow irrigation, and A4 = drip irrigation. The corresponding probabilities found were

P(A1) = 20/50 = 2/5, P(A2) = 15/50 = 3/10, P(A3) = 8/50, and P(A4) = 7/50.

Also, if event A represents “a successful crop”, we found that

P(A|A1) = 0.85, P(A|A1) = 0.90, P(A|A1) = 0.70, and P(A|A1) = 0.60.

Bayes’ rule can be used to determine the probability that a given irrigation method was used given that a successful crop was detected, e.g., for sprinkler irrigation we can calculate

These results indicate that the probability that sprinkler irrigation was used to produce a successful crop is 42.18%, etc.

Combinatorial analysis. In the evaluation of probabilities it is often necessary to determine the number of outcomes of a given event as well as that of the sample space. Combinatorial analysis provide mathematical approaches for determining those numbers particularly when the number of outcomes is large. Some simple techniques of combinatorial analysis are shown next.

Tree diagrams and the fundamental principle of counting. Tree diagrams are sketches of choices of elements showing all possible combinations. The following example illustrates this idea and introduces the fundamental principle of counting for situations to determine the total number of configurations depicted in tree diagrams.

Example 8. Putting together a computer system. Suppose that you are putting together a computer system and you can choose three different motherboards (M1, M2, M3), two different hard disks (H1, H2), and three different CD-R drives (C1, C2, C3). A three diagram can be used to illustrate all possible configurations of the computer system. Start by showing the three branches corresponding to the motherboards as the first step in the tree. Then, branch out of each tip (M1, M2, M3) with two branches corresponding to the two hard disks (H1, H2). Finally,

include the three branches corresponding to the three CD-R drives (C1, C2, C3) for each of the two hard disks branches of step 2. The three diagram is illustrated in the figure shown below.

The tips of the final branching correspond to all possible configurations of the three

components of the computer system. Notice that there are 18 possible configurations, each configuration corresponding to the path followed through the tree diagram from the initial point (to the left of the first branching) to the different terminal tips (to the right of the diagram). For example, configuration 1 corresponds to the path M1-H1-C1, or, using formal set notation, M1∩H1∩C1 (interpreted as “M1 and H1 and C1”), while configuration 9 corresponds to M₂∩H₁∩D₃.

The number of possible configurations (i.e.,18) results from multiplying the number of options in step 1 (3 motherboards), by the number of options in the second step (2 hard disks), by the number of options in the third step (3 CD-R drives), i.e., 3x2x3 = 18. This illustrates the fundamental principle of counting: When putting together configurations of elements selected in k different steps and there are n1 options in the first step, n2 options in the second step, and so on, ending with n_k options in the k-th step, the total number of possible configurations is given by n⋅ n⋅ n …⋅ n.

Conditional probability – Page 8

Solution. The description of this activity follows the conditions of the fundamental principle of counting, namely, we select the sequence of inspection by choosing between 6 bridge piers (n1

= 6), 3 highway decks (n2 = 3), 4 retaining walls (n3 = 4), and 8 steel trusses (n4 = 8). Thus, the possible number of configurations is n1⋅ n2⋅ n3⋅ n4 =6x3x4x8 = 576.

Factorials. The factorial of a positive integer number n is the product n! = n⋅ (n-1) ⋅ (n-2)…3⋅ 2⋅ 1.

Following the definition of factorial we can write:

n! = n⋅ (n-1)! = n⋅ (n-1)⋅ (n-2)! = n ⋅ (n-1)⋅ (n-2)⋅ (n-3)! = … For example,

6! = 6x5! = 6x5x4! = 6x5x4x3! = 6x5x4x3x2! = 6x5x4x3x2x1 Also, for two positive integer numbers n, r, with r < n,

)

The Gamma function and factorials. The Gamma function is a specialized mathematical function that has a variety of applications in probability, differential equations, and other branches of mathematics. It is defined by the following integral:

∫

^∞ ⁻ ⁻

=

Γ

)

( z e

t

dt

The Gamma function has the property that

Γ(z+1) = z⋅Γ (z).

Thus,

Γ(z) = (z-1) ⋅ Γ(z), Γ(z-1) = (z-2) ⋅ Γ(z-2), etc.

If z is an integer, say z = n, then we can write

Γ(n+1) = n⋅(n-1)⋅(n-2)⋅⋅⋅3⋅2⋅1 = n!

The use of the Gamma function allows us to define factorials of non-integer numbers and even of negative numbers (although, the gamma function of all negative integers is infinite).

Maple’s Gamma function and factorials. The Gamma function in Maple is called by using the name GAMMA(x). Here are some values of the GAMMA function in Maple:

> GAMMA(x);#Function statement

Γ x( )

> GAMMA(5);#Should be the same as (5-1)! = 4!

> GAMMA(-3);#For negative integer numbers GAMMA(x) is infinite Error, (in GAMMA) numeric exception: division by zero

> GAMMA(2.5);#GAMMA(x) for a non-integer positive number 1.329340388

> GAMMA(-1.2); );#GAMMA(x) for a non-integer negative number 4.850957141

The following is a plot of the Gamma function produced in Maple:

> plot(GAMMA(x),x=-4..5,-20..20,labels=["x","Gamma(x)"], discont=true);

Conditional probability – Page 10

Permutations. Given n objects we select r (with r<n) of those objects and order them in a line.

The first position allows n possible choices, while there are (n-1) choices in the second position, (n-2) in the third, and so on, ending with (n-r+1) choices in the r-th position. Thus, the fundamental principle of counting provides for n(n-1)(n-2)…(n-r+1) configurations or permutations of the n objects taken r at a time. This is indicated symbolically as

)!

Notice that nPr and P(n,r) are different notations representing the same calculation, i.e., the number of permutations of n objects taken r at a time.

If n = r , i.e., if interested in the number of permutations of n objects taken all at once, the result is

Example 10. Selection of officers in an engineering company. As a member of the board of directors of your own highly-profitable engineering company you are faced with the need to select three people to serve as President, Chief Executive Officer (C.E.O.), and Chief Operating Officer (C.O.O.) out of a pool of 8 equally qualified candidates. The situation is similar to choosing three objects out of eight and placing them in a certain order in a line. The order in this particular situation are the different specific offices to be occupied by the selected candidates. You would like to know how many permutations of candidates are there to fill these three offices, i.e., you want to find _nP_r, or P(n,r), with n = 8 and r = 3. The formula to

You can also reason this calculation by using the fundamental principle of counting (multiplying the possible options) indicating that there are 8 possible candidates for President of your company that you can select as a first step. After having selected a candidate for President, you are left only with 7 choices for C.E.O., and, after having chosen a President and a C.E.O., with only 6 choices for C.O.O. Thus, you have 8x7x6 = 336 possible permutations.

Example 11. Permutations of computers in a computer center. A computer manufacturing company has donated 8 desktop computers to be placed in a single long table in a computer lab. The 8 computers are exactly the same except for the color of the monitor case, i.e., there will be eight monitors of different colors in the table. How many possible permutations of those 8 computers are possible?

In this case we want to find

That’s forty thousand, three hundred and twenty different ways of arranging those computers of different colors!

More on permutations. Suppose that now you have n objects consisting of n1 objects of a certain kind, n2 objects of a second kind, and so on, ending with nk objects of a k-th kind, so that

n1 + n2 + … + nk = n.

Then, the number of different permutations of the n objects is given by

!

Example 12. Permutations of components of a communication system. We are trying to put together a communication system by choosing one out of 3 components of type A, one out of 2 components of type B, and one out of 3 components of type C. The components can be incorporated into the system in any order. Thus, we can use the formula shown above and write

n1 = 3, n2 = 2, n3 = 3, n = n1 + n2 + n3 = 3 + 2 + 3 = 8.

We are interested in finding the possible number of permutations of those eight components by using

Combinations. When we determine the number of permutations of n objects taken r at a time we take into consideration the order in which the objects are arranged and call it a

permutation. Consider the case in which n = 5 and r = 3. Say that the five objects we have to choose from are labeled {A, B, C, D, E}, and suppose that we choose objects {B, D, E}. The arrangement BDE is a permutation of the chosen objects, and so are BED, DBE, DEB, EBD, and EDB.

If the order of the objects is not a consideration, the selection is referred to as a combination.

Thus, BDE, BED, DBE, DEB, EBD, and EDB, represent the same combination of objects {B, D, E}.

The number of combinations of n objects taken r at a time will be, therefore, smaller than the corresponding number of permutations. The number of combinations of n objects taken r at a time is calculated as

Conditional probability – Page 12

NOTE: The notation (ⁿ_r) is also referred to as a binomial coefficient because it represents the r-th coefficient in r-the expansion of r-the binomial (x+y)ⁿ, i.e.,

Notice that in this expansion, the value of r starts at 0 and ends at n, indicating that n+1 terms are involved in the right-hand side of the expression.

Example 13. Selecting members to a board of directors. Suppose that 8 candidates file to be elected members of a 3-person board of directors of a small firm. There is no distinction among the 3 members of the board of directors (which wasn’t the case in Example 8), thus the order of the selection is not relevant. Of interest in this case is to determine the number of combinations of 8 persons taken 3 at a time, i.e.,

56

Thus, there are 56 different ways you can select 3 out of the pool of 8 persons to form the board of directors of the small firm under consideration.

Combinatorial analysis functions in Maple. Combinatorial analysis in Maple can be performed by loading the functions in package

combinat

> with(combinat);

Chi bell binomial cartprod character choose composition conjpart decodepart encodepart fibonacci, , , , , , , , , , , [

firstpart graycode inttovec lastpart multinomial nextpart numbcomb numbcomp numbpart, , , , , , , , ,

numbperm partition permute powerset prevpart randcomb randpart randperm setpartition stirling1, , , , , , , , , , stirling2 subsets vectoint, , ]

To determine the number of permutations of

n

objects taken

r

at a time, use function

numbperm(n,r)

, for example, P(3,2) is calculated as:

> numbperm(3,2);

Alternatively, you can use the definition: P(3,2) = 3!/(3-2)!:

> 3!/(3-2)!;

If you want to see the permutations of 3 numbers taken 2 at a time use function

permute

> permute(3,2);

[[1 2, ],[1 3, ],[2 1, ],[2 3, ],[3 1, ],[3 2, ]]

Function permute takes permutations of size 2 out of the numbers [1,2,3], as shown above.

You can define the elements that need to be permutated by defining a list of objects as the first argument of function

permute

, as in the following two examples:

> permute([A,B,C],2);#Permutations of 3 variables taken 2 at a time [[A B, ],[A C, ],[B A, ],[B C, ],[C A, ],[C B, ]]

> permute(["Jose","Joseph","Iosefa"],2);#Permutations of 3 names taken 2 at a time ["Jose" "Joseph", ],["Jose" "Iosefa", ],["Joseph" "Jose", ],["Joseph" "Iosefa", ],["Iosefa" "Jose", ], [

["Iosefa" "Joseph" ], ]

To determine the number of combinations of

n

objects taken

r

at a time use either function

numbcomb(n,r)

binomial(n,r)

-- as in binomial coefficient--. For example, to calculate

C(5,3)

you can use:

> numbcomb(5,3);

10 or,

> binomial(5,3);

Alternatively, you can use the definition

C(5,3) = 5!/(3!(5-3)!),

for this case

> 5!/(3!*(5-3)!);

To see combinations of

n

numbers taken

r

at a time, use function

choose(n,r)

. For example,

> choose(5,3);

[[1 2 3, , ],[1 2 4, , ],[1 2 5, , ],[1 3 4, , ],[1 3 5, , ],[1 4 5, , ],[2 3 4, , ],[2 3 5, , ],[2 4 5, , ],[3 4 5, , ]]

This application of function

choose

produces the 10 possible combinations of the numbers [1,2,3,4,5] taken 3 at a time. You can also define your own list from which the combinations are selected, for example:

> choose([A,B,C,D,E],3);#Combinations of 5 variables taken 3 at a time

[[A B C, , ],[A B D, , ],[A B E, , ],[A C D, , ],[A C E, , ],[A D E, , ],[B C D, , ],[B C E, , ],[B D E, , ],[C D E, , ]]

Conditional probability – Page 14

Combinatorial analysis functions with the HP 48 G or HP 49 G calculators. Selected

combinatorial analysis functions are available in the HP 48 G calculator by using the following keystrokes:

[MTH][NXT][PROB]

The same functions are available in the HP 49 G calculator by using:

[][MTH][NXT][PROB]

At this point you will have available the following combinatorial analysis and random-number functions:

[COMB][PERM][ ! ][RAND][ RDZ ]

Factorials. Function [ ! ] is used to calculate factorials, for example, to calculate 8!, use:

[8][ ! ] The result is 40320.

Function [ ! ] is also used to calculate the Gamma function of a real number by using the relationship,

Γ(x) = (x-1)!

Thus, to calculate Γ(2.5) = 1.5! use

[2][ . ][5][SPC][1][ - ][ ! ] The result is 1.32934038818.

Similarly, to calculate Γ(-1.8) = (-2.8)! use

[1][ . ][8][+/-][SPC][1][ - ][ ! ] The result is 3.18808591111.

Permutations. Function [PERM] is used to calculate the permutations of n objects taken r at a time. The value of n must be entered first followed by r. For example, to calculate P(5,3) use:

[5][SPC][3][PERM]

The result is 60.

In document Introduction to CEE 3030 page 1 (Page 25-46)