Probability
And of all axioms this shall win the prize.
‘Tis better to be fortunate than wise.
John Webster Men’s judgements are a parcel of their fortunes.
W. Shakespeare, Antony and Cleopatra
1.1 Notation and Experiments
In the course of everyday life, we become familiar with chance and probability in various contexts. We express our ideas and assessments in many ways, such as:
It will very likely rain.
It is almost impossible to hole this putt.
That battery may work for a few more hours.
Someone will win the lottery, but it is most unlikely to be one of us.
It is about a 50-50 chance whether share prices will rise or fall today.
You may care to amuse yourself by noting more such judgments of uncertainty in what you say and in the press.
This large range of synonyms, similes, and modes of expression may be aesthetically pleasing in speech and literature, but we need to become much more precise in our thoughts and terms. To aid clarity, we make the following.
(1) Definition Any well-defined procedure or chain of circumstances is called an experi-ment. The end results or occurrences are called the outcomes of the experiexperi-ment. The set of possible outcomes is called the sample space (or space of outcomes) and is denoted by
the Greek letter.
In cases of interest, we cannot predict with certainty how the experiment will turn out, rather we can only list the collection of possible outcomes. Thus, for example:
Experiment Possible outcomes (a) Roll a die One of the faces (b) Flip a coin Head or tail
24
1.1 Notation and Experiments 25 (c) Buy a lottery ticket Win a prize, or not
(d) Deal a bridge hand All possible arrangements of 52 cards into four equal parts
(e) Run a horse race Any ordering of the runners
Typically, probability statements do not refer to individual outcomes in the sample space; instead, they tend to embrace collections of outcomes or subsets of. Here are some examples:
Experiment Set of outcomes of interest (a) Deal a poker hand Have at least a pair
(b) Buy a share option Be in the money at the exercise date (c) Telephone a call centre Get through to a human in less than 1 hour (d) Buy a car It runs without major defects for a whole year
(e) Get married Stay married
Clearly this is another list that you could extend without bound. The point is that in typical probability statements of the form
the probability of A is p, which we also write as
P( A)= p,
the symbol A represents groups of outcomes of the kind exemplified above. Furthermore, we concluded in Chapter 0 that the probability p should be a number lying between 0 and 1 inclusive. A glance at the Appendix to that chapter makes it clear that P(.) is in fact simply a function on these subsets of, which takes values in [0, 1]. We make all this formal; thus:
(2) Definition An event A is a subset of the sample space.
(3) Definition Probability is a function, defined on events in, that takes values in [0, 1]. The probability of the event A is denoted by P(A).
(4) Example: Two Dice Suppose the experiment in question is rolling two dice. (Note that in this book a “die” is a cube, conventionally numbered from 1 to 6, unless otherwise stated.) Here are some events:
(a) A= Their sum is 7.
(b) B= The first die shows a larger number than the second.
(c) C = They showthe same.
We may alternatively display these events as a list of their component outcomes, so A= {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)},
and so on, but this is often very tedious.
Of course this experiment has the type of symmetry we discussed in Section 0.3, so we can also compute
P( A)= P(C) = 1
6, and P(B) = 15 36 = 5
12.
s
(5) Example: Darts A dart is thrown to hit a chess board at random. Here, “at random”
clearly means that it is equally likely to hit any point of the board. Possible events are:
(a) A= It hits a white square.
(b) B= It lands within one knight’s move of a corner square.
This problem is also symmetrical in the sense of Section 0.3, and so we calculate P( A)= 12
and P(B)= 18.
s
Of course, the brief definitions above raise more questions than they answer. How should the probability function behave when dealing with two or more events? How can it be extended to cope with changes in the conditions of the experiment? Many more such questions could be posed. It is clear that the brief summary above calls for much explanation and elaboration. In the next fewsections, we provide a fewsimple rules (or axioms) that define the properties of events and their probabilities. This choice of rules is guided by our experience of real events and their likelihoods, but our experience and intuition cannot prove that these rules are true or say what probability “really” is. What we can say is that, starting with these rules, we can derive a theory that provides an elegant and accurate description of many random phenomena, ranging from the behaviour of queues in supermarkets to the behaviour of nuclear reactors.
1.2 Events
Let us summarise our progress to date. Suppose we are considering some experiment such as tossing a coin. To say that the experiment is well defined means that we can list all the possible outcomes. In the case of a tossed coin, the list reads: (head, tail). For a general (unspecified) experiment, any particular outcome is denoted byω; the collection of all outcomes is called the sample space and is denoted by.
Any specified collection of outcomes in is called an event. Upper case letters such as A, B, and C are used to denote events; these may have suffices or other adornments such as Ai, ¯B, C∗, and so on. If the outcome of the experiment isω and ω ∈ A, then A is said to occur. The set of outcomes not in A is called the complement of A and is denoted by Ac.
In particular, the event that contains all possible outcomes is the certain event and is denoted by . Also, the event containing no outcomes is the impossible event and is denoted byφ. Obviously, φ = c. What we said in Chapter 0 makes it natural to insist that P() = 1 and that P(φ) = 0.
It is also clear from our previous discussions that the whole point of probability is to say howlikely the various outcomes are, either individually or, more usually, collectively in events. Here are some more examples to illustrate this.
1.2 Events 27 Example: Opinion Poll Suppose n people are picked at random and interrogated as to their opinion (like or dislike or do not care) about a brand of toothpaste. Here, the sample space is all collections of three integers (x, y, z) such that x + y + z = n, where x is the number that like it, y the number that dislike it, and z the number that do not care.
Here an event of interest is
A≡ more like it than dislike it
which comprises all the triples (x, y, z) with x > y. Another event that may worry the manufacturers is
B≡ the majority of people do not care,
which comprises all triples (x, y, z), such that x + y < z.
s
Example: Picking a Lottery Number In one conventional lottery, entries and the drawchoose six numbers from 49. The sample space is therefore all sextuples{x1, . . . , x6}, where all the entries are between 1 and 49, and no two are equal. The principal event of
interest is that this is the same as your choice.
s
Example: Coins If a coin is tossed once, then = {head, tail}. In line with the notation above, we usually write
= {H, T }.
The event that the coin shows a head should strictly be denoted by{H}, but in common with most other writers we omit the braces in this case, and denote a head by H . Obviously, Hc= T and Tc = H.
Likewise, if a coin is tossed twice, then
= {H H, H T, T H, T T },
and so on. This experiment is performed even more often in probability textbooks than it
is in real life.
s
Because events are sets, we use the usual notation for combining them; thus:
A∩ B denotes outcomes in both A and B; their intersection.
A∪ B denotes outcomes in either A or B or both; their union.
A B denotes outcomes in either A or B, but not both; their symmetric difference.
A\B denotes outcomes in A that are not in B; their difference.
∞ j=1
Aj denotes outcomes that are in at least one of the countable collection ( Aj; j ≥ 1);
their countable union.
[Countable sets are in one–one correspondence with a subset of the positive integers.]
A⊆ B denotes that every outcome in A is also in B; this is inclusion.
A= {ω1, ω2, ω3, . . . , ωn} denotes that the event A consists of the outcomes ω1, . . . , ωn
A× B denotes the product of A and B; that is, the set of all ordered pairs (ωa, ωb), where ωa ∈ A and ωb ∈ B.
Figure 1.1 The interior of the rectangle represents the sample space, and the interior of the circle represents an event A. The pointω represents an outcome in the event Ac. The diagram
clearly illustrates the identities Ac∪ A = and \A = Ac.
These methods of combining events give rise to many equivalent ways of denoting an event. Some of the more useful identities for any events A and B are:
A B = (A ∩ Bc)∪ (Ac∩ B) (1)
A= (A ∩ B) ∪ (A ∩ Bc) (2)
A\B = A ∩ Bc (3)
Ac= \A (4)
A∩ Ac= φ (5)
A∪ Ac= .
(6)
These identities are easily verified by checking that every element of the left-hand side is included in the right-hand side, and vice versa. You should do this.
Such relationships are often conveniently represented by simple diagrams. We illustrate this by providing some basic examples in Figures 1.1 and 1.2. Similar relationships hold
Figure 1.2 The interior of the smaller circle represents the event A; the interior of the larger circle represents the event B. The diagram illustrates numerous simple relationships; for example,
the region common to both circles is A∩ B ≡ (Ac∪ Bc)c. For another example, observe that A B = Ac Bc.
1.2 Events 29 between combinations of three or more events and some of these are given in the problems at the end of this chapter.
When A∩ B = φ we say that A and B are disjoint (or mutually exclusive).
(7) Example A die is rolled. The outcome is one of the integers from 1 to 6. We may denote these by {ω1, ω2, ω3, ω4, ω5, ω6}, or more directly by {1, 2, 3, 4, 5, 6}, as w e choose. Define:
A the event that the outcome is even, B the event that the outcome is odd, C the event that the outcome is prime,
D the event that the outcome is perfect (a perfect number is the sum of its prime factors).
Then the above notation compactly expresses obvious statements about these events. For example:
A∩ B = φ A∪ B =
A∩ D = {ω6} C\A = B\{ω1}
and so on.
s
It is natural and often useful to consider the number of outcomes in an event A. This is denoted by|A|, and is called the size or cardinality of A.
It is straightforward to see, by counting the elements on each side, that size has the following properties.
If A and B are disjoint, then
|A ∪ B| = |A| + |B|, (8)
and more generally, for any A and B
|A ∪ B| + |A ∩ B| = |A| + |B|.
(9)
If A⊆ B, then
|A| ≤ |B|.
(10)
For the product A× B,
|A × B| = |A||B|.
(11)
Finally,
|φ| = 0.
(12)
(13) Example The Shelmikedmu are an elusive and nomadic tribe whose members are unusually heterogeneous in respect of hair and eye colour, and skull shape. A persistent anthropologist establishes the following facts:
(i) 75% have dark hair, the rest have fair hair.
(ii) 80% have brown eyes; the rest have blue eyes.
(iii) No narrow-headed person has fair hair and blue eyes.
(iv) The proportion of blue-eyed broad-headed tribespeople is the same as the proportion of blue-eyed narrow-headed tribespeople.
(v) Those who are blue-eyed and broad-headed are fair-haired or dark-haired in equal proportion.
(vi) Half the tribe is dark-haired and broad-headed.
(vii) The proportion who are brown-eyed, fair-haired, and broad-headed is equal to the proportion who are brown eyed, dark-haired, and narrow-headed.
The anthropologist also finds n, the proportion of the tribe who are narrow-headed, but unfortunately this information is lost in a clash with a crocodile on the difficult journey home. Is another research grant and field trip required to find n? Fortunately, not if the anthropologist uses set theory. Let
B be the set of those with blue eyes C be the set of those with narrow heads D be the set of those with dark hair
Then the division of the tribe into its heterogeneous sets can be represented by Figure 1.3.
This type of representation of sets and their relationships is known as a Venn diagram.
The proportion of the population in each set is denoted by the lower case letter in each compartment, so
a= |Bc∩ Cc∩ Dc|/||, b= |B ∩ Cc∩ Dc|/||,
Figure 1.3 Here the interior of the large circle represents the entire tribe, and the interior of the small circle represents those with narrow heads. The part to the right of the vertical line represents
those with dark hair, and the part above the horizontal line represents those with blue eyes. Thus, the shaded quadrant represents those with blue eyes, narrow heads, and fair hair; as it happens,
this set is empty by (iii). That is to say B∩ C ∩ Dc= φ, and so g = 0.
1.2 Events 31 and so on. The required proportion having narrowheads is
n= |C|/|| = e + f + g + h
and, of course, a+ b + c + d + e + f + g + h = 1. The information in (i)–(vii), which survived the crocodile, yields the following relationships:
c+ d + e + f = 0.75 (i)
a+ d + e + h = 0.8 (ii)
g= 0 (iii)
f + g = b + c (iv)
b= c (v)
c+ d = 0.5 (vi)
a= e (vii)
The anthropologist (who has a pretty competent knowledge of algebra) solves this set of equations to find that
n= e + f + g + h = e + f + h = 0.15 + 0.1 + 0.05 = 0.3
Thus, three-tenths of the tribe are narrow-headed.
s
This section concludes with a technical note (which you may omit on a first reading). We have noted that events are subsets of. A natural question is, which subsets of are entitled to be called events?
It seems obvious that if A and B are events, then A∪ B, Ac, A ∩ B, and so on should also be entitled to be events. This is a bit vague; to be precise, we say that a subset A of
can be an event if it belongs to a collection F of subsets of , obeying the following three rules:
∈ F;
(14)
if A∈ F then Ac∈ F;
(15)
if Aj ∈ F for j ≥ 1, then ∞
j=1
Aj ∈ F.
(16)
The collectionF is called an event space or a σ-field.
Notice that using (1)–(6) shows that if A and B are inF, then so are A\B, A B and A∩ B.
(17) Example (7) Revisited It is easy for you to check that {φ, A, B, } is an event space, and{φ, A ∪ C, B\C, } is an event space. However, {φ, A, } and {φ, A, B, D, }
are not event spaces.
s
In general, if is finite, it is quite usual to take F to be the collection of all subsets of , which is clearly an event space. If is infinite, then this collection is sometimes too big to be useful, and some smaller collection of subsets is required.