Probability theory is nothing but common sense reduced to calculation.- Laplace
When you think about probability, the first things that might come to mind are coin flips (“there’s a50-50chance of landing heads”), weather reports (“there’s a20% chance of rain today”), and politi- cal polls (“the incumbent candidate is leading the challenger53% to 47%”). When we speak about probability, we speak about a percent- age chance (0%-100%) for something to happen, although we often write the percentage as a decimal number, between0and1. If the probability of an event is0then it is the same as saying thatyou are certain that the event will never happen. If the probability is1thenyou are certain that itwillhappen. Life is full of uncertainty, so we assign a number somewhere between0and1to describe our state of knowl- edge of the certainty of an event. The probability that you will get
36 s tat i s t i c a l i n f e r e n c e f o r e v e r yo n e
struck by lightning sometime in your life isp = 0.0002, or1out of 5000. Statistical inference is simply the inference in the presence of uncertainty. We try to make the best decisions we can, given incom-
plete information. In this book, our approach is to
determine, for each problem, what degree of confidence we have in all of the possible outcomes. The approach of statistical inference covered in this book is about the procedure of most rationally assign- ing various degrees of confidence (which we callprobability) to the possible outcomes of some process using all the objectively available data.
One can think of probability as a mathematical short-hand for the common sense statements we make in the presence of uncertainty. This short-hand, however, becomes a very powerful tool when our common sense is not up to the task of handling the complexity of a problem. Thus, we will start with examples that will perhaps seem simple and obvious, and move to examples where it would be a challenge for you to determine the answer without the power of statistical inference.
Let’s walk through a simple set of examples to establish the nota- tion, and some of the basic mathematical properties of probabilities.
Card Game
A simple game can be used to explore all of the facets of probability. We use a standard set of cards (Figure1.1) as the starting point, and
use this system to set up the intuition, as well as the mathematical notation and structure for approaching probability problems.
Figure1.1: Standard52-card deck. 13cards of each suit, labeled Spades, Clubs, Diamonds, Hearts.
We start with what I simply call thesimple card game5
, which goes 5
In this description of the game, we do not reshuffle after each draw. The differences between this non- reshuffled version and the one with reshuffling will be explored later, but will only change some small details in the outcomes.
i n t ro du c t i o n t o p ro b a b i l i t y 37
like:
simple card game ≡
From a standard initially shuffled deck, we draw one card, note what card it is and set it aside. We then draw another card, note what card it is and set it aside. Continue until there are no more cards, noting each one along the way.
(1.1)
There are certain principles that guide us in developing the math- ematical structure of probability. We start with some common sense notions, written in English, and then write them as general princi- ples. These principles, then, constrain our mathematics so that we can apply the ideasquantitatively.
When asked “what is the probability of drawing a red on the first draw?” you would generally say50-50, or50%, or equivalently written as a probability,P(R1) = 0.5. The reason for this is that
we are completely ignorant of the initial conditions of the deck (i.e. where each card is located in the deck after the initial shuffling). Given this level of (or lack of) knowledge, we could swap the colors of the two suits and we would have an equivalent state of knowledge - the problem would be identical. We will keep coming back to this concept, but in general:
Principle of Knowledge and ProbabilityEquivalent states of Principle of Knowledge and
ProbabilityEquivalent states of knowledge must yield equivalent probability assignments.
knowledge must yield equivalent probability assignments. Because of this principle, we are led to the conclusion that
P(R1) =P(B1)
whereR1represents the statement “a red on the first draw” andB1
represents “a black on the first draw.” Because these are the only two options, and they are mutually exclusive, then they must add up to1. Thus we have
P(R1) =1−P(B1)
which leads directly to our original assignment
P(R1) =P(B1) =0.5
Mutually ExclusiveIf I have a list ofmutually exclusiveevents, then Mutually ExclusiveIf I have a
list ofmutually exclusiveevents, then that means that only one of them could possibly be true. Examples includes the heads and tails outcomes of coins, or the values of standard6-sided dice. In terms of probability, this means that, for events A and B,
P(AandB) =0.
that means that only one of them could possibly be true. Example events include flipping heads or tails with a coins, rolling a1,2, 3,4,5or6on dice, or drawing a red or black card from a deck of
38 s tat i s t i c a l i n f e r e n c e f o r e v e r yo n e
cards. In terms of probability, this means that, for events A and B, P(AandB) =0.
Non Mutually ExclusiveIf I have a list of events that arenot mutu- Non Mutually ExclusiveIf I have
a list of events that arenot mutually exclusive, then it is possible for two or more to be true. Examples include weather with rain and clouds or holding the high and the low card in a poker game.
ally exclusive, then it is possible for two or more to be true. Examples include weather with rain and clouds or holding the high and the low card in a poker game.
Now, this was a long-winded way to get to the answer we knew from the start, but that is how it must begin. We start working things out where our common sense is strong, so that we know we are proceeding correctly. We can then, confidently, apply the tools in places where our common sense is not strong.
In summary, with no more information than that there are two mutually exclusive possibilities, we assign equal probability to both. If there are only two colors of cards in equal amounts, red and black, then the probability of drawing a red isP(R1) =0.5 and the probabil-
ity for a black is the same,P(B1) =0.5.