If we can count the entire sample space
and each outcome is equally likely, then
Outcomes Possible All of Number A as Count That Outcomes of Number ) (A = P
If A and B are Mutually Exclusive,
)
P(
)
P(
)
(
P
A
B
=
A
+
B
)
P(
)
P(
)
P(
)
P(
A
B
=
A
+
B
−
A
B
The complement of an event are the
outcomes in the sample space that are not in the event.
Sometimes it is easier to compute Complement of is denoted
A
)
(
1
)
(
)
(
1
)
(
A
P
A
P
A
P
A
P
−
=
−
=
A
A
What is the probability that you reach in and
pull out a random marble and it is yellow?
(Trickier) If you pick two marbles at random,
what is the probability that one is yellow and one is red.
Three Urns
Urn A contains 4 white and 3 red marbles, urn
B contains 2 white and 5 red marbles, and urn C contains 2 white and 6 red marbles. A marble is to be selected from each one of the three
urns. What is the probability that the three selected are of the same color?
Total 28 marbles 8 yellow. 20 red.
A bin of machine parts contains 10 percent defectives. A random sample of three is
drawn from the bin with replacement. What is the probability of at least one defective in the sample?
We looked four distributions:
◦ Gaussian (or Normal) distribution
◦ Binomial Distribution
◦ Poisson Distribution
◦ Uniform Distribution
Distributions nicely lay out the probability of
all possible outcomes in a functional form.
Some natural (or unnatural) phenomenon's
yield random variables that behave
Any function, f, can be a probability
distribution function (as long as the area adds up to one).
Often times the probability of an event
changes once we can find out about some related event.
For example,
◦ Probability(Rain Tomorrow)
◦ Probability of (Rain Tomorrow, given that it is cloudy today)
For example,
a) Probability(Rain Tomorrow)
b) Probability of (Rain Tomorrow, given that it is cloudy today)
Since (a) is not same as (b)
In this case, we will say that the event
“cloudy today” is not independent of event “rain tomorrow.”
We write this as follows:
Probability(Rain | Cloudy)
Read as:
Conditional probability calculations are very
helpful.
Often times, when we want to know
probability of an event A, we face two situations:
◦ We want to know before A happens.
We hit a black jack It will rain today
◦ A is unobservable.
There is cheating on an assignments The email is spam
So event A is either unobservable or in the
future.
To better calculate the probability of A, we
try to observe some events that can influence the probability of A.
Example:
◦ Weatherman can look at today’s weather patterns
◦ A detective can look at the crime scene evidence
On the other hand, if we can observe both
event A and event B, conditional probability laws can tell us weather they are related or not.
Two events E and F in the same sample
space.
First conditional probability law.
OR
)
(
)
&
(
)
|
(
F
P
F
E
P
F
E
P
=
)
(
)
|
(
)
&
(
E
F
P
E
F
P
F
P
=
Rolling the die
◦ Given that you rolled an odd number
Rolling two dice
Given that one of the dice is a 3.
Top card from a deck of cards Given that you have a red card.
What is the probability that you have the
Toss a coin twice
Given that first toss was heads.
What is the probability of second toss being
)
(
)
(
)
|
(
)
|
(
B
P
A
P
A
B
P
B
A
P
=
) ( ) | ( ) ( ) | ( ) (B P B A P A P B AC P AC P = + School has 100 students 60 boys and 40 girls 70% of the boys wear shorts 30% wear pants. Half of girls wear shorts, half of girls wear
skirts.
What is the probability of seeing a girl given a
Breast Cancer Screening
Mammograms detect 90% of breast cancers
9.6% of women with no cancer have a
positive result
1% of women have breast cancer
What is the probability of having breast
Breast Cancer Screening
Mammograms detect 98% of breast cancers
7.2% of women with no cancer have a
positive result
1% of women have breast cancer
What is the probability of having breast
In Southern California, it only rains 5% of
the time.
When it rains, the weatherman is correct
90% of the time.
When it doesn’t rain, the weatherman is
says it will rain 10% of the time.
The weatherman predicted rain. What is
the probability it will actually rain tomorrow?
You are a contestant on a game
show. To decide your prize, you are asked to choose one of three doors (door 1, 2 or 3).
Behind one of the doors is a car (good prize). Behind two of the doors is a goat (bad prize).
You choose door 3. The game
show host, who knows what is behind each door, opens door 1 to reveal a goat. He then gives you the option to change your original selection to door 2.”
Is it better to stick with door 3,
change your selection to door 2 or does it not matter?
When problem first appeared in Parade,
approximately 10,000 readers, including 1,000 PhDs, wrote claiming the solution was wrong.
In a study of 228 subjects, only 13% chose to
switch.
Has inspired some studies related to (limitation
of) human and primate brain function dealing with cognitive dissonance and biases.
Note that:
◦ Monty must open a door that reveals a goat
◦ Monty cannot open the door you selected That is:
◦ If you initially selected a door concealing a goat, then there is only one door Monty can open.
Lets solve the Monty Hall problem using a Bayesian analysis
Let Ci represent the state that the car lies behind Door , i i [1,2,3].
Let represent the event that Monty opens door , [1,2,3], revealing a goat.
i
M i i
We seek p(C2 |M1) = p(M1|C2)p(C2)
p(M1)
More specifically, we want to compare the three probabilities.
Probability that car lies behind door 1, 2 or 3 given that Monty has opened door 2.
• Drawers A and B
• A has 15 good coin and 2 fake coin • B has 6 good coins and 4 fake coin • Open a drawer and pick a coin.
• Chance of getting fake one.
• Given the fake one is selected, what is the
)
(
)
(
)
&
(
A
B
P
A
P
B
P
=
)
(
)
|
(
A
B
P
A
P
=
Two dice rolls
In a deck of cards, the suit and the number Two coin tosses
Coin came up heads, and whether it will
Now, we can use information we have
learned so far to build an automatic
classifier that can automatically assign a
probability to a scenario given the evidence and past history.
For the new example (red, , small)
P($| item) = P(red|$)*P(|$)*P(small|$)*P($) / P(item) = 2/3 * 1/3 * 2/3 * 3/5 = 0.08
P(# | item) = P(red|#)*P(| #)*P(small| #)*P(#) / P(item) = 1/2 * 1/2 * 1/2 * 2/5 = 0.05
Only comparing numerator is enough. Ignore the denominator, P(Item), as it is same in both cases.
Day Outlook Water Temperature Pollutants in Water Wind Fish Present
Day1 Sunny Hot High Weak No Day2 Sunny Hot High Strong No
Day3 Overcast Hot High Weak Yes
Day4 Rain Mild High Weak Yes
Day5 Rain Cool Normal Weak Yes
Day6 Rain Cool Normal Strong No
Day7 Overcast Cool Normal Strong Yes
Day8 Sunny Mild High Weak No
Day9 Sunny Cool Normal Weak Yes
Day10 Rain Mild Normal Weak Yes
Day11 Sunny Mild Normal Strong Yes
Day12 Overcast Mild High Strong Yes
Day13 Overcast Hot Normal Weak Yes
Now, we can use information we have
learned so far to build an automatic classifier such as an algorithm that
Lets look a real world (early) algorithm that automatically filters out the SPAM mail.
What I am most interested in is:
◦ Probability (Spam | Words in my email)
These surely don’t sound independent events
◦ Probability (Spam | {student, grades, class})
What I am most interested in is:
◦ Probability (Spam | Words in my email)
These surely don’t sound independent
events
◦ Probability (Spam | {student, grades, class})
Want to know:
Spam Not Spam Degree 50 120 Pharmacy 50 5 Nigeria 90 3 Prince 80 2 Student 20 70 ! 90 30 soon 50 70 Get 120 180
Let’s assume that 100 volunteers helped us label total of 10000 emails.
Total of 3000 normal mails and 7000 spam mail are gathered.
“Get Pharmacy Degree Soon!”
Is it spam??
Prob (“pharmacy” | spam) = 50/7000
Because in spam, there are 50 occurrences of “pharmacy” in 7000 emails.
