This preview shows pages 1–3. Sign up to view the full content.
IE 111
Bayes Theorem and the Rule of Total Probability
Note Set #4
Bayes Theorem
was first published in 1763 by the Reverend Thomas Bayes.
Bayes theorem is quite useful for solving
a particular
type of problem.
To understand
Bayes Theorem, we must first learn "the rule of total probability".
It can be thought of
as a way to “update” probability estimates given new information.
It is the foundation
for a whole branch of statistics called “Bayesian statistics” in which new data is used to
Update estimates of models of prior knowledge.
It is interesting because it combines
“prior knowledge” with data rather than relying only on the data.
The Rule of Total Probability
Consider the case shown in the Venn Diagram below.
In this case we have events E
1
,
E
2
, .
.. , E
k
which are
mutually exclusive and exhaustive
.
Exhaustive means that:
{E
1
∪
E
2
∪
.....
∪
E
k
} = S
The
rule of total probability
says the following:
P(A) = P(A
∩
E
1
) + P(A
∩
E
2
) + P(A
∩
E
3
) + .
.... + P(A
∩
E
k
)
= P(AE
1
)P(E
1
) + P(AE
2
)P(E
2
) + P(AE
3
)P(E
3
) + .
.... + P(AE
k
)P(E
k
)
Note that the second equation can be derived from the first using the definition of
conditional probability which says:
P(A
∩
E
1
) =
P(A E
1
)P(E
1
)
E
1
E
2
E
3
E
4
E
5
E
6
S
A
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThe rule of total probability allows us to use a “divide and conquer” approach, or
dividing a problem into several “cases” each of which is easy to solve, to finding
probabilities.
Example
In a wellshuffled deck, find the probability that the ace of spades is next to the king of
spades.
In this problem it seems easier to divide things up into three cases:
Event E1 = "the ace of spades is the top card in the deck"
Event E2 = "the ace of spades is the bottom card in the deck"
Event E3 = "the ace of spades is in the middle part of the deck"
Let event A = "the ace and king of spades are next to each other"
We have the following:
P(A)=P(AE1)P(E1)+ P(AE2)P(E2)+ P(AE3)P(E3)
=(1/51)(1/52) + (1/51)(1/52) + (2/51)(50/51)
We see that the rule of total probability allows us to calculate the probability of an
event by dividing the situation up into conditional events that are easier to calculate
individually.
This is precisely what we do in the case of a probability tree.
Useful result from the rule of total probability
This is the end of the preview. Sign up
to
access the rest of the document.
 Fall '07
 Storer

Click to edit the document details