Chapter 4
Basic Probability and Probability
Distributions
Probability Terminology
•
Classical Interpretation
: Notion of probability based
on equal likelihood of individual possibilities (coin toss
has 1/2 chance of Heads, card draw has 4/52 chance of
an Ace). Origins in games of chance.
–
Outcome
: Distinct result
of random process (
N
= # outcomes)
–
Event
: Collection of outcomes (
N
e
= # of outcomes in event)
–
Probability of event
E
:
P
(event
E
) =
N
e
/
N
•
Relative Frequency Interpretation
: If an experiment
were conducted repeatedly, what fraction of time would
event of interest occur (based on empirical observation)
•
Subjective Interpretation
: Personal view (possibly
based on external info) of how likely a oneshot
experiment will end in event of interest
Obtaining Event Probabilities
•
Classical Approach
–
List all
N
possible outcomes of experiment
–
List all
N
e
outcomes corresponding to event of interest (
E
)
–
P
(event
E
) =
N
e
/
N
•
Relative Frequency Approach
–
Define event of interest
–
Conduct experiment repeatedly (often using computer)
–
Measure the fraction of time event
E
occurs
•
Subjective Approach
–
Obtain as much information on process as possible
–
Consider different outcomes and their likelihood
–
When possible, monitor your skill (e.g. stocks, weather)
Basic Probability and Rules
•
A,B
≡
Events of interest
•
P
(
A
),
P
(
B
)
≡
Event probabilities
•
Union
: Event
either
A
or
B
occurs
(
A
∪
B
)
•
Mutually Exclusive
:
A, B
cannot occur at same time
–
If
A,B
are mutually exclusive:
P
(either
A
or
B
) =
P
(
A
) +
P
(
B
)
•
Complement of
A
: Event that
A
does not occur (
Ā
)
–
P
(
Ā
) = 1
P
(
A
)
That is:
P
(
A
) +
P
(
Ā
) = 1
•
Intersection
: Event
both
A
and
B
occur
(
A
∩
B
or
AB
)
•
P
(
A
∪
B
) =
P
(
A
) +
P
(
B
) 
P
(
AB
)
Conditional Probability and Independence
•
Unconditional/Marginal Probability
: Frequency which
event occurs in general (given no additional info).
P
(
A
)
•
Conditional Probability
: Probability an event (
A
) occurs
given
knowledge another event (
B
) has occurred.
P
(
A

B
)
•
Independent Events
: Events whose unconditional and
conditional (given the other) probabilities are the same
(
)
(
)
(

)
(
)
(
)
(
)
(
)
(

)
(
)
(
)
(
)
(
)
(
)
(

)
(
)
(

)
,
independent
(
)
(

) &
(
)
(

)
P A
B
P AB
P A B
P B
P B
P A
B
P AB
P B
A
P A
P A
P A
B
P AB
P A P B
A
P B P A B
A B
P A
P A B
P B
P B
A
∩
=
=
∩
=
=
∩
=
=
=
⇒
=
=
John Snow London Cholera Death Study
•
2 Water Companies (Let
D
be the event of death):
–
Southwark&Vauxhall (
S
): 264913 customers, 3702 deaths
–
Lambeth (
L
): 171363 customers, 407 deaths
–
Overall: 436276 customers, 4109 deaths
people)
10000
per
(24
0024
.
171363
407
)

(
people)
10000
per
(140
0140
.
264913
3702
)

(
people)
10000
per
(94
0094
.
436276
4109
)
(
=
=
=
=
=
=
L
D
P
S
D
P
D
P
Note that probability of death is almost 6 times higher for S&V
customers than Lambeth customers (was important in showing how
cholera spread)
John Snow London Cholera Death Study
Cholera
Death
Water
Company
Yes
No
Total
S&V
3702
(.0085)
261211
(.5987)
264913
(.6072)
Lambeth
407
(.0009)
170956
(.3919)
171363
(.3928)
Total
4109
(.0094)
432167
(.9906)
436276
(1.0000)
Contingency Table with joint probabilities (in body of table)
and marginal probabilities (on edge of table)
John Snow London Cholera Death Study
WaterUser
S&V
L
.6072
.3928
Company
Death
D
(.0085)
.0140
.9860
D
C
(.5987)
.0024
.9976
D
(.0009)
D
C
(.3919)
Tree Diagram obtaining joint probabilities by multiplication rule
Bayes’s Rule  Updating Probabilities
• Let
A
1
,…,
A
k
be a set of events that
partition
a sample