Northwestern University
Marciano Siniscalchi
Fall 2009
Econ 3310
BAYESIAN INFERENCE AND THE FREQUENTIST VIEW
1.
Introduction
This lecture has two main objectives. First, it will briefly introduce you to Bayes’ Rule, which
plays an important role in the subjectivist approach to statistical inference, as well as in economics.
Second, it will illustrate that, in a precise sense, the subjectivist approach generalizes the frequentist
view that probability equals longrun frequency; while you might not care so much about this fact
per se, it is handy to know that it is still OK, in a precise sense, to say that “the probability of
Heads in a coin toss is
1
2
”, even if we subscribe to the subjectivist view.
2.
Bayes’ Rule and the Frequency View
Bayes’ Rule is, loosely speaking, a (rather obvious) way of flipping the events
E
and
F
in the
conditional probability
P
(
E

F
). Formally, suppose that
P
(
E
)
>
0 and
P
(
F
)
∈
(0
,
1): then
(1)
P
(
F

E
) =
P
(
E

F
)
P
(
F
)
P
(
E

F
)
P
(
F
) +
P
(
E

F
c
)
P
(
F
c
)
.
To prove Eq. (1), and to understand why it’s useful, note that, by definition,
P
(
F

E
) =
P
(
F
∩
E
)
P
(
E
)
(so we need
P
(
E
)
>
0 for this to make sense). Bayes’ Rule is useful precisely when
P
(
F
∩
E
) and
P
(
E
) are not known.
We know that, by definition,
P
(
E

F
) =
P
(
E
∩
F
)
P
(
F
)
(so we need
P
(
F
)
>
0);
hence,
P
(
F
∩
E
) =
P
(
E
∩
F
) =
P
(
E

F
)
P
(
F
), which explains what’s in the numerator of Eq.
(1). For the denominator, note that
E
= (
E
∩
F
)
∪
(
E
∩
F
c
), so
P
(
E
) =
P
(
E
∩
F
) +
P
(
E
∩
F
c
).
We already know that
P
(
E
∩
F
) =
P
(
E

F
)
P
(
F
): similarly,
P
(
E
∩
F
c
) =
P
(
E

F
c
)
P
(
F
c
), which
explains why we also need
P
(
F
c
)
>
0, or
P
(
F
)
<
1.
Bayes’ Rule is used in virtually all interesting economic applications where agents make decisions
at different points in time, so we will have ample time to appreciate its significance. Moreover, it
suggests a rather natural approach to statistical inference—one that is intimately connected with
the subjective view of probability.
1
We discuss this briefly in the next section.
Perhaps most importantly, this approach provides a way to
reconcile the frequentist and Bayesian
views of probability
. This can be shown formally, but it requires rather formidable mathematics.
Thus, I’ll try to provide you with an intuition of what’s going on, by relying on an analogy.
Consider an urn that contains
N
black balls and 100

N
white balls, with 0
≤
N
≤
100. The
urn is sampled with replacement: that is, a ball is drawn, its color is observed, then the ball is
replaced in the urn. Since we can draw any number of balls from the urn (with replacement), this
is precisely the kind of situation where the frequentist approach makes sense: the probability of
drawing a black ball is the longrun frequency of black draws, as we keep sampling from the urn.
1
It is possible to extend our definition of forecast systems and admissibility to derive the updating formula, and
hence Bayes’ Rule, as a consequence of the assumption that people avoid inadmissible forecasts.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '09
 Marciano
 Economics, Conditional Probability, black balls, Classical Statistician

Click to edit the document details