ISyE8843A, Brani Vidakovic
Handout 5
1
Priors
A prior is a sword and Achilles heel of Bayesian statistics. Priors are carriers of prior information that is
coherently incorporated via Bayes theorem to the inference. At the same time, parameters are unobservable,
and prior specification is subjective in nature. Subjectivity of specifying the prior is fundamental objection
of rabid frequentists to the Bayesian approach.
Frequentists, attacking Bayesians for subjectivity are not saints of objectivity themselves.
The very
elicitation of a model (likelihood) and loss function is highly subjective, and Bayesians merely divide the
necessary subjectivity to two sources  that from the model and from the prior. Since the prior and loss are
not separable
in decision theoretic statistical inference (Herman Rubin (1987)) it follows that a decision
theoretic frequentist and a Bayesian are equally subjective.
Figure 1: Importance of Priors
Being subjective for an engineer is not a bad thing! Being subjective does not mean being nonscientific,
as critics of Bayesian statistic often insinuate. On the contrary – vast amount scientific information coming
from theoretical and physical models is guiding specification of priors and merging such information with
the data for better inference. Examples are abundant and in this course you will see many instances. In
the last several decades Bayesian research also focused on priors that are uninformative and robust as an
answer to criticism that Bayesian inference is overly sensitive to the choice of a prior. We will cover several
such paradigms.
The nature and sources of priors carry deep philosophical load as well and are subject of discussion and
disagreements even among Bayesians.
In this handout we will discuss various priors. Some of them are historic (uniform priors of Laplace and
Bayes), some of them mathematically convenient (conjugate priors; Raiffa and Schlaifer, 1961), and some
robust and noninformative (Jefreys, improper, reference, etc. priors. We will talk as well about hierarchical
priors and priors over various domains (Dirichlet, Wavelets, etc).
1.1
Uniform Priors of Bayes and Laplace
In modern mathematical language, the Bayes’ essay dealt with the following problem.
[year 1764]
A billiard ball
W
is rolled on a line
[0
,
1]
,
with a uniform probability of stopping anywhere. It
stops at
p
. A second ball
O
is then rolled
n
times under the same conditions and
X
times stopped on the left
of
W
. Given
X
what can we say about
p
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
In particular, Bayes was interested in
P
(
a < p < b
)
for
0
≤
a < b
≤
1
.
In our notation the solution is
given by
B
e
(
X
+ 1
, n

X
+ 1)
as the posterior of Binomial/Uniform Bayesian model, but at that time the
Reverend had difficulties of evaluating
P
(
a < p < b
)
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '11
 VIDAKOVIC
 Bayesian probability, Bayesian statistics, Jeffreys, priors

Click to edit the document details