This preview shows pages 1–9. Sign up to view the full content.
STAT 531: Bayesian Methods
HM Kim
Department of Mathematics and Statistics
University of Calgary
Fall 2010
1/43
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document MCMC Sampling from Posterior
The conditional density of
θ
given data is given by
g
(
θ

data
)
=
g
(
θ
)
f
(
data

θ
)
R
g
(
θ
)
f
(
data

θ
)
d
θ
the
posterior
is proportional to
prior
times
likelihood
.
ignore the constants in the prior and likelihood that do not depend on
the parameter, since multiplying either the prior or the likelihood by a
constant won’t aﬀect the results of Bayes’ theorem.
Gibbs sampler and MH algorithm
have been developed to draw an random
sample from the posterior distribution, without having to completely
evaluate it. We can approximate the
posterior distribution
to any accuracy
we wish by taking a large enough random sample from it.
Fall 2010
2/43
Applications
generalized linear models
missing data
hierarchical models (multilevel model)
Fall 2010
3/43
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document Example: Logistic regression model
Load the
turnout
dataset from the
Zelig
library. Implement a Bayesian
logistic regression of vote on age and income using a random walk
MetropolisHasting algorithm with a diﬀuse multivariate Normal prior.
> library(Zelig)
> data(turnout)
> attach(turnout)
> names(turnout)
[1] "race"
"age"
"educate" "income"
"vote"
> turnout[1:5, ]
race age educate income vote
1 white
60
14 3.3458
1
2 white
51
10 1.8561
0
3 white
24
12 0.6304
0
4 white
38
8 3.4183
1
5 white
25
12 2.7852
1
Fall 2010
4/43
vote =
±
1
,
with probability
p
0
,
with probability 1

p
Odds
²
p
1

p
³
are commonly used in the statistical analysis of binary outcomes
since
both probabilities
p
and 1

p
lies between 0 and 1; it follows that the odds
lie between 0 and
∞
.
when the probability is 0
.
5, the odds are 1
the odds are always bigger than the probability
when the probability is small, the odds are very close to the probability
p
=
1
1+exp(

(
β
0
+
β
1
age
+
β
2
income
))
logit
(
p
) = log
´
p
1

p
µ
=
β
0
+
β
1
age
+
β
2
income
Fall 2010
5/43
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document > data(turnout)
> y
< turnout$vote
> X
< cbind(1,turnout$age, turnout$income)
> mle < glm(vote~age+income, data=turnout, family=binomial)
> mle
Call:
glm(formula = vote ~ age + income, family = binomial)
Coefficients:
(Intercept)
age
income
0.63912
0.01806
0.26606
Degrees of Freedom: 1999 Total (i.e. Null);
1997 Residual
Null Deviance:
2267
Residual Deviance: 2113
AIC: 2119
Fall 2010
6/43
Data: (
y
,
x
) = (
vote
,
age
,
income
)
p
(
p

y
,
x
)
∝
"
n
∏
i
=1
p
(
y
i

β
0
,
β
1
,
x
i
)
#
p
(
β
0
,
β
1
,
β
2
)
=
"
n
∏
i
=1
p
y
i
i
(1

p
i
)
1

y
i
#
p
(
β
0
,
β
1
,
β
2
)
log
p
(
y
,
x
,
p
) =
n
∑
i
=1
log(
Bernoulli
(
y
i
,
p
i
))+
log
p
(
β
0
,
β
1
,
β
2
)
Fall 2010
7/43
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document ,
→
Steps
First, use a multivariate Normal jumping distribution
β
0
β
1
β
2
∼
N
3
(
0
,
Σ)
,
Σ =
δ
0
0
0
δ
0
0
0
δ
to draw all the parameters at the same time. Keep track of your
acceptance rate. Note any problems that you encountered.
Next, draw each
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 02/04/2011 for the course STAT 531 taught by Professor Gaborlukacs during the Spring '11 term at Manitoba.
 Spring '11
 GABORLUKACS
 Statistics

Click to edit the document details