{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

531f10BAYES2

# 531f10BAYES2 - STAT 531 Bayesian Methods HM Kim Department...

This preview shows pages 1–11. Sign up to view the full content.

STAT 531: Bayesian Methods HM Kim Department of Mathematics and Statistics University of Calgary Fall 2010 1/43

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
MCMC Sampling from Posterior The conditional density of θ given data is given by g ( θ | data ) = g ( θ ) f ( data | θ ) R g ( θ ) f ( data | θ ) d θ the posterior is proportional to prior times likelihood . ignore the constants in the prior and likelihood that do not depend on the parameter, since multiplying either the prior or the likelihood by a constant won’t affect the results of Bayes’ theorem. Gibbs sampler and MH algorithm have been developed to draw an random sample from the posterior distribution, without having to completely evaluate it. We can approximate the posterior distribution to any accuracy we wish by taking a large enough random sample from it. Fall 2010 2/43
Applications generalized linear models missing data hierarchical models (multilevel model) Fall 2010 3/43

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Example: Logistic regression model Load the turnout dataset from the Zelig library. Implement a Bayesian logistic regression of vote on age and income using a random walk Metropolis-Hasting algorithm with a diffuse multivariate Normal prior. > library(Zelig) > data(turnout) > attach(turnout) > names(turnout) [1] "race" "age" "educate" "income" "vote" > turnout[1:5, ] race age educate income vote 1 white 60 14 3.3458 1 2 white 51 10 1.8561 0 3 white 24 12 0.6304 0 4 white 38 8 3.4183 1 5 white 25 12 2.7852 1 Fall 2010 4/43
vote = 1 , with probability p 0 , with probability 1 - p Odds p 1 - p are commonly used in the statistical analysis of binary outcomes since both probabilities p and 1 - p lies between 0 and 1; it follows that the odds lie between 0 and . when the probability is 0 . 5, the odds are 1 the odds are always bigger than the probability when the probability is small, the odds are very close to the probability p = 1 1+exp( - ( β 0 + β 1 age + β 2 income )) logit ( p ) = log p 1 - p = β 0 + β 1 age + β 2 income Fall 2010 5/43

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
> data(turnout) > y <- turnout\$vote > X <- cbind(1,turnout\$age, turnout\$income) > mle <- glm(vote~age+income, data=turnout, family=binomial) > mle Call: glm(formula = vote ~ age + income, family = binomial) Coefficients: (Intercept) age income -0.63912 0.01806 0.26606 Degrees of Freedom: 1999 Total (i.e. Null); 1997 Residual Null Deviance: 2267 Residual Deviance: 2113 AIC: 2119 Fall 2010 6/43
Data: ( y , x ) = ( vote , age , income ) p ( p | y , x ) " n i =1 p ( y i | β 0 , β 1 , x i ) # p ( β 0 , β 1 , β 2 ) = " n i =1 p y i i (1 - p i ) 1 - y i # p ( β 0 , β 1 , β 2 ) log p ( y , x , p ) = n i =1 log( Bernoulli ( y i , p i ))+ log p ( β 0 , β 1 , β 2 ) Fall 2010 7/43

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
, Steps First, use a multivariate Normal jumping distribution β 0 β 1 β 2 N 3 ( 0 , Σ) , Σ = δ 0 0 0 δ 0 0 0 δ to draw all the parameters at the same time. Keep track of your acceptance rate. Note any problems that you encountered. Next, draw each β separately with a univariate Normal jumping distribution given your draws of the other β s. That is, draw in the following order: 1 β ( t ) 0 | β ( t - 1) 1 , β ( t - 1) 2 2 β ( t ) 1 | β ( t ) 0 , β ( t - 1) 2 3 β ( t ) 2 | β ( t ) 0 , β ( t ) 1 Fall 2010 8/43
> library(mvtnorm) > library(coda) > log.post.func <- function(beta,...){ > pi.i <- 1/(1+exp(-X %*% t(beta))) > log.like <- sum(dbinom(y, size=1, prob=pi.i, log=T)) > log.prior <-dmvnorm(beta, mean=rep(prior.mean,k), sigma=diag(prior.var, k), log=T) > log.post <- log.like + log.prior > return(log.post) > } Fall 2010 9/43

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
> beta.update
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern