{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

531f10BAYES2 - STAT 531 Bayesian Methods HM Kim Department...

Info icon This preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
STAT 531: Bayesian Methods HM Kim Department of Mathematics and Statistics University of Calgary Fall 2010 1/43
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
MCMC Sampling from Posterior The conditional density of θ given data is given by g ( θ | data ) = g ( θ ) f ( data | θ ) R g ( θ ) f ( data | θ ) d θ the posterior is proportional to prior times likelihood . ignore the constants in the prior and likelihood that do not depend on the parameter, since multiplying either the prior or the likelihood by a constant won’t affect the results of Bayes’ theorem. Gibbs sampler and MH algorithm have been developed to draw an random sample from the posterior distribution, without having to completely evaluate it. We can approximate the posterior distribution to any accuracy we wish by taking a large enough random sample from it. Fall 2010 2/43
Image of page 2
Applications generalized linear models missing data hierarchical models (multilevel model) Fall 2010 3/43
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example: Logistic regression model Load the turnout dataset from the Zelig library. Implement a Bayesian logistic regression of vote on age and income using a random walk Metropolis-Hasting algorithm with a diffuse multivariate Normal prior. > library(Zelig) > data(turnout) > attach(turnout) > names(turnout) [1] "race" "age" "educate" "income" "vote" > turnout[1:5, ] race age educate income vote 1 white 60 14 3.3458 1 2 white 51 10 1.8561 0 3 white 24 12 0.6304 0 4 white 38 8 3.4183 1 5 white 25 12 2.7852 1 Fall 2010 4/43
Image of page 4
vote = 1 , with probability p 0 , with probability 1 - p Odds p 1 - p are commonly used in the statistical analysis of binary outcomes since both probabilities p and 1 - p lies between 0 and 1; it follows that the odds lie between 0 and . when the probability is 0 . 5, the odds are 1 the odds are always bigger than the probability when the probability is small, the odds are very close to the probability p = 1 1+exp( - ( β 0 + β 1 age + β 2 income )) logit ( p ) = log p 1 - p = β 0 + β 1 age + β 2 income Fall 2010 5/43
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
> data(turnout) > y <- turnout$vote > X <- cbind(1,turnout$age, turnout$income) > mle <- glm(vote~age+income, data=turnout, family=binomial) > mle Call: glm(formula = vote ~ age + income, family = binomial) Coefficients: (Intercept) age income -0.63912 0.01806 0.26606 Degrees of Freedom: 1999 Total (i.e. Null); 1997 Residual Null Deviance: 2267 Residual Deviance: 2113 AIC: 2119 Fall 2010 6/43
Image of page 6
Data: ( y , x ) = ( vote , age , income ) p ( p | y , x ) " n i =1 p ( y i | β 0 , β 1 , x i ) # p ( β 0 , β 1 , β 2 ) = " n i =1 p y i i (1 - p i ) 1 - y i # p ( β 0 , β 1 , β 2 ) log p ( y , x , p ) = n i =1 log( Bernoulli ( y i , p i ))+ log p ( β 0 , β 1 , β 2 ) Fall 2010 7/43
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
, Steps First, use a multivariate Normal jumping distribution β 0 β 1 β 2 N 3 ( 0 , Σ) , Σ = δ 0 0 0 δ 0 0 0 δ to draw all the parameters at the same time. Keep track of your acceptance rate. Note any problems that you encountered. Next, draw each β separately with a univariate Normal jumping distribution given your draws of the other β s. That is, draw in the following order: 1 β ( t ) 0 | β ( t - 1) 1 , β ( t - 1) 2 2 β ( t ) 1 | β ( t ) 0 , β ( t - 1) 2 3 β ( t ) 2 | β ( t ) 0 , β ( t ) 1 Fall 2010 8/43
Image of page 8
> library(mvtnorm) > library(coda) > log.post.func <- function(beta,...){ > pi.i <- 1/(1+exp(-X %*% t(beta))) > log.like <- sum(dbinom(y, size=1, prob=pi.i, log=T)) > log.prior <-dmvnorm(beta, mean=rep(prior.mean,k), sigma=diag(prior.var, k), log=T) > log.post <- log.like + log.prior > return(log.post) > } Fall 2010 9/43
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
> beta.update
Image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern