This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Approaches to the Bayesian Linear Model Andrew Womack November 14, 2011 1 Introduction We consider models of the form y = x β + (1) where i ∼ N (0 ,σ 2 ). In general, if we have p possible covariates to include then there are 2 p possible models to consider and thus 2 p priors on ( β ,σ 2 ) to construct. We will consider the construction of many types of priors that lend themselves well to Gibbs Sampling. These priors will have hyperparameters, which we can either fix, estimate, or provide with their own priors. Most of the priors that we will construct use a hierarchical framework and can be expressed as mixtures of normals. There are two important lessons to draw from this exercise. The first is the variety of models that can be built under similar frameworks. The second is the consideration of priors that lend themselves easily to Monte Carlo methods, such as Gibbs Sampling. A model will be considered to be a vector of ones and zeros of length p which tells us which covariates are included in the model. For a given one of these vectors γ , we will call the model M γ and subscript x , β ,σ 2 by γ to denote this model. The model M γ has p γ covariates. We also assume that all hyperparameters have fixed values. In a full analysis, these are either estimated or given their own prior distributions. 2 Mulitvariate Normal Prior To facilitate our discussion, we begin with a simple prior for the linear model. We begin with the simple likelihood y | x γ , β γ ,σ 2 γ ,M γ ∼ N ( x γ β γ ,σ 2 γ I ) (2) This likelihood has the form f γ ( y | x γ , β γ ,σ 2 γ ,M γ ) = 1 2 πσ 2 γ n 2 exp- 1 2 σ 2 γ ( y- x γ β γ ) ( y- x γ β γ ) (3) 1 We consider a multivariate normal prior for β γ and inverse gamma prior for σ 2 γ . π γ ( β γ ,σ 2 γ ) = 1 2 π pγ 2 1 | B γ | 1 2 exp- 1 2 σ 2 γ ( β γ- β ,γ ) B- 1 γ ( β γ- β ,γ ) (4) × 1 Γ ( ν γ 2 ) r γ 2 νγ 2 ( σ 2 γ )- νγ 2- 1 exp- r γ 2 σ 2 γ (5) For a particular model, the posterior distribution is given as a normal-inverse gamma distri- bution β...
View Full Document
- Fall '11
- Gibbs sampler, Mγ, xγ, xγ xγ, block Gibbs Sampler