This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: ISyE8843A, Brani Vidakovic Handout 5 1 Priors A prior is a sword and Achilles heel of Bayesian statistics. Priors are carriers of prior information that is coherently incorporated via Bayes theorem to the inference. At the same time, parameters are unobservable, and prior specification is subjective in nature. Subjectivity of specifying the prior is fundamental objection of rabid frequentists to the Bayesian approach. Frequentists, attacking Bayesians for subjectivity are not saints of objectivity themselves. The very elicitation of a model (likelihood) and loss function is highly subjective, and Bayesians merely divide the necessary subjectivity to two sources - that from the model and from the prior. Since the prior and loss are not separable in decision theoretic statistical inference (Herman Rubin (1987)) it follows that a decision theoretic frequentist and a Bayesian are equally subjective. Figure 1: Importance of Priors Being subjective for an engineer is not a bad thing! Being subjective does not mean being nonscientific, as critics of Bayesian statistic often insinuate. On the contrary – vast amount scientific information coming from theoretical and physical models is guiding specification of priors and merging such information with the data for better inference. Examples are abundant and in this course you will see many instances. In the last several decades Bayesian research also focused on priors that are un-informative and robust as an answer to criticism that Bayesian inference is overly sensitive to the choice of a prior. We will cover several such paradigms. The nature and sources of priors carry deep philosophical load as well and are subject of discussion and disagreements even among Bayesians. In this handout we will discuss various priors. Some of them are historic (uniform priors of Laplace and Bayes), some of them mathematically convenient (conjugate priors; Raiffa and Schlaifer, 1961), and some robust and noninformative (Jefreys, improper, reference, etc. priors. We will talk as well about hierarchical priors and priors over various domains (Dirichlet, Wavelets, etc). 1.1 Uniform Priors of Bayes and Laplace In modern mathematical language, the Bayes’ essay dealt with the following problem. [year 1764] A billiard ball W is rolled on a line [0 , 1] , with a uniform probability of stopping anywhere. It stops at p . A second ball O is then rolled n times under the same conditions and X times stopped on the left of W . Given X what can we say about p . 1 In particular, Bayes was interested in P ( a < p < b ) for ≤ a < b ≤ 1 . In our notation the solution is given by B e ( X + 1 ,n- X + 1) as the posterior of Binomial/Uniform Bayesian model, but at that time the Reverend had difficulties of evaluating P ( a < p < b ) ....
View Full Document
This note was uploaded on 10/23/2011 for the course ISYE 8843 taught by Professor Vidakovic during the Spring '11 term at Georgia Tech.
- Spring '11