This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: 1 Bayesian Testing and Model Selection 1.1 Review of Frequentist Testing We begin by reviewing some basic notations of frequentist testing. As a sim- ple example, suppose we observe a random sample y 1 , ..., y n from a normal population with mean and known variance 2 . Suppose we wish to test the simple hypothesis H : = against the simple alternative A : = 1 where < 1 . Since the sample mean y is sufficient, we can consider the single observation y that is normal with mean and variance 2 /n . The likelihood function is L ( ) = ( y ; , 2 /n ) , where ( y ; , 2 ) is the normal density with mean and variance 2 . The most-powerful test of H against A is based on the likelihood ratio = L ( 1 ) L ( ) . This test rejects H when k which is equivalent to rejecting when y c . We set a Type I error probability of and choose the constant c so that P ( y c | = ) = . The most-powerful test of size rejects H when y + z 1- n , where z is the percentile of a standard normal random variable. Here are some comments about this testing procedure. 1. Two types of error? There are two mistakes one can make with a test one can incorrectly reject H when H is true ( = ) or one can incorrectly accept H when A is true ( = 1 ). In a frequentist test, one is controlling only the probability of the first error. 2 1 Bayesian Testing and Model Selection 2. Confidence? This test has a repeated sampling validity. If one performs many tests when H is true, that is, = , then the proportion of times one will incorrectly reject is . 3. Measure of evidence? Suppose one observes an extreme value of y , a value that is unusual if the hypothesis H is true. The frequentist test, as constructed, does not provide a measure of evidence given this extreme value of y . (All one has is the repeated sampling interpretation.) R. A. Fisher proposed the p-value that is the probability of obtaining the ob- served value y or more extreme if indeed H was true. p- value = P ( Y y | = ) . In practice, one typically computes a p-value. This computation allows one to accept or reject the hypothesis H for any value of and provides a measure of the strength of evidence against the null hypothesis H . 1.2 Introduction to Bayesian Testing Lets consider the problem of testing a simple null hypothesis H : = against the simple alternative hypothesis A : = 1 for normal data, known variance, from a Bayesian perspective. Here there are two possible values of the mean, and 1 . Suppose we assign the prior probabilities g ( ) , g ( 1 ) = 1- g ( ) ....
View Full Document
- Spring '10