This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 1 Bayesian Testing and Model Selection 1.1 Review of Frequentist Testing We begin by reviewing some basic notations of frequentist testing. As a sim ple example, suppose we observe a random sample y 1 , ..., y n from a normal population with mean and known variance 2 . Suppose we wish to test the simple hypothesis H : = against the simple alternative A : = 1 where < 1 . Since the sample mean y is sufficient, we can consider the single observation y that is normal with mean and variance 2 /n . The likelihood function is L ( ) = ( y ; , 2 /n ) , where ( y ; , 2 ) is the normal density with mean and variance 2 . The mostpowerful test of H against A is based on the likelihood ratio = L ( 1 ) L ( ) . This test rejects H when k which is equivalent to rejecting when y c . We set a Type I error probability of and choose the constant c so that P ( y c  = ) = . The mostpowerful test of size rejects H when y + z 1 n , where z is the percentile of a standard normal random variable. Here are some comments about this testing procedure. 1. Two types of error? There are two mistakes one can make with a test one can incorrectly reject H when H is true ( = ) or one can incorrectly accept H when A is true ( = 1 ). In a frequentist test, one is controlling only the probability of the first error. 2 1 Bayesian Testing and Model Selection 2. Confidence? This test has a repeated sampling validity. If one performs many tests when H is true, that is, = , then the proportion of times one will incorrectly reject is . 3. Measure of evidence? Suppose one observes an extreme value of y , a value that is unusual if the hypothesis H is true. The frequentist test, as constructed, does not provide a measure of evidence given this extreme value of y . (All one has is the repeated sampling interpretation.) R. A. Fisher proposed the pvalue that is the probability of obtaining the ob served value y or more extreme if indeed H was true. p value = P ( Y y  = ) . In practice, one typically computes a pvalue. This computation allows one to accept or reject the hypothesis H for any value of and provides a measure of the strength of evidence against the null hypothesis H . 1.2 Introduction to Bayesian Testing Lets consider the problem of testing a simple null hypothesis H : = against the simple alternative hypothesis A : = 1 for normal data, known variance, from a Bayesian perspective. Here there are two possible values of the mean, and 1 . Suppose we assign the prior probabilities g ( ) , g ( 1 ) = 1 g ( ) ....
View Full
Document
 Spring '10
 albert

Click to edit the document details