This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: 1 Bayesian Testing and Model Selection 1.1 Review of Frequentist Testing We begin by reviewing some basic notations of frequentist testing. As a sim- ple example, suppose we observe a random sample y 1 , ..., y n from a normal population with mean θ and known variance σ 2 . Suppose we wish to test the simple hypothesis H : θ = θ against the simple alternative A : θ = θ 1 where θ < θ 1 . Since the sample mean ¯ y is sufficient, we can consider the single observation ¯ y that is normal with mean θ and variance σ 2 /n . The likelihood function is L ( θ ) = φ (¯ y ; θ, σ 2 /n ) , where φ ( y ; θ, σ 2 ) is the normal density with mean θ and variance σ 2 . The most-powerful test of H against A is based on the likelihood ratio Λ = L ( θ 1 ) L ( θ ) . This test rejects H when Λ ≥ k which is equivalent to rejecting when ¯ y ≥ c . We set a Type I error probability of α and choose the constant c so that P (¯ y ≥ c | θ = θ ) = α . The most-powerful test of size α rejects H when ¯ y ≥ θ + z 1- α σ √ n , where z α is the α percentile of a standard normal random variable. Here are some comments about this testing procedure. 1. Two types of error? There are two mistakes one can make with a test – one can incorrectly reject H when H is true ( θ = θ ) or one can incorrectly accept H when A is true ( θ = θ 1 ). In a frequentist test, one is controlling only the probability of the first error. 2 1 Bayesian Testing and Model Selection 2. Confidence? This test has a repeated sampling validity. If one performs many tests when H is true, that is, θ = θ , then the proportion of times one will incorrectly reject is α . 3. Measure of evidence? Suppose one observes an extreme value of ¯ y , a value that is unusual if the hypothesis H is true. The frequentist test, as constructed, does not provide a measure of evidence given this extreme value of ¯ y . (All one has is the repeated sampling interpretation.) R. A. Fisher proposed the p-value that is the probability of obtaining the ob- served value ¯ y or more extreme if indeed H was true. p- value = P ( ¯ Y ≥ ¯ y | θ = θ ) . In practice, one typically computes a p-value. This computation allows one to accept or reject the hypothesis H for any value of α and provides a measure of the strength of evidence against the null hypothesis H . 1.2 Introduction to Bayesian Testing Let’s consider the problem of testing a simple null hypothesis H : θ = θ against the simple alternative hypothesis A : θ = θ 1 for normal data, known variance, from a Bayesian perspective. Here there are two possible values of the mean, θ and θ 1 . Suppose we assign the prior probabilities g ( θ ) , g ( θ 1 ) = 1- g ( θ ) ....
View Full Document
- Spring '10
- Normal Distribution, Probability theory, Bayesian statistics, Bayes Factors, Joe IQ