TO INTRODUCTION PROBABILITY AND STATISTICS II
Handout 10 36-226
Interval Estimation: Some Motivation
Spring, 2009
At the beginning of the semester, we discussed some of the elements of statistical inference. Although at the time we didnt dene it, one of those elements of a statistical inference included a measure of precision or accuracy of how good the inference is. We have discussed desirable properties of point estimators and have seen that these properties depend on the probability distribution of the point estima tor, i.e, f (). We will use the standard deviation of the point estimator, i.e., V ar(), as a measure of
where we have a large sample size n and recall that the general form of an interval estimate of , the target parameter, is: z/2 V ar()
the accuracy of an inference. In fact this measure of the variability of the point estimator is so important it is given a special name. It is called the standard error (SE) of . Just to be clear SE() = V ar(), i.e., SE() is just the standard deviation of . How do we use SE() as a measure of the precision of an inference? We consider here the case
(1)
has several different interpretations, in the end, (1) leads to the same scientic inference. The purpose of this handout, is to motivate this expression from two different perspectives. Motivation for Condence Intervals: Approach 1
We have referred to this interval as a (1 )100% condence interval for . We have also noted that the more narrow the interval the more precise our estimate of . Of course, the width of the interval depends on V ar() = SE(). Although the expression in (1) arises from several different perspectives and
The rst justication for (1) comes from a combination of thinking about what would happen in innitely many repetitions of this (1 )100% condence interval procedure, together with the Central Limit Theorem. I should point out that this latter justication is the one that is used exclusively in the vast majority of elementary statistics textbooks. Briey, the interpretation of (1) in this case does not speak about the data we actually observed. Instead, it speaks of the operating characteristics of the condence interval procedure: if we were to estimate repeatedly with new data then, in the long run (with innitely many repetitions), about (1 )100% of the time the intervals ( z/2 SE(), + z/2 SE() would include . We illustrated this idea in class using the following applet: You may want to try this yourself. Go to http://www.stat.cmu.edu/meyer/tea/. Scroll down to the heading entitled Condence Interval and select Condence Intervals I. This will take you to the IPS 4th Ed. page. Choose Applets and click on Condence Interval. Then follow directions. Motivation for Condence Intervals: Approach 2 The second justication for (1) above comes from an application of Bayes Theorem. (See, e.g., homework #2, problem 6 and handout #7.) Ill illustrate this approach in the context of a Binomial 1
experiment where we are interested in estimating p, the probability of success. Let X be the number of successes in a Binomail experiment with n trials. The Binomial probability function is f (x|p) = n x p (1 p)nx x
which was previously by denoted p(x) = P (X = x). Here the notation f (x|p) is used to imply that we are examing the probability distribution of X given the value of p. We would, instead, like to nd the distribution of p given x: the latter distribution would describe what we know about p having observed the data X = x. The reversal of the conditioning is accomplished by Bayes theorem: f (p|x) = f (x, p) = f (x) f (x|p)f (p) . f (x|p)f (p)dp (2)
To apply (2) we need the initial probability distribution f (p) which expresses what we know about p before we see the data X = x. A simple procedure, motivated by the assumption that we know very little about the value of p aside from its having to be between 0 and 1, is to take f (p) = 1 so that the distribution of p is Uniform on (0,1) (which can also be written as a Beta( = 1, = 1). We then obtain f (p|x) = which reduces to f (p|x) =
n x 1 n 0 x
px (1 p)(nx) 1 px (1 p)(nx) 1dp (3)
px (1 p)(nx) . 1 x (nx) dp 0 p (1 p)
This formula is the pdf of a Beta distribution with parameters (x + 1) and (n x) + 1 (see, for example, problem 6 on Homework #2). We note that for large n that the Beta distribution is approximated very well by a Normal distribution having mean = p and standard deviation = SE(). See Figure 1, p where both the Beta distribution and its Normal approximation are plotted for the case in which x = 60 and n = 100; they are essentially identical. This means, in particular, that P p z/2 SE() p p + z/2 SE() = 1 . For example, according to this logic, after we p p observe the value of p, the probability that p is in the interval ( 2 SE(), p + 2 SE()) is approx p p p imately 95% (since Im using z0.025 = 1.96 2) and is of the form of a 95% condence interval for p (see (1)).
2
0.0
0.2
0.4 p
0.6
0.8
1.0
Figure 1: The Beta distribution and its Normal approximation when x = 60 and n = 100. Remarks: 1. Notice that from the formula for the stand...