101: Stat Lecture 9
Summer 2006
Outline
Answer Questions
Box Models
Expected Value and Standard Error
The Central Limit Theorem
Box Models
A Box Model descrives a process in terms of making repeated draws, with replacement, from a box containing numbers. Since draws are made with replacement, the outcomes in a series of draws are independent. The value on the first draw does not affect the value on the second. Box models describe: repeated rolls of a die, repeated tosses of a coin (fair or unfair), and (approximately) drawing a random sample of U.S. citizens and asking for whom they plan to vote.
For a box model, the expected value is the average of the numbers in the box. For categorical data, such as H or T in coin-tossing, or voting for Bush or Kerry, one averages zeroes and ones. When averaging zeroes and ones, the result is just a proportion, or the total number of people voting for Kerry divided by the total number of people whom you ask. The Law of Averages says that if one makes many draws from the box and averages the results, that average will converge to the expected value of the box. But the Law of Averges says nothing about the outcome on the next draw.
Expected Value and Standard Error
Suppose that a box contains B numbers, X1 , . . . , XB . Then the expected value of the box is 1 EV = X = B
B
Xi
i=1
and the standard deviation of the box is, 1 (X1 - EV )2 + . . . + (XB - EV )2 B 1 B
B
sd = =
Xi2
i=1
- EV 2
The box represents a "population" but the calculations are just finding the mean and sd for a list of numbers.
EV is just a number. It is not random. It is a characteristic of the box. The same is true for . The number of elements in the box B has nothing to do with the sample size you draw from the box. The mean of a sample from the box would be written as X and the standard deviation of the sample would be written as sd. They vary around EV and , respectively.
The standard error for the average of n draws from a box (with replacement) is, seaverage = sqrtn
This is the sd of the averages found in many samples of size n. The standard error is the likely size of the difference between the average of n draws from the box and the expected value of the box. Note that as n , the standard error se goes to zero. This is a formal statement of the Law of Averages. It means that the sample average is a good estimate of the average in the box, and the accuracy of the estimate improves as you take more and more draws from the box.
The standard error for the sum of n draws (with replacement) is: sesum = n This is the sd of many sums of size n. Analogously to standard error for averages, the standard error of the sum is the likely size of the difference between the sum of n draws from the box and n times the expected value of the box. Note that as n , this standard error does not go to zero. This means that as the number of draws incresease, the likely difference between the sum of the draws and nEV gets larger, rather than smaller. The behavior of the sum, rather than the behavior of the average, becomes important in contexts such as investment or gambling, where the total return from multiple trials is important, not the average return.
The Central Limit Theorem
The Central Limit Theorem is one of hte high-water marks of mathematical thinking. It was worked upon by James Bernoulli, Abraham de Moivre, and Alan Turing. Over the centuries, the theory improved from special cases to a very general rule. Essentially, the Central Limit Theorem allows one to describe how accurately the Law of Averages works. Most people have a good intuitive understanding of the Law of Averages, in but many cases it is important to determine whether a particular size of deviation between the sample mean and the (usually unknown) expected value is probable or improbable. That is, what is the chance that the sample average is more than some distance d away from the true EV?
Formally, the Central Limit Theorem for averages says: X - EV N(0, 1) / n where, X is the average of n draws, and, imagine a box model, EV is the expected value of the box, and is the standard deviation of the box. This emans that the left-hand side is a random number that is approximately normal with mean zero and standard deviation one. The approximation gets better as n gets larger. A key point is that to use the Central Limit Theorem, we need to know the EV and of the box model.
A version of the Central Limit holds for sums: nX - nEV N(0, 1) n Note that nX is just the sum of the draws from the box. These two formulas are the same. Just multiply top and bottom of the first one by n and you will get the second one. This formula is useful when calculating the cahnce of winning a given amount of money when gambling, or getting more than a specific score on a test. With these two central limit formulas, one can answer all sorts of practical questions.
Problem 1
You want to estimate the average income of people in Durham. Suppose the true mean income is $42,000 with sd of $10,000. You draw a random sample of 100 households. What is the probability that your sample estimate exceeds the true value by $500 or more? What is the box model for this problem? What is the EV (expected value) of the box? What is the of the box?
P(X > 42, 500) = P(X - EV > 500) = P (X - EV )/se > 500/se = P(Z > 500/(10, 000/ 100)) = P(Z > 0.5) From the standard normal table, we know this has chance (1/2)(100 - 38.29) = 30.85%, so the probability of the estimate being too high by $500 is just 0.3085. But this is not really the question one wants to ask in practice, nor is it the kind of information that one really has from a survey.
Problem 1a
You want to estimate the average income of people in Durham. You draw a random sample of 100 households and find the mean is $42,500 and the ...