This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Consider the sequence of sample averages formed by taking the average of progres sively more X i ’s. The first entry in the sequence is ¯ X 1 , which is simply equal to X 1 . The next entry is ¯ X 2 , which is the average of X 1 and X 2 . The next entry is the average of the first three X i ’s, and so forth. Continuing in this way, we obtain an infinite sequence of sample averages: ¯ X 1 , ¯ X 2 , ¯ X 3 ,... . The law of large numbers tells us that our sequence of sample averages converges in probability to μ . That is, ¯ X n → p μ as n → ∞ . This is one of the two most important results in all of probability and statistics; the other is the central limit theorem, which we shall come to later. The law of large numbers is used to justify the practice of estimating a population average ( μ ) using a sample average ( ¯ X n ). If our sample size n is large, then we must be a “long way along” the sequence ¯ X 1 , ¯ X 2 , ¯ X 3 ,... . Since this sequence is converging in probability to μ , our sample average ¯ X n should be close to μ with high probability. The key to proving the law of large numbers is a fundamental result in prob ability theory called Chebyshev’s inequality . Chebyshev’s inequality says that, for any random variable X and any strictly positive number ε > 0, we have P (  X  > ε ) ≤ 1 ε 2 E ( X 2 ) . Let us prove Chebyshev’s inequality in the special case where X is a continuous random variable with pdf f . We begin by writing P (  X  > ε ) = P ( X < ε ) + P ( X > ε ) = Z ε∞ f ( x )d x + Z ∞ ε f ( x )d x. Now, if x > ε then we must have x 2 /ε 2 > 1. Therefore, Z ∞ ε f ( x )d x ≤ Z ∞ ε x 2 ε 2 f ( x )d x. Similarly, if x < ε then x 2 /ε 2 > 1, and so Z ε∞ f ( x )d x ≤ Z ε∞ x 2 ε 2 f ( x )d x. Combining these two equalities with our expression for P (  X  > ε ) yields P (  X  > ε ) ≤ Z ε∞ x 2 ε 2 f ( x )d x + Z ∞ ε x 2 ε 2 f ( x )d x. 11 We can make the righthand side of this inequality larger by including the integral between ε and ε as an extra term. This yields P (  X  > ε ) ≤ Z ε∞ x 2 ε 2 f ( x )d x + Z ε ε x 2 ε 2 f ( x )d x + Z ∞ ε x 2 ε 2 f ( x )d x = Z ∞∞ x 2 ε 2 f ( x )d x = 1 ε 2 Z ∞∞ x 2 f ( x )d x = 1 ε 2 E ( X 2 ) . This completes the proof of Chebyshev’s inequality for the special case where the random variable X is continuous. A similar proof may be used in the discrete case. Along with Chebyshev’s inequality, we shall employ a few basic facts about the expected value and variance to prove the law of large numbers. You should remember these facts from your previous coursework. Let X and Y be arbitrary random variables, and let c be a constant. Then: 1. E ( cX ) = cE ( X ). 2. E ( X + Y ) = E ( X ) + E ( Y ). 3. Var( cX ) = c 2 Var( X ). 4. Var( X + Y ) = Var( X ) + Var( Y ) + 2Cov( X,Y )....
View
Full Document
 Spring '08
 Stohs
 Normal Distribution, Probability theory, probability density function

Click to edit the document details