Consider the sequence of sample averages formed by

Info iconThis preview shows pages 11–13. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Consider the sequence of sample averages formed by taking the average of progres- sively more X i ’s. The first entry in the sequence is ¯ X 1 , which is simply equal to X 1 . The next entry is ¯ X 2 , which is the average of X 1 and X 2 . The next entry is the average of the first three X i ’s, and so forth. Continuing in this way, we obtain an infinite sequence of sample averages: ¯ X 1 , ¯ X 2 , ¯ X 3 ,... . The law of large numbers tells us that our sequence of sample averages converges in probability to μ . That is, ¯ X n → p μ as n → ∞ . This is one of the two most important results in all of probability and statistics; the other is the central limit theorem, which we shall come to later. The law of large numbers is used to justify the practice of estimating a population average ( μ ) using a sample average ( ¯ X n ). If our sample size n is large, then we must be a “long way along” the sequence ¯ X 1 , ¯ X 2 , ¯ X 3 ,... . Since this sequence is converging in probability to μ , our sample average ¯ X n should be close to μ with high probability. The key to proving the law of large numbers is a fundamental result in prob- ability theory called Chebyshev’s inequality . Chebyshev’s inequality says that, for any random variable X and any strictly positive number ε > 0, we have P ( | X | > ε ) ≤ 1 ε 2 E ( X 2 ) . Let us prove Chebyshev’s inequality in the special case where X is a continuous random variable with pdf f . We begin by writing P ( | X | > ε ) = P ( X <- ε ) + P ( X > ε ) = Z- ε-∞ f ( x )d x + Z ∞ ε f ( x )d x. Now, if x > ε then we must have x 2 /ε 2 > 1. Therefore, Z ∞ ε f ( x )d x ≤ Z ∞ ε x 2 ε 2 f ( x )d x. Similarly, if x <- ε then x 2 /ε 2 > 1, and so Z- ε-∞ f ( x )d x ≤ Z- ε-∞ x 2 ε 2 f ( x )d x. Combining these two equalities with our expression for P ( | X | > ε ) yields P ( | X | > ε ) ≤ Z- ε-∞ x 2 ε 2 f ( x )d x + Z ∞ ε x 2 ε 2 f ( x )d x. 11 We can make the right-hand side of this inequality larger by including the integral between- ε and ε as an extra term. This yields P ( | X | > ε ) ≤ Z- ε-∞ x 2 ε 2 f ( x )d x + Z ε- ε x 2 ε 2 f ( x )d x + Z ∞ ε x 2 ε 2 f ( x )d x = Z ∞-∞ x 2 ε 2 f ( x )d x = 1 ε 2 Z ∞-∞ x 2 f ( x )d x = 1 ε 2 E ( X 2 ) . This completes the proof of Chebyshev’s inequality for the special case where the random variable X is continuous. A similar proof may be used in the discrete case. Along with Chebyshev’s inequality, we shall employ a few basic facts about the expected value and variance to prove the law of large numbers. You should remember these facts from your previous coursework. Let X and Y be arbitrary random variables, and let c be a constant. Then: 1. E ( cX ) = cE ( X ). 2. E ( X + Y ) = E ( X ) + E ( Y ). 3. Var( cX ) = c 2 Var( X ). 4. Var( X + Y ) = Var( X ) + Var( Y ) + 2Cov( X,Y )....
View Full Document

{[ snackBarMessage ]}

Page11 / 17

Consider the sequence of sample averages formed by taking...

This preview shows document pages 11 - 13. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online