# Since ε is strictly positive lim n 1 ε 1 2 n 0 it

• Notes
• 17

This preview shows pages 10–13. Sign up to view the full content.

Since ε is strictly positive, lim n →∞ (1 + ε ) 1 - 2 n = 0. It follows that Z n p 1. 9 The law of large numbers Suppose we have an infinite sequence of random variables X 1 , X 2 , X 3 , . . . . Assume that this sequence of random variables is independently and identically distributed , or iid. This means that (1) the different X i ’s are independent of one another, and (2) the different X i ’s all have the same distribution – that is, the same pmf or pdf. A consequence of (2) is that the different X i ’s all have the same expected value and variance. We will denote the common expected value by μ and the common variance by σ 2 . 10

This preview has intentionally blurred sections. Sign up to view the full version.

For any n , we can imagine taking the average of the first n of the X i ’s. This average will be denoted ¯ X n : ¯ X n = 1 n n X i =1 X i . Consider the sequence of sample averages formed by taking the average of progres- sively more X i ’s. The first entry in the sequence is ¯ X 1 , which is simply equal to X 1 . The next entry is ¯ X 2 , which is the average of X 1 and X 2 . The next entry is the average of the first three X i ’s, and so forth. Continuing in this way, we obtain an infinite sequence of sample averages: ¯ X 1 , ¯ X 2 , ¯ X 3 , . . . . The law of large numbers tells us that our sequence of sample averages converges in probability to μ . That is, ¯ X n p μ as n → ∞ . This is one of the two most important results in all of probability and statistics; the other is the central limit theorem, which we shall come to later. The law of large numbers is used to justify the practice of estimating a population average ( μ ) using a sample average ( ¯ X n ). If our sample size n is large, then we must be a “long way along” the sequence ¯ X 1 , ¯ X 2 , ¯ X 3 , . . . . Since this sequence is converging in probability to μ , our sample average ¯ X n should be close to μ with high probability. The key to proving the law of large numbers is a fundamental result in prob- ability theory called Chebyshev’s inequality . Chebyshev’s inequality says that, for any random variable X and any strictly positive number ε > 0, we have P ( | X | > ε ) 1 ε 2 E ( X 2 ) . Let us prove Chebyshev’s inequality in the special case where X is a continuous random variable with pdf f . We begin by writing P ( | X | > ε ) = P ( X < - ε ) + P ( X > ε ) = Z - ε -∞ f ( x )d x + Z ε f ( x )d x. Now, if x > ε then we must have x 2 2 > 1. Therefore, Z ε f ( x )d x Z ε x 2 ε 2 f ( x )d x. Similarly, if x < - ε then x 2 2 > 1, and so Z - ε -∞ f ( x )d x Z - ε -∞ x 2 ε 2 f ( x )d x. Combining these two equalities with our expression for P ( | X | > ε ) yields P ( | X | > ε ) Z - ε -∞ x 2 ε 2 f ( x )d x + Z ε x 2 ε 2 f ( x )d x. 11
We can make the right-hand side of this inequality larger by including the integral between - ε and ε as an extra term. This yields P ( | X | > ε ) Z - ε -∞ x 2 ε 2 f ( x )d x + Z ε - ε x 2 ε 2 f ( x )d x + Z ε x 2 ε 2 f ( x )d x = Z -∞ x 2 ε 2 f ( x )d x = 1 ε 2 Z -∞ x 2 f ( x )d x = 1 ε 2 E ( X 2 ) .

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern