{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lect07 - Lecture Notes 7 Convergence and Limit Theorems...

Info icon This preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture Notes 7 Convergence and Limit Theorems Motivation Convergence with Probability 1 Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and Limit Theorems 7 – 1 Motivation One of the key questions in statistical signal processing is how to estimate the statistics of a r.v., e.g., its mean, variance, distribution, etc. To estimate such a statistic, we collect samples and use an estimator in the form of a sample average How good is the estimator ? Does it “converge” to the true statistic? How many samples do we need to ensure with some confidence that we are within a certain range of the true value of the statistic? Another key question in statistical signal processing is how to estimate a signal from noisy observations, e.g., using MSE or linear MSE Does the estimator converge to the true signal? How many observations do we need to achieve a desired estimation accuracy? The subject of convergence and limit theorems for r.v.s addresses such questions EE 278: Convergence and Limit Theorems 7 – 2
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example: Estimating the Mean of a R.V. Let X be a r.v. with finite but unknown mean E( X ) To estimate the mean we generate X 1 , X 2 , . . . , X n i.i.d. samples drawn according to the same distribution as X and compute the sample mean S n = 1 n n X i =1 X i Does S n converge to E( X ) as we increase n ? If so, how fast? But what does it mean to say that a r.v. sequence S n converges to E( X ) ? First we give an example: Let X 1 , X 2 , . . . , X n be i.i.d. N (0 , 1) We use Matlab to generate 6 sets of outcomes of X 1 , . . . , X n We then plot s n for the 6 sets of outcomes as a function of n Note that each s n sequence appears to be converging to 0 , the mean of the r.v., as n increases EE 278: Convergence and Limit Theorems 7 – 3 Plots of Sample Sequences of S n 1 2 3 4 5 6 7 8 9 10 -2 0 2 10 20 30 40 50 60 70 80 90 100 -0.5 0 0.5 100 200 300 400 500 600 700 800 900 1000 -0.2 0 0.2 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 -0.05 0 0.05 n s n s n s n s n EE 278: Convergence and Limit Theorems 7 – 4
Image of page 2
Convergence With Probability 1 Recall that a sequence of numbers x 1 , x 2 , . . . , x n , . . . converges to x if for every > 0 , there exists an n ( ) such that | x n - x | < for every n n ( ) Now consider a sequence of r.v.s X 1 , X 2 , . . . , X n , . . . all defined on the same probability space Ω . For every ω Ω we obtain a sample sequence (sequence of numbers) X 1 ( ω ) , X 2 ( ω ) , . . . , X n ( ω ) , . . . A sequence X 1 , X 2 , X 3 , . . . of r.v.s is said to converge to random variable X with probability 1 (w.p.1) if P { ω : lim n →∞ X n ( ω ) = X ( ω ) } = 1 This means that the set of sample paths that converge to X ( ω ) , in the sense of a sequence converging to a limit, has probability 1 Equivalently, X 1 , X 2 , . . . , X n , . . . converges w.p.1 if for every > 0 , lim m →∞ P {| X n - X | < for every n m } = 1 EE 278: Convergence and Limit Theorems 7 – 5 Example 1: Let X 1 , X 2 , . . . , X n be i.i.d. Bern(1 / 2) , and define Y n = 2 n Q n i =1 X i . Show that the sequence Y n converges to 0 w.p.1 Solution: To show this, let > 0 (and < 2 m
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern