6441lecture4

6441lecture4 - LECTURE 4 Convergence and Asymptotic...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: LECTURE 4 Convergence and Asymptotic Equipartition Property Last time: Fano’s Inequality • Stochastic Processes • Entropy Rate • Hiden Markov Process • Lecture outline Types of convergence • Weak Law of Large Numbers • Strong Law of Large Numbers • Asymptotic Equipartition Property • Reading: Chapter 3. Convergence of Random Variables A sequence of maps Ω → X converge, w.o.l.g., to 0. Pointwise convergence: for any ω ∈ Ω, X n ( ω ) → 0. Goal The Law of Large Numbers: the av- erage of a sequence of i.i.d. r.v.s converges to the mean. n 1 lim X n E [ X ] n → n →∞ i =1 Need weaker notions. Types of convergence Almost sure convergence (also called con- • vergence with probability 1) P ω : lim Y n ( ω ) = Y ( ω ) = 1 n →∞ write Y n Y a.s. . → Mean-square convergence: • lim E [ Y n − Y 2 ] = 0 n →∞ | | • Convergence in probability: ∀ > lim P ( { ω : Y n ( ω ) − Y ( ω ) > } ) = 0 n →∞ | | Convergence in distribution: the cumula- • tive distribution function (CDF) F n ( y ) = P r ( Y n ≤ y ) satisfy lim F n ( y ) F Y ( y ) → n →∞ at all y for which F is continuous. Relations among types of convergence Venn diagram of relation: Weak Law of Large Numbers X 1 , X 2 , . . . i.i.d....
View Full Document

This note was uploaded on 12/08/2010 for the course MATH 6.041 / 6. taught by Professor Muntherdahleh during the Spring '10 term at MIT.

Page1 / 16

6441lecture4 - LECTURE 4 Convergence and Asymptotic...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online