This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: LECTURE 4 Convergence and Asymptotic Equipartition Property Last time: Fano’s Inequality • Stochastic Processes • Entropy Rate • Hiden Markov Process • Lecture outline Types of convergence • Weak Law of Large Numbers • Strong Law of Large Numbers • Asymptotic Equipartition Property • Reading: Chapter 3. Convergence of Random Variables A sequence of maps Ω → X converge, w.o.l.g., to 0. Pointwise convergence: for any ω ∈ Ω, X n ( ω ) → 0. Goal The Law of Large Numbers: the av erage of a sequence of i.i.d. r.v.s converges to the mean. n 1 lim X n E [ X ] n → n →∞ i =1 Need weaker notions. Types of convergence Almost sure convergence (also called con • vergence with probability 1) P ω : lim Y n ( ω ) = Y ( ω ) = 1 n →∞ write Y n Y a.s. . → Meansquare convergence: • lim E [ Y n − Y 2 ] = 0 n →∞   • Convergence in probability: ∀ > lim P ( { ω : Y n ( ω ) − Y ( ω ) > } ) = 0 n →∞   Convergence in distribution: the cumula • tive distribution function (CDF) F n ( y ) = P r ( Y n ≤ y ) satisfy lim F n ( y ) F Y ( y ) → n →∞ at all y for which F is continuous. Relations among types of convergence Venn diagram of relation: Weak Law of Large Numbers X 1 , X 2 , . . . i.i.d....
View
Full
Document
This note was uploaded on 12/08/2010 for the course MATH 6.041 / 6. taught by Professor Muntherdahleh during the Spring '10 term at MIT.
 Spring '10
 MuntherDahleh
 Systems Analysis, Law Of Large Numbers, Probability

Click to edit the document details