1Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 7 September 18, 2008 Recap: 3 The Asymptotic Equipartition Property (Chapter 3) Markov Inequality: For a nonnegative r.v., Xand a constant 0>ccXEcxP≤≥Chebyshev Inequality: For a r.v., Yand a constant 0>c[ ][ ]2cYVarcYEyP≤≥−Sample Mean: Let nXX,,1Lbe niid r.v.’s with mean Xμand variance 2Xσ. Then nXXYnn++=L1is a r.v. with mean Xμand variance nX2σWeak Law of Large Numbers (WLLN): XnnnXXYμ→++=L1in probability i.e. 1lim=<−∞→εμXnnyPfor any 0>εTheorem (AEP) If L,,21XXare iid with ()Xp, then ()()XHXXpnn→−,,log11Lin probability Definition: The typical set ()nAεwith respect to ( )xpis the set of sequences ()nnxxX∈,,1Lwith the following property: ()()()()()εε−−+−≤≤XHnnXHnxxp2,,21L
This preview has intentionally blurred sections.
Sign up to view the full version.