{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lecture7

# Lecture7 - Colorado State University Ft Collins ECE 516...

This preview shows pages 1–4. Sign up to view the full content.

1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 7 September 18, 2008 Recap: 3 The Asymptotic Equipartition Property (Chapter 3) Markov Inequality : For a nonnegative r.v., X and a constant 0 > c [ ] [ ] c X E c x P Chebyshev Inequality : For a r.v., Y and a constant 0 > c [ ] [ ] [ ] 2 c Y Var c Y E y P Sample Mean : Let n X X , , 1 L be n iid r.v.’s with mean X μ and variance 2 X σ . Then n X X Y n n + + = L 1 is a r.v. with mean X μ and variance n X 2 σ Weak Law of Large Numbers (WLLN) : X n n n X X Y μ + + = L 1 in probability i.e. [ ] 1 lim = < ε μ X n n y P for any 0 > ε Theorem (AEP) If L , , 2 1 X X are iid with ( ) X p , then ( ) ( ) X H X X p n n , , log 1 1 L in probability Definition: The typical set ( ) n A ε with respect to ( ) x p is the set of sequences ( ) n n x x X , , 1 L with the following property: ( ) ( ) ( ) ( ) ( ) ε ε + X H n n X H n x x p 2 , , 2 1 L

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document