Lecture15

# Lecture15 - Colorado State University Ft Collins ECE 516...

This preview shows pages 1–3. Sign up to view the full content.

1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 15 October 21, 2008 Recap: 7.6 Joint Typical Sequences Definition: The set () n A ε of jointly typical sequences with respect to joint PMF y x p , is the set of ( ) n n y x , whose empirical entropies are -close to the true entropies. { () () ()() < < < × = Y X H y x p n Y H y p n X H x p n y x A n n n n n n n n n , , log 1 log 1 , log 1 : , Y X Theorem: (Joint AEP) Let ( ) n n Y X , be drawn iid according to = = n i i i n n y x p y x p 1 , , 1. ( ) ( ) 1 , lim = n n n n A y x p , i.e., ( ) ( ) ( ) > 1 , lim n n n n A y x p for large enough n . 2. ( ) ( ) ( ) + Y X H n n Y X H n A , , 2 2 1 3. If ( ) ( ) ( ) n n n n y p x p y x ~ ~ ~ ~ , ~ , i.e., n x ~ and n y ~ are independent with the marginals ( ) n x p and ( ) n y p obtained from ( ) n n y x p , , then ( ) ( ) ( ) ( ) ( ) 3 ; 3 ; 2 ~ , ~ 2 1 + Y X I n n n n Y X I n A y x p Achievability (direct part): All rates below C are achievable. That is, if R < C , then one can design a code ( ) n nR , 2 such that ( ) n λ , ( ) 0 n e P as n . 7.12 Feedback Capacity (of a DMC) C C FB = on a DMC

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 7.13 Joint Source-channel Coding Theorem So far, we saw two major results 1. Source Coding: () ( ) ε + < < X H n X l E X H n O r H R > is required for lossless compression 2. Channel Coding: C R < is required for reliable communication Two questions: 1. Is C H < sufficient and necessary? 2. Do we lose anything by a two step process? Separate source and channel coding? Recall source coding: We work on a block of n symbols. With almost probability 1, the sequence belongs to the typical set. The typical set contains nH 2 sequences. [] ( ) + = + × H n n nH X l E n n X log 1 1 Recall channel coding: [] [ ] [ ] [] [] Y X I R E P E P E E E P W P P R Y X I n Y X nI nR i i c c nR nR ; if 2 2 2 2 1 | ; ; 2 2 1 2 2 1 < + + + = = = = L If C R < , then 2 P Theorem: (Source-channel coding theorem) If n V V V , , , 2 1 L is a finite-alphabet
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 7

Lecture15 - Colorado State University Ft Collins ECE 516...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online