Lecture12 - Colorado State University, Ft. Collins ECE 516:...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 12 October 7, 2008 Recap: 6 Universal Source Coding (Chapter 13) Arithmetic Coding Lempel-Ziv Coding 7 Channel Capacity (Chapter 7) Channel user A X user B Y p ( X ) p ( Y | X ) Source is a r.v. X with PMF ( ) X p , X x The channel is a conditional PMF, ( ) X Y p | , X x and Y y . We will define “information” channel capacity () Y X I C x p ; max =
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Example of channel capacity: 1. Noiseless binary channel 0 1 0 1 Y = X () ( ) ( ) X H X X I Y X I C x p x p x p max ; max ; max = = = = 2 1 , 2 1 x p and 1 = C bit 2. Binary symmetric channel 0 1 0 1 1- p 1- p p p ()( ) ( ) () () ( ) () () () () () p H p H Y H p H x p Y H x X Y H x p Y H X Y H Y H Y X I X X = = = = = = = 1 | | ; 1 0 1 0 But note when = 2 1 , 2 1 x p , we have = 2 1 , 2 1 y p and 1 = Y H Therefore, ( ) ( ) p H Y X I C x p = = 1 ; max Note,
Background image of page 2
3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 p C 3. Binary erasure channel 0 1 0 1 1- α 1- α α α e ()( ) ( ) () () ( ) () () () () () α H Y H H x p Y H x X Y H x p Y H X Y H Y H Y X I X X = = = = = = = | | ; 1 0 1 0 () ( ) ( ) ( ) H H Y H Y X I C x p x p = = 3 log max ; max But is this achievable by a ( ) x p ? No. Let q X p = = 1 and q X p = = 1 0 Then () ( ) ( ) ( ) q q H Y H = 1 , , 1 1 hard to interpret Define = = otherwise , 0 , 1 e Y E
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Then () ( ) ( ) ( ) ( ) E Y H E H Y E H Y H E Y H | | , 0 + = + = = () () ( ) () ( ) ( ) = = = + = + = 1 0 | | e e E Y H e E p H E Y H E H Y H α ( ) q H e E Y H e = = = | 0 0 | 1 = = = e E Y H e Then ( ) ( ) ( ) ( )() [ ] ( ) ( ) q H H q H H H Y
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/17/2010 for the course ECE 516 taught by Professor Rocky during the Spring '08 term at Colorado State.

Page1 / 9

Lecture12 - Colorado State University, Ft. Collins ECE 516:...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online