{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lecture12

# Lecture12 - Colorado State University Ft Collins ECE 516...

This preview shows pages 1–5. Sign up to view the full content.

1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 12 October 7, 2008 Recap: 6 Universal Source Coding (Chapter 13) Arithmetic Coding Lempel-Ziv Coding 7 Channel Capacity (Chapter 7) Channel user A X user B Y p ( X ) p ( Y | X ) Source is a r.v. X with PMF ( ) X p , X x The channel is a conditional PMF, ( ) X Y p | , X x and Y y . We will define “information” channel capacity ( ) ( ) Y X I C x p ; max =

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Example of channel capacity: 1. Noiseless binary channel 0 1 0 1 Y = X ( ) ( ) ( ) ( ) ( ) ( ) X H X X I Y X I C x p x p x p max ; max ; max = = = ( ) = 2 1 , 2 1 x p and 1 = C bit 2. Binary symmetric channel 0 1 0 1 1- p 1- p p p ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) p H p H Y H p H x p Y H x X Y H x p Y H X Y H Y H Y X I X X = = = = = = = 1 | | ; 1 0 1 0 But note when ( ) = 2 1 , 2 1 x p , we have ( ) = 2 1 , 2 1 y p and ( ) 1 = Y H Therefore, ( ) ( ) ( ) p H Y X I C x p = = 1 ; max Note,