hw6sol - EE 376B/Stat 376B Information Theory Prof. T....

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376B/Stat 376B Handout #17 Information Theory Tuesday, May 23, 2006 Prof. T. Cover Solutions to Homework Set #6 1. Slepian-Wolf for deterministically related sources. Find and sketch the Slepian-Wolf rate region for the simultaneous data compression of ( X, Y ) , where y = f ( x ), and f is a given deterministic function. Solution: Slepian-Wolf for deterministically related sources The quantities defining the Slepian Wolf rate region are H ( )= H ( X ), H ( Y | X 0and H ( X | Y ) 0. Hence the rate region is as shown in the Figure 1. - 6 @ @ @ @ @ @ @ @ H ( Y ) H ( X | Y ) R 1 R 2 0 H ( X ) Figure 1: Slepian Wolf rate region for Y = f ( X ). 2. Slepian-Wolf. Three cards from a three-card deck are dealt, one to sender X 1 , one to sender X 2 ,and one to sender X 3 . At what rates do X 1 ,X 2 , and X 3 need to communicate to some receiver so that their card information can be recovered? 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Decoder - - - - - - X n 3 X n 2 X n 1 k ( X n 3 ) j ( X n 2 ) i ( X n 1 ) - ± ˆ X n 1 , ˆ X n 2 , ˆ X n 3 ² Assume that ( X 1 i ,X 2 i 3 i ) are i.i.d. from a uniform distribution over the permutations of { 1 , 2 , 3 } . Solution: Slepian-Wolf From Theorem 14.4.2 of Cover and Thomas the rate region is given by R 1 H ( X 1 | X 2 3 ) R 2 H ( X 2 | X 1 3 ) R 3 H ( X 3 | X 1 2 ) R 1 + R 2 H ( X 1 2 | X 3 ) R 1 + R 3 H ( X 1 3 | X 2 ) R 2 + R 3 H ( X 2 3 | X 1 ) R 1 + R 2 + R 3 H ( X 1 2 3 ) . Since knowing the cards of any two senders determines the card of the third, H ( X 1 | X 2 3 )=0 H ( X 2 | X 1 3 H ( X 3 | X 1 2 . Furthermore, conditioned on the card of one sender, the other two senders have two possible configuration of cards. Since the distribution on the permutations is uniform, the two configurations are equally likely and therefore H ( X 1 2 | X 3 )=1 H ( X 1 3 | X 2 H ( X 2 3 | X 1 . Finally, since there are six possible permutations of the deck and each is equally likely H ( X 1 2 3 )=log6 . 2
Background image of page 2
Therefore R 1 + R 2 1 R 1 + R 3 1 R 2 + R 3 1 R 1 + R 2 + R 3 log 6 . 3. Broadcast channel. (a) For the degraded broadcast channel X Y 1 Y 2 , find the points a and b where the capacity region hits the R 1 and R 2 axes. (b) Show that b a. Solution: Broadcast channel (a) The capacity region of the degraded broadcast channel X Y 1 Y 2 is the convex hull of regions of the form R 1 I ( X ; Y 1 | U )( 1 ) R 2 I ( U ; Y 2 2 ) over all choices of auxiliary random variable U and joint distribution of the form p ( u ) p ( x | u ) p ( y 1 ,y 2 | x ). The region is of the form illustrated in Figure 2. a b R 1 R 2 Figure 2: Capacity region of degraded broadcast channel The point b on the figure corresponds to the maximum achievable rate from the sender to receiver 2. From the expression for the capacity region, it is the maxi- mum value of I ( U ; Y 2 ) for all auxiliary random variables U . 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
For any random variable U and p ( u ) p ( x | u ), U X Y 2 f o rm saM a rk o v chain, and hence I ( U ; Y 2 ) I ( X ; Y 2 ) max p ( x ) I ( X ; Y 2 ). The maximum can be achieved by setting U = X and choosing the distribution of X to be the one that maximizes I ( X ; Y 2 ). Hence the point b corresponds to R 2 = max p ( x ) I ( X ; Y 2 ) ,R 1 = I ( X ; Y 1 | U )= I ( X ; Y 1 | X )=0 .
Background image of page 4
Image of page 5
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 12

hw6sol - EE 376B/Stat 376B Information Theory Prof. T....

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online