hw6sol

# hw6sol - EE 376B/Stat 376B Information Theory Prof T Cover...

This preview shows pages 1–5. Sign up to view the full content.

EE 376B/Stat 376B Handout #17 Information Theory Tuesday, May 23, 2006 Prof. T. Cover Solutions to Homework Set #6 1. Slepian-Wolf for deterministically related sources. Find and sketch the Slepian-Wolf rate region for the simultaneous data compression of ( X, Y ) , where y = f ( x ), and f is a given deterministic function. Solution: Slepian-Wolf for deterministically related sources The quantities deﬁning the Slepian Wolf rate region are H ( )= H ( X ), H ( Y | X 0and H ( X | Y ) 0. Hence the rate region is as shown in the Figure 1. - 6 @ @ @ @ @ @ @ @ H ( Y ) H ( X | Y ) R 1 R 2 0 H ( X ) Figure 1: Slepian Wolf rate region for Y = f ( X ). 2. Slepian-Wolf. Three cards from a three-card deck are dealt, one to sender X 1 , one to sender X 2 ,and one to sender X 3 . At what rates do X 1 ,X 2 , and X 3 need to communicate to some receiver so that their card information can be recovered? 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Decoder - - - - - - X n 3 X n 2 X n 1 k ( X n 3 ) j ( X n 2 ) i ( X n 1 ) - ± ˆ X n 1 , ˆ X n 2 , ˆ X n 3 ² Assume that ( X 1 i ,X 2 i 3 i ) are i.i.d. from a uniform distribution over the permutations of { 1 , 2 , 3 } . Solution: Slepian-Wolf From Theorem 14.4.2 of Cover and Thomas the rate region is given by R 1 H ( X 1 | X 2 3 ) R 2 H ( X 2 | X 1 3 ) R 3 H ( X 3 | X 1 2 ) R 1 + R 2 H ( X 1 2 | X 3 ) R 1 + R 3 H ( X 1 3 | X 2 ) R 2 + R 3 H ( X 2 3 | X 1 ) R 1 + R 2 + R 3 H ( X 1 2 3 ) . Since knowing the cards of any two senders determines the card of the third, H ( X 1 | X 2 3 )=0 H ( X 2 | X 1 3 H ( X 3 | X 1 2 . Furthermore, conditioned on the card of one sender, the other two senders have two possible conﬁguration of cards. Since the distribution on the permutations is uniform, the two conﬁgurations are equally likely and therefore H ( X 1 2 | X 3 )=1 H ( X 1 3 | X 2 H ( X 2 3 | X 1 . Finally, since there are six possible permutations of the deck and each is equally likely H ( X 1 2 3 )=log6 . 2
Therefore R 1 + R 2 1 R 1 + R 3 1 R 2 + R 3 1 R 1 + R 2 + R 3 log 6 . 3. Broadcast channel. (a) For the degraded broadcast channel X Y 1 Y 2 , ﬁnd the points a and b where the capacity region hits the R 1 and R 2 axes. (b) Show that b a. Solution: Broadcast channel (a) The capacity region of the degraded broadcast channel X Y 1 Y 2 is the convex hull of regions of the form R 1 I ( X ; Y 1 | U )( 1 ) R 2 I ( U ; Y 2 2 ) over all choices of auxiliary random variable U and joint distribution of the form p ( u ) p ( x | u ) p ( y 1 ,y 2 | x ). The region is of the form illustrated in Figure 2. a b R 1 R 2 Figure 2: Capacity region of degraded broadcast channel The point b on the ﬁgure corresponds to the maximum achievable rate from the sender to receiver 2. From the expression for the capacity region, it is the maxi- mum value of I ( U ; Y 2 ) for all auxiliary random variables U . 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
For any random variable U and p ( u ) p ( x | u ), U X Y 2 f o rm saM a rk o v chain, and hence I ( U ; Y 2 ) I ( X ; Y 2 ) max p ( x ) I ( X ; Y 2 ). The maximum can be achieved by setting U = X and choosing the distribution of X to be the one that maximizes I ( X ; Y 2 ). Hence the point b corresponds to R 2 = max p ( x ) I ( X ; Y 2 ) ,R 1 = I ( X ; Y 1 | U )= I ( X ; Y 1 | X )=0 .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 12

hw6sol - EE 376B/Stat 376B Information Theory Prof T Cover...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online