This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE 478 Handout #8 Multiple User Information Theory October 7, 2008 Homework Set #1 Solutions 1. Solution: (a) Consider H ( X  Z ) ≤ H ( X,Y  Z ) = H ( Y  Z ) + H ( X  Y,Z ) ≤ H ( Y  Z ) + H ( X  Y ) . (b) h ( X + Y ) ≥ h ( X ). Since conditioning reduces the differential entropy, h ( X + Y ) ≥ h ( X + Y  Y ) = h ( X ) . (c) I ( g ( X ); Y ) ≤ I ( X ; Y ). I ( g ( X ); Y ) = H ( Y ) H ( Y  g ( X )) ≤ H ( Y ) H ( Y  X ) = I ( X ; Y ) . (d) I ( Y ; Z  X ) ≥ I ( Y ; Z ) if p ( x,y,z ) = p ( x ) p ( y ) p ( z  x,y ). Since X and Y are independent random variables, I ( Y ; Z  X ) = H ( Y  X ) H ( Y  Z,X ) = H ( Y ) H ( Y  Z,X ) ≥ H ( Y ) H ( Y  Z ) = I ( Y ; Z ) . 2. Solution: (a) Consider h ( Y n ) ≤ n X i =1 h ( Y i ) = n X i =1 1 2 log(2 πe ) K ii = 1 2 log(2 πe ) n n Y i =1 K ii . 1 (b) Now since h ( Y n ) = 1 2 log(2 πe ) n  K  , the result in part (a) implies that  K  ≤ n Y i =1 K ii . 3. Solution: The identity can be proved by induction, or more simply using chain rule of mutual information. Notice that by chain rule the mutual information between Z n and W can be expanded in n ! ways, depending on how we order the element of Z n . For instance, these are both valid expansions I ( Z n ; W ) = n X i =1 I ( Z i ; W  Z i 1 ) = n X j =1 I ( Z j ; W  Z n j +1 ) , where the two ordering Z n = ( Z 1 ,...,Z n ) and Z n = ( Z n ,...,Z 1 ) have been used. Therefore, we have n X i =1 I ( X n i +1 ; Y i  Y i 1 ) = n X i =1 n X j = i +1 I ( X j ; Y i  Y i 1 ,X n j +1 ) (1) = n X j =2 j 1 X i =1 I ( X j ; Y i  Y i 1 ,X n j +1 ) (2) = n X j =2 I ( X j ; Y j 1  X n j +1 ) (3) = n X j =1 I ( X j ; Y j 1  X n j +1 ) (4) = n X i =1 I ( Y i 1 ; X i  X n i +1 ) where (1) and (3) follow from chain rule of mutual information, (2) is obtained switch ing the order in the summations, and finally (4) follows from the fact that Y = ∅ ....
View
Full Document
 Spring '11
 Kelly
 Derivative, English Channel, i=1, X&Y, Z N

Click to edit the document details