This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 5620 Spring 2011 Homework #2: Solutions 1. 2. 1 3. 4. 2 5. 3 6. 7. (a) I ( X 1 ,X 2 ,...,X n ; Y ) = n X i =1 I ( X i ; Y  X 1 ,...,X i 1 ) = n X i =1 [ H ( X i  X 1 ,...,X i 1 ) H ( X i  X 1 ,...,X i 1 ,Y )] ( i ) = n X i =1 [ H ( X i ) H ( X i  X 1 ,...,X i 1 ,Y )] ( ii ) n X i =1 [ H ( X i ) H ( X i  Y )] = n X i =1 I ( X i ; Y ) , where ( i ) follows from the independence of the X 1 ,...,X n and ( ii ) follows because conditioning reduces entropy. (b) For equality in ( ii ) we require H ( X i  X 1 ,...,X i 1 ,Y ) = H ( X i  Y ) for all i, which is equivalent to X i Y ( X 1 ,...,X i 1 ) for all i. 8. Mimicking the alternative proof of the concavity of unconditional en tropy, let ( X 1 ,Y 1 ) have joint PMF p 1 ( x,y ) and let ( X 2 ,Y 2 ) have joint PMF p 2 ( x,y ) . Suppose that T has PMF Pr( T = 1) = = 1 Pr( T = 2) . 4 Suppose that ( X 1 ,Y 1 ) , ( X 2 ,Y 2 ) , and T are all independent. Observe that ( X T ,Y T ) has joint PMF p ( x,y ) =...
View
Full
Document
This note was uploaded on 10/03/2011 for the course ECE 5620 at Cornell University (Engineering School).
 '08
 WAGNER

Click to edit the document details