This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 5620 Spring 2011 Homework #2: Solutions 1. 2. 1 3. 4. 2 5. 3 6. 7. (a) I ( X 1 ,X 2 ,...,X n ; Y ) = n X i =1 I ( X i ; Y  X 1 ,...,X i 1 ) = n X i =1 [ H ( X i  X 1 ,...,X i 1 ) H ( X i  X 1 ,...,X i 1 ,Y )] ( i ) = n X i =1 [ H ( X i ) H ( X i  X 1 ,...,X i 1 ,Y )] ( ii ) ≥ n X i =1 [ H ( X i ) H ( X i  Y )] = n X i =1 I ( X i ; Y ) , where ( i ) follows from the independence of the X 1 ,...,X n and ( ii ) follows because conditioning reduces entropy. (b) For equality in ( ii ) we require H ( X i  X 1 ,...,X i 1 ,Y ) = H ( X i  Y ) for all i, which is equivalent to X i ↔ Y ↔ ( X 1 ,...,X i 1 ) for all i. 8. Mimicking the “alternative” proof of the concavity of unconditional en tropy, let ( X 1 ,Y 1 ) have joint PMF p 1 ( x,y ) and let ( X 2 ,Y 2 ) have joint PMF p 2 ( x,y ) . Suppose that T has PMF Pr( T = 1) = θ = 1 Pr( T = 2) . 4 Suppose that ( X 1 ,Y 1 ) , ( X 2 ,Y 2 ) , and T are all independent. Observe that ( X T ,Y T ) has joint PMF p ( x,y ) =...
View
Full Document
 '08
 WAGNER

Click to edit the document details