hw2_soln_fall2011 - Question 1: ECE 563, Fall 2011 Homework...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
We assume that X,Q 1 ,Q 2 are jointly independent. We also assume A ( )i sa deterministic function; i.e., the answer is a function of the question Q and the object X . (a) I ( X ; Q, A ) ( a ) = I ( Q ; X )+ I ( A ; X | Q ) ( b ) = I ( A ; X | Q ) ( c ) = H ( A | Q ) H ( A | ) ( d ) = H ( A | Q ) where (a) Chain rule. (b) X and Q are independent. (c) DeFnition of mutual information. (d) Answer is a deterministic function of X and Q . The interpretation is as follows. The uncertainty removed in X given ( Q, A )is the same as the uncertainty in the answer given the question. (b) Using the result from part (a) and the fact that questions are independent, we can easily obtain the desired relationship. I ( X ; Q 1 ,A 1 2 2 ) ( a ) = I ( X ; Q 1 I ( X ; A 1 | Q 1 I ( X ; Q 2 | A 1 1 ) + I ( X ; A 2 | A 1 1 2 ) ( b ) = I ( X ; A 1 | Q 1 H ( Q 2 | A 1 1 ) H ( Q 2 | X,A 1 1 ) + I ( X ; A 2 | A 1 1 2 ) ( c ) = I ( X ; A 1 | Q 1 I ( X ; A 2 | A 1 1 2 ) = I ( X ; A 1 | Q 1 H ( A 2 | A 1 1 2 ) H ( A 2 | 1 1 2 ) ( d ) = I ( X ; A 1 | Q 1 H ( A 2 | A 1 1 2 ) ( e ) I ( X ; A 1 | Q 1 H ( A 2 | Q 2 ) ( f ) H ( A 1 | Q 1 ) H ( A 1 | 1 H ( A 2 | Q 2 ) ( g ) H ( A 1 | Q 1 H ( A 2 | Q 2 ) ( h ) 2 H ( A 1 | Q 1 ) ( i ) =2 I ( X ; A 1 1 ) ECE 563, Fall 2011 Homework 2 Solution Parts of the solutions are copied from the solution manual of Cover and Thomas. Question 1:
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
(a) Chain rule. (b) X and Q 1 are independent. (c) Q 2 is independent of X , Q 1 ,and A 1 .(d ) A 2 is completely determined given Q 2 and X . (e) Conditioning decreases entropy. (f) Expand the entropy. (g) A 1 is completely determined given Q 1 and X . (h) Identically distributed. (i) Invoke the result of part (a). In the above solution, the mutual information chain rule was applied directly. Here is an alternative solution which some might prefer. Starting over, we want to prove that I ( X ; Q 1 ,A 1 ,Q 2 2 ) 2 I ( X ; Q 1 1 ) .
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/24/2011 for the course ELECTRICAL ECE 571 taught by Professor Kelly during the Spring '11 term at University of Illinois, Urbana Champaign.

Page1 / 9

hw2_soln_fall2011 - Question 1: ECE 563, Fall 2011 Homework...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online