{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# hw2_soln_fall2011 - Question 1 ECE 563 Fall 2011 Homework 2...

This preview shows pages 1–3. Sign up to view the full content.

Solution: Random questions. We assume that X, Q 1 , Q 2 are jointly independent. We also assume A ( X, Q ) is a deterministic function; i.e., the answer is a function of the question Q and the object X . (a) I ( X ; Q, A ) ( a ) = I ( Q ; X ) + I ( A ; X | Q ) ( b ) = I ( A ; X | Q ) ( c ) = H ( A | Q ) H ( A | X, Q ) ( d ) = H ( A | Q ) where (a) Chain rule. (b) X and Q are independent. (c) Definition of mutual information. (d) Answer is a deterministic function of X and Q . The interpretation is as follows. The uncertainty removed in X given ( Q, A ) is the same as the uncertainty in the answer given the question. (b) Using the result from part (a) and the fact that questions are independent, we can easily obtain the desired relationship. I ( X ; Q 1 , A 1 , Q 2 , A 2 ) ( a ) = I ( X ; Q 1 ) + I ( X ; A 1 | Q 1 ) + I ( X ; Q 2 | A 1 , Q 1 ) + I ( X ; A 2 | A 1 , Q 1 , Q 2 ) ( b ) = I ( X ; A 1 | Q 1 ) + H ( Q 2 | A 1 , Q 1 ) H ( Q 2 | X, A 1 , Q 1 ) + I ( X ; A 2 | A 1 , Q 1 , Q 2 ) ( c ) = I ( X ; A 1 | Q 1 ) + I ( X ; A 2 | A 1 , Q 1 , Q 2 ) = I ( X ; A 1 | Q 1 ) + H ( A 2 | A 1 , Q 1 , Q 2 ) H ( A 2 | X, A 1 , Q 1 , Q 2 ) ( d ) = I ( X ; A 1 | Q 1 ) + H ( A 2 | A 1 , Q 1 , Q 2 ) ( e ) I ( X ; A 1 | Q 1 ) + H ( A 2 | Q 2 ) ( f ) H ( A 1 | Q 1 ) H ( A 1 | X, Q 1 ) + H ( A 2 | Q 2 ) ( g ) H ( A 1 | Q 1 ) + H ( A 2 | Q 2 ) ( h ) 2 H ( A 1 | Q 1 ) ( i ) = 2 I ( X ; A 1 , Q 1 ) ECE 563, Fall 2011 Homework 2 Solution Parts of the solutions are copied from the solution manual of Cover and Thomas. Question 1:

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
(a) Chain rule. (b) X and Q 1 are independent. (c) Q 2 is independent of X , Q 1 , and A 1 . (d) A 2 is completely determined given Q 2 and X . (e) Conditioning decreases entropy. (f) Expand the entropy. (g) A 1 is completely determined given Q 1 and X . (h) Identically distributed. (i) Invoke the result of part (a). In the above solution, the mutual information chain rule was applied directly. Here is an alternative solution which some might prefer. Starting over, we want to prove that I ( X ; Q 1 , A 1 , Q 2 , A 2 ) 2 I ( X ; Q 1 , A 1 ) .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 9

hw2_soln_fall2011 - Question 1 ECE 563 Fall 2011 Homework 2...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online