soln_1_229Aspr07

# soln_1_229Aspr07 - EECS 229A Homework 1 solutions Spring...

This preview shows pages 1–2. Sign up to view the full content.

EECS 229A Spring 2007 * * Homework 1 solutions. 1. (a) Determine the entropy of the following probability distributions on a set of size 3 : p = ( 1 6 , 1 2 , 1 3 ) , q = ( 1 5 , 0 , 4 5 ) , r = ( 89 450 , 1 30 , 173 225 ) . (b) Notice that r is a convex combination of p and q . Is your answer consistent with the concavity of entropy as a function of probability distributions ? Solution : (a) H ( p ) = 1 6 log 2 6 + 1 2 log 2 2 + 1 3 log 2 3 = 1 . 46 H ( q ) = 1 5 log 2 5 + 4 5 log 2 5 4 = 0 . 722 H ( r ) = 89 450 log 2 450 89 + 1 30 log 2 30 + 173 225 log 2 225 173 = 0 . 9175 (b) r = 1 15 p + 14 15 q . Since H ( r ) = 0 . 9175 > 0 . 7712 = 1 15 H ( p ) + 14 15 H ( q ) the answer is consistent with the concavity of the entropy function. 2. X and Y are random variables taking values in ﬁnite sets X and Y respectively. You are given that H ( X ) = 11 and that H ( X | Y ) = H ( Y | X ). A Lucent engineer claims that | Y |≥ 3. Is this true or false ? Either prove the claim or give a counterexample. Solution

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 3

soln_1_229Aspr07 - EECS 229A Homework 1 solutions Spring...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online