soln_1_229Aspr07

soln_1_229Aspr07 - EECS 229A * Homework 1 solutions. Spring...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
EECS 229A Spring 2007 * * Homework 1 solutions. 1. (a) Determine the entropy of the following probability distributions on a set of size 3 : p = ( 1 6 , 1 2 , 1 3 ) , q = ( 1 5 , 0 , 4 5 ) , r = ( 89 450 , 1 30 , 173 225 ) . (b) Notice that r is a convex combination of p and q . Is your answer consistent with the concavity of entropy as a function of probability distributions ? Solution : (a) H ( p ) = 1 6 log 2 6 + 1 2 log 2 2 + 1 3 log 2 3 = 1 . 46 H ( q ) = 1 5 log 2 5 + 4 5 log 2 5 4 = 0 . 722 H ( r ) = 89 450 log 2 450 89 + 1 30 log 2 30 + 173 225 log 2 225 173 = 0 . 9175 (b) r = 1 15 p + 14 15 q . Since H ( r ) = 0 . 9175 > 0 . 7712 = 1 15 H ( p ) + 14 15 H ( q ) the answer is consistent with the concavity of the entropy function. 2. X and Y are random variables taking values in finite sets X and Y respectively. You are given that H ( X ) = 11 and that H ( X | Y ) = H ( Y | X ). A Lucent engineer claims that | Y |≥ 3. Is this true or false ? Either prove the claim or give a counterexample. Solution
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/25/2010 for the course ECE 544 taught by Professor Liu during the Spring '10 term at Ill. Chicago.

Page1 / 3

soln_1_229Aspr07 - EECS 229A * Homework 1 solutions. Spring...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online