hw1[1]

hw1[1] - ECE255AN Fall 2011 Homework set 1, due Tue 10/4 ¯...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE255AN Fall 2011 Homework set 1, due Tue 10/4 ¯ 1. Use first principles to show that L( 1 , 1 , 1 ) = 244 3 2 ¯ and L( 1 , 1 , 1 , 1 ) = 2. 4444 2. For n ≥ 2, let Pn = ( 1 , 1 , . . . , 2n1 1 , 21 , 21 ). Find the entropy and an optimal code for n n − 24 (a) Pn , 1 (b) limn→∞ Pn = ( 1 , 4 , 1 , · · · ). 2 8 3. For all r.v.’s X and Y , not necessarily independent, is H (X ) ≤ H (X + Y )? 4. For α ∈ R, let an = 1 n(log n)α and A = ∞ n=2 an . (a) For which values of α is A finite? (b) For finite A, let pn = (c) For β ∈ R, let bn = an A. For which values of α is H (p1 , p2 , . . .) finite? 1 n(log n)2 (log log n)β and B = ∞ n=4 bn . Repeat part (b) for bn and B . ¯ 5. [After your TA’s question last year] In class we saw that for all X , L(X ) < H (X ) + 1. Prove ¯ (X ) ≤ H (X ) . or disprove the stronger statement L 6. Calculate and simplify H (p, 1−p , , 1−p , , 1−p ). Explain your answer. 2 4 4 7. Show that entropy is concave in the underlying distribution. 8. Chapter 2, problem 21. 9. Chapter 5, problems 14, 16, 22. 1 ...
View Full Document

Ask a homework question - tutors are online