hw1[1]

# hw1[1] - ECE255AN Fall 2011 Homework set 1 due Tue 10/4 ¯...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE255AN Fall 2011 Homework set 1, due Tue 10/4 ¯ 1. Use ﬁrst principles to show that L( 1 , 1 , 1 ) = 244 3 2 ¯ and L( 1 , 1 , 1 , 1 ) = 2. 4444 2. For n ≥ 2, let Pn = ( 1 , 1 , . . . , 2n1 1 , 21 , 21 ). Find the entropy and an optimal code for n n − 24 (a) Pn , 1 (b) limn→∞ Pn = ( 1 , 4 , 1 , · · · ). 2 8 3. For all r.v.’s X and Y , not necessarily independent, is H (X ) ≤ H (X + Y )? 4. For α ∈ R, let an = 1 n(log n)α and A = ∞ n=2 an . (a) For which values of α is A ﬁnite? (b) For ﬁnite A, let pn = (c) For β ∈ R, let bn = an A. For which values of α is H (p1 , p2 , . . .) ﬁnite? 1 n(log n)2 (log log n)β and B = ∞ n=4 bn . Repeat part (b) for bn and B . ¯ 5. [After your TA’s question last year] In class we saw that for all X , L(X ) < H (X ) + 1. Prove ¯ (X ) ≤ H (X ) . or disprove the stronger statement L 6. Calculate and simplify H (p, 1−p , , 1−p , , 1−p ). Explain your answer. 2 4 4 7. Show that entropy is concave in the underlying distribution. 8. Chapter 2, problem 21. 9. Chapter 5, problems 14, 16, 22. 1 ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online