# hw1 - p ( b ). Does the entropy decrease or increase due to...

This preview shows page 1. Sign up to view the full content.

ECE 563 Fall 2010 August 24, 2010 HOMEWORK ASSIGNMENT 1 Due Date: September 2, 2010 (in class) 1. Problem 2.1 on page 43 of text. 2. Problem 2.2 on page 44 of text. (Also look at Problem 5 on page 43. You don’t have to turn in the solution to problem 5.) 3. Problem 2.11 on page 45 of text. 4. Problem 2.12 on page 46 of text. 5. Problem 2.14 on page 46 of text. 6. Entropy after Merging Symbols. Consider what happens to the entropy when two symbols (say, a and b ) in the alphabet are merged into a new symbol (say, c ), i.e., the size of the alphabet is reduced by 1, and p ( c ) = p ( a ) +
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: p ( b ). Does the entropy decrease or increase due to the merging operation? 7. Entropy after Mixing Symbols. Consider what happens to the entropy when two sym-bols (say, a and b ) are mixed, i.e., the probability of symbols a and b are both replaced by the average ( p ( a )+ p ( b )) / 2. Does the entropy decrease or increase due to the mixing operation? 8. Maximizing Entropy. Let X be a random variable taking values in the set { 1 , 2 , . . . , m } . Suppose it is known that P { X = 1 } = p 1 . (a) What is max H ( X ) subject to this constraint? (b) What probability mass function achieves this maximum?...
View Full Document

## This note was uploaded on 11/17/2010 for the course ECE 650 taught by Professor Niu during the Spring '10 term at Illinois Tech.

Ask a homework question - tutors are online