This preview shows page 1. Sign up to view the full content.
Unformatted text preview: p ( b ). Does the entropy decrease or increase due to the merging operation? 7. Entropy after Mixing Symbols. Consider what happens to the entropy when two symbols (say, a and b ) are mixed, i.e., the probability of symbols a and b are both replaced by the average ( p ( a )+ p ( b )) / 2. Does the entropy decrease or increase due to the mixing operation? 8. Maximizing Entropy. Let X be a random variable taking values in the set { 1 , 2 , . . . , m } . Suppose it is known that P { X = 1 } = p 1 . (a) What is max H ( X ) subject to this constraint? (b) What probability mass function achieves this maximum?...
View
Full
Document
This note was uploaded on 11/17/2010 for the course ECE 650 taught by Professor Niu during the Spring '10 term at Illinois Tech.
 Spring '10
 Niu

Click to edit the document details