This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 5 Huffman 20 Questions there are 2 n possible sequences. Let b k be the bit representation of the number k(in decimal). Let b ki be the i th bit(0 or 1),starting from the LSB for the number k So the probability of the k th combination(of defective and good) would be P n k = n Y i =1 p b ki i (1 p i ) 1 b ki In general let P j k = j Y i =1 p b ki i (1 p i ) 1 b ki 3 We can write the entropy of the distribution to be 2 n X k =1 P n k log 2 ( P n k ) It is a simple observation that P n k = p n P n 1 k ( mod )2 n if k ≤ 2 n 1 (1 p n ) P n 1 k ( mod )2 n if k > 2 n 1 So the entropy can be rewritten as 2 n X k =1 P n k log 2 ( P n k ) = 2 n 1 X k =1 p n P n 1 k log( p n P n 1 k ) 2 n 1 X k =1 (1 p n ) P n 1 k log((1 p n ) P n 1 k ) which can be written as 2 n X k =1 P n k log 2 ( P n k ) = p n log( p n ) (1 p n ) log(1 p n ) 2 n 1 X k =1 P n 1 k log 2 ( P n 1 k ) This is a recursion and the obvious solution to the recursion is n X i =1 ( p i log( p i ) + (1 p i ) log(1 p i )) So the entropy is a good lower bound. Hence the answer for part (a) is n X i =1 ( p i log( p i ) + (1 p i ) log(1 p i )) 5.1 Part b If we do huffman coding,the longest question will be the one that has got the least probability. Clearly the fact that ”all objects are defective” has the lowest probability. The second lowest probability is for the fact that all objects,except the n th object are defective. So our last question to nature(god?) would be ”is the n th object good atleast? ” 4 5.2 Part c Clearly the upper bound will be attained when we are forced to assign integer lengths to all...
View
Full
Document
 Spring '09
 B.K.Dey
 Probability theory, probability density function

Click to edit the document details