# prob - 5 Huffman 20 Questions there are 2 n possible...

This preview shows pages 1–4. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 5 Huffman 20 Questions there are 2 n possible sequences. Let b k be the bit representation of the number k(in decimal). Let b ki be the i th bit(0 or 1),starting from the LSB for the number k So the probability of the k th combination(of defective and good) would be P n k = n Y i =1 p b ki i (1- p i ) 1- b ki In general let P j k = j Y i =1 p b ki i (1- p i ) 1- b ki 3 We can write the entropy of the distribution to be- 2 n X k =1 P n k log 2 ( P n k ) It is a simple observation that P n k = p n P n- 1 k ( mod )2 n if k ≤ 2 n- 1 (1- p n ) P n- 1 k ( mod )2 n if k > 2 n- 1 So the entropy can be rewritten as- 2 n X k =1 P n k log 2 ( P n k ) =- 2 n- 1 X k =1 p n P n- 1 k log( p n P n- 1 k )- 2 n- 1 X k =1 (1- p n ) P n- 1 k log((1- p n ) P n- 1 k ) which can be written as- 2 n X k =1 P n k log 2 ( P n k ) =- p n log( p n )- (1- p n ) log(1- p n )- 2 n- 1 X k =1 P n- 1 k log 2 ( P n- 1 k ) This is a recursion and the obvious solution to the recursion is- n X i =1 ( p i log( p i ) + (1- p i ) log(1- p i )) So the entropy is a good lower bound. Hence the answer for part (a) is- n X i =1 ( p i log( p i ) + (1- p i ) log(1- p i )) 5.1 Part b If we do huffman coding,the longest question will be the one that has got the least probability. Clearly the fact that ”all objects are defective” has the lowest probability. The second lowest probability is for the fact that all objects,except the n th object are defective. So our last question to nature(god?) would be ”is the n th object good atleast? ” 4 5.2 Part c Clearly the upper bound will be attained when we are forced to assign integer lengths to all...
View Full Document

{[ snackBarMessage ]}

### Page1 / 8

prob - 5 Huffman 20 Questions there are 2 n possible...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online