hw5sol - EE 376A/Stat 376A Handout #17 Information Theory...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE 376A/Stat 376A Handout #17 Information Theory Tuesday, February 15, 2011 Prof. T. Cover Solutions to Homework Set #5 1. Bad codes. Which of these codes cannot be Huffman codes for any probability assignment? (a) { 1 , 01 , 00 } . (b) { 00 , 01 , 10 , 110 } . (c) { 01 , 10 } . Solution: Bad codes. (a) { 1,01,00 } is a Huffman code for the distribution ( 1 2 , 1 4 , 1 4 ). (b) The code { 00,01,10, 110 } can be shortened to { 00,01,10, 11 } without losing its instantaneous property, and therefore is not optimal, so it cannot be a Huffman code. Alternatively, it is not a Huffman code because there is a unique longest codeword. (c) The code { 01,10 } can be shortened to { 0,1 } without losing its instantaneous property, and therefore is not optimal and not a Huffman code. 2. Huffman coding. Consider the random variable X = ( x 1 x 2 x 3 x 4 x 5 x 6 x 7 . 50 0 . 26 0 . 11 0 . 04 0 . 04 0 . 03 0 . 02 ) (a) Find a binary Huffman code for X. (b) Find the expected codelength for this encoding. (c) Find a ternary Huffman code for X. Solution: Huffman coding. 1 (a) The Huffman tree for this distribution is Codeword 1 x 1 0.50 0.50 0.50 0.50 0.50 0.50 1 01 x 2 0.26 0.26 0.26 0.26 0.26 0.50 001 x 3 0.11 0.11 0.11 0.11 0.24 00011 x 4 0.04 0.04 0.08 0.13 00010 x 5 0.04 0.04 0.05 00001 x 6 0.03 0.05 00000 x 7 0.02 (b) The expected length of the codewords for the binary Huffman code is 2 bits. ( H ( X ) = 1 . 99 bits) (c) The ternary Huffman tree is Codeword x 1 0.50 0.50 0.50 1.0 1 x 2 0.26 0.26 0.26 20 x 3 0.11 0.11 0.24 21 x 4 0.04 0.04 222 x 5 0.04 0.09 221 x 6 0.03 220 x 7 0.02 This code has an expected length 1.33 ternary symbols. ( H 3 ( X ) = 1 . 25 ternary symbols). 3. Codes. Let X 1 ,X 2 ,..., i.i.d. with X = 1 , with probability 1 / 2 2 , with probability 1 / 4 3 , with probability 1 / 4 . Consider the code assignment C ( x ) = , if x = 1 01 , if x = 2 11 , if x = 3 ....
View Full Document

Page1 / 7

hw5sol - EE 376A/Stat 376A Handout #17 Information Theory...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online