ECE514_S11_HW2_solution

ECE514_S11_HW2_solution - 1. 1) The entropy of the source...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1. 1) The entropy of the source is 2.3549 2) The Huffman codes for the source are Symbol Probability Huffman Code X1 X2 X3 X4 X5 X6 0.1 0.3 0.05 0.09 0.21 0.25 010 11 0110 0111 00 10 The average length of the Huffman code is 2.38 And the efficiency of the Huffman code is 0.9894 3) The Huffman codes for the new source sequences of length 2 are Symbol X1X1 X1X2 X1X3 X1X4 X1X5 X1X6 X2X1 X2X2 X2X3 X2X4 X2X5 X2X6 X3X1 X3X2 X3X3 X3X4 X3X5 X3X6 Probability 0.0100 0.0300 0.0050 0.0090 0.0210 0.0250 0.0300 0.0900 0.0150 0.0270 0.0630 0.0750 0.0050 0.0150 0.0025 0.0045 0.0105 0.0125 Code 1110000 1110 10110111 1011001 111001 00101 0111 000 011010 00111 1001 1100 11101110 011011 111011110 111011111 1110001 001000 Symbol X4X1 X4X2 X4X3 X4X4 X4X5 X4X6 X5X1 X5X2 X5X3 X5X4 X5X5 X5X6 X6X1 X6X2 X6X3 X6X4 X6X5 X6X6 Probability 0.0090 0.0270 0.0045 0.0081 0.0189 0.0225 0.0210 0.0630 0.0105 0.0189 0.0441 0.0525 0.0250 0.0750 0.0125 0.0225 0.0525 0.0625 Code 1011010 01100 10110110 1011000 101110 111110 111010 1010 1110110 101111 11110 0100 00110 1101 001001 111111 0101 1000 The efficiency of the Huffman code is 0.9932 So, the efficiency of the source code for sequences of length 2 is higher than that for one symbol source. 4) In the same way, the Huffman code for source sequences of length 3 can be generated. Run ece514_hw2_sol.m to see the codes. The new source is in `p_3' and the corresponding Huffman codes are in `h_code_3'. The efficiency of the Huffman code is 0.9958 Compare the efficiency of this code with those in part 2 and 3, it can be discovered that efficiency of the source code for sequences of length 3 is the highest, so the coding efficiency increase with source length, but the efficiency improvement decreases with the length, and the coding complexity increases significantly at the same time. 2. For any two distribution P and Q on the alphabet U, Show that 1) 2) if Solution: 1) Since 1 1 So, 1 1 0 1 1 0 ,0 1 1 2) If , and the random variable a ,a ,...,a with probability take value from a finite alphabet set 1,2, ... , , so, a , 1 0 And 1 With the conclusion in part 1) 0 ...
View Full Document

Ask a homework question - tutors are online