This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Asymptotic Equipartition Property and Data Compression Exercises Exercise 3.3 : The AEP and source coding . A discrete memoryless source emits a sequence of statistically independent binary digits with probabilities p (1) = 0 . 005 and p (0) = 0 . 995. The digits are taken 100 at a time and a binary codeword is provided for every sequence of 100 digits containing three or fewer ones. (a) Assuming that all codewords are the same length, find the minimum length required to provide codewords for all sequences with three or fewer ones. (b) Calculate the probability of observing a source sequence for which no codeword has been assigned. Solution : (a) The number of sequences of 100 digits containing three or few ones is given by N = 100 + 100 1 + 100 2 + 100 3 = 1 + 100 + 4980 + 161700 = 166751 (1) The minimum length required to encode these sequences is given by d log 2 N e = d 17 . 34731 e = 18. (b) The probablity of observing a sequence which has an assigned codeword is given by: P = 1 . 995 100 + 100 . 995 99 . 005 + 4980 . 995 98 . 005 2 + 161700 . 995 97 . 005 3 = 0 . 9983 (2) Hence the probability of observing a sequence which has no codeword is 0 . 0017. Exercise 5.4 : Huffman Coding . Consider the random variable X = x 1 x 2 x 3 x 4 x 5 x 6 x 7 . 49 . 26 . 12 . 04 . 04 . 03 . 02 (3) (a) Find a binary Huffman code for X . (b) Find the expected codelength for this encoding. 1 (c) Find a ternary Huffman code for X (a ternary code is one which uses three symbols, e.g. { , 1 , 2 } , instead of a binary codes two symbols { , 1 } ). Solution : (a) Using the diagram in Figure 1, the Huffman code for X is given in Table 1. x1 x2 x3 x4 x5 x6 x7 0.49 0.26 0.12 0.04 0.04 0.03 0.02 0.05 1 1 0.08 0.13 1 0.25 1 1 0.51 1.00 1 Figure 1: Diagram for designing the binary Huffman code for X in Exercise 5.4....
View Full
Document
 Spring '10
 sd

Click to edit the document details