kraft-mcmillan - Proof of Kraft-McMillan theorem Nguyen...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Proof of Kraft-McMillan theorem * Nguyen Hung Vu 16 2 24 1 Kraft-McMillan theorem Let C be a code with n codewords with lenghth l 1 , l 2 , ..., l N . If C is uniquely decodable, then K ( C ) = N X i =1 2 - l i 1 (1) 2 Proof The proof works by looking at the nth power of K ( C ) . If K ( C ) is greater than one, then K ( C ) n should grw expoentially with n. If it does not grow expoentially with n, then this is proof that P N i =1 2 - l i 1 . Let n be and arbitrary integer. Then h P N i =1 2 - l i i n = = P N i 1 =1 2 - l i 1 · · · P N i 2 =1 2 - l i 2 ” “ P N i N =1 2 - l in and then " N X i =1 2 - l i # n = N X i 1 =1 N X i 2 =1 · · · N X i n =1 2 - ( l i 1 + i 2 + ··· + in ) (2) The exponent l i 1 + l i 2 + · · · + l i n is simply the length of n codewords from the code C . The smallest value this exponent can take is greater than or equal to n, which would be the case if codwords were 1 bit long. If l = max { l 1 , l 2 , · · · , l N } then the largest value that the exponent can take is less than or equal to nl . Therefore, we can write this summation as K ( C ) n = nl X n A k 2 - k where A k is the number of combinations of n codewords that have a combined length of k. Let’s take a look at the size of this coeficient. The number of possibledistinct binary sequences of length k is 2 k . If this code is uniquely decodable, then each se- quence can represent one and only one sequence of codewords. Therefore, the number of possible combinations of codewords whose combined length is k cannot be greater than 2 k . In other words, A k 2 k . * Introduction to Data Compression, Khalid Sayood email: [email protected] This means that K ( C ) n = nl X k = n A k 2 - k nl X k = n 2 k 2 - k = nl - n + 1 (3) But K ( C ) is greater than one, it will grow exponentially with n, while n ( l - 1)+1 can only grow linearly, So if K ( C ) is greater than one, we can always find an n large enough that the inequal- ity (3) is violated. Therefore, for an uniquely decodable code C , K ( C ) is less than or equal ot one. This part of of Kraft-McMillan inequality provides a necces- sary condition for uniquely decodable codes. That is, if a code is uniquely decodable, the codeword lengths have to satisfy the inequality. 3 Construction of prefix code Given a set of integers l 1 , l 2 , · · · , l N that satisfy the inequality N X i =1 2 - l i 1 (4) we can always find a prefix code with codeword lengths l 1 , l 2 , · · · , l N . 4 Proof of prefix code constructing theorem We will prove this assertion by developing a procedure for con- structing a prefix code with codeword lengths l 1 , l 2 , · · · , l N that satisfy the given inequality. Without loss of generality, we can assume that l 1 l 2 ≤ · · · ≤ l N . (5) Define a sequence of numbers w 1 , w 2 , · · · , w N as follows: w 1 = 0 w j = P j - 1 i =1 2 l j - l i j > 1 . The binary representation of w j for j > 1 would take up d log 2 w j e bits. We will use this binary representation to construct a prefix code. We first note that the number of bits in the binary representation of w j is less than or equal to l j . This is obviously true for w 1 . For j > 1 , log 2 w j = log 2 h P j - 1 i =1 2 l j
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern