Module5_2 - Module 5, Lecture 2 Data Compression: Huffman...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
Module 5, Lecture 2 Data Compression: Huffman and Arithmetic Coding G.L. Heileman Module 5, Lecture 2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Huffman Coding For a given RV X , assuming knowledge of only p 1 , . . . , p |X| , Huffman coding will produce an optimal prefix code, C Huf in terms of expected codeword length, L ( C Huf ). That is, no other coding technique, using only p 1 , . . . , p , can produce a better prefix code C , such that L ( C ) < L ( C Huf ). Huffman derived his algorithm based on two observations regarding optimal prefix codes: (1) In an optimal code, symbols with higher probability will have shorter codewords than symbols with lower probability (obvious). (2) In an optimal code, the two symbols with lowest probability will have the same length (not obvious). G.L. Heileman Module 5, Lecture 2
Background image of page 2
Huffman Coding To see why observation (2) is true: Suppose we have an optimal code C in which the two least probable symbols do not have the same length. Since these are the two least probable symbols, by observation (1), no other codeword can be longer these codewords. Assume the longer codeword is k bits longer than the shorter codeword. By simply dropping the last k bits of the longer codeword, we end up with a new prefix code (the new codeword will not become the prefix of any other codeword) in which the two symbols with lowest probability will have the same length. The new prefix code will have a smaller expected codeword length than C . This contradicts our assumption that C is optimal, and establishes the truth of observation (2). G.L. Heileman Module 5, Lecture 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Huffman Coding Lemma Given RV X p ( x ) with alphabet X , let x , y ∈ X be two characters with the smallest probabilities. Then there exists an optimal prefix code C for X in which C ( x ) and C ( y ) are siblings with maximal depth in the code tree. If x and y are siblings in a code tree, since this is a prefix code, they will differ only in their LSBs. Observations (1) and (2), along with the truth of this Lemma, are the basis for the Huffman coding procedure. G.L. Heileman Module 5, Lecture 2
Background image of page 4
Huffman Coding Proof of Lemma: Let T be a tree representing an optimal prefix code, where C ( a ) and C ( b ), a , b ∈ X are the deepest sibling leaves in T . Assume p ( a ) p ( b ), p ( x ) p ( y ), and C ( x ) and C ( y ) do not have maximal depth in T : T x a b y T T' x a b y x a b y Then exchange C ( x ) and C ( a ) to produce a new tree T 0 . Because T is optimal L ( T ) = i p i l T i = L ? , and L ? - L ( T 0 ) = X i p i l T i - X i p i l T 0 i = p ( x ) l T ( x ) + p ( a ) l T ( a ) - p ( x ) l T 0 ( x ) - p ( a ) l T 0 ( a ) G.L. Heileman Module 5, Lecture 2
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Huffman Coding Proof of Lemma (con’t): Thus, L ( T 0 ) L ? , with equality when p ( a ) = p ( x ) or l T ( a ) = l T ( x ). Since L ? is optimal, it must be the case that L ( T 0 ) = L ? , and therefore L ( T 0 ) is an optimal code. Similarly, by exchanging the codewords for y and b we cannot increase the expected length of the code in a new tree T 00 . Thus T 00 is an optimal tree in which C ( x ) and C ( y ) appear as sibling leaves having maximal depth. G.L. Heileman Module 5, Lecture 2
Background image of page 6
Huffman Coding Given RV X p ( x ).
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 05/06/2010 for the course ECE 549 taught by Professor G.l.heileman during the Spring '10 term at University of New Brunswick.

Page1 / 30

Module5_2 - Module 5, Lecture 2 Data Compression: Huffman...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online