Lecture11

# Lecture11 - Colorado State University Ft Collins ECE 516...

This preview shows pages 1–4. Sign up to view the full content.

1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 11 October 2, 2008 Recap: Theorem: (McMillan) The codeword lengths of any uniquely decodable code must satisfy the Kraft inequality 1 i l i D Conversely, given a set of lengths that satisfy this inequality, then it is possible to construct a uniquely decodable code with these lengths. (Indeed, can construct pre- fix code) 5.6 Huffman Codes Huffman’s Algorithm an algorithm that finds optimum prefix code. Example: PMF () 1 . 0 , 15 . 0 , 2 . 0 , 2 . 0 , 35 . 0 , 2 = D 0 1 0 1 0 1 0.2 0 1 0.2 0.4 0.15 0.1 0.35 0.6 101 100 01 00 11 5 4 3 2 1 25 . 2 = L bits, ( ) 2 . 2 = X H bits Every step eliminates D -1 symbols, we want the total number of symbols be 1 1 + D k , i.e., we want the total number of symbols to be 1 mod D -1. Theorem: Huffman coding is optimal, i.e., if C is the Huffman code and C’ is any other (uniquely decodable) code, then ( ) ( ) C L C L .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Lemma: Given D =2. An optimum (instantaneous) code satisfies, 1. If k j p p > then k j l l 2. The two longest codewords have the same length. 3. Two longest codewords differ only in the last bit. Shannon-Fano-Elias Coding Shannon-Fano-Elias gives a codebook construction scheme that is based on the CDF. Given PMF, the CDF is ( ) ( ) = x a a p x F Modified CDF () () ( ) 2 x p a p x F x a + = < Code {} i x using () i x F . In general, ( ) i x F are real numbers. Let 1 1 log first + = i i p y digits of ( ) i x F (codeword of i x ) i y is of length 1 1 log + i p .
3 6 Universal Source Coding (Chapter 13) Arithmetic Coding Motivation: Optimal average length is achieved by encoding a block of source symbols. But, as the block length grows, the complexity of encoding increases. We want to use big block, but don’t want to involve sophisticated encoding and decoding. Use Shannon-Fano-Elias coding to encode a (long) sequence of symbols.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 03/17/2010 for the course ECE 516 taught by Professor Rocky during the Spring '08 term at Colorado State.

### Page1 / 9

Lecture11 - Colorado State University Ft Collins ECE 516...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online