Lecture9 - Colorado State University, Ft. Collins ECE 516:...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Colorado State University, Ft. Collins Fall 2008 ECE 516: Information Theory Lecture 9 September 25, 2008 Recap: Write () = = X 1 0 i i i e p α , where { } i e are the set of linear independent eigenvectors, i e corresponding to eigenvalue i λ . Assume L > = 2 1 1 . Then, ()() () () = = = = = X X 1 1 0 i i i i i i T i T n e e P p P p n n n As n , 1 1 e p = n . (scaling is not important) If a Markov chain is irreducible and aperiodic, and if it is run for a long time, so μ p n , then it us a stationary ergodic process. () () ( ) ( ) ∑∑ ∈∈ = = = = = XX i ij j ij i n n n n n n n P P X X H X X H X X X H H H 1 log | lim | lim , , | lim 1 2 1 1 1 μ L X X () ( ) X log n X H H X 5 Data Compression (Chp. 5) 5.1 Example of Codes Definition: A source code C for a r.v. X is a mapping from X to * D , the finite length strings of symbols from a D-ary alphabet. Definition: The expected length ( ) C L of the source code x C for a r.v. X with x p () ( ) ( ) = X x x l x p C L Definition: A code is non-singular if every element of X maps into a different string in * D , i.e., ( ) ( ) j i j i x C x C x x
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Definition: An extension * C of a code C is a mapping from finite length strings of X to finite length strings of * D () ( ) ( ) ( ) n n x C x C x C x x x C L L 2 1 2 1 = Definition: A code is uniquely decodable if its extension is non-singular. Definition: A code is called a prefix code (short for prefix-free code) if no codeword is a prefix of any other codeword. Prefix codes are instantaneously decodable. All codes can be represented on a D-ary tree with some nodes equal to codewords, and depth of the tree equals to the length of the longest codeword. To be prefix: no code is an ancestor of another code. Prefix codes can be represented by intervals in [ ] 1 , 0 . Consider an i th codeword of length i l : i l y y y , , , 2 1 L where D y j We can represent a codeword i l y y y , , , 2 1 L by an interval [ ) i i i l l l D y y y y y y + L L 2 1 2 1 . 0 , . 0 5.2 Kraft Inequality Theorem: (Kraft Inequality) For any instantaneous (prefix) code over an alphabet of size D , the codeword lengths, m l l l , , , 2 1 L must satisfy the inequality 1 i l i D Conversely, given a set of codeword lengths m l l l , , , 2 1 L that satisfies this inequality, there exists an instantaneous code with these code lengths.
Background image of page 2
3 Utility: If 2 = D , you cannot have a code with lengths 1, 1, 2. Because 1 25 . 1 4 1 2 1 2 1 2 2 2 2 1 1 > = + + = + + But, you can have a code with lengths 1, 2, 2, because 1 1 4 1 4 1 2 1 2 2 2 2 2 1 = + + = + + Proof: Consider D-ary tree representation. Leaves represent codewords.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/17/2010 for the course ECE 516 taught by Professor Rocky during the Spring '08 term at Colorado State.

Page1 / 10

Lecture9 - Colorado State University, Ft. Collins ECE 516:...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online