Hw8sol - EE 376B/Stat 376B Information Theory Prof T Cover Handout#27 Tuesday June 6 2006 Solutions to Homework Set#8 1 Universal data compression

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376B/Stat 376B Handout #27 Information Theory Tuesday, June 6, 2006 Prof. T. Cover Solutions to Homework Set #8 1. Universal data compression. Consider three possible source distributions on X , P a = (0 . 7 , 0 . 2 , 0 . 1) , P b = (0 . 1 , 0 . 7 , 0 . 2) , P c = (0 . 2 , 0 . 1 , 0 . 7) . (a) Find the minimum incremental cost of compression D * = min P max θ D ( P θ || P ) , the associated mass function P = ( p 1 ,p 2 ,p 3 ), and ideal codeword lengths l i = log(1 /p i ). (b) What is the channel capacity of a channel matrix with rows P a , P b , P c ? Solution: Universal data compression. Since D * = C, where C is the capacity of a channel defined by a channel matrix with rows P a ,P b ,P c , and the uniform input distribution p * = (1 / 3 , 1 / 3 , 1 / 3) achieves the capacity C , D * = C = log 3 - H (0 . 1 , 0 . 7 , 0 . 2) . 4282 . Thus, the cost of not knowing the true source distribution is slightly less than half a bit per symbol. 2.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 06/10/2008 for the course ECE 376B taught by Professor Tomcover during the Spring '05 term at Stanford.

Page1 / 3

Hw8sol - EE 376B/Stat 376B Information Theory Prof T Cover Handout#27 Tuesday June 6 2006 Solutions to Homework Set#8 1 Universal data compression

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online