hw6sol - EE 376B Handout #16 Information Theory Tuesday,...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE 376B Handout #16 Information Theory Tuesday, May 31, 2011 Prof. T. Cover Solutions to Homework Set #6 1. One bit quantization of a single Gaussian random variable Let X N (0 , 2 ) and let the distortion measure be squared error. Here we do not allow block descriptions. Show that the optimum reproduction points for 1 bit quantization are 2 , and that the expected distortion for 1 bit quantization is 2 2 . Compare this with the distortion rate bound D = 2 2 2 R for R = 1. Solution: One bit quantization of a single Gaussian random variable With one bit quantization, the obvious reconstruction regions are the positive and negative real axes. Proving that such is optimal needs a bit of care, which we will skip here. The reconstruction point is the centroid of each region. For example, for the positive real line, the centroid a is given as a = x 2 2 2 e x 2 2 2 dx = 2 e y dy = 2 , using the substitution y = x 2 / 2 2 . The expected distortion for one bit quantization is D = x + 2 2 1 2 2 e x 2 2 2 dx + x- 2 2 1 2 2 e x 2 2 2 dx = 2 ( x 2 + 2 2 ) 1 2 2 e x 2 2 2 dx- 2 - 2 x 2 1 2 2 e x 2 2 2 dx = 2 + 2 2- 4 1 2 2 2 1 = 2 - 2 . 3634 2 , which is significantly larger than the distortion rate bound D (1) = 2 / 4. 2. Rate distortion function with infinite distortion Find the rate distortion function R ( D ) = min I ( X ; X ) for X Bernoulli ( 1 2 ) and distortion d ( x, x ) = , x = x, 1 , x = 1 , x = 0 , , x = 0 , x = 1 , where x, x { , 1 } . Thus reconstructing a 0 by a 1 is never allowed. Solution: Rate distortion function with infinite distortion We wish to evaluate the rate distortion function R ( D ) = min p ( x | x ): ( x, x ) p ( x ) p ( x | x ) d ( x, x ) D I ( X ; X ) ....
View Full Document

Page1 / 6

hw6sol - EE 376B Handout #16 Information Theory Tuesday,...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online