This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE 376B Handout #16 Information Theory Tuesday, May 31, 2011 Prof. T. Cover Solutions to Homework Set #6 1. One bit quantization of a single Gaussian random variable Let X N (0 , 2 ) and let the distortion measure be squared error. Here we do not allow block descriptions. Show that the optimum reproduction points for 1 bit quantization are 2 , and that the expected distortion for 1 bit quantization is 2 2 . Compare this with the distortion rate bound D = 2 2 2 R for R = 1. Solution: One bit quantization of a single Gaussian random variable With one bit quantization, the obvious reconstruction regions are the positive and negative real axes. Proving that such is optimal needs a bit of care, which we will skip here. The reconstruction point is the centroid of each region. For example, for the positive real line, the centroid a is given as a = x 2 2 2 e x 2 2 2 dx = 2 e y dy = 2 , using the substitution y = x 2 / 2 2 . The expected distortion for one bit quantization is D = x + 2 2 1 2 2 e x 2 2 2 dx + x 2 2 1 2 2 e x 2 2 2 dx = 2 ( x 2 + 2 2 ) 1 2 2 e x 2 2 2 dx 2  2 x 2 1 2 2 e x 2 2 2 dx = 2 + 2 2 4 1 2 2 2 1 = 2  2 . 3634 2 , which is significantly larger than the distortion rate bound D (1) = 2 / 4. 2. Rate distortion function with infinite distortion Find the rate distortion function R ( D ) = min I ( X ; X ) for X Bernoulli ( 1 2 ) and distortion d ( x, x ) = , x = x, 1 , x = 1 , x = 0 , , x = 0 , x = 1 , where x, x { , 1 } . Thus reconstructing a 0 by a 1 is never allowed. Solution: Rate distortion function with infinite distortion We wish to evaluate the rate distortion function R ( D ) = min p ( x  x ): ( x, x ) p ( x ) p ( x  x ) d ( x, x ) D I ( X ; X ) ....
View Full
Document
 '11
 cover

Click to edit the document details