{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

hw6sol - EE 376B Information Theory Prof T Cover Handout#16...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376B Handout #16 Information Theory Tuesday, May 31, 2011 Prof. T. Cover Solutions to Homework Set #6 1. One bit quantization of a single Gaussian random variable Let X ∼ N (0 , σ 2 ) and let the distortion measure be squared error. Here we do not allow block descriptions. Show that the optimum reproduction points for 1 bit quantization are ± 2 π σ , and that the expected distortion for 1 bit quantization is π 2 π σ 2 . Compare this with the distortion rate bound D = σ 2 2 2 R for R = 1. Solution: One bit quantization of a single Gaussian random variable With one bit quantization, the obvious reconstruction regions are the positive and negative real axes. Proving that such is optimal needs a bit of care, which we will skip here. The reconstruction point is the centroid of each region. For example, for the positive real line, the centroid a is given as a = 0 x 2 2 πσ 2 e x 2 2 σ 2 dx = 0 σ 2 π e y dy = σ 2 π , using the substitution y = x 2 / 2 σ 2 . The expected distortion for one bit quantization is D = 0 −∞ x + σ 2 π 2 1 2 πσ 2 e x 2 2 σ 2 dx + 0 x - σ 2 π 2 1 2 πσ 2 e x 2 2 σ 2 dx = 2 −∞ ( x 2 + σ 2 2 π ) 1 2 πσ 2 e x 2 2 σ 2 dx - 2 0 - 2 2 π 1 2 πσ 2 e x 2 2 σ 2 dx = σ 2 + 2 π σ 2 - 4 1 2 π σ 2 2 π 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
= σ 2 π - 2 π . 3634 σ 2 , which is significantly larger than the distortion rate bound D (1) = σ 2 / 4. 2. Rate distortion function with infinite distortion Find the rate distortion function R ( D ) = min I ( X ; ˆ X ) for X Bernoulli ( 1 2 ) and distortion d ( x, ˆ x ) = 0 , x = ˆ x, 1 , x = 1 , ˆ x = 0 , , x = 0 , ˆ x = 1 , where x, ˆ x ∈ { 0 , 1 } . Thus reconstructing a 0 by a 1 is never allowed. Solution: Rate distortion function with infinite distortion We wish to evaluate the rate distortion function R ( D ) = min p x | x ): ( x, ˆ x ) p ( x ) p x | x ) d ( x, ˆ x ) D I ( X ; ˆ X ) .
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern