HW11s[1] - ECE 534: Elements of Information Theory, Fall...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 534: Elements of Information Theory, Fall 2010 Homework 11 Solutions – all by Kenneth S. Palacio Baus November 17, 2010 1. Problem 10.14. Rate distortion for two independent sources . Can one compress two inde- pendent sources simultaneously better than by compressing the sources individually? The following problem addresses this question. Let { X i } be i.i.d. ∼ p ( x ) with distortion d ( x, ˆ x ) and rate distortion function R X ( D ). Similarly, let { Y i } be i.i.d. ∼ p ( y ) with distortion d ( y, ˆ y ) and rate distortion function R Y ( D ). Suppose we now wish to describe the process { ( X i ,Y i ) } subject to distortions Ed ( X, ˆ X ) ≤ D 1 and Ed ( Y, ˆ Y ) ≤ D 2. Thus, a rate R X,Y ( D 1 ,D 2) is sufficient, where R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y | x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) Now suppose that the { X i } process and the { Y i } process are independent of each other. (a) Show that: R X,Y ( D 1 ,D 2 ) ≥ R X ( D 1 ) + R Y ( D 2 ) Solution: Given: R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y | x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) (1) Considering that { X i } and the { Y i } are independent of each other we have: I ( X,Y ; ˆ X, ˆ Y ) = H ( X,Y )- H ( X,Y | ˆ X, ˆ Y ) (2) = H ( X ) + H ( Y )- H ( X | ˆ X, ˆ Y )- H ( Y | X, ˆ X, ˆ Y ) (3) = H ( X ) + H ( Y )- H ( X | ˆ X )- H ( Y | ˆ Y ) (4) ≥ H ( X )- H ( X | ˆ X ) + H ( Y )- H ( Y | ˆ Y ) (5) ≥ I ( X ; ˆ X ) + I ( Y ; ˆ Y ) (6) Now we obtain: R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y | x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) (7) ≥ min p (ˆ x, ˆ y | x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 h I ( X ; ˆ X ) + I ( Y ; ˆ Y ) i (8) ≥ min p (ˆ x | x ): Ed ( X, ˆ X ) ≤ D 1 I ( X ; ˆ X ) + min p (ˆ y | y ): Ed ( Y, ˆ Y ) ≤ D 2 I ( Y ; ˆ Y ) (9) ≥ R X ( D 1 ) + R Y ( D 2 ) (10) 1 (b) Does equality hold? From part (a), we have that: R X,Y ( D 1 ,D 2 ) ≥ R X ( D 1 ) + R Y ( D 2 )....
View Full Document

This note was uploaded on 01/19/2012 for the course ECE 534 taught by Professor Natashadevroye during the Fall '10 term at Ill. Chicago.

Page1 / 8

HW11s[1] - ECE 534: Elements of Information Theory, Fall...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online