This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 534: Elements of Information Theory, Fall 2010 Homework 11 Solutions – all by Kenneth S. Palacio Baus November 17, 2010 1. Problem 10.14. Rate distortion for two independent sources . Can one compress two inde pendent sources simultaneously better than by compressing the sources individually? The following problem addresses this question. Let { X i } be i.i.d. ∼ p ( x ) with distortion d ( x, ˆ x ) and rate distortion function R X ( D ). Similarly, let { Y i } be i.i.d. ∼ p ( y ) with distortion d ( y, ˆ y ) and rate distortion function R Y ( D ). Suppose we now wish to describe the process { ( X i ,Y i ) } subject to distortions Ed ( X, ˆ X ) ≤ D 1 and Ed ( Y, ˆ Y ) ≤ D 2. Thus, a rate R X,Y ( D 1 ,D 2) is sufficient, where R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y  x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) Now suppose that the { X i } process and the { Y i } process are independent of each other. (a) Show that: R X,Y ( D 1 ,D 2 ) ≥ R X ( D 1 ) + R Y ( D 2 ) Solution: Given: R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y  x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) (1) Considering that { X i } and the { Y i } are independent of each other we have: I ( X,Y ; ˆ X, ˆ Y ) = H ( X,Y ) H ( X,Y  ˆ X, ˆ Y ) (2) = H ( X ) + H ( Y ) H ( X  ˆ X, ˆ Y ) H ( Y  X, ˆ X, ˆ Y ) (3) = H ( X ) + H ( Y ) H ( X  ˆ X ) H ( Y  ˆ Y ) (4) ≥ H ( X ) H ( X  ˆ X ) + H ( Y ) H ( Y  ˆ Y ) (5) ≥ I ( X ; ˆ X ) + I ( Y ; ˆ Y ) (6) Now we obtain: R X,Y ( D 1 ,D 2 ) = min p (ˆ x, ˆ y  x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 I ( X,Y ; ˆ X, ˆ Y ) (7) ≥ min p (ˆ x, ˆ y  x,y ): Ed ( X, ˆ X ) ≤ D 1 ,Ed ( Y, ˆ Y ) ≤ D 2 h I ( X ; ˆ X ) + I ( Y ; ˆ Y ) i (8) ≥ min p (ˆ x  x ): Ed ( X, ˆ X ) ≤ D 1 I ( X ; ˆ X ) + min p (ˆ y  y ): Ed ( Y, ˆ Y ) ≤ D 2 I ( Y ; ˆ Y ) (9) ≥ R X ( D 1 ) + R Y ( D 2 ) (10) 1 (b) Does equality hold? From part (a), we have that: R X,Y ( D 1 ,D 2 ) ≥ R X ( D 1 ) + R Y ( D 2 )....
View
Full
Document
This note was uploaded on 01/19/2012 for the course ECE 534 taught by Professor Natashadevroye during the Fall '10 term at Ill. Chicago.
 Fall '10
 NatashaDevroye

Click to edit the document details