are independent and hence Y 1 and Y 2 are independent Therefore C max p x 1 x 2

# Are independent and hence y 1 and y 2 are independent

This preview shows page 7 - 9 out of 9 pages.

are independent and hence Y 1 and Y 2 are independent. Therefore C = max p ( x 1 ,x 2 ) I ( X 1 , X 2 ; Y 1 , Y 2 ) max p ( x 1 ,x 2 ) I ( X 1 ; Y 1 ) + max p ( x 1 ,x 2 ) I ( X 2 ; Y 2 ) = max p ( x 1 ) I ( X 1 ; Y 1 ) + max p ( x 2 ) I ( X 2 ; Y 2 ) = C 1 + C 2 . with equality iff p ( x 1 , x 2 ) = p * ( x 1 ) p * ( x 2 ) and p * ( x 1 ) and p * ( x 2 ) are the distributions that maximize C 1 and C 2 respectively. 8. A channel with two independent looks at Y. Let Y 1 and Y 2 be conditionally independent and conditionally identically distributed given X. Thus p ( y 1 , y 2 | x ) = p ( y 1 | x ) p ( y 2 | x ) . (a) Show I ( X ; Y 1 , Y 2 ) = 2 I ( X ; Y 1 ) I ( Y 1 ; Y 2 ) . (b) Conclude that the capacity of the channel a45 a45 X ( Y 1 , Y 2 ) is less than twice the capacity of the channel a45 a45 X Y 1 (c) How about 3 independent looks? Compare I ( X ; Y 1 , Y 2 , Y 3 ) to 3 I ( X ; Y 1 ). Solution: A channel with two independent looks at Y. 7
(a) First note that Y 1 and Y 2 are identically distributed since p ( y 1 | x ) = p ( y 2 | x ) and hence p ( y 1 ) = x p ( y 1 | x ) p ( x ) = x p ( y 2 | x ) p ( x ) = p ( y 2 ). Therefore, I ( X ; Y 1 , Y 2 ) = H ( Y 1 , Y 2 ) H ( Y 1 , Y 2 | X ) = H ( Y 1 ) + H ( Y 2 ) I ( Y 1 ; Y 2 ) H ( Y 1 , Y 2 | X ) = H ( Y 1 ) + H ( Y 2 ) I ( Y 1 ; Y 2 ) H ( Y 1 | X ) H ( Y 2 | X ) (7) = 2 H ( Y 1 ) 2 H ( Y 1 | X ) I ( Y 1 ; Y 2 ) (8) = 2 I ( X ; Y 1 ) I ( Y 1 ; Y 2 ) , where Eq. (7) follows from the fact that Y 1 and Y 2 are conditionally independent given X and Eq. (8) follows from the fact that Y 1 and Y 2 are identically distributed and conditionally identically distributed given X . (b) The capacity of the single look channel X Y 1 is C 1 = max p ( x ) I ( X ; Y 1 ) . The capacity of the channel X ( Y 1 , Y 2 ) is C 2 = max p ( x ) I ( X ; Y 1 , Y 2 ) = max p ( x ) 2 I ( X ; Y 1 ) I ( Y 1 ; Y 2 ) max p ( x ) 2 I ( X ; Y 1 ) = 2 C 1 . Hence, two independent looks cannot be more than twice as good as one look. (c) Observe for Y n conditionally independent given X that I ( X ; Y k | Y k - 1 ) = H ( Y k | Y k - 1 ) H ( Y k | Y k - 1 , X ) = H ( Y k | Y k - 1 ) H ( Y k | X ) (9) H ( Y k ) H ( Y k | X ) (10) = I ( X ; Y k ) where Eq. (9) follows from the fact that Y k and Y k - 1 are conditionally independent given X and Eq. (10) follows from the fact that conditioning reduces entropy. Using the above relationship, I ( X ; Y 1 , Y 2 , Y 3 ) = I ( X ; Y 1 ) + I ( X ; Y 2 | Y 1 ) + I ( X ; Y 3 | Y 1 , Y 2 ) I ( X ; Y 1 ) + I ( X ; Y 2 ) + I ( X ; Y 3 ) (11) = 3 I ( X ; Y 1 ) (12) where Eq. (11) follows from the relationship shown above and Eq. (12) follows from the fact that Y 1 , Y 2 and Y 3 are identically distributed. Thus it can be shown that the capacity C 3 of three looks is less than three times the capacity C 1 of one look, C 3 < 3 C 1 . 8

#### You've reached the end of your free preview.

Want to read all 9 pages?

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern