soln_4_229spr07

soln_4_229spr07 - EECS 229A Solutions to Homework 4 1...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EECS 229A Spring 2007 * * Solutions to Homework 4 1. Problem 7.5 on pg. 224 of the text. Solution : Using two channels at once To find the capacity of the product channel we must find the distribution p ( x 1 ,x 2 ) on the input alphabet X 1 × X 2 which maximizes I ( X 1 ,X 2 ; Y 1 ,Y 2 ). Since the joint distribution would be p ( x 1 ,x 2 ,y 1 ,y 2 ) = p ( x 1 ,x 2 ) p ( y 1 | x 1 ) p ( y 2 | x 2 ) , we have I ( X 1 ,X 2 ; Y 1 ,Y 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 ,Y 2 | X 1 ,X 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 | X 1 ,X 2 ) - H ( Y 2 | X 1 ,X 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 | X 1 ) - H ( Y 2 | X 2 ) H ( Y 1 ) + H ( Y 2 ) - H ( Y 1 | X 1 ) - H ( Y 2 | X 2 ) = I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) , where we have used the conditional independence of Y 1 and Y 2 given ( X 1 ,X 2 ) to make the second step, and Markovianity to make the third step. Equality occurs in the fourth step when X 1 and X 2 are independent (so that Y 1 and Y 2 are also independent). Now it follows that we have C = max p ( x 1 ,x 2 ) I ( X 1 ,X 2 ; Y 1 ,Y 2 ) max p ( x 1 ,x 2 ) I ( X 1 ; Y 1 ) + max p ( x 1 ,x 2 ) I ( X 2 ; Y 2 ) = max p ( x 1 ) I ( X 1 ; Y 1 ) + max p ( x 2 ) I ( X 2 ; Y 2 ) = C 1 + C 2 , with equality if p ( x 1 ,x 2 ) = p * ( x 1 ) p * ( x 2 ), where p * ( x 1 ) and p * ( x 2 ) are the input distribu- tions that achieve the capacity of the first and the second channel respectively. The capacity of the product channel is therefore C 1 + C 2 . 2. Problem 7.12 on pg. 226 of the text. Solution : Unused symbols Let α,β,γ respectively denote the probabilities with which the three input symbols are used, α + β + γ = 1. Then H ( Y | X ) = ( α + γ ) h ( 1 3 ) + β log 3, where h ( p ) denotes the binary entropy function. For this choice of input distribution, we have I ( X ; Y ) = H ( Y ) - H ( Y | X ) (1 - β )log 3 - (1 - β ) h ( 1 3 ) , 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
where we have observed that H ( Y ) log 3. Since h ( 1 3 ) < log 3, the upper bound is largest when β = 0, equalling log 3 - h ( 1 3 ) for this choice. However, note that if we set α = γ = 1 2 and β = 0, then the output distribution is indeed uniform, and we therefore achieve this highest possible upper bound. Hence it is optimal to not use the second symbol at all. This may be understood by observing that the distribution on the output conditioned on
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 12/01/2010.

Page1 / 6

soln_4_229spr07 - EECS 229A Solutions to Homework 4 1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online