soln_4_229spr07

# soln_4_229spr07 - EECS 229A Solutions to Homework 4 1...

This preview shows pages 1–3. Sign up to view the full content.

EECS 229A Spring 2007 * * Solutions to Homework 4 1. Problem 7.5 on pg. 224 of the text. Solution : Using two channels at once To ﬁnd the capacity of the product channel we must ﬁnd the distribution p ( x 1 ,x 2 ) on the input alphabet X 1 × X 2 which maximizes I ( X 1 ,X 2 ; Y 1 ,Y 2 ). Since the joint distribution would be p ( x 1 ,x 2 ,y 1 ,y 2 ) = p ( x 1 ,x 2 ) p ( y 1 | x 1 ) p ( y 2 | x 2 ) , we have I ( X 1 ,X 2 ; Y 1 ,Y 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 ,Y 2 | X 1 ,X 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 | X 1 ,X 2 ) - H ( Y 2 | X 1 ,X 2 ) = H ( Y 1 ,Y 2 ) - H ( Y 1 | X 1 ) - H ( Y 2 | X 2 ) H ( Y 1 ) + H ( Y 2 ) - H ( Y 1 | X 1 ) - H ( Y 2 | X 2 ) = I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) , where we have used the conditional independence of Y 1 and Y 2 given ( X 1 ,X 2 ) to make the second step, and Markovianity to make the third step. Equality occurs in the fourth step when X 1 and X 2 are independent (so that Y 1 and Y 2 are also independent). Now it follows that we have C = max p ( x 1 ,x 2 ) I ( X 1 ,X 2 ; Y 1 ,Y 2 ) max p ( x 1 ,x 2 ) I ( X 1 ; Y 1 ) + max p ( x 1 ,x 2 ) I ( X 2 ; Y 2 ) = max p ( x 1 ) I ( X 1 ; Y 1 ) + max p ( x 2 ) I ( X 2 ; Y 2 ) = C 1 + C 2 , with equality if p ( x 1 ,x 2 ) = p * ( x 1 ) p * ( x 2 ), where p * ( x 1 ) and p * ( x 2 ) are the input distribu- tions that achieve the capacity of the ﬁrst and the second channel respectively. The capacity of the product channel is therefore C 1 + C 2 . 2. Problem 7.12 on pg. 226 of the text. Solution : Unused symbols Let α,β,γ respectively denote the probabilities with which the three input symbols are used, α + β + γ = 1. Then H ( Y | X ) = ( α + γ ) h ( 1 3 ) + β log 3, where h ( p ) denotes the binary entropy function. For this choice of input distribution, we have I ( X ; Y ) = H ( Y ) - H ( Y | X ) (1 - β )log 3 - (1 - β ) h ( 1 3 ) , 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
where we have observed that H ( Y ) log 3. Since h ( 1 3 ) < log 3, the upper bound is largest when β = 0, equalling log 3 - h ( 1 3 ) for this choice. However, note that if we set α = γ = 1 2 and β = 0, then the output distribution is indeed uniform, and we therefore achieve this highest possible upper bound. Hence it is optimal to not use the second symbol at all. This may be understood by observing that the distribution on the output conditioned on
This is the end of the preview. Sign up to access the rest of the document.

## This document was uploaded on 12/01/2010.

### Page1 / 6

soln_4_229spr07 - EECS 229A Solutions to Homework 4 1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online