{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

7.20 - EE 376A Prof T Weissman Information Theory Homework...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376A Information Theory Prof. T. Weissman Feb 24, 2010 Homework Set #5 Solution 1. The Z channel. The Z-channel has binary input and output alphabets and transition probabilities p ( y | x ) given by the following matrix: Q = ± 1 0 1 / 2 1 / 2 ² x, y ∈ { 0 , 1 } Find the capacity of the Z-channel and the maximizing input probability distribution. Solutions: The Z channel. First we express I ( X ; Y ), the mutual information between the input and output of the Z-channel, as a function of x = Pr( X = 1): H ( Y | X ) = Pr( X = 0) · 0 + Pr( X = 1) · 1 = x H ( Y ) = H 2 (Pr( Y = 1)) = H 2 ( x/ 2) I ( X ; Y ) = H ( Y ) - H ( Y | X ) = H 2 ( x/ 2) - x Since I ( X ; Y ) = 0 when x = 0 and x = 1, the maximum mutual information is obtained for some value of x such that 0 < x < 1. Using elementary calculus, we determine that d dx I ( X ; Y ) = 1 2 log 2 1 - x/ 2 x/ 2 - 1 , which is equal to zero for x = 2 / 5. (It is reasonable that Pr( X = 1) < 1 / 2 because X = 1 is the noisy input to the channel.) So the capacity of the Z-channel in bits is H (1 / 5) - 2 / 5 = 0 . 722 - 0 . 4 = 0 . 322. 2. Channel capacity: Calculate the capacity of the following channels with probability transition matrices: (a) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 (1) (b) X = Y = { 0 , 1 , 2 } p ( y | x ) = 1 / 2 1 / 2 0 0 1 / 2 1 / 2 1 / 2 0 1 / 2 (2) 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
(c) X = Y = { 0 , 1 , 2 , 3 } p ( y | x ) = p 1 - p 0 0 1 - p p 0 0 0 0 q 1 - q 0 0 1 - q q (3) Channel Capacity:
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 6

7.20 - EE 376A Prof T Weissman Information Theory Homework...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon bookmark
Ask a homework question - tutors are online