sample_final_sol - EE 376A/Stat 376A Prof T Weissman...

Info icon This preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
EE 376A/Stat 376A Information Theory Prof. T. Weissman Friday, March 17, 2006 Solutions to Practice Final Problems These problems are sampled from a couple of the actual finals in previous years. 1. ( 20 points) Errors and erasures. Consider a binary symmetric channel (BSC) with crossover probability p . - - · · · · · · · ·3 Q Q Q Q Q Q Q Qs 1 0 1 0 1 - p 1 - p p p A helpful genie who knows the locations of all bit flips offers to convert flipped bits into erasures. In other words, the genie can transform the BSC into a binary erasure channel. Would you use his power? Be specific. Solution: Errors and erasures. Although it is very tempting to accept the genie’s offer, on a second thought, one realizes that it is disadvantageous to convert the bit flips into erasures when p is large. For example, when p = 1 , the original BSC is noiseless, while the “helpful” genie will erase every single bit coming out from the channel. The capacity C 1 ( p ) of the binary symmetric channel with crossover probability p is 1 - H ( p ) while the capacity C 2 ( p ) of the binary erasure channel with erasure probability p is 1 - p . One would convert the BSC into a BEC only if C 1 ( p ) C 2 ( p ), that is, p p * = . 7729 . (See Figure 1.) 2. (20 points) Code constraint. What is the capacity of a BSC( p ) under the constraint that each of the codewords has a proportion of 1’s less than or equal to α , i.e., 1 n n X i =1 X i ( w ) α, for w ∈ { 1 , 2 , . . . , 2 nR } . (Pay attention when α > 1 / 2.) 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 p 1-H(p) 1-p pstar Solution: Code constraint. Using the similar argument for the capacity of Gaussian channels under the power constraint P , we find that the capacity C of a BSC( p ) under the proportion constraint α is C = max p ( x ): EX α I ( X ; Y ) . Now under the Bernoulli( π ) input distribution with π α , we have I ( X ; Y ) = H ( Y ) - H ( Y | X ) = H ( Y ) - H ( Z | X ) = H ( Y ) - H ( Z ) = H ( π * p ) - H ( p ) , (1) where π * p = (1 - π ) p + π (1 - p ). (Breaking I ( X ; Y ) = H ( X ) - H ( X | Y ) = H ( X ) - H ( Z | Y ) is way more complicated since Z and Y are correlated.) Now when α > 1 / 2 , we have max π H ( π * p ) - H ( p ) = 1 - H ( p ) , with the capacity-achieving π * = 1 / 2 . On the other hand, when α 1 / 2 , π * = α achieves the maximum of (1); hence C = H ( α * p ) - H ( p ) . 3. ( 20 points) Partition. Let ( X, Y ) denote height and weight. Let [ Y ] be Y rounded off to the nearest pound. (a) Which is greater I ( X ; Y ) or I ( X ; [ Y ]) ? (b) Why? 2
Image of page 2
Solution: Partition. (a) I ( X ; Y ) I ( X ; [ Y ]) . (b) Data processing inequality. 4. ( 20 points) Amplify and forward. We cascade two Gaussian channels by feeding the (scaled) output of the first channel into the second. · ¶‡ - - ? - ? · ¶‡ q - - · ¶‡ - ? X 1 P X 2 P Y 1 Y 2 Z 1 N (0 , N ) α Z 2 N (0 , N ) Thus noises Z 1 and Z 2 are independent and identically distributed according to N (0 , N ), EX 2 1 = EX 2 2 = P, Y 1 = X 1 + Z 1 , Y 2 = X 2 + Z 2 , and X 2 = αY 1 , where the scaling factor α is chosen to satisfy the power constraint EX 2 2 = P .
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern