prelim10 - ECE 5620 Spring 2010 Prelim Exam: Mar. 11, 2010...

Info iconThis preview shows pages 1–13. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 5620 Spring 2010 Prelim Exam: Mar. 11, 2010 Name: fa l u‘l’” N’ NetID: Rules: 0 Do not open this exam until you are instructed to do so. 0 You are permitted one letter-sized crib sheet. Otherwise the exam is closed— book and closed—note. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit. 0 Points may be deducted for incorrect statements. a You have two hours to complete this exam. Problem Points Score 1 17 4 10 Good Luck! 1. Concavity of Conditional Entropy Suppose that (X 1, Y1) have joint PMFp1(J:, y) and (X2, Y2) have joint PMF p2($, y). Suppose that T has PMF and T, (X1, Y1), and (X2, Y2) are all independent. (a) (4 pts) Compute the PMF of (XT, YT). Simplify your answer as much as possible. Win“) ‘ PM”) = PCT "J r((“T.TT/ " m4)! T 1") [f rJ 4” r('r= 1) rumm = (m,v)lT-1-) = a Hoax) = (fin-u} + (u— a/ ribtfi/ flaw) [2 to] -: (Mira) + wax ,u, (x,1) [1 H] A Ham) (b) (2 pts) Compute H (YTlXT). Simplify your answer as much as possi- ble. , Mm“) 5—: “my? W12) {'2 W] 7n! ‘3 Haw} + “'m‘ 9'; Hm!) 7-: - E P113277 4 {Pat-3 r3559) [0}, m._:u\__www_wwm_____h____ *1? frfi-fiflihiv +i‘“&,¢’f¢[kig‘f-I 2‘ J i (0) (fits) Compute H(YT|XT, T). Simplify your answer as much as pos— 31 e. “(Yr‘XT;T) : _ i F(XT1VJYT=‘/IT-“jl°y r[YT:Yt)(1:'xtT:£) 1N 7: “ f Gml‘m/L} Mam) 7‘” E F+] — f 6—0151“, {a} Mm} «PI 7L” (d) (3 pts) In general, is H (Y|X ) a concave function of the joint PMF p(x, y)? Explain. a a] H(Y1—[XT’T} é “($11112 re Juur “*r-yy: Y'EJ‘, C oval-"‘VO‘W‘} C‘ H] . , l " &H(‘f.!y.}+{n-‘o! Marx-m} é _’ i r“"’“"’ rt,” V C r] «n! (e) (5 pts) In general, is H (Y|X ) a strictly concave function of the joint PMF p(m, y)? Explain. No. r'l‘1‘ (nun-7er "H: JON; I'MF; C! r if]; fl 7 Y '2. 3 5L On! a 1'; [2 Tb] PI : {’1 l J 1 - J. i. r (j ‘ L! ‘7' 1 .1 V: L): (a “(TIIVJ : HITLH") Z‘PVWJ/ : 0 "I: 7; WIN” bay} N P C1 r41]. TL‘ILI Jig M’n-I €4l‘l *fiowr’pr 2. Entropy of the Geometric Distribution (a) (3 pts) Suppose that X is geometrically distributed over the nonnega- tive integers: Pr(X = k) =p(1—p)k k 2 0. Compute the entropy of X in terms of its mean ,u = (1/10 — 1). HM) :— - % HJ‘rx‘HWE-‘Fwflg [I if k5» : -—l:) p - Z i<r[l‘"r)<[u9 (Ff) Ira, : “In? r - Nhirbt-f) (4] i .1. weg-t ‘=> r7,“ I :3 my) —_~ .1Hr HM ~ we; (I ~ “7, (b) (3 pts) Let Y be a random variable taking values in the nonnegative integers. Suppose that Y has mean ,u. Show that H (Y) g H (X ). "" P flu — D p u w ——‘2’ Id v i— ( T Pg) I<:U PT( I} l'xlls! 0 rt?) M k}: = Hm + “‘“Wt’l' '1 (c) (2 pts) Supppose that Y in (b) satisfies H (Y) = H (X). Can we con- clude that Y is geometrically distributed? Explain. 0 r0. 0L9”! ‘1” FANG-:14”) 5" timing} :9 [1 r43. FVO‘VA 7:7 'pfrrsy C 3Q Suppose a discrete memoryless channel has a two—bit input (X 1, X2) and a two—bit output (Y1, Y2). The input X1 is passed through a binary symmetric channel with crossover probability 6 to obtain Y1. The input X2 is passed through an independent binary symmetric channel with crossover probabil- ity e to obtain Y2. Note that for each channel use, the encoder specifies an input to both of the binary symmetric channels, and the decoder observes both of the outputs. (a) (7 pts) Compute the capacity of this channel. What is a capacity- achieving distribution? [I r’J [I H] (Y Y -_-_- “4.x -'I-\[‘Tt,‘{1!>(.‘¥,j : Mtg ', 1. I, a. r‘xler) Phfi'r-‘f‘j Rina] é Hfttjl NHL} 5 1°; 7- "" “r '3 3 2 {j H] 0 r0 Hirilszfx.‘yr = “[‘myuwr; + Hitzlvdm) r u [I n) -.: HLYJI.) 4v l-l(‘l1]\(.,) w: Mir) 4 on; (b) (6 pts) Suppose T is uniformly distributed over the set {1, 2} and is independent of (X1, X2, Y1,Y2). Suppose that instead of observing (Y1, Y2), the channel observes YT. Thus the decoder now only 0b- serves one bit, which is drawn from a randomly chosen subchannel. Compute the capacity. What is a capacity—achieving distribution? CL‘duvul “tin-J. ftyly}! 0 ad i-C _t L F'- r,‘ L__.A_.__fll_-..__.fi,_.J S I'L- 4.. n ’1 HIYTIY. 3(1) -.—. plala? H (c) 4- (plan) + plug] .1 + pfmj put) “a Mfr} Far] (c) (6 pts) Let T be as in (b), but suppose that the decoder observes (YT, T). Thus the decoder now knows which of the two subchan- nels it is observing at each time. Compute the capacity. What is a capacity-achieving distribution? 6 r0 0 r0 I(XI:K2;rT_Tj : I(K\191.JT)+ T{V'f)";rTiT) -: ‘1" T(¥'v,_l‘f||1':t) +':LT(V~,V‘ Yttrfk) . I ‘~‘ 'i'fi“(¥.lvt;*n} 4 "E Thanh U "j '3 '{TL\( 7) + J‘r-[y- w ~— o a ‘E {5... HI“) .L. .1 (q- chj/ [1W0 ': l— Hts) 7L}! U?r:- Livy! {r $61» In“!!! "I xl 5"! y‘ a" $5.45 UV7I‘urn. (- :kJ'r‘rJFv "I “01- V'?p-‘mr):} 4. (a) (6 .pts) Suppose that X and Y are jointly Gaussian with zero mean and covariance matrix ' H where a} > 0, 0?, > 0, and —1 S p S 1. Compute I(X;Y) and h(Y|X For what value(s) of p is I (X ; Y) largest? For what value(s) of p is it smallest? 2 0X PUXUY PC’XUY 0%? T(~K"} ‘= HAW -L(>f} - (l (g = "-l balk“er “<1 + liq m m + an, (mm: H J 0-,: 6‘; I l w ., W = a- 0 a Gig; ("Ty Lory-35e- lfl=l [l H] fwuul f‘“ [l H] Mth My) .. The”; '3 45!”,(7'159 _ if“) _i_. 2 ._._ u... a; l.” (2°41: (0;?(1- 77,0) 10 (b) (4 pts) Suppose that X and Y are jointly continuous with joint PDF 2 ifD g m 5 y S 1 3:: = . H y) {0 otherwxse. Compute [(X; Y) and h(Y]X). Hint: You may find the following indefinite integral helpful 11193 1 m m+1 _ [CE lnmdm—m ( 1 ( DZ). 5. (a) (3 pts) Suppose that a binary-input discrete memoryless channel has the property that for any 39 in [0, 1], the two input distributions [p 1 — p] and [1 — p p] achieve the same mutual information over the channel. Explain why the uniform input distribution must be capacity achiev- mg. [4 r0 -L- IO. M; Mm) + '11 7! my, f (m); é It :5er + 4; my); MM) WW W-M u tut-[1p [i ’1‘] Iffilfl‘j Ham); ‘1 Liv.“ If; H] I 12 (b) (3 pts) Compute the capacity of the binary-symmetric—erasure channel: Wh6Y6620,520,ande+6g1, " Iva-ff CLovveE Jr'lil'rif‘r {'5'} j by 5pm,“! .‘yrml' "I -th TL!“ H“) = l-f+ Lij [I a] 13 ...
View Full Document

This note was uploaded on 10/03/2011 for the course ECE 5620 at Cornell University (Engineering School).

Page1 / 13

prelim10 - ECE 5620 Spring 2010 Prelim Exam: Mar. 11, 2010...

This preview shows document pages 1 - 13. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online