{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

prelim10

# prelim10 - ECE 5620 Spring 2010 Prelim Exam Mar 11 2010...

This preview shows pages 1–13. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 5620 Spring 2010 Prelim Exam: Mar. 11, 2010 Name: fa l u‘l’” N’ NetID: Rules: 0 Do not open this exam until you are instructed to do so. 0 You are permitted one letter-sized crib sheet. Otherwise the exam is closed— book and closed—note. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit. 0 Points may be deducted for incorrect statements. a You have two hours to complete this exam. Problem Points Score 1 17 4 10 Good Luck! 1. Concavity of Conditional Entropy Suppose that (X 1, Y1) have joint PMFp1(J:, y) and (X2, Y2) have joint PMF p2(\$, y). Suppose that T has PMF and T, (X1, Y1), and (X2, Y2) are all independent. (a) (4 pts) Compute the PMF of (XT, YT). Simplify your answer as much as possible. Win“) ‘ PM”) = PCT "J r((“T.TT/ " m4)! T 1") [f rJ 4” r('r= 1) rumm = (m,v)lT-1-) = a Hoax) = (ﬁn-u} + (u— a/ ribtﬁ/ ﬂaw) [2 to] -: (Mira) + wax ,u, (x,1) [1 H] A Ham) (b) (2 pts) Compute H (YTlXT). Simplify your answer as much as possi- ble. , Mm“) 5—: “my? W12) {'2 W] 7n! ‘3 Haw} + “'m‘ 9'; Hm!) 7-: - E P113277 4 {Pat-3 r3559) [0}, m._:u\__www_wwm_____h____ *1? frﬁ-ﬁﬂihiv +i‘“&,¢’f¢[kig‘f-I 2‘ J i (0) (ﬁts) Compute H(YT|XT, T). Simplify your answer as much as pos— 31 e. “(Yr‘XT;T) : _ i F(XT1VJYT=‘/IT-“jl°y r[YT:Yt)(1:'xtT:£) 1N 7: “ f Gml‘m/L} Mam) 7‘” E F+] — f 6—0151“, {a} Mm} «PI 7L” (d) (3 pts) In general, is H (Y|X ) a concave function of the joint PMF p(x, y)? Explain. a a] H(Y1—[XT’T} é “(\$11112 re Juur “*r-yy: Y'EJ‘, C oval-"‘VO‘W‘} C‘ H] . , l " &H(‘f.!y.}+{n-‘o! Marx-m} é _’ i r“"’“"’ rt,” V C r] «n! (e) (5 pts) In general, is H (Y|X ) a strictly concave function of the joint PMF p(m, y)? Explain. No. r'l‘1‘ (nun-7er "H: JON; I'MF; C! r if]; fl 7 Y '2. 3 5L On! a 1'; [2 Tb] PI : {’1 l J 1 - J. i. r (j ‘ L! ‘7' 1 .1 V: L): (a “(TIIVJ : HITLH") Z‘PVWJ/ : 0 "I: 7; WIN” bay} N P C1 r41]. TL‘ILI Jig M’n-I €4l‘l *ﬁowr’pr 2. Entropy of the Geometric Distribution (a) (3 pts) Suppose that X is geometrically distributed over the nonnega- tive integers: Pr(X = k) =p(1—p)k k 2 0. Compute the entropy of X in terms of its mean ,u = (1/10 — 1). HM) :— - % HJ‘rx‘HWE-‘Fwﬂg [I if k5» : -—l:) p - Z i<r[l‘"r)<[u9 (Ff) Ira, : “In? r - Nhirbt-f) (4] i .1. weg-t ‘=> r7,“ I :3 my) —_~ .1Hr HM ~ we; (I ~ “7, (b) (3 pts) Let Y be a random variable taking values in the nonnegative integers. Suppose that Y has mean ,u. Show that H (Y) g H (X ). "" P flu — D p u w ——‘2’ Id v i— ( T Pg) I<:U PT( I} l'xlls! 0 rt?) M k}: = Hm + “‘“Wt’l' '1 (c) (2 pts) Supppose that Y in (b) satisﬁes H (Y) = H (X). Can we con- clude that Y is geometrically distributed? Explain. 0 r0. 0L9”! ‘1” FANG-:14”) 5" timing} :9 [1 r43. FVO‘VA 7:7 'pfrrsy C 3Q Suppose a discrete memoryless channel has a two—bit input (X 1, X2) and a two—bit output (Y1, Y2). The input X1 is passed through a binary symmetric channel with crossover probability 6 to obtain Y1. The input X2 is passed through an independent binary symmetric channel with crossover probabil- ity e to obtain Y2. Note that for each channel use, the encoder speciﬁes an input to both of the binary symmetric channels, and the decoder observes both of the outputs. (a) (7 pts) Compute the capacity of this channel. What is a capacity- achieving distribution? [I r’J [I H] (Y Y -_-_- “4.x -'I-\[‘Tt,‘{1!>(.‘¥,j : Mtg ', 1. I, a. r‘xler) Phﬁ'r-‘f‘j Rina] é Hfttjl NHL} 5 1°; 7- "" “r '3 3 2 {j H] 0 r0 Hirilszfx.‘yr = “[‘myuwr; + Hitzlvdm) r u [I n) -.: HLYJI.) 4v l-l(‘l1]\(.,) w: Mir) 4 on; (b) (6 pts) Suppose T is uniformly distributed over the set {1, 2} and is independent of (X1, X2, Y1,Y2). Suppose that instead of observing (Y1, Y2), the channel observes YT. Thus the decoder now only 0b- serves one bit, which is drawn from a randomly chosen subchannel. Compute the capacity. What is a capacity—achieving distribution? CL‘duvul “tin-J. ftyly}! 0 ad i-C _t L F'- r,‘ L__.A_.__ﬂl_-..__.ﬁ,_.J S I'L- 4.. n ’1 HIYTIY. 3(1) -.—. plala? H (c) 4- (plan) + plug] .1 + pfmj put) “a Mfr} Far] (c) (6 pts) Let T be as in (b), but suppose that the decoder observes (YT, T). Thus the decoder now knows which of the two subchan- nels it is observing at each time. Compute the capacity. What is a capacity-achieving distribution? 6 r0 0 r0 I(XI:K2;rT_Tj : I(K\191.JT)+ T{V'f)";rTiT) -: ‘1" T(¥'v,_l‘f||1':t) +':LT(V~,V‘ Yttrfk) . I ‘~‘ 'i'ﬁ“(¥.lvt;*n} 4 "E Thanh U "j '3 '{TL\( 7) + J‘r-[y- w ~— o a ‘E {5... HI“) .L. .1 (q- chj/ [1W0 ': l— Hts) 7L}! U?r:- Livy! {r \$61» In“!!! "I xl 5"! y‘ a" \$5.45 UV7I‘urn. (- :kJ'r‘rJFv "I “01- V'?p-‘mr):} 4. (a) (6 .pts) Suppose that X and Y are jointly Gaussian with zero mean and covariance matrix ' H where a} > 0, 0?, > 0, and —1 S p S 1. Compute I(X;Y) and h(Y|X For what value(s) of p is I (X ; Y) largest? For what value(s) of p is it smallest? 2 0X PUXUY PC’XUY 0%? T(~K"} ‘= HAW -L(>f} - (l (g = "-l balk“er “<1 + liq m m + an, (mm: H J 0-,: 6‘; I l w ., W = a- 0 a Gig; ("Ty Lory-35e- lﬂ=l [l H] fwuul f‘“ [l H] Mth My) .. The”; '3 45!”,(7'159 _ if“) _i_. 2 ._._ u... a; l.” (2°41: (0;?(1- 77,0) 10 (b) (4 pts) Suppose that X and Y are jointly continuous with joint PDF 2 ifD g m 5 y S 1 3:: = . H y) {0 otherwxse. Compute [(X; Y) and h(Y]X). Hint: You may ﬁnd the following indeﬁnite integral helpful 11193 1 m m+1 _ [CE lnmdm—m ( 1 ( DZ). 5. (a) (3 pts) Suppose that a binary-input discrete memoryless channel has the property that for any 39 in [0, 1], the two input distributions [p 1 — p] and [1 — p p] achieve the same mutual information over the channel. Explain why the uniform input distribution must be capacity achiev- mg. [4 r0 -L- IO. M; Mm) + '11 7! my, f (m); é It :5er + 4; my); MM) WW W-M u tut-[1p [i ’1‘] Ifﬁlﬂ‘j Ham); ‘1 Liv.“ If; H] I 12 (b) (3 pts) Compute the capacity of the binary-symmetric—erasure channel: Wh6Y6620,520,ande+6g1, " Iva-ff CLovveE Jr'lil'rif‘r {'5'} j by 5pm,“! .‘yrml' "I -th TL!“ H“) = l-f+ Lij [I a] 13 ...
View Full Document

{[ snackBarMessage ]}

### Page1 / 13

prelim10 - ECE 5620 Spring 2010 Prelim Exam Mar 11 2010...

This preview shows document pages 1 - 13. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online