This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 5620 Spring 2010
Prelim Exam: Mar. 11, 2010 Name: fa l u‘l’” N’
NetID: Rules: 0 Do not open this exam until you are instructed to do so. 0 You are permitted one lettersized crib sheet. Otherwise the exam is closed—
book and closed—note. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit.
0 Points may be deducted for incorrect statements. a You have two hours to complete this exam. Problem Points Score
1 17 4 10 Good Luck! 1. Concavity of Conditional Entropy Suppose that (X 1, Y1) have joint PMFp1(J:, y) and (X2, Y2) have joint PMF
p2($, y). Suppose that T has PMF and T, (X1, Y1), and (X2, Y2) are all independent. (a) (4 pts) Compute the PMF of (XT, YT). Simplify your answer as much
as possible. Win“) ‘ PM”) = PCT "J r((“T.TT/ " m4)! T 1") [f rJ
4” r('r= 1) rumm = (m,v)lT1) = a Hoax) = (ﬁnu} + (u— a/ ribtﬁ/ ﬂaw) [2 to] : (Mira) + wax ,u, (x,1) [1 H] A Ham) (b) (2 pts) Compute H (YTlXT). Simplify your answer as much as possi ble.
,
Mm“) 5—: “my? W12) {'2 W]
7n!
‘3 Haw} + “'m‘ 9'; Hm!)
7:  E P113277 4 {Pat3 r3559) [0}, m._:u\__www_wwm_____h____
*1? frﬁﬁﬂihiv +i‘“&,¢’f¢[kig‘fI 2‘ J i (0) (ﬁts) Compute H(YTXT, T). Simplify your answer as much as pos—
31 e. “(Yr‘XT;T) : _ i F(XT1VJYT=‘/IT“jl°y r[YT:Yt)(1:'xtT:£) 1N
7: “ f Gml‘m/L} Mam)
7‘” E F+]
— f 6—0151“, {a} Mm}
«PI 7L” (d) (3 pts) In general, is H (YX ) a concave function of the joint PMF
p(x, y)? Explain. a a] H(Y1—[XT’T} é “($11112 re Juur “*ryy: Y'EJ‘, C oval"‘VO‘W‘} C‘ H] . , l "
&H(‘f.!y.}+{n‘o! Marxm} é _’ i r“"’“"’ rt,” V C r] «n! (e) (5 pts) In general, is H (YX ) a strictly concave function of the joint
PMF p(m, y)? Explain. No. r'l‘1‘ (nun7er "H: JON; I'MF;
C! r if]; fl
7 Y '2. 3 5L On! a 1'; [2 Tb]
PI : {’1 l J 1 
J. i. r (j ‘
L! ‘7'
1 .1
V: L): (a “(TIIVJ : HITLH") Z‘PVWJ/ : 0
"I: 7;
WIN” bay} N P C1 r41].
TL‘ILI Jig M’nI €4l‘l *ﬁowr’pr 2. Entropy of the Geometric Distribution (a) (3 pts) Suppose that X is geometrically distributed over the nonnega
tive integers:
Pr(X = k) =p(1—p)k k 2 0. Compute the entropy of X in terms of its mean ,u = (1/10 — 1). HM) :—  % HJ‘rx‘HWE‘Fwﬂg [I if k5»
: —l:) p  Z i<r[l‘"r)<[u9 (Ff)
Ira,
: “In? r  Nhirbtf) (4]
i .1.
wegt ‘=> r7,“
I
:3 my) —_~ .1Hr HM ~ we; (I ~ “7, (b) (3 pts) Let Y be a random variable taking values in the nonnegative
integers. Suppose that Y has mean ,u. Show that H (Y) g H (X ). "" P flu
— D p u w ——‘2’ Id v i—
( T Pg) I<:U PT( I} l'xlls! 0 rt?)
M k}:
= Hm + “‘“Wt’l' '1 (c) (2 pts) Supppose that Y in (b) satisﬁes H (Y) = H (X). Can we con
clude that Y is geometrically distributed? Explain. 0 r0. 0L9”! ‘1” FANG:14”) 5" timing} :9 [1 r43. FVO‘VA 7:7 'pfrrsy C 3Q Suppose a discrete memoryless channel has a two—bit input (X 1, X2) and a
two—bit output (Y1, Y2). The input X1 is passed through a binary symmetric
channel with crossover probability 6 to obtain Y1. The input X2 is passed
through an independent binary symmetric channel with crossover probabil
ity e to obtain Y2. Note that for each channel use, the encoder speciﬁes an
input to both of the binary symmetric channels, and the decoder observes
both of the outputs. (a) (7 pts) Compute the capacity of this channel. What is a capacity
achieving distribution?
[I r’J [I H] (Y Y __ “4.x 'I\[‘Tt,‘{1!>(.‘¥,j
: Mtg ', 1. I, a.
r‘xler) Phﬁ'r‘f‘j
Rina] é Hfttjl NHL} 5 1°; 7 "" “r '3 3 2
{j H] 0 r0
Hirilszfx.‘yr = “[‘myuwr; + Hitzlvdm) r u
[I n) .: HLYJI.) 4v ll(‘l1]\(.,) w: Mir) 4 on; (b) (6 pts) Suppose T is uniformly distributed over the set {1, 2} and is
independent of (X1, X2, Y1,Y2). Suppose that instead of observing
(Y1, Y2), the channel observes YT. Thus the decoder now only 0b
serves one bit, which is drawn from a randomly chosen subchannel.
Compute the capacity. What is a capacity—achieving distribution? CL‘duvul “tinJ. ftyly}! 0 ad iC
_t
L F' r,‘ L__.A_.__ﬂl_..__.ﬁ,_.J S
I'L
4.. n
’1 HIYTIY. 3(1) .—. plala? H (c) 4 (plan) + plug] .1 + pfmj put) “a Mfr} Far] (c) (6 pts) Let T be as in (b), but suppose that the decoder observes
(YT, T). Thus the decoder now knows which of the two subchan
nels it is observing at each time. Compute the capacity. What is a
capacityachieving distribution? 6 r0
0 r0 I(XI:K2;rT_Tj : I(K\191.JT)+ T{V'f)";rTiT) : ‘1" T(¥'v,_l‘f1':t)
+':LT(V~,V‘ Yttrfk)
. I
‘~‘ 'i'ﬁ“(¥.lvt;*n} 4 "E Thanh U "j
'3 '{TL\( 7) + J‘r[y
w ~— o a
‘E {5... HI“) .L. .1 (q chj/ [1W0
': l— Hts)
7L}! U?r: Livy! {r $61» In“!!! "I xl 5"! y‘
a" $5.45 UV7I‘urn. ( :kJ'r‘rJFv "I “01 V'?p‘mr):} 4. (a) (6 .pts) Suppose that X and Y are jointly Gaussian with zero mean and
covariance matrix ' H where a} > 0, 0?, > 0, and —1 S p S 1. Compute I(X;Y) and
h(YX For what value(s) of p is I (X ; Y) largest? For what value(s)
of p is it smallest? 2
0X PUXUY PC’XUY
0%? T(~K"} ‘= HAW L(>f}  (l (g
= "l balk“er “<1 + liq m m + an, (mm:
H J 0,: 6‘; I l
w ., W = a 0 a
Gig; ("Ty
Lory35e lﬂ=l [l H]
fwuul f‘“ [l H]
Mth My) .. The”; '3 45!”,(7'159 _ if“) _i_. 2 ._._
u... a; l.” (2°41: (0;?(1 77,0) 10 (b) (4 pts) Suppose that X and Y are jointly continuous with joint PDF 2 ifD g m 5 y S 1
3:: = .
H y) {0 otherwxse. Compute [(X; Y) and h(Y]X).
Hint: You may ﬁnd the following indeﬁnite integral helpful 11193 1
m m+1 _
[CE lnmdm—m ( 1 ( DZ). 5. (a) (3 pts) Suppose that a binaryinput discrete memoryless channel has
the property that for any 39 in [0, 1], the two input distributions [p 1 — p]
and [1 — p p] achieve the same mutual information over the channel.
Explain why the uniform input distribution must be capacity achiev
mg. [4 r0 L IO. M; Mm) + '11 7! my, f (m); é It :5er + 4; my); MM) WW WM u tut[1p [i ’1‘] Ifﬁlﬂ‘j Ham); ‘1 Liv.“ If; H] I 12 (b) (3 pts) Compute the capacity of the binarysymmetric—erasure channel: Wh6Y6620,520,ande+6g1,
" Ivaff CLovveE Jr'lil'rif‘r {'5'} j by 5pm,“! .‘yrml' "I th TL!“ H“) = lf+ Lij [I a] 13 ...
View
Full Document
 '08
 WAGNER

Click to edit the document details