This preview shows pages 1–13. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 562 Spring 2007 Prelim Exam: Mar. 8, 2007 Name: Rules: 0 Do not open this exam until you are instructed to do so. gliu+{on5 o This is exam is open—book and open—note. o N o calculators are permitted. a No collaboration is permitted. 0 There is no penalty for guessing. 0 You must show your work to get full credit. 0 You have two hours to complete this exam. Problem Points Score
1 6 Good Luck! 1. Let X1, X2, . . . ,Xn be independent.
(a) Show that I(X1, . . . ,Xn;Y) 2 21(Xi;Y). i=1 — H()(g ) x,’ xm/ W] M ,
z LEEI—xblc)“ Hixclxxwuxcqﬂﬂ
(Ly iv/tlwlthe/ 3 [MN— Mner (fx‘wg (ml4Mn6u) re/vn, en+,.r\/) :1? THU/V) (b) Give a collection of Markov chain conditions that is necessary and
sufﬁcient for equality in (a). 2. Let X1, . . . ,Xn be i.i.d. Bernoulli(p) and let Y1, . . . ,Yn be i.i.d. Bernoulli(q).
Show that l
_;L'10ng(Y1:  ,Yn) converges in probability and determine the limit in terms of p and q. 17' l0) P107“... 1?»! 11¢ Wm law, at law NM“: H” Wva *0 3. Let C be a lossless source code for a binary source.
0 : {O,1}” +—> {0,1}*. Suppose that C is uniquely decodable, and that it makes a least one string
shorter €(3'E) < n for some 33’. Show that it must then make at least one string longer [(37) > n for some 37. furrom +11"?
M; 2(2)“ V >2 “,1 'pw ﬁwt w( w?“ Jraw & (°'+”“¢‘l3an. ﬁ‘wc This page intentionally left blank. 4. Compute the capacity of the following channel in terms of p and q.
0 I 2 ? (’[1—17—(1 O p q
‘ 0 1p—q q p Here p and q are nonnegative numbers satisfying p + q S 1. (= M” IWW) = MN HM'Hl‘ﬂv/
PH) plus) Mm) "r “WWW” “I PW annvelf
FOHOW Je/.‘V‘+,'an A, {w;fU/l.
\ "'c '1’ '3 '2 Ir Y5:
E r. O MérwﬁJ—e C)
H” E); ME) + HUME) = MW 1» \At’ W 3th per mm) raw] lHE) : MN». Heme) = H(TIE=:)VI(E:1) » HWIE: 0mm“; 5 («(6 :1) + \rF/(E:¢) :1 w) qwlziy if “y, if
Wain.“ —§ C LN» YO?! Vw‘lsvw, The“ 'L C —. HlY) — Mm) ‘ ‘ CF‘Y'Ul')(""__/’ ‘CV‘WIU a";
'7. WWW/W (“w
7 + (in)? a)» 7’ 5. Consider two channels, p1 and p2(y}$), in parallel. message channel X p Y p a channel message
W encoder 1 2 decoder W Suppose the ﬁrst channel has capacity 01 and the second has capacity 02.
Let C denote the capacity of the two channels in parallel. (a) Show carefully that C S min(C'1, 02). C 1:: NM 17(X‘f3/ (5*! F0 r In) (WI, Lt; “He. 9M1 In a : Hrs») f“ qwl'lv" ler m (b) Suppose that p1(y:r) is a binary symmetric channel (BSC) with crossover
probability p and p2(yx) is a BSC with crossover probability q. De— termine C in terms of p and q. Does equality hold for the bound in (a)
in this case? o \"P a0 a "$ 0
f’
1
V s
0 0 d~/‘ #0
try ’“F a 1'!”
r~
V.
lr
\IVLcrz \r‘; (0—;/ 4. Xlﬂpj’
1*)
 :1
E ' W‘ (a; JD?! “7+ “ftp/f,,.'(‘/ +H€C r ‘ ; "iVl" '
ﬂ“ . 7 r 2 , 3. > _L
V: 17 7r "' w‘» ‘" t “I
—> "Ht'><i~H(rI—: .mm
9 (c) Suppose we now allow coding between the two channels. The box
marked “coder” can implement arbitrary transformations. message message What do you think the capacity of this system is? Justify your answer
as best you can. ' c = m (6'. 62/
R‘V‘ 'H‘c TL; (chi (an. Jere/c 'H; Weave ‘ (OWMVvI‘t.“,'gn
(Zr/4 ewak, m9 l/(cw‘h It I], ~ “He. (“"9 CLIWML will Olen], Oval [(‘L‘cvc, Vo*£. W‘Iv‘ (Cal (a,in lO 6. Consider two realizations of an i.i.d. source with alphabet {a, b, c, d}. (a) Determine the probability of the second string under the maximum
likelihood distribution of the ﬁrst. 11 We, (b) Find the i.i.d. distribution p that maximizes 1 H 1 #
510gP($1)+ ; 10gp(:v2). "I; $g +L¢ (Ivrdeu'Hlm MC TC”; "05 "MINI
a
7:
WM“ Jif‘iriivh‘cu Wk)”: w‘wv‘”
«» A M33.) HE) :« M x) .3, .3, ~ng “My: up "x 12 (c) Express the distribution in (b) in terms of the types of 11—51 and :32. m if ﬂ W’r/ 13 ...
View
Full
Document
This note was uploaded on 10/03/2011 for the course ECE 5620 at Cornell University (Engineering School).
 '08
 WAGNER

Click to edit the document details