This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 562 S pring 2007 Prelim Exam: Mar. 8, 2007
Name: gllv‘l‘uﬂ Rules: 0 Do not open this exam until you are instructed to do so. 0 This is exam is openbook and opennote. o No calculators are permitted. 0 No collaboration is permitted. 0 There is no penalty for guessing. 0 You must show your work to get full credit. 0 You have two hours to complete this exam. Problem Points 1 6 Good Luck! Score H 1. Let X1, X2, . . . ,Xn be independent.
(a) Show that 'I(>1\,...,xn;\’) = %I(KL~;Y]X.,...,Y:.I) (7 = :(Ettxttx....., m)
— Hm 1 x,‘ x0” Y/] M ,
= E [ mg) — Htx: l xwx VJ]
(L1 Evie/win”) 2 % CHUU— HUJYH \:l (mu: («vi'+\‘ar.'u) "(IVT’J‘ ”JF’W‘I) 2 12:: I(>‘c’/Y) (b) Give a collection of Markov chain conditions that is necessary and
sufﬁcient for equality in (a). 2. Let X1, . . . ,Xn be i.i.d. Bemoulli(p) and let Y1, . . . ,Yn be i.i.d. Bemoulli(q).
Show that 1
—_T—L10ng(Y'1’ " ' aY’n) converges in probability and determine the limit in terms of p and q. a1 to; m (n... KY “1 We»; \«w of ”we. VW““", +L‘l’ (”V‘U" +0 E ~ Ia, mm] = ”W; F — (ewe; (lr)‘ 3. Let C be a lossless source code for a binary source.
C : {O,1}" I—> {0,1}*. Suppose that O is uniquely decodable, and that it makes a least one string
shorter £0?) < n for some 50’. Show that it must then make at least one string longer ﬂy“) > n for some ﬂ. nrrm +Li+ (v.1 ﬁCﬁzj C A p“ JEN' 32; We w?“ )nw A (”+"l‘lﬁu‘. ﬁw‘ 7‘ ' i 2’19)? f '2’“ : ‘1? 6‘00“ 3'8 éSPG‘G"
{'Mtc, [0(2) 4 M [a] (ow; '78 ‘Hzr Iueﬁwktﬁl Wurd
L9 f+r::‘l, l?w} ‘HH/ (0"4"J‘.‘+f kuﬂ‘, lyr;u.l‘iy ‘fvl‘c‘e Ht (0‘): if V“‘)‘vel\) /?laJa’rl/g This page intentionally left blank. 4. Compute the capacity of the following channel in terms of p and q.
0 l 2 3’ ”[1‘P—q 0 p q
‘ 0 110—61 q 27 Here 10 and q are nonnegative numbers satisfying p + q S 1. C = WW! 10190 = MN HM" HMS!)
pm PM “(THi v‘r "W‘erwmt— ,I Ptk)‘ 10 CLMMQU
U/ {VlfU/L Fouow Jﬁ/JVA’I‘OA
\ ff ‘1’ '2— '2 Cf Y=j : O MCVWCJf m 0
HH 5/: Me) + Hms) : Hm ,, “(c r/
EMM ‘por mm) (W) HCE) = MPH»), HWIE) = H(YIE=I) Nth) 1+ HWIE: o) rm: :0) g [. (NEH) + lv F/(Eze) =l w) quit“ if “‘0 ﬁr
V“:(‘QIM __g (Lug you UM3'A W“. 'I'lm c —. \M‘t’) — Mm) = — 6mm} lv (12/ ‘(W/l”) m
'L 7. WWW“) (WW
7 + rm + 7h; } 5. Consider two channels, p1(y:r) and p2(yx), in parallel. message channel X 17 (M53) \f p (ylx) 2 channel message
W encoder 1 2 decoder W Suppose the ﬁrst channel has capacity 01 and the second has capacity 02.
Let C denote the capacity of the two channels in parallel. (a) Show carefully that C S min(Cl, 02). C 7 NM TU)?”
(M For an) (9", ‘9 JA" 9”“ I’”"”""J "WW”, I(,(;'2) é IMHY)‘ (b) Suppose that p1 (ylx) is a binary symmetric channel (BSC) with crossover
probability p and p2(yx) is a BSC with crossover probability q. De
termine C in terms of p and q. Does equality hold for the bound in (a)
in this case? 0 "—M O O o
F ‘3
V ?
0 0 (J 0
l’f ”F
TkC/ :f v1vi‘Va'mJ» +3 ‘Hag CL‘“"€'
l' l"
d
f‘
Y‘
I r
W" r= Mw + IMO. E '4! H (a) J“ '1' ”'4‘ ”fftffln'l‘l Ln”: +gkc
iVA l
M I 3 L 2 .t;: > _L t
V = 77 '9 "’ ‘r w ' V '1 r=i‘ .1
Y. (c) Suppose we now allow coding between the two channels. The box
marked “coder” can implement arbitrary transformations. message
W message What do you think the capacity of this system is? Justify your answer
as best you can. ‘ c = m C". (2/. V a
TL; CQJCI c,“ Jere/a 'HL WU»): RV ‘ ‘ (0W W‘Vw'f.“ :4“
Q’f" Mli)ﬂ«v 0’9 ((CM(‘JQ 1" RI I +La ffcuwo CLIVHL TLff W41” Clﬂlzli lO 6. Consider two realizations of an i.i.d. source with alphabet {a, b, c, d}. "'1 abdbcbba x
a? baadcabc (a) Determine the probability of the second string under the maximum
likelihood distribution of the ﬁrst. VIN. Jv+a.‘Ltt.~M .r ‘“< “the. III
_ 31;; :[i_——]
F32,_[8 7 5 5] t 2 9 ’ ll We. (b) Find the i.i.d. distribution p that maximizes 1 _’ 1 _.
5103P($1)+ ; 10gp(:y2). 7)? .4: +1»: {widenHan aP 1?: $5”,th
.4
7:
Wm} ‘Hne its JvhttmM Huh» wipwhf
HEM'QJ =1 H?)
.3
U. +L¢ +\)YG '£ 'x 12 (c) Express the distribution in (b) in terms of the types of (2'1 and :32. *‘iYﬁ (hf 72 U. fiwyir/ “HM. dVﬁvAJc 13 "P ...
View
Full Document
 '08
 WAGNER

Click to edit the document details