This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 562 Spring 2008
Prelim Exam: Mar. 6, 2008 Name: ﬁlu‘h 04: Rules: 0 Do not open this exam until you are instructed to do so.
0 This is exam is openbook and opennote. o No calculators are permitted. 0 No collaboration is permitted. 0 There is no penalty for guessing. 0 You must show your work to get full credit. 0 You have two hours to complete this exam. Problem Points Score Good Luck! 1. Consider the following discrete memoryless channel
1‘ Y I '1. .7 '1 3' I 1/2 1/6 1/3 0 0
1 1/6 1/2 1/3 0 0
7 0 0 1/3 1/2 1/6
a 0 0 1/3 1/6 1/2 (a) (2 points) Let X and Y denote the input and output, respectively, to
the channel. Suppose'that X is uniform. Compute I (X ; Y). Clix1y} : MY) — HUI!) IL ‘L i: ourlm. 1L Jeanmum a? ‘r 5! H(Y] — dim2— *;ii é : ‘ql'lI’ 2 ’ Jc'lq ‘ ‘ J"}%
.L .1 —'
I‘Kl‘r) '3 ""lwg— "I" (Iqé' ‘1' 11v) 2 Nee) *1 '03” ii; 0'4in hard ):,4,,'[u4'.°m
1‘{!;‘r) = HIV) — HUN)
Hle) I", n“ hp”! 0“ P”), /0 Cluue f’lr/ “N
Htifuut31t l :{ ‘r=3
E 2 Ldlwf
0 0+ (c 0m; “,4. germ; 6‘
¢ the)
Ht‘f) = H(Y)+ HlEIY) : H(YIE) : MEN HUI“)
=0.
Am a; 2 C Cal} VIA"L, Jthud]
‘ "er TLV; 'Hc
TL] Lr [MLen; if fl!) u Wu:
mam W Armum u new, M (b) (6 points) Determine the capacity of the channel. , 2. An Even Spread? Let X be a sequence of i.i.d. Bernoulli(1/2) random variables with length
872. We call the number of ones in X its weight. (a) (3 points) Ciiven that X has weight 4n, consider the event that the ﬁrst
411. bits of X have weight n and the last 4n bits of X have weight 3n.  Compute or upper/lower bound this conditional probability as best you
can. A 7. g 3? Lu WefJb'l ‘lVlS
3:5 V! at m: .t 't w mu «1  Jl _ n MinE
I I “EN1L) L "i 1
TLuvrm “M “0,2 4 PC / _ 2
_ “ .1 1L
1 {wavKﬁllE) L F c) 2 u olm
@un)‘
\ 2En°(zll%): " 19 nos 1.
(gm “)1 Linll)
New): “3"” : mm = rlwrtc)
F4") P(A)_
\ .l
 ?ub{'\l.ll{ L ~(:mp(;\,,/
d ‘_ full} '2 I
L¥“_H)V 2 — L (b) (3 points) For large n, is this conditional probability large or small?
Explain. TL; cippvup¥ coowleer 'H‘e. roL) Mum,1 "F0 OH», / K?.c‘A)4>b d'J‘ vb...» ,5 (c) (3 points) Given that X has weight at least 5n, consider the event
that X has weight at least 611. Compute or lower/upper bound this
conditional probability as best you can. A  \xu Weak“ d’f leor‘l’ ﬂ} 3;{1 I“, Wow'3‘“ t4 luﬂ‘ Ga} 7" K i 5" 0(L “.L
‘ 1) _FA f“ i
t HM: i 2 e (r 5 Z '2
TL onw L?“ “)1, kg“ P ) K=(‘
\_______._..—__._..._.J > 2 9uD(% Mi] I new = '2’? + L3” = ,_ Mi); ,_L(_f.) a! kan Fu
9 {fad—EM) é (nitriharﬂl)
n=tn
g (vh‘up 7:?ng é m/ 5 (9 “*9 2‘ g“ DEM)
meut (hum —r.‘v({h:)é HA) C tam) 29» DEL[Hi
rlle) = 53%;) = rig)
I 2;l(v(%ntl blﬂliUé POW é @A+I/2(2W“) rn(b{{u (d) (3 points) For large n, is this conditional probability large or small?
Explain. r 6* 9+ J1 HIM‘lf
0(3f’llé) “9(7Ilql_))o an e rem.» w J. FLVH >I’ a: m—a—A.
n‘2 nowhl 'p'c‘hr’ r' I 3. Suppose that X1 is a Bernoulli(1 / 2) random variable. Let X2, X3, X4, ...be
generated from X 1 via a cascade of binary symmetric channels " BSC X BSC X5 BSC "' The channels are independent, and each has crossover probability p 171 29
P 1—P' (a) (3 points) Compute H (X 1, X2) in terms of p. maxi) = Hlm+ Him») Him H H
JP
:5
x
,1
,_..
‘C
V!
o
\u
+
r4—
I
L
J
L‘
I.
\_/ Hi‘lwiyi) (b) (4 points) Compute Hm H(X1, . . . ,X"). 11—00 17, _ n H = ‘ 6:
Wax.) — g H H,...,x ) {mtg X‘ee )(K b9 9' Xv.) H U;\x.,..., L..) = uncut") = umlut) = Mr). A — )
1 __._“”"”°“ = ___—' 5‘“ “4” = um.
I!“ n V.
“'5” 4. Fixfree Codes
A code is called fixfree if it is both preﬁxfree and sufﬁxfree. (a) (3 points) A Harvard student claims that if there exists a fixfree code
with codeword lengths £1, ..., 6,, then Is he correct? Why or why not? No (own ~11; to}: ctr) = 0 ,Q‘ =‘ C(2J=" ’{1=‘. “(‘ = u _.
TL; (a It Ff a", {'69 L” {z 1 I 1 [ Tlg. Convcue I‘V'hv‘MJ’ "I a“ 0'“ {anew k“ 4h lull] 10 (b) (3 points) Is it possible to have a ﬁxfree code with codeword lengths e,,...,en such that 3
._[‘ = —?
Z? 4 1, If so. give an example. If not, explain why not. Ye}. (ouzhr +11 to): CU) = 00 cm :01 ct?) l o 7‘"! {11: _
TLc C0 ck I'J’ 4}? ‘ 'r' ’9, M, I 11 5. A SecondAChance I Suppose that W is a random variable with alphabet W. Let W1 and Wg
denote two guesses of W. Let Pc denote the probability that both guesses are wrong . A
PC = Pr(W =,£ W1 and W # W2). Since there is no point in repeating a guess, we assume that W1 75 Wz as. (a) (6 points) Show that
Hm) + Pelog(W — 2) + (1 — PC) 2 H(WW1, W2). ‘ ;[ w ?al 5" E : O °+LcrwJL 6
A A A «
leElvzl'ﬁJ : lew,/w.,) + lWWzIW ~= HlElv‘vUvi/t) + lelvinkhs). =9 mwm. v1.) = HtElv'G‘lv'L) + t4lwlv'0.’ﬁ"s) 12 (b) (3 points) Under what conditions does this inequality hold with equal
ity? U) E 1] (Quaﬁ) w :3 cynlhj Milly +0 ts»! II
C (2) (was. E J W{ 04" Wt
 ‘ ‘. {fuL‘ Cn‘via E : W U tyudl’ ‘ +0 +Lg VAN: l'\ . W\fw',w73 13 6. A Second Chance 11 Consider a channel coding scenario in which the decoder gets two chances
to correctly guess the transmitted message. We say that an error has oc
curred if both guesses are incorrect. Let 02 denote the capacity of the chan
nel under this setup, and let 0' denote the usual Shannon capacity. (a) (2 points) Explain why 02 2 C. (gunk, ‘~ fence“. 0! (Ignlw) Caroc!*\)  ackrzyrnj COJCJ‘ el'u’ r"‘. 40 EOCL C‘A Lg. +Vrnca Wk"! 9* +WV' jut/j (“’1 J.) WAI‘M) “Vi a,L:l,”q fun} jinn. .fr'vwg WC Hill ‘Mvt l4 C énrfict W IND? (b) (6 points) Does Cg = C? Prove or disprove.
of (7" K, a)
x“ = 4‘ l w). a (Na K
‘ OVCI 2"Inu’2‘} fey» (Mg ‘I’es, w] P: > or (WI Hg“) = 7( é; H(Pc) + Pctnﬁ +(lY:)
waw => itw'.».,v>z)2
l
:25 waul’eVH ’—I‘+c
=> (ea 2’? C255 $ Cz—c. 15 LA
I ). ru'uﬁ “ ...
View
Full Document
 '08
 WAGNER
 Information Theory, Conditional Probability, Probability, Probability theory

Click to edit the document details