This preview shows pages 1–15. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 5620 Spring 2009 Prelim Exam: Mar. 5, 2009 Name: 7" ‘ II F’i ' r Rules: 0 Do not open this exam until you are instructed to do so. a You are permitted one lettersized crib sheet. Otherwise the exam is closed
book and closednote. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit.
0 Points may be deducted for incorrect statements. 0 You have two hours to complete this exam. Problem Points Score Good Luck! 1. Consider the channel consisting of a discrete memoryless channel (DMC)
followed by an independent erasure channel (EC): The DMC has input alphabet X, output alphabet 3}, and transition proba bility matrix W(y[:z:). The erasure channel has input alphabet y and output
alphabet 37 = 3? U {6}.
Its erasure probability is 1/2. (a) (i pts) Find the capacity of the overall channel X —> 17’. Simplify the
answer as best you can. a»; 1 ri
(: wri 9“» A
[/1 if Yﬁc‘ irl r m
a. l rl / I (‘
l * “i I“ ‘X E l
= 0.) 3 ' I i a},
3 '? (4.!
‘u _L '*
1’ ‘5‘ a] " 1. [+[YIH1L 0/
i P: V E.
.. 4 W mm; mm A
F 1.
':. 73L T'hﬁlj
a.
_ J. a It???) : 1 CDMC. i r
C — 7,’ "I:
rt) (b) (2 pts) Suppose that the encoder can (causally) observe the output of
the DMC, but not the output of the erasure channel. Find the capacity. [fr{ul— fcedimk 9w.» N4 ‘1“?"4’ fj'fc’L‘he‘L 1‘ TL“; (c) (2 pts) Suppose that, instead of erasing the symbols independently
with probability 1 / 2, the erasure channel simply erases every other
symbol. Thus the encoder and the decoder both know where the era
sures will occur. Find the capacity. 1?‘ ("6' 1"] 943'?! f‘) Wg'] (F‘Iére :1 TL”
,0 MM 2. Let qbl be the density of a Na}, of) tandem variable. Let (ﬁg be the density
of 3 MG], 0%) random variable. (a) (3 pts) Compute D(gb1q52). 0( ¢r 41) = I lint]le Jr 2: 4456;) " [ﬁnd (org? "f(’ Jr r’ ,1.
—.— ml) + elmw +(wmfaw 0‘?
H"? " 1 I 6‘
J : f“ "' ——' H
: ﬂayK??? Jr '2. a: 936‘ r
._ w ‘lrg 1dr wIfl'r
E no rant“! d'"“" a (b) (3 pts) Let f be the density of a random variable, not necessarily Gaus
sian, with zero mean and variance 0%. Is it true that qﬁl is “on the way”
from f to $2, meaning D(f¢52) = D(f¢51)+ D(¢'1l¢2)? Explain
{nu
D(+I1d.ﬁ) =1 [{{WL} )3:
a ’9” f a: if";
I £1}in a + 2] h w ‘chw r
.. i x? __I'_ I u 0mm + [aw My «4—: (3,: d»
_ a)
. 7i __ Uzi“r .
" ‘4‘ I'm! r " : DLFHJI) + J5“) 0::— " 47%? tJLI‘ ‘ A.) I i:
: ofﬂlér} + '3‘ Ivy 5;; '+ '2?wa i
z; pHHCPI} 4 9W1” 9%) (c) (2 pts) Let f be as in (b). Let 9 denote the set of Gaussian densities
with zero mean. Compute ﬁnale). M DUN): DUN.) + OWN?)
Emit? 45.44 3. Let X.n be a random variable with alphabet {1, . . . , 5n} and distribution Pr(X —k)_ Wm”) ifn=1,...:2n
“h _ 2mm ifn=2n+11___,5n Let Ln denote the average codeword length of a Huffman code for X ,3. Let
Kn denote Hog(5n)], the average codeword length without compression. (a) (2 pts) Sh0w that H (Xn) grows with n as log n + c and compute c. HM"): 2" 6" [Fr 5‘" Tn “J "in it
:BLIJ€“*;'[”? 2
A; Z” I
T: In} K 4. I c} mug—m...
L_____t._ ,...,.___J
a: (b) (2 pts) Find a density f such that h( f ) = c. Plot f . or; :ovni UH"Flrur‘ MW”) VW‘F or A: (c) (2 pts) Call Kn — Ln the compression amount. Bound the compression
amount from above and below as best you can. _ _ j
\cn—L“ ‘ HWWE ‘ “W e n} (row  ‘03“ * c 1:“ L 3 ISM“? " M") “I p n
 {what} ‘1'?“ 'c'r
.n Mg 5 '“‘ "C 4. (a) (3 pts) Show carefully that of all densities Over the ﬁnite interval [—a, a],
the uniform density has the largest differential entropy. m y
E
an, Lv 91" 3} ﬁvt'r a! Lc'i 4:. J: an} arm34'] LE; 7. it Uwi'fura. J‘N’J‘CJ.’ l r:
O“; “9(Fl5g) '3 f'ﬁ (Kiwis), elk
:. "' {a} '24
21; H4”) é h} (n: mJ Us): 14. 10 (b) (5 pts) Let U be uniformly distributed over the interval [—a / 2, a / 2].
Consider the additivenoise channel with input alphabet [—a /2, a / 2] and
Y=X+U.
Determine the capacity of this channel.
4.
C .; MM 1:0”) : w‘g HU Hm) I "'
.J
,1”) 7m; Hm): I.) a. 2 p4.»
m) e wit«a w ~ m w —:—/ =mx— rw
.5} C = l[}f?gl_ 19’s = {a} 2 l erk I! r'l' (c) (2 pts) Determine the zeroerror capacity of this channel. Co E“. I C‘afolg ll 5. One way of measuring the “distance” between two discrete random vari
ables X and Y is to use the symmetrized conditional entropy d(X, Y) = H(XY) + H(YX). (a) (1 pt) Suppose that d(X, Y) = 0. What can we say about X and Y? Hism u: Him: : o 12 (b) (6 pts) Does this distance satisfy the triangle inequality,
d(X,Y) +d(Y, Z) 2 d(X, Z) for all X, Y, and Z ? Prove or disprove. Yr: thc U R 44:94 (“flh: '3 Hf 691;; y)+J(\ﬁ?/ = HUI?) + HL‘ru) + MRI?) 4 HHIY/ W I[¥;1l‘r) + Hhtltr'é) + 1:(T'.‘e]*}+ HWY”) W + Irwine] + HUME?) + ﬂxmw * HWYKW WW mg) + gnaw) + 2HLYHJ?)
'3 His; I s) + H L
W L_.._.———("‘*~«~“_..M__... ...—. n. )0. lrk / when) + NEW!) 13 6. “Turbo” Codes Consider a binary symmetric channel (BSC) with crossover probability p:
Y = X EB Z ,
where EB denotes exclusiveor and Z is Bernoulli(p). (a) (1 pt) What is the capacity of this channel? «M C 2 ._ H (r) (In... (unit (b) (2 pts) Suppose now that the decoder observes the channel output
both directly and through a second, independent BSC with the same
crossover probability p. Decoder What is the capacity of this channel? Explain. C : WW TIﬂ’fﬁ) : Haul. Iixﬂi 4' ii};
)4“) Fir) {m d: l H”) 14 (c) (2 pts) Suppose now that the second channel follows the rule Y=Y$X. What is the capacity of this channel? Explain. krel3T '' Ten!an = X.
TLvr “Ha: Joule; (‘m *9361”; {VHH‘} £9 ﬁrk’w?
‘1' ya,“ C a C :3 I $rrv V UVkfbiar)
~. !. 
113,4. C 4‘ Lrtv (I’M: r 7‘ *J «swanM
g C : I Larry
(d) (1 pt) What is the zeroerror capacity of channel in (c)?
% ‘nj V9eappr._: "“‘;=' 'H'I’s 1"! t7ka 15 ...
View
Full
Document
 '08
 WAGNER

Click to edit the document details