prelim09 - ECE 5620 Spring 2009 Prelim Exam: Mar. 5, 2009...

Info iconThis preview shows pages 1–15. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 14
Background image of page 15
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 5620 Spring 2009 Prelim Exam: Mar. 5, 2009 Name: 7" --‘ II F’i -' r Rules: 0 Do not open this exam until you are instructed to do so. a You are permitted one letter-sized crib sheet. Otherwise the exam is closed- book and closed-note. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit. 0 Points may be deducted for incorrect statements. 0 You have two hours to complete this exam. Problem Points Score Good Luck! 1. Consider the channel consisting of a discrete memoryless channel (DMC) followed by an independent erasure channel (EC): The DMC has input alphabet X, output alphabet 3}, and transition proba- bility matrix W(y[:z:). The erasure channel has input alphabet y and output alphabet 37 = 3? U {6}. Its erasure probability is 1/2. (a) (i pts) Find the capacity of the overall channel X —> 17’. Simplify the answer as best you can. a»; 1 ri- (: wri 9“» A [/1 if Yfic‘ irl r m a. l rl / I (‘- l * “i I“ ‘X E- l =- 0.) -3 ' I i a}, 3 '? (4.! ‘u _L '* 1’ ‘5‘ a] " 1. [+[YIH1L 0/ i P: V E. .. 4 W- mm; mm A -F 1. ':. 73L T'hfilj a. _ J. a It???) : 1 CDMC. i r C -— 7,’ "I: rt) (b) (2 pts) Suppose that the encoder can (causally) observe the output of the DMC, but not the output of the erasure channel. Find the capacity. [fr-{ul— fcedimk 9w.» N4 ‘1“?"4’ fj'fc’L‘he‘L 1‘ TL“; (c) (2 pts) Suppose that, instead of erasing the symbols independently with probability 1 / 2, the erasure channel simply erases every other symbol. Thus the encoder and the decoder both know where the era- sures will occur. Find the capacity. 1?‘ ("6' 1"] 943'?! f‘) Wg'] (F‘Iére- :1 TL” ,0 MM 2. Let qbl be the density of a Na}, of) tandem variable. Let (fig be the density of 3 MG], 0%) random variable. (a) (3 pts) Compute D(gb1||q52). 0( ¢r 41) = I lint]le Jr 2: 4456;) " [find (org? "f(’ Jr- r’ ,1. —.— -ml) + elm-w +(wmfaw 0‘? H"? " 1 I 6‘ J- : f“ "' ——' H -: flay-K???- Jr '2. a: 936‘ r ._ w ‘lrg 1dr w-If-l'r E no rant“! d'"“" a (b) (3 pts) Let f be the density of a random variable, not necessarily Gaus- sian, with zero mean and variance 0%. Is it true that qfil is “on the way” from f to $2, meaning D(f||¢52) = D(f||¢51)+ D(¢'1|l¢2)? Explain {nu D(+I1d.fi) =1 [{{WL} )3: a ’9” f a: if"; I £1}in a + 2] h w ‘chw r .. i x? __I'_ I u 0mm + [aw My «4—: (3,: d» _ a) . 7i __ Uzi-“r . " ‘4‘ I'm! r " : DLFHJI) + J5“) 0::— " 47%|? tJLI‘ ‘ A.) I i:- -: offllér} + '3‘ Ivy 5;; '+ '2?wa i z; pHHCPI} 4 9W1” 9%) (c) (2 pts) Let f be as in (b). Let 9 denote the set of Gaussian densities with zero mean. Compute finale). M DUN): DUN.) + OWN?) Emit? 45.44 3. Let X.n be a random variable with alphabet {1, . . . , 5n} and distribution Pr(X —k)_ Wm”) ifn=1,...:2n “h _ 2mm ifn=2n+11___,5n Let Ln denote the average codeword length of a Huffman code for X ,3. Let Kn denote Hog(5n)], the average codeword length without compression. (a) (2 pts) Sh0w that H (Xn) grows with n as log n + c and compute c. HM"): 2" 6" [Fr 5‘" Tn “J "in it :BLIJ€“*;'[”? 2 A; Z” I T: In} K 4. I c} mug—m... L_____t._ ,...,.___J a: (b) (2 pts) Find a density f such that h( f ) = c. Plot f . or; :ovni UH-"Flrur‘ MW”) VW‘F or A: (c) (2 pts) Call Kn — Ln the compression amount. Bound the compression amount from above and below as best you can. _ _ j \cn—L“ ‘- HWWE ‘ “W e n} (row - ‘03“ * c 1:“ -L 3 ISM“? " M") “I p n - {what} ‘1'?“ 'c'r .n Mg 5-- '“‘ "C 4. (a) (3 pts) Show carefully that of all densities Over the finite interval [—a, a], the uniform density has the largest differential entropy. m y E an, Lv 91" 3-} fivt'r a! Lc'i 4:. J: an} arm-34'] LE; 7. it Uwi'fura. J‘N’J‘CJ.’ l r: O“; “9(Fl5g) '3 f'fi (Kiwis), elk -:. "' {a} '24 2-1; H4”) é h} (n:- mJ Us): 14. 10 (b) (5 pts) Let U be uniformly distributed over the interval [—a / 2, a / 2]. Consider the additive-noise channel with input alphabet [—a /2, a / 2] and Y=X+U. Determine the capacity of this channel. 4. C .; MM 1:0”) : w-‘g HU- Hm) I "' .J ,1”) 7m; Hm): I.) a. 2 p4.» m) e wit-«a w ~ m w- —:—/ =mx— r-w .5} C = l[}f?gl_ 19’s -=- {a} 2 l erk I! r'l' (c) (2 pts) Determine the zero-error capacity of this channel. Co E“. I C‘afolg ll 5. One way of measuring the “distance” between two discrete random vari- ables X and Y is to use the symmetrized conditional entropy d(X, Y) = H(X|Y) + H(Y|X). (a) (1 pt) Suppose that d(X, Y) = 0. What can we say about X and Y? Hism u: Him: : o 12 (b) (6 pts) Does this distance satisfy the triangle inequality, d(X,Y) +d(Y, Z) 2 d(X, Z) for all X, Y, and Z ? Prove or disprove. Yr: thc U R 44:94 (“fl-h: '3 Hf 691;; y)+J(\fi-?/ = HUI?) + HL‘ru) + MRI?) 4- HHIY/ W I[¥;1l‘r) + Hhtltr'é) + 1:(T'.‘e]*}+ HWY”) W + Irwin-e] + HUME?) + flxmw * HWY-KW WW mg) + gnaw) + 2HLYHJ?) '3 His; I s) + H L W L_.._.—-—--—("‘*-~«-~“_..M__... ..-.—.- n. )0. lrk / when) + NEW!) 13 6. “Turbo” Codes Consider a binary symmetric channel (BSC) with crossover probability p: Y = X EB Z , where EB denotes exclusive-or and Z is Bernoulli(p). (a) (1 pt) What is the capacity of this channel? «M C 2 |._ H (r) (In... (unit (b) (2 pts) Suppose now that the decoder observes the channel output both directly and through a second, independent BSC with the same crossover probability p. Decoder What is the capacity of this channel? Explain. C : WW TIfl’ffi) : Haul. Iixfli 4' ii}; )4“) Fir) {m d: l- H”) 14 (c) (2 pts) Suppose now that the second channel follows the rule Y=Y$X. What is the capacity of this channel? Explain. krel-3T '-' Ten!an = X. TLvr “Ha: Joule; (‘m *9361”; {VHH‘} £9 firk’w? ‘1' ya,“ C a C :3 I $rrv V UV-kfbiar) ~. !. - 113,4. C 4‘ Lrtv (I’M: r 7‘ *J «swan-M g C- : I Larry (d) (1 pt) What is the zero-error capacity of channel in (c)? % ‘nj V9eappr.--_: "“‘;=' 'H'I’s 1"! t7ka 15 ...
View Full Document

Page1 / 15

prelim09 - ECE 5620 Spring 2009 Prelim Exam: Mar. 5, 2009...

This preview shows document pages 1 - 15. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online