prelim09

# prelim09 - ECE 5620 Spring 2009 Prelim Exam Mar 5 2009 Name...

This preview shows pages 1–15. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 5620 Spring 2009 Prelim Exam: Mar. 5, 2009 Name: 7" --‘ II F’i -' r Rules: 0 Do not open this exam until you are instructed to do so. a You are permitted one letter-sized crib sheet. Otherwise the exam is closed- book and closed-note. o No calculators are permitted. 0 No collaboration is permitted. 0 You must justify your answers to get full credit. 0 Points may be deducted for incorrect statements. 0 You have two hours to complete this exam. Problem Points Score Good Luck! 1. Consider the channel consisting of a discrete memoryless channel (DMC) followed by an independent erasure channel (EC): The DMC has input alphabet X, output alphabet 3}, and transition proba- bility matrix W(y[:z:). The erasure channel has input alphabet y and output alphabet 37 = 3? U {6}. Its erasure probability is 1/2. (a) (i pts) Find the capacity of the overall channel X —> 17’. Simplify the answer as best you can. a»; 1 ri- (: wri 9“» A [/1 if Yﬁc‘ irl r m a. l rl / I (‘- l * “i I“ ‘X E- l =- 0.) -3 ' I i a}, 3 '? (4.! ‘u _L '* 1’ ‘5‘ a] " 1. [+[YIH1L 0/ i P: V E. .. 4 W- mm; mm A -F 1. ':. 73L T'hﬁlj a. _ J. a It???) : 1 CDMC. i r C -— 7,’ "I: rt) (b) (2 pts) Suppose that the encoder can (causally) observe the output of the DMC, but not the output of the erasure channel. Find the capacity. [fr-{ul— fcedimk 9w.» N4 ‘1“?"4’ fj'fc’L‘he‘L 1‘ TL“; (c) (2 pts) Suppose that, instead of erasing the symbols independently with probability 1 / 2, the erasure channel simply erases every other symbol. Thus the encoder and the decoder both know where the era- sures will occur. Find the capacity. 1?‘ ("6' 1"] 943'?! f‘) Wg'] (F‘Iére- :1 TL” ,0 MM 2. Let qbl be the density of a Na}, of) tandem variable. Let (ﬁg be the density of 3 MG], 0%) random variable. (a) (3 pts) Compute D(gb1||q52). 0( ¢r 41) = I lint]le Jr 2: 4456;) " [ﬁnd (org? "f(’ Jr- r’ ,1. —.— -ml) + elm-w +(wmfaw 0‘? H"? " 1 I 6‘ J- : f“ "' ——' H -: ﬂay-K???- Jr '2. a: 936‘ r ._ w ‘lrg 1dr w-If-l'r E no rant“! d'"“" a (b) (3 pts) Let f be the density of a random variable, not necessarily Gaus- sian, with zero mean and variance 0%. Is it true that qﬁl is “on the way” from f to \$2, meaning D(f||¢52) = D(f||¢51)+ D(¢'1|l¢2)? Explain {nu D(+I1d.ﬁ) =1 [{{WL} )3: a ’9” f a: if"; I £1}in a + 2] h w ‘chw r .. i x? __I'_ I u 0mm + [aw My «4—: (3,: d» _ a) . 7i __ Uzi-“r . " ‘4‘ I'm! r " : DLFHJI) + J5“) 0::— " 47%|? tJLI‘ ‘ A.) I i:- -: ofﬂlér} + '3‘ Ivy 5;; '+ '2?wa i z; pHHCPI} 4 9W1” 9%) (c) (2 pts) Let f be as in (b). Let 9 denote the set of Gaussian densities with zero mean. Compute ﬁnale). M DUN): DUN.) + OWN?) Emit? 45.44 3. Let X.n be a random variable with alphabet {1, . . . , 5n} and distribution Pr(X —k)_ Wm”) ifn=1,...:2n “h _ 2mm ifn=2n+11___,5n Let Ln denote the average codeword length of a Huffman code for X ,3. Let Kn denote Hog(5n)], the average codeword length without compression. (a) (2 pts) Sh0w that H (Xn) grows with n as log n + c and compute c. HM"): 2" 6" [Fr 5‘" Tn “J "in it :BLIJ€“*;'[”? 2 A; Z” I T: In} K 4. I c} mug—m... L_____t._ ,...,.___J a: (b) (2 pts) Find a density f such that h( f ) = c. Plot f . or; :ovni UH-"Flrur‘ MW”) VW‘F or A: (c) (2 pts) Call Kn — Ln the compression amount. Bound the compression amount from above and below as best you can. _ _ j \cn—L“ ‘- HWWE ‘ “W e n} (row - ‘03“ * c 1:“ -L 3 ISM“? " M") “I p n - {what} ‘1'?“ 'c'r .n Mg 5-- '“‘ "C 4. (a) (3 pts) Show carefully that of all densities Over the ﬁnite interval [—a, a], the uniform density has the largest differential entropy. m y E an, Lv 91" 3-} ﬁvt'r a! Lc'i 4:. J: an} arm-34'] LE; 7. it Uwi'fura. J‘N’J‘CJ.’ l r: O“; “9(Fl5g) '3 f'ﬁ (Kiwis), elk -:. "' {a} '24 2-1; H4”) é h} (n:- mJ Us): 14. 10 (b) (5 pts) Let U be uniformly distributed over the interval [—a / 2, a / 2]. Consider the additive-noise channel with input alphabet [—a /2, a / 2] and Y=X+U. Determine the capacity of this channel. 4. C .; MM 1:0”) : w-‘g HU- Hm) I "' .J ,1”) 7m; Hm): I.) a. 2 p4.» m) e wit-«a w ~ m w- —:—/ =mx— r-w .5} C = l[}f?gl_ 19’s -=- {a} 2 l erk I! r'l' (c) (2 pts) Determine the zero-error capacity of this channel. Co E“. I C‘afolg ll 5. One way of measuring the “distance” between two discrete random vari- ables X and Y is to use the symmetrized conditional entropy d(X, Y) = H(X|Y) + H(Y|X). (a) (1 pt) Suppose that d(X, Y) = 0. What can we say about X and Y? Hism u: Him: : o 12 (b) (6 pts) Does this distance satisfy the triangle inequality, d(X,Y) +d(Y, Z) 2 d(X, Z) for all X, Y, and Z ? Prove or disprove. Yr: thc U R 44:94 (“fl-h: '3 Hf 691;; y)+J(\ﬁ-?/ = HUI?) + HL‘ru) + MRI?) 4- HHIY/ W I[¥;1l‘r) + Hhtltr'é) + 1:(T'.‘e]*}+ HWY”) W + Irwin-e] + HUME?) + ﬂxmw * HWY-KW WW mg) + gnaw) + 2HLYHJ?) '3 His; I s) + H L W L_.._.—-—--—("‘*-~«-~“_..M__... ..-.—.- n. )0. lrk / when) + NEW!) 13 6. “Turbo” Codes Consider a binary symmetric channel (BSC) with crossover probability p: Y = X EB Z , where EB denotes exclusive-or and Z is Bernoulli(p). (a) (1 pt) What is the capacity of this channel? «M C 2 |._ H (r) (In... (unit (b) (2 pts) Suppose now that the decoder observes the channel output both directly and through a second, independent BSC with the same crossover probability p. Decoder What is the capacity of this channel? Explain. C : WW TIﬂ’fﬁ) : Haul. Iixﬂi 4' ii}; )4“) Fir) {m d: l- H”) 14 (c) (2 pts) Suppose now that the second channel follows the rule Y=Y\$X. What is the capacity of this channel? Explain. krel-3T '-' Ten!an = X. TLvr “Ha: Joule; (‘m *9361”; {VHH‘} £9 ﬁrk’w? ‘1' ya,“ C a C :3 I \$rrv V UV-kfbiar) ~. !. - 113,4. C 4‘ Lrtv (I’M: r 7‘ *J «swan-M g C- : I Larry (d) (1 pt) What is the zero-error capacity of channel in (c)? % ‘nj V9eappr.--_: "“‘;=' 'H'I’s 1"! t7ka 15 ...
View Full Document

{[ snackBarMessage ]}

### Page1 / 15

prelim09 - ECE 5620 Spring 2009 Prelim Exam Mar 5 2009 Name...

This preview shows document pages 1 - 15. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online