# ch_14 - 14-1 14—2 W It suffices to show that[see(14-41...

This preview shows pages 1–12. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 14-1 14—2 W It suffices to show that [see (14-41)] H(A 'BlBj) = H(AIBj) Since AiBj k a j AinBj = and 12minj Inj) =P(AiIBj) {a} k # j (14-40)y1e1ds “(A oBIBj) = —i§k P(AinIBj) log P(A1BkIBj) If a <8, then ¢'(u) >¢'(B) because ¢'(0) ‘ ¢'(B) = 103(8/a) >0. Hence, b b+c I ¢'(0)dﬂ > I ¢'(a)du c >0 a a+c This yields p1+p2 p2 ¢(p1-sz) — ¢(pl) = J ¢'(a)da < I ¢'(a)du = ¢(p2) P1 0 Similarly ¢(p1-+e) - ¢(p1) - ¢(92) +¢(p2— 6) 4 p1 p1+6 p2 ¢'(a)da - I ¢'(a)da > o p2-€ ==a=msF___.__________________________________________________‘ 203 14-3 14—4 Applying the identity H(A1 0A2) - MAI) + H(A2IA1) (1) to the partitions All-A, Az-B - C and Al-A - B, A2 -C, we obtain the first line. The second line follows from the first [see (1)]. The third line is a consequence of the first two. M It follows if we apply the identity 1(A1.A2) ' MAI) + H(A2) - H(A1 ~A2) to the partitions A1 -A - B, A2 -C. 204 14-5 (a) From (14-53) and since (see Prob. I(A,B o C) = MA) + 1105 -c) - H(A o B -C) I(A.c) - um) + 3(a) - H(A - C) 14-4) 1m. -3 -C) - B(A -C) - 11(A onlc) - H(A|C) we conclude with (14-49) that I(A,B o0) -I(A -C) - male) + male) - H(A eBIC) If B -C is observed, then the resulting prediction in the un- certainty of A equals I(A, B -C). But, if B -C is observed, then C is observed, hence, the reduction in the uncertainty of A is at least I(A,C). Hence I(A,B - C) 3 I(A,C‘ with equality only if I(A,BIC) - 0, i.e., if in the subsequence of trials in which C occurred, knowledge of the occurrence of B gives no information about A. 205 14-6 14-7 14-8 The partition 11(A3) has eight elements with respective probabili tics 3 2 2 2 2 2 2 3 P 3? q’P Q9P Qapq 9Pq opq sq Hence 3 2 2 2 3 3 3 3 2 H(A ) = - p 108? - 3!: qlogp q - 3m logpq - q logq 2 2 2 2 " - 313(1) +2pq+q )logp - 3q(p +qu+q )logq = - 3plogp - 3qlogq 8 3H(A) The density of the RV g - x + :1 equals fx(w-a). Hence, H(§+a) = - J fx(w-a) 10g fx(w-a)dw a .. I fx(x) log fx(x)dx = 110:) The joint density of the RVs x and z =x+y equals fxy(x,z-x). Hence [see (14—30)] H(E|§) = — J J fxy(x,z—x) log fxy(x,z-x)4'x(x)dxdz —¢) —m G) on = — J J fxy(x,y) log fxy(x,y}éx(x)dxdy " “Zip —m -w W The RVs :5 and y take the values x iff x=x . = + i and y3 respectively {Ab-15 xi )7:] i and y ==yJ (assumption). Hence, {f=x1+yj}=(§=xi}n{z=yj] ~ This shows that Az =Ax - By. Furthermore, since the RVs-x and y are independent, the events {;§=xi} and {z=yj} are also independent. This shows that the partitions AK and B are independent and [see (14-44) and Prob . 14-1] Y 206 H(AzIAx) = H(Ax -AyIAx) = amyle) = Buy) From this it follows that H(§I§) - HQ!) because [see (14—88) "and (14 -41) ] ugly - H<AZIAX) 14-9 As we see from (14-80) HQ) 8 lna where we assume that aaNG. The RV y takes the values 0,6,...,(N—1)6 with probability UN. The conditional density of § assuming y-k6 is uniform in the interval (k6,k6 +6). Hence, k6+6 H(xly=k6) a -J k6 f(x|y-k6)£n f(x|y=k6)dx = in 6 And as in (14-41) N H(x|y) =2 H(x|y=k6)P{y=k6} - 1:16 - - k'O I. ~ ' Finally [see (14-95)] Hag!) = H(§) - 8635'!) = lna- 2:16 W. 14-10 If yi-g(x1), yj-g(xj) and yi-Iy:l then xi=xj. Hence, Pi i=3 p.Lj - : pi = P{§=xi} and 207 14-11 14-12 From Prob. 10-10 it follows with g(x) - x that 1105,15) 1- 11(3). And since [see (14-103)] 1105,35) - H(§|§) + H05) we conclude that H(x|x) = 0. From Prob. 14—3 it follows that H(y,x|x) - nu -AxIAx) - aux. Ax) + no.ny -Ax) - ~ - y - H(AyIAx) - mzlf) becauseA -A -A and H(A -A ) -H(::,::) -0. x x x x x .. WW E{x}-0 E{x2}-S E{y}-0 2 " -2k 2 -20 2 Hzn} - 2 2 E{§n_ L7 E{§nzn} - Ehfn} = 5 k-O (a) From (14-95) ,(14-84) , and (15—86) with 1111 -5, 1122 =20/3, and ulz =5 11(3) - 2n¢101re HQ) s lnMOne/3 Hng) = lnlOnt/fi- I(x,y) = 9412 (b) The process y(t) is the output of the system L(z) =..—1._._:1 £0 1 ‘005 z = l with input 35“. Since 13(3) = HQ) and [see (12A-1) 3117 I ln|L(ej¢)|d¢ ¥ 2112.0 = 0 -1r (14-133)y1e1ds ﬁ(y) = {1(35) = 11(5) - £m’101re. 208 14—13 6 .. 1 H(§)-H(§)--%J£n§dx-£n2 4 And as in Prob. 14-12 with 2.0 I 5, 171(y) - I-l(x) + in 5 - in 10 14—14 Given that f(x) - 0 for |xl>l and E{x) - 0.3, find f(x). With g(x) = x, (14-143) yields f(x) - Ae""‘ where 1 A A I 9"" dx - -—(e" - ﬂ) - 1 .1 A 1 A I -1 xe"‘* dx - 4:70" - e"‘) - 35—0.," - e“) - 0.31 Solving, we obtain A :2 0.425, A z - l ---- gu————_———-—-——-n—--—--—-—--—_--—-——-———-——_---—----————-———_-—--—————-- 14—15 {(x) - Ar“ for l<x<5 and 0 otherwise, 5 5 A J- 9"" dx - 0.31 A I xe-de - 3 3-7- 1 , 60 209 14—16 From (14-151) with xk=k, 310(k) = gl(k) = k, k=l, ..., 6 ( ) {o k-1,3,5 Ac“ k-l,3,5 k l k-2,4,6 k 641"": k-2,4,6 Since p1 + p, + p5 = 0.5 and E{x} = 4.44, we conclude with z = e42 and w = e'A2 that A(2+23+25) = Aw(zz+z4zs) A(z +3z3+525) + Aw(222+4z4+6z°) = 4.44 This yields A a: 0.0437, 2 = l/w g 1.468 14—17 (a) The transformation y = 3x is one-to-one, hence, H(y) = H(x) (b) From (14-113) with g(x) = 3x: 111(2) = HQ) + in 3 l l - l4 ?, P(ll) = —— P{neither 7 nor 1]) = —— 14—18 (a) For fair dice, P(7) = 18, 18 l l l 1 l4 l4 HA=- —— —— — —— —— =. () [6ln6+18£n18+18£n18) 0655 (b) From (14-10) with n=100 and N=3: nT g enHWg 2.79 x 1028 na 2 N“ g 5.16 x 1047 210 14 —19 The process xn is WSS with entropy rate ﬁ(x). Show that, if n 8“ =k§0 r“-k ﬁk then lim—l—lﬂw ,...,w) =ﬁ(x) +£n|l I (i) n+1 ~o -n 0 11+» Proof. The RVs genuyn are linear transformations of the RVs xo,...,xn and the transformation matrix equals 9.0 0 ... 0 2.1 9.0 ... 0 in gin-10¢. 0 Since the determinant of this transformation equals [2. (14—115) yields I n+1 o I H(yo.-u.yn) = 11(§°,...§n) + (n+1)£nl£ol Dividing by (n+1) we obtain (31) as n-W’. 14—20 As in Example 14-19, f(p) = A e-Ap. To find A, we usc the k—n curve of Fig. 14-16. This yields A = - 1.23 f(p) = 0.51 e1'23p 211 14-21 As in Example 14-22,!)k = A e-‘Xk. To find A, we use the w-r. curve of Fig. 14-17. This yields (see also Jayncs) w = 1.449 A = - 0.371 pl 92 D3 94 P5 P6 0.051. 0.079 0.114 0.165 0.240 0.348 14-22 The unknown density is normal as in (14-157) Where 4 1 1 A = 1 4 -— lo 2 +' 2 + 56 "‘23 “‘23 m23 1 m23 4 The moment x323 = E{x2x3} must be such as to maximize A. This yields 11123 = 0.25. 14-23 6AMMOM 212 Illa II/I to NO 213 14-24 If 25“ = 0, then J-tn - 000 and g“ = 1 iff §n consists of one 0 or no zeros. The probability of one and only one zero equals 382(1..s) [see (3-I3)]; the probability of no zeros equals 83. Hence, M2“ - 1|35n - 0} - 382(1-8) + 83 Thus, the redundantly coded channel of Example 14-29 is symmetrical as in (14-191) with probability of error 81- 82. 'W 14-25 If the received information is always wrong, then Pfyn=1l§n=0} = B - 1, hence C - 1—r(8) = 1 214 ...
View Full Document

{[ snackBarMessage ]}

### Page1 / 12

ch_14 - 14-1 14—2 W It suffices to show that[see(14-41...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online