Assignment 1A Solution

3076171875 4 out57 c a b a the entropy for the given

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: lt; 9/10*low + 1/10*high Code[low_, high_, x_] := {b, 9/10*low + 1/10*high, 7/10*low + 3/10*high} /; And[x >= 9/10*low + 1/10*high, x < 7/10*low + 3/10*high] Code[low_, high_, x_] := {c, 7/10*low + 3/10*high, high} /; x >= 7/10*low + 3/10*high Clear[CodeF] (* executes decoding using Code *) CodeF[low_, high_, x_, 0] := {} CodeF[low_, high_, x_, n_] := Module[{aux1 = Code[low, high, x]}, Join[{aux1[[1]]}, CodeF[aux1[[2]], aux1[[3]], x, n - 1]] /; n > 0 ¤¢ ¥¡ h hh   ¦ BQh BQQh QBh  &0 Finally, we find that the code that corresponds to is caba : In[57]:= CodeF[0, 1, 0.3076171875, 4] Out[57]= {c, a, b, a} The entropy for the given model is so one would expect an average -bit message to take bits of information. However, our message contains two a’s and one b who all have low probability and therefore a large self-information. Hence, the sum of self-informations is relatively high X f¦b`  GdQt h Wh ¦ 2 ( ' GdQ` G§…' 9 ( hb' ` ¦ X   for our specific message caba. By Theorem 3.3.1. from the handouts, it is the sum of self-informations that determines the length of the code: the length of the code is bounded by To sum up, our code is relatively long (11 bits) because the corresponding message caba contains a relatively high amount of self-information.  dQdW¦ YXWQV¦ ‚ bte hh ¦ ¢‚ § ¥¤£¡9 ¨ ' Problem 4(a) # !   © $" S¦¨ ...
View Full Document

Ask a homework question - tutors are online