Assignment 1A Solution

# 3076171875 4 out57 c a b a the entropy for the given

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: lt; 9/10*low + 1/10*high Code[low_, high_, x_] := {b, 9/10*low + 1/10*high, 7/10*low + 3/10*high} /; And[x &gt;= 9/10*low + 1/10*high, x &lt; 7/10*low + 3/10*high] Code[low_, high_, x_] := {c, 7/10*low + 3/10*high, high} /; x &gt;= 7/10*low + 3/10*high Clear[CodeF] (* executes decoding using Code *) CodeF[low_, high_, x_, 0] := {} CodeF[low_, high_, x_, n_] := Module[{aux1 = Code[low, high, x]}, Join[{aux1[[1]]}, CodeF[aux1[[2]], aux1[[3]], x, n - 1]] /; n &gt; 0 ¤¢ ¥¡ h hh   ¦ BQh BQQh QBh  &amp;0 Finally, we ﬁnd that the code that corresponds to is caba : In[57]:= CodeF[0, 1, 0.3076171875, 4] Out[57]= {c, a, b, a} The entropy for the given model is so one would expect an average -bit message to take bits of information. However, our message contains two a’s and one b who all have low probability and therefore a large self-information. Hence, the sum of self-informations is relatively high X f¦b`  GdQt h Wh ¦ 2 ( ' GdQ` G§' 9 ( hb' ` ¦ X   for our speciﬁc message caba. By Theorem 3.3.1. from the handouts, it is the sum of self-informations that determines the length of the code: the length of the code is bounded by To sum up, our code is relatively long (11 bits) because the corresponding message caba contains a relatively high amount of self-information.  dQdW¦ YXWQV¦  bte hh ¦ ¢ § ¥¤£¡9 ¨ ' Problem 4(a) # !   © \$&quot; S¦¨ ...
View Full Document

Ask a homework question - tutors are online