2. Problem 3.5 (a) Since X1 , . . . , Xn are i.i.d, so are q (X1 ), . . . , q (Xn ). Then, applying the strong law of large numers, lim 1 1 log q (X1 , . . . , Xn ) = lim n n
log q (Xi )
p(x) q (x) = D(p|q ) + H (p) = p(x) log (b) Again, by the st
5.36Solution. (a) l = (1, 2, 2) can be the word lengths of a binary Huffman code, but
l = (2, 2,3,3) (assume that l1 = 2, l2 = 2, l3 = 3, l4 = 3 ) cannot, because by the
Huffman code construction procedure, we should always delete the secondbitof l
-ith raw error pFobebilttyp Is eguhmlent to a single BX wlth e mr probabih ,
- Zp)m) and hence &at
731Ws and c h a d . W rkMit e* re k on
a E mmlllIol) pr
VI,VZ1 for transmiss .
(a) The exponential density, f (x) = h elh, x 1 0.
(b) The Laplace density,
8.1 Differential entropy. Evaluate the differential entropy h(X) = - flnf for t he following:
f(x) = :~e-"l"'
( c) The sum of X , and X , where X I an
EECE 580B Modern Coding Theory WWW: http:/dde.binghamton.edu/filler/mct/hw/1
Homework assignment #1 Due date: September 15, 2:50pm
Gaussian elimination in binary arithmetic
Problem description: You are given a binary matrix of size ( rows and columns) and
EECE 580B Modern Coding Theory WWW: http:/dde.binghamton.edu/filler/mct/hw/8
Homework assignment #8 Due date: December 3, 2:50pm
Belief propagation algorithm over the BSC channel
In this assignment, you are asked to implement the Belief propagation algori