ECE255AN
Fall 2014
Homework set 1, due Tue 10/14
1
1. For n 2, let Pn = ( 1 , 1 , . . . , 2n1 , 21 , 21 ). Find the entropy and an optimal code for
n
n
2 4
(a) Pn ,
1
(b) limn Pn = ( 1 , 4 , 1 , ).
2
8
2. Find the entropy of the geometric distribution G(p
ECE 255AN
Fall 2014
Homework set 7 - solutions
Solutions to Chapter 8 problems
3. Uniformly distributed noise. The probability density function for Y = X+Z is the convolution
of the densities of X and Z. Since both X and Z have rectangular densities, the
ECE 255AN
Fall 2014
Homework set 8 - solutions
Solutions to Chapter 9 problems
6. Parallel channels and waterlling. By the result of Section 10.4, it follows that we will put
all the signal power into the channel with less noise until the total power of n
ECE 255AN
Fall 2014
Homework set 6 - solutions
Solutions to Chapter 7 problems
3. Channels with memory have a higher capacity.
Yi = Xi Zi ,
where
Zi =
1 with probability p
0 with probability 1 p
and Zi are not independent.
I(X n ; Y1 , Y2 , . . . , Yn )
=
ECE 255AN
Fall 2014
Homework set 2 - solutions
Solutions to Chapter 3 problem
1. Markovs inequality and Chebyshevs inequality.
(a) If X has distribution F (x),
EX =
xdF
0
=
xdF +
0
xdF
xdF
dF
= Prcfw_X .
Rearranging sides and dividing by we get,
Prcfw_X
ECE 255AN
Fall 2014
Homework set 4 - solutions
Solutions to Chapter 7 problems
2. A sum channel.
Y =X +Z
X cfw_0, 1,
Z cfw_0, a
We have to distinguish various cases depending on the values of a.
a = 0 In this case, Y = X, and max I(X; Y ) = max H(X) = 1.
ECE 255AN
Fall 2014
Homework set 3 - solutions
Solutions to Chapter 2 problem
25. Venn Diagrams. To show the rst identity,
I(X; Y ; Z) = I(X; Y ) I(X; Y |Z)
by denition
= I(X; Y ) (I(X; Y, Z) I(X; Z)
by chain rule
= I(X; Y ) + I(X; Z) I(X; Y, Z)
= I(X; Y