EE376A: Homeworks #1 Solutions
1. Entropy of Hamming Code.
Consider information bits X1 , X2 , X3 , X4 cfw_0, 1 chosen at random, together with
check bits X5 , X6 , X7 chosen to make the parity of the circles even.
X5
X2
X3
X1
X7
X6
X4
Thus, for example,
EE376A: Homework #1 Solutions
Due on Thursday, January 22, 2015
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Entropy of Hamming Code.
EE376A: Homework #2 Solutions
Due on Thursday, January 29, 2015
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
Some denitions that may be
EE376A: Homeworks #1
Due on Thursday, January 24, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Entropy of Hamming Code.
Consider
EE376A: Homeworks #2 Solutions
1. Entropy and pairwise independence.
Let X, Y, Z be three binary Bernoulli( 1 ) random variables that are pairwise indepen2
dent; that is, I (X ; Y ) = I (X ; Z ) = I (Y ; Z ) = 0.
(a) Under this constraint, what is the min
EE376A: Homeworks #4
Due on Friday, February 15, 2013
You can hand in the homework, after class, before midterm or deposit it, before 5 PM, in
the EE376A drawer of the class le cabinet on the second oor of the Packard Building.
1. Data Processing Inequali
EE376A: Homeworks #6 Solutions
Due on Thursday, February 26, 2015
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Exact number of types.
EE376A: Homework #7 Solutions
Due on Tuesday, March 10, 2015
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Conditionally typical set.
EE376A: Homeworks #4 Solutions
Due on Thursday, February 12, 2015
You can hand in the homework, after class or deposit it, before 5 PM, in the EE376A drawer
of the class le cabinet on the second oor of the Packard Building.
1. Proofs
Consider f (x), g(x)
EE376A: Homeworks #5 Solutions
1. Postprocessing the output.
One is given a communication channel with transition probabilities p(y | x) and channel
capacity C = max I (X ; Y ). A helpful statistician postprocesses the output by forming
p(x)
Y = g (Y ), y
EE376A: Homeworks #8 Solutions
1. One bit quantization of a single Gaussian random variable.
Let X N (0, 2 ) and let the distortion measure be squared error. Here we do not allow
block descriptions. Show that the optimum reproduction points for 1bit quant
EE376A: Homeworks #6 Solutions
1. Cascaded BSCs.
Consider the two discrete memoryless channels (X , p1 (y |x), Y ) and (Y , p2 (z |y ), Z ).
Let p1 (y |x) and p2 (z |y ) be binary symmetric channels with crossover probabilities 1
and 2 respectively.
1 1
0
EE376A: Homeworks #4 Solutions
1. Data Processing Inequality.
Let X, Y, Z be random variables such that X and Z are conditionally independent
given Y which means
p(X = x, Z = z |Y = y ) = p(X = x|Y = y )P (Z = z |Y = y )
for all x, y, z . We denote this c
EE376A: Homeworks #0 Solutions
1. Independence.
(a) Let Z be a bernoulli random variables (r.v.) with parameter 1/2, (Bern(1/2),
i.e. P (Z = 0) = 1/2, P (Z = 1) = 1/2. Let X be a bernoulli random variables
with parameter p and X is independent of Z . Show
EE376A: Homeworks #2
Due on Thursday, January 31, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Entropy and pairwise independence
EE376A: Homeworks #3
Due on Thursday, February 7, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Relative entropy is cost of misco
EE376A: Homeworks #3 Solutions
1. Relative entropy is cost of miscoding.
Let the random variable X have ve possible outcomes cfw_1, 2, 3, 4, 5. Consider two
distributions on this random variable
Symbol p(x) q (x) C1 (x) C2 (x)
1
1/2 1/2
0
0
1/4 1/8
10
100
EE376A: Homeworks #5
Due on Thursday, February 21, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Postprocessing the output.
One i
EE376A: Homeworks #3 Solutions
1. Shannon code.
Consider the following method for generating a code for a random variable X which
takes on m values cfw_1, 2, . . . , m with probabilities p1 , p2 , . . . , pm . Assume that the
probabilities are ordered so
EE376A: Homeworks #6
Due on Thursday, February 28, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Cascaded BSCs.
Consider the two
EE376A: Homeworks #7
Due on Thursday, March 7, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. Typical Average Lemma.
For X p(x) an
EE376A: Homeworks #7 Solutions
1. Typical Average Lemma.
For X p(x) and 0 < < 1, dene the strong typical set by
1
T (X ) = cfw_xn : | N (x|xn ) p(x)| p(x) for all x X .
n
Also, for X, Y p(x, y ) and 0 < < 1, dene the joint strong typical set by
1
T (X, Y
EE376A: Homeworks #8
Due on Thursday, March 14, 2013
You can hand in the homework either after class or deposit it, before 5 PM, in the EE376A
drawer of the class le cabinet on the second oor of the Packard Building.
1. One bit quantization of a single Ga
EE376A: Homeworks #0
Note: HW0 has NO eect on your grade of EE376A. There is no requirement to hand it in.
Those are warm-up exercises in probability. However, we would like to hear your feedbacks.
If you have some cute problems in probability, please sha