This preview shows pages 1–2. Sign up to view the full content.
Harvard SEAS
ES250 – Information Theory
Homework 4
(Due Date: Nov. 13, 2007)
1. Consider a binary symmetric channel with
Y
i
=
X
i
⊕
Z
i
, where
⊕
is mod 2 addition, and
X
i
, Y
i
∈
{
0
,
1
}
.
Suppose that
{
Z
i
}
has constant marginal probabilities Pr(
Z
i
= 1) =
p
and Pr(
Z
i
= 0) = 1

p
, but
that
Z
1
, Z
2
,
···
, Z
n
are not necessarily independent. Let
C
= 1

H
(
p
). Show that
max
p
(
x
1
,x
2
,
···
,x
n
)
I
(
X
1
, X
2
,
···
, X
n
;
Y
1
, Y
2
,
···
, Y
n
)
≥
nC
Comment on the implications.
2. Consider the channel
Y
=
X
+
Z
(mod 13), where
Z
=
1
,
with probability
1
3
2
,
with probability
1
3
3
,
with probability
1
3
and
X
∈ {
0
,
1
,
···
,
12
}
.
(a) Find the capacity.
(b) What is the maximizing
p
*
(
x
)?
3. Using two channels.
(a) Consider two discrete memoryless channels (
X
1
, p
(
y
1

x
1
)
,
Y
1
) and (
X
2
, p
(
y
2

x
2
)
,
Y
2
) with capac
ities
C
1
and
C
2
respectively.
A new channel (
X
1
× X
2
, p
(
y
1

x
1
)
×
p
(
y
2

x
2
)
,
Y
1
× Y
2
) is formed
in which
x
1
∈ X
1
and
x
2
∈ X
2
, are
simultaneously
sent, resulting in
y
1
, y
2
. Find the capacity of
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.