Information Theory I
Prof. Dr. A. Lapidoth
Signal and Information
Processing Laboratory
Institut für Signal und
Informationsverarbeitung
Model Answers to Exercise 10 of November 21, 2012
http://www.isi.ee.ethz.ch/teaching/courses/it1
Problem 1
Properties of
R
(
D
)
We have the two rate distortion functions
R
(
D
) =
min
P
ˆ
X

X
:
E
[
d
(
x,
ˆ
x
)]
≤
D
I
(
X
;
ˆ
X
)
,
and
R
′
(
D
) =
min
P
ˆ
X

X
:
E
[
d
′
(
x,
ˆ
x
)]
≤
D
I
(
X
;
ˆ
X
)
,
where the distortion measures
d
(
·
,
·
) and
d
′
(
·
,
·
) relate through
d
′
(
i,j
) =
d
(
)
−
w
i
. We will show
that
R
′
(
D
) =
R
(
D
+ ¯
w
) by expressing
E
b
d
′
(
X,
ˆ
X
)
B
in terms of
E
b
d
(
X,
ˆ
X
)
B
:
E
b
d
′
(
X,
ˆ
X
)
B
=
m
s
i
=1
m
s
j
=1
p
(
) (
d
(
)
−
w
i
)
=
m
s
i
=1
m
s
j
=1
p
(
)
d
(
)
−
m
s
i
=1
m
s
j
=1
p
(
)
w
i
=
E
b
d
(
X,
ˆ
X
)
B
−
m
s
i
=1
p
i
w
i
m
s
j
=1
p
(
j

i
)
±
²³
´
=1
=
E
b
d
(
X,
ˆ
X
)
B
−
¯
w.
Hence, we have
R
′
(
D
) =
min
P
ˆ
X

X
:
E
[
d
′
(
X,
ˆ
X
)
]
≤
D
I
(
X
;
ˆ
X
)
=
min
P
ˆ
X

X
:
E
[
d
(
X,
ˆ
X
)
]
−
¯
w
≤
D
I
(
X
;
ˆ
X
)
=
min
P
ˆ
X

X
:
E
[
d
(
X,
ˆ
X
)
]
≤
D
+ ¯
w
I
(
X
;
ˆ
X
)
=
R
(
D
+ ¯
w
)
.
To conclude that there is no loss in generality in assuming that min
j
d
(
) = 0 for all
i
∈
{
1
,
2
,... ,m
}
, it remains to note that for every distortion measure
d
(
·
,
·
), there exists a corre
sponding distortion measure
d
′
(
·
,
·
) with min
j
d
′
(
) = 0 for every
i
∈ {
1
,
2
}
(by choosing
w
i
= min
j
d
(
)). And as we have shown, the rate distortion function
R
(
·
) then follows directly
from
R
′
(
·
) by
R
(
D
) =
R
′
(
D
−
¯
w
)
.
c
c
Amos Lapidoth, 2012
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentProblem 2
Erasure Distortion
Note frst that For rate
R
= 0 the distortion
D
= 1 is achievable through the choice
P
ˆ
X

X
(“?”

x
) = 1.
±urthermore,
d
(1
,
0) =
∞
and
d
(0
,
1) =
∞
imply that the rate distortion Function is certainly only
achieved by conditional laws satisFying Pr
b
ˆ
X
= 0
v
v
v
X
= 1
B
= 0 and Pr
b
ˆ
X
= 1
v
v
v
X
= 0
B
= 0, since
otherwise the expected distortion would be infnite. It Follows that
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '11
 AmosLapidoth
 Information Theory, Prof. Dr. A. Lapidoth, PX X

Click to edit the document details