EE 376A/Stat 376A
Information Theory
Prof. T. Weissman
Thursday, February 8, 2007
Midterm
1. (
35 points
)
Throwing a die
Suppose you have a die with three sides. The probability of outcome of each side is
given as
X
=
1
,
w.p.
1
/
2
2
,
w.p.
1
/
3
3
,
w.p.
1
/
6
You throw this die independently
n
times, and collect the sequence of numbers (
X
1
, X
2
,
· · ·
, X
n
).
(a) (
15 points
) Suppose you are building an
n
dimensional rectangular box with side
lengths
R
n
= (
X
1
, X
2
,
· · ·
, X
n
). The volume of the box is
V
(
R
n
) =
n
i
=1
X
i
. The
n
dimensional cube with “effective side length” is a cube with the same volume
as the box, namely
V
(
R
n
). When
n
is large, what is the effective side length
for
the rectangular box? More precisely, identify the value of
for which
(
V
(
R
n
))
1
/n
→
in probability as
n
→ ∞
.
(b) (
20 points
) Now, let
P
(
R
n
) be the probability of box
R
n
. Identify the value of
p
for which
(
P
(
R
n
)
1
/n
)
→
p
in probability as
n
→ ∞
. Interpret your result.
(
Hint for both parts:
You may want to evaluate the logarithm of the limit.)
Solution: Throwing a die
(a) Let’s take logarithm and get the limit, first.
1
n
log
V
(
R
n
) =
1
n
log
n
i
=1
X
i
=
1
n
n
i
=1
log
X
i
→
E
(log
X
i
)
in probability
where the convergence follows from the weak law of large numbers, and
X
i
’s being
i.i.d. Since
E
(log
X
i
) =
1
2
log 1 +
1
3
log 2 +
1
6
log 3 =
1
3
+
1
6
log 3
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 Prof.T.Weissman
 Probability distribution, probability density function

Click to edit the document details