Introduction to Information Theory (67548)
November 27, 2008
Assignment 0: Solution
Lecturer: Prof. Michael Werman
Due:
Problem 1
Probability and Conditional Probability
Part (a)
Leting
A
be the event that there is a girl in the family, and
B
be the event that there is a boy in the
family, the answer is simply Pr(
A

B
) = Pr(
A
∩
B
)
/
Pr(
B
), which is equal to (1
/
2)
/
(3
/
4) = 2
/
3.
Part (b)
Given that the boy is the first child, the second child is either a girl or a boy with equal probability, so
the answer is 1
/
2. The same applies if the boy is the second child.
Part (c)
The problem is that this statistical experiment is not well defined. For instance, one scenario is that all
families decide that boys, when available, should open the door. In this case, the fact that a boy opens
the door only rules out the possibility of two girls in the family, and the answer is as in Part (a). If one of
the two childs opens the door ar random, we can think of the sample space as Ω = (
{
bb, bg, gb, gg
}×{
1
,
2
}
,
where
{
1
,
2
}
signifies whether the first or the second child opens the door. All 8 outcomes are equally
likely. Letting
A
be the event that there is a girl in the family, and
B
the event that a boy opens the
door, we have that
Pr(
A

B
) =
P
(
A
∩
B
)
P
(
B
)
=
P
(
{
(
bg,
1)
,
(
gb,
2)
}
)
P
(
{
(
bb,
1)
,
(
bb,
2)
,
(
bg,
1)
,
(
gb,
2)
}
)
=
2
/
8
4
/
8
=
1
2
.
Problem 2
Geometric Distribution
Part (a)
By the definition of a geometric distribution, Pr(
X
=
k
) = (1

p
)
k

1
p
. The expectation is equal to
E
[
X
] =
∞
X
k
=1
k
Pr(
X
=
k
) =
∞
X
k
=1
k
(1

p
)
k

1
p
=
p
∞
X
k
=1
k
(1

p
)
k

1
.
Notice that
∑
∞
k
=1
k
(1

p
)
k

1
is simply the derivative, according to
p
, of

∑
∞
k
=1
(1

p
)
k
, which is equal
to

1
/p
+ 1. Therefore, the expectation is equal to
p

1
p
+ 1
0
=
p
p
2
=
1
p
.