HMWK 2
1. 3 black balls and 3 white balls are distributed among two urns (labeled 1,2). At each
“move”, a ball is randomly selected from each urn and the two are swapped (interchanged).
Let
X
n
denote the number of Black balls in Urn 1 after the
n
th
move. Argue that
X
n
forms a Markov chain, and give the transition probability matrix (
P
i,j
).
Solution:
The state space is
S
=
{
0
,
1
,
2
,
3
}
. We assume that each urn contains 3 balls
each.
Given
X
n
=
i
∈ S
, we know that the ﬁrst urn contains
i
black (
B
) balls and 3

i
white (
W
) balls, whereas the second urn contains 3

i B
balls and
i W
balls. We thus
know the complete distribution of the balls among the urns if we know the state of
X
n
.
Randomly selecting one ball from each urn and swapping them, indeed yields a MC: If
X
n
=
i
∈ {
0
,
1
,
2
,
3
}
, then, independent of the past, the ball chosen from the ﬁrst urn
will be
B
wp=
i/
3, and
W
wp= (3

i
)
/
3, whereas (independently) the ball chosen from
the second urn will be
B
wp= (3

i
)
/
3, and
W
wp=
i/
3. This leads to the following
transition probabilities:
P
01
= 1
, P
10
=
1
9
, P
21
=
4
9
, P
32
= 1
, P
11
=
4
9
, P
22
=
4
9
, P
12
=
4
9
, P
23
=
1
9
.
P
=
0
1
0
0
1
9
4
9
4
9
0
0
4
9
4
9
1
9
0
0
1
0
.
For example
P
10
= 1
/
3
×
1
/
3 = 1
/
9 because this results from the events : Chose
B
from
the ﬁrst urn and chose
W
from the second urn.
P
22
=
4
9
because either a
W
was swapped
with a
W
(wp= 2
/
9), or a
B
was swapped with a
B
(wp= 2
/
9), yielding a total (sum) of
4
/
9.
2. Consider modeling the weather where we now assume that the weather today depends
(at most) on the previous three days weather. Letting
W
n
denote weather on the
n
th
day (0 = no rain, 1 = rain), let
X
n
= (
W
n

2
,W
n

1
,W
n
). Argue that
{
X
n
}
forms a
MC. There are 8 states, and we can relabel them 0—7 (as was pointed out in lecture).
(0
,
0
,
0) = 0
,
(1
,
0
,
0) = 1
,
(0
,
1
,
0) = 2
,
(0
,
0
,
1) = 3
,
(1
,
1
,
0) = 4
,
(1
,
0
,
1) = 5
,
(0
,
1
,
1) =
6
,
(1
,
1
,
1) = 7. Assume that if it has rained for the past 3 days, then it will rain today
with probability 0.7; if it did not rain on any of the past three days, then it will rain
today with probability 0.10. In any other case assume that the weather today will with
probability 0.8 be the same as the weather yesterday. Derive the transition matrix.
Solution: