Math331, Spring 2008
Instructor: David Anderson
1st Markov Chain Homework
1. Suppose there are three white and three black balls in two urns distributed so that
each urn contains three balls. We say the system is in state
i
,
i
= 0
,
1
,
2
,
3, if there are
i
white balls in urn one. At each stage one ball is drawn at random from each urn and
interchanged. Let
X
n
denote the state of the system after the
n
th draw, and compute
the transition matrix for the Markov chain
{
X
n
:
n
≥
0
}
.
Solution
: Consider a system in state 0. Then there are zero white balls in urn 1, and
therefore, three white balls in urn 2. Thus, after interchanging one ball from each urn,
the next state is necessarily 1. Therefore,
p
(0
,
0) = 0
, p
(0
,
1) = 1
, p
(0
,
2) = 0
,
and
p
(0
,
3) = 0
.
Now suppose that the system is in state 1. Then urn one has one white ball and urn
two has 2 white balls. Therefore, letting
X
be one if a white ball is drawn from urn
one and zero otherwise, and
Y
be one if a white ball is drawn from urn two and zero
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Anderson
 Math, Probability, weather, Markov chain, Rain, white balls, Markov Chain Homework

Click to edit the document details