ORIE 3510 – Homework 4
Instructor: Mark E. Lewis
due 2PM, Wednesday February 22, 2012 (ORIE Hallway drop box)
1. Show that if state
i
is recurrent and state
i
does not communicate with state
j
, then
p
ij
= 0. (This implies that once a process enters a recurrent class of states, it can
never leave that class. For this reason, a recurrent class is often referred to as a
closed
class).
2. Consider a Markov chain with state space
S
=
{
1
,
2
,...,
9
}
and transition matrix given
by
0 1
0
0
0
0
0
0
0
0 0
0
0
0
0
0
0
1
0 0 1
/
2 0
0
0 1
/
2
0
0
0 0 1
/
2 0
0
0 1
/
2
0
0
0 0
0
0
0
0
0
2
/
3 1
/
3
1 0
0
0
0
0
0
0
0
0 0
0
1
0
0
0
0
0
0 0
0
0 1
/
4 0 3
/
4
0
0
0 0
0
0
0
1
0
0
0
a) Classify the states.
b) Which classes are transient? Which classes are recurrent?
c) What are the periods of each state?
3. A transition probability matrix
P
is said to be doubly stochastic if the sum over each
column equals one; that is
X
i
p
ij
= 1 for all
j
If such a chain is irreducible and aperiodic and consists of
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 LEWIS
 Markov chain, pij

Click to edit the document details