This preview shows pages 1–2. Sign up to view the full content.
Midterm Solutions
CA: Mark Peters
February 15, 2005
Problem 1
(a) TRUE
If the class were not closed, then some state would be transient since there would be some
positive probability of leaving the class and never returning from that state. If one state
is transient, then all communicating states would be transient. Hence, the class would be
transient. (See Fact 4 from Lecture 4).
(b) FALSE
For an example, take the random walk on the integers with a positive drift. Let the class be
all the integers  an inﬁnite class. If
P
(
X
n
+1
=
i
+ 1

X
n
=
i
)
>
0
.
5
, then each state is
transient. Thus, the class is transient and closed.
(c) FALSE
Our deﬁnition for reversibility requires that
π
i
P
i,j
=
π
j
P
j,i
. There is no requirement that
π
i
=
π
j
. Thus,
P
i,j
doesn’t necessarily need to equal
P
j,i
. For example, consider a Markov
Chain with the following transition matrix.
P
=
±
1

α
α
β
1

β
¶
where
α
6
=
β
.
We can verify that the invariant distribution is
π
0
=
β
α
+
β
,π
1
=
α
α
+
β
. Now, this chain will
pass the requirement for reversibility but
P
i,j
6
=
P
j,i
.
(d) TRUE
Since the Markov chain is irreducible and has an invariant distribution, it is positive recurrent.
Since the MC is positive recurrent, we know that
E
[
T
i
]
<
∞
for all
i
. Since
π
i
=
1
E
[
T
i
]
, then
we know that
π
i
>
0
for all
i
.
(e) FALSE
In addition to positive recurrence, we require irreducibility for there to be a unique invariant
distribution. Consider the two state chain, where
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 02/03/2011 for the course MS&E 221 taught by Professor Ramesh during the Winter '11 term at Stanford.
 Winter '11
 Ramesh

Click to edit the document details