E3106, Solutions to Homework 2
Columbia University
Exercise 13
.
Proof
For
∀
n>r
and
∀
i,j
, according to Chapman-Kolmogorov
equations, we have
P
n
ij
=
X
k
P
n
−
r
ik
P
r
kj
.
Since
X
k
P
n
−
r
ik
=1
,
it follows that there exists some state
k
0
such that
P
n
−
r
ik
0
>
0. And because
P
r
k
0
j
>
0, it follows that
P
n
ij
≥
P
n
−
r
ik
0
P
r
k
0
j
>
0
,
which completes the proof.
Exercise 14
. (1) The classes of the states of the Markov chain with transition
probability
P
1
is
{
0
,
1
,
2
}
. Because it is a
f
nite-state Markov chain, all the
states are recurrent.
(2)The classes of the states of the Markov chain with transition probability
P
2
is
{
0
,
1
,
2
,
3
}
. And because it is a
f
nite-state Markov chain, all the states
are recurrent.
(3)The classes of the states of the Markov chain with transition probability
P
3
is
{
0
,
2
}
,
{
1
}
,
{
3
,
4
}
. It can be easily seen from a graph that
{
0
,
2
}
and
{
3
,
4
}
are recurrent, while
{
1
}
is transient.
Here we also give a rigorous proof of this claim.
(3.1) States 0,2 are recurrent. Firstly, we prove by induction that for all
n
,
P
n
00
=
1
2
and
P
n
02
=
1
2
. Obviously the result is true for
n
=1
. Thensuppose
P
n
−
1
00
=