STAT455/855
Fall 2009
Applied Stochastic Processes
Assignment #2, Solutions
Total Marks: 55 for 455 and 65 for 855.
1. From Sheet.
(10 marks)
(a) (4 marks)
(i) (4 marks) Let
D
k
and
N
k
denote, respectively, the period and the number of
equivalence classes of the Markov chain whose transition matrix is given by
P
k
. By considering various example values of
d
and
k
one can guess that the
correct values of
D
k
and
N
k
should be
D
k
=
d
gcd(
d,k
)
and
N
k
= gcd(
d,k
)
.
(ii) (8 bonus marks) For
k
= 1, the values
D
1
=
d
and
N
1
= 1 are clearly
correct. So let us assume
k
∈ {
2
,...,d
}
. We use the following notation. Let
X
denote the original Markov chain and let
X
(
k
)
denote the Markov chain
whose transition matrix is
P
k
. We use the term “1step” to mean a single
step in the original Markov chain
X
and the term “
k
step” to mean a single
step in the Markov chain
X
(
k
)
(so a single
k
step consists of
k
1steps of
X
).
Let
D
k
and
N
k
be as given in part(i) and let
C
0
,...,C
d

1
denote the cyclic
classes of
X
. Let
p
ij
(
n
) denote the
n
step transition probability from
i
to
j
in
the original chain and let
p
(
k
)
ij
(
n
) denote the “
n
step” transition probability
from
i
to
j
in the chain with transition matrix
P
k
, so each step here is a
k
step. We divide the proof into 4 steps:
Step 1.
Proof that
p
ij
(
nd
)
>
0
for all
n
big enough for
i
and
j
in the same
cyclic class:
From class we have that if
n
1
,n
2
,...
is an inﬁnite sequence of
positive integers with gcd 1 then there is a ﬁnite subset of this sequence,
say
b
1
,...,b
r
, that also has gcd 1 and a ﬁnite integer
M
such that for every
n > M
, there exist nonnegative integers
d
1
,...,d
r
(which depend on
n
) such
that
n
=
d
1
b
1
+
...
+
d
r
b
r
. Now consider an arbitrary state
j
. Since the period
is
d
each possible return time to
j
is a multiple of
d
. Let
m
1
d,m
2
d,.
..
be the
times at which a return to state
j
has positive probability; i.e.,
p
jj
(
m
s
d
)
>
0