STAT455/855
Fall 2009
Applied Stochastic Processes
Final Exam, Solutions
1. (15 marks)
(a) 3 marks)
True
. Take the simple random walk with
p
= 1
/
2.
(b) (3 marks)
True
.
If state
k
were recurrent then the vector
ρ
(
k
) = (
ρ
i
(
k
))
i
∈
S
,
where
ρ
i
(
k
) is the mean number of visits to state
i
during a sojourn from
k
back
to
k
, satisfies
ρ
(
k
) =
ρ
(
k
)
P
.
(c) (3 marks)
False
.
Take
S
to be the integers and for each
i
∈
S
suppose that
p
i,i
+1
>
0,
p
i,i

1
>
0,
p
i,i
+2
>
0, and
p
i,i

2
>
0. Then from state
i
one can make
the transitions
i
→
i
+1
→
i
so that
p
ii
(2)
>
0. One can also make the transitions
i
→
i
+ 2
→
i
+ 1
→
i
so that
p
ii
(3)
>
0. Since the greatest common divisor of 2
and 3 is 1, the period must be 1.
(d) (3 marks)
False
.
Take
S
to be the integers and for each
i
∈
S
suppose that
p
ii
=
r
, where 0
< r <
1,
p
i,i
+1
= (1

r
)
/
2, and
p
i,i

1
= (1

r
)
/
2. If a stationary
distribution existed then this chain is time reversible because for any states
i
and
j
and any path from
i
to
j
, the reverse path will have the same probability. So
the stationary distribution must satisfy the local balance equations, which are
π
i
1

r
2
=
π
i

1
1

r
2
for all
i
. These reduce to
π
i
=
π
i

1
for all
i
. Thus, all components of
π
must be
the same. But this is impossible since
S
is infinite.
(e) (3 marks)
False
.
Take the simple, symmetric random walk (
p
= 1
/
2), or the
counterexample from part(d), where
P
is symmetric but no stationary distribution
exists.
(f) (2 bonus marks)
True
. If
X
were time reversible with stationary distribution
π
and
P
were symmetric then the local balance equations
π
i
p
ij
=
π
j
p
ji
would reduce to
π
i
=
π
j
for all
i, j
∈
S
, since
p
ij
=
p
ji
. That is, all components
of
π
would have to be equal, which is impossible if
S
is infinite.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
STAT 455/855  Final Exam Solutions, 2009
Page 2 of 7
2. (15 marks) We check the local balance equations for all parts.
(a) (5 marks) The local balance equations are
π
i
n
2

1
=
π
j
n
2

1
for all
i, j
such that
j
is equal to
i
except with two components interchanged.
Thus, we get
π
i
=
π
j
for all
i, j
∈
S
. Since
S
has
n
! elements, we get
π
i
=
1
n
!
for all
i
∈
S.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 GLENTAKAHARA
 Markov chain, Πi, πi Pij, local balance equations, vbj πbj

Click to edit the document details