STAT455/855
Fall 2008
Applied Stochastic Processes
Final Exam, Solutions
1. (15 marks)
(a) (9 marks) The classes and classifications are:
(i)
{
0
}
, which is positive recurrent with period 1, and
{
1
,
2
}
, which is positive
recurrent with period 2.
(ii)
{
0
}
, which is positive recurrent with period 1, and
{
1
,
2
}
, which is positive
recurrent with period 1.
(iii)
{
0
}
, which is positive recurrent with period 1, and
{
1
,
2
}
, which is transient
with period 1.
(b) (6 marks) We find lim
n
→∞
P
j
(
n
), for
j
= 0
,
1
,
2, by conditioning on the initial
state:
P
j
(
n
)
=
P
(
X
n
=
j
)
=
1
3
P
(
X
n
=
j
X
0
= 0) +
P
(
X
n
=
j
X
0
= 1) +
P
(
X
n
=
j
X
0
= 2)
.
(i) For
j
= 0,
P
(
X
n
= 0
X
0
= 0) = 1 and
P
(
X
n
= 0
X
0
=
k
) = 0 for
k
= 1 or
k
= 2. Therefore,
P
(
X
n
= 0) = 1
/
3 for all
n
so the limiting value is 1
/
3. For
j
= 1,
P
(
X
n
= 1
X
0
= 0) = 0 for all
n
,
P
(
X
n
= 1
X
0
= 1) = 1 for
n
even
and 0 for
n
odd, and
P
(
X
n
= 1
X
0
= 2) = 0 for
n
even and 1 for
n
odd.
Therefore, for all
n
(either
n
even or
n
odd), we obtain
P
(
X
n
= 1) = 1
/
3,
and so the limiting value is 1
/
3. By subtraction, we obtain
P
(
X
n
= 2) = 1
/
3
for all
n
and in the limit.
(ii) For
j
= 0, we obtain
P
(
X
n
= 0) = 1
/
3 for all
n
as in part(i). For
j
= 1 or
j
= 2, we have
P
(
X
n
=
j
X
0
= 0) = 0 for all
n
as in part(i). If
X
0
=
k
with
k
= 1 or 2 then lim
n
→∞
P
(
X
n
=
j
X
0
=
k
) is the stationary probability of
state
j
for the Markov chain with state space
{
1
,
2
}
and onestep transition
matrix
.
4
.
6
.
7
.
3
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
STAT 455/855  Final Exam Solutions, 2008
Page 2 of 7
Letting
π
1
and
π
2
denote the stationary probabilities of states 1 and 2, re
spectively, we have from the global balance equations for this chain that
π
1
and
π
2
satisfy
π
1
=
.
4
π
1
+
.
7
π
2
and
π
2
=
.
6
π
1
+
.
3
π
2
,
together with the normalization constraint
π
1
+
π
2
= 1. Both balance equa
tions above give
.
6
π
1
=
.
7
π
2
or
π
2
= (6
/
7)
π
1
. Thus,
π
1
satisfies
π
1
(1+6
/
7) =
1, or
π
1
= 7
/
13. This then gives
π
2
= 6
/
13. Thus, we have
lim
n
→∞
P
(
X
n
= 1
X
0
=
k
)
=
7
13
lim
n
→∞
P
(
X
n
= 2
X
0
=
k
)
=
6
13
,
for
k
= 1 or 2. Therefore,
lim
n
→∞
P
1
(
n
)
=
1
3
7
13
+
7
13
=
14
39
,
lim
n
→∞
P
2
(
n
)
=
1
3
6
13
+
6
13
=
12
39
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 GLENTAKAHARA
 Markov chain, Xn, πi αik

Click to edit the document details