STAT 150 FINAL EXAM, Spring 2006, JWP
Lastname,Firstname and SID#:
1. Suppose a Markov chain (
X
n
,n
= 0
,
1
,...
) has transition matrix
P
. Let
i,j,k
be states
of the chain, and
N
a positive integer. Derive expressions involving summations and
powers of
P
for the following quantities:
a) Given
X
0
=
i
, the expected number of times
n
with 1
≤
n
≤
N
and
X
n
=
j
.
b) Given
X
0
=
i
, the expected number of times
n
with 1
≤
n
≤
N
and
X
n
=
j
and
X
n
+1
=
k
.
c) Given
X
0
=
i
, the probability that
T
i
>
2, where
T
i
is the least
n
≥
1 such that
X
n
=
i
, and
T
i
=
∞
if there is no such
n
.
2. Suppose that (
X
1
(
t
)
,t
≥
0) and (
X
2
(
t
)
,t
≥
0) are two independent Markov chains
with continuous time parameter, each with state space
{
0
,
1
}
, with transition rates
λ
from 0 to 1 and
μ
from 1 to 0, for
i
= 1
,
2. Let
S
2
(
t
) :=
X
1
(
t
) +
X
2
(
t
).
a) Explain why (
S
2
(
t
)
,t
≥
0) is a Markov chain, and describe its transition rate
matrix.
b) What is the limit distribution of
S
2
(
t
) as
t
→ ∞
?
c) Generalize part b) to