STA 447/2006S, Winter 2008: InClass Test
(DRAFT) SOLUTIONS
1.
[8 points] Let (
p
ij
) be the transition probabilities for random walk on the graph whose
vertices are
V
=
{
1
,
2
,
3
,
4
}
, with a single edge between each of the four pairs (1,2), (2,3),
(3,1), and (3,4), and no other edges. Compute (with full explanation) lim
n
→∞
p
(
n
)
13
.
Solution:
Since
1
↔
2
↔
3
↔
4
, the graph is connected, so the random walk
is irreducible. Also, state 3 is aperiodic since e.g. it is possible to get from 3 to 3
in two steps (
3
→
4
→
3
) or in three steps (
3
→
1
→
2
→
3
), and gcd
(2
,
3) = 1
.
So, by irreducibility, all states are aperiodic.
Also, from class, the chain has
stationary distribution given by
π
u
=
d
(
u
)
/
∑
v
d
(
v
) =
d
(
u
)
/
2

E

. In this case,
d
(1) =
d
(2) = 2
,
d
(3) = 3
, and
d
(4) = 1
, so
∑
v
d
(
v
) = 2 + 2 + 3 + 1 = 8
(or
equivalently,
2

E

= 2
·
4 = 8
). Hence, by the Markov chain convergence theorem,
lim
n
→∞
p
(
n
)
13
=
π
3
=
d
(3)
/
8 = 3
/
8
.
2.
Consider the Markov chain with state space
S
=
{
1
,
2
,
3
}
, and transition probabilities
given by
p
11
= 1
/
6,
p
12
= 1
/
3,
p
13
= 1
/
2,
p
22
=
p
33
= 1, and
p
ij
= 0 otherwise.
(a)
[4 points] Compute (with explanation)
f
12
(i.e., the probability, starting from 1, that
the chain will eventually visit 2).
Solution:
When the chain leaves the state 1, it goes to either 2 or 3 and then
stays there. So,
f
12
=
P
(
X
1
= 2

X
0
= 1
, X
1
= 1) =
p
12
/
(1

p
11
) = 1
/
3
/
(1

1
/
6) =
(1
/
3)
/
(5
/
6) = 2
/
5
.
Alternatively, since
f
32
= 0
(because
p
33
= 1
), we have that
f
12
=
p
12
+
∑
j
=2
p
1
j
f
j
2
=
p
12
+
p
11
f
12
+
p
13
f
32
= (1
/
3) + (1
/
6)
f
12
+ (1
/
2)(0)
, so
(5
/
6)
f
12
= 1
/
3
, so
f
12
= 2
/
5
.
(b)
[3 points] Prove that
p
(
n
)
12
≥
1
/
3, for any positive integer
n
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '09
 Markov chain, 0 j, Xn, pij, j Pij, 0 j

Click to edit the document details