STA 447/2006S, Winter 2008: InClass Test
(DRAFT) SOLUTIONS
1.
[8 points] Let (
p
ij
) be the transition probabilities for random walk on the graph whose
vertices are
V
=
{
1
,
2
,
3
,
4
}
, with a single edge between each of the four pairs (1,2), (2,3),
(3,1), and (3,4), and no other edges. Compute (with full explanation) lim
n
→∞
p
(
n
)
13
.
Solution:
Since
1
↔
2
↔
3
↔
4
, the graph is connected, so the random walk
is irreducible. Also, state 3 is aperiodic since e.g. it is possible to get from 3 to 3
in two steps (
3
→
4
→
3
) or in three steps (
3
→
1
→
2
→
3
), and gcd
(2
,
3) = 1
.
So, by irreducibility, all states are aperiodic. Also, from class, the chain has
stationary distribution given by
π
u
=
d
(
u
)
/
∑
v
d
(
v
) =
d
(
u
)
/
2

E

. In this case,
d
(1) =
d
(2) = 2
,
d
(3) = 3
, and
d
(4) = 1
, so
∑
v
d
(
v
) = 2 + 2 + 3 + 1 = 8
(or
equivalently,
2

E

= 2
·
4 = 8
). Hence, by the Markov chain convergence theorem,
lim
n
→∞
p
(
n
)
13
=
π
3
=
d
(3)
/
8 = 3
/
8
.
2.
Consider the Markov chain with state space
S
=
{
1
,
2
,
3
}
, and transition probabilities
given by
p
11
= 1
/
6,
p
12
= 1
/
3,
p
13
= 1
/
2,
p
22
=
p
33
= 1, and
p
ij
= 0 otherwise.
(a)
[4 points] Compute (with explanation)
f
12
(i.e., the probability, starting from 1, that
the chain will eventually visit 2).
Solution:
When the chain leaves the state 1, it goes to either 2 or 3 and then
stays there. So,
f
12
=
P
(
X
1
= 2

X
0
= 1
, X
1
6
= 1) =
p
12
/
(1

p
11
) = 1
/
3
/
(1

1
/
6) =
(1
/
3)
/
(5
/
6) = 2
/
5
.
Alternatively, since
f
32
= 0
(because
p
33
= 1
), we have that
f
12
=
p
12
+
∑
j
6
=2
p
1
j
f
j
2
=
p
12
+
p
11
f
12
+
p
13
f
32
= (1
/
3) + (1
/
6)
f
12
+ (1
/
2)(0)
, so
(5
/
6)
f
12
= 1
/
3
, so
f
12
= 2
/
5
.
(b)