Winter 2009
PSTAT 160A
Homework # 6 solutions
Problem 1.
Pb 14
1)
{
1
,
2
,
3
}
recurrent.
d
= 1
2)
{
1
,
2
,
3
,
4
}
recurrent.
d
= 1 since can do 1431 and 143241: 3 and 5 steps. greatest divisor
is 1.
3)
{
2
}
transient,
{
1
,
3
}
recurrent,
{
4
,
5
}
recurrent . all classes
d
= 1 since have positive
P
i
i
.
4)
{
4
}
transient,
{
5
}
transient,
{
1
,
2
}
recurrent.
d
= 1
5) with modified
P
1
, the states are still all recurrent, the period is still
d
= 1 since it can do 121,
1321 ie 2 or 3 steps.
Problem 2.
1)
P
5
has all entries positive. This means at the 5
th
step, the chain can be in any state, starting
from any step. This means all state communicates with each others.
2) (
P
5
)
31
=
.
2808
3) Need to make 1 absorbing
→
chose
Q
as defined in the problem set. (
Q
5
)
31
=
.
8319
4) Add an extra absorbing state that can be reached only through 1.
→
chose
Q
a
as defined in the
problem set. (
Q
5
a
)
31
=
.
0720
5) Just 1P(ever visited 1 in first 5 step)= 1

.
8319 =
.
1681. This also can be found by arguing
that the chain with the extra state is in either 2 or 3, so 0 +
.
1681
6) Oops, typo, same as 5)!
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '09
 Matrices, Markov chain, Reach, required probability, State highways in Indiana, steps. greatest divisor

Click to edit the document details