•
States 1 and 3 - Recurrent positive, aperiodic.
•
States 2 and 4 - Transient.
(b)
The set
{
1
,
3
,
4
}
is closed and irreducible. Starting at states 2 or 5
sooner or later lands us in this set. Also
P
(3
,
3)
>
0. These yield
•
States 1,3,4 - Recurrent positive, aperiodic
•
States 2,5 - Transient.
(c)
The states can be split into 2 closed irreducible sets as
{
1
,
3
,
4
}
and
{
2
,
5
}
. We have
P
(1
,
1)
>
0 and
P
(2
,
2)
>
0. This means that both sets are
aperiodic and recurrent positive. Therefore all the states in this chain are
recurrent positive and aperiodic.
(d)
The sets
{
1
,
3
,
6
}
and
{
2
,
5
}
are closed and irreducible. Starting at states
4 or 7 leads us into one of these 2 sets.
Therefore, 4 and 7 are transient
states. Also,
P
(1
,
1)
>
0 and
P
(5
,
5)
>
0. Combining these observations,
we have
•
States 1,3,6 - Recurrent positive, aperiodic
•
States 2,5 - Recurrent positive, aperiodic
•
States 4,7 - Transient
(e)
The set
{
1
,
3
,
5
}
is closed and irreducible as is the set
{
4
,
6
}
. The tran-
sitions in
{
1
,
3
,
5
}
are such that we can come back to the starting state only
in an even number of steps.