Stat 333  Tutorial 8 Answers/Solutions
1.
(a) 0.6665
(b) 0.6667
2.
(a) Single Closed Class:
{
0
,
1
,
2
}
. All states are positive recurrent since we have a ﬁnite
state space.
(b) Single Closed Class:
{
0
,
1
,
2
,
3
}
. . One can note also that the states are periodic with
d
= 3
.
(c) Classes:
{
0
,
2
}
(closed) and
{
3
,
4
}
(closed) and
{
1
}
(open). States 0, 2, 3, 4 are positive
recurrent since we have a ﬁnite state spaces. State 1 is transient since it is an open
class.
(d) Classes:
{
0
,
1
}
(closed) and
{
2
}
(closed),
{
3
}
(open) and
{
4
}
(open). States 0, 1 and 2
are positive recurrent since we have a ﬁnite state spaces. States 3 and 4 is transient
since they belong to open classes.
3. Deﬁne
0 :=
TT
,
1 :=
HH
,
2 :=
HT
and
3 :=
TH
.
(a)
i.
P
=
1p
0
0
0
0
p 1p
0
1p
0
0
0
0
p 1p
0
ii. Aperiodic, Positive Recurrent (and therefore Ergodic), and Irreducible.
iii. As mentioned in class, you need to ﬁnd
~π
such that
~π
·
P
=
~π
with
∑
3
j
=0
π
j
= 1
.
You can do this either the regular way or by means of row reduction on
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Chisholm
 Probability, Probability theory, Markov chain, finite state spaces, nth trial, Single Closed Class

Click to edit the document details