ECEN 646 Homework 8 Solutions
1
C
¸inlar Exercise (4.2), Chapter 5
(a)
Since
X
is a Markov chain, the transitions are conditionally inde
pendent of the past given the present. Therefore, the required probability
can be written as
P
{
X
5
=
b,X
6
=
r,X
7
=
b,X
8
=
b

X
4
=
w
}
=
P
{
X
5
=
b

X
4
=
w
}
P
{
X
6
=
r

X
5
=
b
}
P
{
X
7
=
b

X
6
=
r
}
P
{
X
8
=
b

X
7
=
b
}
=
P
(2
,
3)
P
(3
,
1)
P
(1
,
3)
P
(3
,
3)
=
(0
.
6)(0
.
8)(1)(0
.
2)
=
0
.
096
(b)
We can define a new variable
Y
=
f
(
X
) which will once again be
Markov since the function
f
(
.
) is invertible. This new chain will have the
state space
E
′
=
{
2
,
4
,
7
,
3
}
. The transition probability remains unchanged
by this relabelling of states.
E
[
f
(
X
5
)
f
(
X
6
)

X
4
=
y
]
=
E
[
Y
5
Y
6

Y
4
= 3]
=
summationdisplay
i
∈
E
′
summationdisplay
j
∈
E
′
i
∗
jP
(
Y
5
=
i,Y
6
=
j

Y
4
= 3)
=
summationdisplay
i
∈
E
′
summationdisplay
j
∈
E
′
i
∗
jP
(
Y
5
=
i

Y
4
= 3)
P
(
Y
6
=
j

Y
5
=
i
)
=
14
.
41
2
C
¸inlar Exercise (4.4), Chapter 5
(a)
It is clear that if we start from either state 1 or state 3, we remain
forever within these 2 states. Therefore, the set
{
1
,
3
}
is a closed, irreducible
set.
Also, starting at states 2 and 4, we enter the states
{
1
,
3
}
sooner or
later. Therefore the states 2 and 4 are transient. States 1 and 3 are recurrent
positive. Since,
P
(1
,
1)
>
0, state 1 is aperiodic. Since states 1 and 3 belong
to the same irreducible set, state 3 is aperiodic as well.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
•
States 1 and 3  Recurrent positive, aperiodic.
•
States 2 and 4  Transient.
(b)
The set
{
1
,
3
,
4
}
is closed and irreducible. Starting at states 2 or 5
sooner or later lands us in this set. Also
P
(3
,
3)
>
0. These yield
•
States 1,3,4  Recurrent positive, aperiodic
•
States 2,5  Transient.
(c)
The states can be split into 2 closed irreducible sets as
{
1
,
3
,
4
}
and
{
2
,
5
}
. We have
P
(1
,
1)
>
0 and
P
(2
,
2)
>
0. This means that both sets are
aperiodic and recurrent positive. Therefore all the states in this chain are
recurrent positive and aperiodic.
(d)
The sets
{
1
,
3
,
6
}
and
{
2
,
5
}
are closed and irreducible. Starting at states
4 or 7 leads us into one of these 2 sets.
Therefore, 4 and 7 are transient
states. Also,
P
(1
,
1)
>
0 and
P
(5
,
5)
>
0. Combining these observations,
we have
•
States 1,3,6  Recurrent positive, aperiodic
•
States 2,5  Recurrent positive, aperiodic
•
States 4,7  Transient
(e)
The set
{
1
,
3
,
5
}
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '11
 SerapSavari
 Markov chain, Andrey Markov, probability transition matrix, Markovian

Click to edit the document details