This preview shows pages 1–3. Sign up to view the full content.
IE 336: Operations Research  Stochastic Models
Homework Set #4
Solutions
Spring 2008
1. Let P=
p
q
0
3
q
0
r
+
q
0
0
1
be the transition matrix of a Markov chain
(a) Determine transition diagram:
1
2
3
1
3q
p
q
r+q
(b) If
q > p
2
, is it possible that
p
=
2
3
? Explain
It is impossible that
p
=
2
3
. Let assume that
p
=
2
3
, then
q >
(
2
3
)
2
=
4
9
.
⇒
p
+
q >
2
3
+
4
9
=
10
9
(*)
But P is a transition matrix
⇒
p
+
q
= 1 (**)
Then (*) and (**)
⇒
p
must be diFerent from
2
3
(c) If
r
=
1
5
, determine the values of
p
and
q
P is a transition matrix
⇒
b
p
+
q
= 1
4
q
+
r
= 1
Then
q
=
1
5
and
p
=
4
5
2. Consider the ”eye color” example from class (section 2.21 in the book). Show that transition matrix
P
=
aa
ab
bb
aa
p
1

p
0
ab
p
2
1
2
1

p
2
bb
0
p
1

p
For the frst row o± the transition matrix
P
{
X
1
=
aa

X
0
=
aa
}
=
P
{
X
1
=
aa

X
0
=
aa,father
=
aa
}
P
{
father
=
aa
}
+
P
{
X
1
=
aa

X
0
=
aa,father
=
ab
}
P
{
father
=
ab
}
+
P
{
X
1
=
aa

X
0
=
aa,father
=
bb
}
P
{
father
=
bb
}
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document Since
P
{
X
1
=
aa

X
0
=
aa,father
=
aa
}
= 1,
P
{
X
1
=
aa

X
0
=
aa,father
=
ab
}
=
1
2
,
P
{
X
1
=
aa

X
0
=
aa,father
=
bb
}
= 0,
P
{
father
=
aa
}
=
p
2
,
P
{
father
=
ab
}
= 2
p
(1

p
)
⇒
P
{
X
1
=
aa

X
0
=
aa
}
=
p
Clearly,
P
{
X
1
=
bb

X
0
=
aa
}
= 0
⇒
P
{
X
1
=
aa

X
0
=
aa
}
+
P
{
X
1
=
ab

X
0
=
aa
}
= 1
⇒
P
{
X
1
=
ab

X
0
=
aa
}
= 1

p
For the second row of the transition matrix
P
{
X
1
=
aa

X
0
=
ab
}
=
P
{
X
1
=
aa

X
0
=
ab,father
=
aa
}
P
{
father
=
aa
}
+
P
{
X
1
=
aa

X
0
=
ab,father
=
ab
}
P
{
father
=
ab
}
+
P
{
X
1
=
aa

X
0
=
ab,father
=
bb
}
P
{
father
=
bb
}
Since
P
{
X
1
=
aa

X
0
=
ab,father
=
aa
}
=
1
2
,
P
{
X
1
=
aa

X
0
=
ab,father
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 03/29/2010 for the course IE 383 taught by Professor Leyla,o during the Spring '08 term at Purdue University.
 Spring '08
 Leyla,O
 Operations Research

Click to edit the document details