This preview shows pages 1–2. Sign up to view the full content.
ORIE 361/523 – Homework 4 Solutions
Instructor: Mark E. Lewis
February 26, 2008
1. For large
n
we will estimate
P
(
X
n
= 1

X
0
= 1) by the steady state probability of state 1.
Suppose the steady state probability or the stationary distribution is given by
π
= (
π
1
,π
2
,π
3
).
Then we know that
π
satisﬁes
π
=
πP
and
π
1
+
π
2
+
π
3
= 1
.
To solve these equations is same
solving the system
Aπ
0
=
b
where
A
=

1
0
2
/
5
1

3
/
4 3
/
5
1
1
1
and
b
=
0
0
1
Solving this we get
π
= (0
.
1463
,
0
.
4878
,
0
.
3659). So the answer for our problem is 0
.
1463
2. Suppose
X
n
is a Markov chain modeling this and
X
n
= 1 or 2 depending on whether dividend
has been paid or not. We need to solve for the stationary distribution of the stochastic matrix
P
=
±
0
.
9 0
.
1
0
.
6 0
.
4
²
The stationary distribution is given by (6
/
7
,
1
/
7). That means, dividend will be paid every 6
times out of 7 on an average in the long run.
3. Let
X
n
be 1, 2 or 3 depending on whether the
n
th day is sunny, cloudy or rainy respectively.
Then
X
n
is a Markov chain with transition probability matrix
P
=
0
1
/
2 1
/
2
1
/
4 1
/
2 1
/
4
1
/
4 1
/
4 1
/
2
To get the stationary distribution we need to solve for
π
= (
π
1
,π
2
,π
3
) from the equations
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 03/01/2010 for the course ORIE 361 taught by Professor Lewis,m. during the Spring '07 term at Cornell University (Engineering School).
 Spring '07
 LEWIS,M.

Click to edit the document details