ORIE 361/523 – Homework 4 Solutions
Instructor: Mark E. Lewis
February 26, 2008
1. For large
n
we will estimate
P
(
X
n
= 1

X
0
= 1) by the steady state probability of state 1.
Suppose the steady state probability or the stationary distribution is given by
π
= (
π
1
, π
2
, π
3
).
Then we know that
π
satisfies
π
=
πP
and
π
1
+
π
2
+
π
3
= 1
.
To solve these equations is same
solving the system
Aπ
0
=
b
where
A
=

1
0
2
/
5
1

3
/
4
3
/
5
1
1
1
and
b
=
0
0
1
Solving this we get
π
= (0
.
1463
,
0
.
4878
,
0
.
3659). So the answer for our problem is 0
.
1463
2. Suppose
X
n
is a Markov chain modeling this and
X
n
= 1 or 2 depending on whether dividend
has been paid or not. We need to solve for the stationary distribution of the stochastic matrix
P
=
0
.
9
0
.
1
0
.
6
0
.
4
The stationary distribution is given by (6
/
7
,
1
/
7). That means, dividend will be paid every 6
times out of 7 on an average in the long run.
3. Let
X
n
be 1, 2 or 3 depending on whether the
n
th day is sunny, cloudy or rainy respectively.
Then
X
n
is a Markov chain with transition probability matrix
P
=
0
1
/
2
1
/
2
1
/
4
1
/
2
1
/
4
1
/
4
1
/
4
1
/
2
To get the stationary distribution we need to solve for
π
= (
π
1
, π
2
, π
3
) from the equations
π
=
πP
and
π
1
+
π
2
+
π
3
= 1
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 LEWIS,M.
 yn, Markov chain, stationary distribution

Click to edit the document details