Unformatted text preview: ISyE 3232
H. Ayhan Stochastic Manufacturing and Service Systems Fall 2011 Homework 8
October 24, 2011
(Due: at the start of class on Monday, October 31)
1. Consider two stocks. Stock 1 always sells for $10 or $20. If stock 1 is selling for $10 today, there is a
.80 chance that it will sell for $10 for tomorrow. If it is selling for $20 today, there is a .90 chance that
it will sell for $20 tomorrow.
Stock 2 always sells for $10 or $25. If stock 2 sells today for $10 there is a .90 chance that it will sell
tomorrow for $10. If it sells today for $25, there is a .85 chance that it will sell tomorrow for $25.
Let Xn denote the price of the 1st stock and Yn denote the price of the 2nd stock during the nth day.
Assume that {Xn : n ≥ 0} and {Yn : n ≥ 0} are discrete time Markov chains.
(a)
(b)
(c)
(d)
(e) What is the transition matrix for {Xn : n ≥ 0}? Is {Xn : n ≥ 0} irreducible?
What is the transition matrix for {Yn : n ≥ 0}? Is {Yn : n ≥ 0} irreducible?
What is the stationary distribution of {Xn : n ≥ 0}?
What is the stationary distribution of {Yn : n ≥ 0}?
On January 1st, your grand parents decide to give you a gift of 300 shares of either Stock 1 or
Stock 2. You are to pick one stock. Once you pick the stock you cannot change your mind. To
take advantage of a certain tax law, your grand parents dictate that one share of the chosen stock
is sold on each trading day. Which stock should you pick to maximize your gift account by the
end of the year? (Explain your answer.) 2. A Markov chain is said to be doubly stochastic if both the rows and columns of the transition matrix
sum to 1. Assume that the state space is {0, 1, . . . , m}, and that the Markov chain is doubly stochastic
and irreducible. Determine the stationary distribution π . (Hint: there are two approaches. One is to
solve π = π P and i∈S π (i) = 1 in general for doubly stochastic matrices. The other is to ﬁrst solve
a few examples, then make an educated guess about π , and ﬁnally show that your guess is correct.)
3. Consider the following transition matrix: 0 0 .5 0 0.5
0.6 0 0.4 0 P = 0 0.7 0 0.3
0.8 0 0.2 0 (1) (a) Is the Markov chain periodic? Give the period of each state.
(b) Is (π1 , π2 , π3 , π4 ) = (33/96, 27/96, 15/96, 21/96) the stationary distribution of the Markov Chain?
100
101
100
101
(c) Is P11 = π1 ? Is P11 = π1 ? Give an expression for π1 in terms of P11 and P11 .
4. For each of the following transition matrices, do the following: (1) Determine whether the Markov
chain is irreducible; (2) Find a stationary distribution π ; is the stationary distribution unique; (3)
Determine whether the Markov chain is periodic; (4) Give the period of each state. (5) Without using
any software package, ﬁnd P 100 approximately
(a) 0 1/3 2/3
P = 2/3 0 1/3
1/3 2/3 0
(b) .2
.5
P =
0
0 .8
.5
0
0 0
0
0
.5 0
0 1
.5 ISyE 3232 Stochastic Manufacturing and Service Systems Fall 2011 H. Ayhan
Solutions to Homework 8
0. 2
. It is irreducible.
0. 9
0. 9
0. 1
(b) The state space is {$10, $25}. The transition matrix is
. It is irreducible.
0.15 0.85 1. (a) The state space is {$10, $20}. The transition matrix is 0.8
0.1 X
X
(c) The stationary distribution is (π$10 , π$20 ) = (1/3, 2/3). Y
Y
(d) The stationary distribution is (π$10 , π$25 ) = (3/5, 2/5).
300
300
(e) What you need look at is E( i=1 Xi ) and E( i=1 Yi ). And choose the one with larger
expectation. By the stationary distribution obtained in (b) and (c), we have E( 300
Xi ) = 300(10 × 1
2
+ 20 × ) = 5000,
3
3 300
Yi ) = 300(10 × 3
2
+ 25 × ) = 4800,
5
5 i=1 E( i=1 so you should pick stock 1. 2. First you should solve a few small examples, for instance:
0.25 0.75
0.75 0.25
and 0.2
0.3
0.5 0. 4
0. 3
0. 3 0.4
0.4
0.2 In the ﬁrst case, the stationary distribution is {0.5, 0.5}, and, in the second case, the stationary distribution is {0.3¯, 0.3¯, 0.3¯}. At this point, it is reasonable to suspect that the
3
3
3
stationary distribution for an arbitrary doubly stochastic transition matrix with state space
{0, 1, 2, . . . , m} is given by {1/(m + 1), 1/m + 1, . . . , 1/(m + 1)}. Because the Markov chain
is irreducible, the stationary distribution is unique. Therefore if we check that the equation
π P = π is satisﬁed with π = {1/(m +1), 1/m + 1, . . . , 1/(m +1)} then π must be the stationary
distribution. To verify, we must perform the matrix multiplication. We will consider element
i of the vector π P . [π P ]i =
= m
j =0
m
i=0 =
= 1 πj pji
1
pji
m+1
m 1
pji
m + 1 i=0
1
= πi
m+1 Because this equation holds for all i, π = {1/(m + 1), 1/m + 1, . . . , 1/(m + 1)} is the unique
stationary distribution.
3. Suppose the state space to be {0, 1, 2, 3, }, one can draw the transition diagram (a) Yes, it is. Each state has period 2. 4
(b) Yes, it is. π satisﬁes the balance equation π P = π and i=1 πi = 1. Since the Markov
chain is irreducible, the stationary distribution is unique.
100
101
(c) P11 = 11/16 = π1 , P11 = 0 = π1 . Since the Markov chain has period 2, π1 = 100
101
P11 + P11
.
2 4. (a) Assume the state space is {0, 1, 2}. One can draw the transition diagram as the following: (1) Since any state commutes with each other, the Markov chain is irreducible.
(2) The stationary distribution is π = (1/3, 1/3, 1/3), which is unique because the state
space is irreducible.
(3) The Markov chain is aperiodic.
(4) Each state can be visited again either in 2 steps (e.g. 0 → 1 → 0) or 3 steps (e.g.
0 → 2 → 1 → 0). The period is the greatest common factor of 2 and 3, which is 1. 2 n
(5) Since the Markov chain is irreducible, aperiodic, and has a ﬁnite state space, limn→∞ Pij =
πj , where π is the unique stationary distribution obtained in (2). And n = 100 is
long enough to ”reach” the steady state, so 1/3 1/3 1/3
P 100 ≈ 1/3 1/3 1/3 .
1/3 1/3 1/3 (b) Assume the state space is {0, 1, 2, 3}. One can draw the transition diagram as the following: (1) It is NOT irreducible since state 0 can only commute with state 1.
(2) π = {5/26, 8/26, 1/6, 2/6} is a stationary distribution. But it is not unique since
π = {5/39, 8/39, 2/9, 4/9} is also one. Both solve π = π P and
π = 1.
(3) Periodicity is only deﬁned for irreducible Markov chains.
(4) State 4 can come back to itself in either 2 steps or 3 steps, so the period is 1. Any
other state can come back to itself in one step, so its period is 1.
(5) The Markov chain can be decomposed into two Markov chains with state space {0, 1}
and {2, 3} respectively. And each of them is irreducible and aperiodic, having ﬁnite
state space,
n
lim Pij = πj for i, j ∈ {0, 1} or i, j ∈ {2, 3}
n→∞ where {π0 , π1 } is the unique stationary distribution for Markov chain with state space
{0, 1} and {π2 , π3 } is the unique stationary distribution for Markov chain with state
n
space {2, 3}. Pij = 0, for all other (i, j ) pairs, of which one is from {0, 1} and the
other is from j ∈ {2, 3}. So 5/13 8/13 0
0
5/13 8/13 0
0
.
P 100 ≈ 0
0
1/3 2/3
0
0
1/3 2/3 3 ...
View
Full
Document
This note was uploaded on 03/08/2012 for the course ISYE 3232 taught by Professor Billings during the Fall '07 term at Georgia Tech.
 Fall '07
 Billings

Click to edit the document details