hmwk8_11 - ISyE 3232 H. Ayhan Stochastic Manufacturing and...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ISyE 3232 H. Ayhan Stochastic Manufacturing and Service Systems Fall 2011 Homework 8 October 24, 2011 (Due: at the start of class on Monday, October 31) 1. Consider two stocks. Stock 1 always sells for $10 or $20. If stock 1 is selling for $10 today, there is a .80 chance that it will sell for $10 for tomorrow. If it is selling for $20 today, there is a .90 chance that it will sell for $20 tomorrow. Stock 2 always sells for $10 or $25. If stock 2 sells today for $10 there is a .90 chance that it will sell tomorrow for $10. If it sells today for $25, there is a .85 chance that it will sell tomorrow for $25. Let Xn denote the price of the 1st stock and Yn denote the price of the 2nd stock during the nth day. Assume that {Xn : n ≥ 0} and {Yn : n ≥ 0} are discrete time Markov chains. (a) (b) (c) (d) (e) What is the transition matrix for {Xn : n ≥ 0}? Is {Xn : n ≥ 0} irreducible? What is the transition matrix for {Yn : n ≥ 0}? Is {Yn : n ≥ 0} irreducible? What is the stationary distribution of {Xn : n ≥ 0}? What is the stationary distribution of {Yn : n ≥ 0}? On January 1st, your grand parents decide to give you a gift of 300 shares of either Stock 1 or Stock 2. You are to pick one stock. Once you pick the stock you cannot change your mind. To take advantage of a certain tax law, your grand parents dictate that one share of the chosen stock is sold on each trading day. Which stock should you pick to maximize your gift account by the end of the year? (Explain your answer.) 2. A Markov chain is said to be doubly stochastic if both the rows and columns of the transition matrix sum to 1. Assume that the state space is {0, 1, . . . , m}, and that the Markov chain is doubly stochastic and irreducible. Determine the stationary distribution π . (Hint: there are two approaches. One is to ￿ solve π = π P and i∈S π (i) = 1 in general for doubly stochastic matrices. The other is to first solve a few examples, then make an educated guess about π , and finally show that your guess is correct.) 3. Consider the following transition matrix: 0 0 .5 0 0.5 0.6 0 0.4 0 P = 0 0.7 0 0.3 0.8 0 0.2 0 (1) (a) Is the Markov chain periodic? Give the period of each state. (b) Is (π1 , π2 , π3 , π4 ) = (33/96, 27/96, 15/96, 21/96) the stationary distribution of the Markov Chain? 100 101 100 101 (c) Is P11 = π1 ? Is P11 = π1 ? Give an expression for π1 in terms of P11 and P11 . 4. For each of the following transition matrices, do the following: (1) Determine whether the Markov chain is irreducible; (2) Find a stationary distribution π ; is the stationary distribution unique; (3) Determine whether the Markov chain is periodic; (4) Give the period of each state. (5) Without using any software package, find P 100 approximately (a) 0 1/3 2/3 P = 2/3 0 1/3 1/3 2/3 0 (b) .2 .5 P = 0 0 .8 .5 0 0 0 0 0 .5 0 0 1 .5 ISyE 3232 Stochastic Manufacturing and Service Systems Fall 2011 H. Ayhan Solutions to Homework 8 ￿ 0. 2 . It is irreducible. 0. 9 ￿ ￿ 0. 9 0. 1 (b) The state space is {$10, $25}. The transition matrix is . It is irreducible. 0.15 0.85 1. (a) The state space is {$10, $20}. The transition matrix is ￿ 0.8 0.1 X X (c) The stationary distribution is (π$10 , π$20 ) = (1/3, 2/3). Y Y (d) The stationary distribution is (π$10 , π$25 ) = (3/5, 2/5). ￿300 ￿300 (e) What you need look at is E( i=1 Xi ) and E( i=1 Yi ). And choose the one with larger expectation. By the stationary distribution obtained in (b) and (c), we have E( 300 ￿ Xi ) = 300(10 × 1 2 + 20 × ) = 5000, 3 3 300 ￿ Yi ) = 300(10 × 3 2 + 25 × ) = 4800, 5 5 i=1 E( i=1 so you should pick stock 1. 2. First you should solve a few small examples, for instance: ￿ ￿ 0.25 0.75 0.75 0.25 and 0.2 0.3 0.5 0. 4 0. 3 0. 3 0.4 0.4 0.2 In the first case, the stationary distribution is {0.5, 0.5}, and, in the second case, the stationary distribution is {0.3¯, 0.3¯, 0.3¯}. At this point, it is reasonable to suspect that the 3 3 3 stationary distribution for an arbitrary doubly stochastic transition matrix with state space {0, 1, 2, . . . , m} is given by {1/(m + 1), 1/m + 1, . . . , 1/(m + 1)}. Because the Markov chain is irreducible, the stationary distribution is unique. Therefore if we check that the equation π P = π is satisfied with π = {1/(m +1), 1/m + 1, . . . , 1/(m +1)} then π must be the stationary distribution. To verify, we must perform the matrix multiplication. We will consider element i of the vector π P . [π P ]i = = m ￿ j =0 m ￿ i=0 = = 1 πj pji 1 pji m+1 m 1￿ pji m + 1 i=0 1 = πi m+1 Because this equation holds for all i, π = {1/(m + 1), 1/m + 1, . . . , 1/(m + 1)} is the unique stationary distribution. 3. Suppose the state space to be {0, 1, 2, 3, }, one can draw the transition diagram (a) Yes, it is. Each state has period 2. ￿4 (b) Yes, it is. π satisfies the balance equation π P = π and i=1 πi = 1. Since the Markov chain is irreducible, the stationary distribution is unique. 100 101 (c) P11 = 11/16 ￿= π1 , P11 = 0 ￿= π1 . Since the Markov chain has period 2, π1 = 100 101 P11 + P11 . 2 4. (a) Assume the state space is {0, 1, 2}. One can draw the transition diagram as the following: (1) Since any state commutes with each other, the Markov chain is irreducible. (2) The stationary distribution is π = (1/3, 1/3, 1/3), which is unique because the state space is irreducible. (3) The Markov chain is aperiodic. (4) Each state can be visited again either in 2 steps (e.g. 0 → 1 → 0) or 3 steps (e.g. 0 → 2 → 1 → 0). The period is the greatest common factor of 2 and 3, which is 1. 2 n (5) Since the Markov chain is irreducible, aperiodic, and has a finite state space, limn→∞ Pij = πj , where π is the unique stationary distribution obtained in (2). And n = 100 is long enough to ”reach” the steady state, so 1/3 1/3 1/3 P 100 ≈ 1/3 1/3 1/3 . 1/3 1/3 1/3 (b) Assume the state space is {0, 1, 2, 3}. One can draw the transition diagram as the following: (1) It is NOT irreducible since state 0 can only commute with state 1. (2) π = {5/26, 8/26, 1/6, 2/6} is a stationary distribution. But it is not unique since ￿ π = {5/39, 8/39, 2/9, 4/9} is also one. Both solve π = π P and π = 1. (3) Periodicity is only defined for irreducible Markov chains. (4) State 4 can come back to itself in either 2 steps or 3 steps, so the period is 1. Any other state can come back to itself in one step, so its period is 1. (5) The Markov chain can be decomposed into two Markov chains with state space {0, 1} and {2, 3} respectively. And each of them is irreducible and aperiodic, having finite state space, n lim Pij = πj for i, j ∈ {0, 1} or i, j ∈ {2, 3} n→∞ where {π0 , π1 } is the unique stationary distribution for Markov chain with state space {0, 1} and {π2 , π3 } is the unique stationary distribution for Markov chain with state n space {2, 3}. Pij = 0, for all other (i, j ) pairs, of which one is from {0, 1} and the other is from j ∈ {2, 3}. So 5/13 8/13 0 0 5/13 8/13 0 0 . P 100 ≈ 0 0 1/3 2/3 0 0 1/3 2/3 3 ...
View Full Document

Ask a homework question - tutors are online