Unformatted text preview: ﬁt is 11.54 2.43 = 9.11 dollars per day.
At this point we have computed
2· policy
proﬁt per day 0,3
$9.11 1,3
$9.62 2,3
$9.40 so the 1,3 inventory policy is optimal. 1.6
1.6.1 Special Examples
Doubly stochastic chains Deﬁnition 1.2. A transition matrixP is said to be doubly stochastic if its
p
COLUMNS sum to 1, or in symbols x p(x, y ) = 1.
The adjective “doubly” refers to the fact that by its deﬁnition a transition probP
ability matrix has ROWS that sum to 1, i.e., y p(x, y ) = 1. The stationary
distribution is easy to guess in this case: Theorem 1.24. If p is a doubly stochastic transition probability for a Markov
chain with N states, then the uniform distribution, ⇡ (x) = 1/N for all x, is a
stationary distribution.
Proof. To check this claim we note that if ⇡ (x) = 1/N then
X
1X
1
= ⇡ (y )
⇡ (x)p(x, y ) =
p(x, y ) =
Nx
N
x Looking at the second equality we see that conversely, if ⇡ (x) = 1/N then p is
doubly stochastic.
Example 1.25. Symmetric reﬂecting random walk on the line. The
state space is {0, 1,...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details