Stochastic

# Proof to check this claim we note that if x 1n then x

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ﬁt is 11.54 2.43 = 9.11 dollars per day. At this point we have computed 2· policy proﬁt per day 0,3 \$9.11 1,3 \$9.62 2,3 \$9.40 so the 1,3 inventory policy is optimal. 1.6 1.6.1 Special Examples Doubly stochastic chains Deﬁnition 1.2. A transition matrixP is said to be doubly stochastic if its p COLUMNS sum to 1, or in symbols x p(x, y ) = 1. The adjective “doubly” refers to the fact that by its deﬁnition a transition probP ability matrix has ROWS that sum to 1, i.e., y p(x, y ) = 1. The stationary distribution is easy to guess in this case: Theorem 1.24. If p is a doubly stochastic transition probability for a Markov chain with N states, then the uniform distribution, ⇡ (x) = 1/N for all x, is a stationary distribution. Proof. To check this claim we note that if ⇡ (x) = 1/N then X 1X 1 = ⇡ (y ) ⇡ (x)p(x, y ) = p(x, y ) = Nx N x Looking at the second equality we see that conversely, if ⇡ (x) = 1/N then p is doubly stochastic. Example 1.25. Symmetric reﬂecting random walk on the line. The state space is {0, 1,...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online