{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# lec7 - What is a Markov Chain Discrete time stochastic...

This preview shows pages 1–4. Sign up to view the full content.

1 Tue, 18 Sep 2001 1 What is a Markov Chain? ! Discrete time stochastic process ! Markovian Property: P(X t+1 |X t ,X t–1 ,…,X 0 ) = P(X t+1 |X t ) ! Transition probabilities, p ij ! Stationarity Assumption: p ij = P(X t+1 =j|X t =i) independent of t Tue, 18 Sep 2001 2 Definitions ! Transitions ! Path ! Reachable ! Communicate ! Types of states: ! Absorbing ! Transient ! Recurrent

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Tue, 18 Sep 2001 3 Maintenance Policy ! Transition probabilities 0 0 0 1 3 1/2 1/2 0 0 2 1/8 1/8 3/4 0 1 1/16 1/16 7/8 0 0 3 2 1 0 state Tue, 18 Sep 2001 4 More Definitions ! Ergodic Markov chain: ! Recurrent ! Aperiodic ! Communicate
3 Tue, 18 Sep 2001 5 MC’s in Economic Analysis ! Markov processes can be used to evaluate: ! Profitability of business venture. ! Value of certain business. ! Long run expected income. ! Market share analysis. Tue, 18 Sep 2001 6 Example: Willy’s ! Willy’s Rentals rents Big Wheels for recreational use at East Coast Parkway. ! Willy’s estimates that its profit is a function of the weather conditions.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 5

lec7 - What is a Markov Chain Discrete time stochastic...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online