lec7 - What is a Markov Chain? Discrete time stochastic...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Tue, 18 Sep 2001 1 What is a Markov Chain? ! Discrete time stochastic process ! Markovian Property: P(X t+1 |X t ,X t–1 ,…,X 0 ) = P(X t+1 |X t ) ! Transition probabilities, p ij ! Stationarity Assumption: p ij = P(X t+1 =j|X t =i) independent of t Tue, 18 Sep 2001 2 Definitions ! Transitions ! Path ! Reachable ! Communicate ! Types of states: ! Absorbing ! Transient ! Recurrent
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Tue, 18 Sep 2001 3 Maintenance Policy ! Transition probabilities 0 0 0 1 3 1/2 1/2 0 0 2 1/8 1/8 3/4 0 1 1/16 1/16 7/8 0 0 3 2 1 0 state Tue, 18 Sep 2001 4 More Definitions ! Ergodic Markov chain: ! Recurrent ! Aperiodic ! Communicate
Background image of page 2
3 Tue, 18 Sep 2001 5 MC’s in Economic Analysis ! Markov processes can be used to evaluate: ! Profitability of business venture. ! Value of certain business. ! Long run expected income. ! Market share analysis. Tue, 18 Sep 2001 6 Example: Willy’s ! Willy’s Rentals rents Big Wheels for recreational use at East Coast Parkway. ! Willy’s estimates that its profit is a function of the weather conditions. ! The weather is classified into 3 states: ! Sunny ! Cloudy ! Rainy.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Tue, 18 Sep 2001 7 Willy’s: Weather Conditions Today’s weather Tomorrows weather 075 020 005 045 040 015 035 ... Sunny Sunny Cloudy
Background image of page 4
Background image of page 5
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 05/05/2008 for the course IE 221 taught by Professor Georgewilson during the Spring '08 term at Lehigh University .

Page1 / 5

lec7 - What is a Markov Chain? Discrete time stochastic...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online