IOE+316+Notes+1

# IOE+316+Notes+1 - Course IOE 316 Winter 2010 Introduction...

This preview shows pages 1–9. Sign up to view the full content.

1 Course IOE 316 - Winter 2010 Introduction to Markov Processes Instructor: Dennis Blumenfeld GSIs: Yinghao Ni Howard Wu Tuesday 03/09/10

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Course Objectives Analyze sequences of events that vary randomly according specific probabilistic laws Apply probability concepts to predict sequence behavior under uncertainty IOE 316 : Introduction to Markov Processes US Retail Sales (\$ million) Date Example:
3 IOE 316 : Introduction to Markov Processes What is a Markov Process ? It is a special case of a Stochastic Process So. ...... What is a Stochastic Process ? It is any system that varies randomly in time or space according to probabilistic laws

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 Stochastic Processes Examples : Stock market: Random variations in price over time Weather conditions each day Number of customers at a bank or post office Number of orders waiting for production Distribution of vehicles along a highway Number of traffic accidents Importance for IOE : Need mathematical models to analyze how processes behave under uncertainty
5 Stochastic Process: Example X n = Number of items sold on Day n ( n = 0, 1, 2, . ....... ) 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 Day, n Number of items sold X n Daily Sales of a Product

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 Stochastic Process: Example X n = Number of items sold on Day n ( n = 0, 1, 2, . ....... ) X 0 , X 1 , X 2 , X 3 , . ........ are a sequence of random variables. The different values that X n can take are called states In our example: If the number of items sold on day n is i , then X n = i and the process is in state i on day n In general, n can be any time units. Need not be a day. Can be a minute, week, month, etc. The probability that the process is in state i at time n is denoted by P { X n = i }
7 Stochastic Processes For a stochastic process in general: Probability that the process will be in state j at time n +1 depends on: the state i at time n (i.e., the current state), and the states at times n - 1, n - 2, . ........ , 1, 0 (i.e., the past states) For a Markov process: Probability that the process will be in state j at time n +1 depends only on: the state i at time n (i.e., the current state) and is independent of all past states i.e., if P i j denotes the conditional probability that the process will be in state j at time n +1, given that the process is in state i at time n , then { } i X j X P P n n j i = = = + 1 This is known as the Markov property (or memoryless property )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
8 Markov Process P i j = conditional probability that process will be in state j at time n +1, given that the process is in state i at time n ( n = 0, 1, 2, . ....) { } i X j X P P n n j i = = = + 1 In our example, the process changes from one state to another at discrete points in time, n = 0, 1, 2, . .... This type of Markov chain is called a
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 25

IOE+316+Notes+1 - Course IOE 316 Winter 2010 Introduction...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online