{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

ClassNotes05

# ClassNotes05 - Markov Chains OEM 2009 ESI 6321 Applied...

This preview shows pages 1–6. Sign up to view the full content.

Markov Chains OEM 2009 ESI 6321 –Applied Probability Methods in Engineering 2 Stochastic processes The last module of this course deals with describing and analyzing systems under uncertainty In particular, we will study how a random variable changes over time Examples: Price of a stock or portfolio of stocks Inventory level of a good in a warehouse Career path (workforce planning) Number of customers present in a store or bank etc.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3 Stochastic processes We will focus on systems that we observe at discrete points in time E.g., we evaluate the value of a stock portfolio at the end of each trading day Denote the time points by t =0,1,2,… Denote the value (at time t ) of the characteristic of the system that we are interested in by X t X t is a random variable The sequence X 0 , X 1 , X 2 ,… is called a stochastic process 4 Example 1 We are interested in tracking the stock market on a daily basis Let X t denote the value of the Dow Jones index at the end of trading day t We are currently at the end of trading day 0, and observe X 0 = x 0 Can we model/study the relationship between the random variables X t ( t =0,1,2,…)?
5 Example 2 You are visiting Las Vegas, and have a gambling budget of \$ x 0 You participate in a game in which you repeatedly bet \$1 if you win (which happens with probability p ) you receive \$2 if you loose (which happens with probability 1- p ) you receive nothing you stop playing as soon as you are broke or have doubled your initial budget Denote your total wealth at time t by X t 6 States and state space The set of values that the random variable X t can take is called the state space of the stochastic process Example 1: [0, ) Example 2: {0,1,2,…2 x 0 } We will restrict ourselves to situations in which the state space consists of a finite number of elements only often: S = {1,2,…, s } If X t = i we say that the stochastic process is in state i at time t .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document