{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lecturenotes02 - MS&E 223 Simulation Peter J Haas Lecture...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
MS&E 223 Lecture Notes #2 Simulation Stochastic Process Models for DESS Peter J. Haas Spring Quarter 2009-10 Page 1 of 14 Some Stochastic Process Models for Discrete-Event Stochastic Systems 1. Discrete-Event Stochastic Systems Recall our previous definition of a discrete-event stochastic system: the system makes stochastic state transitions only at an increasing sequence of random times. If X(t) is the state of the system at time t, then a typical sample path of the underlying stochastic process of the simulation, i.e. (X(t): t 0), looks like this (for a finite or countably infinite state space): For a given simulation model, we need to precisely specify the process (X(t): t 0), so that we can generate sample paths of the process and obtain meaningful point and interval estimates of system characteristics based on well-defined properties of the process. We will describe several types of stochastic processes that can serve as the underlying process of a simulation. 2. Discrete-Time Markov Chains (DTMC’s) Sometimes we are interested in the discrete-time process (X n : n 0), where X n is the state of the system just after the n th state transition. (Sometimes we might only look at the system state at a specified subsequence of the state transitions.) If the state space is finite or countably infinite and the Markov property 1 1 1 0 0 1 { | , , , } { | } n n n n n n n n P X x X x X x X x P X x X x + + = = = = = = = holds, then (X n : n 0) is a discrete-time Markov chain. A (time-homogeneous) DTMC can be specified by giving: (1) the transition matrix P , where P = (P(x, y): x, y S). Here P(x, y) = P{X n+1 = y | X n = x} and S is the state space of (X n : n 0) . (2) the initial distribution μ , where μ = ( μ (x):x S). Here { } 0 ( ) x P X x μ = = for x S . t X(t)
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
MS&E 223 Lecture Notes #2 Simulation Stochastic Process Models for DESS Peter J. Haas Spring Quarter 2009-10 Page 2 of 14 Example (The Markovian Jumping Frog of Calaveras County) : 1/2 1/3 1/2 3/4 2/3 1/4 1 2 3 The frog starts in states 1 and 2 with equal probability. Let X n be the lily pad occupied by the frog at jump n. Then (X n : n 0) is a DTMC, with P = 0 4 / 1 4 / 3 3 / 2 0 3 / 1 2 / 1 2 / 1 0 and μ = 0 2 / 1 2 / 1 (we’ll assume that all vectors are column vectors). Suppose that we are interested in the probability that the frog will be on lily pad 2 after the k th jump, where k is fixed. We can express this quantity in the form E[f(X k )], where f(x) =1 if x = 2 and f(x) = 0 if x = 1 or 3---the function f is called an indicator function and we sometimes write I(X k = 2) instead of f(X k ).We can compute E[f(X k )] in one of two ways: (1) Numerically, letting n v (i) P(frog on pad i after nth jump) = and writing n n n n v (1) v v (2) v (3) = for n 0 : Set v 0 = μ t t m 1 m v v + = P for m 0.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern