6.262.Lec20

6.262.Lec20 - DISCRETE STOCHASTIC PROCESSES Lecture 20...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 20 4/21/2010 Discrete Stochastic Processes 1 DISCRETE STOCHASTIC PROCESSES Lecture 20 Reading: Sections 6.1, 6.3, 6.5 Review: Countable-State Markov Processes (aka Continuous-Time Markov Chains) Forward Kolmogorov Equation Steady State as Solution to Forward Kolmogorov Equation Birth-Death Processes and the M/M/1 Queue
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Lecture 20 4/21/2010 Discrete Stochastic Processes 2 M ARKOV P ROCESSES A Markov process is a semi-Markov process with exponentially distributed inter- transition intervals (rate ν i ) that are independent of the next state. There can be a different rate of departure for each state the chain may be in. A Markov process is defined to be irreducible if the embedded chain is irreducible; assume this throughout. For now, we also assume P ii = 0 , i.e., there are no self transitions allowed in the embedded chain. We have defined a Markov process in terms of an exponential departure rate for each state ( i ) and embedded Markov chain probabilities ( P ij ).
Background image of page 2
Lecture 20 4/21/2010 Discrete Stochastic Processes 3 , i j, ij ij i qP ν = j, ij ij i = Alternate Description of a Markov Process Given I'm in state i, there are a bunch of exponential processes racing. If process i j fires first I go to state j . (For example, this corresponds to an M/M/1 queue with a positive number of customers having exponential interarrivals increasing the state by 1 and independent different rate exponential interdepartures decreasing the state by 1.) For each i, j q ij is the rate of the exponential intertransition time racing to move the process to state j. Then the intertransition time of leaving state i is exponential with rate ii j ji q =
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 13

6.262.Lec20 - DISCRETE STOCHASTIC PROCESSES Lecture 20...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online