6.262.Lec20

# 6.262.Lec20 - DISCRETE STOCHASTIC PROCESSES Lecture 20...

This preview shows pages 1–4. Sign up to view the full content.

Lecture 20 4/21/2010 Discrete Stochastic Processes 1 DISCRETE STOCHASTIC PROCESSES Lecture 20 Reading: Sections 6.1, 6.3, 6.5 Review: Countable-State Markov Processes (aka Continuous-Time Markov Chains) Forward Kolmogorov Equation Steady State as Solution to Forward Kolmogorov Equation Birth-Death Processes and the M/M/1 Queue

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Lecture 20 4/21/2010 Discrete Stochastic Processes 2 M ARKOV P ROCESSES A Markov process is a semi-Markov process with exponentially distributed inter- transition intervals (rate ν i ) that are independent of the next state. There can be a different rate of departure for each state the chain may be in. A Markov process is defined to be irreducible if the embedded chain is irreducible; assume this throughout. For now, we also assume P ii = 0 , i.e., there are no self transitions allowed in the embedded chain. We have defined a Markov process in terms of an exponential departure rate for each state ( i ) and embedded Markov chain probabilities ( P ij ).
Lecture 20 4/21/2010 Discrete Stochastic Processes 3 , i j, ij ij i qP ν = j, ij ij i = Alternate Description of a Markov Process Given I'm in state i, there are a bunch of exponential processes racing. If process i j fires first I go to state j . (For example, this corresponds to an M/M/1 queue with a positive number of customers having exponential interarrivals increasing the state by 1 and independent different rate exponential interdepartures decreasing the state by 1.) For each i, j q ij is the rate of the exponential intertransition time racing to move the process to state j. Then the intertransition time of leaving state i is exponential with rate ii j ji q =

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 10/21/2010 for the course EE 5581 taught by Professor Moon,j during the Spring '08 term at Minnesota.

### Page1 / 13

6.262.Lec20 - DISCRETE STOCHASTIC PROCESSES Lecture 20...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online