6.262.Lec18

6.262.Lec18 - Discrete Stochastic Processes Lecture 18...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 17 - 4/14/2010 Discrete Stochastic Processes 1 Discrete Stochastic Processes Lecture 18 Countable-State Markov Processes Definitions – what depends on what Example – radioactive decay Example – M/M/1 Queue Transition rates and alternate characterizations Steady state Anomalous Case
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 17 - 4/14/2010 Discrete Stochastic Processes 2 M ARKOV P ROCESSES A Markov process is a Markov chain with continuous, exponentially distributed holding times at each state. The mean holding time can depend on the state, but the holding times for distinct passages through the state are independent. There can be a different rate of departure for each state the chain may be in. More specifically, the holding interval U n between the time that state X n-1 = k and the time that state X n is entered is a nonnegative exponentially distributed random variable with parameter k ν , i.e., () 1 ( u | X k) = 1 - e k u nn PU υ ≤= Furthermore, conditional on X n-1 , U n is jointly independent of X m , for all m n-1 and of U m , for all m n. Within a holding time the future is conditionally independent of the past, given the present state ( ) X t . So the process has the Markov Property
Background image of page 2
Lecture 17 - 4/14/2010 Discrete Stochastic Processes 3 Definition A countable-state Markov process X(t), t 0, satisfies X(t) = X n , S n t < S n+1 , S 0
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/21/2010 for the course EE 5581 taught by Professor Moon,j during the Spring '08 term at Minnesota.

Page1 / 12

6.262.Lec18 - Discrete Stochastic Processes Lecture 18...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online