This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Dealing with Some Assumptions Computation of Transition Probabilities Solving the Backward or Forward Equations Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1/ 19 Dealing with Some Assumptions Computation of Transition Probabilities Solving the Backward or Forward Equations Embedded DTMC has No SelfLoops Explosiveness (no need for finite state space) Embedded DTMC has No SelfLoops We assumed that the embedded DTMC has Q ii = 0. Is this required? Recall we are trying to model a natural phenomenon by CTMCs...and so we would not actually view self loops (it would just stay in the same state at time t ). Just for fun, suppose we want a CTMC from an embedded DTMC with Q ii > 0 for some i and some rates v i for the exponential times between jumps (including selfloops) So the rate at which we see jumps from i to j (for i 6 = j ) should be Q ij v i (as before) But the rate at which actually leave state i is not v i 2/ 19 Dealing with Some Assumptions Computation of Transition Probabilities Solving the Backward or Forward Equations Embedded DTMC has No SelfLoops Explosiveness (no need for finite state space) Embedded DTMC has No SelfLoops We assumed that the embedded DTMC has Q ii = 0. Is this required? Recall we are trying to model a natural phenomenon by CTMCs...and so we would not actually view self loops (it would just stay in the same state at time t ). Just for fun, suppose we want a CTMC from an embedded DTMC with Q ii > 0 for some i and some rates v i for the exponential times between jumps (including selfloops) So the rate at which we see jumps from i to j (for i 6 = j ) should be Q ij v i (as before) But the rate at which actually leave state i is not v i 2/ 19 Dealing with Some Assumptions Computation of Transition Probabilities Solving the Backward or Forward Equations Embedded DTMC has No SelfLoops Explosiveness (no need for finite state space) Embedded DTMC has No SelfLoops We assumed that the embedded DTMC has Q ii = 0. Is this required? Recall we are trying to model a natural phenomenon by CTMCs...and so we would not actually view self loops (it would just stay in the same state at time t ). Just for fun, suppose we want a CTMC from an embedded DTMC with Q ii > 0 for some i and some rates v i for the exponential times between jumps (including selfloops) So the rate at which we see jumps from i to j (for i 6 = j ) should be Q ij v i (as before) But the rate at which actually leave state i is not v i 2/ 19 Dealing with Some Assumptions Computation of Transition Probabilities Solving the Backward or Forward Equations Embedded DTMC has No SelfLoops Explosiveness (no need for finite state space) Embedded DTMC has No SelfLoops We assumed that the embedded DTMC has Q ii = 0. Is this required?...
View
Full
Document
This note was uploaded on 10/07/2010 for the course ORIE 3510 taught by Professor Resnik during the Spring '09 term at Cornell University (Engineering School).
 Spring '09
 RESNIK

Click to edit the document details