Chapt 10e - 10.8 Continuous-Time Markov Chains 373 In the...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 10.8 Continuous-Time Markov Chains 373 In the steady state, p j ( t ) → p j and lim t →∞ braceleftbigg dp i ( t ) dt bracerightbigg = Thus, we obtain = − v i p i summationdisplay j negationslash= i p ij + summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i Alternatively, we may write v i p i summationdisplay j negationslash= i p ij = summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i The left side of the first equation is the rate of transition out of state i , while the right side is the rate of transition into state i . This “balance” equation states that in the steady state the two rates are equal for any state in the Markov chain. 10.8.1 Birth and Death Processes Birth and death processes are a special type of continuous-time Markov chains. Consider a continuous-time Markov chain with states 0 , 1 , 2 ,... . If p ij = 0 when- ever j negationslash= i − 1 or j negationslash= i + 1, then the Markov chain is called a birth and death process. Thus, a birth and death process is a continuous-time Markov chain with states 0 , 1 , 2 ,... , in which transitions from state i can only go to either state i + 1 or state i − 1. That is, a transition either causes an increase in state by one or a decrease in state by one. A birth is said to occur when the state increases by one, and a death is said to occur when the state decreases by one. For a birth and death process, we define the following transition rates from state i : λ i = v i p i ( i + 1 ) μ i = v i p i ( i − 1 ) Thus, λ i is the rate at which a birth occurs when the process is in state i and μ i is the rate at which a death occurs when the process is in state i . The sum of these two rates is λ i + μ i = v i , which is the rate of transition out of state i . The state- transition-rate diagram of a birth and death process is shown in Figure 10.18. It is called a state-transition-rate diagram as opposed to a state-transition diagram 374 Chapter 10 Some Models of Random Processes Figure 10.18 State-Transition-Rate Diagram for Birth and Death Process because it shows the rate at which the process moves from state to state and not the probability of moving from one state to another. Note that μ = 0, since there can be no death when the process is in an empty state. The actual state-transition probabilities when the process is in state i are p i ( i + 1 ) and p i ( i − 1 ) . By definition, p i ( i + 1 ) = λ i /(λ i + μ i ) is the probability that a birth occurs before a death when the process is in state i . Similarly, p i ( i − 1 ) = μ i /(λ i + μ i ) is the probability that a death occurs before a birth when the process is in state i ....
View Full Document

This note was uploaded on 01/05/2010 for the course STAT 350 taught by Professor Carlton during the Fall '07 term at Cal Poly.

Page1 / 10

Chapt 10e - 10.8 Continuous-Time Markov Chains 373 In the...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online