This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 10.8 ContinuousTime Markov Chains 373 In the steady state, p j ( t ) → p j and lim t →∞ braceleftbigg dp i ( t ) dt bracerightbigg = Thus, we obtain = − v i p i summationdisplay j negationslash= i p ij + summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i Alternatively, we may write v i p i summationdisplay j negationslash= i p ij = summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i The left side of the first equation is the rate of transition out of state i , while the right side is the rate of transition into state i . This “balance” equation states that in the steady state the two rates are equal for any state in the Markov chain. 10.8.1 Birth and Death Processes Birth and death processes are a special type of continuoustime Markov chains. Consider a continuoustime Markov chain with states 0 , 1 , 2 ,... . If p ij = 0 when ever j negationslash= i − 1 or j negationslash= i + 1, then the Markov chain is called a birth and death process. Thus, a birth and death process is a continuoustime Markov chain with states 0 , 1 , 2 ,... , in which transitions from state i can only go to either state i + 1 or state i − 1. That is, a transition either causes an increase in state by one or a decrease in state by one. A birth is said to occur when the state increases by one, and a death is said to occur when the state decreases by one. For a birth and death process, we define the following transition rates from state i : λ i = v i p i ( i + 1 ) μ i = v i p i ( i − 1 ) Thus, λ i is the rate at which a birth occurs when the process is in state i and μ i is the rate at which a death occurs when the process is in state i . The sum of these two rates is λ i + μ i = v i , which is the rate of transition out of state i . The state transitionrate diagram of a birth and death process is shown in Figure 10.18. It is called a statetransitionrate diagram as opposed to a statetransition diagram 374 Chapter 10 Some Models of Random Processes Figure 10.18 StateTransitionRate Diagram for Birth and Death Process because it shows the rate at which the process moves from state to state and not the probability of moving from one state to another. Note that μ = 0, since there can be no death when the process is in an empty state. The actual statetransition probabilities when the process is in state i are p i ( i + 1 ) and p i ( i − 1 ) . By definition, p i ( i + 1 ) = λ i /(λ i + μ i ) is the probability that a birth occurs before a death when the process is in state i . Similarly, p i ( i − 1 ) = μ i /(λ i + μ i ) is the probability that a death occurs before a birth when the process is in state i ....
View
Full
Document
This note was uploaded on 01/05/2010 for the course STAT 350 taught by Professor Carlton during the Fall '07 term at Cal Poly.
 Fall '07
 Carlton
 Markov Chains

Click to edit the document details