Chapt 10e - 10.8 Continuous-Time Markov Chains 373 In the...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
10.8 Continuous-Time Markov Chains 373 In the steady state, p j ( t ) p j and lim t →∞ braceleftbigg dp i ( t ) dt bracerightbigg = 0 Thus, we obtain 0 = − v i p i summationdisplay j negationslash= i p ij + summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i Alternatively, we may write v i p i summationdisplay j negationslash= i p ij = summationdisplay j negationslash= i p j p ji v j 1 = summationdisplay i p i The left side of the first equation is the rate of transition out of state i , while the right side is the rate of transition into state i . This “balance” equation states that in the steady state the two rates are equal for any state in the Markov chain. 10.8.1 Birth and Death Processes Birth and death processes are a special type of continuous-time Markov chains. Consider a continuous-time Markov chain with states 0 , 1 , 2 ,... . If p ij = 0 when- ever j negationslash= i 1 or j negationslash= i + 1, then the Markov chain is called a birth and death process. Thus, a birth and death process is a continuous-time Markov chain with states 0 , 1 , 2 ,... , in which transitions from state i can only go to either state i + 1 or state i 1. That is, a transition either causes an increase in state by one or a decrease in state by one. A birth is said to occur when the state increases by one, and a death is said to occur when the state decreases by one. For a birth and death process, we define the following transition rates from state i : λ i = v i p i ( i + 1 ) μ i = v i p i ( i 1 ) Thus, λ i is the rate at which a birth occurs when the process is in state i and μ i is the rate at which a death occurs when the process is in state i . The sum of these two rates is λ i + μ i = v i , which is the rate of transition out of state i . The state- transition-rate diagram of a birth and death process is shown in Figure 10.18. It is called a state-transition-rate diagram as opposed to a state-transition diagram
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
374 Chapter 10 Some Models of Random Processes Figure 10.18 State-Transition-Rate Diagram for Birth and Death Process because it shows the rate at which the process moves from state to state and not the probability of moving from one state to another. Note that μ 0 = 0, since there can be no death when the process is in an empty state. The actual state-transition probabilities when the process is in state i are p i ( i + 1 ) and p i ( i 1 ) . By definition, p i ( i + 1 ) = λ i /(λ i + μ i ) is the probability that a birth occurs before a death when the process is in state i . Similarly, p i ( i 1 ) = μ i /(λ i + μ i ) is the probability that a death occurs before a birth when the process is in state i . Recall that the rate at which the probability of the process being in state i changes with time is given by dp i ( t ) dt = − v i p i ( t ) summationdisplay j negationslash= i p ij + summationdisplay j negationslash= i p j ( t ) p ji v j = − i + μ i ) p i ( t ) + μ i + 1 p i + 1 ( t ) + λ i 1 p i 1 ( t ) In the steady state, lim t →∞ braceleftbigg dp i ( t ) dt bracerightbigg = 0 , which gives i + μ i ) p i ( t ) = μ i + 1 p i + 1 ( t ) + λ i 1 p i 1 ( t )
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern