This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: J. Virtamo 38.3143 Queueing Theory / Markov processes 1 Markov processes (Continuous time Markov chains) Consider (stationary) Markov processes with a continuous parameter space (the parameter usually being time). Transitions from one state to another can occur at any instant of time. • Due to the Markov property, the time the system spends in any given state is memoryless: the distribution of the remaining time depends solely on the state but not on the time already spent in the state ⇒ the time is exponentially distributed. A Markov process X t is completely determined by the so called generator matrix or transition rate matrix q i,j = lim Δ t → P { X t +Δ t = j  X t = i } Δ t i negationslash = j probability per time unit that the system makes a transition from state i to state j transition rate or transition intensity The total transition rate out of state i is q i = summationdisplay j negationslash = i q i,j  lifetime of the state ∼ Exp( q i ) This is the rate at which the probability of state i decreases. Define q i,i = q i J. Virtamo 38.3143 Queueing Theory / Markov processes 2 Transition rate matrix and time dependent state probability vector The transition rate matrix in full is Q = q , q , 1 . . . q 1 , q 1 , 1 . . . . . . . . . . . . =  q q , 1 . . . q 1 , q 1 . . . . . . . . . . . . row sums equal zero: the probability mass flowing out of state i will go to some other states (is conserved) State probability vector π ( t ) is now a function of time evolving as follows d dt π ( t ) = π ( t ) · Q ⇒ π ( t + Δ t ) = π ( t ) + π ( t ) · Q Δ t + o (Δ t ) = π ( t )( I + Q Δ t ) + o (Δ t ) Transition probability matrix over time interval Δ...
View
Full
Document
 Spring '09
 all
 Markov chain, Continuoustime Markov process, Xt, J. Virtamo

Click to edit the document details