{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

E_markov

# E_markov - J Virtamo 38.3143 Queueing Theory Markov...

This preview shows pages 1–3. Sign up to view the full content.

J. Virtamo 38.3143 Queueing Theory / Markov processes 1 Markov processes (Continuous time Markov chains) Consider (stationary) Markov processes with a continuous parameter space (the parameter usually being time). Transitions from one state to another can occur at any instant of time. Due to the Markov property, the time the system spends in any given state is memoryless: the distribution of the remaining time depends solely on the state but not on the time already spent in the state the time is exponentially distributed. A Markov process X t is completely determined by the so called generator matrix or transition rate matrix q i,j = lim Δ t 0 P { X t t = j | X t = i } Δ t i negationslash = j - probability per time unit that the system makes a transition from state i to state j - transition rate or transition intensity The total transition rate out of state i is q i = summationdisplay j negationslash = i q i,j | lifetime of the state Exp( q i ) This is the rate at which the probability of state i decreases. Define q i,i = - q i

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
J. Virtamo 38.3143 Queueing Theory / Markov processes 2 Transition rate matrix and time dependent state probability vector The transition rate matrix in full is Q = q 0 , 0 q 0 , 1 . . . q 1 , 0 q 1 , 1 . . . . . . . . . . . . = - q 0 q 0 , 1 . . . q 1 , 0 - q 1 . . . . . . . . . . . . row sums equal zero: the probability mass flowing out of state i will go to some other states (is conserved) State probability vector π ( t ) is now a function of time evolving as follows d dt π ( t ) = π ( t ) · Q π ( t + Δ t ) = π ( t ) + π ( t ) · Q Δ t + o t ) = π ( t )( I + Q Δ t ) + o t ) Transition probability matrix over time interval Δ t is P t ) = I + Q Δ t - tends to the identity matrix I as Δ t 0 - Q = P prime (0) is the time derivative of the transition prob. matrix (transition rate matrix)
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}