E_markov - J Virtamo 38.3143 Queueing Theory Markov...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: J. Virtamo 38.3143 Queueing Theory / Markov processes 1 Markov processes (Continuous time Markov chains) Consider (stationary) Markov processes with a continuous parameter space (the parameter usually being time). Transitions from one state to another can occur at any instant of time. • Due to the Markov property, the time the system spends in any given state is memoryless: the distribution of the remaining time depends solely on the state but not on the time already spent in the state ⇒ the time is exponentially distributed. A Markov process X t is completely determined by the so called generator matrix or transition rate matrix q i,j = lim Δ t → P { X t +Δ t = j | X t = i } Δ t i negationslash = j- probability per time unit that the system makes a transition from state i to state j- transition rate or transition intensity The total transition rate out of state i is q i = summationdisplay j negationslash = i q i,j | lifetime of the state ∼ Exp( q i ) This is the rate at which the probability of state i decreases. Define q i,i =- q i J. Virtamo 38.3143 Queueing Theory / Markov processes 2 Transition rate matrix and time dependent state probability vector The transition rate matrix in full is Q =       q , q , 1 . . . q 1 , q 1 , 1 . . . . . . . . . . . .       =      - q q , 1 . . . q 1 ,- q 1 . . . . . . . . . . . .       row sums equal zero: the probability mass flowing out of state i will go to some other states (is conserved) State probability vector π ( t ) is now a function of time evolving as follows d dt π ( t ) = π ( t ) · Q ⇒ π ( t + Δ t ) = π ( t ) + π ( t ) · Q Δ t + o (Δ t ) = π ( t )( I + Q Δ t ) + o (Δ t ) Transition probability matrix over time interval Δ...
View Full Document

Page1 / 9

E_markov - J Virtamo 38.3143 Queueing Theory Markov...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online