132B_1_Sec12_CTMC_090710

132B_1_Sec12_CTMC_090710 - Continuous-Time Markov Chains...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Continuous-Time Markov Chains Professor Izhak Rubin Electrical Engineering Department UCLA rubin@ee.ucla.edu © 2010-2011 by Izhak Rubin
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
© Prof. Izhak Rubin 2 Continuous-Time Markov Chains: Definition {} t Definition: The stochastic process , 0 whose states assume values in a countable state space S, where X denotes the state of the process at time t 0, is said to be a continuous-time (or cont t XX t =≥ () inuous parameter) Markov Chain (CTMC) if it satisfies the Markov Prope Markov Prope rty (MP): , r , , , t: 0. y ts u t PX yX u t PX yX y Sts ++ = ≤= =
Background image of page 2
© Prof. Izhak Rubin 3 Transition Probability Function () , ,0 , Its Transition Probability Function (TPF) is defined as , , x, , 0 Consider only time-homogeneous Markov chains, for which: , , , x, 0 st t s t s Px y P X y X x y S ts y P x y P x y y S −− == =≡ ± . The Transition Probability Function (TPF) satisfies the following properties: 1. , 0, , 0 2. , 1, 0, 3. , , , , , , s, 0. (Chap t t yS t s zS Pxy xy St t x S y P x z P z y x y St + ≥∀ =∀ man Kolmogorov Equation: CKE)
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
© Prof. Izhak Rubin 4 Chapman-Kolmogorov Equation and Poisson Counting Process () {} 0 0 00 0 Proof of Chapman Kolmogorov Equation: , , , ,, s,t 0, x,y S. Example: Let , 0 be the Pois ts t zS t t t t t s t Px y P X y X x PX yX zX x xPX zX x yX zPX zX x PxzP zy NN t ++ + + + ∈∈ == = = = = = = = = = = = ∀≥ =≥ ∑∑ with intensity , over the state space of non-negative integers S = {0,1,2,. ..}. Then, we readily show that: 1. satisfies the Markov Property 2. , 0 , t 0 ! son Counting Process yx t t N t Pxy e Iy x λ , x,y S.
Background image of page 4
© Prof. Izhak Rubin 5 Probabilistic Structure () 0 We consider a CTMC X with a standard TPF for which lim , , , We define to designate the residual time period occupied by the state assumed by the process at time .
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/12/2011 for the course EE 132B taught by Professor Izhakrubin during the Fall '09 term at UCLA.

Page1 / 16

132B_1_Sec12_CTMC_090710 - Continuous-Time Markov Chains...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online