232B_1_Section 1.2 232B_Rubin_CTMC_12 12 2015A

232B_1_Section 1.2 232B_Rubin_CTMC_12 12 2015A - Continuous...

Info icon This preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Professor Izhak Rubin EE232B 1 Continuous Time Markov Chains Class Notes By Professor Izhak Rubin Electrical Engineering Department UCLA Electrical Engineering 232B © 2015-2016 by Izhak Rubin
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
EE232B 2 Professor Izhak Rubin Continuous Time Markov Chains: Definitions t Definition: The stochastic process , 0 whose states assume values in a countable state space S, where X denotes the state of the process at continuous-time t 0, is said to be a continuous-ti t X X t Markov Property: me (or continuous parameter) Markov Chain (CTMC) if it satisfies the Markov Property (MP): , , , , 0 Its Transition Probability Function (TPF) is define t s u t s t P X y X u t P X y X y S t s   , , 0, d as , , x, , 0 Consider only time-homogeneous Markov chains, for which: , , , , x, , 0. The Transition Probability Function (TPF) satisfie s t t s s t t s t s P x y P X y X x y S t s P x y P x y P x y y S t s s the following properties: 1. , 0, , , 0 2. , 1, 0, 3. , , , , , , s, 0. (Chapman Kolmogorov Eq.- CKE) t t y S t s t s z S P x y x y S t P x y t x S P x y P x z P z y x y S t  
Image of page 2
EE232B 3 Professor Izhak Rubin Chapman-Kolmogorov Equation and Poisson Counting Process 0 0 0 0 0 Proof of Chapman Kolmogorov Equation: , , , , , s,t 0, x,y S. Example: Let , 0 be the Pois t s t s t s t z S t s t t z S t s t t t s z S z S t P x y P X y X x P X y X z X x P X y X z X x P X z X x P X y X z P X z X x P x z P z y N N t with intensity , over the state space of non-negative integers S = {0,1,2,...}. Then, we readily show (where I(A) denotes the indicator function of event A, so that I(A) = 1 if A h son Counting Process olds and = 0, otherwise): 1. satisfies the Markov Property 2. , 0 , t 0, x,y S. ! y x t t N t P x y e I y x y x
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon