132B_1_Recitation7_Discrete_Time_Markov_Chains

132B_1_Recitation7_D - EE132B Recitation 7 Discrete Time Markov Chains Prof Izhak Rubin [email protected] Electrical Engineering Department UCLA 1

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
EE132B - Recitation 7 Discrete Time Markov Chains Prof. Izhak Rubin [email protected] Electrical Engineering Department UCLA 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Markov Chains   A stochastic process (SP) is a collection of random variables over some index set. , 0,1,2,. .. is a discrete time stochastic process if states assumes values from a countable state space 0,1, k X X k S    2,. ..     1 0 1 2 1 is considered to be a Markov Chain if it satisfies the : Future evolution of the process is independent of the past given the p Mar resent : | , ko , ,. v Prope .., | : rty n n n n X DTMC P X j X X X X P X j X CTMC P      | | t s u t s t X j X u t P X j X k X k X 1 X 2 X 3 X 4 X 5 1 2 3 1 2 3 4 5 2
Background image of page 2
Transition Probability Function         1 1 0 The one-step transition probability is independent of . ,, Given a time homogeneous state space = {1, 2, 3}, the transition probability function (TPF) and state diagram n n n n P i j P X j X i P X j X i P i j is shown below.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/12/2011 for the course EE 132B taught by Professor Izhakrubin during the Fall '09 term at UCLA.

Page1 / 8

132B_1_Recitation7_D - EE132B Recitation 7 Discrete Time Markov Chains Prof Izhak Rubin [email protected] Electrical Engineering Department UCLA 1

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online