132B_1_Recitation7_Discrete_Time_Markov_Chains

# 132B_1_Recitation7_D - EE132B Recitation 7 Discrete Time Markov Chains Prof Izhak Rubin [email protected]/* <![CDATA[ */!function(t,e,r,n,c,a,p){try{t=document.currentScript||function(){for(t=document.getElementsByTagName('script'),e=t.length;e--;)if(t[e].getAttribute('data-cfhash'))return t[e]}();if(t&&(c=t.previousSibling)){p=t.parentNode;if(a=c.getAttribute('data-cfemail')){for(e='',r='0x'+a.substr(0,2)|0,n=2;a.length-n;n+=2)e+='%'+('0'+('0x'+a.substr(n,2)^r).toString(16)).slice(-2);p.replaceChild(document.createTextNode(decodeURIComponent(e)),c)}p.removeChild(t)}}catch(u){}}()/* ]]> */ Electrical Engineering Department UCLA 1

This preview shows pages 1–4. Sign up to view the full content.

EE132B - Recitation 7 Discrete Time Markov Chains Prof. Izhak Rubin [email protected] Electrical Engineering Department UCLA 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Markov Chains   A stochastic process (SP) is a collection of random variables over some index set. , 0,1,2,. .. is a discrete time stochastic process if states assumes values from a countable state space 0,1, k X X k S    2,. ..     1 0 1 2 1 is considered to be a Markov Chain if it satisfies the : Future evolution of the process is independent of the past given the p Mar resent : | , ko , ,. v Prope .., | : rty n n n n X DTMC P X j X X X X P X j X CTMC P      | | t s u t s t X j X u t P X j X k X k X 1 X 2 X 3 X 4 X 5 1 2 3 1 2 3 4 5 2
Transition Probability Function         1 1 0 The one-step transition probability is independent of . ,, Given a time homogeneous state space = {1, 2, 3}, the transition probability function (TPF) and state diagram n n n n P i j P X j X i P X j X i P i j is shown below.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 01/12/2011 for the course EE 132B taught by Professor Izhakrubin during the Fall '09 term at UCLA.

### Page1 / 8

132B_1_Recitation7_D - EE132B Recitation 7 Discrete Time Markov Chains Prof Izhak Rubin [email protected] Electrical Engineering Department UCLA 1

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online