{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Stat150_Spring06_Markov_intro

# Stat150_Spring06_Markov_intro - Statistics 150 Spring 2007...

This preview shows pages 1–10. Sign up to view the full content.

Statistics 150: Spring 2007 February 7, 2007 0-1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1 Markov Chains Let { X 0 ,X 1 , ···} be a sequence of random variables which take values in some countable set S , called the state space . Each X n is a discrete random variables that takes one of N possible values, where N = | S | ; it may be the case that N = . Deﬁnition. The process X is a Markov Chain if it satisﬁes the Markov condition : P ( X n = s | X 0 = x 0 ,X 1 = x 1 ,...,X n - 1 = x n - 1 ) = P ( X n = s | X n - 1 = x n - 1 ) for all n 1 and all s,x 1 , ··· ,x n - 1 S . 1
Deﬁnition. The chain X is called homogeneous if P ( X n +1 = j | X n = i ) = P ( X 1 = j | X 0 = i ) for all n,i,j . The transition matrix P = ( p ij ) is the | S | × | S | matrix of transition probabilities p ij = P ( X n +1 = j | X n = i ) . Henceforth, all Markov chains are assumed homogeneous unless otherwise speciﬁed. 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Theorem. The transition matrix P is a stochastic matrix, which is to say that: (a) P has non-negative entries, or p ij 0 for all i,j , (b) P has row sums equal to one, or j p ij = 1 for all i. Proof. An easy exercise. ± Deﬁnition. The n-step transition matrix P ( m,m + n ) = ( p ij ( m,m + n )) is the matrix of n-step transition probabilities p ij ( m,m + n ) = P ( X m + n = j | X m = i ) . 3
Theorem. Chapman-Kolmogorov equations. p ij ( m,m + n + r ) = X k p ik ( m,m + n ) p kj ( m + n,m + n + r ) . Therefore, P ( m,m + n + r ) = P ( m,m + n ) P ( m + n,m + n + r ) , and P ( m,m + n ) = P n , the n th power of P . Proof. We have as required that p ij ( m,m + n + r ) = P ( X m + n + r = j | X m = i ) = X k P ( X m + n + r = j,X m + n = k | X m = i ) 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
= X k P ( X m + n + r = j | X m + n = k,X m = i ) P ( X m + n = k | X m = i ) = X k P ( X m + n + r = j | X m + n = k ) P ( X m + n = k | X m = i ) where we have used the fact that P ( A T B | C ) = P ( A | B T C ) P ( B | C ) , together with the Markov property. The established equation may be written in matrix form as P ( m,m + n + r ) = P ( m,m + n ) P ( m + n,m + n + r ) , and it follows by iteration that P ( m,m + n ) = P n . ± 5
Let μ ( n ) i = P ( X n = i ) be the mass function of X n , and write μ n for the row vector with entries ( μ n i : i S ) . Lemma μ ( m + n ) = μ ( m ) P n , and hence μ ( n ) = μ (0) P n . 6

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Exercises 1) A die is rolled repeatedly. Which of the following are Markov chains? For those that are, supply the transition matrix. (a) The largest number X n shown up to the n th roll. (b) The number N n of sixes in n rolls. (c) At time r , the time C r since the most recent six. (d) At time r , the time B r until the next six. 7
2) Let X be a Markov chain on S , and let T be a random variables taking values in { 0 , 1 , 2 , ···} with the property that the indicator function 1 { T = n } of the event that T = n is a function of the variables X 1 ,X 2 , ··· ,X n . Such a random variables T is called a stopping time , and the above deﬁnition requires that it is decidable whether or not T = n with a knowledge only of the past and present,

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 45

Stat150_Spring06_Markov_intro - Statistics 150 Spring 2007...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online