160a_ho5

160a_ho5 - Pstat160a Handout 5 Intro Markov Chains Example...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
Pstat160a Handout 5 Intro Markov Chains Example 1: Imagine an object moving at random along the points 1 , 2 , 3 , 4. Each moves depends only on the current position, as depicted in the figure below and according to the following rules: from 1 , moves to 2 with probability 1 from 2 , stays in 2 .2 moves to 1 .5 moves to 3 .3 from 3 , moves to 4 1 from 4 , moves to 1 .6 moves to 2 .4 1 2 3 4 .5 .3 1 .4 .6 .2 1 These rules can be represented in the following matrix form: P = 0 1 0 0 . 5 . 2 . 3 0 0 0 0 1 . 6 . 4 0 0 Where the entry P ij represent the probability to go from i to j . For instance, P 21 = . 5 means that the proability to move from 2 to 1 is .5. This is an example of a finite state Markov Chain . It is a very special case of stochastic process , that is a sequence of random number { X 0 ,X 1 ,...,X n ,... } , where the ordering is though to represent time , indexed by n . The state space is the set of value that the process takes, for us it would usual encoded as being 1
Background image of page 1
This is the end of the preview. Sign up to access the rest of the document.
Ask a homework question - tutors are online