Pstat160a
Handout 5
Intro Markov Chains
Example 1:
Imagine an object moving at random along the points 1
,
2
,
3
,
4.
Each moves depends only on the current position, as
depicted in the figure below and according to the following rules:
from
1
,
moves to
2
with probability
1
from
2
,
stays in
2
”
.2
moves to
1
”
.5
moves to
3
”
.3
from
3
,
moves to
4
”
1
from
4
,
moves to
1
”
.6
moves to
2
”
.4
1
2
3
4
.5
.3
1
.4
.6
.2
1
These rules can be represented in the following matrix form:
P
=
0
1
0
0
.
5
.
2
.
3
0
0
0
0
1
.
6
.
4
0
0
Where the entry
P
ij
represent the probability to go from
i
to
j
. For
instance,
P
21
=
.
5 means that the proability to move from 2 to 1 is .5.
This is an example of a
finite state Markov Chain
. It is a very special case of
stochastic process
, that is a sequence
of random number
{
X
0
, X
1
, . . . , X
n
, . . .
}
, where the ordering is though to represent
time
, indexed by
n
. The
state space
is
the set of value that the process takes, for us it would usual encoded as being 1
,
2
, . . . , M
. Each points in the state space is
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '10
 bonnet
 Markov Chains, Probability theory, Stochastic process, Markov chain, Intro Markov Chains, ﬁnite state Markov

Click to edit the document details