This preview shows page 1. Sign up to view the full content.
Pstat160a
Handout 5
Intro Markov Chains
Example 1:
Imagine an object moving at random along the points 1
,
2
,
3
,
4. Each moves depends only on the current position, as
depicted in the ﬁgure below and according to the following rules:
from
1
, moves to
2
with probability 1
from
2
,
stays in
2
”
.2
moves to
1
”
.5
moves to
3
”
.3
from
3
, moves to
4
”
1
from
4
, moves to
1
”
.6
moves to
2
”
.4
1
2
3
4
.5
.3
1
.4
.6
.2
1
These rules can be represented in the following matrix form:
P
=
0
1
0
0
.
5
.
2
.
3 0
0
0
0
1
.
6
.
4
0
0
Where the entry
P
ij
represent the probability to go from
i
to
j
. For
instance,
P
21
=
.
5 means that the proability to move from 2 to 1 is .5.
This is an example of a
ﬁnite state Markov Chain
. It is a very special case of
stochastic process
, that is a sequence
of random number
{
X
0
,X
1
,...,X
n
,...
}
, where the ordering is though to represent
time
, indexed by
n
. The
state space
is
the set of value that the process takes, for us it would usual encoded as being 1
This is the end of the preview. Sign up
to
access the rest of the document.
 Winter '10
 bonnet
 Markov Chains

Click to edit the document details