1
Markov Chain
Chapter 1
2
Finding a cheese in the maze
Cheese
Black
hole
mouse
3
Assumptions
The mouse does not know where the
cheese is.
The mouse will get nothing and leaves the
game if it enters the black hole.
4
Questions
A biologist may want to know the
following things.
Can the mouse detect the cheese by smell?
Can the mouse self-train if it is put back to
the game after entering the black hole?
Would the chance of getting cheese be the
same for different initial position?

This ** preview** has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
5
Establish a probabilistic model
To determine if the mouse can find cheese
by smell, we use the following idea.
Determine the probability of success under
the alternative hypothesis.
Conduct experiments and compare the results
with the probability.
If the successful rate is not higher than the
theoretical probability significantly, then the
hypothesis is rejected.
6
Establish
a Probabilistic model
Self-train hypothesis. Idea:
Use special materials to make the maze so
that the mouse cannot find cheese by smell.
Determine the probability under the
alternative hypothesis.
If the successful rate is not higher than the
theoretical probability significantly, then the
hypothesis is rejected.
7
Model construction: ass. room #
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
8
Model construction: stochastic
Define the stochastic variable X
n
be the
position of a mouse at step n.
X
0
= 0 means the mouse is initially placed
in room 0.
Successful probability:
P(X
n
= 13, for some n)
Failure probability:
P(X
n
= 7, for some n)