CSci 5451, S’2010
Programming Assignment # 3
Due Date: April 26th, 2010
1
Background
This lab is about Markov chain models and it will imple
ment and test a “random walk on a 2D Grid”. Consider
the grid pictured on the next Figure. The fgure shows a
cartesian network grid and a particle. At any given time
the particle can jump to any o± its nearest neighbors in
the horizontal or vertical direction with di²erent prob
abilities; ±or example 0.3 going north, 0.2 going south,
0.1 going west, 0.2 going east, and 0.2 remaining where
it is.
One can think o± a particle starting in a certain position
in the grid (in this assignment it will start at the origin)
which then makes a hop to one o± its neighbors either
north, south, to the west, to the east or to the point
itsel± (i.e. remains at this point ±or one hop).
i,j
i+1,j
i,j+1
i1,j
i,j1
The problem is to determine the probability that the particle will be at any given node a±ter
a very long time. A “state” is the location o± the particle within the grid. When the particle
jumps to another node it changes states. The matrix
P
o± transition propabilities is defned by
p
ij
= probability ±or a state to jump ±rom state
i
to state
j
. States are ordered in row major
order on a grid.
So, assuming an
n
×
n
grid, the gridpoint (0,0) corresponds to sate 0, the point
(0,1) to state 1, .
.., the point (0
,n

1) to state
n

1, the point (1
,
0) to state
n
etc.
.. So fxing
i
in (
i,j
) corresponds to moving EastWest and fxing
j
corresponds to moving NorthSouth.
The method consists o± taking a given initial probability vector and then to iterate:
u
k
+1
=
u
k
P
±or a certain number o± steps until
u
k
converges. There±ore, the basic problem is one o± per
±orming a number o± matrixvector products. The fnal result is a vector o± probabilities.
2
Details
Each point o± the grid is assigned a set o± probabilities (WEST,EAST,SOUTH,NORTH) where
WEST is the probability that when a particle is at this point it will hop to the west, EAST
is probability that it hops to the east and so on. The probability that a particle will stay at
the same location (termed “REMAIN”) ±or one hop is
REMAIN
= 1

(
WEST
+
EAST
+
SOUTH
+
NORTH
). Each point in the grid may have a di²erent set o± probabilities. We
will label by (R,W,E,S,N) the fve probabilities corresponding to remain, west, east, south, and
north respectively.
Assume that a particle starts at state
k
. We want to fnd what is the probability that a±ter
m
hops the particle will end at any given point. Let
u
0
= (0
,
0
,...,
1
,
0
,
0
,...
0) where the “1” is
located at position
k

1 (indices start at zero as in the C/C++ language). Then
u
0
can be