1
Chapter 17
Graph-Theoretic Analysis of Finite Markov Chains
J. P. Jarvis
D. R. Shier
17.1
Introduction
Markov chains arise frequently in the modeling of physical and conceptual pro-
cesses that evolve over time. For example, the di±usion of liquids across a semi-
porous membrane, the spread of disease within a population, and the ﬂow of
personnel within the ranks of an organization can all be modeled using Markov
chains. In each of these cases, the system can be found in any of a ²nite number
of states, and transitions between states occur at discrete instants according to
speci²ed probabilities.
As one illustration, suppose that there are
M
molecules in a vessel, separated
into two chambers by a membrane, across which molecules can pass. A typical
con²guration of the system at any instant can be described by the distribution
of the
M
molecules between the two chambers. If there are
k
1
molecules in
the ²rst chamber, then there will be
k
2
=
M
−
k
1
molecules in the second
chamber. Transitions from the current state (
k
1
,k
2
) can occur by the movement
of a single molecule from the ²rst chamber to the second, or from the second
chamber to the ²rst. These two new states are represented by (
k
1
−
1
2
+1)
and (
k
1
+1
2
−
1), respectively. In one possible model of this process, the
probability of transition from (
k
1
2
)to(
k
1
−
1
2
+1)is given by
k
1
/M
,
whereas the probability of transition to (
k
1
2
−
1) is
k
2
=1
−
k
1
.
This quanti²es the idea that if more molecules are present in (say) chamber
1, then it is more likely for some molecule to transfer next from chamber 1 to
chamber 2. Using this mathematical model, one can answer questions such as:
(a) under what conditions do the molecules achieve an equilibrium con²guation,
(b) what are the (probabilistic) characteristics of this con²guration, and (c) at
what rate is this equilibrium approached?
The above is an instance of a ²nite-state Markov chain, which is the topic of
the present chapter. Normally, this subject is presented in terms of the (²nite)
matrix describing the Markov chain. Our objective here is to supplement this
viewpoint with a graph-theoretic approach, which provides a useful visual repre-
sentation of the process. A number of important properties of the Markov chain
(typically derived using matrix manipulations) can be deduced from this picto-
rial representation. Moreover, certain concepts from modern algebra will also be
illuminated by developing this approach. In addition, the graph-theoretic rep-