markov - 1 Chapter 17 Graph-Theoretic Analysis of Finite...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Chapter 17 Graph-Theoretic Analysis of Finite Markov Chains J. P. Jarvis D. R. Shier 17.1 Introduction Markov chains arise frequently in the modeling of physical and conceptual pro- cesses that evolve over time. For example, the di±usion of liquids across a semi- porous membrane, the spread of disease within a population, and the flow of personnel within the ranks of an organization can all be modeled using Markov chains. In each of these cases, the system can be found in any of a ²nite number of states, and transitions between states occur at discrete instants according to speci²ed probabilities. As one illustration, suppose that there are M molecules in a vessel, separated into two chambers by a membrane, across which molecules can pass. A typical con²guration of the system at any instant can be described by the distribution of the M molecules between the two chambers. If there are k 1 molecules in the ²rst chamber, then there will be k 2 = M k 1 molecules in the second chamber. Transitions from the current state ( k 1 ,k 2 ) can occur by the movement of a single molecule from the ²rst chamber to the second, or from the second chamber to the ²rst. These two new states are represented by ( k 1 1 2 +1) and ( k 1 +1 2 1), respectively. In one possible model of this process, the probability of transition from ( k 1 2 )to( k 1 1 2 +1)is given by k 1 /M , whereas the probability of transition to ( k 1 2 1) is k 2 =1 k 1 . This quanti²es the idea that if more molecules are present in (say) chamber 1, then it is more likely for some molecule to transfer next from chamber 1 to chamber 2. Using this mathematical model, one can answer questions such as: (a) under what conditions do the molecules achieve an equilibrium con²guation, (b) what are the (probabilistic) characteristics of this con²guration, and (c) at what rate is this equilibrium approached? The above is an instance of a ²nite-state Markov chain, which is the topic of the present chapter. Normally, this subject is presented in terms of the (²nite) matrix describing the Markov chain. Our objective here is to supplement this viewpoint with a graph-theoretic approach, which provides a useful visual repre- sentation of the process. A number of important properties of the Markov chain (typically derived using matrix manipulations) can be deduced from this picto- rial representation. Moreover, certain concepts from modern algebra will also be illuminated by developing this approach. In addition, the graph-theoretic rep-
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 resentation immediately suggests several computational schemes for calculating important structural characteristics of the underlying problem. This chapter in- dicates how appropriate data structures and algorithms enable such calculations to be carried out in an efficient manner.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/31/2011 for the course AMS 546 taught by Professor Arkin,e during the Fall '08 term at SUNY Stony Brook.

Page1 / 17

markov - 1 Chapter 17 Graph-Theoretic Analysis of Finite...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online