markov_practice_problems

markov_practice_problems - state from any other state...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Some practice problems involving Markov Chains December 20, 2010 1. Show that if T is a Markov chain such that there is a positive probability of transitioning from any vertex to any other vertex in a single time step (note: we are not saying that there is a positive probability of transitioning from vertex i to itself in one time step), then T has a unique equilibrium distirubiton. 2. Suppose there is an epidemic in which every month half of those who are well become sick, and a quarter of those who are sick become well. Find the steady state for the corresponding Markov process. 3. Suppose that an ant starts at position (0 , 0), and cane move up, down, left, or right one unit, so long as the new position says inside the grid { ( x, y ) : 0 x, y 2 } . Determine the probability that the ant reaches position (2 , 2) within 20 moves. 4. Find an example of a connected Markov chain – you can reach any
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: state from any other state (ignoring edge directions) that has more than one equilibrium distribution. 5. Population dynamics can often be modeled by composing polynomials, and in this problem we will explore this possibility: suppose that you have a population of bacteria whereby from one generation to the next, a single individual will produce j ospring with probability p j , and that that individual dies in the next generation. Let f ( x ) = s j =0 p j x j . 1 Say that in our case f ( x ) = 1 / 2 + 1 / 4 x + 1 / 4 x 2 . If at time 0 the population has 1 individual, determine the expected number of individuals for when time t = 4. Hint: think about what f ( f ( x )) means, and then f ( f ( f ( x ))), etc. 2...
View Full Document

Page1 / 2

markov_practice_problems - state from any other state...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online