EE 351K Probability, Statistics, and Random Processes
SPRING 2009
Instructor: Shakkottai/Vishwanath
{
shakkott,sriram
}
@ece.utexas.edu
Homework 9: Markov Chains
Due Monday, April 27th 2009
Problem 1
A spider and a fly move along a straight line in unit increments.
The spider always moves
towards the fly by one unit. The fly moves towards the spider by one unit with probability 0.3, moves away
from the spider with probability 0.3 and stays in place with probability 0.4. When the spider and fly land in
the same position, the spider captures the fly.
1. Construct a Markov chain that describes the relative location of the spider and fly.
2. Identify the transient and recurrent states.
Problem 2
A professor gives tests that are hard, medium or easy. If she gives a hard test, her next test will
be either medium or easy with equal probability. However, if she gives a medium or easy test, there is a 0.5
probability that her next test will be of the same difficulty, and a 0.25 probability for each of the other two
levels of difficulty. Construct an appropriate Markov chain and find the steadystate probabilities.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 BARD
 Probability theory, Stochastic process, Markov chain, Continuoustime Markov process, Andrey Markov, Random walk

Click to edit the document details