IE 336
Handout #7
Mar. 5, 2011
Due Mar. 11, 2011
Homework Set #6
1.
Consider a Markov chain given by the following transition matrix
P
=
2
q
r
2
p
3
r
p
2
q
6
q
2
p
3
r
3
.
(a)
Compute
p
,
q
, and
r
. Determine the transition diagram.
(b)
If the vector of state probabilities at time step 4 is
p
(4)
=
£
0
.
2
0
.
6
0
.
2
/
determine the
vector of state probabilities at time step 8.
(c)
Compute the vector of steadystate probabilities.
2.
Assume the setup of problem 1.
(a)
If at some point the process is in state 2, determine the average number of steps that
will pass before the process leaves state 2.
(b)
Compute the probability that if the process started from state 3 it will be in state 2 for
the first time four steps later.
(c)
Find the mean first passage time to reach state 2 from state 1.
(d)
Assume that at some point the process is in state 3. Determine the mean number of
steps that will pass before the process returns to state 3.
3.
Assume the setup of problem 3 from HW #3.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Xangi
 Probability theory, Dallas

Click to edit the document details