CS/EE 147
HW 2: Practice with DTMCs
Guru: Lina
Assigned: 04/06/10
Due: 04/16/10, Raga’s mailbox, 1pm
We encourage you to discuss these problems with others, but you need to write up the actual solutions alone.
At the top of your homework sheet, list all the people with whom you discussed. Crediting help from other
classmates will not take away any credit from you. Start early and come to office hours with your questions!
Note before you begin
This is a long and probably hard problem set. Start early – do not put it off until the last minute!
1
DTMC Warmup [20 points]
(a) For each of the following state whether the chain is irreducible, aperiodic, and/or positive recurrent.
(i)
1
4
1
4
1
2
0
0
1
1
0
0
(ii)
0
1
0
0
1
0
1
0
0
(iii)
1
3
0
2
3
1
4
3
4
0
0
0
1
(b) Consider the following probability transition matrices
P
(1)
=
0
2
/
3
0
1
/
3
1
/
3
0
2
/
3
0
0
1
/
3
0
2
/
3
2
/
3
0
1
/
3
0
P
(2)
=
1
/
3
2
/
3
0
0
1
/
3
0
2
/
3
0
0
1
/
3
0
2
/
3
0
0
1
/
3
2
/
3
For each of these,
(i) Draw the corresponding Markov chains.
(ii) Solve for the stationary probabilities. First, try to use the local balance equations, and if they don’t work
use the balance equations.
(iii) For those chain(s) where the local balance equations worked, explain why it makes sense that for all states
i
,
j
in the chain, the rate of transitions from
i
to
j
should equal the rate of transitions from
j
to
i
.
2
Finite state DTMCs [15 points]
(a) Prove the following two class property theorems.
(i) Nullrecurrence is a class property, i.e., if
i
is nullrecurrent and
i
communicates with
j
then
j
is
nullrecurrent.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 Adam, Markov chain, Random walk

Click to edit the document details