STAT 580 Assignment 1 solutions
1. You have three pennies in a row, and at each time you turn over a randomly selected
penny.
(a) Describe a Markov chain that models this process. Explicitly dene the state space
S, and Xn the state at time n. Write down t
STAT 580 Final Exam solutions
1. Prove or disprove: Mt := Yt2 E(Yt2 ) is a martingale, where Xt is a Poisson process
with rate , and Yt = Xt t.
Solution: Since E(Yt2 ) is the variance of a Poisson random variable with mean t, we
also have E(Yt2 ) = t. We
STAT 580 Assignment 5 solutions
Notation:
In this assignment, (Sn )n0 is a simple symmetric random walk on the integers starting at zero, and (Wt )t0 is a standard one-dimensional Brownian motion starting
at zero.
1. Let (Xt ) be a continuous Markov chain
STAT 580 Assignment 1 solutions
1. Let (Xn ) be a independent sequence of fair tosses of a 6 sided die. For n 1, dene
Mn to be the maximum of the values X1 , X2 , . . . , Xn .
(a) Assuming that (Mn ) has the Markov property, nd its state space S and trans
STAT 580 Assignment 2 solutions
1. Do the following for each of the three Markov chains in questions 1, 2, and 3 of the
previous assignment.
(a) Divide the state space into positive and null classes.
(b) Determine all of the invariant probability vectors
STAT 580 Assignment 3 solutions
1. Find the value of the start position for the nancial opportunity pictured below. The
underlying Markov chain is a random walk on the graph, except for the state labelled
absorbing.
(a) Calculate the value of the start po
STAT 580 Assignment 1 solutions
1. Show that if (Xn ) is a Markov chain with transition matrix P , then (X2n ) is a
n=0
n=0
Markov chain with transition matrix P 2 .
Solution:
Using Theorem B-2 on Markov chain probabilities we compute
P (X2n = in j X2n2 =
STAT 580 Term Exam I solutions
1. Recall the frogs random hop from assignment 2.
(a) Starting in the south-west corner, what is the probability that the frog is in the
center square at time n = 1000000?
(b) Find the invariant probability for this Markov c
STAT 580 Term Exam II solutions
1. The graph below represents the payo function for a coin ipping game. You begin
with ve dollars and a fair coin. At each time, you can keep the current payo or decide
to ip the coin. If you choose to ip the coin, you move
STAT 580 Term Exam I solutions
1. Here is the P matrix for a Markov chain with state space S = f1, 2, 3g.
1
2
3
1
1
0
0
P = 2 1/2 1/2 0
3 1/3 1/3 1/3
Find all the communicating classes, and decide which of the states are recurrent, transient,
absorbing.
STAT 580 Assignment 1 solutions
1. Imagine a three state Markov chain with transition probabilities as shown in the
diagram.
B
1/2
3/4
1/4
1/3
1/2
A
C
2/3
(a) There are ve ways (with non-zero probability) to begin at state C and be in state B
after four t
STAT 580 Term Exam I solutions
1. For the Markov chain with the following transition matrix
1
0
0
P = 1/2 1/2 0 ,
1/3 1/3 1/3
calculate the average time to absorption, starting at state .
Solution: Dene the function f (x) = Ex (V ). This function satises
STAT 580 Term Exam I solutions
1. This diagram illustrates all the transitions between distinct states having non-zero
probability. Find all the communicating classes of the Markov chain, and determine
whether each class is recurrent or transient.
d
c
e
b
STAT 580 Assignment 4 solutions
1. A fair die is rolled with outcome X. The sample space is = f1 , 2 , 3 , 4 , 5 , 6 g,
where all outcomes are equally likely and X(i ) = i for 1 i 6.
Hint 1: Let P represent the information contained in the hint: I will te