5
Homework 5 - Additional Problems
5.1. (This is exercise 1.8 in Introduction to Stochastic Processes by Lawler). Consider a simple random walk on the graph below. (Recall that simple random walk on a graph is the markov chain which at each time moves to
5
Homework 5 - Additional Problems
5.1. (This is exercise 1.8 in Introduction to Stochastic Processes by Lawler). Consider a simple random walk on the graph below. (Recall that simple random walk on a graph is the markov chain which at each time moves to
2
Homework 2 - Additional Problems
2.1. Let cfw_Xn n0 be a Markov chain on a finite state space I. Define Tcov to be the cover time of the Markov chain to be the first time that the Markov chain has visited every site in I. That is, Tcov = mincfw_n 0 : i
7
Homework 7 - Additional Problems
7.1. Let X1 , X2 , . . . Xn be independent, identically distribution random variables all with distribution Exp(). Let X = maxcfw_X1 , X2 , . . . , Xn . a) Prove that X has the same distribution as Yj Exp(j) for j = 1, 2
8
Homework 8 - Additional Problems
8.1. Suppose that N = i1 (Xi ,Yi ) is a homogeneous Poisson point process on R2 with intensity . Let R = mini1 Xi2 + Yi2 be the distance of the closest point to the origin. a) Compute the density fR (t) of R. Hint: first
9
Homework 9 - Additional Problems
9.1. Let N be a non-homogeneous Poisson point process on (0, ) with density (x) = x, and let N (t) = N (0, t]) be the number of points in the interval (0, t]. a) Compute E[N (5) | N (2) = 3].
b) Compute P (N (2) = 3 | N
10
Homework 10 - Additional Problems
10.1. Consider the following modification to the Poisson janitor example (example 3.5 in the book). Suppose that the average lifetime of the lightbulbs is F = 60 and that the janitor comes to check on the bulb accordin
11
Homework 11 - Additional Problems
11.1. In this problem we will analyze the length of busy periods for an M/G/1 queue. That is the arrivals are according to a Poisson process with rate , and the service times are i.i.d. with some general distribution (
12
Homework 12 - Additional Problems
12.1. Consider the continuous time Ehrenfest process Xt with N balls (that is, each ball switches between the two urns at rate 1 independently of all the other balls).
a) Compute the jump rates q(i, j) for this Markov
13
Homework 13 - Additional Problems
13.1. In the standard branching process model individuals reproduce offspring at rate and die at rate (see Example 4.4 in the book). Consider the following modification where there the population can also grow by immig
14
Homework 14 - Additional Problems
14.1. Let Xt be a branching process with birth rate and death rate (see Example 4.4 in the book). a) Let M2,n (t) = E[Xt2 | X0 = n]. Use the Kolmogorov forward equations to derive a system of differential equations for
1
4.2.
Homework 12 - Book Problems
a) The stationary distribution is = (2/5, 1/5, 3/10, 1/10). b) Customers arrive according to a Poisson process with rate 2. By the PASTA principle, the fraction of them that enter a store with at least one computer for s
6
Homework 6 - Additional Problems
6.1. Consider a random walk on the following graph.
1 2 4 5 7 8 9 3 6 10
If the random walk starts at the top of the triangle (state 1), what is the distribution when it exits the bottom of the triangle. That is, let T =
4
4.1.
Homework 4 - Additional Problems
a) Let Xn be a Markov chain on I = cfw_1, 2, 3, 4 with transition probability matrix 1-a a 0 0 b 1-b 0 0 , with 0 < a, b, c, d < 1. p= 0 0 1-c c 0 0 d 1-d Compute limn pn (x, y) for any x, y I. Note that this limit
3
Homework 3 - Additional Problems
3.1. There is a typo on page 20 of the book, just after the bold heading Using the TI83 calculator is easier. The book says .we write (1.10) in matrix form as -.2 .1 1 (1 , 2 , 3 ) .2 -.5 1 = (0, 0, 1) .3 .3 1 If we let
2
Homework 2 - Additional Problems
2.1. Let cfw_Xn n0 be a Markov chain on a finite state space I. Define Tcov to be the cover time of the Markov chain to be the first time that the Markov chain has visited every site in I. That is, Tcov = mincfw_n 0 : i
3
Homework 3 - Additional Problems
3.1. There is a typo on page 20 of the book, just after the bold heading Using the TI83 calculator is easier. The book says .we write (1.10) in matrix form as -.2 .1 1 (1 , 2 , 3 ) .2 -.5 1 = (0, 0, 1) .3 .3 1 If we let
10
Homework 10 - Additional Problems
10.1. Consider the following modification to the Poisson janitor example (example 3.5 in the book). Suppose that the average lifetime of the lightbulbs is F = 60 and that the janitor comes to check on the bulb accordin
13
Homework 13 - Additional Problems
13.1. In the standard branching process model individuals reproduce offspring at rate and die at rate (see Example 4.4 in the book). Consider the following modification where there the population can also grow by immig
Stationary distributions of continuous time Markov chains
Jonathon Peterson April 13, 2012
The following are some notes containing the statement and proof of some theorems I covered in class regarding explicit formulas for the stationary distribution and
1
Stopping Times
In class we defined a stopping time in the following way. T is a stopping time for the Markov chain cfw_Xn n0 if for any n the event cfw_T = n can be expressed in terms of X0 , X1 , . . . , Xn . Lemma 1. There are two other equivalent def
1
Homework 1 - Additional Problems
1.1. Let cfw_Xn n0 be a random walk on Z with momentum that we discussed in class. That is, let (1/2, 1) be fixed and let X0 = 0, and P (Xn+1 - Xn = Xn - Xn-1 ) = , P (Xn+1 - Xn = -(Xn - Xn-1 ) = 1 - . P (X1 = 1) = P (X1
4.7 Time reversibility
Time reversed Markov chain (discrete time): Let Xn be a discrete time irreducible Markov
chain under stationary distribution j . Fix a larte time N > 0. The time reversed process
Xn obtained by time reversing Xn at time T is defined