Math 564 Homework 3. Solutions.
Problem 1. Here we systematically develop the solution of the system (11.2.4), which is the formula
for hi , that satises the recursion
hi = phi+1 + qhi1 ,
h0 = 1.
(1)
a. Show that any constant solution hi = A satises (1).
Math 564 Homework 1. Solutions.
Problem 1. Prove Proposition 0.2.2. A guide to this problem: start with the open set S = (a, b),
for example.
First assume that a > , and show that the number a has the properties that it is a lower
bound for S , and, for a
Math 564 Homework 2. Solutions.
Problem 1. Let X, Y, Z, W be independent U (0, 1) random variables. Use a Monte Carlo method
to compute E[XY 2 + eZ cos(W )]. How much computation should you do to be condent in your
answer to three decimal places? (Turn in
Part II
Discrete-time Markov chains
61
Chapter 6
Introduction to Stochastic Processes
This chapter of the book is modeled on Chapter 1 of [Nor07], but with some additional
material and a dierent structure.
Denition 6.0.6. Let (, B ) be a probability space
Bibliography
[1] Patrick Billingsley. Probability and Measure. Wiley, 1995.
[2] John P. DAngelo and Douglas B. West. Mathematical Thinking. Prentice Hall, 2000.
[3] J. R. Norris. Markov Chains. Cambridge, 2007.
[4] H. L. Royden. Real Analysis. MacMillan,
Math 564. Fall 2013. Midterm 2.
Instructions.
Take-home exam. It is legal to use any non-human resource to help in aiding you in
your solutions.
Legal aids:
Books
Articles
Any computer software
Buses, trucks
Livestock and/or feed animals
Non-legal aids
Applied Stochastic Processes Math 564
Fall 2011
Midterm Solutions
This midterm has four questions. Choose three of them. Then do them. Write on the front
of your exam booklet which three you would like me to grade.
1. For each of the following statements,
Applied Stochastic Processes Math 564
Fall 2011
Midterm October 19
This midterm has four questions. Choose three of them. Then do them. Write on the front
of your exam booklet which three you would like me to grade.
1. For each of the following statements
Applied Stochastic Processes Math 564
Fall 2011
Homework 9 Solutions
1. Let Bt be a Brownian motion, and dene
(c)
Bt
(c)
Prove that Bt
1
= Bct .
c
is also a Brownian motion.
(c)
Solution: By the denition given in class, we need to show that the increments
Applied Stochastic Processes Math 564
Fall 2011
Homework 8 Solutions
1. Consider the CTMC with state space I = cfw_1, 2, . . . , n and with nearest-neighbor interactions, i.e.
qi,i+1 > 0 for i = 1, . . . , n 1 and qi,i1 > 0 for all i = 2, . . . , n and qi
Bibliography
[Bil95] Patrick Billingsley, Probability and measure, Wiley, 1995.
[DW00] John P. DAngelo and Douglas B. West, Mathematical thinking, Prentice Hall, 2000.
[Nor07] J. R. Norris, Markov chains, Cambridge, 2007.
[Roy88] H. L. Royden, Real analys
Chapter 0
Background
0.1
Introduction
In these notes we give an overview of all of the concepts that are prerequisite for the course.
It mostly focuses on background in probability theory, but also contains some elementary
linear algebra and set theory, a
Chapter 1
Discrete-time Markov chains
1.1
Introduction to Stochastic Processes
This chapter of the book is modeled on Chapter 1 of [3], but with some additional material
and a dierent structure.
Denition 1.1.1. Let (, B ) be a probability space, I be a se
Part III
Continuous-time Markov chains
137
Chapter 12
Denition of continuous-time Markov
chain
12.1
Motivation of denition, our wish list
We would like to extend the notion of Markov chains to the continuous time domain, i.e. to
consider a family of stoch
Part I
Background
7
Chapter 1
Introduction
In these notes we give an overview of all of the concepts that are prerequisite for the course.
It mostly focuses on background in probability theory, but also contains some elementary
linear algebra and set theo
Applied Stochastic Processes Math 564
Fall 2011
Homework 7 Solutions
1. Let T be an exponential random variable with rate . Show that T is an exponential random variable
with rate /.
Solution: By denition, we know that
P(T > t) = et .
Then
P(T > t) = P(T
Applied Stochastic Processes Math 564
Fall 2011
Homework 6 Solutions
1. Consider any Markov chain dened on cfw_0, 1, . . . , N where each state talks only to its neighbors, i.e.
p0j = 0 for j 2,
p00 , p01 > 0,
pN,j = 0 for j N 2,
pN,N , pN,N 1 > 0,
and f
Applied Stochastic Processes Math 564
Fall 2011
Homework 8 Solutions
1. Consider the CTMC with state space I = cfw_1, 2, . . . , n and with nearest-neighbor interactions, i.e.
qi,i+1 > 0 for i = 1, . . . , n 1 and qi,i1 > 0 for all i = 2, . . . , n and qi
Applied Stochastic Processes Math 564
Fall 2011
Homework 7 Solutions
1. Let T be an exponential random variable with rate . Show that T is an exponential random variable
with rate /.
Solution: By denition, we know that
P(T > t) = et .
Then
P(T > t) = P(T
Applied Stochastic Processes Math 564
Fall 2011
Homework 6 Solutions
1. Consider any Markov chain dened on cfw_0, 1, . . . , N where each state talks only to its neighbors, i.e.
p0j = 0 for j 2,
p00 , p01 > 0,
pN,j = 0 for j N 2,
pN,N , pN,N 1 > 0,
and f
Applied Stochastic Processes Math 564
Fall 2011
Homework 5 Solutions
1. Show that if P is an irreducible Markov chain, then, for any 0 < < 1, the lazy version of
the chain, P = P + (1 )I , is irreducible and aperiodic. Finally, show that if is an invarian
Applied Stochastic Processes Math 564
Fall 2011
Homework 4 Solutions
1. Let Zk be independent identically distributed (i.i.d.) random variables and dene Xn =
Show that
N
k=1
Zk .
n
V (Xn ) =
V (Zk ) = nV (Z1 ).
k=1
Hint: The hard part is to show that the
Applied Stochastic Processes Math 564
Fall 2011
Homework 3 Solutions
1. Solve the recursion relation for the exit time in the Gamblers Ruin problem, i.e. show that the solution
to
ki = pki+1 + qki1 + 1
(1)
is
ki = A + B
q
p
i
+
i
,
qp
ki = A + Bi i2 ,
p =
Applied Stochastic Processes Math 564
Fall 2010
Homework 2 Solutions
1. As stated in class, prove that if cfw_xn satises the recursion relation
n=0
xn = axn1 + b,
x0 = c,
(1)
where a, b, c are real numbers, a = 1, then
c
xn =
b
1a
an +
b
.
1a
(2)
Hints.
Applied Stochastic Processes Math 564
Fall 2010
Homework 1 Solutions
1. Give examples of the following, all dened on Z:
(a) A distribution;
(b) A nite measure that is not a distribution;
(c) A -nite measure that is not nite;
(d) A measure which is not -ni
Applied Stochastic Processes Math 564
Fall 2011
Final Exam Solutions
1. Let (Xt ) be the Poisson process with rate and (Yt ) the Poisson process with rate .
(a) Show that Xt and Yt/ have the same distribution.
(b) For each k > 0, show that pk (t) := P(Xt