Bernardo DAuria
Stochastic Processes - 2010/11
Notes
March 01st , 2011
Exercise 2.10 pag 90 in [1]
Buses arrive at a certain stop according to a Poisson process with rate . If you take the bus from that stop
then it takes R, measured from the time at whic
Homework 5 (Stats 620, Fall 2013)
Due Tuesday Oct 22, in class
1. Prove that if the number of state is n, and if state j is accessible from state i, then it is
accessible in n or fewer steps.
k
Solution: j is accessible from i if, for some k 0, Pij > 0. N
Homework 2 (Stats 620, Fall 2013)
Due *Tuesday Sept 24*, in class
1. Let cfw_N (t), t 0 be a Poisson process with rate . Calculate E N (t)N (t + s) .
Comment: Please state carefully where you make use of basic properties of Poisson processes,
such as stat
Homework 4 (Stats 620, Fall 2013)
Due Tuesday Oct 8, in class
1. Let A(t) and Y (t) denote respectively the age and excess at t. Find:
(a) Pcfw_Y (t) > x|A(t) = s.
(b) Pcfw_Y (t) > x|A(t + x/2) = s.
(c) Pcfw_Y (t) > x|A(t + x) > s for a Poisson process.
(
IEOR 6711: Stochastic Models I, Professor Whitt
Solutions to Homework Assignment 6
Problem 3.1 (a) Yes since it is a contrapositive statement of (3.2.1).
(b) No. Note that we can have Xn+1 = 0, so that Sn = Sn+k = 0 for some k 1. Hence
We can have Sn = t,
Homework 1 (Stat 620, Fall 2013)
Due Thu Sept 12, in class
1. (a) Let N denote a nonnegative integer-valued random variable. Show that
Pcfw_N k =
E[N ] =
k=1
Pcfw_N > k .
k=0
(b) In general show that if X is nonnegative with distribution F , then
E[X ] =
Homework 1 (Stat 620, Fall 2013)
Due Thu Sept 12, in class
1. (a) Let N denote a nonnegative integer-valued random variable. Show that
Pcfw_N k =
E[N ] =
k=1
Pcfw_N > k .
k=0
(b) In general show that if X is nonnegative with distribution F , then
E[X ] =
Homework 7. Due by 5pm on Wednesday 10/16.
Collaborative research & Human participants and animal subjects
Most statisticians do not have to worry directly about running large scientic research groups
and dealing with the bureaucracy of ethical data colle
Homework 3 (Stats 620, Fall 2013)
Due Tuesday October 1, in class
1. Prove the renewal equation
t
m(t x) dF (x)
m(t) = F (t) +
0
Hint: One approach is to use the identity E[X ] = E E[X |Y ] for appropriate choices of X
and Y .
Solution:
m(t) = E(N (t)
= E
Homework 4 (Stats 620, Fall 2013)
Due Tuesday Oct 8, in class
1. Let A(t) and Y (t) denote respectively the age and excess at t. Find:
(a) Pcfw_Y (t) > x|A(t) = s.
(b) Pcfw_Y (t) > x|A(t + x/2) = s.
(c) Pcfw_Y (t) > x|A(t + x) > s for a Poisson process.
(
Homework 2 (Stats 620, Fall 2013)
Due *Tuesday Sept 24*, in class
1. Let cfw_N (t), t 0 be a Poisson process with rate . Calculate E N (t)N (t + s) .
Comment: Please state carefully where you make use of basic properties of Poisson processes,
such as stat
4. Markov Chains
A discrete time process cfw_Xn , n = 0, 1, 2, . . .
with discrete state space Xn cfw_0, 1, 2, . . . is a
Markov chain if it has the Markov property:
P[Xn+1 = j |Xn = i, Xn1 = in1 , . . . , X0 = i0 ]
= P[Xn+1 = j |Xn = i]
In words, the p
Homework 9 (Stats 620, Fall 2013)
Due Tuesday Dec 3, in class
1. Consider successive ips of a coin having probability p of landing heads. Use a martingale
argument to compute the expected number of ips until the following sequences appear:
(a) HHTTHHT
(b)
Statistics 620
Final exam, Fall 2012
Name:
UMID #:
There are 5 questions, each worth 10 points.
You are allowed two single-sided sheets of notes.
You are not allowed to make use of a calculator, or any other electronic device,
during the exam.
Credit
Homework 8 (Stats 620, Winter 2015)
Due Tuesday March 31, in class
1. Let cfw_N (t) be a Poisson processwith rate . Since cfw_N (t) is also a continuous time Markov
chain, it can also be defined in terms of its transition rates qij and rates i of leaving
Stats 620 Final Exam
The exam will be in the usual classroom (271
Dennison), on Thursday April 23, 1:30-3:30.
Four questions will cover the material since the
midterm, and one question will be based on
material in the first half of the course.
You may b
Statistics 620
Midterm exam, Fall 2012
Name:
UMID #:
Midterm Exam
There are 4 questions, each worth 10 points.
You are allowed a single-sided sheet of notes.
You are not allowed to make use of a calculator, or any other electronic device,
during the ex
Statistics 620
Final exam, Fall 2012
1. This question asks you to confirm the validity of a variation on the Hastings-Metropolis algorithm. Let Q = cfw_qij be a symmetric irreducible transition probability matrix (i.e., Q specifies the
one-step transitio
7. Brownian Motion & Diffusion Processes
A continuous time stochastic process with
(almost surely) continuous sample paths which
has the Markov property is called a diffusion.
almost surely means with probability 1,
and we usually assume all sample path
Stats 620: Applied Probability
and Stochastic Models
1. Review of Probability
An event E is a subset of a sample space S.
The probability of E is written as P(E).
Formally, P(E) may be undefined for certain
non-measurable events. Here, we assume P(E)
i
Statistics 620
Final exam, Fall 2013
1. Suppose that traffic on a road follows a Poisson process with rate cars per minute. A chicken
needs a gap of length at least c minutes in the traffic to cross the road. To compute the time the
chicken will have to w
Statistics 620
Final exam, Fall 2013
Name:
UMID #:
There are 5 questions, each worth 10 points.
You are allowed two single-sided sheets of notes.
You are not allowed to make use of a calculator, or any other electronic device,
during the exam.
Credit
The Duality Principle for Random Walks
(This topic will not be in the final exam)
Pn
For Sn = i=1 Xi with X1 , X2 , . . . iid, we note
that (X1 , . . . , Xn ) has the same joint distribution
as (Xn , . . . , X1 )
This obvious property has surprising
con
3. Renewal Theory
Definition 3 of the Poisson process can be
generalized: Let X1 , X2 , . . . , iid F (x) be
non-negative interarrival times. Set
Pn
Sn = i=1 Xi and N (t) = max cfw_n : Sn t.
Then cfw_N (t) is a renewal process.
Set = E[Xn ]. We assume >
6. Martingales
For casino gamblers, a martingale is a betting
strategy where (at even odds) the stake doubled
each time the player loses. Players follow this
strategy because, since they will eventually win,
they argue they are guaranteed to make money!
4. Markov Chains
A discrete time process cfw_Xn , n = 0, 1, 2, . . .
with discrete state space Xn cfw_0, 1, 2, . . . is a
Markov chain if it has the Markov property:
P[Xn+1 = j|Xn = i, Xn1 = in1 , . . . , X0 = i0 ]
= P[Xn+1 = j|Xn = i]
In words, the pas
Statistics 620
Midterm exam, Fall 2013
1. Richard catches trout according to a Poisson process with rate 0.1 minute1 . Suppose that the
trout weigh an average of 4 pounds with a standard deviation of 2 pounds. Find expressions for
the mean and standard de
Statistics 620
Midterm exam, Winter 2015
1. Let X and Y be independent, non-negative, continuous random variables with densities fX (x)
and fY (y), and cumulative distribution functions FX (x) and FY (y). Define the failure rate function
for X to be
P[x <
2. The Poisson Process
A counting process cfw_N (t), t 0 is a Poisson
process with rate if . . .
Definition 1.
(i) N (0) = 0,
(ii) N (t) has independent increments,
(iii) N (t) N (s) Poisson (t s) for s < t.
This can be shown to be equivalent to
Definitio
Statistics 620
Midterm exam, Fall 2013
Name:
UMID #:
Midterm Exam
There are 4 questions, each worth 10 points.
You are allowed a single-sided sheet of notes.
You are not allowed to make use of a calculator, or any other electronic device,
during the ex