Introduction to
Introduction
Stochastic processes
Week 1: Review on basic tools
What is probability?
Probability is a concept trying to quantify
the likelihood for events which are not
certain to happen.
Probability cannot be simply described by
a sente
4.3 Birth and Death Process
An obvious generalization of the pure
birth and the pure death process
discussed in Section 4.1 and 4.2 is to
permit X(t) both to increase and to
decrease. Thus it at time t the process is
in state n it may, after a random sojo
Spatial Poisson Processes
Chapter V, Section 5 of the text book
1
Notations
Let S be a set of n-dimensional space
and A be a family of subsets of S.
A point process in S is a stochastic
process N(A) indexed by the sets A in
A and having the set of nonne
Spatial Poisson Processes
Chapter V, Section 5 of the text book
1
Notations
Let S be a set of n-dimensional space
and A be a family of subsets of S.
A point process in S is a stochastic
process N(A) indexed by the sets A in
A and having the set of nonne
Distributions associated with
the Poisson Process
Chapter V, Section 3 of the text book
1
Definitions
Wn: the waiting time of occurrence of
the n-th event. We often set that W0
= 0.
Sn: the sojourn time measures the
duration that the Poisson process
soj
Poisson Distribution and the
Poisson Process
Chapter V of text book
1
Poisson Distribution
The Poisson distribution with parameter > 0 is
given by
e
Pr X k p k
k!
k
Let X be a Poisson r.v. Then, E[X]= .
Moreover, E[X2] = 2+ and
Var(X) = .
Note: Poisson
The Classification of
States
Chapter IV, Section 3 of the
text book
1
Examples of irregular Markov
Example 1: identity matrix
1 0
1 0
n
P
P 0 1
0 1
Limiting probability exists but depends on initial state
Example 2:
0 1
1 0 2 n 1 0 1
2n
P
P 0 1 , P
1
Week 5
Long Run Behavior of
Markov Chains
Chapter IV of text book
1
Some Pn are convergent
It is not hard to verify that (using mathematical
induction) for a transition matrix
1 a
P
b
a
where
1b
0 a,b 1
we have
P
n
b
1
ab b
Note
lim
n
that
P
n
a
a
1
ab
a
First Step
Analysis of
Markov Chain
Chapter 3.4 of textbook
1
Simple First Step Analysis
The Markov Chain cfw_Xn with state
space cfw_0,1,2 has the Markov matrix:
1
P
0
0
0
0
1
2
Questions
Let the time of absorption be
T minn 0 | X n 0 or X n 2
Find
u
Markov Chain
Chapter 3 in the text
book
1
Markov chain: Definition
A Markov process cfw_Xt is a stochastic process
with the property that, givn the value Xt, the
value of Xs for s>t are not influenced by the
value of Xu for u<t. A discrete-time Markov ch
Week 2
Week
General definition of
Stochastic process, brief
introduction to
Martingales
and Branching processes
Stochastic Processes
A stochastic Processes is a family
of random variables X(t), or Xt,
where t belongs to an index set T.
Elements of a sto
Renewal Phenomena
Chapter 5
1
5.1 Definition of a renewal Process
and Related Concepts
Renewal theory began with the study of
stochastic systems whose evolution through time
was interspersed with renewals or regeneration
times when, in a statistical sense