Math341: Stochastic Modeling(No.9)
By Lin Yuanyuan
April. 13, 2011
1. (Poisson Processes(continued) Key points.
(1). Denition,
(2). Properties of Wn and Sk .
Remark:
(1). cfw_Wr t = cfw_X (t) r; cfw_Wr = t = cfw_X (t) = r, X (t) = r 1; cfw_Wr <
t = cfw_X
Math341: Stochastic Modeling(No.11)
By Lin Yuanyuan
Apr. 27, 2011
Example 1. Let T be the time to extinction in the linear death process with
parameters X (0) = N and .
(a) Using the sojourn time viewpoint, show that
[
]
11
1
E (T ) =
+
+ + 1 .
n N 1
(b)
Math341: Stochastic Modeling(No.4)
(Branching Processes)
By Lin Yuanyuan
Mar. 9, 2011
1. Key points.
Xn : the size of the n-th generation.
(n)
(n)
i : # of the ospring of i-th individual of the n-th generation. In general, i
independent, identically distr
Math341: Stochastic Modeling(No.3)
By Lin Yuanyuan
Mar. 2, 2011
1 (First-step Analysis) Key points.
Suppose cfw_Xn , n 0 is a Markov chain with state space cfw_0, 1, ., r 1, r, r +1, . . . N
where cfw_r, ., N are absorbing states and cfw_0, 1, ., r 1 ar
Math341: Stochastic Modeling
By Lin Yuanyuan
Feb. 16, 2011
Review key points:
(1) Markov Process, Markov Chain (Markov process with discrete state space),
Discrete time Markov Chain, Continuous time Markov Chain.
Check discrete time Markov Chain:
P (Xn+1
Math341: Stochastic Modeling(No.13)
By Lin Yuanyuan
May 11, 2011
Key Points
(i). Elementary renewal theorem
M (t) =
t
+ o(t),
as t .
(ii). Rened renewal theorem
M (t) =
t
2 2
+
+ o(1),
22
as t .
(iii). C.L.T. for N (t)
N (t) t/
N (0, 1).
t 2 /3
(iv). Li
Math341: Stochastic Modeling(No.10)
By Lin Yuanyuan
Apr. 20, 2011
Chapter 6: Continuous time Markov Chain
1. Key points.
(1). Pure birth processes.
Denition:(three postulates)
Xt is a CTMC with nondecreasing path, X0 = 0,
P 1).
P(Xt+h Xt = 1|Xt = k ) = k
Math341: Stochastic Modeling(No.8)
By Lin Yuanyuan
Apr.6, 2011
1. (Poisson Processes) Key points.
(1). Denition,
() Independent increment, X (t1 ) X (t0 ) X (t2 ) X (t1 ) if t0 < t1 < t2 .
() Poisson marginal distribution, X (t + s) X (s) (t), X (0) = 0.
Math341: Stochastic Modeling(No.7)
By Lin Yuanyuan
Mar. 30, 2011
1. (The classication of states) Key points.
(1). Check whether or not a M.C. is irreducible by denition i j for all i, j .
(k )
Communicate: i j if and only if (i) there exists some k, l > 0
Math341: Stochastic Modeling(No.6)
By Lin Yuanyuan
Mar. 23, 2011
1. (The long run behavior of M.C.) Key points.
(1). Two methods to check regularity.
Method 1: Denition.
If there exists a k > 0 such that all the elements in Pk are positive, then P is
reg
Math341: Stochastic Modeling(No.5)
By Lin Yuanyuan
Mar. 16, 2011
1. Consider the Markov chain whose transition probability matrix is given by
0
1
2
3
0
1
0
0
0
P=
1
0.1 0.2
0.5
0.2
2
0.1 0.2
0.6
0.1
0
1
3
0
0
Starting in state 1, determine the mean time t