Summer 2007
STAT333
Summary of the continuous Markov chain
We already had a summary of the discrete Markov Chain.
Almost all of deFnitions and
results in the discrete Markov chain can be extended to the continuous Markov chain. Let
me just review some special things in continuous Markov Chain.
±or continuous Markov chain, we are interested in the following two things:
1. how long are you going to stay in state
i
, starting from state
i
?
2. which place will the process enter into if the process is going to leave the current place?
Knowing the above two things can help us to learn everything about the continuous Markov
Chain. All the answers contain in the generator or the rate matrix
R
.
♣
Rate matrix
(
R
).
(a) DeFnition:
R
= lim
h
→
0
+
P
(
h
)
−
I
h
,
where
P
(
h
)isthe
h
step transition matrix and
I
is the identity matrix.
(b) Properties:
i.
r
ii
≤
0, for
i
∈
S
(state space);
ii.
r
ij
≥
0, for
i, j
∈
S
and
i
±
=
j
;
iii.
∑
j
∈
S
r
ij
=0.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '10
 MenZhongxian
 Derivative, Probability, Markov chain, Markov models, rate matrix, Continuous Markov chain

Click to edit the document details