ORIE3510
Introduction to Engineering Stochastic Processes
Spring 2010
Section 3
Review
•
Recurrence (positive and null), transience, periodicity and ergodicity
•
Limiting and stationary distributions & longrun proportions.
Problem 4.29
We have
N
employees, with
N
large.
There are 3 job classifications and employees change job
classifications (independently) according to a DTMC with probability matrix
0
.
7
0
.
2
0
.
1
0
.
2
0
.
6
0
.
2
0
.
1
0
.
4
0
.
5
We want to know what percentage of employees are in each job classification.
Solution:
The chain is finite state and irreducible which implies that all states are recurrent. To
see why all states must be recurrent (for general number of states
M
), assume that all states are
transient. Then, since state 0 is transient, there must exist a time
T
0
after which 0 is never visited.
Likewise, for any other state
i
, there must exist a time
T
i
after which
i
is never visited. Hence,
there exists a time
T
= max
{
T
0
, . . . , T
M
}
after which no state is visited. This is a contradiction,
so all states must be recurrent.
Note, by Hw3, that
P
ij
>
0 for all
i, j
∈
S
, so we have
P
(
n
)
ij
>
0 for all
n
≥
1 and all
i, j
∈
S
. This
implies that the chain is aperiodic, and hence ergodic. Therefore, we know that the chain has a
unique stationary distribution,
π
, which is also the limiting distribution.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '09
 RESNIK
 Markov chain, longrun proportion

Click to edit the document details