4.4 The Kolmogoroff Differential Equations.
Like the Chapman-Kolmogoroff equations, the Kolmogoroff differential equations are simplest to write
down in matrix form. They say that the transition matrices P(t) satisfy
(1)
= P(t)R
(2)
= RP(t)
for all t 0. H
4.2 Markov Processes.
Let's summarize the assumptions from section 4.1 that we will be making in the rest of this chapter. We
have a system that at any non-negative time t can be in any of N states numbered i = 1, 2, ., N. Let
X(t) = state at time t
(1)
T
5.2 Multiple Server Queues.
The previous section considered single server queues. In this section we extend the analysis to queues
with c servers. This is often called an M/M/c queue. We let
= arrival rate of new customers
= service rate = rate at which
4
Continuous Time Markov Processes
4.1 Basic Principles.
Many systems in the real world are more naturally modeled in continuous time as opposed to discrete
time. For example, if we are modeling the number of customers waiting for service at a bank, then
3
Continuous Probability
3.1 Probability Density Functions.
So far we have been considering examples where the outcomes form a finite or countably infinite set. In
many situations it is more natural to model a situation where the outcomes could be any rea
3.2 Joint Probability Distributions.
Recall that in section 1.9 we discussed two ways to describe two or more discrete random variables.
We could either use the joint probability mass function or conditional probability mass functions. We
have two similar
4.7 Revenues and Costs.
In the previous sections we saw how to compute the transition matrices P(t) = etR where R is the
generator matrix and we saw that P(t) approaches a limiting matrix P() as t tends to . The rows of
P() are often identical and their v
4.6 Steady State Probablilites.
In the previous section we saw how to compute the transition matrices P(t) = etR where R is the
generator matrix. We saw that each element of P(t) was a constant plus a sum of multiples of etj where
the j are the eigenvalue
3.3 Sums and Other Functions of Random Variables.
We often form new random variables by performing the usual mathematical operations to existing
random variables. In this case we may be interested in the mass or density function of the new random
variable
4.5 Solving Kolmogoroff's Equation The Matrix Exponential.
In the last section we discussed the Kolmogoroff differential equations which say that the transition
matrices P(t) satisfy
(1)
= P(t)R
(2)
= RP(t)
for all t 0. Here R is the generator matrix disc