Unformatted text preview: Probability and Statistics with Reliability,
Queuing and Computer Science Applications
Second edition
by K.S. Trivedi
PublisherJohn Wiley & Sons Chapter 8 (Part 6) :Continuous Time Markov Chains
Solution Techniques
Dept. of Electrical & Computer engineering
Duke University
Email:kst@ee.duke.edu
URL: www.ee.duke.edu/~kst
Copyright © 2003 by K.S. Trivedi 1 CTMC Solutions
• Transient Solutions
– Solving Kolmogorov differential equation • SteadyState
– Solving the linear system of equations πQ = 0 • Closedform solutions
– Possible for very small CTMCs or highly structured
CTMCs • Numerical Solutions
– Most other cases
Copyright © 2003 by K.S. Trivedi 2 Outline of This Part of Chapter 8
• Methods for SteadyState Analysis
– Power method
– SOR • Methods for Transient Analysis
– Fully symbolic method
– Numerical Methods
– Uniformization
Copyright © 2003 by K.S. Trivedi 3 Methods for SteadyState
Analysis Copyright © 2003 by K.S. Trivedi 4 Methods for SteadyState Analysis
• Direct methods (e.g., Gaussian elimination)
– Fullmatrix
– Memoryintensive
– Upper bound on the time • Iterative methods (e.g., power method, SOR, GaussSeidel)
– Sparse storage methods and sparsity preserving
– Less storage and less computing resources needed
– Long time to converge • Q matrix for practical CTMC problems is highly sparse.
Copyright © 2003 by K.S. Trivedi 5 Power Method
• Rewrite Eq. (8.23) as
where
• Note
is a transition probability
matrix of a DTMC.
• Iteration:
• Initial value: Copyright © 2003 by K.S. Trivedi 6 Example
• Consider the following twostate
availability model • Assume repair rate µ > failure rate λ
• Choose q=max{µ, λ} = µ
Copyright © 2003 by K.S. Trivedi 7 Example – cont’d
• The matrix is
• After n iterations,
• As n approaches infinity, Copyright © 2003 by K.S. Trivedi 8 Successive Overrelaxation (SOR)
• Start with an initial guess
• Iterate until some criteria for convergence are
satisfied: • Where
is the solution vector at the kth iteration,
L is a lower triangular matrix, U is an upper
triangular matrix, and D is a diagonal matrix.
• GaussSeidel is a special case of SOR with
Copyright © 2003 by K.S. Trivedi 9 Example
• Consider the CTMC in (a) • w/ matrix
Copyright © 2003 by K.S. Trivedi 10 Example – cont’d
• This CTMC will not converge using the
GuassSeidel or SOR.
• The iteration will converge using
underrelaxation, i.e.,
• Renumbering the states can also cause the
iteration to converge for GaussSeidel and
SOR. See Fig. (b).
Copyright © 2003 by K.S. Trivedi 11 Methods for Transient Analysis Copyright © 2003 by K.S. Trivedi 12 Fully Symbolic Method
• Taking the Laplace transform on both sides of the Kolmogorov
differential equation and rearranging the terms: • The transient state probability is obtained by computing the inverse
Laplace transform of
• Solutions obtained will be closedform and full symbolic in both
the system parameters and time t.
• However, in general, computing the inverse Laplace transform is
extremely difficult.
• Semisymbolic solutions
– Assuming entries in the Q matrix are all numerical, but final solution
being a symbolic function in time parameter t.
– Simpler than the full symbolic method.
– Implemented in the SHARPE software package.
Copyright © 2003 by K.S. Trivedi 13 Numerical Methods
• Discretization methods are standard to solve ODEs such as
Kolmogorov differential equations.
– Discretize the time interval into a finite number of subintervals and
compute the solution step by step. • Implicit discretization methods
– More stable and high solution accuracy
– For stiff Markov chains
– For example: TRBDF2 and the implicit RungeKutta • Explicit discretization methods
– For nonstiff Markov chains
– For example: RungeKutta Copyright © 2003 by K.S. Trivedi 14 Uniformization
• ODE solvers are for the evolution profile of the state
probabilities.
– Require a very small step to obtain accurate results. • Uniformization method
– Also known as randomization method or Jensen’s method
– Starts with the formal solution
where the matrix exponential is defined by the infinite
series: Copyright © 2003 by K.S. Trivedi 15 Practical problems using this approach
1. Q has both negative and positive entries and
hence the computation has both additions and
subtractions
– poor numerical behavior 2. Raising the matrix Q to its powers is costly and
fills in zeros in the matrix
– In practice, Q is very large yet sparse. 3. The infinite series needs to be truncated. Copyright © 2003 by K.S. Trivedi 16 Solution to Problem 1
• Use an integrating factor
• Let
• Kolmogorov differential equations becomes
• We have
• Hence
• DTMC matrix has no negative entries. Copyright © 2003 by K.S. Trivedi 17 Solution to Problem 2
• Rewrite last equation as • Problem 2 (raising matrix Q to its powers) is
avoided. Copyright © 2003 by K.S. Trivedi 18 Solution to Problem 3
• To solve problem 3, either left or right truncate
the infinite series: • The value of l and r can be determined from the
specified truncation error tolerance by: Copyright © 2003 by K.S. Trivedi 19 Example
• Consider the 2state homogenous CTMC • The Kolmogorov differential equations are: Copyright © 2003 by K.S. Trivedi 20 Example – cont’d
• Note
we obtain
•
(1)
• Assume µ > λ, let q= µ. Now define
• Differentiate both sides Copyright © 2003 by K.S. Trivedi 21 Example – cont’d
• From Eq. 1, we obtain
• Solving the equation,
• We have
• Given initial condition
Copyright © 2003 by K.S. Trivedi 22 Example – cont’d
• So we have • Similarly, Copyright © 2003 by K.S. Trivedi 23 ...
View
Full
Document
This note was uploaded on 04/08/2010 for the course COMPUTER E 409232 taught by Professor Mohammadabdolahiazgomiph.d during the Spring '10 term at Islamic University.
 Spring '10
 MohammadAbdolahiAzgomiPh.D

Click to edit the document details