chap8_p6 - Probability and Statistics with Reliability...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Probability and Statistics with Reliability, Queuing and Computer Science Applications Second edition by K.S. Trivedi Publisher-John Wiley & Sons Chapter 8 (Part 6) :Continuous Time Markov Chains Solution Techniques Dept. of Electrical & Computer engineering Duke University URL: Copyright © 2003 by K.S. Trivedi 1 CTMC Solutions • Transient Solutions – Solving Kolmogorov differential equation • Steady-State – Solving the linear system of equations πQ = 0 • Closed-form solutions – Possible for very small CTMCs or highly structured CTMCs • Numerical Solutions – Most other cases Copyright © 2003 by K.S. Trivedi 2 Outline of This Part of Chapter 8 • Methods for Steady-State Analysis – Power method – SOR • Methods for Transient Analysis – Fully symbolic method – Numerical Methods – Uniformization Copyright © 2003 by K.S. Trivedi 3 Methods for Steady-State Analysis Copyright © 2003 by K.S. Trivedi 4 Methods for Steady-State Analysis • Direct methods (e.g., Gaussian elimination) – Full-matrix – Memory-intensive – Upper bound on the time • Iterative methods (e.g., power method, SOR, Gauss-Seidel) – Sparse storage methods and sparsity preserving – Less storage and less computing resources needed – Long time to converge • Q matrix for practical CTMC problems is highly sparse. Copyright © 2003 by K.S. Trivedi 5 Power Method • Rewrite Eq. (8.23) as where • Note is a transition probability matrix of a DTMC. • Iteration: • Initial value: Copyright © 2003 by K.S. Trivedi 6 Example • Consider the following two-state availability model • Assume repair rate µ > failure rate λ • Choose q=max{µ, λ} = µ Copyright © 2003 by K.S. Trivedi 7 Example – cont’d • The matrix is • After n iterations, • As n approaches infinity, Copyright © 2003 by K.S. Trivedi 8 Successive Overrelaxation (SOR) • Start with an initial guess • Iterate until some criteria for convergence are satisfied: • Where is the solution vector at the kth iteration, L is a lower triangular matrix, U is an upper triangular matrix, and D is a diagonal matrix. • Gauss-Seidel is a special case of SOR with Copyright © 2003 by K.S. Trivedi 9 Example • Consider the CTMC in (a) • w/ matrix Copyright © 2003 by K.S. Trivedi 10 Example – cont’d • This CTMC will not converge using the Guass-Seidel or SOR. • The iteration will converge using underrelaxation, i.e., • Renumbering the states can also cause the iteration to converge for Gauss-Seidel and SOR. See Fig. (b). Copyright © 2003 by K.S. Trivedi 11 Methods for Transient Analysis Copyright © 2003 by K.S. Trivedi 12 Fully Symbolic Method • Taking the Laplace transform on both sides of the Kolmogorov differential equation and rearranging the terms: • The transient state probability is obtained by computing the inverse Laplace transform of • Solutions obtained will be closed-form and full symbolic in both the system parameters and time t. • However, in general, computing the inverse Laplace transform is extremely difficult. • Semi-symbolic solutions – Assuming entries in the Q matrix are all numerical, but final solution being a symbolic function in time parameter t. – Simpler than the full symbolic method. – Implemented in the SHARPE software package. Copyright © 2003 by K.S. Trivedi 13 Numerical Methods • Discretization methods are standard to solve ODEs such as Kolmogorov differential equations. – Discretize the time interval into a finite number of subintervals and compute the solution step by step. • Implicit discretization methods – More stable and high solution accuracy – For stiff Markov chains – For example: TR-BDF2 and the implicit Runge-Kutta • Explicit discretization methods – For nonstiff Markov chains – For example: Runge-Kutta Copyright © 2003 by K.S. Trivedi 14 Uniformization • ODE solvers are for the evolution profile of the state probabilities. – Require a very small step to obtain accurate results. • Uniformization method – Also known as randomization method or Jensen’s method – Starts with the formal solution where the matrix exponential is defined by the infinite series: Copyright © 2003 by K.S. Trivedi 15 Practical problems using this approach 1. Q has both negative and positive entries and hence the computation has both additions and subtractions – poor numerical behavior 2. Raising the matrix Q to its powers is costly and fills in zeros in the matrix – In practice, Q is very large yet sparse. 3. The infinite series needs to be truncated. Copyright © 2003 by K.S. Trivedi 16 Solution to Problem 1 • Use an integrating factor • Let • Kolmogorov differential equations becomes • We have • Hence • DTMC matrix has no negative entries. Copyright © 2003 by K.S. Trivedi 17 Solution to Problem 2 • Rewrite last equation as • Problem 2 (raising matrix Q to its powers) is avoided. Copyright © 2003 by K.S. Trivedi 18 Solution to Problem 3 • To solve problem 3, either left or right truncate the infinite series: • The value of l and r can be determined from the specified truncation error tolerance by: Copyright © 2003 by K.S. Trivedi 19 Example • Consider the 2-state homogenous CTMC • The Kolmogorov differential equations are: Copyright © 2003 by K.S. Trivedi 20 Example – cont’d • Note we obtain • (1) • Assume µ > λ, let q= µ. Now define • Differentiate both sides Copyright © 2003 by K.S. Trivedi 21 Example – cont’d • From Eq. 1, we obtain • Solving the equation, • We have • Given initial condition Copyright © 2003 by K.S. Trivedi 22 Example – cont’d • So we have • Similarly, Copyright © 2003 by K.S. Trivedi 23 ...
View Full Document

This note was uploaded on 04/08/2010 for the course COMPUTER E 409232 taught by Professor Mohammadabdolahiazgomiph.d during the Spring '10 term at Islamic University.

Ask a homework question - tutors are online