CS 797 Independent Study Report on
Markov Chains and Queueing Networks
By:
Vibhu Saujanya Sharma
(Roll No. Y211165, CSE, IIT Kanpur)
Under the supervision of:
Prof. S. K. Iyer
(Dept. Of Mathematics, IIT Kanpur)
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentAbstract
Quantitative analysis of computer systems has become very important as they are being used to handle
various mission critical and performance intensive applications. In this independent study I studied
methods to model computer systems for analyzing them for reliability, availability, and performance
metrics. The techniques that were covered include modeling using Discrete Time Markov Chains
(DTMC), Continuous Time Markov Chains (CTMC) and Queueing Networks. Each of the topics, which
were studied, are very extensive and have many books devoted solely to each one of them. Thus, in this
report I present a summary of the important definitions, assumptions, and results pertaining to each one of
these as well as provide some handle on using them practically.
1. Introduction
Modeling of computer systems has become very necessary with quantitative analysis gaining
importance these days. Almost in every related field the emphasis is now on concrete numbers rather than
intuitive arguments in support of the system being built or sold. Ascertaining a number of desirable
properties of computer systems such as reliability, availability, and performance metrics such as
throughput, response times, etc. mandate the need for accurately modeling these systems. Queueing
networks and Markov chains offer fairly simple yet effective and practical modeling solutions to such
problems. The aim of this independent study was to study these topics in detail and also learn about their
practical applications. The text
Probability and Statistics with Reliability, Queuing, and Computer Science
Applications
by Dr. Kishor S. Trivedi [1] was used as the guiding reference for this study along with some
reading from [2] and [3]. Discrete Time Markov Chains, Continuous Time Markov Chains, Stochastic Petri
Nets, and Network of Queues, were studied as a part of this course along with some other paradigms and
basics required for stochastic modeling. I also solved many problems given in the text as a part of this
course.
This report contains a gist of some of the important and basic topics, which were studied. I have tried
to present the important results and techniques along with some intuition behind them. I have also included
some examples showing the use of the techniques. The reader is suggested to refer to [1], for detailed
derivations as well as other examples.
2 Discrete Time Markov Chains
2.1 Introduction
A Markov Chain is a stochastic process whose state space I is discrete (finite or countably infinite) and
the probability distributions for its future development depend only on the present state and not on the path
(consisting of past states), that was followed to reach this state. If we further assume that the parameter
space T is also discrete, then we have a
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 MohammadAbdolahiAzgomiPh.D
 Markov chain, transition probability matrix, DTMC, time Markov Chains, CTMCs

Click to edit the document details