Renewal Theory
Goal for these notes:
Recall what we mean by a law of large numbers.
Dene the renewal counting process. It is a generalization of the Poisson counting process, where we will permit the inter-event times to be
non-exponential.
Dene a rene

Renewal Theory
Goal for these notes:
Recall what we mean by a law of large numbers.
Dene the renewal counting process. It is a generalization of the Poisson counting process, where we will permit the inter-event times to be
non-exponential.
Dene a rene

Continuous Markov Chains (CTMC)
A continuous time stochastic process:
cfw_X(t) : t [0, ),
is a continuous time Markov chain, if
1. the state space is discrete;
2. it satises the Markov property, i.e. if for every n 0, for the times
0 t0 , t1 , . . . , tn

Markov Chains
Classication of States and Potential Theory
Classication Example 1: Consider the (homogeneous) DTMC with
the following transition diagram.
3
0.3
0.05
1
5
2
0.99
0.6 0.7
0.03
0.01
0.92
1
4
0.4
Since there is a path (with non-zero probability)

Introduction to Markov Chains
Consider the stochastic process:
cfw_X(t) : t T ,
that is a set of random variables.
Terminology:
1. X(t) is a random variable;
2. t is a parameter that often represents time;
3. T is the parameter space;
(a) if T is discrete

Reliability Theory (Survival Analysis)
In reliability theory, we are usually interested in the time to an event
(which is usually a failure in engineering - in biostatistics it is onset of a
disease or even death). Say X is the time to failure and its rel

Continuous Random Variables
Denition: A random variable X is said to be continuous if its cumulative distribution function FX is an absolutely continuous function. This
means that there exits a function f (x) such that
x
f (u) du.
FX (x) =
Remark: Absolut

Phase type distributions - Part II
Up to now, we have introduced few phase type distributions: exponential,
Erlang-r and a mixture of an Erlang-(r 1) and Erlang-r.
Figure 1: Phase Diagram for an exponential
1
2
r
Figure 2: Phase Diagram for an Erlang-r
r1

Phase type distributions
Why are phase type distributions important?
Can be used for modeling certain systems.
Dense in the set of positive-valued distributions, i.e. either the true
distribution is a phase type distribution or it can be approximated by

Poisson Process Part II
Event Epochs and Inter-event times
Now that we have the machinery to work with continuous models, we are
ready to think of the Poisson process in terms of its inter-event times.
Denition: Consider the sequence random variables S1 ,

MAT 4371 - SYS 5120 (Winter 2014)
Conditional Distributions
In these notes, we will discuss conditioning on the value of a discrete
random variable Y that takes on the values y1 , y2 , . . ., with probabilities
p1 , p2 , . . .
Consider another random vari

Introduction to R (statistical computing)
R is a free software environment for statistical computing and graphics.
It compiles and runs on a wide variety of Linux platforms, Windows and
MacOS. Visit the following webpage: http:/www.r-project.org/
These no

Dominated Convergence
These notes concern one important result from mathematical analysis.
It is a special case of Lebesgues Dominated Convergence Theorem from
Measure Theory, which is one of the most important results in mathematical
analysis. We will re

Discrete Probability Models
One mathematical tool that will allow us to manipulate models is the
probability generating function (sometime called the z-transform). Let X
taken values on cfw_0, 1, 2, 3, . . . with the corresponding probabilities p0 , p1 ,

Poisson Process Part I
The Poisson process is one of the most important random processes in
probability theory. It is widely used to model random events in time and
space, such as the times of radioactive emissions, the arrival times of customers at a ser

Random Variables
Denition :
Let be a sample space. A function X : R, that
associates a real number X(s) to each outcome s is called a random variable.
Notation : The range of the random variable is denoted RX .
Note :
We use upper-case letter (often at th

Mathematical Preliminaries
To build our probability models, we will need some mathematical tools.
Here are a few.
little o notation :
f (x) is o(x),
Remark :
if
f (x)
= 0.
x0 x
lim
It means that the functions converges to zero faster than x.
Example :
x i

The Art of Counting
Tree Diagram: If the experiment can be described as a sequence of k steps,
then the sample space can be illustrated with a tree diagram. Any path through
the tree signies a single outcome in the sample space.
Example 1:
Roll a fair six

Introduction to Probability
Denition : A random experiment is an experiment or a process for
which the outcome cannot be predicted with certainty.
Denition : The sample space (denoted ) of a random experiment is
the set of all possible outcomes.
Example 1