Markov Chains
Stationary Distribution
Consider a (homogeneous) DTMC whose state is cfw_1, 2, . . . , N . If the
state space is countably infinite, then we take N to be .
Stationary Distribution
Recall: For an ergodic irreducible DTMC the DTMC has a unique
Introduction to Markov Chains
Consider the stochastic process:
cfw_X(t) : t T ,
that is a set of random variables.
Terminology:
1. X(t) is a random variable;
2. t is a parameter that often represents time;
3. T is the parameter space;
(a) if T is discrete
Random Variables
Denition :
Let be a sample space. A function X : R, that
associates a real number X(s) to each outcome s is called a random variable.
Notation : The range of the random variable is denoted RX .
Note :
We use upper-case letter (often at th
Poisson Process Part II
Event Epochs and Inter-event times
Now that we have the machinery to work with continuous models, we are
ready to think of the Poisson process in terms of its inter-event times.
Definition: Consider the sequence random variables S1
Poisson Process Part I
Simple Point Process
Definition: Consider the sequence random variables S1 , S2 , S3 , . . . such
that
S1 < S2 < S3 < . . .
We say that the sequence is a simple point process. We say that Si is an
event epoch, that is we interpret i
Introduction to R (statistical computing)
R is a free software environment for statistical computing and graphics.
It compiles and runs on a wide variety of Linux platforms, Windows and
MacOS. Visit the following webpage: http:/www.r-project.org/
These no
Discrete Probability Models
One mathematical tool that will allow us to manipulate models is the
probability generating function (sometime called the z-transform). Let X
taken values on cfw_0, 1, 2, 3, . . . with the corresponding probabilities p0 , p1 ,
Continuous Random Variables
Definition: A random variable X is said to be continuous if its cumulative distribution function FX is an absolutely continuous function. This
means that there exits a function f (x) such that
Z x
f (u) du.
FX (x) =
Remark: Abs
Discrete Probability Models
One mathematical tool that will allow us to manipulate models is the
probability generating function (sometime called the z-transform). Let X
taken values on cfw_0, 1, 2, 3, . . . with the corresponding probabilities p0 , p1 ,
Phase type distributions
Motivation:
We have the exponential distribution that can be used as a model for
time to an event. However, the exponential satisfies the memoryless
property. This means that the exponential is often not a reasonable
model.
In t
Markov Chains
Classification of States and Potential Theory
Classification Example 1: Consider the (homogeneous) DTMC with
the following transition diagram.
3
0.3
0.05
1
5
2
0.99
0.6 0.7
0.03
0.01
0.92
1
4
0.4
Since there is a path (with non-zero probabil
Renewal Theory
Goal for these notes:
We will state the strong law of large numbers.
Define the renewal counting process. It is a generalization of the Poisson counting process, where we will permit the inter-event times to be
non-exponential.
Define a
Mathematical Preliminaries
To build our probability models, we will need some mathematical tools.
Here are a few.
little o notation :
f (x) is o(x),
Remark :
if
f (x)
= 0.
x0 x
lim
It means that the functions converges to zero faster than x.
Example :
x i
Dominated Convergence
These notes concern one important result from mathematical analysis. It
is a special case of Lebesgues Dominated Convergence Theorem from Measure Theory, which is one of the most important results in measure theory.
We will restrict
Markov Chains
Classification of States and Potential Theory
Classification Example 1: Consider the (homogeneous) DTMC with
the following transition diagram.
3
0.3
0.05
1
5
2
0.99
0.6 0.7
0.03
0.01
0.92
1
4
0.4
Since there is a path (with non-zero probabil
The Art of Counting
Tree Diagram: If the experiment can be described as a sequence of k steps,
then the sample space can be illustrated with a tree diagram. Any path through
the tree signifies a single outcome in the sample space.
Example 1:
Roll a fair s
Introduction to Probability
Definition : A random experiment is an experiment or a process for
which the outcome cannot be predicted with certainty.
Definition : The sample space (denoted ) of a random experiment is
the set of all possible outcomes.
Examp
The Art of Counting
Tree Diagram: If the experiment can be described as a sequence of k steps,
then the sample space can be illustrated with a tree diagram. Any path through
the tree signifies a single outcome in the sample space.
Example 1:
Roll a fair s
f (x)
f (x)
= 0.
x!0 x
o(x),
lim
x
p
x
o(x)
a 2 R f (x)
f (x)
f (x)
o(x)
g(x)
x2
o(x)
a f (x)
o(x)
f (x) + g(x)
f (x) ! 0
o(1)
o(x)
o(x)
x!0
a0 , a1 , a2 , . . .
f (x) =
1
X
ai xi = a0 + a1 x + a2 x2 + . . . .
i=0
1
R
|x| < R
x = R
R > 0 f (x)
R<x<R
ai
xi
Name Student #
SYS 5130 Midterm
October 20, l7:30-19:30
Closed Book
In total you have 10 questions, 2 marks for each question.
1. (i) State the three basic components of optimization, and indicate which
is essential.
(Ebjective functionts), variables (-eW
Anti-tuberculosis Campaign
1. Classification of the Individuals Targeted by the Antituberculosis Campaign
The government of a developing country is carrying out an anti-tuberculosis campaign within its own
border. In the model time is discrete and denoted
Let p = (p )L be a price vector. Consumer i solves the following utility maximization problem:
(1)
max(xi1 ,.,xi ,.,xiL ) ui [xi1 , ., xi , ., xiL ]
subject to the budget constraint
(2)
p1 xi1 + . + p xi + . + pL xiL = wi ,
where wi is her income.
Sol
The Location of Trauma Centers
1. Introduction
A trauma center (or trauma centre) is a hospital equipped and staffed to provide care for patients suffering from major traumatic injuries such as falls, motor vehicle collisions, or gunshot wounds. A trauma
THE OPTIMUM DISTRIBUTION OF
NATURAL GAS
1. THE PROBLEM
This problem was about the optimal distribution outside the South-West region of
France of the natural gas from the Lacq field. In 1951, La Socit Nationale des
Ptroles dAquitaine (The National Oil Com
The Basic Economic Model of Private
Health Care
1. Introduction
In the model, there are L goods and services, indexed by , = 1, ., L. These goods and services are
produced by J firms, indexed by j, j = 1, ., J. A production plan for a firm, say j, is a li
Assignment 3 (Due: Nov. 4, Wednesday, in class)
Instructions:
1. Group Assignment
2. Do all Questions by network.exe. Copy and paste the results to word.
3. Print out your assignment and submit
1. (2 points). A company has two plants producing motors that
Let vi [p, wi ] denote the indirect utility function of consumer i = 1, ., I. That is, vi [p, wi ] represents the
maximum utility enjoyed by consumer i, given that her income is wi and p is the prevailing price vector.
Let U[u1 , ., ui , ., uI ] denot
Family Name_
First Name_
Student #_
_
SYS 5130 Midterm-October 9, 19:00-20:30
Closed Book
(In total you have 8 questions, 20 marks in total.)
1. (2 points) The main topics in this course are:
(a) _Systems Optimization_
(b) _Management_
2. (2 points) Consi
Family Name_
First Name_
Student #_
_
SYS 5130 Midterm-October 19, 19:00-20:30
Closed Book
(In total you have 6 questions, 20 marks in total.)
Marking scheme: 2+3+3+2+3+3+4
1. Consider the following system:
-3x1 2x2 + x3 + x4 0,
2x2 5x3 + x4 3,
5x1
- x3
=