46
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 15: Reviewing The Exponential Distribution
Discrete-time Markov chains have a notable limitation: There is no real notation of time. In
particular, we considered time to be a transition betwe

ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
51
Lecture 16: The Poisson Process
Counting Process
A counting process counts the number of events that occur during time intervals. More
formally, a stochastic process cfw_N (t) : t 0 is a counting pro

42
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 14: More on DTMC on Innite State Spaces
Recall the random-walk example from last lecture:
If cfw_Xn : n 0 is a random walk on the integers with transition probabilities
Pi,i+1 = p = 1 Pi,i1 ,

38
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 13: More on Classication of States
Periodicity
n
We are interested in determining whether a limit for Pi,j = P(Xn = j|X0 = i) exists as
n . As we know from deterministic sequences, a convergi

30
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 11: Chapman-Kolmogorov Equations
Recap: Unconditional Probabilities. We might be interested in the unconditional probability of being in a state, or in the probability of some path. Let
:= (

36
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 12: Classication of States
Classication of States
To better understand what a well-behaved M.C. is, we need to study the structure of M.Cs.
n
State j is said to be accessible from state i if

28
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 10: More on DTMC
The Probability of a Path
The Markov property makes it easy to compute the probability that the process moves
through a specic (nite) path. Suppose that we are interested in

ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
25
Lecture 9: Stochastic Processes and DTMC
Denition: A stochastic process cfw_Xt : t T is a collection of random variables: for
each t T , Xt is a r.v. with T being a parameter set, for example, T = c

56
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 18: More on the Poisson Process
Conditional Distribution of the Arrival Times
Suppose that we know an event occurred over the interval [0, t] in a Poisson process cfw_N (t) :
t 0. Does this g

54
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 17: Poisson Process - An Alternative View
Recall from last lecture:
A counting process with unit jumps, having stationary and independent increments is a
Poisson process. That is, the number

58
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 19: More Examples of Using Poisson Processes
EXAMPLE 1: Exponential Batteries. A ashlight needs two batteries to be operational. Consider such a ashlight along with a set of n functional batt

94
IEE 470 - Fall 2013- Instructor: Dr. Soroush Saghafian
Lecture 27: M/G/1 Queue
M/G/1 Queue:
Poisson arrivals (exponentially distributed interarrival times)
General distribution for service times
One server
M/G/1 Queue Parameters:
Arrival rate: customer

88
IEE 470 - Fall 2013- Instructor: Dr. Soroush Saghafian
Lecture 26: M/M/c Queue
Poisson arrivals (exponentially distributed interarrival times)
Exponentially distributed service times
c service channels
M/M/c Queue Parameters:
Arrival rate: customers pe

71
IEE 470 - Fall 2013- Instructor: Dr. Soroush Saghafian
Lecture 23: Queueing Theory-Contd
Basic relationships in G/G/c- Contd
Let pB = probability that a server is busy (i.e., fraction of time a server is busy)
Average number of customers in service at

75
IEE 470 - Fall 2013- Instructor: Dr. Soroush Saghafian
Lecture 24: M/M/1 Queue
Poisson arrivals (exponentially distributed interarrival times)
Exponentially distributed service times
One server
Objective:
Obtain steady state probability distribution of

66
IEE 470 - Fall 2013- Instructor: Dr. Soroush Saghafian
Lecture 22: Queueing Theory
Introduction
Queueing: Waiting in line
Queue: Line of customers waiting for service
Representation of Queueing System:
Service
Customers
Arrivals
Departures
Queue
Exampl

64
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 21: More Examples- Contd
Example: Emails. Suppose you get emails according to a Poisson process with rate = 0.2
per hour. You check your email every hour.
(a) What is the chance that you nd 0

62
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 20: More Examples- Contd
In this lecture we will review and rework on the last parts of the Jamaica Example from the
previous lecture to make sure we clearly understand how we can calculate d

ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
21
Lecture 8
Three Modes of Convergence
We now discuss three dierent modes of convergence associated with random variables, and
corresponding fundamental limit theorems.
1. Convergence with Probability

ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
17
Lecture 7
Conditional Probability and Expectation
Conditional Probability
We can condition on an event or on another r.v.
Conditioning on an event. The conditional cdf of X given the event A with P(A

14
ASU-IEE 470 Fall 2013 Instructor: Dr.Soroush Saghaan
Lecture 6
Joint Distributions
We may be interested in the joint distribution of two or more r.v.s. For example, we may
be interested in the distribution of the vector (X, Y ), where both X and Y are

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 5 (Solution)
(I) Customers arrive at a specic store in Tempe according to a Poisson process with rate
4 per hour. The store opens at 9AM. What is the probability t

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 4
Due Date: 10/17/13 (In class, before the lecture)
(1) An absent minded professor of Stochastic Operations Research has two umbrellas. If it
rains on a day and an

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 4 (Solution)
(1) An absent minded professor of Stochastic Operations Research has two umbrellas. If it
rains on a day and an umbrella is available where he is (hom

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 3
Dues Date: 10/03/2012, (In class, before the lecture)
(1) A Markov chain cfw_Xn : n 0 with three states 0, 1, 2 has the following transition
probability matrix:

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 3 (Solution)
(1) A Markov chain cfw_Xn : n 0 with three states 0, 1, 2 has the following transition
probability matrix:
1 1 1
2
3
6
P = 0
1
3
2
3
1
2
0
1
2
If P(X0

IEE 470: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 2
Due Date: 09/24/13 (In class, before the lecture)
(1) Suppose one is collecting coupons to obtain a retail discount. There are r dierent types
of coupons. We ass

IEE 470/598: Stochastic Operation Research
Fall 2013, Prof. Soroush Saghaan
Homework Assignment 2 (Solutions)
(1) Suppose one is collecting coupons to obtain a retail discount. There are r dierent types
of coupons. We assume that the probability that a co