The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Solution to Homework 2
Part II (Textbook Chapter 2): 2.1, 2.3, 2.4, 2.5, 2.16, 2.20, 2.21, 2.25, 2.30, 2.39
2.1 The conditions (i) and (ii) of definition 2.1.2 are apparent from the definition 2.1.1.
Hence it is sufficient to show that definition 2.1.1 im
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Solution to Homework 1
Part II (Textbook Chapter 1): 1.1, 1.2, 1.3, 1.8, 1.9, 1.12, 1.30, 1.31
1.1 This problem is about the tailintegral formula for expected value. It can be proved
by converting to double sum(integral) and change the order. The interch
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Solution to Homework 3
Part II (Textbook Chapter 3): 3.1, 3.2, 3.4, 3.10, 3.11, 3.9, 3.13, 3.14, 3.15, 3.18,
3.21, 3.22, 3.28, 3.29, 3.31
3.1 (a) Yes, since it is equivalent with statement (3.2.1) in the textbook.
(b) No. We can have N (t) = n but Sn < t
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Solution to Homework 4
Part II (Textbook Chapter 4): 4.1, 4.4, 4.5, 4.7, 4.8, 4.11, 4.18, 4.19, 4.22, 4.26 (a),
(b) and (c), 4.31, 4.41, 4.42, 4.44
4.1 Let Dn be the random demand of time period n. Clearly Dn is i.i.d. and independent of all Xk for k < n.
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Applied Probability Qualifying Review Exam Questions
May, 2009
time allowed: approximately 1/2 hr per question
1. Consider a continuoustime Markov chain cfw_X(t), t 0 with stationary probabilities
cfw_Pi , i 0. Denote the transition rates by cfw_qij and
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Solution to Homework 5
Part II (Textbook Chapter 5): 5.4, 5.9, 5.10, 5.12, 5.13, 5.15, 5.19, 5.21, 5.22
N 1
5.4 Let Ti denote the time to go from i to i + 1. Then i=0
Ti is the time to go from
0 to N . Further Ti is exponential with rate i and Ti are inde
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Lecture 1: Introduction and Preliminaries
Ning Cai
Dept of IELM, HKUST
Ning Cai
Lecture 1 of IELM 5250
1 / 29
Course Information
Course title: Stochastic Models of Operations Research
Instructor: Ning Cai
Office: Room 5559D Email: ningcai@ust.hk
Office ho
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Lecture 3: Renewal Theory
Ning Cai
Dept of IELM, HKUST
Ning Cai
Lecture 3 of IELM 5250
1 / 44
Definition of renewal process
Generalization of Poisson process.
Generalization of rate: Nonhomogeneous Poisson process
Generalization of jump size: Compound Poi
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Lecture 5: ContinuousTime Markov Chains
Ning Cai
Dept of IELM, HKUST
Ning Cai
Lecture 5 of IELM 525
1 / 28
Definition of ContinuousTime Markov Chains
Consider a continuoustime stochastic process cfw_X (t)
taking on values 0, 1, . It is called a continu
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Lecture 4: Markov Chains
Ning Cai
Dept of IELM, HKUST
Ning Cai
Lecture 4 of IELM 5250
1 / 51
Definition of Markov chains
Consider a stochastic process cfw_Xn : n = 1, 2, taking on
a finite or countable number of possible values (assumed
to be the set of
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Homework 1
Part I (Reading): Example 1.3 (A), 1.3(C), 1.5(C), 1.5(F), 1.6(B) and 1.7(A)
Part II (Textbook Chapter 1): 1.1, 1.2, 1.3, 1.8, 1.9, 1.12, 1.30, 1.31
Part III (Some problems in Chapter 1 with solutions given in the end of
textbook): 1.5, 1.6, 1.
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Lecture 2: Poisson Processes
Ning Cai
Dept of IELM, HKUST
Ning Cai
Lecture 2 of IELM 5250
1 / 22
Counting Processes
A counting process N(t) represents the total number of
events" that have occurred up to time t.
N(t) 0;
N(t) is integer valued;
N(t) is non
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Homework 5
Part I (Reading): Example 5.4(C), 5.5(B),
Part II (Textbook Chapter 5): 5.4, 5.9, 5.10, 5.12, 5.13, 5.15, 5.19, 5.21, 5.22
Hint: (1) 5.12 (b): The total
R t time that a CTMC X(s) spends in the state i in the interval
[0, t] can be expressed as
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Homework 4
Part I (Reading): Example 4.1(D),
Part II (Textbook Chapter 4): 4.1, 4.4, 4.5, 4.7, 4.8, 4.11, 4.18, 4.19, 4.31, 4.22, 4.26
(a), (b) and (c), 4.41, 4.42, 4,44
Hint: (1) 4.7 (a) can be solved in dierent ways, which are either fundamental (e.g.,
The Hong Kong University of Science and Technology
Stochastic Model in Operation research
IELM 5250

Fall 2016
Homework 2
Part I (Reading): Example 2.3(C), 2.6(A)
Part II (Textbook Chapter 2): 2.1, 2.3, 2.4, 2.5, 2.16, 2.20, 2.21, 2.25, 2.30, 2.39
Part III (Some problems in Chapter 2 with solutions given in the end of
textbook): 2.8, 2.14, 2.15, 2.22
Requirement: