Problem Set 11 and 12
DueDec4by11:59pm
Points20
Submittingonpaper
PS11:Writeandsolveagoodoriginalexamquestiononthematerialsincethelastmidterm.
PS12:Onseparatepaper:
Chapter7:47,48,50.(allsemiMarkov)
1.Ifthetimebetweenrenewalsiseither5withprobability1/3and
IEOR263A Sample Midterm 1 Solution
1. True or false:
(a) False. A discrete-time Markov chain has the property that the
future, conditioning on the present, is independent of the past.
The past and future, without conditioning on the present, could
be depe
IEOR263A Sample Midterm 1 Solution
1. True or false:
(a) False. A discrete-time Markov chain has the property that the
future, conditioning on the present, is independent of the past.
The past and future, without conditioning on the present, could
be depe
HW2 Solution
September 11
2.60, 2.63, 2.71(Var(X) using Yj), 3.21, 3.22, 3.32.
2.60 Calculate the moment generating function of the uniform distribution
on (0, 1). Obtain E[X] and Var[X] by dierentiating.
Answer:
2.63 Calculate the moment generating funct
IEOR 263A, Solution to Problem Set 1, 09/04/2014
1. (a) The -eld generated by cfw_a is F = cfw_a, b, c, d, cfw_, cfw_a, cfw_b, c, d.
The -eld generated by cfw_a and cfw_a, b is
F = cfw_a, b, c, d, cfw_, cfw_a, cfw_b, c, d, cfw_a, b, cfw_c, d, cfw_a, c, d,
IEOR 263A Sample Midterm 1
September 18, 2012
Please show all your work on these pages. Good luck! This midterm occurred
a bit later in the semester than yours e.g. 1 involves the gambler's ruin.
1. (20 points) Frankie goes to Las Vegas to gamble. He want
IEOR 263A Sample Midterm 2
175 points (4 questions)
1. (5 points each) For the following statements, indicate whether they are T (true) or F (false).
a. _ For a nonhomogeneous Poisson process with continuous rate function (t), the distribution of the numb
IEOR 263A Sample Final
190 points
Also study delayed/equilibrium renewal processes and semi-Markov processes.
1. Consider an alternating renewal process with on times Ziexp( ) and off times Yiexp( ) where
Zi's and Yi's are i.i.d and independent of each ot
Sample Final Solutions
1.
2
a) EXi = 1/1 + 1/2 and EXi = 1/2 + 1/2 + (1/1 + 1/2 )2 . Hence
1
2
EY (t) =
1/2 + 1/2 + (1/1 + 1/2 )2
1
2
2(1/1 + 1/2 )
b) By memoryless property of the exponential distribution
E[Y (t)|state is on at t] = 1/1 + 1/2
2.
Let the
IEOR 263A Sample Midterm 1
165 points (6 questions)
1. (5 points each) For the following statements, indicate whether they are T (true) or F (false).
a. _ A discrete-time Markov chain is a stochastic process that has the property that the future is
indepe
IEOR 263A Midterm 1 F/11
150 points
NAME:
Please show all your work on these pages. Good luck!
This midterm occurred a bit later in the semester than yours e.g. 1 involves the gamblers ruin.
1. (20 points) Frankie goes to Las Vegas to gamble. He wants to
3.41 A rat is trapped in a maze. Initially it has to choose one of two directions. If it goes to the right, then it will wander around in the maze
for three minutes and will then return to its initial position. If it goes
to the left, then with probabilit
1
Limiting Mean Excess and Key Renewal Theorem
P
Recall, for xed n: P fSn xg = P f n Xi xg = Fn (x), the n-fold convolution of F , and
i=1
P
P
m(t) = EN (t) = 1 Fn (t), so dm(t) = 1 dFn (t).
n=1
n=1
Proposition 1.1
P fSN (t)
sg = P fSN (t) = 0g +
Z
s
0
dF
Renewal Theory Summary:
Time between events Xi
Sn =
Pn
i=1 Xi
F i.i.d and EX =
= time of n'th event
N (t)
Sn
= 1=rate!
Fn = n-fold convolution
n () Sn
N (t) = maxfn : Sn
t
tg
m(t) = EN (t) = mean value function or renewal function
Another way to get m(t):
Problem Set 10
DueNov20by11am
Points10
Submittingonpaper
Chapter7problems:32,34,37,42,43,44(optional).Alsodothefollowing.
1.UsingtherenewalrewardtheoremshowthatwhenES=1foranM/G/1/nqueue,the
expectednumberoflossesduringabusyperiodis1foralln.Hint:Notethatal
Problem Set 12 Solution
!1
2
P1=22/155, P2=66/155, P3=67/155.
The time the state is i is broken into 2 parts the time ti
waiting at i, and the time traveling. Hence, the proportion
of time the taxi is waiting at state i is Piti/i and the
proportion of tim
are
into 2 partsthe
traveling. Hence,
waiting at state i
time it is travel-
s consisting of n
hich is exponenand consider the
at any time is the
time. Hence, this
ate 1 to 2 to 3 . . . to
pent in each state
clearly the limitkov chain is Pi =
Pcfw_Y (t) <
Page 1 of 6
Problem Set 9 Solution"
Chapter 7 problems: 13, 14, 15, 16, 23.
Page 2 of 6
Solution:"
"
"
Page 3 of 6
Page 4 of 6
(d)
Page 5 of 6
son process with rate . If a customer will enter the bank only if the server is free
when he arrives, and if th
population size is N or larger.
(a) Set this up as a birth and death model.
(b) If N = 3, 1 = = , = 2, determine the proportion of time that immigration
is restricted.
13. A small barbershop, operated by a single barber, has room for at most two customers
Problem Set 10 Solution
Problem Set 10 Solution
Problem Set 10 Solution!
XN(t)+1 < c if and only if time t lies in a cycle of length less than c.
Therefore, we can assign reward Xi to a cycle of length less than c,
and zero otherwise. By the renewal-rewar
Page 1 of 8
Problem Set 7 Solution"
"
Chapter 5 - 63 (optional), 66, 70 (optional), 72 (optional), 80, 83 (optional), 86, 94
(optional)
Chapter 6 - 1, 6 (add d: Determine E[time to go from state 4 to 0] assuming i
, i , and < .).
"
"
"
"
"
"
"
"
Page 2 o
PS 4 Solution
1. The expected discounted reward is
E eX R = E[eX ]E[R] = X ()E[R]
where X (t) is the Laplace transform of X. The dependence on R is
only via the mean E[R], and the distribution of R does not matter
other than that.
2. Let P n be the n-step
4-52!
5-6
!
!
!
!
!
!
!
sij
i 6= j sij = fij sjj + (1
fij
fij
fij ) 0 = fij sij sjj = 1 + fjj sjj
sjj =
1
1 fjj
4. (a) We assume that if B shoots when both A and C are alive, B will
shoot C; if C shoots when both A and B are alive, C will shoot B.
In shor
IEOR 263A Problem Set 4
Due 9/25: Write and solve a good, original exam question on the material for Midterm 1 (on
Chapters 1, 2, 3, and 4.1-4.4, but skipping sections 2.7, 3.6.2-3.6.6, 3.7).
The rest of this problem set is not to turn in, but to prepare
IEOR 263A Problem Set 6
Due October 8. Write and solve a good, original exam question on the
material so far.
The rest of this assignment is not to turn in but to use to study for
the midterm. We will probably postpone some of the Poisson process
material
IEOR 263A Problem Set 4
Due 9/24: Optional questions are not to turn in but may be good exam questions . Read Chapter
4 and do Chapter 4 problems: 3, 5, 12, 14, 15, 17, 18. Also:
1. (optional) Suppose a reward of value R ~ F will be earned upon completion
IEOR 263A Problem Set 5
Due 10/1: Read Chapter 5. Do Chapter 4 problems 20, 25, 38, 42, 43 (optional), 45, 52 and
Chapter 5 problem 6. Also,
1. (optional) Using the expression for the gambler's ruin probability, nd expressions for
fij for j > i, for j < i
IEOR 263A Problem Set 3
Due 9/18. Read Chapter 4, but well skip Sections 4.5.2, 4.5.3, 4.8-4.11 and probably 4.7.
Do problems 3.41 (add b: Compute the variance of the time the rat will be trapped in the
maze), 3.59 (just get to expressions involving pis o