This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MS&E 221  ' Final Examination
Ramesh Johari  March 14, 2005 Instructions . 1. Take altemate seating if possible. 2. Answer all questions in the spaces provided on these sheets. If needed, additional paper will be avail—
able at the front of the room. Answers given on any other paper will not be counted. 3. You may use a calculator, but no notes or books. 4. The examination begins at'82'30 am, and ends at 11:30 am. Honor Code In taking this examination, I acknowledge and accept the Stanford University Honor Code. NAME (signed) ' gal/MTIONS . '  NAME (printed) Useful Formulas 1. If T is an exponentially distributed random variable with mean 1/A, then the density of T is given by fT, where: .
fT(t) = Ae’”, t > 0. 2. If S is a random variable with a gamma distribution of parameters n and A, where n is a positive
integer, then the density of S is givenrby f3, where: ' ~As n—l
h®=§%é%%+yszﬂ 3. For an M /M / 1 queue with arrival rate A and service rate ,u with A < p, the equilibrium distribution
is: WQ=ﬁ=U—MH,j=QLZm,
where p = A/u. 4. For an M /M / oo queue with arrival rate A and service rate p, the equilibrium distribution is: . 6"" . I 1:0’1,2,"', where p = /\/,u. 5. For anM /M / K / K queue with arrival rate A and service rate ,u,’ the equilibrium distribution is: Q WM .
P(Q=J)=Wa 9:0,1,2a,Ka where p '= A/ PART A —' Short answer questions. (60 points; 12 points per question) 1. Either justify the following statement if true, or provide a counterexample if false: If a state 2' is recur
rent for a continuous time Markov chain Xt, then it is also recurrent for the jump chain corresponding t0 Xt. .
(I ream/Mi" ﬂy X; a; it an, ave 1w 4w: J 74,4m
P(,X{.=l (yo‘l)=1 2. Suppose that Xn is a discrete time Markov chain on'the ﬁnite state space X = {1, . . . , K Assume
that all states are aperiodic. Suppose also that this Markov chain has exactly two closed communicat—
ing classes, A1 and A2 (although there may be other communicating classes that are not closed). For
any two states 2' and j does the limit: '  limIP(Xn='jX0=z') = Lg. n—voo J always exist? Justify your answer. we mm remitted {to M .‘s ivveducilole M KM
sledc Space => ’ V‘V‘rqme ivwm‘awr distribution was, if?“ Haiti) M MD = Wm Ak l X0=U k;=l,Z
glmce ﬁﬁle alt/VA mo’r closed 3' rfmwflimi’r mm + mm = I lww" git/Ice i' #4? cbm‘m I .‘5 'apm‘l’df‘li, "ﬂ )6 AU “"6”
lag = NO 1mg); he )‘e A. mm L;)~= hzfihlwi‘); “MA . Cﬁ A7 UA’Z, 2 56 Hm Wit ADA/mgr exx‘g’g, .3. Consider an inﬁnite server queue, where customers arrive according to a Poisson process of rate A.
(Thus every arriving customer gets assigned his own server.) The service times of customers are
independent, with expOnential distribution of mean 1/“. Assume that the queue starts in equilibrium
at time —00; i.e., the number of customers in the queue is a twosided stationary process. Suppose
we note the times at which customers depart from the queue. What is the distribution of the length of
time between successive departures? Justify your answer. The “MW/'00 (7W {5 raveagitate. (I) Twat; ' 42mm rm: 6% mm at we a.
(a) Wepar’rwes M rammed Pam: are 170 GEM a! rake} 3 ~
( 3) {’MQS I)” "imam! [W'CC’B are Patron 0? raftg R % Wmtcmy) . 2?? Lat/9H1 a? «Hm Latvia/w Successive WatWrit ~ cxfa (9‘). 4. Either justify the following statement if true, or provide a counterexample if false: A continuous time
Markov chain is reversible if and only if the corresponding jump chain is reversible. (You should
assume that the Continuous time Markov chain is irreducible, and that the state space is ﬁnite.) 4M1? jnredotCrtu'c ~“' thilc Sink 5/1066 5") [/L‘M4“?l/L[
,4, , CTMC (“WWWt Aﬁ+n17w+7m T l
W 9x1 : habitMg Faramgjer ﬁ/ (Shiv ( m
. . “   Z) 1r < *
IWMMTL «w. .m m J'qu Chm («M W t 'L k °°
since rH/xe sink jfﬁce «‘5 ﬁnale).
AM; 10,), My (j/j; Tn. , 15.62,; [CTMC (Mailed tawch
<9 A; c. Tr). a). _
€=> . ﬁg —— ﬁipjb [pm/1c 0Wle balawte] 4 S9 i5 waibk @ Jam? 00min {3‘ rmwnﬂe‘ 5. You want to estimate the joint distributionof two random variables X and Y. Each of the random
variables takes values on the ﬁnite state space {1, . . . , 10}. You are given the following conditional
distributions, for all values of a: and y: IP<X = W = y) = my);
WY = le = m) = you). You should assume that f(:vy) > 0 and g(yac) _> 0 for all values of a: and y. We denote the joint
distribution of X and Y by IP(X = x, Y = y) = h(m, y). Using only f and 9, construct a Markov
chain with invariant distribution given by h. Justify your answer. L. .
(Aw7ue [P (yum % xer, yum : 3M: ll 5m My) >0, jrxlgxw v 76.3, m cm is
{wet/{Wilde 0w £me 34qu SFMc 53> (AWN imvwiawl 413+,
Check. (bladed Enigma, (9W9 Md *0 check (“Wang/y),
(OWN —? (793W '(wlm x'a‘x, y“):
my WW I M) :gwwilrx'w
_ 177le
g : _, MUG”) T](X/j {XII
Similarly 093),“? [1,)”. 50, char“ )5 M% (/{l/l‘r71/Le (In Val/I‘MVH’ (X13) 
5 PART B. (120 points) Problem 1. (35 points) A gambler needs to win $10 quickly, and he plays the following game. At each
stage, the gambler chooses a wager, and a fair coin is tossed (i.e., the coin has probability 1 / 2 of coming up
heads, and probability 1/2 of coming up tails). If the coin Comes up heads, the gambler wins an additional
amount equal to his Wager. If the coin comes up tails, the gambler loses his wager. The gambler chooses the following (aggressive) policy. If the gambler’s CUrrent wealth is less than or
equal to $5, the gambler wagers everything at the next stage. If the gambler’s current wealth. is greater than
$5, he wagers just enough on the next stage so that if the coin ﬂip comes up heads, his resultingtotal wealth would be exactly $10.  (a) (5 points) Describe the gambler’s current wealth using a Markov chain model, and identify the com—
municating classes. ,wmmmmM ' m '
E 0 i 
O t 2 * 4 9’ '6 ‘ D F“ WNW{ad we Fm,“ lav/.117 '/; _. P00 = (710.10 :— I  For (' § 6', PM = PL‘IZJ '1 FM 1. >6) Pg“):
_ ' gIR‘F’IO ’1 '/Z .
CUWWMV‘iCWg cum; = M, {10%, m, m, 261, :71, W,
{2, 4,6,9 E,
(b) (10 points) Assuming that the gambler starts with $2, calculate the probability that he eventually
reaches $10. I q I
in? = Wm l0 7t, =15).
, L0 : O ‘ 
bl” : l
a l
hi ” 144' ~—9 h a 414
[/1 = i? 8 ‘ 2 r
144 {its . " M. +3 “'9 W a.
g, = “ z + ,
Z 2 —9 bub = 3' in, +3 T
63
H
w—
3‘
<>\
.1. (c) (10 points) Assuming that the gambler starts with $8, calculate the probability that he eventually loses
all his wealth. le Cz’ﬂll/t/l is gamma so (ll/Le OMKWM (Almalivelyl [_ “Lb : 1,4142 '/ If; (d) (10 points) If the gambler starts with $2, what is the expected number of stages until the game ends
(i.e., until either the gambler loses all his wealth, or he earns $10)? M’ ear/l1 sage, W45 W/Fwé. "/2, 0/ l
{5 jail/MW r?) mum f’ﬂ Slt'djpf 4v Problem 2. (45 points) You have an email program that has an inbox that can hold at most three e—mail
messages. Email messages arrive/to your inbox according to a Poisson process of rate A. For any given
message, there is a probability p that the message is spam; in this case your spam ﬁlter always catches the
message and sends it to your trash folder, so that it never enters your inbox. When your inbox is full, any
messages that arrive are blocked. (a) (5 points) For the ﬁrst four parts of this question, you should assume that whenever you check email,
you instantaneously empty the inbox (i.e., you take zero time to process and delete all your email).
After checking your e—mail, you wait an exponentially distributed amount of time with mean 1/?”
before checking your email again. Give a Markov chain description of the number of messages in
your inbox. Staff” lb passages (in )1"be . I (NM "5pm «WWW W?
I 170165in [AU—I7») (b) (10 points) What is the longrun fraction of time that your inbox is full? ' 7!; 2617) = [11“. +1; +19% : (lerrolr
f9 : V“ my = T]; 2(ivr) ;
o . , ). '3 i
t, (+1“? £971}: gar) 3
Tl,l (f+ 7‘0"?» 5 Th) “((1’) (Yvi )UT»
'MiF) ‘
5) Tr, : V ‘ '1‘ t
(r + )(H’ﬁz Eff/:44 ﬁg“, 1T
 ' 0x4
TL [rt ﬁUfn : Tit/MP?) L “A 3'
Lita—.—
3 IF; 3 3 (tr—t 2(1’FD
8 .. (c) (5 points) Assume that the Markov chain is in equilibrium. What is the probability that an arriving
message does not enter your inbox? ' W
7 l
Nﬂmge is ‘ Mexajc (‘5 v~0+‘ I
spam ’ 54?an lam/i (vaax is: div” (Ly Pkg—k) 5 r (d) (10 points) Now suppose that whenever you check your email, if your inbox is not empty, you require
an exponentially diStributed amount of time with mean 1/3 to process and delete all yOur email
(regardless of the number of messages in your inbox). For simplicity, assume that you take your
computer offline while you are checking e—mail, so that no messages arrive while you are checking
email. After having ﬁnished processing your e—mail, you wait an exponentially distributed amount
of time with mean 1/?” before checking your email again. What is the longrun fraction of time that you spend checking email? C z: 0%!ch WW“  0 LZJ‘ #41 “Ways I‘vv‘
' I Mbux_ “Fag = (1T,+1rz+W3)Y
: (I'mTQM? m Mth = TQS V l
=i
‘
3‘
4g.
%‘ (e) (10 points) Now suppose that whenever you check your email, each message in your inbox requires '
an exponentially distributed amount of processing time with mean 1 /,u. As before, we assume that no
messages arrive while you are checking email. We also continue to» assume that after you are ﬁnished _
emptying your inbox, you wait an exponentially distributed amount of time with mean 1/1" before I
checking your email again. Write down a system of equations we can use to determine the long—run
fraction of time that you spend checking e—mail. Express the longrun fraction of time that you spend
checking email in terms of the solution to these equations; you do not need to solve the equations. 35/2393 gt: g "4556323 mt (mung Wu.
NW) I V, "g! i Nitajﬂ, CMCll/lldj mail.
‘9" r I M— l: 11 »
_ . ./’~>o as 1T0 {7 ml"
. : (l’ ) '
m’l’) M V l‘ _' ﬁll (“17) l”) T“ 7‘“ l’ y
ﬁ>o Trzﬂ +V) : W’A l’r ' " 1L2 _. ,
war) / . 1r” v 3 «ﬂ ’Mt/f)
00 f‘ Tn; ’4 = Tl’m‘V + Was It
' Trust» “rumr may {MP = $3an Haw/’70" at 4W WM? Mm = m + 193 + 1% .
(f) (5 points) In terms of the solution to the quations in part (f), what is the longrun rate at which you
process messages (i.e., the longrun average number of messages processed per unit time)? Faxes; msgaﬁwﬁ (are r wmg Check/(by Mail, WHO 0 ($67“? 10 Problem 3. (40 points) Consider a customerservice center to which calls arrive according to a Poisson
process of rate A. Arriving customers have two choices: they can either immediately request to speak with
a customerservice agent, or they can request to use an automated customer—service system. The customer
service center has only one human customerservice agent, so' customers who request to speak with this agent
must queue fer service. The agent can serve customers at a rate of ,u per hour; service times are independent
and exponentially distributed. If customers choose to use the automated system, they immediately begin
service; calls which use the automated system last an exponentially distributed amount of time with mean
1/01. The customerservice center can hold inﬁnitely many calls, including both those that are queuing for , the agent, and those Which are using the automated system. V
Assume that arriving customers choose to wait for the agent with probability p, and otherwise with
probability 1 — p they choose to use the automated system. (a) (8 points) Give a Markov chain model we can use to analyze this system. Determine the necessary
and sufﬁcient condition(s) that must be satisﬁed by A, ,u, (1, and/or p under which this Markov chain is positive recurrent. 5 (#1 0% WSW m/ﬁi'ﬁl/Lﬂ. (law/1%,
# 04th Vian git/1%., system) 3 (703) e' {0,1,2vwi‘xMILL"‘3 Q{((?€39), burW) = AI, , y: ,i.z,. . .‘;3=o,',1~—,~
Qih’rhghﬁnjl) = ’4, x=0,\,Z,. .'./'v=.0,\,‘Z/...
a; z’ 7T0!!! )l 3: OI’IZ'I' “
Q((’V,5H), I Opel)“, 7=0,/2,.1,lj:OI))ZI._. CQUVGy), my») > o Nth/me. » (a, (we + (7.3»). Tim. age/mi": qw i: IM/M/yf' m awmmd 5414M r: M/M /ao. ﬂe’. 47WS we F‘ k M (K. SW 19,8 . q
M/M /( ("2 isiaLie MAW WM < W'e (“it £9 Need ( ’XE < '5 Ly (Fm/Hm recrAwe/Me. 11 (b) (7 points) If the assumption in part (a) is satisﬁed, what is the longrun average number of customers
in the customer—service center? : Mum il’ llﬂ M /M /l W/ ' MMV'M MIL: A F1
Wile rat}! I I; V I + ‘
MM Hull: M M/M/no 4/ nan/9’01? ml: (A07)
WM rah. o< U Jﬁq+ ﬂip).
r”? r (c) (5 points) Calculate the value of p such that the mean waiting time of a customer arriving in equilib
‘ rium will be the same regardless of whether they choose to wait for the agent, or they choose to use
the automated system. (For this part of the question, assume that A < ,u and a < ,u.) ~ MW" Warlﬁj limit M M/M/l alga/n" I .
rilr
Ill/lwa Miami/1.3 4am uh M/M/m outan f. 51% 12 (d) (10 points) If a customer is using the automated system, they may become impatient. Suppose that
each customer who chooses the automated system is willing to wait at most an exponentially dis~
tributed amount of time with mean 1/;6; if after this time they have not completed service in the
autOmated system, they quit and join the queue for the human agent instead. What is the longrun
fraction of time that the human agent is busy serving customers? (For this part of the question, as
sume that A < ,u.) rot/f 40 wamm age/wt 714.4%
a 9x1; + arrpﬁzﬂ), MM" gem/fa? «hwb. WWW agwi'
1, i“
:z) i5 man v’lYMHW lawman mgw’l‘ ('5 WWI/lg a At l— )(l’E)/F//K+F))I
F (e) (10 points) Suppose that currently, one customer is being served by the agent, and two customers are
using the automated system. Find the probability that the entire system empties before another call arrives to the center. F13! Sjslewt (iv «M59411, W 4km MMﬁL' MON/C lamb/c mmav'al "
lm’r +IM Walla/V Wm}? ’wmliWf . Famed/13 : 2:” )(dfa) o( iii,“ :1
4. ’A+_2¢_+') [xviMt) 0”")
jg « « _L
f/{erl‘g 'pkot+) fit) 13 ...
View
Full Document
 Winter '11
 Ramesh
 Markov chain

Click to edit the document details