This preview shows page 1. Sign up to view the full content.
Unformatted text preview: +0.5(1+0.5~,*) p,* = 2.73 (time periods) Markov Decision Model Markov Reward Model Mail Order Company Given:
l Markov Process with a known
stationary dlstrlbution (Steady State
Vector)
l
Set of values reflecting the net effect
of each outcome (Reward Vector)
The long run expected value for the
system can be estimated A mail order company is studying the
reliability of its order entry computer
l Using a series of 10 minute samplings
of the current system, it was found that
the computer behaves as follows:
. If the computer was observed in the UP
condition, 5% of the time it was DOWN
at the next observation
l If it was DOWN, it had been brought
back UP by the next sampling 90% of
the time l EM602 / QM710 (NJ) Lecture IO
Page 104 Example (cont’d) Example (cont’d) Mail Order Company Mail Order Company A proposed expansion of the computer
maintenance department will improve
the situation so that:
if in the UP condition, it will be DOWN
at the next observation 4% of the time
if it is DOWN, it can be brought back
UP by the next sampling 95% of the
time
Evaluate the current and proposed
systems Current System l UP DOWN
0.95 0.05
UP
0.10 I
DOWNI 0.90 l l For which we can compute
the Steady  State Vector
x = [0.947 0.053] Example (cont’d) Example (cont’d) Mail Order Company Mail Order Company *Proposed System Determine whether the company should
implement the proposal if:
The net revenue when operatify is
$4000 I day
The net loss when down is $5000 I day
The net cost of implementing the
maintenance plan is $300 per day UP DOWN l l l For which we can compute
the new Stead! State Vecctor
?l = [0.95% o.O.Ku] Example (cont’d) Example Mail Order Company Barber Shop Problem Calculate the expected monetary value
(EMV) of each state under each plan
Current System
EMV = + 4000 (0.947)  5000 (0.053)
= 53523
Proposed System
EMV = +4000 (0.9596)  5000 (0.0404) 300
= $3336 l l l l l l l A barber shop has one barber chair
and 3 chairs for waiting
A haircut always takes 15 minutes
Customers arrive according to some
(known) empirical distribution
if a customer arrives and there is a
place to sit, they wait and get a haircut
if a customer arrives and the shop iS
full, they immediately leave EM602 / QM710 (NJ) Lecture 10
Page 105 ,. Example (cont’d) Example (cont’d)...
View
Full
Document
This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.
 Spring '94
 DonaldC.Johnson

Click to edit the document details