Lecture10

# 2z 105105 p 273 time periods markov decision

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: +0.5(1+0.5~,*) p,* = 2.73 (time periods) Markov Decision Model Markov Reward Model Mail Order Company Given: l Markov Process with a known stationary dlstrlbution (Steady State Vector) l Set of values reflecting the net effect of each outcome (Reward Vector) The long run expected value for the system can be estimated A mail order company is studying the reliability of its order entry computer l Using a series of 10 minute samplings of the current system, it was found that the computer behaves as follows: . If the computer was observed in the UP condition, 5% of the time it was DOWN at the next observation l If it was DOWN, it had been brought back UP by the next sampling 90% of the time l EM-602 / QM-710 (NJ) Lecture IO Page 10-4 Example (cont’d) Example (cont’d) Mail Order Company Mail Order Company A proposed expansion of the computer maintenance department will improve the situation so that: if in the UP condition, it will be DOWN at the next observation 4% of the time if it is DOWN, it can be brought back UP by the next sampling 95% of the time Evaluate the current and proposed systems Current System l UP DOWN 0.95 0.05 UP 0.10 I DOWNI 0.90 l l For which we can compute the Steady - State Vector x = [0.947 0.053] Example (cont’d) Example (cont’d) Mail Order Company Mail Order Company *Proposed System Determine whether the company should implement the proposal if: The net revenue when operatify is \$4000 I day The net loss when down is \$5000 I day The net cost of implementing the maintenance plan is \$300 per day UP DOWN l l l For which we can compute the new Stead! -State Vecctor ?l = [0.95% o.O.Ku] Example (cont’d) Example Mail Order Company Barber Shop Problem Calculate the expected monetary value (EMV) of each state under each plan Current System EMV = + 4000 (0.947) - 5000 (0.053) = 53523 Proposed System EMV = +4000 (0.9596) - 5000 (0.0404) 300 = \$3336 l l l l l l l A barber shop has one barber chair and 3 chairs for waiting A haircut always takes 15 minutes Customers arrive according to some (known) empirical distribution if a customer arrives and there is a place to sit, they wait and get a haircut if a customer arrives and the shop iS full, they immediately leave EM-602 / QM-710 (NJ) Lecture 10 Page 10-5 ,. Example (cont’d) Example (cont’d)...
View Full Document

## This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.

Ask a homework question - tutors are online