Lecture10

Period 2 ww em 602 i qm 710 nj lecture 10 page 1 o 2

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: .. EM-602 I QM-710 (NJ) Lecture 10 Page 1 O-2 First visit was in period 1 -- First Recurrence Time First Passage Time Expected ttrst Passage time is defined as the expected value of the number of time periods required to: l Start at state i, and end at state j in the nlh period, and l The visit in the nth period to state j was the ttrst visit In that case, the expected ttrst passage time will be given the symbol &ii(“) l l p’ii < c*) if the state is Transient First Passage Time Graphical Analogy Solution State 2 can be reached from all other states First passage probability is computed according to: k=J if the state is Recurrent CL1 =m First Passage Time l A special case of first passage time is Expccted First Return or Expected Recurrence Tlme p$” For a given state: . For example, if there are 4 states kz = 1 f PA p42 + PI, CL,2 + PI, k2 .. Example (solution) Numerical Example First Passage Probability First Passage Probabiltty l Lkvclop thr equation from me f;s’ = p;J’_ f~"p$l _ f;"PA" A stochastic system (with states 1,2, S 3) behaves according to the one-step transition matrtx shown below: I.2 .3 .q In order to solve, first compurc: JO] A3 .20 i 30 ‘=G :: :I Find the probability of first passage from state 1 to state 2 in the third epoch A9 P’?) = .32 l /.,I A5 .25 I EM-602 I QM-710 (NJ) Lecture IO Page IO-3 $?cncml romda Example (solution) Numerical Example First Passage Probability Expected First Passage Time l A stochastic system (with states 1, 2, 6 3) behaves according to the one-step transition matrix shown below: f:,” = (47) - (.2)(.48) - f,:“(.6) r.2 .3 ?'=/A 51 f,‘;’ = p:,’ - f,‘;‘P;;’ = (49) - (.3X.6) l Substituting and solving: 0; L.5 .5 In order to solve, we need to find /I:” .6 OJ Find the expected first passage time from state 1 to state 2 /,:” = 0.185 Example (solution) Homework Problem #l Expected First Passage Time Lecture 10 Use the transition matrix from Problem 18-17 to find l (a) the probability of first passage from state 1 to 3 in the third time period l (b) the expected first passage time from state 1 to all other states k-1 Pll = 1 + 4, PI2 + P,, P32 pu P’n = 1 + Pa + P,, PI2 After substituting, combine and solve ~,*-o.2~,z = 1...
View Full Document

This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.

Ask a homework question - tutors are online