This preview shows page 1. Sign up to view the full content.
Unformatted text preview: .. EM602 I QM710 (NJ) Lecture 10
Page 1 O2 First visit was in period 1  First Recurrence Time First Passage Time
Expected ttrst Passage time is defined as
the expected value of the number of
time periods required to:
l Start at state i, and end at state j in the
nlh period, and
l The visit in the nth period to state j was
the ttrst visit
In that case, the expected ttrst passage
time will be given the symbol &ii(“) l l p’ii < c*) if the state is Transient First Passage Time
Graphical Analogy Solution State 2 can be reached from all other states First passage probability is computed
according to: k=J if the state is Recurrent CL1 =m First Passage Time
l A special case of first passage time is
Expccted First Return or Expected
Recurrence Tlme p$”
For a given state: . For example, if there are 4 states
kz = 1 f PA p42 + PI, CL,2 + PI, k2 .. Example (solution) Numerical Example First Passage Probability First Passage Probabiltty
l Lkvclop thr equation from me
f;s’ = p;J’_ f~"p$l _ f;"PA" A stochastic system (with states 1,2, S 3)
behaves according to the onestep
transition matrtx shown below:
I.2 .3 .q In order to solve, first compurc:
JO] A3 .20 i 30 ‘=G :: :I
Find the probability of first passage from
state 1 to state 2 in the third epoch A9 P’?) = .32
l /.,I A5 .25 I EM602 I QM710 (NJ) Lecture IO
Page IO3 $?cncml romda Example (solution) Numerical Example First Passage Probability Expected First Passage Time
l A stochastic system (with states 1, 2, 6
3) behaves according to the onestep
transition matrix shown below: f:,” = (47)  (.2)(.48)  f,:“(.6) r.2 .3 ?'=/A 51 f,‘;’ = p:,’  f,‘;‘P;;’ = (49)  (.3X.6) l Substituting and solving: 0; L.5 .5 In order to solve, we need to find /I:” .6 OJ Find the expected first passage time
from state 1 to state 2 /,:” = 0.185 Example (solution) Homework Problem #l Expected First Passage Time Lecture 10
Use the transition matrix from Problem
1817 to find
l (a) the probability of first passage from
state 1 to 3 in the third time period
l (b) the expected first passage time
from state 1 to all other states k1 Pll = 1 + 4, PI2 + P,, P32 pu P’n = 1 +
Pa + P,, PI2
After substituting, combine and solve
~,*o.2~,z = 1...
View
Full
Document
This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.
 Spring '94
 DonaldC.Johnson

Click to edit the document details