4106-08-Notes-RP

4106-08-Notes-RP - Copyright c 2008 by Karl Sigman 1...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
± 2008 by Karl Sigman 1 Regenerative Processes Continuous-time case Given a stochastic process X = { X ( t ) : t 0 } , suppose that there exists a (proper) random time τ = τ 1 such that { X ( τ + t ) : t 0 } has the same distribution as X and is independent of the past C 1 = {{ X ( t ) : 0 t < τ } , τ } . Then we say that X regenerated at time τ , meaning that it has “started over again” probabilistically, as if it was time t = 0 again, and its future is independent of its past. In particular, X ( τ ) has the same distribution as X (0), and is independent of the regeneration time τ . C 1 is called the first cycle and X 1 = τ 1 is the first cycle length . But if such a τ 1 exists, then there must be a second such time τ 2 > τ 1 yielding an identically distributed second cycle C 2 = {{ X ( τ 1 + t ) : 0 t < τ 2 - τ 1 } , τ 2 - τ 1 } . Continuing in this fashion we conclude that there is a renewal process of such times { τ k : k 1 } (with τ 0 def = 0) with iid cycle lengths X k = τ k - τ k - 1 , k 1 , and such that the entire cycles C k = {{ X ( τ k - 1 + t ) : 0 t < X k } , X k } are iid. Note how in particular, X ( τ k ) is independent of the time τ k : Upon regeneration the value of the process is independent of what time it is. The above defines what is called a regenerative process . It is said to be positive recurrent if the renewal process is so, that is, if 0 < E ( X 1 ) < . Otherwise it is said to be null recurrent if E ( X 1 ) = . As an example: Any recurrent continuous-time Markov chain is regenerative. Fix any state i and let X (0) = i . Then let τ = τ i,i denote the first time that the chain returns back to state i (after first leaving it). By the (strong) Markov property, the chain indeed starts all over again in state i at time τ independent of its past. Since i is recurrent, we know that the chain must in fact return over and over again, and we can let τ k denote the k th such time. We also know that positive recurrence of the chain, by definition, means that E ( τ i,i ) < , which is thus equivalent to being a positive recurrent regenerative process. Because { τ k } is a renewal process and the cycles { C k } are iid, we can apply the renewal reward theorem in a variety of ways to a regenerative process so as to compute various time average quantities of interest. The general result can be stated in words as the time average is equal to the expected value over a cycle divided by the expected cycle length. For example, we can obtain the time-average of the process itself as lim t →∞ 1 t Z t 0 X ( s ) ds = E ( R ) E ( X ) , wp 1 , where X = X 1 and R = R 1 = R X 0 X ( s ) ds . The proof is derived by letting N ( t ) denote the number of renewals (regenerations) by time t , and observing that Z t 0 X ( s ) ds N ( t ) X j =1 R j , where R j = Z τ j τ j - 1 X ( s ) ds. 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 9

4106-08-Notes-RP - Copyright c 2008 by Karl Sigman 1...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online