MIT6_262S11_lec12 - 6.262 Discrete Stochastic Processes Lecture 12 Renewal rewards stopping trials and Walds equality Outline Review strong law for

Theorem: If { Z n ; n ≥ 1 } converges to α WP1, (i.e., Pr { ω : lim n ( Z n ( ω ) − α ) = 0 } = 1 ), and f ( x ) is continu ous at α . Then Pr { ω : lim n f ( Z n ( ω )) = α } = 1 . For a renew al process with inter-renew als X i , 0 < X < ∞ , Pr ω : lim n ( 1 S n ( ω ) − X ) = 0 = 1 ) n n 1 Pr ω : lim = = 1 . n →∞ S n ( ω ) X For renewal processes, n/S n and N ( t ) /t are related by ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ❅ ❅ ❅ ❅ ❘ Slope = N ( t ) ❄ Slope = N ( t ) t S N ( t ) Slope = N ( t ) S ❆ N ( t )+1 ❆ ❆ ❆  N ( t ) t 0 S 1 S N ( t ) S N ( t )+1 2 6.262: Discrete Stochastic Processes 3/14/11 Lecture 12 : Renewal rewards, stopping trials, and Wald’s equality Outline: • Review strong law for renewals • Review of residual life • Time-averages for renewal rewards • Stopping trials for stochastic processes • Wald’s equality • Stop when you’re ahead 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

The strong law for renewal processes follows from this relation between n/S n and N ( t ) /t . Theorem: For a renewal process with X < ∞ , Pr ω : lim N ( t, ω ) /t = 1 /X = 1 . t →∞ This says that the rate of renewals over the infinite time horizon (i.e., lim t N ( t ) /t ) is 1 /X WP1. This also implies the weak law for renewals, N ( t ) 1 lim Pr t − →∞ t X > = 0 for all > 0 3 X 5 ( ω ) ❅ ❅ ❅ X 2 ( ω ) ❅ Y ( t, ω ) ❅ X 4 ( ω ) ❅ ❅ ❅ ❅ X X ( ω ) ❅ ❅ 6 ( ω 1 ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅❅ ❅ t ❅ ❅ ❅ ❅ S 1 ( ω ) S 2 ( ω ) S 4 ( ω ) S 5 ( ω ) S 6 ( ω ) N ( t, ω ) 2 N ( t, ω )+1 X ( ω ) 1 t 2 X ( ω ) i Y ( t, ω ) dt i 2 t ≤ t 0 n ≤ 2 t =1 n =1 Review of residual life Def: The residual life Y ( t ) of a renewal process at time t is the remaining time until the next renewal, i.e., Y ( t ) = S N ( t )+1 − t . Residual life is a random process; for each sample point ω , Y ( t, ω ) is a sample function. 4

N ( t, ω ) ( )+1 2 N t, ω 2 X ( ω ) 1 X i ≤ t ( ω ) Y ( t, ω ) dt 2 t 0 i t ≤ 2 t i =1 i =1 Going to the limit t → ∞ ( 1 N t, ω ) t 2 X ( ω ) N ( t, ω ) lim Y ( t, ω ) dt = lim i t →∞ t 0 t →∞ 2 N ( t, ω ) t n =1 E 2 X = 2 E [ X ] This is infinite if E 2 X = ∞ . Think of example where p X ( ) = 1 − , p X (1 / ) = .