Discrete-time stochastic processes

0 is a very small number say 106 1 1 12 0 12

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: izarre behavior does not occur and E [ZN ] = E [Z1 ]. To get a better understanding of when E [ZN ] = E [Z1 ], note that for any n, we have ∗ E [Zn ] = = n X i=1 n X i=1 ∗ ∗ E [Zn | N = i] Pr {N = i} + E [Zn | N &gt; n] Pr {N &gt; n} (7.89) E [ZN | N = i] Pr {N = i} + E [Zn | N &gt; n] Pr {N &gt; n} . (7.90) The left side of this equation is E [Z1 ] for all n. If the ﬁnal term on the right converges to 0 as n → 1, then the sum must converge to E [Z1 ]. If E [|ZN |] &lt; 1, then the sum also converges to E [ZN ]. Without the condition E [|ZN |] &lt; 1, the sum might consist of alternating terms which converge, but whose absolute values do not converge, in which case E [ZN ] does not exist (see Exercise 7.23 for an example). Thus we have established the following theorem. Theorem 7.6. Let N be a stopping time for a martingale {Zn ; n ≥ 1}. Then E [ZN ] = E [Z1 ] if and only if lim E [Zn | N &gt; n] Pr {N &gt; n} = 0 n→1 and E [|ZN |] &lt; 1. (7.91) Example 7.7.1 (Random walks with thresholds). Recall the generating function product martingale of (7.49) in which {Zn = exp[rSn − n∞ (r)]; n ≥ 1} is a martingale deﬁned in terms of the random walk {Sn = X1 +· · ·+Xn ; n ≥ 1}. From (7.86), we have E [Zn ] = E [Z1 ], and since E [Z1 ] = E [exp{rX1 − ∞ (r}] = 1, we have E [Zn ] = 1 for all n. Also, for any possi∗ bly defective stopping time N , we have E [Zn ] = E [Z1 ] = 1. If N is a non-defective stopping time, and if (7.91) holds, then E [ZN ] = E [exp{rSN − N ∞ (r)}] = 1. (7.92) If there are two thresholds, one at α &gt; 0, and the other at β &lt; 0, and the stopping rule is to stop when either threshold is crossed, then (7.92) is just the Wald identity, (7.24). The nice part about the approach here is that it also applies naturally to other stopping rules. For example, for some given integer n, let Nn+ be the smallest integer i ≥ n for which Si ≥ α or Si ≤ β . Then, in the limit β → −1, Pr {SNn + ≥ α} = Pr {∪1 n (Si ≥ α)}. i= © ™ Assuming X &lt; 0, we can ﬁnd an upper bound to Pr SNn+ ≥ α for any r &gt; 0 and ∞ (r) ≤ 0 (i.e., for 0 &lt; r ≤ r∗ ) by the following steps £ § © ™ 1 = E exp{rSNn+ − Nn+ ∞ (r)} ≥ Pr SNn+ ≥ α exp[rα − n∞ (r)] © ™ Pr SNn+ ≥ α ≤ exp[−rα + n∞ (r)]; 0 ≤ r ≤ r∗ . (7.93) 7.7. STOPPED PROCESSES AND STOPPING TIMES 311 This is almost the same result as (7.38), except that it is slightly stronger; (7.38) bounded the probability that the ﬁrst threshold crossing crossed α at some epoch i ≥ n, whereas this includes the possibility that Sm ≥ α and Si ≥ α for some m &lt; n ≤ i. 7.7.1 Stopping times for martingales relative to a process In Section 7.6.2, we deﬁned a martingale {Zn ; n ≥ 1} relative to a joint process {Zn , Xn ; n ≥ 1} as a martingale for which (7.59) is satisﬁed, i.e., E [Zn | Zn−1 , Xn−1 , . . . , Z1 , X1 ] = Zn−1 . In the same way, we can deﬁne a submartingale or supermartingale {Zn ; n ≥ 1} relative to a joint process {Zn , Xn ; n ≥ 1} as a submartingale or supermartingale satisfying (7.59) with the = sign replaced by ≥ or ≤ respectively. The purpose of this added complication is to make it easier to deﬁne useful stopping rules. As before, a stopping rule is a rule that determines a collection of stopping nodes. A stopping node here, is an initial segment (i, x1 , z1 , . . . , xi , zi ) of a joint sample sequence of {Zn , Xn ; n ≥ 1}. If an initial segment is a stopping node for one joint sample sequence, it is a stopping node for all joint sample sequences with that initial segment, and no shorter initial segment is a stopping node. The stopping time for joint sequences with that initial segment is the length i of that segment. Theorems 7.4, 7.5, 7.6 all carry over to martingales (submartingales or supermartingales) relative to a joint process. These theorems are stated more precisely in Exercises 7.24 to 7.27. To summariz...
View Full Document

Ask a homework question - tutors are online