Discrete-time stochastic processes

# The purpose of this exercise is to illustrate that

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: hen stays at that point beyond the threshold as an artiﬁce to simplify analysis. The use of stopped processes is similar to the artiﬁce that we employed in Section 4.5 for ﬁrst passage times in Markov chains; recall that we added an artiﬁcial trapping state after the desired passage to simplify analysis. We next show that the possibly defective stopped process of a martingale is itself a martingale; the intuitive reason is that, before stopping, the stopped process is the same as the ∗ ∗ martingale, and, after stopping, Zn = Zn−1 . The following theorem establishes this and the corresponding results for submartingales and supermartingales. Theorem 7.4. Given a stochastic process {Zn ; n ≥ 1} and a possibly defective stopping ∗ time N for the process, the stopped process {Zn ; n ≥ 1} is a submartingale if {Zn ; n ≥ 1} is a submartingale, is a martingale if {Zn ; n ≥ 1} is a martingale, and is a supermartingale if {Zn ; n ≥ 1} is a supermartingale. ∗ Proof: First we show that, for all three cases, the stopped process satisﬁes E [|Zn |] &lt; 1 ∗ = Z , so for any given n ≥ 1. Conditional on N = i for some i &lt; n, we have Zn i ∗ E [|Zn | | N = i] = E [|Zi | | N = i] &lt; 1 for each i &lt; n such that Pr {N = i} &gt; 0. The reason for this is that if E [|Zi | | N = i] = 1 and Pr {N = i} &gt; 0, then E [|Zi |] = 1, contrary to the assumption that {Zn ; n ≥ 1} is a martingale, submartingale, or super∗ martingale. Similarly, for N ≥ n, we have Zn = Zn so ∗ E [|Zn | | N ≥ n] = E [|Zn | | N ≥ n] &lt; 1 if Pr {N ≥ n} &gt; 0. 7.7. STOPPED PROCESSES AND STOPPING TIMES 309 Averaging, ∗ E [|Zn |] = n−1 X i=1 ∗ ∗ E [|Zn | | N =i] Pr {N =i} + E [|Zn | | N ≥ n] Pr {N ≥ n} &lt; 1. Next assume that {Zn ; n ≥ 1} is a submartingale. For any given n &gt; 1, consider an arbitrary initial sample sequence Z1 = z1 , Z2 = z2 , . . . , Zn−1 = zn−1 . First, suppose that for some i ≤ n − 1, (i, z1 , . . . , zi ) ∈ T where T is the set of stopping nodes. Then for the ∗ ∗ ∗ stopped process under these assumptions, zn−1 = · · · = zi = zi and zj = zj for 1 ≤ j &lt; i. ∗ ∗ Also, Zn = zn−1 = zi . Thus £∗ ∗ § ∗ ∗ ∗ ∗ E Zn |Zn−1 =zn−1 , ..., Z1 =z1 = zn−1 . (7.83) Next, contrary to the above assumption, assume there is no i ≤ n − 1, (i, z1 , . . . , zi ) ∈ T ∗ where T is the set of stopping nodes. Given this assumption, zj = zj for 1 ≤ j ≤ n − 1. ∗ = Z . Using the fact that {Z ; n ≥ 1} is a submartingale, Conditional on this, Zn n n £∗ ∗ § ∗ ∗ ∗ ∗ E Zn |Zn−1 =zn−1 , ..., Z1 =z1 ≥ zn−1 . (7.84) The same argument works for martingales and supermartingales by replacing the inequality in (7.84) by equality for the martingale case and the opposite inequality for the supermartingale case. Theorem 7.5. Given a stochastic process {Zn ; n ≥ 1} and a possibly defective stopping ∗ time N for the process, the stopped process {Zn ; n ≥ 1} satisﬁes the following conditions for all n ≥ 1 if {Zn ; n ≥ 1} is a submartingale, martingale, or supermartingale respectively: ∗ E [Z1 ] ≤ E [Zn ] ≤ E [Zn ] (submartingale) (7.85) = E [Zn ] (martingale) (7.86) E [Z1 ] ≥ ≥ E [Zn ] (supermartingale). (7.87) E [Z1 ] = ∗ E [Zn ] ∗ E [Zn ] ∗ Proof: Since a process cannot stop before epoch 1, Z1 = Z1 in all cases. First consider ∗ the case in which {Zn ; n ≥ 1} is a submartingale. Theorem 7.4 shows that {Zn ; n ≥ 1} is ∗ ] for all n ≥ 1. This establishes the ﬁrst a submartingale, and from (7.76), E [Z1 ] ≤ E [Zn half of (7.85) and we next prove the second half. First condition on the set of sequences for ∗ which some given initial segment (i, z1 , . . . , zi ) ∈ T for some i &lt; n. Then E [Zn ] = zi . From (7.74), E [Zn ] ≥ zi , proving (7.85) for this case. For those sequences not having such an ∗ initial segment, Zn = Zn , establishing (7.85) in that case. Averaging over these two cases gives (7.85) in general. Finally, if {Zn ; n ≥ 1} is a supermartingale, then {−Zn ; n ≥ 1} is a submartingale, verifying (7.87). Since a martingale is both a submartingale and supermartingale, (7.86) follows and the proof is complete. Consider a (non-defective) stopping time N for a martingale {Zn ; n ≥ 1}. Since the stopped process is also a martingale, we have ∗ ∗ E [Zn ] = E [Z1 ] = E [Z1 ] ; n ≥ 1. (7.88) 310 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES ∗ Since Zn = ZN for all n ≥ N and since N is ﬁnite with probability 1, we see that ∗ limn→1 Zn = ZN with probability 1. Unfortunately, in general, E [ZN ] is unequal to ∗ limn→1 E [Zn ] = E [Z1 ]. An example in which this occurs is the binary product martingale in (7.48). Taking the stopping time N to be the smallest n for which Zn = 0, we have ∗ ∗ ZN = 0 with probability 1, and thus E [ZN ] = 0. But Zn = Zn for all n, and E [Zn ] = 1 for all n. The problem here is that, given that the process has not stopped by time n, Zn and ∗ Zn each have the value 2n . Fortunately, in most situations, this type of b...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online