This preview shows page 1. Sign up to view the full content.
Unformatted text preview: hen stays at that point beyond the threshold as an artiﬁce to simplify analysis. The use of
stopped processes is similar to the artiﬁce that we employed in Section 4.5 for ﬁrst passage
times in Markov chains; recall that we added an artiﬁcial trapping state after the desired
passage to simplify analysis.
We next show that the possibly defective stopped process of a martingale is itself a martingale; the intuitive reason is that, before stopping, the stopped process is the same as the
∗
∗
martingale, and, after stopping, Zn = Zn−1 . The following theorem establishes this and the
corresponding results for submartingales and supermartingales.
Theorem 7.4. Given a stochastic process {Zn ; n ≥ 1} and a possibly defective stopping
∗
time N for the process, the stopped process {Zn ; n ≥ 1} is a submartingale if {Zn ; n ≥ 1}
is a submartingale, is a martingale if {Zn ; n ≥ 1} is a martingale, and is a supermartingale
if {Zn ; n ≥ 1} is a supermartingale.
∗
Proof: First we show that, for all three cases, the stopped process satisﬁes E [Zn ] < 1
∗ = Z , so
for any given n ≥ 1. Conditional on N = i for some i < n, we have Zn
i
∗
E [Zn   N = i] = E [Zi   N = i] < 1 for each i < n such that Pr {N = i} > 0. The reason for this is that if E [Zi   N = i] = 1 and Pr {N = i} > 0, then E [Zi ] = 1,
contrary to the assumption that {Zn ; n ≥ 1} is a martingale, submartingale, or super∗
martingale. Similarly, for N ≥ n, we have Zn = Zn so
∗
E [Zn   N ≥ n] = E [Zn   N ≥ n] < 1 if Pr {N ≥ n} > 0. 7.7. STOPPED PROCESSES AND STOPPING TIMES 309 Averaging,
∗
E [Zn ] = n−1
X
i=1 ∗
∗
E [Zn   N =i] Pr {N =i} + E [Zn   N ≥ n] Pr {N ≥ n} < 1. Next assume that {Zn ; n ≥ 1} is a submartingale. For any given n > 1, consider an
arbitrary initial sample sequence Z1 = z1 , Z2 = z2 , . . . , Zn−1 = zn−1 . First, suppose that
for some i ≤ n − 1, (i, z1 , . . . , zi ) ∈ T where T is the set of stopping nodes. Then for the
∗
∗
∗
stopped process under these assumptions, zn−1 = · · · = zi = zi and zj = zj for 1 ≤ j < i.
∗
∗
Also, Zn = zn−1 = zi . Thus
£∗ ∗
§
∗
∗
∗
∗
E Zn Zn−1 =zn−1 , ..., Z1 =z1 = zn−1 .
(7.83)
Next, contrary to the above assumption, assume there is no i ≤ n − 1, (i, z1 , . . . , zi ) ∈ T
∗
where T is the set of stopping nodes. Given this assumption, zj = zj for 1 ≤ j ≤ n − 1.
∗ = Z . Using the fact that {Z ; n ≥ 1} is a submartingale,
Conditional on this, Zn
n
n
£∗ ∗
§
∗
∗
∗
∗
E Zn Zn−1 =zn−1 , ..., Z1 =z1 ≥ zn−1 .
(7.84) The same argument works for martingales and supermartingales by replacing the inequality
in (7.84) by equality for the martingale case and the opposite inequality for the supermartingale case.
Theorem 7.5. Given a stochastic process {Zn ; n ≥ 1} and a possibly defective stopping
∗
time N for the process, the stopped process {Zn ; n ≥ 1} satisﬁes the following conditions
for all n ≥ 1 if {Zn ; n ≥ 1} is a submartingale, martingale, or supermartingale respectively:
∗
E [Z1 ] ≤ E [Zn ] ≤ E [Zn ] (submartingale) (7.85) = E [Zn ] (martingale) (7.86) E [Z1 ] ≥ ≥ E [Zn ] (supermartingale). (7.87) E [Z1 ] = ∗
E [Zn ]
∗
E [Zn ] ∗
Proof: Since a process cannot stop before epoch 1, Z1 = Z1 in all cases. First consider
∗
the case in which {Zn ; n ≥ 1} is a submartingale. Theorem 7.4 shows that {Zn ; n ≥ 1} is
∗ ] for all n ≥ 1. This establishes the ﬁrst
a submartingale, and from (7.76), E [Z1 ] ≤ E [Zn
half of (7.85) and we next prove the second half. First condition on the set of sequences for
∗
which some given initial segment (i, z1 , . . . , zi ) ∈ T for some i < n. Then E [Zn ] = zi . From
(7.74), E [Zn ] ≥ zi , proving (7.85) for this case. For those sequences not having such an
∗
initial segment, Zn = Zn , establishing (7.85) in that case. Averaging over these two cases
gives (7.85) in general. Finally, if {Zn ; n ≥ 1} is a supermartingale, then {−Zn ; n ≥ 1} is a submartingale, verifying
(7.87). Since a martingale is both a submartingale and supermartingale, (7.86) follows and
the proof is complete.
Consider a (nondefective) stopping time N for a martingale {Zn ; n ≥ 1}. Since the stopped
process is also a martingale, we have
∗
∗
E [Zn ] = E [Z1 ] = E [Z1 ] ; n ≥ 1. (7.88) 310 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES ∗
Since Zn = ZN for all n ≥ N and since N is ﬁnite with probability 1, we see that
∗
limn→1 Zn = ZN with probability 1. Unfortunately, in general, E [ZN ] is unequal to
∗
limn→1 E [Zn ] = E [Z1 ]. An example in which this occurs is the binary product martingale in (7.48). Taking the stopping time N to be the smallest n for which Zn = 0, we have
∗
∗
ZN = 0 with probability 1, and thus E [ZN ] = 0. But Zn = Zn for all n, and E [Zn ] = 1 for
all n. The problem here is that, given that the process has not stopped by time n, Zn and
∗
Zn each have the value 2n . Fortunately, in most situations, this type of b...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details