This preview shows page 1. Sign up to view the full content.
Unformatted text preview: nothing conceptually diﬀerent. Let T be the collection of stopping nodes for the given random
walk with two thresholds. We refer to each stopping node as a pair, (n, x ), where n is the
length of the initial segment and x = (x1 , . . . , xn ) is the segment itself. The probability
that this initial segment occurs is then p(x1 )p(x2 ) · · · p(xn ). From Lemma 7.1, stopping
occurs with probability 1, so
n
XY p(xi ) = 1. (n,x )∈T i=1 Now consider the same set of sample sequences, but with a diﬀerent IID probability measure.
In particular, each rv Xi is replaced with a ‘tilted’ rv with the PMF q (X ) = p(X )erX −∞ (r) 7.6. MARTINGALES AND SUBMARTINGALES 299 for any ﬁxed r, r− < r < r+ . Note that q (X ) sums to 1 over the sample values of X , so it
is in fact a valid PMF. Note also that we can consider the same random walk as with the
original rv’s, with only the probability measure changed. That is, the same set of stopping
nodes correspond to threshold crossings as before. Applying Lemma 7.1 to this modiﬁed
random walk, we get
n
XY q (xi ) = 1. (n,x )∈T i=1 Expressing this in terms of p(x),
n
XY (n,x )∈T i=1 p(xi ) exp[rxi − ∞ (r)] = 1. For any given (n, x ) ∈ T , the product above can be written as
n
Y
i=1 where s(n, x ) = p(xi ) exp[rxi − ∞ (r)] = Pr {(n, x )} exp[rs(n, x ) − n∞ (r)]. Pn i=1 xi . Thus
X
Pr {(n, x )} exp[rs(n, x ) − n∞ (r)] = 1. (7.43) (n,x )∈T Note that the stopping time N and the stopping value SN are functions of the stopping node,
and a stopping node occurs with probability 1. Thus the expectation in Wald’s identity can
be taken over the stopping nodes, and that expectation is given in (7.43), completing the
proof.
The proof of this theorem makes it clear that the theorem can be generalized easily in
a number of directions. The only requirement is that the random walk must stop with
probability 1, and the tilted version must also stop with probability 1. Thus, for example,
the thresholds could vary as a function of the trial number, the thresholds could be replaced
by the ﬁrst time some given pattern occurs, etc. It is usually not possible, however, to
replace two thresholds with one, since the most useful stopping nodes in that case do not
occur with probability 1. 7.6 Martingales and submartingales A martingale is deﬁned as an integertime stochastic process {Zn ; n ≥ 1} with the properties
that E [Zn ] < 1 for all n ≥ 1 and
E [Zn  Zn−1 , Zn−2 , . . . , Z1 ] = Zn−1 ; for all n ≥ 2. (7.44) The name martingale comes from gambling terminology where martingales refer to gambling
strategies in which the amount to be bet is determined by the past history of winning or 300 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES losing. If one visualizes Zn as representing the gambler’s fortune at the end of the nth
play, the deﬁnition above means, ﬁrst, that the game is fair in the sense that the expected
increase in fortune from play n − 1 to n is zero, and, second, that the expected fortune on
the nth play depends on the past only through the fortune on play n − 1.
There are two interpretations of (7.44); the ﬁrst and most straightforward is to view it as
shorthand for E [Zn  Zn−1 =zn−1 , Zn−2 =zn−2 , . . . , Z1 =z1 ] = zn−1 for all possible sample
values z1 , z2 , . . . , zn−1 . The second is that E [Zn  Zn−1 =zn−1 , . . . , Z1 =z1 ] is a function of
the sample values z1 , . . . , zn−1 and thus E [Zn  Zn−1 , . . . , Z1 ] is a random variable which is
a function of the random variables Z1 , . . . , Zn−1 (and, for a martingale, a function only of
Zn−1 ). The student is encouraged to take the ﬁrst viewpoint initially and to write out the
expanded type of expression in cases of confusion.
It is important to understand the diﬀerence between martingales and Markov chains. For
the Markov chain {Xn ; n ≥ 1}, each rv Xn is conditioned on the past only through Xn−1 ,
whereas for the martingale {Zn ; n ≥ 1}, it is only the expected value of Zn that is conditioned on the past only through Zn−1 . The rv Zn itself, conditioned on Zn−1 can be
dependent on all the earlier Zi ’s. It is very surprising that so many results can be developed using such a weak form of conditioning.
In what follows, we give a number of important examples of martingales, then develop some
results about martingales, and then discuss those results in the context of the examples. 7.6.1 Simple examples of martingales Example 7.6.1 (Zeromean random walk). One example of a martingale is a zeromean random walk, since if Zn = X1 + X2 + · · · + Xn , where the Xi are IID and zero
mean, then
E [Zn  Zn−1 , . . . , Z1 ] = E [Xn + Zn−1  Zn−1 , . . . , Z1 ]
= E [Xn ] + Zn−1 = Zn−1 . (7.45)
(7.46) Extending this example, suppose that {Xi ; i ≥ 1} is an arbitrary sequence of IID random
e
variables with mean X and let Xi = Xi − X . Then {Sn ; n ≥ 1} is a random walk with
e
e
Sn = X1 + · · · + Xn and {Zn ; n ≥ 1} is a martingale with Zn = X1 + · · · + Xn . The random
w...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details