Discrete-time stochastic processes

Random walks large deviations and martingales

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: nothing conceptually diﬀerent. Let T be the collection of stopping nodes for the given random walk with two thresholds. We refer to each stopping node as a pair, (n, x ), where n is the length of the initial segment and x = (x1 , . . . , xn ) is the segment itself. The probability that this initial segment occurs is then p(x1 )p(x2 ) · · · p(xn ). From Lemma 7.1, stopping occurs with probability 1, so n XY p(xi ) = 1. (n,x )∈T i=1 Now consider the same set of sample sequences, but with a diﬀerent IID probability measure. In particular, each rv Xi is replaced with a ‘tilted’ rv with the PMF q (X ) = p(X )erX −∞ (r) 7.6. MARTINGALES AND SUBMARTINGALES 299 for any ﬁxed r, r− < r < r+ . Note that q (X ) sums to 1 over the sample values of X , so it is in fact a valid PMF. Note also that we can consider the same random walk as with the original rv’s, with only the probability measure changed. That is, the same set of stopping nodes correspond to threshold crossings as before. Applying Lemma 7.1 to this modiﬁed random walk, we get n XY q (xi ) = 1. (n,x )∈T i=1 Expressing this in terms of p(x), n XY (n,x )∈T i=1 p(xi ) exp[rxi − ∞ (r)] = 1. For any given (n, x ) ∈ T , the product above can be written as n Y i=1 where s(n, x ) = p(xi ) exp[rxi − ∞ (r)] = Pr {(n, x )} exp[rs(n, x ) − n∞ (r)]. Pn i=1 xi . Thus X Pr {(n, x )} exp[rs(n, x ) − n∞ (r)] = 1. (7.43) (n,x )∈T Note that the stopping time N and the stopping value SN are functions of the stopping node, and a stopping node occurs with probability 1. Thus the expectation in Wald’s identity can be taken over the stopping nodes, and that expectation is given in (7.43), completing the proof. The proof of this theorem makes it clear that the theorem can be generalized easily in a number of directions. The only requirement is that the random walk must stop with probability 1, and the tilted version must also stop with probability 1. Thus, for example, the thresholds could vary as a function of the trial number, the thresholds could be replaced by the ﬁrst time some given pattern occurs, etc. It is usually not possible, however, to replace two thresholds with one, since the most useful stopping nodes in that case do not occur with probability 1. 7.6 Martingales and submartingales A martingale is deﬁned as an integer-time stochastic process {Zn ; n ≥ 1} with the properties that E [|Zn |] < 1 for all n ≥ 1 and E [Zn | Zn−1 , Zn−2 , . . . , Z1 ] = Zn−1 ; for all n ≥ 2. (7.44) The name martingale comes from gambling terminology where martingales refer to gambling strategies in which the amount to be bet is determined by the past history of winning or 300 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES losing. If one visualizes Zn as representing the gambler’s fortune at the end of the nth play, the deﬁnition above means, ﬁrst, that the game is fair in the sense that the expected increase in fortune from play n − 1 to n is zero, and, second, that the expected fortune on the nth play depends on the past only through the fortune on play n − 1. There are two interpretations of (7.44); the ﬁrst and most straightforward is to view it as shorthand for E [Zn | Zn−1 =zn−1 , Zn−2 =zn−2 , . . . , Z1 =z1 ] = zn−1 for all possible sample values z1 , z2 , . . . , zn−1 . The second is that E [Zn | Zn−1 =zn−1 , . . . , Z1 =z1 ] is a function of the sample values z1 , . . . , zn−1 and thus E [Zn | Zn−1 , . . . , Z1 ] is a random variable which is a function of the random variables Z1 , . . . , Zn−1 (and, for a martingale, a function only of Zn−1 ). The student is encouraged to take the ﬁrst viewpoint initially and to write out the expanded type of expression in cases of confusion. It is important to understand the diﬀerence between martingales and Markov chains. For the Markov chain {Xn ; n ≥ 1}, each rv Xn is conditioned on the past only through Xn−1 , whereas for the martingale {Zn ; n ≥ 1}, it is only the expected value of Zn that is conditioned on the past only through Zn−1 . The rv Zn itself, conditioned on Zn−1 can be dependent on all the earlier Zi ’s. It is very surprising that so many results can be developed using such a weak form of conditioning. In what follows, we give a number of important examples of martingales, then develop some results about martingales, and then discuss those results in the context of the examples. 7.6.1 Simple examples of martingales Example 7.6.1 (Zero-mean random walk). One example of a martingale is a zeromean random walk, since if Zn = X1 + X2 + · · · + Xn , where the Xi are IID and zero mean, then E [Zn | Zn−1 , . . . , Z1 ] = E [Xn + Zn−1 | Zn−1 , . . . , Z1 ] = E [Xn ] + Zn−1 = Zn−1 . (7.45) (7.46) Extending this example, suppose that {Xi ; i ≥ 1} is an arbitrary sequence of IID random e variables with mean X and let Xi = Xi − X . Then {Sn ; n ≥ 1} is a random walk with e e Sn = X1 + · · · + Xn and {Zn ; n ≥ 1} is a martingale with Zn = X1 + · · · + Xn . The random w...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online