This preview shows page 1. Sign up to view the full content.
Unformatted text preview: onvert this branching process into a martingale by scaling it, however. That is, deﬁne
n
Zn = Xn /Y . It follows that
∑
∏
Xn
Y Xn−1
E [Zn  Zn−1 , . . . , Z1 ] = E
= Zn−1 .
(7.68)
n  Xn−1 , . . . , X1 =
n
Y
Y
Thus {Zn ; n ≥ 1} is a martingale. We will see the surprising result later that this implies
that Zn converges with probability 1 to a limiting rv as n → 1. 7.6.5 Partial isolation of past and future in martingales Recall that for a Markov chain, the states at all times greater than a given n are independent
of the states at all times less than n conditional on the state at time n. The following
lemma shows that at least a small part of this independence of past and future applies to
martingales. 7.6. MARTINGALES AND SUBMARTINGALES 305 Lemma 7.2. Let {Zn ; n ≥ 1} be a martingale. Then for any n > i ≥ 1,
E [Zn  Zi , Zi−1 , . . . , Z1 ] = Zi . (7.69) Proof: For n = i + 1, E [Zi+1  Zi , . . . , Z1 ] = Zi by the deﬁnition of a martingale. Now
consider n = i + 2. Then E [Zi+2  Zi+1 , . . . , Z1 ] is a rv equal to Zi+1 . We can view
E [Zi+2  Zi , . . . , Z1 ] as the conditional expectation of the rv E [Zi+2  Zi+1 , . . . , Z1 ] over
Zi+1 conditional on Zi , . . . , Z1 .
E [Zi+2 Zi , . . . , Z1 ] = E [E [Zi+2  Zi+1 , Zi , . . . , Z1 ]  Zi , . . . , Z1 ]
= E [Zi+1  Zi , . . . , Z1 ] = Zi . (7.70) For n = i +3, (7.70), with i incremented, shows us that the rv E [Zi+3  Zi+1 , . . . , Z1 ] = Zi+1 .
Taking the conditional expectation of this rv over Zi+1 conditional on Zi , . . . , Z1 , we get
E [Zi+3  Zi , . . . , Z1 ] = Zi .
This argument can be applied successively to any n > i.
The same argument can also be used (see Exercise 7.18) to show that
E [Zn ] = E [Z1 ] 7.6.6 for all n > 1. (7.71) Submartingales and supermartingales Submartingales and supermartingales are simple generalizations of martingales that provide many useful results for very little additional work. We will subsequently derive the
Kolmogorov submartingale inequality, which is a powerful generalization of the Markov inequality. We use this both to give a simple proof of the strong law of large numbers and
also to better understand threshold crossing problems for random walks.
Deﬁnition 7.4. A submartingale is an integer time stochastic process {Zn ; n ≥ 1} that
satisﬁes the relations
E [Zn ] < 1 ; E [Zn  Zn−1 , Zn−2 , . . . , Z1 ] ≥ Zn−1 ; n ≥ 1. (7.72) A supermartingale is an integer time stochastic process {Zn ; n ≥ 1} that satisﬁes the relations
E [Zn ] < 1 ; E [Zn  Zn−1 , Zn−2 , . . . , Z1 ] ≤ Zn−1 ; n ≥ 1. (7.73) In terms of our gambling analogy, a submartingale corresponds to a game that is at least
fair, i.e., where the expected fortune of the gambler either increases or remains the same.
A supermartingale is a process with the opposite type of inequality.4
4 The easiest way to remember the diﬀerence between a submartingale and a supermartingale is to remember that it is contrary to what common sense would dictate. That is, a submartingale is bigger than a
supermartingale. Why this terminology became standard is a mystery. 306 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES Since a martingale satisﬁes both (7.72) and (7.73) with equality, a martingale is both a
submartingale and a supermartingale. Note that if {Zn ; n ≥ 1} is a submartingale, then
{−Zn ; n ≥ 1} is a supermartingale, and conversely. Thus, some of the results to follow
are stated only for submartingales, with the understanding that they can be applied to
supermartingales by changing signs as above.
Lemma 7.2, with the equality replaced by inequality, also applies to submartingales and
supermartingales. That is, if {Zn ; n ≥ 1} is a submartingale, then
E [Zn  Zi , Zi−1 , . . . , Z1 ] ≥ Zi ; 1 ≤ i < n, (7.74) ; 1 ≤ i < n. (7.75) and if {Zn ; n ≥ 1} is a supermartingale, then
E [Zn  Zi , Zi−1 , . . . , Z1 ] ≤ Zi Equations (7.74) and (7.75) are veriﬁed in the same way as Lemma 7.2 (see Exercise 7.20).
Similarly, the appropriate generalization of (7.71) is that if {Zn ; n ≥ 1} is a submartingale,
then
E [Zn ] ≥ E [Zi ] ; for all i, 1 ≤ i < n. (7.76) for all i, 1 ≤ i < n. (7.77) and if {Zn ; n ≥ 1} is a supermartingale, then
E [Zn ] ≤ E [Zi ] ; A random walk {Sn ; n ≥ 1} with Sn = X1 + · · · + Xn is a submartingale, martingale,
or supermartingale respectively for X ≥ 0, X = 0, or X ≤ 0. Also, if X has a semiinvariant moment generating function ∞ (r) for some given r, and if Zn is deﬁned as Zn =
exp(rSn ), then the process {Zn ; n ≥ 1} is a submartingale, martingale, or supermartingale
respectively for ∞ (r) ≥ 0, ∞ (r) = 0, or ∞ (r) ≤ 0. The next example gives an important way
in which martingales and submartingales are related.
Example 7.6.4 (Convex functions of martingales). Figure 7.8 illustrates the graph
of a convex function h of a real variable x. A function h of a real variable is deﬁned to be
convex if, for each point x1 ,...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details