Discrete-time stochastic processes

# What is the decision rule now exercise 75 for the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: onvert this branching process into a martingale by scaling it, however. That is, deﬁne n Zn = Xn /Y . It follows that ∑ ∏ Xn Y Xn−1 E [Zn | Zn−1 , . . . , Z1 ] = E = Zn−1 . (7.68) n | Xn−1 , . . . , X1 = n Y Y Thus {Zn ; n ≥ 1} is a martingale. We will see the surprising result later that this implies that Zn converges with probability 1 to a limiting rv as n → 1. 7.6.5 Partial isolation of past and future in martingales Recall that for a Markov chain, the states at all times greater than a given n are independent of the states at all times less than n conditional on the state at time n. The following lemma shows that at least a small part of this independence of past and future applies to martingales. 7.6. MARTINGALES AND SUBMARTINGALES 305 Lemma 7.2. Let {Zn ; n ≥ 1} be a martingale. Then for any n > i ≥ 1, E [Zn | Zi , Zi−1 , . . . , Z1 ] = Zi . (7.69) Proof: For n = i + 1, E [Zi+1 | Zi , . . . , Z1 ] = Zi by the deﬁnition of a martingale. Now consider n = i + 2. Then E [Zi+2 | Zi+1 , . . . , Z1 ] is a rv equal to Zi+1 . We can view E [Zi+2 | Zi , . . . , Z1 ] as the conditional expectation of the rv E [Zi+2 | Zi+1 , . . . , Z1 ] over Zi+1 conditional on Zi , . . . , Z1 . E [Zi+2 |Zi , . . . , Z1 ] = E [E [Zi+2 | Zi+1 , Zi , . . . , Z1 ] | Zi , . . . , Z1 ] = E [Zi+1 | Zi , . . . , Z1 ] = Zi . (7.70) For n = i +3, (7.70), with i incremented, shows us that the rv E [Zi+3 | Zi+1 , . . . , Z1 ] = Zi+1 . Taking the conditional expectation of this rv over Zi+1 conditional on Zi , . . . , Z1 , we get E [Zi+3 | Zi , . . . , Z1 ] = Zi . This argument can be applied successively to any n > i. The same argument can also be used (see Exercise 7.18) to show that E [Zn ] = E [Z1 ] 7.6.6 for all n > 1. (7.71) Submartingales and supermartingales Submartingales and supermartingales are simple generalizations of martingales that provide many useful results for very little additional work. We will subsequently derive the Kolmogorov submartingale inequality, which is a powerful generalization of the Markov inequality. We use this both to give a simple proof of the strong law of large numbers and also to better understand threshold crossing problems for random walks. Deﬁnition 7.4. A submartingale is an integer time stochastic process {Zn ; n ≥ 1} that satisﬁes the relations E [|Zn |] < 1 ; E [Zn | Zn−1 , Zn−2 , . . . , Z1 ] ≥ Zn−1 ; n ≥ 1. (7.72) A supermartingale is an integer time stochastic process {Zn ; n ≥ 1} that satisﬁes the relations E [|Zn |] < 1 ; E [Zn | Zn−1 , Zn−2 , . . . , Z1 ] ≤ Zn−1 ; n ≥ 1. (7.73) In terms of our gambling analogy, a submartingale corresponds to a game that is at least fair, i.e., where the expected fortune of the gambler either increases or remains the same. A supermartingale is a process with the opposite type of inequality.4 4 The easiest way to remember the diﬀerence between a submartingale and a supermartingale is to remember that it is contrary to what common sense would dictate. That is, a submartingale is bigger than a supermartingale. Why this terminology became standard is a mystery. 306 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES Since a martingale satisﬁes both (7.72) and (7.73) with equality, a martingale is both a submartingale and a supermartingale. Note that if {Zn ; n ≥ 1} is a submartingale, then {−Zn ; n ≥ 1} is a supermartingale, and conversely. Thus, some of the results to follow are stated only for submartingales, with the understanding that they can be applied to supermartingales by changing signs as above. Lemma 7.2, with the equality replaced by inequality, also applies to submartingales and supermartingales. That is, if {Zn ; n ≥ 1} is a submartingale, then E [Zn | Zi , Zi−1 , . . . , Z1 ] ≥ Zi ; 1 ≤ i < n, (7.74) ; 1 ≤ i < n. (7.75) and if {Zn ; n ≥ 1} is a supermartingale, then E [Zn | Zi , Zi−1 , . . . , Z1 ] ≤ Zi Equations (7.74) and (7.75) are veriﬁed in the same way as Lemma 7.2 (see Exercise 7.20). Similarly, the appropriate generalization of (7.71) is that if {Zn ; n ≥ 1} is a submartingale, then E [Zn ] ≥ E [Zi ] ; for all i, 1 ≤ i < n. (7.76) for all i, 1 ≤ i < n. (7.77) and if {Zn ; n ≥ 1} is a supermartingale, then E [Zn ] ≤ E [Zi ] ; A random walk {Sn ; n ≥ 1} with Sn = X1 + · · · + Xn is a submartingale, martingale, or supermartingale respectively for X ≥ 0, X = 0, or X ≤ 0. Also, if X has a semiinvariant moment generating function ∞ (r) for some given r, and if Zn is deﬁned as Zn = exp(rSn ), then the process {Zn ; n ≥ 1} is a submartingale, martingale, or supermartingale respectively for ∞ (r) ≥ 0, ∞ (r) = 0, or ∞ (r) ≤ 0. The next example gives an important way in which martingales and submartingales are related. Example 7.6.4 (Convex functions of martingales). Figure 7.8 illustrates the graph of a convex function h of a real variable x. A function h of a real variable is deﬁned to be convex if, for each point x1 ,...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online