Discrete-time stochastic processes

# We wish to nd pr sn in the absence of a lower

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: there is a real number c with the property that h(x1 ) + c(x − x1 ) ≤ h(x) for all x. Geometrically, this says that every tangent to h(x) lies on or below h(x). If h(x) has a derivative at x1 , then c is the value of that derivative and h(x1 ) + c(x − x1 ) is the tangent line at x1 . If h(x) has a discontinuous slope at x1 , then there might be many choices for c; for example, h(x) = |x| is convex, and for x1 = 0, one could choose any c in the range −1 to +1. A simple condition that implies convexity is a nonnegative second derivative everywhere. This is not necessary, however, and functions can be convex even when the ﬁrst derivative does not exist everywhere. For example, the function ∞ (r) in Figure 7.5 is convex even though it blows up at r = r+ . Jensen’s inequality states that if h is convex and X is a random variable with an expectation, then h(E [X ]) ≤ E [h(X )]. To prove this, let x1 = E [X ] and choose c so that h(x1 ) + c(x − 7.7. STOPPED PROCESSES AND STOPPING TIMES h(x) 307 h(x1 ) + c(x − x1 ) c = h0 (x1 ) x1 Figure 7.8: Convex functions: For each x1 , there is a value of c such that, for all x, h(x1 ) + c(x − x1 ) ≤ h(x). If h is diﬀerentiable at x1 , then c is the derivative of h at x1 . x1 ) ≤ h(x). Using the random variable X in place of x and taking expected values of both sides, we get Jensen’s inequality. Note that for any particular event A, this same argument applies to X conditional on A, so that h(E [X | A]) ≤ E [h(X ) | A]. Jensen’s inequality is very widely used; it is a minor miracle that we have not required it previously. Theorem 7.3. If h is a convex function of a real variable, {Zn ; n ≥ 1} is a martingale, and E [|h(Zn )|] &lt; 1 for al l n, then {h(Zn ); n ≥ 1} is a submartingale. Proof: For any choice of z1 , . . . , zn−1 , we can use Jensen’s inequality with the conditioning probabilities to get E [h(Zn )|Zn−1 =zn−1 , . . . , Z1 =z1 ] ≥ h(E [Zn | Zn−1 =zn−1 , . . . , Z1 =z1 ]) = h(zn−1 ). (7.78) For any choice of numbers h1 , . . . , hn−1 in the range of the function h, let z1 , . . . , zn−1 be arbitrary numbers satisfying h(z1 )=h1 , . . . , h(zn−1 )=hn−1 . For each such choice, (7.78) holds, so that E [h(Zn ) | h(Zn−1 )=hn−1 , . . . , h(Z1 )=h1 ] ≥ h(E [Zn | h(Zn−1 )=hn−1 , . . . , h(Z1 )=h1 ]) = h(zn−1 ) = hn−1 . (7.79) completing the proof. Some examples of this result, applied to a martingale {Zn ; n ≥ 1}, are as follows: {|Zn |; n ≥ 1} is a submartingale 2 {Zn ; £ 2§ n ≥ 1} is a submartingale if E Zn &lt; 1 (7.80) (7.81) {exp(rZn ); n ≥ 1} is a submartingale for r such that E [exp(rZn )] &lt; 1. (7.82) A function of a real variable h(x) is deﬁned to be concave if −h(x) is convex. It then follows from Theorem 7.3 that if h is concave and {Zn ; n ≥ 1} is a martingale, then {h(Zn ); n ≥ 1} is a supermartingale (assuming that E [|h(Zn )|] &lt; 1). For example, if {Zn ; n ≥ 1} is a positive martingale and E [| ln(Zn )|] &lt; 1, then {ln(Zn ); n ≥ 1} is a supermartingale. 7.7 Stopped processes and stopping times The discussion of stopping times in Section 7.5.1 applies to arbitrary integer time processes {Zn ; n ≥ 1} as well as to IID sequences. Recall that a collection T of stopping nodes for a 308 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES sequence {Zn ; n ≥ 1} of rv’s is a collection of initial segments such that no initial segment in T is an initial segment of any other segment in T . For any sample sequence which has an initial segment in T , the stopping time for that sample sequence is the length of that initial segment. If the set of sample sequences containing an initial segment in T has probability 1, then the stopping time N is a random variable, and otherwise it is a defective random variable. For some of the results to follow, it is unimportant whether N is a random variable or a defective random variable (i.e., whether or not the process stops with probability 1). If it is not speciﬁed whether N is a random variable or a defective random variable, we refer to the stopping time as a possibly defective stopping time; we consider N to take on the value 1 if the process does not stop. Given a possibly defective stopping time N for a process {Zn ; n ≥ 1}, the corresponding ∗ ∗ stopped process is deﬁned as the process {Zn ; n ≥ 1} in which Zn = Zn for n ≤ N and ∗ = Z for n &gt; N . Zn N As an example, suppose Zn models the fortune of a gambler at the completion of the nth trial of some game, and suppose the gambler then modiﬁes the game by deciding to stop gambling under some given circumstances (i.e., at the stopping time). Thus, after stopping, the fortune remains constant, so the stopped process models the gambler’s fortune in time, including the eﬀect of the stopping time. As another example, consider a random walk with a positive and negative threshold, and consider the process to stop after reaching or crossing a threshold. The stopped process t...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online