Markov Martingales
1
Introduction
Besides chain, another class of fundamental stochastic processes is martingales. Developed by mathematician and probabilist J.L. Doob, the theory of martingales has become the corner stone for much of the modern probability. We will discuss the basics of discrete time martingales, but all the results and denitions can also be carried over to the continuous time martingales. The general denition of martingale involves the concept of -algebras and ltration, which are probabilistic terminologies that describe the information ow. In order to avoid such technicality, we will instead give a more restricted version of the denition. A stochastic process X = {Xn } is said to be a martingale with respect to a reference process Z = {Zn } if 1. (adaptedness) for every n, Xn is a function of {Z0 , . . ., Zn }, 2. E[Xn+1 |Zn , , Z0 ] = Xn .
Notation: When there is no ambiguity about the reference process Z, we will denote the conditional expectation E[X|Zn , . . . , Z0 ] by E[X|Fn ], and write Fn = {Z0 , . . . , Zn }.
1.1
Basic properties of martingales
We should start with a generalized tower property. Proposition 1.1. For any random variable X, and any random vectors Y and Z, E[E[X|Y, Z]|Z] = E[X|Z].
Proof: For the proof we will restrict ourselves to the case where X, Y , and Z are continuous random variables with joint density f(x, y, z). Then the joint density for Y, Z and the density for Z are fY,Z (y, z) =
R
f(x, y, z) dx,
fZ (z) =
R R
f(x, y, z) dxdy.
1
It follows that . h(y, z) = E[X|Y = y, Z = z] =
R
x
f(x, y, z) dx fY,Z (y, z)
and E[E[X|Y, Z]|Z = z] = = = = = =
R
E[h(Y, Z)|Z = z] f(x, y, z) x dx f(y|z) dy fY,Z (y, z) R R fY,Z (y, z) f(x, y, z) dx dy x fY,Z (y, z) fZ (z) R R f(x, y, z) x dydx fZ (z) R R fX,Z (x, z) x dx fZ (z) R xf(x|z)dx E[X|Z = z]
= This completes the proof.
Lemma 1.2. Suppose that X = {Xn } is a martingale with respect to a reference process Z = {Zn }. Then for any n and m 0, E[Xn+m |Zn , . . . , Z0 ] = E[Xn+m|Xn , . . . , X0 ] = Xn . In particular, E[Xn] = E[X0] for all n.
Proof: Using notation E[X|Fn ] = E[X|Zn , . . . , Z0 ], by tower property E[Xn+m |Fn ] = = = = = E[E[Xn+m |Fn+m1 ]|Fn ] E[Xn+m1 |Fn ] E[Xn+1 |Fn ] Xn .
Furthermore, since Xn is a function of {Z0 , . . . , Zn }, it follows that E[Xn+m |Xn , , X0 , Fn ] = E[Xn+m |Fn ] = Xn
2
By tower property again, we have E[Xn+m |Xn , Xn1 , , X0 ] = E [E[Xn+m |Xn , , X0 , Fn ]|Xn , , X0 ] = E[Xn |Xn , , X0 ] = Xn . The claim of E[Xn ] = E[X0 ] is trivial by tower property.
Martingales can be considered as appropriate models for fair games if Xn denotes the total wealth that a player has at the n-th play, then the players average gain of fortune at each play is zero regardless of the play history.
1.2
Examples of martingales
Example: Suppose Z = {Zn } is a sequence of iid random variables with expected value . Dene
n
Sn = x +
i=1
Zi .
and Yn = Sn n. Then Y = {Yn } is a martingale with respect to Z.
Proof. Clearly Yn is a function of {Z1 , . . . , Zn }. Moreover, E[Yn+1 |Zn , , Z0 ] = = = = E[Yn + Zn+1 |Zn , , Z0 ] Yn + E[Zn+1 |Zn , . . . , Z0 ] Yn + E[Zn+1 ] Yn .
Therefore Y is a martingale.
Example: Suppose Z = {Zn } is a sequence of iid random variables with mean 0 and variance 2 . Dene
n
Sn = x +
i=1
Zi
and
2 Yn = Sn n 2 .
3
Then Y = {Yn } is a martingale with respect to Z.
Proof.
Example (Walds martingale). Suppose Z = {Zn } is a sequence of iid random variables with moment generating function . Then for any , . Mn = 1 eSn , n () . Sn =
n
Zi
i=1
is a martingale with respect to Z.
Proof.
Example (Doobs martingale). Let Z = {Zn } be any sequence of random variables and X any random variable. Dene . Yn = E[X|Zn, , Z0 ] = E[X|Fn]. Then it is an easy consequence from tower property that Y = {Yn } is a martingale with respect to Z. Example: Suppose Z = {Zn } is a Markov chain with transition probability matrix P = [Pij ] and state space S. A function h : S R is said to be 4
harmonic if for all i S h(i) =
jS
Pij h(j)
. If h is harmonic, then Xn = h(Zn ) is a martingale with respect to Z.
Proof: Clearly Xn is a function of {Zn , . . . , Z0 }. Furthermore, for any i S, E[Xn+1 |Zn = i, Zn1 , , Z0 ] =
j
h(j)P(Zn+1 = j|Zn = i, Zn1 , , Z0 ) h(j)Pij
j
= = It follows that
h(i).
E[Xn+1 |Zn , Zn1 , , Z0 ] = h(Zn ) = Xn , and we complete the proof.
Example (Polyas Urn). Consider an urn with red and green balls. Assume that initially there is one ball of each color in the urn. Everyday, a ball is randomly chosen from the urn, and it is returned along with an extra ball of same color. Let Zn denote the number of red balls in the urn after n days, and dene . Zn Mn = . n+2 Then M = {Mn } is a martingale with respect to Z = {Zn }.
Proof: Note that at day n, there are totally n + 2 balls in the urn. Clearly Z is a Markov chain with transition probability P(Zn+1 = k + 1|Zn = k, Zn1 , , Z0 ) P(Zn+1 = k|Zn = k, Zn1 , , Z0 ) It follows that E[Zn+1 |Zn , , Z0 ] = or equivalently E[Mn+1 |Zn , , Z0 ] = Mn . Zn n + 2 Zn Zn (Zn + 1) + Zn = (n + 3), n+2 n+2 n+2 = = k , n+2 k 1 . n+2
5
Example (likelihood ratio). Let Z = {Zn } be a sequence of iid random variables with common density f (z). Suppose that h(z) is any probability density and dene . h(Zn ) h(Z0 ) . Xn = f (Zn ) f (Z0 ) Then X = {Xn } is a martingale with respect to Z = {Zn }.
Proof. Clearly Xn is a function of {Z0 , . . . , Zn }. Furthermore, E[Xn+1 |Fn ] = = = = h(Zn+1 ) Fn f(Zn+1 ) h(Zn+1 ) Xn E f(Zn+1 ) h(z) Xn f(z) dz f(z) R Xn . E Xn
Example (preliminary stochastic integral). Suppose M = {Mn } is a martingale with respective to a reference process Z = {Zn }. Further assume that process Y = {Yn } is predictable, that is, Yn is a function of {Z0 , , Zn1 }. Dene n . Xn = Yj (Mj Mj1 ) := Y M
j=1
with X0 = 0. Then X = {Xn } is a martingale with respect to Z.
Proof. Clearly Xn is a function of {Z0 , . . . , Zn }. Furthermore, E[Xn+1 |Fn ] Xn = = = = E[Xn+1 Xn |Fn ] E[Yn+1 (Mn+1 Mn )|Fn ] Yn+1 E[Mn+1 Mn |Fn ] 0.
The third equality holds because Yn+1 is a function of {Z0 , . . . , Zn }.
2
Optional Sampling Theorem
The optional sampling theorem is the fundamental result in martingale theory. Suppose X = {Xn} is a martingale with respect to Z = {Zn }. Recall the denition of stopping times. A random variable : 6
{0, 1, } {} is said to be a stopping time if for every n 0 the event { = n} is determined by {Z0 , . . . , Zn }, or we can write 1{=n} = Fn (Z0 , . . . , Zn) for some function Fn . The question of interest is that whether E[X ] = E[X0] holds or not. It is clear that this cannot be true in general. Example: (Can we beat a fair game?) Let S = {Sn } be a symmetric simple random walk on real line, starting from 0. Then S is a martingale. Dene stopping time . = inf {n 0 : Sn = 1} . Since P( < ) = 1, it follows that E[S ] = 1 = E[S0 ] = 0. However, with appropriate assumptions on the process X and the stopping time , the claim is true. We should give the simplest version of optional sampling theorems, followed by comments. A note on the notation: for two stopping times and , their minimum is denoted by . = min{, }. Theorem 2.1. (Optional Sampling Theorem). Suppose that X = {Xn } is a martingale with respect to Z = {Zn } and an arbitrary stopping time. Then the stopped process Y = {Yn } where . Yn = Xn is also a martingale with respect to Z = {Zn }. In particular, E[Xn ] = E[X0] for all n 0.
Proof: We rst show that Yn is a function of {Z0 , . . . , Zn }. Indeed
n1
Yn = Xn 1{n} +
i=0
Xi 1{=i} .
7
Observing that
n1
1{n} = 1 1{n1} = 1
i=0
1{=i}
is a function of {Z0 , . . . , Zn }, it is clear that so is Yn . Furthermore,
n
E[Yn+1 |Fn ] = = = = = This concludes the proof.
E[Xn+1 1{n+1} +
i=0
Xi 1{=i} Fn ]
n
1{n+1}E[Xn+1 |Fn ] +
i=0 n
Xi 1{=i}
1{n+1} Xn +
i=0
Xi 1{=i}
Xn Yn .
Remark 2.2. The question of interest is whether E[X ] = E[X0 ]. The answer will be armative if one can argue that E[X ] = lim E[Xn ]
n
holds. Alas, this seemingly natural equality is not true in general and some appropriate conditions need to be imposed. In the appendix a few relevant limit theorems are stated without proof. Theorem 2.1, combined with these limit theorems, is a very powerful tool of analysis.
Corollary 2.3. The equality E[X ] = E[X0] holds if either of the following conditions holds. 1. The stopping time is bounded, i.e., there exists a constant T such that P( T ) = 1. 2. is nite, i.e., P( < ) = 1, and there exists a constant K such that |Xn | K for all n.
Proof. If Assumption 1 holds, then we can let n = T and observe E[X ] = E[XT ] = E[X0 ]. If Assumption 2 holds, then by dominated convergence theorem (see appendix) E[X ] = E[lim Xn ] = lim E[Xn ] = E[X0 ].
n n
8
3
3.1
Applications of Optional Sampling Theorem
Simple random walk
Let Z = {Zn } be a sequence of iid random variables with distribution . P(Zi = 1) = p, P(Zi = 1) = 1 p = q. Dene a simple random walk starting from x: . Xn = x + 3.1.1 Gamblers ruin probability
n
Zi .
i=1
Assume that x (0, b). Dene the hitting times T0 = inf{n 0 : Xn = 0}, Tb = inf{n 0 : Xn = b}.
and we are interested in the probabilities . . h(x) = Px(T0 < Tb ), g(x) = Px (Tb < T0 ), . Let = T0 Tb. Then is a nite stopping time and h(x) + g(x) = 1.
1. p = q: In this case, X = {Xn } is a martingale with respect to Z = {Zn }. Since 0 XnT b for every n, Corollary 2.3 implies that x = E[X0 ] = E[X ] = h(x) 0 + g(x) b = x. Combined with h(x) + g(x) = 1 we have x bx , g(x) = . b b . Sn 2. p = q. Consider the process Mn = (q/p) . It is not dicult to show that M = {Mn } is a martingale with respect to Z = {Zn }. Furthermore, h(x) = 1 Mn (q/p)b for all n 0. By Corollary 2.3, (q/p)x = E[M0 ] = E[M ] = (q/p)0 h(x) + (q/p)b g(x). Combined with h(x) + g(x) = 1, we have h(x) = (q/p) (q/p) 1 (q/p)
b x b
,
g(x) =
1 (q/p)
x b
1 (q/p)
.
These results agree with those in the chapter of Simple Random Walk.
9
3.1.2
Pattern in coin tossing
Consider tossing a coin with P(H) = p and P(T ) = 1 p. What is the expected number of tosses needed for a particular pattern, such as say HT T HT HHT T T H, to appear? Of course, we can do what we have done using Markov chains. More precisely, we set up a Markov chain with nitely many states, with each state corresponds to a segment of the pattern. The state which corresponds to the full pattern is the sole absorbing state of the Markov chain. One can write down the probability transition matrix and use rst step analysis to compute the expected time from state 0 (starting point) to the absorbing state. See the chapter of Markov chains for more details. The disadvantage of this Markov chain approach is that it requires case by case calculation, at least on the surface, and a general formula seems to be a remote possibility. We will use martingale theory to solve this problem, and it will be a lot easier to write down the expected time. For illustration, we assume that the pattern is HHT T HH. This patterns total length is 6. Consider the following gambling scenario. You will toss the coin once at the end of each day. At the beginning of each day, a new gambler will join you and bet against you on the outcome of your coin toss at the end of that day. This gambler will use the following strategy. He bets 1 dollar on his rst day on the rst digit of the pattern (in this case H). If he loses, i.e., the outcome that day is T , he loses his 1 dollar and leaves the game and never comes back again. If he wins, i.e., the outcome that day is H, then his net win will be q/p dollars or his total wealth will become 1/p dollars. In this case, he will come back again the next day to bet against you. But on his second day he will bet on the 2nd digit of the pattern, which is H, and he will bet his total wealth 1/p. As his rst day, if he loses, he loses all his bet and will leave the game forever; if he wins, his total wealth will become (1/p)2. Furthermore, if he wins, he will come back next day. But on his 3rd day, he will bet all his wealth (1/p)2 on the 3rd digit of the pattern, which is T . If he loses, adios. If he wins, however, for each of his 1 dollar bet he net wins p/q dollars, and therefore his total wealth will become (1/p)2(1/q). The gambler will keep coming back until either he loses all his bet (or his wealth) or the pattern appears, in which case, he has been betting and winning for 6 days straight, and his total wealth at the end of 6th day will be (1/p)4(1/q)2 . Note that each day you will not only face a new gambler, but also some winning old gamblers too. But when the pattern appears, the game permanently ends and everyone goes home whether he is a loser or a winner. Let Xn denote the your total net winning at the end of day n. X = {Xn } 10
is a martingale since each bet is a fair game. Let be the time when the pattern rst appears. By optional sampling theorem E[Xn ] = E[X0] = 0. One can argue that we indeed have the identity (see Remark 3.1) E[X ] = lim E[Xn] = 0.
n
However, one can easily see that each of the new gamblers that started on day 1, 2, . . ., 6 was a loser and each of them lost exactly 1 dollar in total. The new gambler who started on day 5 is a winner and his net win is (1/p)4 (1/q)2 1. Each of the new gamblers that started on day 4, 3, 2 was a loser and lost 1 dollar each. The new gambler who started on day 1 is a winner for two days and his net winning is (1/p)2 1. The new gambler who started on day is also a winner with net winning (1/p) 1. It follows that X = ( 6) [(1/p)4(1/q)2 1] + 1 + 1 + 1 [(1/p)2 1] [(1/p) 1] = [(1/p)4(1/q)2 + (1/p)2 + (1/p)], and that E[] = (1/p)4 (1/q)2 + (1/p)2 + (1/p). Exercise: Argue that the expected time to the pattern HHT is (1/p)2 (1/q) while the expected time to the pattern HT H is (1/p)2(1/q) + (1/p).
Remark 3.1. Observe that there exists a constant c such that for all n |Xn | (n ) + c + c. But E[] < by the Markov chain approach (why?). Therefore it follows from dominated convergence theorem that E[X ] = E[Xn ].
4
Option pricing with binomial tree
In binomial asset pricing model, stock prices are modeled as follows. If the stock price at time n is S, then at time n + 1 the stock price will have probability p to climb to uS and probability 1 p to drop to dS. Here u and d are given constants with d < 1 < u and ud = 1. Even though the 11
binomial pricing model seems too simple compared with the real-world stock price movement, it provides a reasonable approximation in many occasions and has superior computational tractability. The main goal of this section is to illustrate the key idea of arbitrage free pricing and risk-neutral probability in the context of option pricing.
4.1
A preliminary example
In some sense, the arbitrary free pricing and risk-neutral pricing are deterministic pricing out of probabilistic models. To be more concrete, we will start with a one-period binomial model as follows. Furthermore, we assume that there is no ination 1 dollar today is 1 dollar tomorrow.
Consider one share of European call option issued at time 0 with strike price K that expires at time 1. The holder of this call option has the right, not the obligation (whence the name option), to buy one share of the stock at the strike price at the expiration date. That is, the payo from this option is C = (S1 K)+ = max{S1 K, 0} at time 1. Now the question is: what should be the price or the value of this option at time 0?
12
Remark 4.1. A share of European put option with strike price K and expiration date 1 gives the holder of the right to sell one share of stock with strike price K at the expiration date. That is, the payo of this put option is C = (K S1 )+ = max{K S1 , 0}. at time 1. Remark 4.2. The name European means that the option can only be exercised at the expiration date. On contrast, American options can be exercised at any time before or at the expiration date. The price (value) of an American option is obviously at least as much as that of its European counterpart.
Arbitrage Free Principle. Suppose that at time 0 we construct a portfolio that consists of one share of the call option and x shares of the stocks [sometimes when x > 0 it is called a long position, and when x < 0 it is called a short position]. The value of x is yet to be determined. Suppose the value or price of the option at time 0 is v. The value of this portfolio is 10x + v at time 0. Now at time 1, the value of the portfolio is either 20x + 6 if the stock price goes up to 20 or 5x if the stock price comes down to 5. However, if we pick x so that 2 x = = 0.4, 5 then the portfolio yields a deterministic value of 20x + 6 = 5x = 2 at time 1. The arbitrage free principle says that 20x + 6 = 5x or 10x + v = 2 or 4 + v = 2 or v = 2.
That is, the option is worth 2 dollar at time 1. Remark 4.3. Note that the real-life probability probabilities p and q play no roles at all in option pricing. Risk-neutral Probability. The risk-neutral probability is an (articial) probability measure, under which the expected return of the option payo equals the option price. In this example, one can nd the risk-neutral probability by solving the equations p + q = 1, or equivalently, 6p + 0q = 2
1 2 p = , q = . 3 3 13
4.2
General pricing formula
Now consider a general one-period binomial pricing model and assume that the interest rate is r. That is, one dollar at time 0 is worth R = 1 + r dollar at time 1. The European option that expires at time 1 has payo C = Cu if the stock price goes up and C = Cd if the stock price goes down. We also assume that d < R < u. Note that here we temporarily drop the assumption ud = 1 this assumption is only imposed for the sake of easy numerical implementation and has no bearing on the general pricing principle whatsoever.
Let v denote the price or the value of the option. We can construct a portfolio with 1 share of the option and x shares of stocks. The goal is to choose x so that the portfolio will have a deterministic value at time 1, that is, Cu Cd . uS x + Cu = dS x + Cd or x = uS dS Since the portfolios value at time 0 is xS + p, we must have, by arbitrage free principle, (xS + v)R = uS x + Cu which implies that v= 1 R Rd uR Cu + Cd ud ud
Dene the risk-neutral probability P by (p , q ) = Rd uR , ud ud ,
That is, under risk neutral probability measure, the stock price will have probability p to increase to uS and probability q to drop to dS. 14
There are a few properties that one can observe regarding the risk neutral probability measure 1. Under the risk neutral probability measure, the discounted stock price is a martingale. That is EP
S1 S0 = S0 . R
This can be easily veried, observing that 1 R Rd uR uS + dS ud ud = S,
2. The option price is the discounted expected payo under risk neutral probability. That is, 1 v = E P [C]. R With this price, the model is arbitrage free. 3. The real world probabilities plays no role in the pricing of options. 4. The risk neutral probability is determined by the structure of the binomial model, and is independent of the option payos. Exercise. Let v be the price for an option whose payo is C. With initial wealth v, construct a portfolio so that its value at time 1 exactly replicates the option payo C. Example: Consider the following multi-period binomial tree pricing model. Find the price of the call option with strike price K = 12 and expiration date at time 3. Interest rate is assumed to be r = 0.
15
5
Supermartingales and submartingales
Analogous to martingales, we can dene supermartingales and submartingales. A stochastic process X = {Xn } is said to be a supermartingale with respect to a reference process Z = {Zn } if 1. (adaptedness) for every n, Xn is a function of {Z0 , . . ., Zn }, 2. E[Xn+1 |Zn , , Z0 ] Xn . Submartingales are similarly dened, except that in condition 2 is replaced by . The theory of supermartingale/submartingale is also analogous to that of martingales. For example, optional sampling theorem holds with = replaced by or respectively. To avoid repetitiveness, the details are left for the students to ll in.
A
Appendix. Basic limit theorems in probability
In this appendix we state three basic limit theorems in probability theory. They are all concerned with exchanging the order of taking limit and taking expectation. Proofs can be found in any graduate level probability textbook. Theorem A.1. (Fatou Lemma) For any nonnegative random variables {Xn } E[lim inf Xn ] lim inf E[Xn].
n n
Theorem A.2. (Monotone Convergence Theorem). Suppose that 0 X1 X2 is a sequence of increasing nonnegative random variables. Then E[lim Xn ] = lim E[Xn].
n n
Theorem A.3. (Dominated Convergence Theorem). Suppose that {Xn } is a sequence of random variables such that limn Xn = X for some random variable X. If there exists a random variable Y with E[Y ] < , such that |Xn | Y for every n. Then lim E[Xn] = E[lim Xn ] = E[X].
n n
16