MATH 56A SPRING 2008
STOCHASTIC PROCESSES
65
2.2.5. proof of extinction lemma. The proof of Lemma 2.3 is just like the proof of the lemma I did on Wednesday. It goes like this. Suppose that a is the smallest positive solution of the equation a = (a). Then
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
31
1.3. Invariant probability distribution.
Denition 1.4. A probability distribution is a function
: S [0, 1]
from the set of states S to the closed unit interval [0, 1] so that
(i) = 1.
iS
is:
When the set of s
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
45
1.6. The substo chastic matrix Q. On Monday, I used the MouseCat-Cheese problem to explain the use of the matrix Q. On Wednesday,
I explained this further and applied it to the Leontief model.
1.6.1. mouse-cat-
48
FINITE MARKOV CHAINS
1.6.2. expected time. Today I did a more thorough explanation of the expected time until we reach a recurrent state. I started by reviewing the basics of substochastic matrices. Denition 1.20. A substochastic matrix is a square mat
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
51
1.6.3. Leontief model. The nal example is the Leontief economic model.
In this model, the numbers in a substochastic matrix are interpreted
as being the input requirements of several industries or factories. I
Math 56a: Introduction to Stochastic Processes and
Models
A stochastic process is a random process which evolves with time.
The basic model is the Markov chain. This is a set of states together
with transition probabilities from one state to another. For
MATH 56A: STOCHASTIC PROCESSES HOMEWORK From the syllabus: There will be weekly homework. The rst HW might have the problem: Using the SIR model, prove that the number of infected reaches its highest point when the size of the susceptible population reach
MATH 56A: STOCHASTIC PROCESSES HOMEWORK From the syllabus: There will be weekly homework. Students are encouraged to work on their homework in groups and to access all forms of aid including expert advice, internet and other resources. The work you hand i
MATH 56A: STOCHASTIC PROCESSES HOMEWORK From the syllabus: There will be weekly homework. Students are encouraged to work on their homework in groups and to access all forms of aid including expert advice, internet and other resources. The work you hand i
MATH 56A: STOCHASTIC PROCESSES HOMEWORK From the syllabus: There will be weekly homework. Students are encouraged to work on their homework in groups and to access all forms of aid including expert advice, internet and other resources. The work you hand i
MATH 56A: STOCHASTIC PROCESSES HOMEWORK From the syllabus: There will be weekly homework. Students are encouraged to work on their homework in groups and to access all forms of aid including expert advice, internet and other resources. The work you hand i
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
209
9.5. Black-Scholes Equation. On the last day of class I derived the Black-Scholes equation and solved it in the case when the drift and volatility of the stock is constant. 9.5.1. the question. First I explain
202
STOCHASTIC INTEGRATION
9.4. Extensions of Its formula. First I explained the idea of coo variation and the product rule. Then I used this to derive the second and third version of Its formula. o 9.4.1. covariation. Denition 9.25. The covariation proce
102
OPTIMAL STOPPING TIME
4. Optimal Stopping Time
4.1. Denitions. On the rst day I explained the basic problem using
one example in the book. On the second day I explained how the
solution to the problem is given by a minimal superharmonic and
how you co
108
OPTIMAL STOPPING TIME
4.4. Cost functions. The cost function g (x) gives the price you must
pay to continue from state x. If T is your stopping time then XT is
your stopping state and f (XT ) is your payo. But your cost to play to
that point was
cost
114
OPTIMAL STOPPING TIME
4.5. discounted payo. Here we assume that the payo is losing
value at a xed rate so that after T steps it will only be worth T f (x)
where is the discount rate, say = .90. If there is no cost, the value
function will satisfy the
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
157
8. Brownian Motion We will spend 5 days on this chapter and cover the following topics: 8.1. introduction 8.2. strong Markov property and the reection principle 8.3. fractal dimension of the zero set 8.4. the
162
BROWNIAN MOTION
8.2. strong Markov prop erty and reection principle. These are concepts that you can use to compute probabilities for Brownian motion. 8.2.1. strong Markov property. a) Brownian motion satises the Markov property : For t > s, Xt depend
168
BROWNIAN MOTION
8.3. fractal dimension of zero set. Today we calculated the fractal
dimension of the zero set.
First, I went over the calculation from last time. Suppose that Xt is
Brownian motion which is centered with drift zero (CDZ). Then
P(Xs = 0
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
179
8.5. Recurrence and transience. The question is: Does Brownian
motion make particle go o to ?
(1)
(2)
(3)
(4)
Set up the probabilistic equation
Convert to a dierential equation by time reversal
Solve the diere
MATH 56A SPRING 2008 STOCHASTIC PROCESSES
KIYOSHI IGUSA
Contents 9. Stochastic integration 9.0. Concept, Its formula, quadratic variation o 9.1. Discrete stochastic integral 185 185 189
Date : April 14, 2008.
184
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
MATH 56A SPRING 2008
STOCHASTIC PROCESSES
197
9.3. Its formula. First I stated the theorem. Then I did a simple o example to make sure we understand what it says. Then I proved it. The key point is Lvys theorem on quadratic variation. e 9.3.1. statement o
202
STOCHASTIC INTEGRATION
9.4. Extensions of Its formula. First I explained the idea of coo
variation and the product rule. Then I used this to derive the second
and third version of Its formula.
o
9.4.1. covariation.
Denition 9.25. The covariation proce
MATH 56A: STOCHASTIC PROCESSES HOMEWORK
Homework 5 Martingales These problems are due Wednesday, March 26. Answers will be posted the following week. First problem: Suppose that X1 , X2 , are i.i.d. random variables with expected value = E(Xi ). a) Show t
MATH 56A: STOCHASTIC PROCESSES HOMEWORK
Homework 6 Renewal These problems are due Thursday, April 3. Answers will be posted the following week. First problem: Suppose that we have a renewal process with a uniform distribution f (t) = 1/10 0 if 0 < t 10 ot
MATH 56A: FALL 2006 HOMEWORK AND ANSWERS Math 56a: Homework 4 4. Homework 4 (Chap 2) p. 59 #2.7, 8, 18 2.7. Are these positive recurrent, null recurrent or transient? (a) This process is null recurrent: x+1 1 , p(x, x + 1) = x+2 x+2 In this process, you k
MATH 56A: FALL 2006
HOMEWORK AND ANSWERS
Math 56a: Homework 5
1. Homework 5 (Chap 3)
p. 84 #3.5, 8, 11, 12
3.5. Let Xt be a Markov chain with state space S = cfw_1, 2 and rates (1, 2) = 1, (2, 1) = 4.
Find Pt .
The innitesmal generator is
1 1
4 4
This mat
6. Homework 6 (Chap 4) p. 98 #4.1, 2, 3, 4 4.1. We have a simple random walk with absorbing walls on cfw_0, 1, 2, , 10 with payo function: x : 0 1 2 3 4 5 6 7 8 9 10 f (x) : 0 2 4 3 10 0 6 4 3 3 0 Find the optimal stopping time rule and the value function
5. Homework 7 (Chap 5) p. 125 #5.2,5,7,15 5.2. If Xt is a Poisson process with = 1 then nd E(X1 | X2 ), E(X2 | X1 ). By the denition of a Poisson process, E(X2 X1 ) = E(X1 ) = = 1. Also, X2 X1 is independent of X1 . So, E(X2 | X1 ) = E(X2 X1 | X1 ) + E(X1
MATH 56A: STOCHASTIC PROCESSES
ANSWERS TO HOMEWORK 2006
Math 56a: Homework 8, Chap 6: Renewal
8. Time is money (Chap 6)
M/G/1 queueing is explained on p. 148.
8.1. The insurance company. We have an insurance company which starts with a certain
amount of c