Homework 1 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, P1.1, P1.2, P1.5)
1. (P1.1)
Yes. It is possible to model it by a Markov chain. There are six states: cfw_0, 1, 2, 3,
4606 Midterm Fall 2010
1. Fill in the spots provided below to get a transition probablity for a markov
chain with states 0,and 1that is ergodic but neither doubly stochastic nor
constant down columns.
Homework 11 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P8.8)
1. (P8.8)
First, we prove that Yt = tX1/t has independent increments and they are jointly normal. For
Homework 2 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P1.14, P2.2, P2.5.)
1. (P1.14)
(a) Yes, the chain is irreducible. Since p(5, 5) =
1
2
> 0, it is aperiodic.
(
Midterm
1. An organization has N employees, where N is large. Each employee has 1
of 3 possible job classications and changes jobs independently according
to a Markov chain with transition probabiliti
Homework 7 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P5.1, P5.2)
1. (P5.1)
Method 1: AS Y is the sum of two dice, its sample space is cfw_2, 3, , 12. For any 2 y
Homework 10 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P8.4, P8.9)
1. (P8.4)
(a) Let Z denote a standard normal random variable and be its cumulative distribution
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1
Time Reversed Markov Chains
Proposition:
The time-reversed markov chain Xn , Xn1 , ., X1 is markov. If Xn
has its stationary distribution, it is a time-homogeneous markov
chain with transition matrix
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1
Stopping Times
Let Ft be a ltration. A random variable 0 is a stopping time
if cfw_ t Ft .
Examples: Let Ft be the Brownian ltration.
Any xed time.
Stopping Times
Let Ft be a ltration. A random variab
Introduction
Topics:
Recurrence of Brownian Motion in one and Several
Dimensions
Introduction
Topics:
Recurrence of Brownian Motion in one and Several
Dimensions
Properties of the zero set of one dime
Figure : Wieners 1948 book, Cybernetics, was a sensation
From Wikiquote.org
When he and his family moved to a new house a few blocks away,
his wife gave him written directions on how to reach it, sinc
Generalities: E x cfw_h(X0 , X1 , . . . ) means E cfw_h(X0 , X1 , . . . )|X0 = x.
Ergodic: positive recurrent, irreducible aperiodic. Ergodic chains have stationary distributions and a Law of Large Nu
4635 Midterm, Spring 2013
March 21, 2013
Rules:
1. All electronic devices out of sight, no calculators , no phones no nothing.
2. Spread out, leave at least one empty seat between you and your neighbo
4606 Solved Problems
M Hogan
March 1, 2013
1
Elementary Properties
1. Fill in the spots provided below to get a transition probablity for a markov
chain with states 0, and 1and that is ergodic but nei
Exponential Random Variables
A random variable is exponential with parameter if its survival
function is
S (x ) = e x , x > 0
Exponential Random Variables
A random variable is exponential with paramet
Stochastic Processes for Finance
October 22, 2012
1
Martingales, Submartingales, Supermartingales
Let Xi , i = 1, . . . be a sequence of random variables. The sequence is a martingale if Ecfw_Xn+1 |X1
Homework 6 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P3.8 a,b.)
1. (P3.8)
(a) By solving
A = 0
3791
with constraint that 1 = 1 we have = ( 38 , 38 , 38 , 2 ).
(b)
Homework 5 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P3.2, P3.5.)
1. (P3.1)
(a) Since Nt P P ( = 4),
P (N1 < 2) =P (N1 = 0) + P (N1 = 1)
=e1 (1 + 4) = 5e4 .
(b)
P
Homework 4 Solution
Xuan
1. If Xi s are independent Exp(i ) random variables, then the distribution of X1 |X1 < X2 is
Exp(1 + 2 ).
For any t > 0,
P (X1 > t|X1 < X2 ) =
Since
P (X1 < X2 ) =
0 x1
P (t
Homework 8 Solution
Xuan
1. Suppose that cfw_Xi ; i = 1, . . . , n is a martingale, and cfw_Yi ; i = 1, . . . , n is a submartingale. If
Yn Xn show that Yi Xi for all 1 i n.
(Assume Xi and Yi are adap
Xiaoou Li
Solution to W4635 HW2
1. First, we prove that the nite eld can be generated by a partition of the sample
space . Let set Ax = AF A, where x , and F is the eld we are interested in.
Then it i
Homework 9 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P5.14)
1. (P5.14)
For any n N, we have
T n
E (ST n 1) = E
Xi
= E (T n) E (X ) .
i=1
Since = EX < 0 and ST n 0
Xiaoou Li
Solution to W4635 HW5
1. An irreducible and aperiodic nite state Markov chain is positive recurrent if and only
if it has invariant distribution. Thus, we only need to nd the condition when
Xiaoou Li
Solution to W4635 HW3
1. Xn is negative, so |Xn | = Xn . Xn is a submartingale, so E (Xn ) is increasing,
E (|Xn |) = E (Xn ) is decreasing.
2. First we prove that E (|X |) =
0
P (|X | > x)d
Xiaoou Li
Solution to W4635 HW1
9. Zn+1 Xn+1 E (Zn+1 |Fn ) E (Xn+1 |Fn ) = Xn . Similarly, E (Zn+1 |Fn ) Yn , so
E (Zn+1 |Fn ) min(Xn , Yn ) = Zn , which means that Zn is a supermartingale.
23. : Supp
Distribution at Time n
Proposition:
Suppose we have a nite state markov chain with states
S = cfw_1, ., n, transition matrix P and initial distribution . Then
n = P n .
the distribution of Xn i given
Ergodic Markov Chains
The chain is Ergodic if it is irreducible, aperiodic,and positive
recurrent
Ergodic Markov Chains
The chain is Ergodic if it is irreducible, aperiodic,and positive
recurrent
all