4606 Midterm Fall 2010
1. Fill in the spots provided below to get a transition probablity for a markov
chain with states 0,and 1that is ergodic but neither doubly stochastic nor
constant down columns.
Record this matrix in your bluebook.
With these transi
Homework 1 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, P1.1, P1.2, P1.5)
1. (P1.1)
Yes. It is possible to model it by a Markov chain. There are six states: cfw_0, 1, 2, 3, 4, 5 or more.
The respective transition matrix is
10000
Homework 11 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P8.8)
1. (P8.8)
First, we prove that Yt = tX1/t has independent increments and they are jointly normal. For
any 0 < t1 t2 tn , since Xt is a standard Brownian moti
Homework 10 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P8.4, P8.9)
1. (P8.4)
(a) Let Z denote a standard normal random variable and be its cumulative distribution
function.
P(X2 > 2) = P( 2Z > 2) = 1 ( 2) or 0.079.
(b)
Homework 7 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P5.1, P5.2)
1. (P5.1)
Method 1: AS Y is the sum of two dice, its sample space is cfw_2, 3, , 12. For any 2 y 12,
E (X |Y = y ) =
=
y
=.
2
y 1
x=1 xP(X = x, Y = y )
Midterm
1. An organization has N employees, where N is large. Each employee has 1
of 3 possible job classications and changes jobs independently according
to a Markov chain with transition probabilities
.7 .2 .1
.2 .6 .2
.1 .4 .5
What percentage of employ
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Rescale time and space so that
Time Reversed Markov Chains
Proposition:
The time-reversed markov chain Xn , Xn1 , ., X1 is markov. If Xn
has its stationary distribution, it is a time-homogeneous markov
chain with transition matrix Pji = ji Pij .
Time Reversed Markov Chains
Proposition:
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Random Walk
Xi are i.i.d. with mean 0 and variance 1.
Sn = X1 + . + Xn , n = 1, ., N
Rescale time and space so that
Stopping Times
Let Ft be a ltration. A random variable 0 is a stopping time
if cfw_ t Ft .
Examples: Let Ft be the Brownian ltration.
Any xed time.
Stopping Times
Let Ft be a ltration. A random variable 0 is a stopping time
if cfw_ t Ft .
Examples: Let Ft
Introduction
Topics:
Recurrence of Brownian Motion in one and Several
Dimensions
Introduction
Topics:
Recurrence of Brownian Motion in one and Several
Dimensions
Properties of the zero set of one dimensional Brownian Motion
Recurrence of One dimensional B
Figure : Wieners 1948 book, Cybernetics, was a sensation
From Wikiquote.org
When he and his family moved to a new house a few blocks away,
his wife gave him written directions on how to reach it, since she
knew he was absent-minded. But when he was leavin
Generalities: E x cfw_h(X0 , X1 , . . . ) means E cfw_h(X0 , X1 , . . . )|X0 = x.
Ergodic: positive recurrent, irreducible aperiodic. Ergodic chains have stationary distributions and a Law of Large Numbers applies.
Matrix inverse:
a
c
b
d
1
is the matrix
4635 Midterm, Spring 2013
March 21, 2013
Rules:
1. All electronic devices out of sight, no calculators , no phones no nothing.
2. Spread out, leave at least one empty seat between you and your neighbor.
Do not share space for scratch paper with your neigh
4606 Solved Problems
M Hogan
March 1, 2013
1
Elementary Properties
1. Fill in the spots provided below to get a transition probablity for a markov
chain with states 0, and 1and that is ergodic but neither doubly stochastic
nor constant down columns.
?
?
?
Exponential Random Variables
A random variable is exponential with parameter if its survival
function is
S (x ) = e x , x > 0
Exponential Random Variables
A random variable is exponential with parameter if its survival
function is
S (x ) = e x , x > 0
It
Stochastic Processes for Finance
October 22, 2012
1
Martingales, Submartingales, Supermartingales
Let Xi , i = 1, . . . be a sequence of random variables. The sequence is a martingale if Ecfw_Xn+1 |X1 , . . . , Xn = Xn . In principal this conditional exp
Homework 6 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P3.8 a,b.)
1. (P3.8)
(a) By solving
A = 0
3791
with constraint that 1 = 1 we have = ( 38 , 38 , 38 , 2 ).
(b) When the initial state is 1, the amount of time until
Homework 4 Solution
Xuan
1. If Xi s are independent Exp(i ) random variables, then the distribution of X1 |X1 < X2 is
Exp(1 + 2 ).
For any t > 0,
P (X1 > t|X1 < X2 ) =
Since
P (X1 < X2 ) =
0 x1
P (t < X1 < X2 ) =
t
It follows
P (t < X1 < X2 )
.
P (X1 <
Homework 8 Solution
Xuan
1. Suppose that cfw_Xi ; i = 1, . . . , n is a martingale, and cfw_Yi ; i = 1, . . . , n is a submartingale. If
Yn Xn show that Yi Xi for all 1 i n.
(Assume Xi and Yi are adapted to the same ltration.) We prove it by contradiction
Xiaoou Li
Solution to W4635 HW2
1. First, we prove that the nite eld can be generated by a partition of the sample
space . Let set Ax = AF A, where x , and F is the eld we are interested in.
Then it is easy to prove that for x = y , either Ax = Ay or Ax =
Homework 9 Solution
Xuan
(Gregory F. Lawler, Introduction to Stochastic Processes, 2nd ed. P5.14)
1. (P5.14)
For any n N, we have
T n
E (ST n 1) = E
Xi
= E (T n) E (X ) .
i=1
Since = EX < 0 and ST n 0, it follows that
E (T n) =
E (ST n 1)
1
1
=
.
E (X )
E
Xiaoou Li
Solution to W4635 HW5
1. An irreducible and aperiodic nite state Markov chain is positive recurrent if and only
if it has invariant distribution. Thus, we only need to nd the condition when the
Markov chain we are interested in has an invariant
Xiaoou Li
Solution to W4635 HW3
1. Xn is negative, so |Xn | = Xn . Xn is a submartingale, so E (Xn ) is increasing,
E (|Xn |) = E (Xn ) is decreasing.
2. First we prove that E (|X |) =
0
P (|X | > x)dx. In fact, we have
E (|X |) = E
1cfw_0<x<|X | dx =
0
P
Xiaoou Li
Solution to W4635 HW1
9. Zn+1 Xn+1 E (Zn+1 |Fn ) E (Xn+1 |Fn ) = Xn . Similarly, E (Zn+1 |Fn ) Yn , so
E (Zn+1 |Fn ) min(Xn , Yn ) = Zn , which means that Zn is a supermartingale.
23. : Suppose T is bounded by M . From 20, we know that cfw_XT n
Distribution at Time n
Proposition:
Suppose we have a nite state markov chain with states
S = cfw_1, ., n, transition matrix P and initial distribution . Then
n = P n .
the distribution of Xn i given by
Distribution at Time n
Proposition:
Suppose we have
Ergodic Markov Chains
The chain is Ergodic if it is irreducible, aperiodic,and positive
recurrent
Ergodic Markov Chains
The chain is Ergodic if it is irreducible, aperiodic,and positive
recurrent
all of those are class properties
Ergodic Markov Chains
The