Recurrence and Transience
For n Z+ , dene:
( n)
fi,j = P (Xn = j, Xn1 = j, . . . , X2 = j, X1 = j |X0 = i) i, j.
This is the probability that starting from state i, the rst visit to state j occurs at time n.
(1)
(n)
Note that fi,j = Pi,j . More generally,
STAT 433/833 PROBLEM SET 1
1. (Exercise 1.14, page 39 of the textbook) Let
cfw_1, 2, 3, 4, 5 with transition matrix
0 1/2
0
0
0
0
P =
1
0
1/2
0
cfw_Xn n=0,1,. be a Markov chain on state space
1/2
0
0
0
0
0
1/5
2/5
0
0
0
4/5
3/5 .
0
1/2
(a) Is this chain i
STAT 433/833 PROBLEM SET 2
1. (Resnick, pp. 151-152) The business of a restaurant fluctuates in successive years between three
states: 0(bankruptcy), 1(verge of bankruptcy) and 2(solvency). The transition matrix giving the
probabilities of evolving from s
STAT 433/833 PROBLEM SET 3
1. (Exercise 3.7, page 84 of the textbook.) Let Xt be an irreducible, continuous-time Markov chain.
Show that for each i, j and every t > 0,
P (Xt = j|X0 = i) > 0.
2. (Exercise 3.12, page 85 of the textbook.) Consider the linear
CMTH500. Introduction to Stochastic Processes. Summer 2016.
Due: Due July 29th, during the lecture.
Name:
Note: include details and intermediate steps in your answers.
Question 1
A stock, priced at $ 100 at spot(t = 0), can go up to $ 120 or down to $ 90
STAT 433/833 - Stochastic Processes
Test 1
Friday, October 11, 2013
6:30 p.m. - 8:00 p.m.
Instructors: S. Drekic & Y. Shen
Total Marks = 35
1. Consider a discrete-time Markov chain with transition probability matrix given by
P=
0
1
0
1
1
,
1
where 0 < 1 a
STAT 433/833 Problem Set 2
1. Exercise 3.7, pg. 84 of the textbook.
2. Exercise 3.11, pg. 84 of the textbook.
3. Exercise 3.12, pg. 85 of the textbook.
In addition, add in the following part: Suppose now that = 0, X (0) = 1, and that at
(deterministic) ti
STAT 433/833 Problem Set 1
1. Exercise 1.7, pg. 36 of the textbook.
2. Exercise 2.2, pg. 57 of the textbook.
3. Exercise 2.4, pg. 58 of the textbook.
4. Exercise 2.5, pg. 58 of the textbook.
5. Exercise 2.7, pg. 59 of the textbook.
6. Suppose that the num
Discrete-time Markov Chains
Generally speaking, cfw_X (t), t T is called a stochastic process if X (t) is a random
variable (or random vector) for any xed t T . T is referred to as the index set, and
is often interpreted in the context of time. As such,
Understanding the Limiting Behaviour of Markov Chains
As the previous three numerical examples show, there is variation in the limiting behaviour of a MC. In particular, it is worthwhile to investigate whether a set of conditions
exist which ensure a MC e
Theory of Algorithms
Homework Set #2, due Wed. Feb. 10
1. In class, we found that the solution to the mergesort recurrence
T (n) = T
n
2
+T
n
2
+ n; T (1) = 0
satised f (n) = (n log n). Find an expression for the exact value of T (n). (Hint:
generate the