Recurrence and Transience
For n Z+ , dene:
fi,j = P (Xn = j, Xn1 = j, . . . , X2 = j, X1 = j |X0 = i) i, j.
This is the probability that starting from state i, the rst visit to state j occurs at time n.
Note that fi,j = Pi,j . More generally,
CMTH500. Introduction to Stochastic Processes. Summer 2016.
Due: Due July 29th, during the lecture.
Note: include details and intermediate steps in your answers.
A stock, priced at $ 100 at spot(t = 0), can go up to $ 120 or down to $ 90
STAT 433/833 - Stochastic Processes
Friday, October 11, 2013
6:30 p.m. - 8:00 p.m.
Instructors: S. Drekic & Y. Shen
Total Marks = 35
1. Consider a discrete-time Markov chain with transition probability matrix given by
where 0 < 1 a
STAT 433/833 Problem Set 2
1. Exercise 3.7, pg. 84 of the textbook.
2. Exercise 3.11, pg. 84 of the textbook.
3. Exercise 3.12, pg. 85 of the textbook.
In addition, add in the following part: Suppose now that = 0, X (0) = 1, and that at
STAT 433/833 Problem Set 1
1. Exercise 1.7, pg. 36 of the textbook.
2. Exercise 2.2, pg. 57 of the textbook.
3. Exercise 2.4, pg. 58 of the textbook.
4. Exercise 2.5, pg. 58 of the textbook.
5. Exercise 2.7, pg. 59 of the textbook.
6. Suppose that the num
Discrete-time Markov Chains
Generally speaking, cfw_X (t), t T is called a stochastic process if X (t) is a random
variable (or random vector) for any xed t T . T is referred to as the index set, and
is often interpreted in the context of time. As such,
Understanding the Limiting Behaviour of Markov Chains
As the previous three numerical examples show, there is variation in the limiting behaviour of a MC. In particular, it is worthwhile to investigate whether a set of conditions
exist which ensure a MC e
Theory of Algorithms
Homework Set #2, due Wed. Feb. 10
1. In class, we found that the solution to the mergesort recurrence
T (n) = T
+ n; T (1) = 0
satised f (n) = (n log n). Find an expression for the exact value of T (n). (Hint: