Math 632 Fall 2011
Homework 2 Solutions
1. 2.2, page 147
If the original Markov chain had transition probabilities pij then the new one will have
p(i,j ),(k,l) = P (Xn+1 = k, Yn+1 = l|Xn = i, Yn = j ) =
P (Xn+1 = k, Yn+1 = l, Xn = i, Yn = j )
P (Xn = i, Y
632 Introduction to Stochastic Processes Spring 2004 Final Exam
Instructions: Show calculations and give concise justifications for full credit. Points add up to 200. 1. Four cages labeled A, B, C and D are connected by tubes as indicated in the diag
632 Introduction to Stochastic Processes Spring 2004 Midterm Exam II
Instructions: Show calculations and give concise justifications for full credit. Points add up 100. 1. (20 pts) Eileen is catching fish at the Poisson rate of per hour. Each fish i
Math 632 Fall 2011
Homework 3 Solutions
1. 2.16 a), p. 151
One can do this with the methods learned about absorption, but its a lot easier than that.
Note that if we are at state 1 or 2 then with probability 1/2 we go to 0 and with probability
1/2 we go t
Math 632 Fall 2011
Homework 4 Solutions
1. 3.4, p. 282
Although the problem does not state this, we can assume that pk = P (N = k ). Then the
distribution function of SN = X1 + + XN is
P (SN x) =
P (SN x, N = k ) =
k=0
pk F k (x).
P (Sk x)P (N = k ) =
k=0
Math 632 Fall 2011
Homework 5 - Solutions
1. 4.3, page 350
For x 1, let I (x) satisfy
I (x)
1
1
dt = x ln(I (x) = x I (x) = ex .
t
Then, by the work in section 4.3, we know that if N is a unit rate homogeneous Poisson
process with points n , then for t 1
Math 632 Fall 2011
Homework 6
Due: December 15, beginning of the class. Late homework will not be accepted.
1. 5.11, page 453
2. 5.15, page 454 (Do not do the part about binary splitting.)
3. 5.16, page 455
4. Consider the continuous time Markov chain wit
Math 632 Fall 2011
Homework 2 Hints
3. It is enough to show a Markov chain and a function f for which f (Xn ) is not a Markov chain.
According to the hint of the textbook f will not be one-to-one.
How to show that a process is not Markov chain? If its Mar
Math 632 Spring 2017
Homework 10
Due date: Friday April 14th
Please read the instructions/suggestions on the course webpage.
In particular, remember that credit comes from your reasoning, not
your numerical answer.
Hand in the following problems:
Exercise
Math 632 Spring 2017
Homework 9
Due date: Friday April 7th
Please read the instructions/suggestions on the course webpage.
In particular, remember that credit comes from your reasoning, not
your numerical answer.
Hand in the following problems:
Exercises
Math 632 - Spring 2017
March 1, 2017, 19:15-20:45
First Midterm Exam
Name
There are 4 problems on the exam, each of them have multiple parts.
Budget your time wisely! Read the problems carefully!
You are not allowed to use a calculator or other handhel
Since we will follow the book quite closely for Chapter 2 and 3, I will not provide
you with detailed lecture notes, but just list down what we cover in class (plus
some extra).
Friday, March 3rd
Exponential random variables
From the book:
the probabilit
Math 632 Fall 2011
Homework 1 Solutions
1. Suppose that A1 , A2 , . . . are events with A1 A2 A3 . . . . Using the basic properties of
probability measures show that
lim P (An ) = P ( An ).
n=1
n
(See the basic probability notes posted online for the prop
Math 632 Fall 2011
Homework 0 Solutions
1. You play a dice game with two dice. If you roll at most 5 (i.e. the sum of the two numbers is
2, 3, 4 or 5) then you win, if you roll 7 or 10 then you lose, otherwise you have to roll again.
(a) What is the proba
632 Introduction to Stochastic Processes Fall 2003 Midterm Exam II
Instructions: Justify non-obvious statements for full credit. Quote results from class accurately. Points add up 100. 1. (a) (20 pts) Let N be a rate Poisson process. Find the condit
632 Introduction to Stochastic Processes Spring 2007 Midterm Exam 2
Instructions: Show calculations and give concise justifications for full credit. Points add up to 100.
In-Class Part
1. Consider a homogeneous rate Poisson point process on the no
632 Introduction to Stochastic Processes Spring 2007 Final Exam
Instructions: Show calculations and give concise justifications for full credit. Points add up to 200. 1. Born again branching process. Let {pk }0k< be the offspring distribution of a br
632 Introduction to Stochastic Processes Spring 2007 Midterm Exam 1
Instructions: Show calculations and give concise justifications for full credit. Points add up to 100.
In-Class Part
1. Consider the Markov chain transition matrix 1/3 1/2 P= 1/4
632 Introduction to Stochastic Processes Fall 2003 Final Exam
Instructions: Justify non-obvious statements for full credit. Quote results from class accurately. Points add up to 200. 1. Consider the discrete-time Markov chain with state space S = Z+
632 Introduction to Stochastic Processes Fall 2002 Midterm Exam II Instructions: Hand in problem 1 for 50 points, problem 2 for 30 points, and one other problem for 20 points. Show calculations and justify nonobvious statements for full credit. 1. Fi
632 Introduction to Stochastic Processes Fall 2002 Final Exam Instructions: Hand in all four problems. The points add up to 100. Show calculations and justify non-obvious statements for full credit. 1. (a) (15 pts) Suppose Xn is a discrete-time Marko
632 Introduction to Stochastic Processes Fall 2003 Midterm Exam I
Instructions: Show calculations and justify non-obvious statements for full credit. When you quote a result proved in class, state the hypotheses and conclusion clearly, and justify wh
632 Introduction to Stochastic Processes Spring 2004 Midterm Exam I
Instructions: Show calculations and give concise justifications for full credit. Don't forget that theoretical ideas can help avoid tricky computations. The points add up 100. Genera
632 Introduction to Stochastic Processes Fall 2002 Midterm Exam I
Instructions: Hand in problem no. 1 for 60 points, and one other problem for 40 points. Show calculations and justify non-obvious statements for full credit. General notation: Px (A) i
Continuous Time Markov Chains
Monday, March 27th
Continuous time Markov chains incorporate the structure of a discrete time Markov chain
in a continuous time Markov process. Substantially, when thinking on a Continuous time
Markov chain you should think o