MTHE/STAT455, STAT855
Fall 2013
Stochastic Processes
Assignment 5
Due Friday, Nov.29
1. Consider a continuous time Markov chain with state space S = cfw_1, 2 and generator
matrix
G=
,
where > 0 and > 0.
(a) Show by induction that
Gn =
(1)n ( + )n1 (1)

MTHE/STAT455, STAT855
Fall 2013
Stochastic Processes
Assignment 4
Due Wednesday, Nov.20
1. Let N1 = cfw_N1 (t) : t 0 and N2 = cfw_N2 (t) : t 0 be independent Poisson processes
with rates 1 and 2 , respectively, and let N = cfw_N (t) : t 0, where N (t) = N

MTHE/STAT455, STAT855
Fall 2013
Stochastic Processes
Assignment #2
Due Wednesday, Oct.16
Starred questions are for 855 students only.
1. In the following we assume that cfw_Xn : n 0 is a time-homogeneous Markov chain.
(a) The Markov property does not impl

MTHE/STAT455, STAT855
Fall 2013
Stochastic Processes
Assignment #3
Due day, Nov.6
Starred questions are for 855 students only.
1. This problem refers back to Problem 3 on Assignment 2. In part(b) of that problem
you showed that
uj
(1)
gij (n) = lji (n)
ui

MTHE/STAT455, STAT855 - Final Exam, 2013
Page 1 of 4
QUEENS UNIVERSITY
DEPARTMENT OF MATHEMATICS AND STATISTICS
FACULTY OF ARTS AND SCIENCE
MTHE/STAT455, STAT855 FALL 2013, FINAL EXAM
7:00PM, DECEMBER 4, 2013
GLEN TAKAHARA
Instructions:
Proctors are unab

MTHE/STAT455, STAT855 - Final Exam, 2012
Page 1 of 4
QUEENS UNIVERSITY
DEPARTMENT OF MATHEMATICS AND STATISTICS
FACULTY OF ARTS AND SCIENCE
MTHE/STAT455, STAT855 FALL 2012, FINAL EXAM
7:00PM, DECEMBER 13, 2012
GLEN TAKAHARA
Instructions:
Proctors are una

MTHE/STAT455, STAT855
Fall 2013
Stochastic Processes
Assignment #1
Due Monday, Sep.30
Starred questions are for 855 students only.
1. Consider the simple random walk, cfw_Xn : n 0, starting at 0 (X0 = 0), where the
probability of moving up at each step is

MTHE/STAT455, STAT855
Fall 2012
MTHE/STAT455, STAT855, Stochastic Processes
Midterm Exam
Instructions:
(a) The exam is closed book. No books are allowed. You may use one 8.5 11 inch sheet
of notes and a calculator.
(b) There are 3 questions. Stat 855 stud

STAT455/855
Fall 2010
Stat 455/855, Stochastic Processes
Midterm Exam
Instructions:
(a) The exam is closed book. No books are allowed. You may use one 8.5 11 inch sheet
of notes and a calculator.
(b) There are 3 questions. Stat 855 students must do all of

MTHE/STAT455, STAT855 Partial Solutions: Practice
Midterm Problems
Fall, 2012
1. (a) If the rst draw is a red ball then we are in the situation where we have just
drawn one consecutive red ball, while if the rst draw is a blue ball then we must
start over

MTHE/STAT455, STAT855
Fall 2012
MTHE/STAT455, STAT855, Stochastic Processes
Midterm Practice Problems
1. An urn contains red balls and blue balls, where the proportion of red balls is p. Balls
are drawn from the urn, one at a time and with replacement. We

MTHE/STAT455, STAT855
Fall 2013
MTHE/STAT455, STAT855, Stochastic Processes
Midterm Exam
Instructions:
(a) The exam is closed book. No books are allowed. You may use one 8.5 11 inch sheet
of notes and a calculator.
(b) There are 3 questions. Stat 855 stud

STAT 455/855 - Final Exam, 2010
Page 1 of 3
QUEENS UNIVERSITY
DEPARTMENT OF MATHEMATICS AND STATISTICS
FACULTY OF ARTS AND SCIENCE
STAT 455/855 FALL 2010, FINAL EXAM
9:00AM, DECEMBER 10, 2010
GLEN TAKAHARA
Instructions:
Proctors are unable to respond to

25
Continuous-Time Markov Chains - Introduction
Prior to introducing continuous-time Markov chains today, let us start
o with an example involving the Poisson process. Our particular
focus in this example is on the way the properties of the exponential
di

21
The Exponential Distribution
From Discrete-Time to Continuous-Time:
In Chapter 6 of the text we will be considering Markov processes in continuous time. In a sense, we already have a very good understanding of
continuous-time Markov chains based on our

9
Markov Chains: Introduction
We now start looking at the material in Chapter 4 of the text. As we
go through Chapter 4 well be more rigorous with some of the theory
that is presented either in an intuitive fashion or simply without proof
in the text.
Our

1
Introduction
Purpose of the Course
The purpose of this course is to study mathematically the behaviour
of stochastic systems.
Examples
1. A CPU with jobs arriving to it in a random fashion.
2. A communications network multiplexor with packets arriving t

13
Introduction to Stationary Distributions
We rst briey review the classication of states in a Markov chain
with a quick example and then begin the discussion of the important
notion of stationary distributions.
First, lets review a little bit with the f