Formulation of Finite State Markov Chains
Friday, September 23, 2011
Note special lecture series by Emmanuel Candes on
compressed sensing Monday and Tuesday 4 -5 PM (room
information on rpinfo)
First an addendum to generating functions.
They can a
Continuous-Time Markov Chains Models
Tuesday, November 15, 2011
The fundamental object describing the dynamics
of a CTMC (continuous-time Markov chain) is the
probability transition (matrix) function:
(implicitly assuming time-homogeneity from her
Poisson Point Process
Tuesday, November 22, 2011
Reference: Karlin and Taylor, Sec. 5.9
Homework 3 is due Tuesday, November 29 at 2 PM
Homework 4 is due Wednesday, December 14 at 5 PM
To the Poisson counting process (which is
Classification of Birth-Death Chains
Tuesday, November 08, 2011
Homework 3 posted, due Tuesday, November 29.
Continuing with our classification of birth-death chains on
nonnegative integers. Last time we started by looking for an
Analyzing FSDT Markov chains
Friday, September 30, 2011
Simulating FSDT Markov chains, as we have said is very straightforward,
either by using probability transition matrix or stochastic update rule. So it
may seem that to answer questions about
Derivation of Existence, Uniqueness, Limit Properties of Stationary
Friday, October 07, 2011
The main reference is Resnick Secs. 2.12-2.15, which extends to countably infinite
Markov chains as well as finite Markov chains.
Karlin & T
Countable-state, discrete-time Markov chains
Tuesday, November 01, 2011
Readings: Lawler Ch. 2
Karlin & Taylor Chs. 2 & 3
Resnick Ch. 1
Recall that a countably infinite state space is one that can be placed in
one-to-one correspondence with the in
Friday, September 09, 2011
I will post Homework 1 soon, probably over the weekend, due
Friday, September 30.
No class or office hours next week. Next class is on Tuesday,
Readings: Karlin and Taylor Sec. 1.1-1.3
Classification of Countable State Markov Chains
Friday, November 04, 2011
Continuing the discussion of long -time behavior of countable-state Markov chains from last
time, we now turn to transient communication classes.
These also have similar pro
Applications of Absorption Probability and Accumulated
Cost/Reward Formulas for FDMC
Friday, October 21, 2011
No class next week. No office hours either.
Next class will be 11/01.
The cost/reward formula has two specific widely used applications:
Tuesday, September 20, 2011
Homework 1 posted, due Friday, September 30, 2 PM.
Independence of random variables:
We say that a collection of random variables
Is independent if for any finite subset
One important consequence of
Extinction Probability for Branching Processes
Friday, November 11, 2011
Continuing the calculation of the absorption
probability for a branching process.
Homework 3 is due Tuesday, November 29.
The nontrivial situation is when:
So the probability
Overview of Stochastic Processes
Friday, September 02, 2011
Stochastic processes entails a dynamical approach to probability theory, with
uncertainty entering a system over time.
Mathematically a stochastic process can be thought of as a random fu
Review of Probability Theory
Tuesday, September 06, 2011
Office hours: Wednesdays 11 AM- 12 PM (this class preference), Mondays 2 PM - 3 PM
Wednesdays 3 PM - 4 PM (DE class preference)
Readings: Karlin and Taylor Secs. 1.1-1.3: con
Classification of Markov chains
Tuesday, October 04, 2011
Note special talk on mathematics and music, Wednesday, October 5 at
2 PM at EMPAC.
To address the questions of whether the stationary distribution is
unique and whether it serves as a limit
Modeling with Finite State Markov Chains
Tuesday, September 27, 2011
Homework 1 due Friday, September 30 at 2 PM.
Office hours on 09/28: Only 11 AM-12 PM (not at 3 PM)
Important side remark about exponentially distributed
random variables T
Poisson Paradox and Renewal Theorem
Tuesday, December 06, 2011
Homework 4 due Thursday, December 15 at 5 PM. Hard
Final exam is on Wednesday, December 14 at 3 PM. If you want
to take it let me know by Tuesday, December 13 at 5 PM.