Formulation of Finite State Markov Chains
Friday, September 23, 2011
2:04 PM
Note special lecture series by Emmanuel Candes on
compressed sensing Monday and Tuesday 4 -5 PM (room
information on rpinfo)
First an addendum to generating functions.
They can a
Continuous-Time Markov Chains Models
Tuesday, November 15, 2011
2:02 PM
The fundamental object describing the dynamics
of a CTMC (continuous-time Markov chain) is the
probability transition (matrix) function:
(implicitly assuming time-homogeneity from her
Poisson Point Process
Tuesday, November 22, 2011
2:03 PM
Reference: Karlin and Taylor, Sec. 5.9
Homework 3 is due Tuesday, November 29 at 2 PM
Homework 4 is due Wednesday, December 14 at 5 PM
(absolute deadline).
To the Poisson counting process (which is
Classification of Birth-Death Chains
Tuesday, November 08, 2011
2:02 PM
Homework 3 posted, due Tuesday, November 29.
Continuing with our classification of birth-death chains on
nonnegative integers. Last time we started by looking for an
invariant measure
Analyzing FSDT Markov chains
Friday, September 30, 2011
2:03 PM
Simulating FSDT Markov chains, as we have said is very straightforward,
either by using probability transition matrix or stochastic update rule. So it
may seem that to answer questions about
Derivation of Existence, Uniqueness, Limit Properties of Stationary
Distributions
Friday, October 07, 2011
3:16 PM
The main reference is Resnick Secs. 2.12-2.15, which extends to countably infinite
Markov chains as well as finite Markov chains.
Karlin & T
Countable-state, discrete-time Markov chains
Tuesday, November 01, 2011
2:05 PM
Readings: Lawler Ch. 2
Karlin & Taylor Chs. 2 & 3
Resnick Ch. 1
Recall that a countably infinite state space is one that can be placed in
one-to-one correspondence with the in
Random Variables
Friday, September 09, 2011
2:02 PM
I will post Homework 1 soon, probably over the weekend, due
Friday, September 30.
No class or office hours next week. Next class is on Tuesday,
September 20.
Readings: Karlin and Taylor Sec. 1.1-1.3
One
Classification of Countable State Markov Chains
Friday, November 04, 2011
2:02 PM
Continuing the discussion of long -time behavior of countable-state Markov chains from last
time, we now turn to transient communication classes.
These also have similar pro
Applications of Absorption Probability and Accumulated
Cost/Reward Formulas for FDMC
Friday, October 21, 2011
2:28 PM
No class next week. No office hours either.
Next class will be 11/01.
The cost/reward formula has two specific widely used applications:
Generating Functions
Tuesday, September 20, 2011
2:00 PM
Homework 1 posted, due Friday, September 30, 2 PM.
Independence of random variables:
We say that a collection of random variables
Is independent if for any finite subset
One important consequence of
Extinction Probability for Branching Processes
Friday, November 11, 2011
2:05 PM
Continuing the calculation of the absorption
probability for a branching process.
Homework 3 is due Tuesday, November 29.
The nontrivial situation is when:
So the probability
Overview of Stochastic Processes
Friday, September 02, 2011
2:07 PM
Stochastic processes entails a dynamical approach to probability theory, with
uncertainty entering a system over time.
Mathematically a stochastic process can be thought of as a random fu
Review of Probability Theory
Tuesday, September 06, 2011
2:05 PM
Office hours: Wednesdays 11 AM- 12 PM (this class preference), Mondays 2 PM - 3 PM
(free-for-all),
Wednesdays 3 PM - 4 PM (DE class preference)
Readings: Karlin and Taylor Secs. 1.1-1.3: con
Classification of Markov chains
Tuesday, October 04, 2011
2:06 PM
Note special talk on mathematics and music, Wednesday, October 5 at
2 PM at EMPAC.
To address the questions of whether the stationary distribution is
unique and whether it serves as a limit
Modeling with Finite State Markov Chains
Tuesday, September 27, 2011
1:54 PM
Homework 1 due Friday, September 30 at 2 PM.
Office hours on 09/28: Only 11 AM-12 PM (not at 3 PM)
Important side remark about exponentially distributed
random variables T
They h
Poisson Paradox and Renewal Theorem
Tuesday, December 06, 2011
11:11 AM
Homework 4 due Thursday, December 15 at 5 PM. Hard
deadline.
Final exam is on Wednesday, December 14 at 3 PM. If you want
to take it let me know by Tuesday, December 13 at 5 PM.
Picki