E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
Filtering : c Vikram Krishnamurthy 2008
Part 2: Optimal
Filtering
Aim: The key question answered here is:
Given a stochastic signal observed in noise, how does one
construct an optimal estimator of the signal?
The key results will be covered using element
E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
: c Vikram Krishnamurthy 2008
Part 3: ML
Parameter
Estimation
Aim: The key question answered here is: Given a partially
observed stochastic dynamical system, how does one
estimate the parameters of the system?
Also joint recursive parameter and state esti
E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
Markov decision processes c Vikram Krishnamurthy 2008
Part 4: Markov
Decision
Processes
Aim: This part covers discrete time Markov Decision
processes whose state is completely observed. The key
ideas covered is stochastic dynamic programming (SDP).
SDP is
E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
c Vikram Krishnamurthy 2008
1
Assignment 1: Due 28th May in class
.
1. Let X be a n dimensional vector random variable. Show that its
covariance matrix P is a positive semidenite symmetric matrix. (A
positive semidenite matrix P is one which satises x P
E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
: c Vikram Krishnamurthy 2008
1
Assignment 2 and References
This assignment is due in class on Monday on Monday, June 9 in class.
1. Derive the recursion for an optimal predictor for a general partially
observed state space model in additive iid noise. Re
E:\University\University\Ph.D\Project\Theoretical Stuff\Correlated Equilibrium\Stochastic Est
ELECTRICAL EQ2810

Spring 2008
: c Vikram Krishnamurthy 2008
1
Assignment 3
This assignment is due on Friday the 13th!.
Simulate a 2 state Markov xk with state levels q , transition probability
matrix A chosen as
[q1 , q2 ] = [2 2] , A =
0.9 0.1
0.2 0.8
Simulate the Hidden Markov Mode
Estimation & Stochastic Control : c Vikram Krishnamurthy 2008
1
Part I: Stochastic Models
Aim: The aim here is to review key results in random
processes and present some basic signal processing models.
The results in Part 1 are essential in understanding