Copyright c 2013, Lang Tong. EE5640: Statistical Inference and Decision
ECE 5640: Statistical Inference and Decision
Probability Models
http:/blackboard.cornell.edu
Lang Tong
School of Electrical and Computer Engineering
Cornell University, Ithaca, NY 148
ECE 5640 Homework 2 Solution
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
1. An Inventory Control Problem
st stock
at units ordered
orders arrive D demand is fullled
t
t+1
(a) The MDP formulation of the problem is summarized as below
ECE 5640 Homework 7
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due by 5pm on December 4
1
Materials
Markov chain and Markov reward process.
MDP under average reward over an innite horizon.
2
Assignment
1. Markov chain and Markov
ECE 5640 Homework 6
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on November 18
1
Materials
Negative dynamic programming.
The optimal stopping problem.
2
Assignment
1. Negative Dynamic Programming
Prove the following s
ECE 5640 Homework 5
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on October 21
1
Materials
MDP over Innite Horizon under Discounted Reward:
Policy evaluation.
Value iteration.
Policy iteration.
2
Assignment
1. Random
ECE 5640 Homework 4
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on October 7
1
Materials
Techniques for establishing structured optimal policies:
Interchange argument.
Threshold optimal policies.
Modularity and mono
ECE 5640 Homework 1
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on September 2
1. A die is rolled repeatedly. Which of the following sequences cfw_Xn are Markov chains? Prove
n=0
your statement. For those that are, giv
ECE 5640 Homework 2
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on September 23
1
Materials
Formulation of MDP over a nite horizon.
Policy evaluation.
Finding the optimal policy using backward induction.
2
Assignment
ECE 5640 Homework 3
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
Due in class on September 30
1
Materials
POMDP
2
Assignment
1. Dynamic Multichannel Access: (10 points)
Consider N time-slotted channels. In each slot t, the state of
ECE 5640 Homework 3 Solution
Instructor: Qing Zhao
Oce: 325 Rhodes Hall
Email: qz16@cornell.edu
1. Dynamic Multichannel Access:
(a) Formulating the problem as a POMDP:
Decision times: t = 1, 2, . . . , T .
The state space S = cfw_(s1 , . . . , sN ), i si