IOE512 Dynamic Programming
Assignment # 2 Due September 18, 2014
Note: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of the so
IOE512 Dynamic Programming
Assignment # 4 Due Oct 13, 2015
Note: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of the solution
IOE512 Dynamic Programming
Assignment # 5 October 27, 2015
Note: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of the solution
IOE12 Dynamic Programming
Assignment # 1 Due 3pm, September 17, 2015
Note: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of th
IOE512 Dynamic Programming
Assignment # 6 Due November 24, 2015
Note 1: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of the s
IOE512 Dynamic Programming
Assignment # 3 Oct 6, 2015
Note: When answering assignment questions be sure to show all of your work including intermediate
results leading up to the final solution. Points are allocated to all of the steps of the solution proc
Proof of Theorem 4.7.4
The following proof is for the case of a nondecreasing monotonic optimal policy (the proof is very similar
for a nonincreasing optimal policy):
As discussed in class, from Lemma 4.7.1 it is sufficient to prove that (, ) is superaddi
Theorem (6.2.2 Puterman): If there exists a such that = , then = .
This proof is in two parts: (a) prove that if then and (b) prove if then .
From which it follows that if = and = .
Part (a): For some arbitrary policy = (, , , . ) if then max cfw_ + . It
Theorem (6.2.3 Puterman): Suppose L is a contraction mapping, then for arbitrary v 0 the sequence
cfw_v 0 , , v n defined by v n+1 = Lv n converges to v as n , where = .
Proof:
First, show that cfw_ 0 , , converges to a limit point. For some n:

+1

Theorem (6.2.4 Puterman): If 0 < < 1 then L is a contraction mapping
Proof:
Let and be arbitrary vectors. Consider an arbitrary state such that () () and
cfw_(, ) + (, )()
Then it follows that
0 () () (, ) + (, )() (, ) (, )()
(, )() ()
(, )  =
#include<stdio.h>
int main ()
cfw_
int i,given,wanted,ratio,diff,b,geometric,arithmetic,fibonacci,a,c,f;
int list[14];
while ( scanf("00",&given,&wanted) != EOF )cfw_
for (i=0;i<given;i+)cfw_
scanf("0",&list[i]);
a=list[given1];
ratio=list[1]/list[0];
f
Last Time
Last time we took a forward
approach and a backwards approach
(policy evaluation algorithm (PEA)
and found the same results for the
problem:
Question:
Policy : At vertex S go up. At all
future states choose to go up if you
have ever gone up in t
Review
Problems we have covered so far:
Shortest path problem
Production and inventory control (WagnerWhitin)
Resource allocation (e.g. knapsack)
Travelling Salesperson Problem
Pattern recognition (e.g. DNA comparison)
Today
Shortest path problems o
Today
Today we will cover
How to evaluate policies
Reading: Today we will start covering topics discussed in
Chapter 4.14.3
Examples
Following are some applications we will cover in the classes
ahead:
Inventory control
Medical treatment decisions
Finance
Last Time
Resource allocation problems (e.g. Knapsack)
TSP
Evaluation of computational effort
Ctools for detailed description of examples from
lecture 3
1
Today
Pattern recognition:
DNA sequencing
Speech recognition
Other pattern recognition problems (e.g
Last Time
Components of DP formulations
What are the main elements of a DP formulation?
1
Sports
A dynamic programming model for baseball
http:/www.footballcommentary.com/bbmodel.htm
Other applications of DP to sports:
Other Sports: http:/mat.tepper.cmu.e
IOE512: Dynamic Programming
Today:
Course summary/expectations
Introduction to dynamic programming
1
Course Contact Information
Instructor:
Brian Denton
Office: IOE 2893
Email: btdenton@umich.edu
Office Hours: To be determined
Course Web Site: Ctools
No
Announcements
Assignment 1 solutions available on canvas
Assignment 2 due today, Assignment 3 next
Thursday
Reminder: regrade requests must be in writing
within a week
Today
Introduction to stochastic dynamic programming
Refresher on Markov chains
Defini
TSP Example, Lecture 4
Following is the dynamic programming formulation of the travelling salesperson problem (TSP)
discussed in lecture 4 (see Dreyfus and Law for a more detailed description).
Let = cfw_2,3, , 1, + 1, , be the set of all locations excep
Last Time
Lotsizing Problem
WagnerWhitin algorithm
1
Important Points
Assignment 1 due on Thursday at the beginning of
class (for question 4 you can assume your opponent
uses a specific strategy (e.g. picks up the same
number of coins each time)
Assignm
Announcements
Office Hours:
Mondays 34pm, Tues/Thurs after class
1
Today
Resource allocation
Travelling salesperson problem
Todays material is covered in:
Chapter 3 of Dreyfus and Law
2
Resource Allocation Problem
The problem of determining the allocatio
Review
Problems we have covered so far:
Shortest path problem
Production and inventory control (WagnerWhitin)
Resource allocation (e.g. knapsack)
Travelling Salesperson Problem
Pattern recognition (e.g. DNA comparison)
Today
Shortest path problems o