16:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Assignment # 3 (due Oct. 26, 2016)
Problem 1.
A pharma company may be in two states: good (1) and fair (2). In each state
two actions are possible: research (R) and no research (N). The correspo

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Label correction algorithms
for determining the shortest path in a graph
1
Introduction
A graph G = (N , A) is given by a set of nodes N and a set of arcs A. The arcs are ordered pairs
a = (i, j

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Equipment Replacement Model
1
The Model
We consider operating a machine that can be in one of the states x cfw_0, 1, 2, . . . , with 0 representing
new machine, and higher values of x correspond

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Finite-Horizon Deterministic Dynamic Programming Problems
1
Introduction
In optimization of dynamical systems, one of the most popular and powerful methodology is dynamic
programming. It address

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Finite-Horizon Stochastic Problems
1
The problem
We consider controlled Markov models as described in Class 1. We may allow the controlled transition
kernels Pt and the feasible control sets Ut

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Dynamic Programming Models
1
Controlled Markov Models
We shall be mainly concerned with optimal decisions in controlled discrete-time Markov models, which
have the following components.
Time Epo

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Inventory Model
We consider an inventory model, with stock at the beginning of period t denoted by xt , orders at the
beginning of period t by u t , and random demand in period t (observed only

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Assignment # 2 (due Oct 12, 2016)
Problem 1.
Consider the equipment replacement problem discussed in class, with the following data:
operating cost per period c0 + c1 x, x = 0, 1, 2, . . . ;
r

26:711:557 DYNAMIC PROGRAMMING
Professor Andrzej Ruszczynski
Infinite-Horizon Discounted Problem
1
The problem
We consider infinite horizon discounted controlled Markov models as described in Class 1. For a given
(0, 1), our aim is to find a policy = cfw