Dynamic Programming Lecture #14
Outline:
Inventory Control
Inventory Control DP
xk+1 = xk + uk wk
x=
inventory x > 0
backlog
x<0
u = production
w = demand
Dene
h |z| z > 0
p |z| z < 0
r(z) =
p
T
Dynamic Programming Lecture #10
Outline:
Stochastic DP
LQ Optimal Control
Finite State Markov Chains
Stochastic DP
System:
xk+1 = fk (xk , uk , wk )
xk Sk ,
uk Uk (xk ),
wk Wk (xk )
Assume:
wk i
Dynamic Programming Lecture #15
Outline:
Finish inventory control (Lecture #14)
Termination problems
Monotonicity property
Termination Problems
Two possibilities:
Play until end:
N 1
realized cos
Dynamic Programming Lecture #12
Outline:
Viterbi algorithm
Controlled Markov chains
Viterbi Algorithm
Given an observation sequence:
Z = cfw_z1 , z2 , ., zN
what is the most likely state sequence?
Dynamic Programming Lecture #8
Outline:
Probability review, cont
Expectation
Law of large numbers
Stochastic DP
Expectation
Define: The expected value of X
E [X] =
xpX (x)
x
Expected value is a
Dynamic Programming Lecture #9
Outline:
Stochastic DP
Examples
Repeated Prisoners Dilemma
Stochastic DP
System:
xk+1 = fk (xk , uk , wk )
xk Sk ,
uk Uk (xk ),
wk Wk (xk )
Assume:
wk is an RV on
Dynamic Programming Lecture #5
Outline:
Worst case DP
Stochastic DP preview
Probability review
Probability space
Conditional probabilities
Value Iteration
Setup:
xk+1 = fk (xk , uk , wk )
uk Uk
Dynamic Programming Lecture #6
Outline:
Stochastic DP preview
Probability review
Probability space
Conditional probabilities
Total probability
Bayes rule
Independent events
Stochastic DP Previe
Dynamic Programming Lecture #4
Outline:
Deterministic DP Review
Variations
Worst case DP
Deterministic DP Review
System:
xk+1 = fk (xk , uk )
State: xk Sk
Control (decision): uk Uk (xk )
Policy
Dynamic Programming Lecture #1
Outline:
Problem formulation(s)
Examples
Motivation: Staged Optimization
Q: How to formulate optimization for problems that occur in stages?
Standard Optimization:
m