SE-ME-EC 710 Notes. Michael Caramanis, draft May 2016
VI. Continuous Time DP problems with Controllable Markov Jump
Process Dynamics
VI.1 Asides
VI.1.1 Aside on Combination of Poisson Arrivals
-Independent Poisson Arrivals of multiple classes i=1,2,3,. of
Homework Assignment
ME-SE-EC 710
Solutions to Problem Set 3
1) Chapter 2 Problem 1
L.Q. Problem with forecast
x k +1 = A k x k + Bk u k + w k
Cost =
E
N 1
x N Q N x N + (x k Q k x k + u k R k u k )
k =0
wk
k = 0,1, N 1
y k = forecast available at the begi
Homework Assignment
ME-SE-EC 710
Solutions to Problem Set 1
1) Dynamics
x k1 x 2k w k u k
k 0, 1,
, N 1
k 0, 1,
,N
k 0, 1,
, N 1
P(wk 1) p u k
P(wk 0.5) 1 p u k
g x k , u k , w k x 2k
if x k 1
if x k 1
x 2k
k (x k )
0.5
1.1) S0 Set of real numbers
Howe
ME-SE-EC 710
Solutions to Homework Assignment 2
Chapter 1 Problem 3
In setting up the problem it is crucial to carefully represent the sequence of events, i.e.
what happens at the beginning, during and at the end of each week. The approach taken
here is o
Homework Assignment
ME-SE-EC 710
Problem Set 1
1) A scalar state variable system has dynamics
x k +1 = x 2k + w k u k
k = 0, 1, , N 1
The disturbance is a discrete random variable with range space D = cfw_0.5, 1 and the probability
mass function is given
MN-SC 710 Midterm Examination, Spring 2014
This is a closed book examination. A 3x5 card with notes is allowed. All answers should be justified. Please
write clearly. Good Luck.
Problem 1. (30 points)
Consider a stochastic DP problem that models the optim
710 Midterm solutions. Spring 2014
Problem 1
1.1
This is a vanilla state augmentation problem discussed in the first set of notes.
Since the mean of wk depends on the observed value of wk-1, the augmented state at time k is
(xk,yk) with dynamics: xk =xk-1
Homework Assignment
ME-SE-EC 710
Solutions to Problem Set 4
1) Asset Selling Problem
(w 1) 2
p(w k ) k
0
1
From class notes, k
1
But the integral
for w k 0
otherwise
k 1
k 1
0
k 1
w p( w)dw 1
1
dw log w 1
diverges,
1 w
(1 w) 2
k 1
w
k 1
p( w)
Homework Assignment
ME-SE-EC 710
Problem Set 3
1) From text do Chapter 4 Problem 1
Hint: y k = forecast at the beginning of period k, y k cfw_1, 2, , n
Define Pki = apriori probability of y k = i . Then D.P. algorithm is
J N (x N , y N ) = x N Q N x N
n
Homework Assignment
ME-SE-EC 710
Problem Set 2
From Text do:
Problems 1.3, 1.8, and 1.13 from chapter 1.
In doing problem 1.3 note that the following sequence of information and time period
definitions can be helpful to you.
x j1 u j1
j
j1
w
j+1 week
xj
Homework Assignment
ME-SE-EC 710
Problem Set 4
1) Given the asset selling problem with the disturbance w k having probability density function:
(w k + 1) 2
p(w k ) =
0
for w k 0
otherwise
find N , N 1 . Does the steady state policy associated with = lim
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Gr
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Co
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Ca
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Pr
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Ma
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Fa
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
In
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Ex
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
Le
SE524/EC524 Optimization Theory and Methods
Yannis Paschalidis
[email protected], http:/ionia.bu.edu/
Department of Electrical and Computer Engineering,
Division of Systems Engineering,
and Center for Information and Systems Engineering,
Boston University
An