EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 3 Solutions
1. Total-variation distance. The total variation distance between distributions and
is given by
dTV (, ) = max Prob(E ) Prob(E ) ,
E X
i.e., the maximu
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 5
1. A rened inventory model. We consider an inventory model that is more rened than
the one youve seen in the lectures. The amount of inventory at time t is denote
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 2 Solutions
1
Problem Solutions
7.1
Computing and Optimizing Policy Value Functions
1. Since Vt (x) = T=t1 R (x , (x ) + RT (xT ), to calculate Vt (x) for a single
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 4
1. Markov web surng model. A set of n web pages labeled 1, . . . , n contain (directed)
links to other pages. We dene the link matrix L Rnn as
Lij =
1 if page i l
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 Homework 1 solutions
1.1 Optimal disposition of a stock. You must sell a total amount B > 0 of a stock in two
rounds. In each round you can sell any nonnegative amount of the stock; b
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 5 Solutions
1. A rened inventory model. We consider an inventory model that is more rened than
the one youve seen in the lectures. The amount of inventory at time t
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 4 Solutions
1. Markov web surng model. A set of n web pages labeled 1, . . . , n contain (directed)
links to other pages. We dene the link matrix L Rnn as
Lij =
1 i
EE365/MS&E251 Stochastic Decision Models and Control
May 15, 2012
Lecture on Control of Stochastic Systems
Professors S. Boyd, S. Lall, and B. Van Roy
1
Model and Problem Formulation
We will develop an approach to modeling a dynamic system and using the m
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 7 Solutions
1. Bi-directional supply chain via LQR. In this problem we manage the ow of a single
commodity across a chain consisting of n warehouses and n links con
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 7
1. Bi-directional supply chain via LQR. In this problem we manage the ow of a single
commodity across a chain consisting of n warehouses and n links connecting th
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 6 Solutions
1. For a single-site inventory model, we proceed as in the notes:
S = cfw_DT, . . . , X
For any x S , A(x) = cfw_0, . . . , X x.
T =T
0. 5
Pt (x, a
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 3
1. Total-variation distance. The total variation distance between distributions and
is given by
dTV (, ) = max Prob(E ) Prob(E ) ,
E X
i.e., the maximum dierence
EE365, Spring 2011-12
Professors S. Boyd, S. Lall, and B. Van Roy
EE365 / MS&E251 Homework 1
1.1 Optimal disposition of a stock. You must sell a total amount B > 0 of a stock in two
rounds. In each round you can sell any nonnegative amount of the stock; b