This preview shows pages 1–6. Sign up to view the full content.
1
Dynamic Programming
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 2
Investment Problem
•
Decision variable
–
X
ij
=1
means we allocate
j
units of investment to
opportunity
i
–
y =
unused investment
0
},
1
,
0
{
8
,...,
1
for
,
1
10
subject to
5
.
0
max
4
0
8
1
4
0
8
1
4
0
≥
∈
=
=
=
+
+
∑
∑∑
∑∑
=
= =
= =
y
X
i
X
y
jX
y
X
r
ij
j
ij
i j
ij
i j
ij
ij
3
The DP Method
•
Solve the problem sequentially
–
At each stage, one investment opportunity is considered
•
Let
f
(
i
,
s
)
be the maximum total profit from stage
i
,
i
+1,…,
to stage 8, given
s
•
Dynamic Programming recursion
f
(
i
,
s
) = max
d
{
r
id
+
f
(
i
+1,
s
–
d
) 
0 ≤
d
≤ 4,
d
≤
s
}
•
Initial condition
f
( 8,
s
) =max
d
{
r
8d
+0.5(
s
–
d
)  0 ≤
d
≤ 4,
d
≤
s
}
•
The optimal solution of the problem is obtained from
f
(1,10)
: the maximum profit from stage 1 to stage 8 with 10 units
of money initially available
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document 4
Dynamic Programming Formulation
•
Problem includes a series of decisions
–
Over “real” time, or simply a decision sequence
•
Stage, state, decision, and value function
–
Sometimes naturally, sometimes not obviously
•
A recursive equation
–
To link different stages
–
An initial condition to make calculation of DP
feasible
Advantages of DP
•
DP formulation is often the best way to gain
understanding of a problem and its solution
•
Computational efficiency
–
The number of states we have to consider: 8*11, or
I*S
, where
I
is the number of stages and
S
is the
number of possible states
–
For each state, we need to compare up to 5 possible
actions, or
D
decisions, in general
–
Overall calculation:
I
*
S
*
D
•
More information can be obtained
–
What happens if we only have 7 units of money?
–
We only need to see the value of
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 01/05/2011 for the course IELM IELM202 taught by Professor D during the Fall '10 term at HKUST.
 Fall '10
 D

Click to edit the document details