EE292E
Spring 2008
Analysis & Control of Markov Chains
April 10, 2008
Prof. Ben Van Roy
Homework Assignment 1 : Solutions
1.16
(a) Each state is a set
S
k
⊂ {
2
, ...N
}
.
The allowable states at stage
k
are those sets
S
k
of cardinality
k
. The allowable controls are
u
k
∈ {
2
, ..., N
} 
S
k
. This control represents the
multiplication of the term ending in
M
u
k

1
by the one starting in
M
u
k
. The system equation is
S
k
+1
=
S
k
∪
u
k
. The terminal state is
S
N
=
{
2
, ..., N
}
, with cost
0
. The cost at stage
k
is given
by the number of multiplications:
g
k
(
S
k
, u
k
) =
n
a
n
u
k
n
b
where
a
= max
{
i
∈ {
1
, . . . , N
+ 1
}
i /
∈
S
k
, i < u
k
}
b
= min
{
i
∈ {
1
, . . . , N
+ 1
}
i /
∈
S
k
, i > u
k
}
For example, let
N
= 3
and
M
1
be
1
×
10
,
M
2
be
10
×
1
,
M
3
be
1
×
10
. The order
(
M
1
M
2
)
M
3
corresponds to controls
u
1
= 2
and
u
2
= 3
, giving cost:
(
u
1
= 2) :
n
1
n
2
n
3
= 10(
a
= 1
, b
= 3)
(
u
2
= 3) :
n
1
n
3
n
4
= 10(
a
= 1
, b
= 4)
with a total cost of
20
.
whereas
M
1
(
M
2
M
3
)
gives:
(
u
1
= 3) :
n
2
n
3
n
4
= 100(
a
= 2
, b
= 4)
(
u
2
= 2) :
n
1
n
2
n
4
= 100(
a
= 1
, b
= 4)
with a total cost of
200
!
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 Bertsekas
 UK, Shortest path problem, shortest path, sK

Click to edit the document details