856
E.A.
FAINBERG
REFERENCES
1]
D.
YLVISAKER,
A
note
on
the
absence
of
tangencies
in
Gaussian
sample
paths,
Ann.
Math.
Statist.,
39,
(1968),
pp.
261-262.
[2]
A.
V.
SKOROKHOD,
A
note
on
Gaussian
measures
in
Banach
space,
Theory
Prob.
Applications,
15,
3
(1970),
p.
508.
[3]
X.
FERNIQUE,
Intgrabilit
des
vecteurs
gaussiens,
C.R.
Acad.
Sci.
Paris,
Set.
A,
270,
25
(1970),
pp.
1698-1699.
[4]
H.
J.
LANDAU
AND
L.
A.
SHEPP,
On
the
supremum
ofa
Gaussian
process,
Sankhya,
Ser.
A,
32,
4
(1970),
pp.
369-378.
[5]
V.
N.
SUDAKOV
AND
B.
S.
TSIREL’SON,
Extremal
properties
of
half-spaces
for
spherically
invariant
measures,
Zapiski
Nauchn.
Seminarov
LOMI,
41
(1974),
pp.
14-24.
(In
Russian.)
[6]
V.I.
SMIRNOV,
Course
in
Higher
Mathematics,
Vol.
2,
Addison-Wesley,
Reading,
Mass.,
1964.
ON
CONTROLLED
FINITE
STATE
MARKOV
PROCESSES
WITH
COMPACT
CONTROL
SETS
E.
A.
FA
INBER
G
(Translated
by K. Durr)
1.
In
this
paper
we
consider
the
maximization
of
the
average
gain
per
unit
step
in
controlled
finite
state
Markov
chains
with
compact
control
sets.
In
[1]
and
[2]
a
stationary
optimal
strategy
was
shown
to
exist
under
the
assumption
that
the
control
sets
are
finite.
In
I-3] it
was
proved
that
if
the
control
sets
are
compact
and
coincide
with
the
transition
probability
sets,
the
gain
functions
are
continuous
and
any
stationary
strategy
yields
a
Markov
chain
with
one
ergodic
class
and
without
transient
states,
then
there
exists
a
stationary
optimal
strategy.
In
the
general
case,
conditions
of
compactness
of
the
control
sets
and
of
continuity
of
the
gain
and
transition
functions
are
not
sufficient
for
the
existence
of
an
optimal
strategy
(see
[4],
Example
3).
In
this
paper
the
existence
is
established
of
a
stationary
optimal
strategy
under
the
condition
that
the
control
sets
are
compact,
the
gain
functions
are
upper
semi-continuous,
the
transition
functions
depend
continuously
on
the
controls
and
that
one
of
the
following
conditions
holds:
(i)
any
stationary
strategy
yields
a
Markov
chain
with
one
ergodic
class,
and
possibly
with
transient
states
(Section
3);
(ii)
for
each
state
the
set
of
transition
probabilities
contains
a
finite
set
of
extreme
points
(Section
4).
2.
Let
X
be
a
state
space
consisting
of
a
finite
number
of
points
(X
{1,
2,
,
s}).
For
each
state
there
is
given
a
control
set
Ax(x
1,
2,...,
s).
On
the
sets
Ax
there
are
defined
functions
qx(a)
(the
gain
from
the
control
a
Ax
when
the
process
is
in
the
state
x),
and
probability
measures
Px
("
[a)
on
X
(the
transition
functions
under
the
condition
that
the
process
is
in
the
state
x
and
the
control
a
Ax
is
chosen).
Set
A
U=
A.
Let
Xo,
al,
x,
a2,
x2,’"
be
the
succession
of
states
and
controls.
At
each
moment
t=
1,2,...
a
choice
of
control
is
made
which
is
given
by
a
probability
measure
r,(da,
lxo,
a,xa,...,
a,_l,
x,-1)
on
the
set
A,,_,
measurably
depending
on
the
past.
The
collection
of
these
measures
for
1,
2,
defines
the
strategy
7r.
The
strategy
7r
is
called
stationary
if
the
measures
zr,
are
concentrated
at
the
points
at
p(x,_),
where
p
is
a
selector,
i.e.,
a
mapping
of
X
into
A
such
that
p
(x)
A.