{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

TV1975 - 856 E.A FAINBERG REFERENCES 1 D YLVISAKER A note...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
856 E.A. FAINBERG REFERENCES 1] D. YLVISAKER, A note on the absence of tangencies in Gaussian sample paths, Ann. Math. Statist., 39, (1968), pp. 261-262. [2] A. V. SKOROKHOD, A note on Gaussian measures in Banach space, Theory Prob. Applications, 15, 3 (1970), p. 508. [3] X. FERNIQUE, Intgrabilit des vecteurs gaussiens, C.R. Acad. Sci. Paris, Set. A, 270, 25 (1970), pp. 1698-1699. [4] H. J. LANDAU AND L. A. SHEPP, On the supremum ofa Gaussian process, Sankhya, Ser. A, 32, 4 (1970), pp. 369-378. [5] V. N. SUDAKOV AND B. S. TSIREL’SON, Extremal properties of half-spaces for spherically invariant measures, Zapiski Nauchn. Seminarov LOMI, 41 (1974), pp. 14-24. (In Russian.) [6] V.I. SMIRNOV, Course in Higher Mathematics, Vol. 2, Addison-Wesley, Reading, Mass., 1964. ON CONTROLLED FINITE STATE MARKOV PROCESSES WITH COMPACT CONTROL SETS E. A. FA INBER G (Translated by K. Durr) 1. In this paper we consider the maximization of the average gain per unit step in controlled finite state Markov chains with compact control sets. In [1] and [2] a stationary optimal strategy was shown to exist under the assumption that the control sets are finite. In I-3] it was proved that if the control sets are compact and coincide with the transition probability sets, the gain functions are continuous and any stationary strategy yields a Markov chain with one ergodic class and without transient states, then there exists a stationary optimal strategy. In the general case, conditions of compactness of the control sets and of continuity of the gain and transition functions are not sufficient for the existence of an optimal strategy (see [4], Example 3). In this paper the existence is established of a stationary optimal strategy under the condition that the control sets are compact, the gain functions are upper semi-continuous, the transition functions depend continuously on the controls and that one of the following conditions holds: (i) any stationary strategy yields a Markov chain with one ergodic class, and possibly with transient states (Section 3); (ii) for each state the set of transition probabilities contains a finite set of extreme points (Section 4). 2. Let X be a state space consisting of a finite number of points (X {1, 2, , s}). For each state there is given a control set Ax(x 1, 2,..., s). On the sets Ax there are defined functions qx(a) (the gain from the control a Ax when the process is in the state x), and probability measures Px (" [a) on X (the transition functions under the condition that the process is in the state x and the control a Ax is chosen). Set A U= A. Let Xo, al, x, a2, x2,’" be the succession of states and controls. At each moment t= 1,2,... a choice of control is made which is given by a probability measure r,(da, lxo, a,xa,..., a,_l, x,-1) on the set A,,_, measurably depending on the past. The collection of these measures for 1, 2, defines the strategy 7r. The strategy 7r is called stationary if the measures zr, are concentrated at the points at p(x,_), where p is a selector, i.e., a mapping of X into A such that p (x) A.
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern