Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 856 E.A. FAINBERG REFERENCES 1] D.YLVISAKER,A note on theabsence of tangenciesinGaussiansamplepaths,Ann.Math.Statist., 39, (1968),pp.261-262. [2] A.V.SKOROKHOD, A note on Gaussian measures inBanach space,TheoryProb.Applications, 15,3 (1970),p.508. [3] X.FERNIQUE, Intgrabilitdesvecteursgaussiens,C.R. Acad. Sci.Paris,Set.A,270,25 (1970), pp. 1698-1699. [4] H.J.LANDAU AND L.A.SHEPP,On thesupremum ofa Gaussianprocess,Sankhya,Ser.A,32,4 (1970),pp.369-378. [5] V. N. SUDAKOV A N D B. S. TSIREL’SON, Extremal properties of half-spaces for spherically invariantmeasures,ZapiskiNauchn.Seminarov LOMI,41(1974),pp.14-24.(InRussian.) [6] V.I.SMIRNOV,CourseinHigher Mathematics,Vol.2,Addison-Wesley,Reading,Mass.,1964. ON CONTROLLED FINITE STATE MARKOV PROCESSES WITH COMPACT CONTROL SETS E. A. FAINBERG (TranslatedbyK.Durr) 1. Inthis paperwe consider the maximization of the average gain per unit step in controlledfinitestate Markov chainswith compact controlsets. In[1]and [2]a stationaryoptimalstrategywas shown to existunder theassumption thatthecontrolsetsare finite.In I-3]it was proved thatifthecontrolsetsare compact and coincide with the transition probability sets, the gain functions are continuous and any stationary strategy yields a Markov chain with one ergodic class and without transient states,then there exists a stationary optimal strategy. Inthe general case, conditions of compactnessofthecontrolsetsandofcontinuityofthegainandtransitionfunctionsarenot sufficientfortheexistenceofan optimalstrategy(see[4],Example 3). Inthispaper the existence isestablishedofa stationaryoptimal strategyunder the conditionthatthecontrolsetsare compact,thegainfunctionsareuppersemi-continuous, thetransitionfunctionsdepend continuouslyon thecontrolsand thatone ofthefollowing conditions holds: (i)anystationarystrategyyields a Markov chain with one ergodic class, and possibly with transient states (Section 3); (ii)for each state the set of transition probabilitiescontains a finitesetofextreme points(Section4). 2. LetX be a statespace consistingofa finitenumber ofpoints (X {1,2, , s}). Foreach state there isgiven a controlsetAx(x 1,2,...,s).On the sets Ax there are definedfunctionsqx(a)(thegainfromthecontrola Ax when theprocessisinthestatex), and probabilitymeasures Px ("[a) on X (thetransitionfunctions under theconditionthat theprocessisin thestate x and thecontrola Ax is chosen).SetA U= A. Let Xo, al, x, a2, x2,’" be the succession of states and controls. At each moment t= 1,2,... a choice of control is made which is given by a probability measure r,(da, lxo, a,xa,..., a,_l, x,-1) on the set A,,_, measurablydepending on the past. The collectionofthesemeasuresfor 1,2, definesthestrategy 7r. The strategy 7r is called stationary ifthe measures zr,are concentrated at the points at p(x,_), where p is a selector,i.e.,amappingofX intoA suchthat p (x) A....
View Full Document

This note was uploaded on 12/06/2011 for the course MATH 101 taught by Professor Eugenea.feinberg during the Fall '11 term at State University of New York.

Page1 / 7


This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online