JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 76, No. 3, MARCH 1993
Denmmerable State Stochastic Games
with Limiting Average Payoff 1
V. S. BORKAR x AND M. K. GHOSH 3
Communicated by P. Varaiya
We study stochastic games with countable state space, com-
pact action spaces, and limiting average payoff. For N-person games,
the existence of an equilibrium in stationary strategies is established
under a certain Liapunov stability condition. For two-person zero-sum
games, the existence of a value and optimal strategies for both players
are established under the same stability condition.
Ergodic occupation measure, stationary strategies,
Shapley equation, equilibrium.
We study noncooperative stochastic games with countable state space,
compact action spaces, and with ergodic or limiting average payoff. The
existing literature in stochastic games with limiting average payoff seems to
be very limited. To the best of our knowledge, notable contributions are
due to Gillette (Ref. t), Sobel (Ref. 2), Bewley and Kohlberg (Ref. 3),
Federgruen (Ref. 4), and Mertens and Neyrnan (Ref. 5). As in Markov
decision processes (MDP), the standard approach to stochastic games with
limiting average payoff is to treat it as a limiting case of B-discount payoff
as the discount factor fl~ 1. For B-discount payoff criterion, much more is
known. For two-person zero-sum games with B-discount payoff, the
1The authors wish to thank Prof. T. Parthasarathy for pointing out an error in an earlier
version of this paper. M, K. Ghosh wishes to thank Prof. A. Arapos~athis and Prof. S. I.
Marcus for their hospitality and support.
2Associate Professor, Department of Electrical Engineering, Indian Institute of Science, Banga-
3Assistant Professor, Department of Mathematics, Indian lr~stitute of Science, Bangalore, India.
0022-3239/93/0300-0539507.00/0 © t993 Plenum Publlsl~ing Corporation