This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Markov Decision Evolutionary Games with Expected Average Fitness E. Altman INRIA, MAESTRO Group 2004 Route des Lucioles F-06902, Sophia-Antipolis Cedex, France E-mail: [email protected] Y. Hayel, H. Tembine, and R. El-Azouzi LIA/CERI, University of Avignon 339, chemin des Meinajaries Agroparc BP 1228, F-84911 Avignon Cedex, France Abstract Aim: To model and characterize evolutionary games where individ- uals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly se- lected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov Decision Processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of Evolutionarily Stable Strategies based on the concept of Occupation Measures. We relate this set to the concept of Evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show existence of an Occupation Measure ESS (OMESS). Keywords: evolutionary games, occupation measure evolutionarily sta- ble strategy, Markov decision process, energy control in wireless networks. 1 Introduction Evolutionary games have been introduced to model the evolution of population sizes as a result of competition between them that occurs through many local pairwise interactions, i.e. interactions between randomly chosen pairs of indi- viduals (see Fisher 1930 , Hamilton 1963,1964 , Maynard Smith 1972 ). Central in evolutionary games is the concept of Evolutionarily Stable Strategy (ESS) introduced by Maynard Smith & Price in 1973. ESS is a distribution of (deter- ministic or mixed) actions such that if used, the population is immune against penetration of mutations. This notion is stronger than that of Nash equilib- rium as ESS is robust against a deviation of a whole fraction of the population where as the Nash equilibrium is defined with respect to possible deviations of a single player ( Nash, 1951 ). A second foundation of evolutionary games is the replicator dynamics that describes the dynamics of the sizes of the populations as a result of the fitness they receive in interactions. Maynard Smith formally introduced both, without needing an explicit modeling of stochastic features....
View Full Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
- Spring '09