This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: A Survey on Optimality and Equilibria in Stochastic Games Frank Thuijsman Department of Mathematics, Maastricht University 1 Abstract In this paper we discuss the main existence results on optimality and equilibria in two-person stochastic games with finite state and action spaces. Several examples are provided to clarify the issues. 1 The Stochastic Game Model In this introductory section we give the necessary definitions and notations for the two-person case of the stochastic game model and we briefly present some basic results. In section 2 we discuss the main existence results for zero-sum stochastic games, while in section 3 we focus on general-sum stochastic games. In each section we discuss several examples to illustrate the most important phenomena. It all started with the fundamental paper by Von Neumann  in which he proves the so called minimax theorem which says that for each finite matrix of reals A = [ a ij ] m n i =1 ,j =1 there exist probability vectors ¯ x = (¯ x 1 , ¯ x 2 ,..., ¯ x m ) and ¯ y = (¯ y 1 , ¯ y 2 ,..., ¯ y n ) such that for all x and y it holds that 2 xA ¯ y ≤ ¯ xA ¯ y ≤ ¯ xAy . In other words: max x min y xAy = min y max x xAy . This theorem can be interpreted to say that each matrix game has a value. A matrix game A is played as follows. Simultaneously, and independent from each other, player 1 chooses a row i and player 2 chooses a column j of A . Then player 2 has to pay the amount a ij to player 1. Each player is allowed to randomize over his available actions and we assume that player 1 wants to maximize his expected payoff, while player 2 wants to minimize the expected payoff to player 1. The minimax theorem tells us that, for each matrix A there is a unique amount val( A ), which player 1 can guarantee as his minimal expected payoff, while at the same time player 2 can guarantee that the expected payoff to player 1 will be at most this amount. Later Nash  considered the n-person extension of matrix games, in the sense that all n players, simultaneously and independently choose actions that determine a payoff for each and every one of them. Nash  showed that in such games there always exists at least one (Nash-)equilibrium: a tuple of strategies such that each player is playing a best reply against the joint strategy of his opponents. For the two-player case this boils down to a “bimatrix game” where players 1 and 2 receive a ij and b ij respectively in case their choices determine entry ( i,j ). The result of Nash says that there exist ¯ x and ¯ y such that for all x and y it holds that ¯ xA ¯ y ≥ xA ¯ y and ¯ xB ¯ y ≥ ¯ xBy , where A = [ a ij ] and B = [ b ij ] are finite matrices of the same size....
View Full Document
- Spring '09
- Game Theory, Matrix games, stochastic games, Vrieze, Thuijsman