Synthesis of strategic games with multiple pre-set Nash equilibria - an artificial inference approach using Fuzzy ASA

This paper presents an extension of the results obtained in previous work concerning the application of global optimization techniques to the design of finite strategic games with mixed strategies. In that publication the Fuzzy ASA global optimization method was applied to many examples of synthesis of strategic games with one previously specified Nash equilibrium, evidencing its ability in finding payoff functions whose respective games present those equilibria, possibly among others. That is to say, it was shown it is possible to establish in advance a Nash equilibrium for a generic finite state strategic game and to compute payoff functions that will make it feasible to reach the chosen equilibrium, allowing players to converge to the desired profile, considering that it is an equilibrium of the game as well. Going beyond this state of affairs, the present article shows that it is possible to ”impose” multiple Nash equilibria to finite strategic games by following the same reasoning as before, but with a fundamental change: using the same fundamental theorem of Richard McKelvey, modifying the originally prescribed objective function and globally minimizing it. The proposed method, in principle, is able to find payoff functions that result in games featuring an arbitrary number of Nash equilibria, paving the way to a substantial number of potential applications.


Introduction
Although game theory has been traditionally used in the investigation of economic systems, in recent years its importance is being evidenced in several fields, like network protocol design and analysis [8], computational resources allocation and several other phenomena [1]. This trend may be explained by the fact that game theory is all about modelling interactions among agents and respective results, and this type of context can be frequently found almost everywhere and in several kinds of scenarios. Often the decision of a single player may affect drastically the result of a given game. In order to model players' benefits (or losses) when acting during the game, payoff functions are used, and to cope with uncertainty, mixed strategies are employed, representing the belief opponents have with respect to possible moves of other participants. In this fashion, and as a significant analytical tool, it was endowed with the concept of Nash equilibrium, which has been and is considered a good model for stable outcomes of strategic games. So, despite the existence of other equilibrium concepts in the literature, John Nash's one is typically considered the most interesting and/or useful. Being a condition that may be described as a set of strategies representing optimal responses to other players' strategic decisions [11,12], it catches a special type of stable condition in which each player acts selfishly, searching for the maximum gain and holding supposedly right expectations with respect to other agents' decisions. As expected, computing Nash equilibria of nontrivial finite normal form games is often a hard problem and, over the years, many powerful methods have been developed to address this issue [11,12,22]. In general, the most adequate approach for computing Nash equilibria of strategic games depends on several factors, as whether the the objective is to obtain pure or mixed strategy equilibria, or it is necessary to find all existing equilibria.
The present article intends to introduce an optimization-based method of strategic game design aimed at obtaining payoff functions whose resulting games have previously established multiple Nash equilibria. That is to say, it becomes possible to specify in advance a set of Nash equilibria for a generic finite state, normal form game and the proposed algorithm computes payoff functions that will realize the desired equilibria -this work may be viewed as the continuation and extension of [21], that solved the problem for a single Nash equilibrium.
Although any effective evolutionary global optimization method could be used to find the desired payoff functions, the Fuzzy Adaptive Simulated Annealing method [6] is again the chosen algorithm, considering its amazing performance and flexibility in preliminary simulations. Among the successful initiatives for finding Nash equilibria of finite games are those using global optimization techniques [11,12,23]. One of them, due to R.D. McKelvey [11], may be used not only in the "given game => Nash equilibria" direction, but also in the opposite one: "given Nash equilibria => new game" -this viewpoint was explored in [21] and in this paper as well, demonstrating the large potential of the original result.
That point of view makes it possible to synthesize games with not only one, but several previously established Nash equilibria. This possibility is due to the way optimization-based design takes place -every design objectives and constrants must be translated into parametric functions that, when optimzed, conduct to the optimal result. Accordingly, in order to impose several equilibria to one single strategic game, it is sufficient to insert additional terms to the overall cost function, each one corresponding to a distinct Nash equilibrium -this approach was used to solve the inverse problem in [21], but using only one term, considering that there was just one given equilibrium, and the statement of the original theorem is compatible with this context. Here, we will show the reasons why it is possible to obtain even more, with the only inconvenient that objective functions get more and more complex, needing very powerful GO methods to find their global minima.
In the sequence, the paper will briefly describe the Fuzzy ASA paradigm, its application to the underlying problem and the obtained results.

The global optimization method
The global optimization method used in this paper is Fuzzy ASA [19], based on the simulated annealing paradigm and introducing a large set of dynamic and static control possibilities, which contribute for its good performance. Among its most significant features it is possible to cite re-annealing, quenching and several parameters that make it possible to change the behavior of the optimization process even in runtime, allowing the construction of control mechanisms aimed at improving the convergence process. It was designed so that users have the possibility to change the dynamics of the simulations, driving optimization sessions with enormous freedom. Other simple but very useful features of the original ASA method include the ability of adjusting each dimension of hyperrectangular domains and generating candidates with full control of each parametric dimension. Although the use of quenching may provoke the convergence to suboptimal regions, the Fuzzy ASA controller [17,19] usually is effective in avoiding such an inadequate behavio, making it possible to reach global minima. Actually, the original ASA structure is viewed as a dynamical system, and the added code implements a feedback loop by sampling the current value of the cost function, changing its inputs according to a control law that simulates a sort of rational behavior.
In summary, fuzzy control and built-in ASA mechanisms conduct temperature profiles, avoiding premature convergence.

Contribution of the present paper, motivation and related definitions
Although the problem of finding Nash equilibria of finite strategic games received a great deal of attention during the last decades and many solutions nowadays exist, the inverse problem of obtaining a complete set of payoff functions that realize one desired Nash equilibrium, with preestablished number of agents and individual strategies is new, having already some proposals for finding solutions, obtained by means of evolutionary techniques [21]. The present work aims to go further, also using stochastic global optimization to obtain games with multiple and predefined equilibria. To that end, some associated concepts must be precisely established. It is worth saying that this problem is closely related to important issues addressed in mechanism design theory [2,5,4,9,10,14,15].

Motivation and significance
Eric Maskin highlights in [9] the importance of the notion of Nash equilibrium as a solution concept in game theory. Besides, despite presenting limitations is some scenarios, the possible shortcomings are not really significant in the setting of mechanism design. The exprression "mechanism design" is related to a previous specification of a certain goal (Nash equilibrium or equilibria, representing favorable results) and a mechanism (here, a finite, normal-form game) that must be designed in order to achieve the desired result. In addition, in [9] it is stated that, in mechanism design, a set of preestablished constraints may be imposed, as having a unique Nash equilibrium [10].

Finite normal form games
In principle, finite, normal form games are models for interactions between agents. Each agent owns a number of alternatives to choose from a set of possible decisions. The benefits or gain a given player may receive when taking initiatives are established by means of payoff functions, quantifying the effect of actions of all decision makers.
A finite normal form game with pure strategies is formed by: A finite set {1, 2, ..., N } of agents or players.
The action sets S i , containing pure strategies for each player -a finite group of alternatives.
being m i the number of actions available to agent i.
By a −i is denoted the strategic profile (a 1 , ..., a i−1 , a i+1 , ..., a N ) obtained from (a 1 , ..., a i−1 , a i , a i+1 , ..., a N ) by excluding the action of player i. Besides, the notation (a i , a −i ) represents the original profile of actions.
N payoff functions associating real values to profiles.

Nash equilibrium -definition and some digressions
The study of strategic games is typically related to the need of predicting decisions agents will make in the most diverse contexts. Predominantly, the general belief is that players will follow the most (materially) favorable route, with best decisions depending on the other players' actions and, at decision time, they create a set of possibilities about the likelihood of other players' possible decisions. In this fashion, a method by means of which this can be made is necessary. A good starting point is the estimation of degrees of possibility for pure strategies, obtained from past experiences -this information should be enough to make predictions of how other players will choose their future actions -this will produce the mixed strategies, representing a refinement and creating an additional dimension of uncertainty handling. The definition of Nash (mixed) equilibrium includes all these components and it is possible to say that it is a decision profile such that players are not able to individually benefit from changing their present state -intuitively, when a game reaches a Nash equilibrium, no agent would have advantages with unilateral moves. From another viewpoint, Nash equilibria can be said to represent stable states, in which no agent wishes to abandon the respective position. Now, some formal definiitions are needed: The concept of mixed strategies is fundamental for the practical quantitative use of the perception of players with respect to the behavior of opponents. In this direction, probability mass funcions (p i ) are the devices by means of which this information is made effective -players are able to associate importances to each possible action of each player. By using them it is possible to compute expected payoffs for each agent. The corresponding formula for player i is: Accordingly, for a Nash equilibrium p * , no player has alternative actions that may result in higher gains than received by p * i , provided the others fix their positions in p * j (j ̸ = i).

The key proposition
As cited above, an amazing result due to McKelvey [11], relating global minimization of a specific function and Nash equilibria of the corresponding strategic game, made it possible to solve nontrivial finite normal form games by finding even certain unknown equilibria by means of evolutionary global minimization techniques [20]. The main cost function is founded on three matrix functions, defined below. Namely: where p ∈ ∆.
The final cost function is The original statement and use of the proposition may be expressed as: [11,12,23]).

The problem of attributing multiple Nash equilibria to a single game
Given a preestablished generic strategic game structure (number of players and alternatives available to each player), the problem of synthesizing payoff functions which realize one desired Nash equilibrium has been successfully addressed in [21]. However, in many practical settings it may be convenient to impose several mixed equilibria upon a single game in order to model real world scenarios. Hence, a proposal for attaining that goal follows.

Proposed method
Considering the previous proposition, relating a mixed profile which is a global minimizer of function v to a Nash equilibrium of the game that originated v (where the object to be found is a mixed profile), the problem to be solved now is: departing from a finite set of mixed profiles, would it be possible to find payoff functions whose associated game has those profiles as its Nash equilibria ? As noted above, if there is only one Nash equilibrium to be treated, the answer is affirmative if at least one associated global minimizer exists, as found in [21] -this is also tacitly conveyed by the original proposition, as highlighted in the just cited work. But another subtle interpretation of the theorem is worth noting: by constructing not one, but many "v functions", one for each desired Nash equilibrium, and summing them, one obtains a new "collective" cost function V -it happens that a global minimizer for V is also a global minimizer for each individual v, making all of them Nash equilibria for the same game, specified by the optimal parameters representing the payoff functions. Hence, as in so many other optimization-based design tasks, the global optimization process produces solutions in an automatic way. As before, the optimization method used gives the designer the possibiity of limiting the values of each component of payoff functions.
As expected, additional numerical conditions may be ensured by means of constraints so as to customize solutions for several types of problems.

Approach for solution
It is worth noting that, although the theoretical foundations of the proposed method are sound, its successful practical implementation totally depends on the good quality of the available global minimization apparatus. That said, the algorithm is now described (using Fuzzy ASA): Departing from the chosen game structure, compute the dimension of the configuration space; Find, for each imposed Nash equilibrium, the function corresponding to (7) In positive case, obtain the payoff functions and finish.

Numerical evidences
In order to illustrate the efficacy of the proposed paradigm, a small but representative group of tests was selected and may be used to obtain numerical results. The presented examples also demonstrate the power of the optimization-based design.
Comparing with the related literature it is possible to observe that in this paper (as in [21]) the opposite direction is taken, that is, the equilibria are given and one compatible game is obtained.
In terms of performance, in all tests Fuzzy ASA was able to reach global minimizers, furnishing the guarantee that proper payoff functions (and respective games) were found. This is so despite the fact that each cost function evaluation during the optimization process demands the creation and assessment of a different game -even so, most synthesis sessions have demonstrated low demand of computational resources. As usual, the Gambit software package [13] ( http://www.gambit-project.org ) was (and may be) used to compute sets of equilibria for games produced by the new algorithm, although in a few cases Gambit was not able to find all equilibria. Specifications compatible with Gambit are provided in a different file so that the given information may be easily confirmed by means of an independent tool. It is very important to note that the numerical patterns of each Nash equilibrium in the examples have been chosen so that it would be probabilistically infeasible to find the associated strategic games by means of "brute force" methods in the direction [game → equilibrium], that is, the validation is done by using regular configurations of probabiilities, mainly in the mixed equilibria. As expected, each global equilibrium has two probability mass functions and four probabilities assigned to each one, corresponding to four pure strategies. Each payoff function is formed by sixteen entries, and the game needs a total of 32 parameters to be defined. The respective Gambit specifications for the obtained games are:  presenting three probability mass functions and three probabilities attributed to each one. Each payoff function has 27 entries, and the game, 81 parameters to adjust.

Interpretation of results and recommendations for independent testing
Once more we remind the readers that the problem and solution presented in this work are new and, as such, can not be compared to previous results, considering they simply do not exist. Despite this, the furnished Gambit specifications make it possible to easily verify that the synthesized games have the initially given equilibria. Another important fact is that, in all cases, the final objective functions' values were under 10 −15 , that is, global minima were reached in practice.

Let us now comment the examples:
In Example 1 it is presented a scenario with two actors deciding about four distinct lines of action -this could be 2 food industries regulated by a central management unit which is trying to establish the most convenient dynamics so that the proportion for production of 4 different products of each one be given by 2, 3 , 4, or 5 possible and simultaneous Nash equilibria, representing multiple favorable consumption configurations, for instance. In this fashion, with the help of the proposed algorithm it is possible to open this possibility by positioning the preestablished Nash equilibria group into the overall set of equilibria of the resulting game.
Example 2 represents a context with 3 agents and 3 distinct moves available to each one -this setting may correspond to 3 firms forming a productive cluster and controlled by a central office which considers satisfactory to reach one out of 2 production profiles in the overall site so that the proportion of the production level of 9 different products (3 x 3) be given by any one of the proposed Nash equilibria. The obtained payoff functions contain the "rewards" capable of making a certain set of equilibria attractive and feasible.
Example 3 corresponds again to the interaction among 3 players, this time each one has 2 alternatives to choose from. As in the previous examples, the 3 chosen profiles were chosen to demonstrate the diversity of situations the proposed algorithm can handle and, moreover, the regular numeric patterns help to evidence the precision and efficacy of the reverse design. In this case, 2 experiments with 2 and 3 simultaneous Nash equilibria were used, conveying the desire of allowing the occurrence of multiple satisfactory production profiles.

The original problem
In [3], Section 4.8, it is formulated a very interesting problem involving a small group of large commercial firms (giant retailers) and their pricing policies in a given setting in space and time (USA/1989). The scenario to be studied is described as a price competition between two firms and follows some well-defined rules, including their approaches for attracting customers and improving their profits. In particular, normal and sale prices are fixed and assumptions about customer behavior are established. After a detailed explanation, it was possible to model the problem as a 2-player strategic game and obtain payoff matrices for both agents. Each player has a two element strategic set whose elements represent the decisions of charging normal (NP=US$600) or sale prices (SP=US$500), with NP higher than SP. Also, the unit cost of the goods to be sold is informed to be US$450.
Finally, the complete payoff matrix is shown to be Giant 2 N P SP Giant 1 N P 7500, 7500 7500, 8500 SP 8500, 7500 5500, 5500 Two pure and one mixed strategy Nash equilibria are found, and the mixed one indicates the existence of a pricing cycle in the dynamical sales processes. The meaning of this is: firms keep on alternating between normal and sale prices obeying a certain proportion of time. Due to the symmetry of the game, at the mixed equilibrium, each player keeps charging NP 2/3 of the time, and SP 1/3 in the rest. Therefore, the expected payoff of each one results in US$ 7500, exactly the same as in the (NP,NP) profile. So, in order to stay operating at the mixed equilibrium both companies must sustain this cyclic pricing behavior.

A derived problem
What if firm 1 decides to improve its expected profit and, at the same time, to change the time proportions of charging NP and SP, not necessarily with the same prices as before? Suppose that it wants to reach an expected payoff of US$ 10000 and practice a proportion of 5/7 for charging NP and 2/7 for charging SP, for operational reasons, meaning that five days a week NP holds and in the remaining two days, SP is practiced by agent 1. Also assume that firm 2 maintains the original scheme, that is, the desired mixed strategy equililbrium is {5.0/7.0, 2.0/7.0, 2.0/3.0, 1.0/3.0}. In principle, the solution is to find a game that features at least such a Nash equilibrium and, once played, is able to reach the preestablished conditions. How can this type of problem be solved?
What payoff matrix would be capable of inducing these new desired conditions?

The solution for the new scenario
Starting with the second answer, the payoff matrix which solves the overall problem is Giant 2 N P SP  The first question may be answered by saying that the main presented method is able to find a game with the new, desired Nash equilibrium. In addition, by modifying the overall cost function it is (and was) possible to impose the new expected payoff of player 1. This is done simply by adding a term which becomes null when the expected payoff of player 1 reaches the desired new value. Thus the global cost function is composed by terms that, once assuming the value 0, automatically satisfy the corresponding condition. In order to illustrate the implementation for this particular constraint, the code (in the C++ programming language) relative to the cited term is shown below. This code corresponds to the condition that player 1 must have expected payoff equal to US$10000, and the variable OverallCost was already being used as an accumulator for the objective function value in each iteration. At this point it already holds the numerical conditions for the positioning of the desired equilibria, and the shown addition of aux 2 combines the two constraints. Array PlayerPayoff holds the payoff matrices and VectorPMF1 the proportions for the two types of prices.

Discussion for the example
This simple example shows very well how it is possible to reach certain results, hardly obtainable by conventional means, using the proposed method.
In this fashion, it is possible to impose a certain set of Nash equilibria and, in many occasions, other conditions, provided the overall cost function keeps its global minimum value at zero. Although the problem demanded only one special Nash equilibrium, it would be possible to have further ones by modifying the existing cost function. Naturally, the resulting payoff matrix has to be implemented in practice in order to make it feasible even the theoretical existence of the equilibrium. To do that, player 1 must adjust some parameters in order to reach found payoff values. Although the number of buyers is one of the components influencing payoff values, it is usually beyond the control of firms to set such a variable. Therefore, NP and SP of player 1 are the control parameters that can be changed to obtain the desired effect. By doing so, the symmetry of the original game is commonly lost and some premises must be altered (or abandoned) as well. In the present setting, it is certainly possible to get adequate numbers which allow designers to arrive to the payoff matrix.

Conclusion
This work describes a new approach to synthesize finite strategic games with given structure and pre-selected multiple Nash equilibria in an automatic way, showing how that is possible by using an evolutionary global optimization method. The new findings are based on two main pillars: a proposition due to R. D. McKelvey, relating Nash equilibria and global minima of certain objective functions (Liapunov functions for specific dynamical systems, as described in [11]) and the utilization of an effective evolutionary global optimization algorithm (Fuzzy ASA). It is worth to highlight that the results here described could hardly be obtained without the flexible structure of evolutionary methods, which make it possible to employ techniques that may be considered infeasible when dealing with conventional, gradient-based methods. The usefulness of the results resides in the possibility of not only positioning several pre-set equilibria simultaneously, but also fine tuning the frequency of the several alternatives available to each player. Other positive points are the easiness of implementation, independent tuning of the range of payoff functions' components and provision for inserting further constraints. For the sake of better understanding presented results, a final example describing a price competition game between two large retailers is explained, including their approaches for attracting customers and improving sales. In particular, normal and sale prices are initially fixed and premises about customer behavior are established. After some reasoning, it was shown to be possible to model the problem as a 2-player strategic game and obtain payoff matrices for both agents, as exposed in [3]. In order to apply the new method, some significant modifications in player 1 behavior were proposed. After submitting all requisites and previous conditions to the algorithm, new payoff matrices were produced and they were able to satisfy all previously stated constraints.