Inefficient Learning Leads to Efficient Coordination: A Repeated Stag-Hunt Game with Learning Agents

1. Introduction

The emergence of cooperation among humans has become a significant area of research since the late XX Century [1,2]. Much of this body of work examines the evolutionary dynamics that favor altruistic behavior in natural environments, framing the issue as a problem of population dynamics [3,4]. However, cooperation is pervasive and can also be observed in pair interactions within competitive contexts. In this type of interaction, cooperation between both individuals is usually desirable, as it allows everyone to obtain the maximum possible benefit, the so-called social optimum. But from an individual point of view, the choice to cooperate is not always so straightforward. In some scenarios, cooperating or defecting may be the best and safest choice for the individual, regardless of the actions of the other, making it easier to make a decision (or predict actions). However, in other situations, this is not always the case: there is no universal best choice, and predicting the implications of one’s decisions becomes more difficult, as they are conditioned by the actions of the other.

In game theory, this situation of strategic uncertainty is formally described by the existence of multiple Nash equilibria, outlining a fundamental coordination problem. To study this phenomenon, numerous frameworks, under the name of coordination games, have been proposed. In these settings, unlike social dilemmas characterized by an implicit conflict of interest, players achieve better results if they make the same choices, with the consequent absence of a unique dominant strategy [5].

From a theoretical point of view, the presence of multiple equilibria represents a deadlock: standard equilibrium analysis does not allow behavioral predictions to be made, forcing theorists to be agnostic about the actual equilibrium selection. As highlighted by [6], this gap has stimulated experimental exploration of coordination, revealing a general tendency for people to coordinate, albeit not always on the most efficient outcome [5,6]. In this broad and diverse strand of literature, scholars have investigated different aspects and possible determinants of coordination. Sociocultural factors (e.g. [7,8,9,10,11,12]), such as the existence of shared social representations and cultural standards, may facilitate successful coordination thanks to their effect on players’ expectations (implicit or otherwise). Studies on cognitive aspects, on the other hand, have instead investigated internal reasoning processes, exploring decision-making styles (intuitive vs. deliberative) and players’ levels of sophistication (e.g., [13,14,15,16,17,18,19]). The question of equilibrium selection has also been extensively addressed by more theoretical strands of the literature, such as evolutionary game theory. These numerical studies have investigated coordination games in both well-mixed populations and network structures, testing different update rules, such as myopic best response or imitative mechanisms [20]. These studies have shown how topological features can, in some cases, facilitate coordination [21,22], although this kind of effect is not generally universal [23]. Finally, a minority of studies have explored coordination by moving away from the evolutionary approach in favor of cognitive and learning mechanisms [24,25,26,27]. A common feature of these works is the use of relatively simple learning processes and a deliberately unsophisticated representation of the agents’ cognitive mechanisms. By eschewing the traditional assumption of perfect rationality, this perspective reveals that coordination on the social optimum can be successfully achieved even through minimal adaptive heuristics.

Following this latter perspective, in this work we studied the coordination dynamics between two agents who participate in a repeated coordination game. In particular, we explored how fundamental learning processes can shape cooperation and strategic coordination in an exclusively pair-wise interaction, where the dynamics of the game and the strategic outcomes are initially unknown and more than one Nash equilibrium is possible. The objective of this work is twofold, as it aims to i) model some recent experimental results and ii) to show how simple and imperfect cognitive mechanisms are sufficient to navigate strategic uncertainty and resolve coordination problems in dyadic settings, without the need to employ complex deductive reasoning.

The paper is organized as follows: Section 2 provides the general configuration and dynamics of the model; the results obtained from the numerical simulations and a comparison with some recent experimental results are reported in Section 3. Finally, Section 4 discusses the results, limitations, and future prospects.

2. Materials and Methods

A classic game-theory framework that investigates coordination between two agents is the stag-hunt (SH) game [28]. The game represents a situation in which two hunters can cooperate to capture a larger prey, the Stag, or decide not to do so and settle for a smaller one (Hare). In the first case, the payoff is greater, but there is a risk of getting nothing if the other player decides not to cooperate. In the other case, the hare does not require coordination between the two players and represents a sure payoff, albeit smaller than the stag. This scenario inevitably generates tension between two strategic criteria, payoff and risk dominance, resulting in two possible equilibria. In the first case, both agents coordinate by choosing Stag. This is the Pareto optimal and payoff-dominant equilibrium, as both players will obtain the maximum expected payoff. On the other hand, agents both choose Hare, obtaining a secure but significantly lower payoff. This risk-dominant equilibrium is the result of the strategic uncertainty inherent in the SH game: choosing Hare is in fact an optimal strategy given the high risk of miscoordination. The SH game therefore exhibits two pure Nash equilibria, and once one of these is reached, the agents have no incentive to change their strategy.

The following paragraphs in this section describe the general configuration adopted for the SH game, the update rule that governs the strategic choices of agents, and the different scenarios tested.

2.1. General configuration

Let us consider two players,“player 1” and “player 2,” engaged in a repeated SH game. Each agent i is characterized by a probability $p_i^S$ of choosing Stag (S), while $p_i^R=1-p_i^S$ is the complementary probability of choosing Hare (H). Depending on their chosen strategy, each player receives a reward determined by the payoff matrix $\widehat{P}$ given in Table 1, then updates the probability of the adopted strategy according to the overall reward obtained up to that round of the game.

When both agents choose Stag, the payoff obtained by each player is given by $r+\epsilon$, where r is the gain for (Hare, Hare) and the parameter $\epsilon$ represents the coordination prize for both moving on Stag. It is important to note that in this version of the game, a player adopting the Hare strategy gains the reward r independently of the opponent’s choice. The introduction of this variation allowed us to test the ability of the players to learn the best option even though the alternative guaranties the same reward in any case. This makes such SH closer to a Prisoner’s Dilemma game, with the difference that here the best option is also a Nash equilibrium.

At the end of each round, the agents’ cumulative payoff is updated based on the strategies adopted. The latter represents the agent’s total “energy”, and a portion of it is required to play the game. Players repeat the game until a final configuration is reached, that is, the probabilities have reached stable values, at least on average. If the energy of any agent reaches zero, the game stops. This mechanism therefore imposes a constraint on the agents and reflects a key feature of the experimental version described in Section 3.3 and in the Appendix A.

2.2. Dynamics

2.2.1. Initial conditions

The initial energy of each agent is set to a value of 10 units at the beginning of each realization. Every time they play a game stage, they lose 0.5 energy units: this quantity represents the effort players put into playing the game. We would like to emphasize that, in all the simulations performed and in the different scenarios tested, the agents managed to obtain sufficient energy and never reached zero. For this reason, we will not address the energy dynamics of the agents in the following sections.

As regards initial strategies, at time step $t=0$, the probability of each agent choosing Stag $p_i^S$ is equal to $p_1^S=p_2^S=0.5=p_1^R=p_2^R$. This implies that, at the beginning of each simulation, agents do not know the rules of the game and the payoffs associated with the strategies.

2.2.2. Update rule

At the end of each game round, players update the probabilities associated with the two strategies based on the payoff obtained. More precisely, if at the N-th stage the player i has chosen the strategy K, the probability of choosing K will be updated according to the following learning rule:

$$p_i^K\left(new\right)=p_i^K+{\alpha }_i(1-p_i^K){\displaystyle \frac{{\mathrm{\Pi }}_K^i}{(r+\epsilon )N}},$$

(1)

where ${\alpha }_i$ is the learning rate of the player i, ${\mathrm{\Pi }}_K^i$ is the total amount of reward gained by the player until the N-th stage when adopting the strategy K, and the normalization factor $1/\left[\right(r+\epsilon \left)N\right]$ guaranties that $p_i^K\in [0,1]$ at each time.

The learning rate ${\alpha }_i\in (0,1]$ of each agent i is assigned during initialization and does not vary over time. This parameter modulates the impact of the result obtained by the strategy K adopted at the time step t, dictating the extent to which the agent reacts to experiences. The introduction of this parameter allows us to increase the realism of the model, representing the cognitive abilities of the agent, as well as allowing us to study the interaction of heterogeneous agents, characterized by different values of $\alpha$.

2.2.3. Error probability

At the beginning of each round, each agent i adopts a strategy K based on the corresponding probability $p_i^K$. This implies that a successful strategy, in terms of payoff, will have a higher probability of being adopted. This mechanism, although not deterministic, does not take into account the possibility of the agent making mistakes. For this reason, among some of the scenarios, we will include an error probability ${\mu }_i$, assigned to each agent and constant over time. In these cases, agent i initially determines the strategy K to adopt based on $p_i^K$; once the strategy has been selected, agent i can choose the opposite strategy with probability ${\mu }_i$.

2.3. Simulations

The learning rates ${\alpha }_i$ and the error probabilities ${\mu }_i$ allowed us to explore different scenarios. The first case (baseline model) we tested refers to the representation of perfectly rational agents, where players exhibit the maximum learning rate (${\alpha }_1={\alpha }_2=1$) and do not make mistakes (${\mu }_1={\mu }_2=0$). Next, we evaluated the impact of the learning rate alone by testing different combinations of the values of ${\alpha }_1$ and ${\alpha }_2$ (keeping ${\mu }_1={\mu }_2=0$). Similarly, we repeated the simulations to evaluate the influence of the error probabilities $\mu$ alone on the agents’ strategies, for different combinations of ${\mu }_1$ and ${\mu }_2$, while keeping ${\alpha }_1={\alpha }_2=1$. Finally, we investigated the combined effect of both learning rate and error probability.

For each scenario, we studied the impact of $r\in [1,10]$ on the evolution of the probabilities associated with the two strategies, Stag and Hare. After assigning the agents their respective values for the parameters $\alpha$ and $\mu$, we performed $T={10}^3$ independent simulations for each scenario and r value, and computed the final average probabilities. For each run agents continued to interact until convergence or the maximum number of iterations ($t={10}^5$) was reached. The convergence criterion was satisfied if the variation in the average probability $(p_1^S+p_2^S)/2$ was less than $ϵ<0.01$ in a time window of ${10}^2$ consecutive iterations.

3. Results

3.1. Baseline model

In the baseline version of the model, we set ${\alpha }_i=1$ and ${\mu }_i=0$, $\forall i$. Therefore, agents learn at the fastest pace possible and never make mistakes. Considering perfectly rational learning agents is of course a huge approximation, nevertheless, we start with this toy version for two main reasons: firstly, if we are able to capture some of the experimental results with such simplified model, this would indicate that the selected features are sufficiently powerful to provide a reasonable description of reality; secondly, it can work as a baseline to compare the effect of further refinements of the model itself.

Figure 1 reports the average probability of the two agents choosing Stag (S), which is equivalent to the probability that a single simulated experiment ends up with players always choosing S. As is easy to see, for $r\to +\infty$ we have $p^S\to 1^{-}$: an expected outcome, since if the Stag option is infinitely more profitable than Hare, there is no chance that the Hare choice can be selected.

When $r=4$, the average probability that both agents choose S is $p^S=0.5$. This indicates that players learn with a probability larger than $50\%$ that $(S,S)$ is actually the best option when the associated reward is at least four times more profitable than the Hare’s payoff.

This outcome can theoretically be justified as follows. From Eq. (1), we can write down the average elementary variation $\Delta P_t$ of the probability $p^S$ after the $t-$th interaction (of course, the same can be done for $p^H=1-p^S$). Assuming that ${\alpha }_i=1,\forall i$ and that, by symmetry, the probability of choosing the strategy K evolves equally for both players, after a few passages we get the following:

$$\Delta P_t^S=P_t^S(1-P_t^S){\displaystyle \frac{P_t^S{\mathrm{\Pi }}_S-{\mathrm{\Pi }}_H}{(r+\epsilon )t}},$$

(2)

where

$${\mathrm{\Pi }}_s={\displaystyle \sum _{j=0}^N}{\left(P_j^S\right)}^2·(r+\epsilon );{\mathrm{\Pi }}_h={\displaystyle \sum _{j=0}^N}(1-P_j^S)·r,$$

(3)

that are the average rewards gained up to the N-th round when a player has adopted the Stag/Hare strategy, respectively. From Eq. (2) it is easy to see that $P_N^S=P^{*}=0$ and $P_N^S=P^{+}=1$ are two fixed points of the dynamics, which is confirmed in simulations since for every independent run both players end up with probability to choose Stag equal to 0 or 1 for both, i.e. they always coordinate. There is also a third equilibrium, $\overline{P}$, that is,

$$\overline{P}={\displaystyle \frac{{\mathrm{\Pi }}_H}{{\mathrm{\Pi }}_S}}.$$

Now, if $\overline{P}$ is a physical equilibrium, that is, if $\overline{P}\in (0,1)$, then it is unstable. Indeed, let us assume that both players have reached $P^S=\overline{P}$, if they go again to Stag, for the rule given in Eq. (1), at the subsequent stage the probability will tend to increase again and go away from $\overline{P}$.

Therefore, we have established that in a single run players coordinate, and the final configuration is always one where both always go to the Stag or always go to the Hare. We can evaluate the probability that a single run ends up with both players choosing Stag, i.e. we can explain the results shown in Figure 1. To this end, let us consider the interactions in the very early stages of the game, when we can pose $p^S=p^H=0.5$ for both players. Focusing on a given player, at the first interaction its probability to choose Stag will change as follows:

$$\Delta p_0^S=\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}}\hfill & \mathrm{with}\mathrm{probability}{\displaystyle \frac{1}{4}}\hfill \\ \hfill \\ -{\displaystyle \frac{r}{2(r+\epsilon )}}\hfill & \mathrm{with}\mathrm{probability}{\displaystyle \frac{3}{4}}\hfill \end{array}\right.$$

(4)

Of course, on average, the double Stag is always the best configuration for the interacting players, but since at the beginning they do not know the rewards, they have to discover that it is the most convenient option just by trying. Now, in order to understand that mutual Stag is the best option, both players have to go both on S to collect the payoff: on average, this happens once every four runs. More precisely, in the early stages of the game, both players select the strategy at random, so that only with a probability of $1/4$ they go for the Stag at the same time: therefore, only for $r\ge 4$ it becomes convenient for the Stag strategy, as it rightly appears in the figure.

3.2. Learning rate and error probability

In the baseline model, agents update the probability of choosing Stag based on the payoffs obtained. The impact of the latter is maximal and, in the first interactions, it is able to significantly modify the behavior of agents. Furthermore, decisions are not subject to errors and agents possess the same characteristics. However, this scenario may not represent reality. Humans do not learn at the same speed and are prone to making mistakes. In order to better capture these characteristics, we will therefore evaluate the impact of two factors on the behavior of agents: i) the learning rate ($\alpha$) and ii) the possibility of making mistakes ($\mu$). These two characteristics will be studied separately and in combination. The analysis of the following results will focus on the case where the payoff of stag $S=4$, i.e., the payoff where agents show a probability of cooperating equal to $p^s=0.5$ in the baseline model.

3.2.1. Learning rate effect

The pair of agents 1 and 2 are assigned ${\alpha }_1$ and ${\alpha }_2$, respectively, where ${\alpha }_1,{\alpha }_2\in (0,1]$. The learning rate controls how quickly the agent learns from its experiences: a high value of $\alpha$, especially in early interactions, can cause the agent to drastically change its strategy, while low values make the decision-making process more stable and slower. In the baseline model, both agents had a value of $\alpha =1$. In this first extension, we will consider not only the impact of lower learning rates, but also the interaction between heterogeneous agents characterized by different values of $\alpha$.

Figure 2 displays the evolution of $p^S$ as the Stag’s payoff ${\pi }_S$ increases, for four different scenarios that reflect the interaction between agents with different values of the learning rate $\alpha$, including the baseline case (${\alpha }_1={\alpha }_2=1$). The scenarios were not selected based on any specific criteria, but are intended simply to explore different combinations of $\alpha$, where the agents exhibit increasing values of the learning rate ($0.2,0.5,1$), in addition to a mixed case ($0.8,0.2$) where agents learn at different speeds. As we have already seen in Section 3.1, as the Stag’s payoff increases, we observe higher values of $p^S$ for any combination of ${\alpha }_1$ and ${\alpha }_2$. However, the highest values are observed as the learning rate decreases, particularly when both agents exhibit a very low learning rate $\alpha$, as we can see for ${\alpha }_1={\alpha }_2=0.2$. Also in the case of heterogeneous agents, such as in the scenario ${\alpha }_1=0.8$ and ${\alpha }_2=0.2$, where only one of the agents has a low learning rate, we observe higher average values of $p^S$ compared to the baseline.

To better understand the underlying dynamics, we can examine Figure 3, which shows the probability of choosing Stag for each combination of the values ${\alpha }_1$ and ${\alpha }_2$ when ${\pi }_S=4$. The dark red areas indicate higher values of $p^S$, while the dark blue areas indicate a low probability of coordinating on Stag. As can be clearly seen, the probability $p^S$ varies widely depending on the pair of values ${\alpha }_1$ and ${\alpha }_2$, ranging from $p^S\sim 0.5$ in the baseline model ($\alpha =1$) to $p^S\sim 0.9$ for ${\alpha }_1={\alpha }_2=0.1$. As the values of $\alpha$ decrease for both agents, the probability of cooperation increases monotonically.

The highest value of $p^S$ is therefore reached when both agents exhibit the lowest learning rate tested ($\alpha =0.1$). In heterogeneous pairs of agents with different $\alpha$, the presence of a “slow learner” is generally beneficial for cooperation, increasing the overall probability of choosing Stag. For example, in the case where ${\alpha }_1=1$, $p^S$ tends to increase as the value of ${\alpha }_2$ decreases. The results therefore seem to suggest that slower learning benefits cooperation between agents. Gradually updating one’s behavioral rule provides sufficient space to test different strategies and avoid becoming stuck in a strategic stalemate.

3.2.2. Error probability effect

In the previous section, we examined the effect of the learning rate on the probability of coordination in Stag, but without considering the possibility of making mistakes. To analyze this different scenario, let us again consider two agents, each endowed with its own probability of making mistakes, $\mu$. For now, let us assume a constant and maximum learning rate ${\alpha }_1={\alpha }_2=1$.

As we can see in Figure 4, the introduction of $\mu$ can either increase or decrease the probability of coordinating on Stag $p^S$. Compared to the baseline model (${\mu }_1={\mu }_2=0$), a small probability of making mistakes can be beneficial, as can be observed when $\mu =0.3$ for both agents. Conversely, high values of $\mu$ tend to result in values of $p^S$ that are lower than the baseline as the payoff of Stag ${\pi }_S$ increases. In the case of heterogeneous agents (${\mu }_1=0.3,{\mu }_2=0.8$), the presence of a single agent with a low probability of making errors is sufficient to offset the negative effect of a high $\mu$, resulting in an overall increase in $p^S$ compared to the baseline.

If we examine the values of $p^S$ for different combinations of ${\mu }_1$ and ${\mu }_2$ (${\mu }_1,{\mu }_2\in [0,1]$) when ${\pi }_S=4$ (Figure 5), we can see how the probability of choosing Stag initially tends to rise as $\mu$ increases. The maximum point is reached when both agents have a value of $\mu$ in the range 0.2-0.3. Moving towards higher values, the probability of cooperating gradually decreases. When the maximum value $\mu =1$ is reached, the value of $p^S$ is close to the case $\mu =0$. In this region, high behavioral instability due to frequent mistakes makes any form of learning difficult, if not impossible. Keeping the error probability ${\mu }_1$ of one of the two agents in the optimal region 0.2-0.3 fixed, we can observe that for increasing values of ${\mu }_2$, the probability $p^S$ is still relatively high ($p_{{\mu }_2=1}^S\sim 0.75$), compared to the baseline model.

In this scenario, despite rapid learning, the possibility of making mistakes represents another way to experiment with different strategies, with the same positive effects seen in the case of a low learning rate. In fact, even a small chance of making mistakes is enough to allow agents to break through possible strategic deadlocks. In the case of an agent with a high probability of error, this can still be beneficial if the other party exhibits a relatively low probability of errors.

3.2.3. Combined effect of learning rate and error probability

Let’s now consider the joint impact of the learning rate $\alpha$ and the error probability $\mu$ on the probability of cooperating $p^S$. Figure 6 shows three different scenarios: for each combination of ${\mu }_1$ and ${\mu }_2$, we assign the agents the maximum (${\alpha }_1={\alpha }_2=1$) and minimum (${\alpha }_1={\alpha }_2=0.1$) learning rate values, as well as a mixed case (${\alpha }_1=0.8,{\alpha }_2=0.2$).

As seen in the previous section, when the learning rate of both agents is maximum, we observe the smallest probability of choosing Stag; conversely, when $\alpha =0.1$, the value of $p^S$ is the highest. On the other hand, the effect of the error $\mu$ is non-linear: an error probability between 0.2-0.3, even for a single agent, is usually beneficial.

Combining the two factors, we observe how their interplay contributes to widening the region of high coordination (red areas). In particular, as the learning rate decreases, the optimal interval for coordination given by $\mu$ tends to widen. From a narrow range of approximately $0.2\le \mu \le 0.3$ with ${\alpha }_1={\alpha }_2=1$, the interval widens significantly, up to approximately $0\le \mu \le 0.5$ when ${\alpha }_1={\alpha }_2=0.1$. We can observe this effect even in the case of heterogeneous agents (${\alpha }_1=0.8,{\alpha }_2=0.2$), where the presence of a single slow learner is capable of extending the positive effect of $\mu$.

These results therefore seem to suggest a specific synergy between the learning rate and the error probability that can promote coordination. While decreasing learning rates tend to increase $p^S$, the noise introduced by $\mu$ is beneficial only if limited. However, the detrimental effect of overly frequent errors can be counterbalanced by a gradual learning process, promoting cooperation even in situations where behavior is more erratic.

3.3. Comparison with experimental results

In this section, we will compare the results of our simulations with a recent experiment (see Appendix A), where pairs of participants played a computerized version of the SH game. Unlike other experimental designs, players were not provided with any information about the structure and rules of the game, in order to stimulate learning and exploration. During the game, participants were able to observe their partner’s moves, but no communication was possible. In this version of the SH, each player was presented with a horizontal grid, whose two ends represented the Stag and the Hare, and could move to the right or the left to reach one of the two ends. Each move involved a cost in “energy” to be subtracted from their initial endowment, similarly to the mechanism employed in our computational model. Each session consisted of at least 100 rounds. In total, 45 dyads completed the experiment, resulting in 5186 rounds. The payoff matrix used is reported in Table 2.

To evaluate the dynamics of cooperation, we analyzed when and how the dyads reached a stable joint strategy within each round. For the purpose of empirical comparison, cooperation is intended as convergence on a strategy. Furthermore, convergence was defined as the first moment within a round when both players simultaneously selected a defined joint state: $(S,S)$ (BothStag), $(H,H)$ (BothHare), $(S,H)$ (Split). For each dyad and round, we identified the earliest move corresponding to a valid joint state and computed the convergence time as the maximum elapsed time between the two participants, to account for slower players. Players quickly learned that the $(S,S)$ strategy was the most profitable, adopting it most frequently (2915 rounds), followed by the $(H,H)$ strategy (1408 rounds) and the least profitable $(S,H)$ strategy (863 rounds). The behavioral results also highlighted a rapid conversion within each round, with a mean convergence time of 334 milliseconds (Figure 7). Most rounds reached a stable configuration within 834 milliseconds, implying early mutual adaptation.

Early convergence suggests that participants rapidly coordinated their strategies, requiring only a few movement corrections. The speed of the players’ actions would seem to suggest a tendency toward a more intuitive and less deliberate decision-making approach, as assumed in our model. Moreover, the predominance of the $(S,S)$ state (BothStag), indicates that cooperative balance was not only achievable, but also more stable once reached (Figure 8). Finally, similar to our computational results, in more than half of the cases players were able to coordinate by both choosing Stag.

4. Discussion

In this work, we simulated the learning dynamics of a couple of agents interacting in a Stag Hunt game, where one strategy (Hare) provides a reward independently from the opponent’s choice, while the other one (Stag) guarantees a larger payoff but only if both players adopt it. Therefore, to achieve the best outcome, agents have to learn not only which is the best behavior, but also to coordinate and choose it simultaneously. In order to describe and analyze the dynamics at stake, we considered a very simple model where the agents update their inner probability to choose a given strategy on the basis of the rewards accumulated during the entire interaction when such strategy has been adopted. In a refined version of this model, we tested the effect of agents’ (bounded) cognitive abilities on the fate of the dynamics, i.e. the possibility to learn at a slower rate and to make mistakes, that is, to adopt the opposite strategy selected by applying the learning rule.

Despite the oversimplification of the models, we have been able to reproduce some important features observed in real experiments. In particular, we see that the players always coordinate, that is, they learn that adopting the same behavior is always better (or, at least, not worse) than choosing different strategies: this is in good accordance with empirical findings described in Section 3.3, where the couples of human subjects who do not learn to coordinate are much less likely to be observed than the opposite case. These results also appear to be consistent with other experimental findings in the literature: for example, in [29], dyads formed by the same players over time are able to coordinate to achieve the most efficient outcome.

As shown by the experimental results described in the previous section, in our model, the probability to find the best option (coordinating on Stag) increases with the reward provided by the corresponding strategy. This finding also emerges in some experimental studies on the optimization premium [30] or the salience of the payoff [31], although the literature remains mixed on the role played by payoff differences in the strategies adopted.

Counterintuitively, but not surprisingly, adding a small but not negligible amount of deficiencies to the agents’ behavior helps them to find the best option more efficiently. In fact, for fast-learning and flawless agents, selecting Hare always increases the likelihood of this strategy, regardless of the partner’s behavior, but the same cannot be said for choosing Stag. Therefore, high-performing agents can understand that coordinating on Stag is the most profitable behavior only if both players opt for Stag at the beginning of the interaction, which on average happens with probability 1/4 (indeed, the final average probability to choose Stag becomes larger than 1/2 when the reward for mutual Stag is larger than Hare’s reward). The asymmetry inherent in the SH game and its payoff structure can therefore penalize the best-performing agents, who may remain stuck in a less remunerative position: therefore, letting the players to "make mistakes" allows them to explore more strategic options and understand if there are better configurations, given that this probability to make such mistakes is not too high in order to avoid random behavior. This result appears to be consistent with previous studies in the literature, in which agents equipped with simpler strategic mechanisms, such as imitation [32], random walks [24], or the use of heuristics [13], are able to coordinate their actions to achieve the most efficient outcome.

The main limit of the models presented is that they are classical game-theoretical models where players have to decide instantaneously their strategy with no information about the opponent’s intentions. In the experimental setting described in Section 3.3, each player decided whether to choose Stag or Hare while being able to observe the direction taken by their partner. Information about the behavior of other players can certainly have an impact on decision-making processes, in addition to the learning and evaluation abilities of each individual player that we investigated in this work. For this reason, we believe that future extensions of this model should integrate the role of such information and different game mechanics, such as the absence of simultaneous choices, in the decision-making processes of agents.