Eco-evolutionary dynamics with environmental feedback: Cooperation in a changing world

A unified approach to understand the feedback-evolving games in which the strategies and the environment coevolve is proposed by Weitz et al. in [35]. We begin from the introduction of a generalized environment-dependent payoff structure in two-player games:

$$\begin{eqnarray} A(n)&= (1-n) A_0 +n A_1\\ &= (1-n) {\left[\begin{array}{c@{\quad}c} R_0 & S_0\\ T_0 P_0 \end{array} \right]} + n {\left[\begin{array}{cc} R_1 & S_1\\ T_1 P_1 \end{array} \right]}, \end{eqnarray} \tag{ 1 }$$

where $0\leq n \leq 1$ denotes the current state of the environment. A₀ and A₁ represent the payoff matrices that have a unique Nash equilibrium corresponding to mutual cooperation and mutual defection, respectively. Therefore, R₀ > T₀ and S₀ > P₀ while R₁ < T₁ and S₁ < P₁. Intuitively, when n = 0, i.e., the environment is depleted, incentives are offered to encourage cooperation behaviors so that the resources can be restored. On the other hand, when n = 1, i.e., the environment is replete, the payoffs favor unilateral defection, which accounts for the overuse or overexploitation of a resource. This environment-dependent payoff matrix can be further written as

$$\begin{eqnarray} A(n) = {\left[\begin{array}{c@{\quad}c} R_0-(R_0-R_1)n & S_0-(S_0-S_1)n\\ T_0-(T_0-T_1)n & P_0-(P_0-P_1)n \end{array} \right]}.\quad \end{eqnarray} \tag{ 2 }$$

Thus, the fitnesses of player 1 and player 2, denoted as r₁ and r₂, can be calculated as

$$\begin{equation} \left\{\begin{array}{r@{~}c@{~}l} r_1(x,A(n)) &=& x(R_0-(R_0-R_1)n)\\ &&+ (1-x)(S_0-(S_0-S_1)n),\\ r_2(x,A(n)) &=& x(T_0-(T_0-T_1)n)\\ &&+ (1-x)(P_0-(P_0-P_1)n). \end{array}\right. \end{equation} \tag{ 3 }$$

Finally, the replicator dynamics of this coupled evolutionary system are described as follows:

$$\begin{equation} \left\{\begin{array}{r@{~}c@{~}l} \epsilon \dot{x} &=& x(1-x)\left[ r_1(x,A(n))-r_2(x,A(n))\right],\\ \dot{n} &=& n(1-n)f(x), \end{array}\right. \end{equation} \tag{ 4 }$$

where represents the relative speed at which individual actions reshape the environment. The logistic term $n(1-n)$ guarantees that the environment state is confined to $[0,1]$. The sign of feedback function f(x) determines the evolution direction of n, which either decreases towards environmental degradation or increases towards environmental enhancement when f < 0 or f > 0, respectively. In [35], the environment state is assumed to be modified by the population actions of cooperation and a simple linear feedback mechanism is adopted:

$$\begin{equation} f(x) = \theta x-(1-x), \end{equation} \tag{ 5 }$$

where $\theta>0$ indicates the strength of cooperators in enhancing the environment. See the complete modeling schematic in fig. 1.

Zoom In Zoom Out Reset image size

Under this framework, Weitz et al. show the emergence of persistent oscillating loops in a special case where the payoff matrices A₀ and A₁ have an embedded symmetry. The orientation of orbits in the $x\text{-}n$ phase plane is always counterclockwise, which in intuitive sense can be explained as four phases that the system evolution experiences: beginning from a depleted environment with few cooperators, to motivated cooperation, to a restored environment, to invading defection, and finally going back to the depleted environmental state. Further, they extend the analysis to asymmetric payoffs conditions and find that the fast-slow $(0<\epsilon \ll 1)$ dynamics can converge both to a heteroclinic cycle and to a fixed point. The former is called "oscillating tragedy of the commons" and it emerges when

$$\begin{equation} \frac{P_1-S_1}{T_1-R_1}>\frac{S_0-P_0}{R_0-T_0}. \end{equation} \tag{ 6 }$$

It is also worth noting that the qualitative outcome of coupled systems in [35] is independent of the relative feedback speed . However, in contrary, Tilman et al. proposed a more general framework of eco-evolutionary games and found that the dynamical behaviors actually largely depend on the relative timescale [36]. Three different environmental dynamics are incorporated in their work: intrinsic resource growth, intrinsic resource decay, or environmental tipping points. Generally, this eco-evolutionary system is written as

$$\begin{equation} \left\{\begin{array}{l} \dot{x} = \epsilon_3 x(1-x)\left[ \pi_1(x,n)-\pi_2(x,n)\right],\\ \dot{n} = \epsilon_1 f(n) + \epsilon_2 h(x,n), \end{array}\right. \end{equation} \tag{ 7 }$$

where $\epsilon_1$, $\epsilon_2$ and $\epsilon_3$ represent the timescale of intrinsic dynamics of the environment, the timescale of the environmental impact of the current population strategies and the timescale of strategy evolution, respectively. In particular, f(n) governs the intrinsic dynamics of the environmental factor n, while h(x, n) aggregates the influence of population cooperation behaviors on environmental changes. And $\pi_1$ and $\pi_2$ are the fitnesses of cooperation and defection, respectively. Accordingly, a renewable resource can be described as

$$\begin{equation} \dot{n} = \epsilon (r-q(e_L n+e_H (1-n))) (x-n), \end{equation} \tag{ 8 }$$

where is the unified relative timescale, r is the intrinsic resource-renewing rate. e_L and e_H are the resource harvest efforts of low- and high-effort strategies, respectively. And q maps harvesting efforts into the rate of resource reduction. See [36] for detailed derivation. Similarly, a decaying resource can be calculated as

$$\begin{equation} \dot{n} = \epsilon \alpha (x-n), \end{equation} \tag{ 9 }$$

here α is the decay rate of the resource. Finally, the environment feedback described in [35] can be considered as a special case caused by a single environmental tipping point without any intrinsic environmental dynamics. Based on this framework, Tilman et al. show that all the abundant physical phenomena arose from the feedback-evolving system can be interpreted by the incentives for individual behavioral changes and the relative timescale of environmental vs. strategic changes.

The modeling extensions that take into consideration the effects of spatially structured interactions or the environmental heterogeneity can be further obtained in [39] and [40], respectively.

Coevolutionary dynamics in public goods game. It has been experimentally proven that eco-evolutionary feedback loops widely exist in microbial systems where cooperations often arise due to the secretion or the release of public goods [41–44]. In specific, cooperators are naturally given preferential access to these common goods, which leads to the asymmetrical payoff-dependent feedback that drives the coevolution of ecological properties and the strategies. Likewise, such group cooperation can also be obtained in many psychological–economic systems where coevolutionary dynamics is engineered by the asymmetrical environmental feedback. One typical example is the crowdsourcing that aims at completing a project by soliciting contributions from a number of individuals or online communities, in which the focal organizer leads the game and often encourages cooperation via providing a higher payoff structure for cooperators, e.g., promising a higher multiplication factor. While previous works exclusively focus on two-player games, in [37], we propose an extended model where individual strategies coevolve with the multiplication factors in public goods game (PGG) to study the emergence of group cooperation in coevolutionary dynamics (fig. 1). The replicator equation for the fraction of cooperators x is

$$\begin{eqnarray} \dot{x} &= x(1-x)(P_c-P_d)\\ &= x(1-x)\left(-c+\displaystyle\frac{(sx+1)c}{s+1}r_c-\displaystyle\frac{sxc}{s+1}r_d\right), \end{eqnarray} \tag{ 10 }$$

where the focal individual randomly chooses s other participants in a well-mixed infinitely population to join the PGG and P_c , P_d are the expected payoffs of cooperators and defectors. In classic PGG, the cooperators first pay c cost to the common pool, then the total contribution will be amplified by a multiplication factor r and finally equally distributes to every player. For simplicity and without loss of generality, in [37] we set the initial contribution of each cooperator be 1, i.e., c = 1. In addition, to mimic the asymmetrical feedback-evolving characters, we assume that the multiplication factor of defectors r_d is invariant while the multiplication factor of cooperators r_c co-evolves in response to the global payoffs distributions, which is described as

$$\begin{equation} \dot{r_c} = \epsilon (r_c-\alpha)(\beta-r_c) f(x,r_c), \end{equation} \tag{ 11 }$$

here we confine r_c in $[\alpha, \beta]$ and $1<\alpha<\beta<s+1$ according to the social dilemma in PGG. is the relative feedback speed of r_c  vs. x. Moreover, to describe the fact that in a crowdsourcing project the authoritative organizer may decide the global incentives distributions for cooperators and defectors in oder to facilitate the collaboration, we define the feedback mechanism f(x, r_c ) as a linear function of global payoffs of cooperators (xP_c ) and defectors $(({1-x})P_d$) and take into consideration the limitation of total rewards for the project:

$$\begin{equation} f(x,r_c) = -xP_c+\theta(1-x)P_d, \end{equation} \tag{ 12 }$$

where $\theta>0$ is the distribution ratio of cooperator's and defector's total payoff expectations. Under this circumstance, cooperators are favored through increasing r_c when x is small, whereas the continuous consumption of resources results in the decrease of cooperator's rewards, which is also in line with the law of diminishing marginal utility in economics. The final ordinary differential equations (ODEs) for our model can thus be written as

$$\begin{equation} \!\left\{\begin{array}{rcl} \dot{x} &=& x(1-x)\left(\displaystyle\frac{sx+1}{s+1}r_c-\displaystyle\frac{sx}{s+1}r_d-1\right),\\ \dot{r_c} &= & \epsilon(r_c-\alpha)(\beta-r_c)\left[-x\left(-1+\displaystyle\frac{r_c(1+sx)}{s+1}\right)\right.\\ &&\left.+\theta(1-x)\displaystyle\frac{r_dsx}{s+1}\right] \end{array}\right. \end{equation} \tag{ 13 }$$

We highlight our main finding that the coevolutionary dynamics with asymmetrical environmental feedback can give rise to oscillating convergence to persistent group cooperation, but only if the feedback updates quickly and promptly enough compared to the strategy change. Mathematically, the unique interior fixed point

$$\begin{equation} \left\{\begin{array}{rcl} x^*&=&\displaystyle\frac{\theta}{1+\theta},\\ r_c^*&=&\displaystyle\frac{\theta r_ds+(s+1)(\theta+1)}{\theta s+\theta+1} \end{array}\right. \end{equation} \tag{ 14 }$$

is stable only when the relative feedback speed of the cooperator's multiplication factor exceeds a threshold, i.e., $\epsilon>\epsilon^*$, where $\epsilon^*$ depends on s, r_d and θ:

$$\begin{equation} \epsilon^*=\frac{(1-x^*)s(r_c^*-r_d)}{(sx^*+1)(r_c^*-\alpha)(\beta-r_c^*)}. \end{equation} \tag{ 15 }$$

Our work sheds lights on how to successfully organize and sustain a desired group collaboration, which provides useful insights into avoiding the traps of social dilemma in many real scenarios such as knowledge discovery and management, crisis mapping, crowdfunding, scientific cooperation, etc.

Following a similar idea, a further exploration of coevolutionary dynamics with asymmetrical feedback mechanism in spatial threshold PGG is performed in [45], where simulations are conducted on both square lattice and scale-free networks. The evolutionary dynamics of a nonlinear PGG, which incorporates a discounted or synergistically enhanced value of accumulated cooperative benefits [46], with different types of ecological variations is also studied in [47]. Note that the environmental feedbacks in [47] are all time-dependent, not strategy- or payoff-dependent.

An open problem: controlling eco-evolutionary games with external feedback laws. While great efforts have been made to reveal the rich dynamical outcomes of eco-evolutionary games as well as the detailed conditions for breaking the "tragedy of the commons", a fundamental theoretical gap in how to effectively steer such coevolutionary dynamics to a desired population state with external control laws remains unsolved. In [38], we develop a novel and bold manifold control method based on a generalized framework of coevolutionary multi-player games with asymmetrical feedback driven by a nonlinear selection gradient [48,49]. The general form of our feedback control laws f(x, r_c ) is given by

$$\begin{eqnarray} f(x, r_c) &= \Phi_0(x, r_c)(\Phi_1(x, r_c)-a_1)\qquad \nonumber\\ & \quad*(\Phi_2(x, r_c)-a_2)\ldots(a_n-\Phi_n(x, r_c)), \end{eqnarray} \tag{ 16 }$$

in which

$$\begin{equation} \left\{\begin{array}{rcl} \Phi_0(x, r_c) &=& P_c-P_d,\\ \Phi_i(x, r_c) &=& \theta_i P_c-P_d \end{array}\right. \end{equation} \tag{ 17 }$$

and $a_i\ge 0$, $\theta_i>0$, n + 1 denotes the order of the control law. Here $\Phi_0(x, r_c)= P_c-P_d$ is the simplest form of linear selection gradient. Hence eq. (16) represents a sequence of control functions driven by a nonlinear selection gradient in a general polynomial form. When n = 1, we reach the simplest situation where f is a quadratic function and the eco-evolutionary model reads

$$\begin{equation} \left\{\begin{array}{rcl} \dot{x} &=& x(1-x)(P_c-P_d),\\ \dot{r_c} &=& \epsilon (r_c-\alpha)(\beta-r_c)(P_c-P_d)(a_1-(\theta_1 P_c-P_d)). \end{array}\right. \end{equation} \tag{ 18 }$$

Surprisingly, we find the emergence of multiple segments of stable and unstable equilibrium manifolds in phase graphs with different feedback control functions, which naturally extends the concept of population equilibrium points in previous models to a manifold (i.e., curve) of stable equilibria. In particular, our result of unstable equilibrium manifold circumstance is consistent with the separatrix phenomenon obtained by the experimental study in [44]. In addition, we find that a larger relative feedback speed $(\epsilon)$ can not only accelerate the convergence process, but also increase the attraction basin of the stable manifolds.

Based on this framework, we further prove the existence of external switching control laws, either time-dependent or state-dependent, for steering the eco-evolutionary dynamics to any desired region when given an initial population state. For a clear understanding, two detailed control examples are provided in fig. 2. In the first case (fig. 2(a), (b) and (e)), the controlled evolution path is the blue trajectory followed by the red one. Beginning from $(x_0, r_0)=(0.2,1.6)$, the trajectory first converges to the stable equilibrium manifold with control law $\epsilon_1f_1$. We then change the external control law to $\epsilon_2 f_2$ so that the equilibrium manifold becomes unstable. Finally, a small disturbance is applied and the trajectory will automatically evolve to the final desired state $(x_0, r_0)=(1,2.5)$. A similar steering process is performed on the second case (fig. 2(a), (c), (d) and (e)), where the evolution path consists of three segments: the blue, green and pink trajectories. See more details in [38].

Zoom In Zoom Out Reset image size

According to the experiments in [42], r_c in our model can be modulated by changing the histidine concentration in the growth medium, which can manipulate the relative growth rate of the cooperators compared to the defectors. Therefore, our framework can be applied to many microbial experiments, which is of great significance in systems biology and microbial ecology.