Evolution of Cooperation
Robert Axelrod
The Prisoners Dilemma
Circle around the prisoners dilemma used in the book
- The players can communicate with each other only through the sequence of their own behaviour. p.12.
Why you might value the future less?
- Payoffs tend to be valued less the further in the future they are (you might not be around say).
- There is always a chance you don’t meet again.
The payoff of the next move is always valued less than the current for these reasons.
-
The importance (weight w) of the next move compared to the current move is the discount parameter.
-
If your strategy is to always defect (like Thrasymachus) against a TFT player .
-
A strategy (or decision rule) is a specification of what to do in any situation that might arise.
-
A key point here is that there is no best strategy independent of the other strategies that exist.
-
This is the case if the future is in any way important.
Proposition 1
If the discount parameter, w, is sufficiently high, there is no best strategy independent of the strategy used by the other player.
What patterns of behaviour now emerge when w is sufficiently high enough?
General Framework for the dilemma
- Payoffs for each player can be for different things, cooperation for me could result in a payoff that’s different from the other player.
- The magnitude of the payoff does not have to be the same, it’s just about the payoff relative to other options for the player.
- Strategies don’t have to be rational, for instance, they could also be ‘rules of thumb’.
- The actions taken may not even be conscious choices, they can still be considered a strategy (although not a purposeful
one) and analysed in that way.
- This is why organisms acting ‘unconsciously’ can be analysed with game-theoretic thinking.
it is not necessary to assume very much about how deliberate and insightful people are. Nor is it neces- sary to assume, as the sociobiologists do, that important aspects of human behavior are guided by one’s genes. The approach here is strategic rather than genetic.
From the tournament, what did well was nice strategies. Mainly, because they did so well with eachother and did not cause cascades of defection although if another strategy wanted to do this, they’d go down that slope with them.
p.38 that you might be setting yourself up for failure, based on your own strategy if you don’t take into account the reaction you’ll have when they react to your strategy.
The DOWNING strategy is interesting and would have won if not for pessimistic initial assumptions.
Two strategies TRANQUILIZER and TESTER did well on strategies that weren’t aggresive in retaliation.
Even in round two no ‘mean’ strategies, or those that tried to take advantage of niceness was able to do so well that it ultimately did better. They were not able to benefit more than they were hurt.
Is TFT robust
- Given time for everyone to learn about it and that, will it keep succeeding in tournament after tournament?
- Each tournament is considered a generation and the representation in the next generation is weighted by score.
At first, a rule that is successful with all sorts of rules will proliferate, but later as the unsuccessful rules disappear, success requires good performance with other successful rules.
Not being nice, as a strategy is necessarily self-defeating because it doesn’t work well with itself. So in the ecological simulation (where there are no variations in the strategies from generation to generation) an increase in population of a not nice strategy doesn’t really help it.
The clarity of TFT.
Its forgiveness helps restore mutual cooperation. And its clarity makes it intelligible to the other player
The chronology of cooperation
What makes a strategy collectively stable.
Collective stability is when invasive strategies can appear, so not the ecological simulation above.
Axelrod starts by assuming that TFT has a memory of one move.
- The strategy invading must defect at some point.
- Because of this, After the first defection the invading strategy can cooperate again or defect.
- The first defection will always be greater than R and then you’ll get the worse than R payoff.
Proposition 2: TIT FOR TAT is collectively stable if w is large enough. This w value is a function of T, R, P and S.
- A player is less likely to defect if w is high enough. Does the reward value, the reward for cooperation, dictace morality in a sense.
- Caesar’s explanation of why Pompey’s allies stopped cooperating with him.
-
”They regarded his [Pompey’s] prospects as hopeless and acted according to the common rule by which man’s friends become his enemies in adversity”
- Similarly when a supplier company is going bankrupt, it seems that buyers might not pay etc. knowing the supplier is doomed.
- ”The interaction must last long enough for the gain from temptations to be nullified over future moves.”
Proposition 3: For any strategy which is the first to cooperate, to be collectively stable, w must be sufficiently large.
Proposition 4: For a nice strategy to be collectively stable, it must be provoked on the first defection of the other player.
Proposition 5: Always defecting is always collectively stable.
This starts this notion of why cooperation might work in a world full of ‘meanies’.
Invasion works if the newcomers do better than the average with themselves and also interact a sufficient amount with themselves. This is why clustering can promote invasion.
The strategies which can invade all D are those that are maximally discriminating, like TFT. It has a very good heuristic for when it’s interacting with itself vs all D.
This allows it to invade a world full of ‘meanies’ with the smallest proportion in the population p.
Thus cooperation can emerge even in a world of uncon- ditional defection. The development cannot take place if it is tried only by scattered individuals who have no chance to interact with each other. But cooperation can emerge from small clusters of discriminating individuals,
Live and let live in trench
- Ethics kind of evolved here, that the payoffs were altered by the sustained mutual cooperation. The two sides generally started to care about eachother based on the cooperation that was established.
- Only look at your own payoff. Do not look at it relative to others. Think of how you can do in the situation rather than how well you’re doing to others in an absolute sense.
- I see this in blog posts those I admire right. I consider it a failure if I can’t compete with something of that character but in reality I should be looking at the best I can do. That hoping for a payoff of the other is just not attainable. This probably doesn’t really have anything to do with game theory.
30/03/23 17:30:58
Had a thought on my walk. The notion of a strategy as something that evolves or adapts to other strategies in a pool of strategies. It can be applied to a lot of you view strategy as this atomic thing. For instance, business strategies evolve and adapt to one another. This is like what Dawkins talks about with memes. But I’ve forgotten a lot of that. The environment for the strategy then becomes the other strategies. This is how you could look at something like farming and see how it might be locally optimal but not globally (will have to flesh this out).
I’ve been thinking about strategies in light of Axelrod’s book. in the book he talks about a kind of ‘survival of the fittest’ of strategies in a pool of strategies. The strategy (I think) similar to Dawkin’s notion of a meme (correct me if I’m wrong). With this, could we look at some idea like ‘farming’ and the agricultural revolution as an equilibrium to something like the hunter gatherer lifestyle. That, possibly, it’s not objectively optimal (for whatever that might mean) but just kind of, ‘worked’. Yes, you’re correct that Axelrod’s book discusses the evolution of strategies and the idea of a “survival of the fittest” in a pool of strategies. And you’re also correct that the concept of memes, as introduced by Richard Dawkins, is similar in some ways to the concept of strategies in Axelrod’s book.
Regarding your question about farming and the agricultural revolution, it’s certainly possible to view the adoption of agriculture as an equilibrium that emerged through a process of cultural evolution. The shift from a hunter-gatherer lifestyle to an agricultural lifestyle represented a significant change in the way that humans lived and interacted with their environment. It’s likely that this change was driven by a combination of factors, including population growth, environmental factors, and cultural innovations.
From a game-theoretic perspective, we could view the shift to agriculture as a strategic decision made by early human societies. By adopting agriculture, these societies were able to increase their food production and create a surplus that could be used to support larger populations and more complex societies. However, there were also costs associated with this shift, such as the need for intensive labor to cultivate crops and the risks associated with relying on a single food source.
Overall, it’s difficult to say whether the shift to agriculture was objectively optimal, since this would depend on the specific criteria used to evaluate the different lifestyle options. However, it’s clear that agriculture was a successful strategy in terms of its ability to support larger, more complex societies over the long term.
Conor Redington: What if the criteria was something like community and generally humans being more communal? theres evidence to suggest (i think) that farming made us more agressive. That’s an interesting point. If the criteria for evaluating different lifestyle options was based on factors like community and social cohesion, then it’s possible that the shift to agriculture might not be viewed as an optimal strategy. There is evidence to suggest that the shift to agriculture was associated with changes in social organization and increased social inequality, which could have negative effects on community and social cohesion.
For example, some researchers have argued that the adoption of agriculture led to the emergence of social hierarchies and inequalities, as some individuals gained more power and resources through control of land and food production. This could have contributed to increased conflict and aggression within and between communities.
At the same time, it’s important to note that the shift to agriculture also had positive effects on social organization and community. For example, agriculture allowed for the development of more settled, permanent communities, which in turn could have led to the emergence of more complex social structures and the development of shared cultural practices.
Overall, the question of whether the shift to agriculture was objectively optimal depends on the specific criteria used to evaluate the different lifestyle options. It’s likely that the shift to agriculture had both positive and negative effects on community and social cohesion, and the balance between these effects would depend on a variety of factors, including cultural context and environmental conditions.