346_lect14.pdf

Lecture Notes 14: Evolutionary Games

Rational versus Evolutionary Strategy Choices

The standard setup in game theory is that a player rationally chooses among his strategies. The setup in an evolutionary game is different. Instead of making a conscious choice, we assume that players are pre-programmed to play certain strategies. The overall configuration in the population may change over time if some strategies prove to be more successful than others. But each member of the population does not actually choose a strategy.

We can motivate this by thinking about biological evolution. The evolution of cats is a good example. Wild desert cats who were genetically pre-programmed to be tame around humans had evolutionary success because they could live in human communities and eat table scraps. The cats didn’t consciously make a choice to behave this way, but the ones who were pre-programmed genetically to do so were more successful and reproduced. This same reasoning is exactly the basis of evolutionary games. Even when people don’t consciously choose their strategies, some strategies will be more successful than others and will tend to survive and reproduce over time.

Basic Terminology

• There is a population consisting of a large number of individuals. Each individual is endowed with a phenotype that governs his behavior.

These phenotypes need not refer to literally to genetic code and biological evolution. We can think of evolution of social norms where people in a society are brought up to behave a certain way, for example.

• The success of a phenotype is called its fitness. Through the process of selection,

phenotypes that are more fit become more numerous in the next generation.

Again, the obvious motivation is biological evolution, but we need not restrict ourselves to this case. If there are a variety of social norms (e.g. related to smoking) and one is more successful than another, it is easy to imagine that the more successful social norm is likely to be more prevalent in future generations.

• From time to time, there are mutations that disturb the balance of phenotypes.

smokes or where women don’t seek employment outside of the home, there will occasionally be people who come along and behave differently.

• These mutations may or may not be successful. If the mutation leads to a phenotype that is more fit, then we say that this new phenotype invades the existing population.

Cats with genetic mutations causing them to be tame around humans ended up doing better than wild desert cats and eventually became much more numerous. With respect to social norms, if people who don’t smoke do better and households where women seek gainful employment do better, then these norms will spread to become a significant part of the population.

In biology, the example is simple genetics – more successful phenotypes are more likely to pass on their chromosomes to the next generation. For socioeconomic evolution, the usual story is that people with more successful phenotypes transmit this information to other people in their social groups, who then copy this successful behavior. Mutation and invasion is just experimentation with new norms and behaviors, which happens all the time.

• A configuration of a population is evolutionary stable if it cannot be successfully invaded by any mutant.

Game Theory Analogues

The phenotypes in an evolutionary game are the analogy of strategies in the usual game theory setup. The difference is just that players are pre-programmed to use these strategies rather than consciously making a choice.

The interactions in the game determine the fitness associated with any particular phenotype, which is basically the payoff in a normal game theory setup. Note that average fitness/payoffs may depend on the environment. A tame cat may do differently in an environment with many other tame cats than in an environment with mostly hostile cats. Being a working wife may have a different success level in a society with many working women versus very few working women.

When different pairs of phenotypes meet up, it leads to different combinations of strategies and payoffs. These are analogous to the cells in a usual game matrix. What happens when a tame cat meets another tame cat? What happens when a tame cat meets a hostile cat?

Rules of Play

• Pairs of phenotypes are randomly selected to interact with each other.

• The payoff / fitness is determined by the phenotypes of both players, and is read from the usual game matrix.

Note that the payoff matrices in evolutionary games are always symmetric. There is nothing special about which player we call “player 1” or “player 2” in the interaction.

• The fitness of a phenotype is determined by the average payoff when playing against other members of the same population. It may depend on the current configuration.

As we said earlier, being a tame cat could lead to different fitness depending upon the configuration among other members of the population – whether tame cats mostly meet other tame cats or whether they meet hostile cats.

• Phenotypes with higher fitness will reproduce and this phenotype will proliferate. Phenotypes with lower fitness become less numerous as time goes on.

• Occasionally, a mutant may emerge with some phenotype not currently in use.

As we said earlier, a configuration is evolutionary stable if it cannot successfully be invaded by any mutant. There are two types of stable configurations.

• The outcome might be a monomorphism, where a single phenotype is fitter than the others and this phenotype takes over the whole population. This phenotype then constitutes an

evolutionary stable strategy (ESS).

• The outcome might be a polymorphism, where different phenotypes consistently proliferate and they permanently co-exist in certain proportions.

A polymorphism is, in a sense, similar to a mixed strategy. But there is an important difference. In a mixed strategy, each player randomizes among different strategies. In an evolutionary polymorph, each individual doesn’t mix strategies/phenotypes. Rather, there is a mix within the population of different phenotypes.

Assurance Game

Consider the following evolutionary interaction with individuals endowed with either an 𝐴𝐴 or a 𝐵𝐵 phenotype. What are the evolutionary stable configurations in this game?

A B

A 1,1 0,0

B 0,0 2,2

Let us first calculate the payoff from each strategy. It is easiest to write it as a function of a single variable, so we will let 𝑎𝑎 designate the fraction of the population endowed with phenotype 𝐴𝐴. Note that it doesn’t matter whether we read these payoffs across the rows or down the columns since the game is symmetric.

Π𝐴𝐴 = 1𝑎𝑎 + 0(1 − 𝑎𝑎) = 1𝑎𝑎 Π𝐵𝐵 = 0𝑎𝑎 + 2(1 − 𝑎𝑎) = 2 − 2𝑎𝑎

The diagram below graphs both of these payoffs as a function of 𝑎𝑎.

The crossing point occurs at:

Π𝐴𝐴 = Π𝐵𝐵 1𝑎𝑎 = 2 − 2𝑎𝑎

It is easy to see the following from the diagram (or from solving the inequalities directly).

• For 𝑎𝑎 <2

3, 𝐵𝐵 is the more fit phenotype: Π𝐵𝐵 > Π𝐴𝐴

• For 𝑎𝑎 >2

3, 𝐴𝐴 is the more fit phenotype: Π𝐴𝐴 > Π𝐵𝐵

To use the diagram to think about evolutionary stable strategies, we need to ask whether a small invasion is going to be successful.

• The monomorph where all players have a 𝐵𝐵 phenotype (i.e. 𝑎𝑎 = 0) is evolutionary stable. Even if there is a small invasion of the 𝐴𝐴 phenotype in the population (moving slightly to the right of the edge), it will still be the case that Π_𝐵𝐵 > Π_𝐴𝐴. The invasion will not be successful.

• The monomorph where all players have an 𝐴𝐴 phenotype (i.e. 𝑎𝑎 = 1) is also evolutionary stable. Even if there is a small invasion of the 𝐵𝐵 phenotype in the population (moving slightly to the left of the edge), it will still be the case that Π_𝐴𝐴 > Π_𝐵𝐵. The invasion will not be successful.

• The polymorph is not stable. If 𝑎𝑎 =2

3 then both phenotypes have equal payoffs. But if there

are mutations and 𝑎𝑎 is disturbed slightly above 2

3, then Π𝐴𝐴 > Π𝐵𝐵 and the 𝐴𝐴 phenotype will

grow and take over. If there 𝑎𝑎 is disturbed slightly below 2

3 by a mutation, then Π𝐵𝐵> Π𝐴𝐴,

and the 𝐴𝐴 phenotype will die out until we reach the monomorph where the whole population has a 𝐵𝐵 phenotype.

So a monomorph where everyone has an 𝐴𝐴 phenotype and a monomorph where everyone has a 𝐵𝐵 phenotype are both evolutionary stable.

This is intuitive to understand. If everyone else is 𝐵𝐵 then the most successful strategy is for you to be 𝐵𝐵 also. A mutant 𝐴𝐴 in this world isn’t going to do very well when he meets up with 𝐵𝐵 members of the population. But even if everyone has an 𝐴𝐴 phenotype, this is also evolutionary stable. A few mutants coming in with a 𝐵𝐵 phenotype won’t do very well against a population where everyone else has an 𝐴𝐴 phenotype because they mostly match with other 𝐴𝐴 types.

Chicken

Consider a population of individuals who are matched to interact and play chicken with each other Each member of the population is endowed either with a 𝑊𝑊 phenotype (waver), or with an 𝑆𝑆 phenotype (go straight). The interaction is described by the following payoff matrix.

W S

W 0,0 -1,1

S 1,-1 -2,-2

Letting 𝑤𝑤 be the proportion of the population that wavers, the payoffs for a player endowed with each phenotype are as follows:

Π𝑊𝑊 = 0𝑤𝑤 + −1(1 − 𝑤𝑤) = −1 + 𝑤𝑤 Π𝑆𝑆 = 1𝑤𝑤 + −2(1 − 𝑤𝑤) = −2 + 3𝑤𝑤

The diagram below shows the fitness of each phenotype as a function of 𝑤𝑤. It is straightforward

to solve Π𝑊𝑊 = Π𝑆𝑆 to see that the crossing point occurs at 𝑤𝑤 =1

2

• The monomorph where all players are 𝑆𝑆 types (i.e. 𝑤𝑤 = 0) is not evolutionary stable. In this case Π𝑊𝑊 > Π𝑆𝑆 and so a mutant population of 𝑊𝑊 phenotypes will enjoy higher fitness and will continue to reproduce. Since 𝑆𝑆 can be successfully invaded, it is not evolutionary stable.

• The polymorph at 𝑤𝑤 =1

2 is stable. If 𝑤𝑤 increases slightly above this level, then Π𝑆𝑆 > Π𝑊𝑊

and so the population of 𝑊𝑊 types will fall again. But if 𝑤𝑤 falls slightly below this level,

then Π_𝑊𝑊 > Π_𝑆𝑆 and the population of 𝑊𝑊 types will rise back towards 𝑤𝑤 =1

2. In other words,

even with small mutations, the configuration tends to return back towards 𝑤𝑤 =1

2.

The only stable configuration in this game is the polymorph where half of the population is endowed with the waver phenotype and half of the population is endowed with the phenotype to go straight.

In a sense, this evolutionary interaction creates the opposite incentives that the assurance game creates. In the assurance game, it’s better to have the phenotype that other people have. If everyone else is an 𝐴𝐴 type, you want to be an 𝐴𝐴 type also. Thus, the two monomorphs were evolutionary stable. But chicken is different. If most people are wavering, then you’re more successful if you go straight. But if most other people go straight, then you’re better off if you waver. Thus, the two monomorphs are not evolutionary stable and, in fact, the stable configuration is a polymorph where

𝑊𝑊 and 𝑆𝑆 types permanently coexist.

Hawk-Dove Game

The hawk-dove game is a very well-known evolutionary interaction. We imagine a population where individuals are endowed either with an aggressive hawk phenotype (𝐻𝐻) or a peaceful dove phenotype (𝐷𝐷).

There is a resource with value 𝑉𝑉. If two doves meet, they split it in half. If a hawk meets a dove, the hawk gets the whole resource. If two hawks meet, they fight it out and incur a cost 𝐶𝐶 associated with fighting. There’s an even chance for each of them to win the fight, so the average payoff

associated with two hawks fighting is 𝑉𝑉−𝐶𝐶

2 for each hawk. We summarize this interaction on the

table below

H D

H 𝑉𝑉−𝐶𝐶

2 , 𝑉𝑉−𝐶𝐶

2 𝑉𝑉, 0

D 0, 𝑉𝑉 𝑉𝑉

2, 𝑉𝑉 2

Fighting is Less Costly (𝐶𝐶 < 𝑉𝑉)

Suppose that 𝑉𝑉 = 6 and 𝐶𝐶 = 2. Then the payoffs for the interactions are as follows.

H D

H 2,2 6,0

D 0,6 3,3

Letting ℎ designate the proportion of hawks in the population, the payoffs for each phenotype are as follows.

Π𝐻𝐻 = 2ℎ + 6(1 − ℎ) = 6 − 4ℎ Π𝐷𝐷 = 0ℎ + 3(1 − ℎ) = 3 − 3ℎ

The diagram shows the fitness of each phenotype as a function of the proportion of hawks ℎ.

This game is easy to analyze. No matter the value of ℎ, 𝐻𝐻 is a more fit phenotype than 𝐷𝐷. Thus, the only evolutionary stable strategy is a monomorph where all players are hawks. No matter how many other people are doves or hawks, you are always better off being a hawk. This is similar in principle to a dominant strategy.

Fighting is More Costly (𝐶𝐶 > 𝑉𝑉)

Suppose now that 𝑉𝑉 = 4 and 𝐶𝐶 = 6. The payoffs are as follows.

H D

H -1,-1 4,0

The payoffs for each phenotype are as follows:

Π𝐻𝐻= −1ℎ + 4(1 − ℎ) = 4 − 5ℎ Π𝐷𝐷 = 0ℎ + 2(1 − ℎ) = 2 − 2ℎ

The diagram below shows the fitness of each phenotype as a function of the proportion of hawks

ℎ. You can solve Π𝐻𝐻 = Π𝐷𝐷 to find the crossing point at ℎ =2

3. As usual, the arrows indicate that

the proportion of hawks will rise when Π𝐻𝐻 > Π𝐷𝐷, meaning that hawks are more fit. The proportion of hawks will fall when Π_𝐷𝐷 > Π_𝐻𝐻.

• A monomorph with no hawks is not stable. If ℎ = 0 then Π_𝐻𝐻 > Π_𝐷𝐷 and so mutant hawks will be successful and grow in number.

• A monomorph with all hawks is not stable. If ℎ = 1 then Π_𝐻𝐻 < Π_𝐷𝐷 and so mutant doves will be successful and grow in number, with ℎ falling.

• The polymorph with ℎ =2

3 is a stable configuration. You can see from the diagram that

when ℎ rises or falls slightly away from ℎ =2

3, the dynamics tend to bring the population

configuration back to ℎ =2

3. Since a disturbance away from this polymorphic configuration

will initiate dynamics that cause it to move back again, it is evolutionary stable.

Prisoners’ Dilemma with One Repetition

Consider a population where players are matched together in a prisoners’ dilemma, and are endowed with either a 𝐶𝐶 phenotype to cooperate or a 𝐷𝐷 phenotype to defect.

C D

C 2,2 -1,4

D 4,-1 0,0

As a function of the proportion of cooperators 𝑐𝑐, the payoffs for each phenotype are:

Π𝐶𝐶 = 2𝑐𝑐 + −1(1 − 𝑐𝑐) = −1 + 𝑐𝑐 Π𝐷𝐷 = 4𝑐𝑐 + 0(1 − 𝑐𝑐) = 4𝑐𝑐

It is easy to verify that, for any 𝑐𝑐 that satisfies 0 ≤ 𝑐𝑐 ≤ 1, the payoff for a defector Π𝐷𝐷 exceeds the payoff for a cooperator Π_𝐶𝐶. Thus, defectors are more fit than cooperators for any value of 𝑐𝑐 and so defecting is an ESS of this game. A monomorphic population of defectors is the only configuration that is evolutionary stable.

Repeated Prisoners’ Dilemma with some Tit-for-Tat Types

Suppose now that the population consists of two types.

• The 𝐴𝐴 phenotype always defects

• The 𝑇𝑇 phenotype starts by cooperating and then plays tit-for-tat – copying his opponent’s action from the previous period. He even will “irrationally” cooperate in the last period.

These two types are randomly matched up and repeat the prisoners’ dilemma from above n times.

Tit-for-tat types who meet will continue to cooperate with each other, earning a payoff of 2 every period for 𝑛𝑛 periods, leading to a total payoff of 2𝑛𝑛. Defector 𝐴𝐴 types who meet with each other will obviously get a payoff of 0.

What about when a 𝑇𝑇 type meets an 𝐴𝐴 type? In the first period, the 𝐴𝐴 type defects against the tit-for-tat player, leading to a payoff of (4, −1). Thereafter, both defect and earn a payoff of 0. Together, the payoffs for the game are.

T A

T 2n, 2n -1,4

Letting 𝑡𝑡 measure the proportion of tit-for-tat types in the population, each phenotype’s fitness is:

Π𝑇𝑇 = (2𝑛𝑛)𝑡𝑡 + −1(1 − 𝑡𝑡) = (2𝑛𝑛 + 1)𝑡𝑡 − 1 Π𝐴𝐴 = 4𝑡𝑡 + 0(1 − 𝑡𝑡) = 4𝑡𝑡

In order to diagram the payoff for each phenotype as a function of 𝑡𝑡, we first calculate the intersection point.

Π𝑇𝑇 = Π𝐴𝐴 (2𝑛𝑛 + 1)𝑡𝑡 − 1 = 4𝑡𝑡

𝑡𝑡 =_{2𝑛𝑛 − 3}1

The diagram is below.

There are two evolutionary stable configurations – Everyone is a defector or everyone is a tit-for-tat player. But the interesting thing to note is that as 𝑛𝑛 rises, the crossing point moves farther and farther to the left. In other words, more and more initial values of 𝑡𝑡 will converge to the ESS where all society consists of tit-for-tat types (𝑡𝑡 = 1) rather than to the ESS where society consists of only

𝐴𝐴 types who defect (𝑡𝑡 = 0).

For example, if 𝑛𝑛 = 100, then as long as 𝑡𝑡 > 0.0051, society will converge to the ESS with only tit-for-tat players in a monomorphic configuration. In other words, even when society has only a few tit-for-tat types to start with, it is so much more valuable to be a tit-for-tat type when the game is repeated 100 times that they will eventually take over the population.

Problems

1. Find the evolutionary stable strategy or strategies for the following interaction. Be sure to consider both monomorphic and polymorphic configurations.

A B

A 2,2 1,2

B 2,1 2,2

2. Consider the evolutionary interaction given below among velociraptors, which can be aggressive (𝐴𝐴) or tame (𝑇𝑇). Let 𝑎𝑎 denote the population proportion of aggressive velociraptors.

a. If 𝑎𝑎 = 0.5, what happens to the population of aggressive velociraptors? b. If 𝑎𝑎 = 0.75, what happens to the population of aggressive velociraptors?

c. What is the equilibrium polymorphic configuration of aggressive and tame velociraptors? Is it stable?

A T

A 0,0 7,2

3. There are two types of racers – tortoises (𝑇𝑇) and hares (𝐻𝐻) who race against one another in randomly drawn pairs. Hares always beat tortoises. If hares race together, they tie but are completely exhausted by the end of the race. When two tortoises race, they always tie, but they enjoy a pleasant conversation along the way that generates a payoff 𝛽𝛽 > 0. The table below summarizes the payoffs for this interaction. Let 𝑡𝑡 designate the proportion of tortoises in the population.

a. If 𝑡𝑡 = 0.5, for what values of 𝛽𝛽 are tortoises more fit than hares? b. If 𝑡𝑡 = 0.1, for what values of 𝛽𝛽 are tortoises more fit than hares?

c. If 𝛽𝛽 = 1, can a population of tortoises be successfully invaded by hares?

d. In terms of 𝑡𝑡, how large must 𝛽𝛽 be in order for tortoises to have greater fitness than hares?

e. In terms of 𝛽𝛽, what is the level of 𝑡𝑡 in a polymorphic equilibrium? Is it stable?

T H

T 𝛽𝛽, 𝛽𝛽 -1,1