Gains in evolutionary dynamics - PowerPoint Presentation

Slide1

Gains in evolutionary dynamicsA unifying and intuitive approach to linking static and dynamic stability

Dai Zusai

Philadelphia, U.S.A.

Lyapunov function

i

)

attains the minimum value only at an equilibrium;

ii)

never increases; and

iii)

must decrease until L reaches its minimum value.

Consider dynamic

on a multidimensional space.

Suppose that we’ve found function

that maps (multi-dim) “position” to a scalar (one-dim) value s.t.

Then, we can declare that the equilibrium is stable.

Zusai, Introduction to Formal Mathematics for Economic Modeling,Under a contract with Temple University Press for publication as an open access textbook (expected around the end of 2020).

History

Modified in the paper

The theorem does not tell

how

to find a Lyapunov function



What’s our Lyapunov!?

In the literature on evolutionary dynamics in population games, dynamic stability of equilibrium is proven for each of major dynamics:

Smith ‘84,

Cressman

‘97,

Hofbauer

’95, ‘01,

H

ofbauer

&

S

andholm

‘09, Sandholm ‘10, M

etrikopolis & Sandholm ’18HS ‘09: “[U]nlike potential games, stable games do not come equipped with an all-purpose Lyapunov function. To prove convergence results, we must construct a suitable Lyapunov function for each dynamic we wish to consider.”

Any general principles?

Mixing heterogeneous dynamics?Robust to misspecification?

What’s our Lyapunov!?

FS ;13: δ-passivity.

HS ‘09: integrabilityMS ‘18: Riemannian geometry (only for local stability of an ESS, requiring negative definiteness–-not only

semidefiniteness)

Testable from experiments/data?

Applying to other situations?

Economic intuition?

What’s our gravity?

Greed works.

Greed clarifies, cuts through, and captures the essence of the evolutionary spirit.

Gordon Gekko

(Wall Street, 1987)

: Rock-Paper-Scissors

with payoff =1 for a win

=0 for a draw

=−0.9 for a lose

Strict stable game

: pairwise payoff comparison dynamic

(Smith dynamic

s.t.

)

While

converges to the equilibrium, the aggregate gross gain does not monotonically decrease.

 What’s Greed? Does it really work?Greed: Exploit opportunities for greater profits/payoffs(Gross) gain = Payoff improvements from switches. cf. At eqm, no room for payoff improvement 

=0  

Decision rules in Evolutionary dynamicsPossibly, not exactly optimizing.Possibly, reluctant to switch.

Switch

Give up linking?

No, rather reconsider

what’s economically reasonable.

What’s economically reasonable?About economic principles, ask a “principle” textbook

Wait, evolutionary game theory is thinking about deviation from rationality!

Best Response Dynamic

Imitative dynamics

Excess Payoff

Dynamic

Pairwise comparison dynamics

Find the

optimal strategy

(simply, greatest payoff)

among all the strategies.

Switch to it, regardless of the amount of the payoff improvement.

Sample another

agent randomly and observe the agent’s strategy.If it performs better than my current strategy, switch to it with a probability proportional to the payoff difference from my current payoff.

Sample another strategy randomly. If it performs better than my current strategy, switch to it with a probability proportional to the payoff difference from my current payoff.

Sample another strategy randomly. Switch to it with a probability proportional to the payoff difference from the average payoff.Agent’s decision making in evolutionary dynamicsPossibly, not exactly optimizing.Possibly, reluctant to switch.

Constraint on available strategies

Hidden stochastic costs to switch

Excluded from our scope---Imitation is truly more than economic/incentive-based reasoning. It is indeed known NOT to guarantee dynamic stability of

eqm in stable games or of regular ESS.

Section 1.3. The First Principle of Economics: Optimization

Acemoglu, Laibson and List,

Microeconomics

(’18,Ed. 2, Pearson)

Optimization means that you weight the information that you have, not that you perfectly foresee the future. … Rational action does not require a crystal ball [to perfectly find the best outcome],

just a

logical appraisal of the costs, benefits and risks that are known to the economic agent.

Evolutionary dynamics: construction

Game

Shares of strategies

Payoffs of strategies

Evolutionary dynamics

At each moment of time,

for each strategy, we count

the agents who switches to it

and those who switches from it.

Aggregation of individual agents’ switches

Individual agent’s

switching rate

An agent occasionally reconsider the choice,

when it receives a “revision opportunity,” which arrives randomly.

(To make the dynamic differentiable with respect to infinitesimal change in time.)

At that opportunity, the agent

finds the candidate

of a new strategy, and

decides whether or not to switch to it

.

Individual agent’s decision of switching the choice

Choices

Incentives

Modified framework:

an agent can take a mixed strategy over available actions.

Imagine

a birth-death process

, where a new agent born with default mixed strategy

(the population’s current action distribution) replaces an old agent at a “revision” opportunity.

Economically reasonable dynamics

Say, an agent has been taking action

so far and the current payoff vector is

.

0. Receive a revision opportunity from a Poisson process.

1. Draw a set of

available new actions

from prob

dist

over a power set of

, and

a

switching cost

from prob

dist

with cumulative

dist

function Q over

.

2. Find the best

available

action, say

, among actions in

and

calculate

payoff improvement

.

3. Switch to action

if

; Keep the current action (

status quo

)

if

.

⇒ Given the best available action b, switch occurs with

prob

.

Best Response Dynamic

: any action is always available, i.e.,

.

: switching cost is always 0, i.e.,

.

Pairwise comparison dynamics

: only one action is available, i.e.,

for each

: any (to have switching rate Q increasing with the payoff improvement)

Excess Payoff

Dynamic

: any action is always available, i.e.,

: any (to have switching rate Q increasing with the payoff improvement)

Smooth Best Response

Dynamic

Further, we introduce a control cost that prevents an agent from taking a pure strategy. (Covered in another in-progress paper.)

Q1

for any

.

A0

does not depend on

.

A1-i)

Any action is available with some positive probability.

A1-ii)

Availability of an action does not vary with the current action,

unless the action has been currently taken (then, it must be certainly available as a status quo).

Apdx

:

assumpt’ns

Net gain as a general Lyapunov function

Our economic reasonable dynamic allows us to define Net gain of switch from a to b

:= [Payoff improvem’t

] – [Switching cost

]

Further, by taking expectation over

and

, we define

Ex-ante net gain for action-

player

Ex-ante second-order gain

Aggregate:

.

Embedded to game F:

.

Therefore,

works as a Lyapunov function to derive dynamic stability of equilibrium under economically reasonable dynamics from static stability.

Switch

Status-quo

Static stability

Economically reasonable

dyn

(esp., Assumption

A1-ii)

maximized among all

available

actions in

.

Static stability

Net gain always decreases,

until it completely vanishes,

i.e., until the dynamic reaches equilibrium!

Game

Share of

strategies

Evolutionary dynamics

Choices

Incentives

Payoff of

strategies

Combined dynamics

Net gain

with

Main theorem

Extended to boundary equilibria (regular ESS),

a society of (finitely many) heterogeneous populations who follow different payoff functions and revision protocols.

Behind those theorems, I also proved several mathematical theorems (a modified version of Lyapunov stability theorem for a set-valued differential equation, etc.)

The paper is posted on

Arxiv

, linked from my web page (easily found from Google).

Wrap up

The approach proposed here:

Construct an

economically reasonable dynamic

from optimization, possibly with additional costs and constraints to explain distortions from exact best responses.

Calculate the

net gain

as the maximal payoff improvement

minus switching cost

.

Static stability should imply monotone decrease in

the aggregate net gain

over time,

and thus dynamic stability of equilibrium

Benefits

Aggregate gain is just a sum of individual gains: easily extended to

heterogeneous setting

.

Relying on

qualitative characterizations

: robust to misspecifications

Approximation of finite-agent dynamics.

(Ellison,

Fudenberg

&

Imhof

’16 JET on Lyapunov)

Intuitive: applicable to complicated settings

(e.g. multitasking:

Sawa

& Z, accepted to JEBO)

Historical note

Economic theory: Stability of

t

â

tonnement process

in general

eqm

model

Market: no single universal axiomatization/formulation

Economic

agent: agreed to formalize from

optimization (even in freshman textbooks).

Price adjustment process in a

market

.

Strategy adjustment process of an

agent

Lyapunov

thm

Modified Lyapunov stability theorem

Lyapunov

thm

Assumptions on

and

Q1

for any

:

As long as there is a positive payoff improvement, switch occurs with some positive prob.

A0

does not depend on . A1-i) Any action is available as a candidate with some positive probability.

A1-ii) Availability of any candidate actions does not vary with the current action, unless any of those candidate actions is not currently taken (then, it must be certainly available as a status quo).Q1 & A1-i) are for stationarity of a Nash equilibriumA0 excludes imitative dynamics.A1-ii) is to make it economically natural that an agent chooses the next action simply by maximizing the payoff improvement If A1-ii) does not hold, then the choice of a new action affects the possible payoff improvement at the

next revision opportunity. So, an economic agent should base the decision not only on the payoff improvement at the present revision opportunity but also the mobility to a further better action in the next revision opportunity. 

AppendixZusai, Gains16 Return: definition