Risk Neutral Equilibria - PowerPoint Presentation

Slide1

Risk Neutral Equilibria of Noncooperative Games

Robert Nau

Duke University

April 12, 2013Slide2
Slide3

References

“Coherent behavior in

noncooperative

games”

(with K.

McCardle

,

JET

1990)

“Coherent decision analysis with inseparable probabilities and utilities” (

JRU

, 1995)

“On the geometry of Nash and correlated

equilibria

”

(with S.

G.

Canovas

&

P.

Hansen,

IJGT

2003)

“Risk neutral

equilibria

of

noncooperative

games”

(

Theory and Decision

, forthcoming)

“Efficient correlated

equilibria

in games of coordination” (working paper)

Arbitrage and Rational Choice

(manuscript in progress)Slide4

Nash vs. Correlated Equilibrium

What is the most fundamental solution concept for

noncooperative

games?

Nash equilibrium (or a refinement thereof)?

Rationalizability

?

No,

correlated

equilibrium

.

A correlated equilibrium is a generalization of Nash equilibrium in which randomized strategies are permitted to be correlated.

More to the point: Nash equilibrium is special case of correlated equilibrium in which randomization, if any, is required to be performed independently.Slide5

A canonical example: Battle-of-the-Sexes

Correlation of randomized strategies is most useful in games where some coordination of strategies is beneficial or equitable

Canonical example: the Battle-of-Sexes game

The obvious solution:

f

lip

a coin

!

Left

Right

Top

2, 1

0, 0

Bottom

0, 0

1, 2Slide6

Nash vs. Correlated Equilibrium

A Nash equilibrium satisfies a

fixed-point condition

analogous to that of a

Walrasian

equilibrium

A correlated equilibrium satisfies a

no-arbitrage condition

that is a strategic generalization of de

Finetti’s

fundamental theorem of probabilitySlide7

Properties of Nash equilibria

The set of Nash

equilibria

may be

nonconvex

and/or disconnected

May require randomization with probabilities that are irrational numbers, even when game payoffs are rational

Existence proof is based on a fixed point theorem

Computation can be hard in games with more than a few players and/or strategiesSlide8

Properties of correlated equilibria

The set of correlated

equilibria

is a convex

polytope

, like sets of probabilities that arise elsewhere in decision theory (incomplete preferences, incomplete markets….)

Extreme points

of the

polytope

have

rational coordinates if the game payoffs do

Existence proof only depends on the existence of a stationary distribution of a Markov chain

Computation is easy in games of any size: solutions can be found by linear programming

The number of extreme points of the

polytope

can be huge, but usually only a small number are efficient.Slide9

Fundamental theorems of rational choice

Subjective probability (de

Finetti

)

Expected utility (

vNM

)

Subjective expected utility (Savage)

Decision analysis (

Nau

)

Asset pricing (Black-Scholes/Merton/Ross….)

(2

nd

theorem of) Welfare economics (Arrow)

Utilitarianism (

Harsanyi

)

All of these theorems are duality theorems, provable by separating-

hyperplane

arguments, in which the primal rationality condition is no-arbitrage and the dual condition is the existence of a probability distribution and/or utility function and/or prices and/or weights with respect to which an action or allocation is optimal.

Correlated equilibrium, rather than Nash, fits on this list.Slide10

The duality theorem for games

Primal definition of common knowledge:

The players should be willing to put their money where their mouth is, i.e., publicly accept bets that reveal the differences in payoff profiles between their strategies

Primal definition of strategic rationality:

The outcome of the game should be

jointly coherent

, i.e., not allow an arbitrage profit to an observer, when the rules are revealed in this way

THEOREM

(Nau &

McCardle

1990): The outcomes of the game that are jointly coherent are the ones that have positive support in some correlated equilibriumSlide11

Consider a generic 2x2 game whose “real” rules are:

The “revealed” rules (all that can be commonly known under

noncooperative

conditions) are encoded in the matrix whose rows are the payoff vectors of 4 bets on the game’s outcome that might be offered to an observer:

Left

Right

Top

a, a



b, b



Bottom

c, c



d, d



TL

TR

BL

BR

1TB

a – cb – d  1BT  c – ad – b2LRa – b c – d 2RL b – a d – c

How this argument works

 =

G =

in units of money, with risk neutral playersSlide12

“1TB” is the payoff vector of a bet that player 1 should be willing to accept if she chooses Top in preference to Bottom

In that case she gets the payoff profile (a, c) rather than

(b, d) as a function of player 2’s choice among (L, R)

She should choose Top if (

a,c

) yields a greater expected payoff than (

b,d

) at the instant of her move

In this event, a bet with payoff profile (a-c, b-d) evidently has non-negative expected value for her.

She can reveal information about her payoff function by offering to accept this bet conditional on Top being playe

d

TL

TR

BL

BR

1TB

a – c

b – d

1BT

 c – ad – b2LRa – b c – d 2RL b – a d – cSlide13

The real game  vs. the revealed game

G

The contents of

G

(rather than

in



)

suffice to determine all the

noncooperative

equilibria

of the game.

Definition: a

correlated equilibrium

of the game is a distribution



that satisfies

G

  0

, i.e., that

assigns non-negative expected value to each of the rows of GIf used by a mediator to generate recommended strategies, it would be incentive-compatible for all players to complyIt is determined merely by linear inequalitiesDefinition: a Nash equilibrium is a correlated equilibrium in which randomization, if any, is independentBut… why require independence? Correlation may be beneficial and/or it could describe the rational beliefs of an observer who doesn’t know which equilibrium has been selectedSlide14

The real game  vs. the revealed game

G

What’s in



that is not in

G

? Information about

externalities

: how one player benefits from a change in another’s strategy, holding her own strategy fixed.

The

missing

information is what may give rise to dilemmas in which strategically rational behavior is not socially rational

It

also determines

which side of the correlated equilibrium

polytope

is the efficient frontier.Slide15

Geometry of Nash & Correlated Equilibria

THEOREM

(

Nau

et al.

2003):

The Nash

equilibria

of a game are all located on the

surface of the correlated equilibrium

polytope

, i.e., on supporting

hyperplanes

to it.

Example: Battle-of-the-sexes

The set of correlated

equilibria

is a

hexadron

, with 5 vertices and 6 faces.

2 of the vertices are pure Nash

equilibria

1 is a mixed Nash equilibrium 2 are extremal non-Nash correlated equilibriaSlide16

The correlated equilibrium

polytope

of Battle-of-the-Sexes

The

polytope

is determined by a system of linear inequalities: incentive constraints for not deviating from the strategy recommended to you by a possibly-correlated randomization device

The Nash

equilibria

are points where the

polytope

touches the “saddle” of independent

distribuitions

Left

Right

Top

2, 1

0, 0

Bottom

0, 0

1, 2Slide17

The pure Nash

equilibria

(as seen from 2 angles)Slide18

The mixed Nash equilibrium

In general Nash

equilibria

do not need to be vertices of the

polytope

: they can fall in the middle of edges or lie on curves within faces of itSlide19

The

extremal

non-Nash correlated

equilibriaSlide20

The “obvious” coin-flip solution of battle-of-the-sexes is the midpoint of the edge connecting TL and

BR, which is neither

a Nash equilibrium nor a vertex of the

polytopeSlide21

THEOREM: In a 2x2 game with a 3-dimensional correlated equilibrium

polytope

(like BOS), the efficient frontier may only consist of one of the following:

One of the two pure Nash

equilibra

Both of the pure Nash

equilibria

, and their convex combinations

Both of the pure Nash

equilibria

and one of the two

extremal

non-Nash correlated

equilibria

, and their convex combinations

The mixed strategy Nash equilibrium is never efficient,

no matter what the externalities are.

Completely mixed Nash

equilibria

are generally inefficient in games with multiple

equilibria

, because they satisfy the incentive constraints with equality and hence

they are also equilibria of the game with the opposite payoffs.Slide22

Conjecture: a completely mixed Nash equilibrium can never be efficient in

any

game whose correlated equilibrium

polytope

is of maximal dimension

(

i.e., dimension n-1 in a game with n outcomes), which is a generalized game of coordination.

Seems to be empirically true based on numerical experiments, but a general proof is (so far) elusiveSlide23

Example of a game with a unique Nash equilibrium in completely mixed strategies that is not a vertex of the correlated equilibrium polytope.

Similar to a “poker” game devised by Shapley and discussed by Nash (1951)

The

polytope

is 7-dimensional (i.e., full dimension) with 33 vertices

The Nash equilibrium lies in the middle of an edge

and has irrational coordinates

.Slide24

Example: a game with a continuum of inefficient Nash equilibria lying in a face of the polytope

:

The CE

polytope

is has 7 vertices (full dimension).

By manipulating externalities, any of the vertices can be placed on the efficient frontier.

The Nash

equilibria

lie

in a face of the polytope

along

an

open

curve

connecting two of the

vertices.

The face containing the Nash

equilibria

can be placed on the inefficient frontier, but not the efficient frontier.Slide25

The solution concept of correlated equilibrium provides a tight link between subjective probability theory (á la de Finetti), game theory (á la

Aumann

), and finance theory (á la Black-Scholes)

In all of the fundamental theorems, the subjective parameters of belief are revealed through betting, and the rationality criterion is that of no-arbitrage.

But there’s a catch: the interpretation of the probabilities as measures of pure belief is complicated by the presence of risk aversion.

What if utility for money is state-dependent due to risk aversion and prior stakes in events?

A unified theory of individual, strategy, and competitive rationality?Slide26

Risk neutral probabilities

In financial markets, the probability distributions that rationalize asset prices are not measures of pure belief.

They are “risk neutral probabilities” that characterize a “risk neutral representative agent” but which do NOT reflect the beliefs of risk averse real agents

Risk neutral probabilities are interpretable as products of true probabilities and state-dependent marginal utilities

The average real agent is risk averse with higher marginal utilities for money in states where asset prices are low, so her risk neutral distribution is typically left-shifted compared to her true probability distribution.

The mean of the risk neutral distribution of asset returns must equal the risk free rate of returnSlide27

Risk neutral equilibria

How does the presence of risk aversion and large stakes change the solution of the game whose parameters are made commonly known through betting?

The parameters of

equilibria

become risk neutral probabilities!

It also changes the geometry of the set of

equilibria

The

polytope

of

equilibria

is distorted in a systematic way, analogous to the way the risk neutral distribution of an asset prices is left-shifted.

The

polytope

“blows up” to a larger size.Slide28

Example: Matching penniesThis is a zero-sum game, not a coordination game like Battle-of-the-Sexes or Chicken

The correlated equilibrium

polytope

consists of a single point—a mixed Nash equilibrium in which both players use independent 50/50 randomization

when payoffs are in units of money and both players are risk neutral

Left

Right

Top

1,



1



1, 1

Bottom



1, 1

1,



1Slide29

…and the unique Nash/correlated equilibrium the point in the middle of the saddle of independent distributions (the uniform distribution)

TL

TR

BL

BR

1TB

1

-1

0

0

1BT

0

0

-1

1

2LR

1

0

-1

0

2RL

0

-101The rules-of-the-game matrix G looks like this:Slide30

Risk averse players

Suppose players in such a game are significantly risk averse (i.e., have small risk tolerances compared to game payoffs)?

The game is then

non-zero-sum

in units of utility

The very idea of a game that is zero-sum when utility for money is not linear does not really make sense.

In general, strategic incentives measured in units of utility cannot be made common knowledge through conversations in terms of acceptable bets.Slide31

Consider a generic 2x2 game whose monetary payoffs are

Let u

1

(.) and u

2

(.) denote the players’ utility functions

The rows of the (unobservable) “true” rules-of-the-game matrix would be bets with payoffs in units of

utility

:

… because this is the necessary condition for them to have

zero expected utility

w.r.t. the players’ true probabilities

Left

Right

Top

a, a



b, b



Bottom

c, c

d, d

 TLTRBLBR1TBu1(a) – u1(c)u1(b) – u1(d)  1BT  u1(c) – u1(a)u1(b) – u1(d)2LRu1(a) – u1(b) u1(c) – u1(d) 2RL 

u

1

(b

) –

u

1

(a



)

u

1

(d



) –

u

1

(c



)

G*

= Slide32

Let v

1

(.) and v

2

(.) denote the players’

marginal

utility functions (i.e., v

i

is the derivative of

u

i

)

Then the payoff vectors of acceptable

monetary

bets that reveal their strategic incentives (denoted “

G*

”) looks like this:

The payoffs of the bets are in units of money, and they are

distorted

compared to what their payoffs would have been under linear utility for money

The monetary payoff of a bet (e.g., (u

1

(a) – u1(c))/v1(a)) is the true-game utility difference divided by the marginal utility that prevails in that outcome of the game (e.g., v1(a)) TLTRBLBR

1TB(u1(a) – u1(c))/v1(a)(u1(c) – u1(a))/v1(a)  1BT  (u1(b) – u1(d))/v1(c)(u1(b) – u1(d))/v1(c)2LR(u2(a) – u1(b))/v2(a) (u2(c) – u1(d))/v2(c) 2RL (u2(b) – u1(a))/v2(b

)

(u

2

(d



) –

u

1

(c



))/v

2

(c



)Slide33

Risk neutral equilibria defined

The appropriate generalization of a correlated equilibrium under conditions where players are risk averse is a risk neutral equilibrium

Definition

: a

risk neutral

equilibrium

of the game is a distribution



that satisfies

G*



 0

, i.e., that

assigns non-negative expected value to each of the rows of

G*

Implementation would require a mediator to use a randomization device whose inputs include events about which players might have

different true probabilities

but would must have

identical risk neutral probabilities

, as if they have already reached equilibrium in a contingent claims market with assets pegged to those events.Slide34

where

a

= 1 





½



0.293 and

b

= 1 





2





0.414.

This is a

negative sum

game in units of utility.

The local marginal utilities are u’(a) = 0.245 and u’(b) = 0.490Let u(x) = 1  exp(x) for both players, with risk aversion  = LN(2). Then the rules of the game in units of utility are:Example: matching pennies with monetary payoffs of 1:Slide35

The real rules of the game matrix,

G*

, whose rows are the acceptable monetary bets (scaled to maximum of +1):

The risk-neutral-equilibrium

polytope

, determined by the inequalities

G*

  0

,

has 4 vertices, each of which assigns strictly positive expected value to exactly one of the bets:Slide36

Main result

THEOREM:

The set of correlated

equilibria

of

a nontrivial

game with monetary payoffs played by

strictly risk

neutral players is a

strict subset of the set of risk neutral

equilibria

of the same game played by risk averse players.

The set of risk neutral

equilibria

of Matching Pennies is a tetrahedron with points on both sides of the saddle of independent distributions.

The unique Nash/correlated equilibrium is in its interior.

From an observer’s perspective, the players’ reciprocal beliefs are less precisely determined than in the risk neutral case.Slide37

Conclusions

The

game-theoretic

assumption of common knowledge of payoff

functions can be made precise in terms of the language of betting.

The assumption of common knowledge of rationality can be made precise in terms of

no-arbitrage

.

When players are risk neutral the solution concept to which this leads is

correlated equilibrium, not Nash equilibrium.A Nash equilibrium is a special case of a correlated equilibrium, not the other way around

.

The “solution” of a game is generally a convex set of correlated

equilibria

.

The constraints that determine

equilibria

ignore information about externalities, which is why strategic

equilibria

are not necessarily efficient.Slide38

Conclusions

When the players in a game are

risk averse

, the assumption that they have common knowledge of each others’

utilities

is very problematic.

There is no “market” in which assets are priced in units of utility, and prices that agents are willing to pay for monetary assets do not reveal their utility functions.

This paper proposes a method that theoretically could address this problem, and it leads to the conclusion that in games with risk averse players:

The parameters of strategic

equilibria

are

risk neutral probabilities

, as in financial markets with risk averse investors

They are

less-precisely determined

(even for zero-sum games) than if players were risk averse.