COLOR TEST - PowerPoint Presentation

Slide1

COLOR TESTCOLOR TESTCOLOR TESTCOLOR TESTSlide2

Dueling Algorithms

Nicole Immorlica, Northwestern University with A. Tauman Kalai

, B. Lucier, A. Moitra, A. Postlewaite, and M. TennenholtzSlide3

Social ContextsNormal-form games: Players choose strategies to maximize expected von Neumann-Morgenstern utility.Social context games

[AKT’08]: Players choose strategies to achieve particular social status among peers.Slide4

Social ContextsRanking games [BFHS’08]: Players choose strategies to achieve particular payoff rank among peers.Slide5

Two-Player Ranking GamesG

Alice

Bob

RG payoff of Alice

:

1

Alice beats Bob in G

Alice and Bob play game

:

½

Alice ties Bob in G

0

Alice loses to Bob in GSlide6

Implicit RepresentationsSuccinct games [FIKU’08]: Payoff matrix represented by boolean circuit. NE hard to solve or approximate.

Blotto games [B’21, GW’50, R’06, H’08]: Distribute armies to battlefields.Slide7

Implicit RepresentationsOptimization duels [this work]: Underlying game is optimization problem. Goal is to optimize better than opponent.Slide8

Ranking DuelA search engine is an algorithm that inputsset Ω = {1, 2, …, n} of itemsprobabilities p

1 + … + pn = 1 of eachand outputs a permutation π of

Ω.Monopolist objective: minimize Ei~

p

[

π

(

i

)].Slide9

Ranking DuelCompetitive objective: Let the expected score of a ranking π versus a ranking π’ be

Pri~p[ π(i) <

π’(i) ] + (½) Pri~p[

π

(

i

) =

π

’(

i

) ].

Then objective is to output a

π

that maximizes expected score given algorithm of opponent.Slide10

Optimizing a Search Engine

?

User searches for object drawn according to known probability dist.Slide11

0.19

0.16

0.27

0.07

0.22

0.09

Search:

pretty shape

1.

(27%)

2.

(22%)

3.

(19%)

4.

(16%)

5.

(09%)

6.

(07%)

Greedy is optimal.Slide12

Choosing a Search Engine

Search for “pretty shape”.

See which search engine ranks my favorite shape higher.Thereafter, use that one.Slide13

0.19

0.16

0.27

0.07

0.22

0.09

Search:

pretty shape

1.

(27%)

2.

(22%)

3.

(19%)

4.

(16%)

5.

(09%)

6.

(07%)

Search:

pretty shape

6.

(27%)

1.

(22%)

2.

(19%)

3.

(16%)

4.

(09%)

5.

(07%)Slide14

Questions Can we efficiently compute an equilibrium of a ranking duel?

How poorly does greedy perform in a competitive setting? What consequences does the duel have

for the searcher?Slide15

Optimization Problems as Duels

Ranking

Binary Search

Routing

Parking

Compression

Hiring

Start

Finish

?

?

?

?

?

?

?Slide16

Duel FrameworkFinite feasible set X of strategies.Prob. distribution p over states of nature

Ω.Objective cost c: Ω

× X R.Monopolist: choose x to minimize Eω

~

p

[c

ω

(x)]

.Slide17

Duel Framework

Players select strategies

x, x’

from

X

.

Nature selects state

ω

from

Ω

according to

p

.

Payoffs

v(

x,x

’), (1-v(

x,x

’))

are realized.

1 if c

ω

(x) < c

ω

(x’)

0 if c

ω

(x) > c

ω

(x’)

½ if c

ω

(x) = c

ω

(x’)

v(

x,x

’) = E

ω

~

p

Slide18

Results: Computation An LP-based technique to compute

exact equilibria, A low-regret learning technique to compute approximate equilibria,

… and a demonstration of these techniques in our sample

settingsSlide19

Computing Exact EquilibriaFormulate game as bilinear duel:Efficiently map strategies to points X in R

n.Define constraints describing K=convex-hull(X).Define payoff matrix M that computes values.Maps

points in K back to strategies in original setting.Slide20

Bilinear Duels If feasible strategies X are points in Rn, and

payoff v(x, x’) is xtMx’ for some M in

Rnxn, then maxv,x v

s.t

.

x

t

Mx

’ ≥ v for all x’ in X

x is in K (=convex-hull(X))

Exponential, but equivalent poly-sized LP.Slide21

Ranking DuelFormulate game as bilinear duel:Efficiently map strategies to points X in Rn

. X = set of permutation matrices (entry xij indicates item

i placed in position j)Define constraints describing K=convex-hull(X). K = set of doubly stochastic matrices (entry

y

ij

= prob. item i placed in position j)Slide22

Ranking DuelFormulate game as bilinear duel:Design “rounding alg.” that maps points in K back to strategies in original setting.

Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).Slide23

Ranking DuelFormulate game as bilinear duel:Define payoff matrix M that computes values.

Ep,y,y’[v(x,x

’)] = ∑i p(i

) ( ½

Pr

y,y

’

[x

i

=

x’

i

] +

Pr

y,y

’

[x i

> x’

i ])

= ∑i p(i

) (∑

i

y

ij

( ½

y’

ij

+ ∑

k>j

y’

ik

))

which is bilinear in

y,y

’ and so can be written

ytMy’.Slide24

Ranking DuelResult: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach. Technique also applies to hiring duel and binary search duel.Slide25

Compression Duel

data

Goal

: smaller compression (i.e., lower depth in tree).

(each with prob. p(.))Slide26

Classical AlgorithmHuffman coding: Repeatedly pair nodes with lowest probability.Slide27

Compression DuelFormulate game as bilinear duel:Efficiently map strategies to points X in Rn

. X = subset of zero-one matrices* (entry

xij indicates item i placed at depth j)Define constraints describing K=convex-hull(X).

K = subset of

row-stochastic

matrices*

(entry

y

ij

= prob. item i placed at depth j)

* Must correspond to depth profile of some binary tree!Slide28

Compression DuelFormulate game as bilinear duel:Define payoff matrix M that computes values.

Ep,y,y’[v(x,x’)] = ∑

i p(i) (∑i

y

ij

( ½

y’

ij

+ ∑

k>j

y’

ik

))

which is bilinear in

y,y

’ and so can be written

ytMy’.Slide29

Compression DuelBilinear Form: maxv,x v

s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))

Problems: 1. How to round points in K back to a random binary tree with right depth profile? 2. How to succinctly express constraints describing K?Slide30

Approximate Minimax Defn. For any ε > 0, an

approximate minimax strategy guarantees payoff not worse than best possible value minus ε.

Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε

.Slide31

Best-Response Oracle Idea. Use approximate best-response oracle to get approximate minimax

strategies. 1. Low-regret learning

: if x1,…,xT and x’

1

,…,

x’

T

have

low regret

, then

ave

. is approx

minimax

.

2.

Follow expected leader

: on round t+1, play best-response to x

1

,…,

x

t to get low-regret.Slide32

Compression Best-Response

Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.

Multiple-choice Knapsack

: Slide33

Compression Best-Response

Depth

:

1

2

3

4Slide34

Compression Best-Response(each with prob. p(.))

x’ in K

For j from 1..n, list of depth j

:

v( ) = Pr[win at depth j |

x’

]

w( ) = 2

-j

…

Kraft inequalitySlide35

Other DuelsHiring duel: constraints defining Euclidean subspace correspond to hiring probabilities.Binary

search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees).Racing duel: seems computationally hard, even though single-player problem easy.Slide36

ConclusionEvery optimization problem has a duel.Classic solutions (and all deterministic algorithms) can usually be badly beaten.Duel can be easier or harder to solve, and can lead to inefficiencies.OPEN QUESTION

: effect of duel on the solution to the optimization problem?