COLOR TEST COLOR TEST COLOR TEST Dueling Algorithms Nicole Immorlica Northwestern University with A Tauman Kalai B Lucier A Moitra A Postlewaite and M ID: 271590
Download Presentation The PPT/PDF document "COLOR TEST" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
COLOR TESTCOLOR TESTCOLOR TESTCOLOR TESTSlide2
Dueling Algorithms
Nicole Immorlica, Northwestern University with A. Tauman Kalai
, B. Lucier, A. Moitra, A. Postlewaite, and M. TennenholtzSlide3
Social ContextsNormal-form games: Players choose strategies to maximize expected von Neumann-Morgenstern utility.Social context games
[AKT’08]: Players choose strategies to achieve particular social status among peers.Slide4
Social ContextsRanking games [BFHS’08]: Players choose strategies to achieve particular payoff rank among peers.Slide5
Two-Player Ranking GamesG
Alice
Bob
RG payoff of Alice
:
1
Alice beats Bob in G
Alice and Bob play game
:
½
Alice ties Bob in G
0
Alice loses to Bob in GSlide6
Implicit RepresentationsSuccinct games [FIKU’08]: Payoff matrix represented by boolean circuit. NE hard to solve or approximate.
Blotto games [B’21, GW’50, R’06, H’08]: Distribute armies to battlefields.Slide7
Implicit RepresentationsOptimization duels [this work]: Underlying game is optimization problem. Goal is to optimize better than opponent.Slide8
Ranking DuelA search engine is an algorithm that inputsset Ω = {1, 2, …, n} of itemsprobabilities p
1 + … + pn = 1 of eachand outputs a permutation π of
Ω.Monopolist objective: minimize Ei~
p
[
π
(
i
)].Slide9
Ranking DuelCompetitive objective: Let the expected score of a ranking π versus a ranking π’ be
Pri~p[ π(i) <
π’(i) ] + (½) Pri~p[
π
(
i
) =
π
’(
i
) ].
Then objective is to output a
π
that maximizes expected score given algorithm of opponent.Slide10
Optimizing a Search Engine
?
User searches for object drawn according to known probability dist.Slide11
0.19
0.16
0.27
0.07
0.22
0.09
Search:
pretty shape
1.
(27%)
2.
(22%)
3.
(19%)
4.
(16%)
5.
(09%)
6.
(07%)
Greedy is optimal.Slide12
Choosing a Search Engine
Search for “pretty shape”.
See which search engine ranks my favorite shape higher.Thereafter, use that one.Slide13
0.19
0.16
0.27
0.07
0.22
0.09
Search:
pretty shape
1.
(27%)
2.
(22%)
3.
(19%)
4.
(16%)
5.
(09%)
6.
(07%)
Search:
pretty shape
6.
(27%)
1.
(22%)
2.
(19%)
3.
(16%)
4.
(09%)
5.
(07%)Slide14
Questions Can we efficiently compute an equilibrium of a ranking duel?
How poorly does greedy perform in a competitive setting? What consequences does the duel have
for the searcher?Slide15
Optimization Problems as Duels
Ranking
Binary Search
Routing
Parking
Compression
Hiring
Start
Finish
?
?
?
?
?
?
?Slide16
Duel FrameworkFinite feasible set X of strategies.Prob. distribution p over states of nature
Ω.Objective cost c: Ω
× X R.Monopolist: choose x to minimize Eω
~
p
[c
ω
(x)]
.Slide17
Duel Framework
Players select strategies
x, x’
from
X
.
Nature selects state
ω
from
Ω
according to
p
.
Payoffs
v(
x,x
’), (1-v(
x,x
’))
are realized.
1 if c
ω
(x) < c
ω
(x’)
0 if c
ω
(x) > c
ω
(x’)
½ if c
ω
(x) = c
ω
(x’)
v(
x,x
’) = E
ω
~
p
Slide18
Results: Computation An LP-based technique to compute
exact equilibria, A low-regret learning technique to compute approximate equilibria,
… and a demonstration of these techniques in our sample
settingsSlide19
Computing Exact EquilibriaFormulate game as bilinear duel:Efficiently map strategies to points X in R
n.Define constraints describing K=convex-hull(X).Define payoff matrix M that computes values.Maps
points in K back to strategies in original setting.Slide20
Bilinear Duels If feasible strategies X are points in Rn, and
payoff v(x, x’) is xtMx’ for some M in
Rnxn, then maxv,x v
s.t
.
x
t
Mx
’ ≥ v for all x’ in X
x is in K (=convex-hull(X))
Exponential, but equivalent poly-sized LP.Slide21
Ranking DuelFormulate game as bilinear duel:Efficiently map strategies to points X in Rn
. X = set of permutation matrices (entry xij indicates item
i placed in position j)Define constraints describing K=convex-hull(X). K = set of doubly stochastic matrices (entry
y
ij
= prob. item i placed in position j)Slide22
Ranking DuelFormulate game as bilinear duel:Design “rounding alg.” that maps points in K back to strategies in original setting.
Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).Slide23
Ranking DuelFormulate game as bilinear duel:Define payoff matrix M that computes values.
Ep,y,y’[v(x,x
’)] = ∑i p(i
) ( ½
Pr
y,y
’
[x
i
=
x’
i
] +
Pr
y,y
’
[x i
> x’
i ])
= ∑i p(i
) (∑
i
y
ij
( ½
y’
ij
+ ∑
k>j
y’
ik
))
which is bilinear in
y,y
’ and so can be written
ytMy’.Slide24
Ranking DuelResult: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach. Technique also applies to hiring duel and binary search duel.Slide25
Compression Duel
data
Goal
: smaller compression (i.e., lower depth in tree).
(each with prob. p(.))Slide26
Classical AlgorithmHuffman coding: Repeatedly pair nodes with lowest probability.Slide27
Compression DuelFormulate game as bilinear duel:Efficiently map strategies to points X in Rn
. X = subset of zero-one matrices* (entry
xij indicates item i placed at depth j)Define constraints describing K=convex-hull(X).
K = subset of
row-stochastic
matrices*
(entry
y
ij
= prob. item i placed at depth j)
* Must correspond to depth profile of some binary tree!Slide28
Compression DuelFormulate game as bilinear duel:Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑
i p(i) (∑i
y
ij
( ½
y’
ij
+ ∑
k>j
y’
ik
))
which is bilinear in
y,y
’ and so can be written
ytMy’.Slide29
Compression DuelBilinear Form: maxv,x v
s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))
Problems: 1. How to round points in K back to a random binary tree with right depth profile? 2. How to succinctly express constraints describing K?Slide30
Approximate Minimax Defn. For any ε > 0, an
approximate minimax strategy guarantees payoff not worse than best possible value minus ε.
Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε
.Slide31
Best-Response Oracle Idea. Use approximate best-response oracle to get approximate minimax
strategies. 1. Low-regret learning
: if x1,…,xT and x’
1
,…,
x’
T
have
low regret
, then
ave
. is approx
minimax
.
2.
Follow expected leader
: on round t+1, play best-response to x
1
,…,
x
t to get low-regret.Slide32
Compression Best-Response
Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.
Multiple-choice Knapsack
: Slide33
Compression Best-Response
Depth
:
1
2
3
4Slide34
Compression Best-Response(each with prob. p(.))
x’ in K
For j from 1..n, list of depth j
:
v( ) = Pr[win at depth j |
x’
]
w( ) = 2
-j
…
Kraft inequalitySlide35
Other DuelsHiring duel: constraints defining Euclidean subspace correspond to hiring probabilities.Binary
search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees).Racing duel: seems computationally hard, even though single-player problem easy.Slide36
ConclusionEvery optimization problem has a duel.Classic solutions (and all deterministic algorithms) can usually be badly beaten.Duel can be easier or harder to solve, and can lead to inefficiencies.OPEN QUESTION
: effect of duel on the solution to the optimization problem?