The Modified Stochastic Game - PowerPoint Presentation

Slide1

The Modified Stochastic Game

Eilon Solan, Tel Aviv University

Omri N. SolanTel Aviv University

withSlide2

Multiplayer Absorbing Games

I = a finite set of players.

Ai = a finite set of actions of player i. A := A

1 × … × AI.

r : A →

R

I = nonabsorbing payoff function. p : A → [0,1] = probability of absorption. r* : A → RI = absorbing payoff function.

0

1

1

0

*

*

Example: the Big MatchSlide3

Multiplayer Absorbing Games

I = a finite set of players.

Ai = a finite set of actions of player i. A := A

1 × … × AI.

r : A →

R

I = nonabsorbing payoff function. p : A → [0,1] = probability of absorption. r* : A → RI = absorbing payoff function.

Discounted Payoff

γ

λ

i(x) = λr

i(x) + (1- λ)(p(x)r*i

(x) + (1-p(x)) γλi

(x)

Expected absorbing payoffSlide4

Multiplayer Absorbing Games

r : A → R

I = nonabsorbing payoff. p : A → [0,1]

= prob. of absorption. r* : A → R

I

= absorbing payoff.Discounted Payoffγλi

(x) = λr

i(x) + (1- λ)(p(x)r*

i (x) + (1-p(x)) γλi

(x)

λri

(x) + (1- λ)p(x)r*i (x)

1-(1-

λ)(1-p(x))

γ

λi(x) =

λ

r

i

(x) + (1-

λ

)p(x)r*

i

(x)

λ

+ (1-

λ

)p(x)

=Slide5

Multiplayer Absorbing Games

Discounted Equilibrium

λri(x

λ) + (1- λ)p(x

λ

)r*

i (xλ)γλi (xλ

) =

If

p(xλ) = o(

λ) then lim γ

λ(xλ) = r(x

0), and x0

is a uniform ε-equilibrium.

r : A →

RI = nonabsorbing payoff.

p : A → [0,1] = prob. of absorption.

r* : A → RI

= absorbing payoff.

If

p(x

λ

) =

ω

(

λ

)

then

lim

γ

λ

(x

λ

) = lim r*(x

λ), and

x

λ

is a uniform

ε

-equilibrium provided

λ

is sufficiently small.

≥

vλi (=min-max value)

λ

+ (1-

λ

)p(x

λ

)Slide6

Multiplayer Absorbing Games

Discounted Equilibrium

λri(x

λ) + (1- λ)p(x

λ

)r*

i (xλ)γλi (xλ

) =

r : A →

RI = nonabsorbing payoff.

p : A → [0,1] = prob. of absorption. r* : A →

RI = absorbing payoff.

If

p(xλ) = Θ(

λ) then lim γ

λ(xλ) is a convex combination of

r*(xλ) and r(x

λ

)

. When |I|=2, we have (a)

r(x

0

)

≥

lim

γ

λ

(x

λ

)

or (b)

lim r*(x

λ1

,x02)

≥

lim

γ

λ

(x

λ

)

or (c)

lim r*(x01,xλ2) ≥

lim

γ

λ

(x

λ

)

. There is a uniform

ε-equilibrium (Vrieze and Thuijsman 89).

≥ vλi (=min-max value)

λ

+ (1-

λ

)p(x

λ

)Slide7

Modified Absorbing Games

Modified Discounted Payoff

λRi

(x) + (1- λ)p(x)r*i (x)

Γ

λ

i (x) :=

r : A →

RI = nonabs. payoff.

Ri(x) := min{ ri(x), v

0i }.

p : A → [0,1] = prob. of absorption.

r* : A → RI = absorbing payoff.

λ

+ (1- λ)p(x)

Theorem:

Vλi := min max

Γλi (x

i

,x

-i

)

satisfies

V

0

i

=v

0

i

.

x

-i

x

i

Theorem

: The modified game admits a discounted stationary equilibrium.Slide8

Modified Stochastic Games 0

0

2

Attempt 1

: The modified payoff is the minimum between the stage payoff and the stage max-min value.

Original game:

0

1

Modified game:

The max-min value changed!Slide9

Modified Stochastic Games 1

Let v1i

,…,vLi be the different limit max-min value of player i

.tλ(s

1

,

σ;l) := E [ Σn=1 λ(1-λ)n-11 ]

uλ

i(s1,σ

;l) := E [ Σn=1

λ(1-λ)

n-1ri(s

n,an) 1 ]

Γλi

(s1,σ

) := Σl=1L

min{uλi(s

1,σ;l) ,

v

l

i

t

λ

(s

1

,

σ

;l)

}

∞

{v

0

i

(s(n)) =v

l

i

}

The modified game is the normal-form game

( I,

Σ

i

, (

Γ

λ

i

(s

1

,

σ

))

{i in I

} )

.

{v0

i

(s(n)) =v

l

i

}

s

1

,

σ

The max-min value

s

1

,

σ

Note

: The modified game depends on the initial state.Slide10

Modified Stochastic Games 2

Let τk be the

k-th time in which the limit max-min value of player i changes:τ0 := 1

τk+1 :=

min{

n >

τk : v0i(s(n)) ≠ v0i(s(τk))

}t

λ(s1,

σ;k) := E [ Σ

n=1 λ(1-λ

)n-11 |

H(τk)

]uλi

(s1,σ

;k) := E[ Σn=1

λ(1-λ)

n-1ri(sn

,a

n

) 1

|

H

(

τ

k

)

]

Γ

λ

i

(s

1

,

σ

) := E[ Σ

k=0

min{

u

λ

i

(s

1

,

σ;k) , v0i(s(τ

k

)) t

λ

(s

1

,

σ

;k) } ]

∞s

1

,

σ

{

τ

k

≤ n <

τ

k+1

}

{

τ

k

≤ n <

τ

k+1

}

∞

∞Slide11

Results

Γλ

i (s1,σ) :=

Σl=1L min{u

λ

i

(s1,σ;l) , vli tλ(s1

,σ;l) }

Γλi

(s1,σ

) := E[ Σk=0

min{ uλi

(s1,σ;k)

, v0i(s(

τk)) tλ

(s1,σ;k)

} ]

∞

Theorem

: In both modified games,

V

0

i

(s

1

) =v

0

i

(s

1

)

.

Theorem

: The first modified game admits a discounted stationary equilibrium (that depends on the initial state). The second modified game admits a more complex equilibrium.

Question

: Does there exist a stationary strategy that is almost optimal for all initial states?

Theorem

: Analog results hold for min-max modification.Slide12

Monovex Sets

Definition: A set X in

Rd is monovex if for every x,y in

X there is a continuous monotone path from x to y in

X

.

Theorem: In the first modified game, if the other players play stationary strategies, then player i has an optimal stationary best response. Moreover, the set of his stationary best responses is monovex.

Question

: Is every monovex set contractible?Slide13

Monovex Sets

Definition: A set X in

Rd is monovex if for every x,y in

X there is a continuous monotone path from x to y in

X

.

Theorem: Every upper semi-continuous set-valued function from a compact convex subset of Rd to itself with monovex nonempty values has a fixed point.

Question

: Is every monovex set contractible?Slide14

Merci

תודה רבה

Thank you

شكرا

To be continued

IHP, February 15, 2016, 10AM