# The Modified Stochastic Game PowerPoint Presentation 2016-09-01 66K 66 0 0

##### Description

Eilon Solan. , . Tel Aviv University. Omri N. Solan. Tel Aviv University. with. Multiplayer Absorbing Games. . I. = a finite set of players.. . A. i. = a finite set of actions of player . i. . . A := A. ID: 458635

Embed code:

DownloadNote - The PPT/PDF document "The Modified Stochastic Game" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

### Presentations text content in The Modified Stochastic Game

Slide1

The Modified Stochastic Game

Eilon Solan, Tel Aviv University

Omri N. SolanTel Aviv University

with

Slide2

Multiplayer Absorbing Games

I = a finite set of players. Ai = a finite set of actions of player i. A := A1 × … × AI. r : A → RI = nonabsorbing payoff function. p : A → [0,1] = probability of absorption. r* : A → RI = absorbing payoff function.

0

1

1

0

*

*

Example: the Big Match

Slide3

Multiplayer Absorbing Games

I = a finite set of players. Ai = a finite set of actions of player i. A := A1 × … × AI. r : A → RI = nonabsorbing payoff function. p : A → [0,1] = probability of absorption. r* : A → RI = absorbing payoff function.

Discounted Payoff

γλi(x) = λri(x) + (1- λ)(p(x)r*i(x) + (1-p(x)) γλi(x)

Expected absorbing payoff

Slide4

Multiplayer Absorbing Games

r : A → RI = nonabsorbing payoff. p : A → [0,1] = prob. of absorption. r* : A → RI = absorbing payoff.

Discounted Payoff

γλi(x) = λri(x) + (1- λ)(p(x)r*i (x) + (1-p(x)) γλi(x)

λri(x) + (1- λ)p(x)r*i (x)

1-(1- λ)(1-p(x))

γλi(x) =

λ

ri(x) + (1- λ)p(x)r*i (x)

λ + (1- λ)p(x)

=

Slide5

Multiplayer Absorbing Games

Discounted Equilibrium

λri(xλ) + (1- λ)p(xλ)r*i (xλ)

γλi (xλ) =

If

p(xλ) = o(λ) then lim γλ(xλ) = r(x0), and x0 is a uniform ε-equilibrium.

r : A → RI = nonabsorbing payoff. p : A → [0,1] = prob. of absorption. r* : A → RI = absorbing payoff.

If p(xλ) = ω(λ) then lim γλ(xλ) = lim r*(xλ), and xλ is a uniform ε-equilibrium provided λ is sufficiently small.

≥ vλi (=min-max value)

λ

+ (1-

λ

)p(x

λ

)

Slide6

Multiplayer Absorbing Games

Discounted Equilibrium

λri(xλ) + (1- λ)p(xλ)r*i (xλ)

γλi (xλ) =

r : A → RI = nonabsorbing payoff. p : A → [0,1] = prob. of absorption. r* : A → RI = absorbing payoff.

If p(xλ) = Θ(λ) then lim γλ(xλ) is a convex combination of r*(xλ) and r(xλ). When |I|=2, we have (a) r(x0) ≥ lim γλ(xλ) or (b) lim r*(xλ1,x02) ≥ lim γλ(xλ) or (c) lim r*(x01,xλ2) ≥ lim γλ(xλ). There is a uniform ε-equilibrium (Vrieze and Thuijsman 89).

≥ vλi (=min-max value)

λ

+ (1-

λ

)p(x

λ

)

Slide7

Modified Absorbing Games

Modified Discounted Payoff

λRi (x) + (1- λ)p(x)r*i (x)

Γλi (x) :=

r : A → RI = nonabs. payoff. Ri(x) := min{ ri(x), v0i }. p : A → [0,1] = prob. of absorption. r* : A → RI = absorbing payoff.

λ + (1- λ)p(x)

Theorem: Vλi := min max Γλi (xi,x-i) satisfies V0i=v0i.

x-i

xi

Theorem

: The modified game admits a discounted stationary equilibrium.

Slide8

Modified Stochastic Games 0

0

2

Attempt 1

: The modified payoff is the minimum between the stage payoff and the stage max-min value.

Original game:

0

1

Modified game:

The max-min value changed!

Slide9

Modified Stochastic Games 1

Let v1i,…,vLi be the different limit max-min value of player i.tλ(s1,σ;l) := E [ Σn=1 λ(1-λ)n-11 ]uλi(s1,σ;l) := E [ Σn=1 λ(1-λ)n-1ri(sn,an) 1 ]Γλi (s1,σ) := Σl=1L min{uλi(s1,σ;l) , vli tλ(s1,σ;l) }

{v0i(s(n)) =vli}

The modified game is the normal-form game ( I, Σi, (Γλi (s1,σ)){i in I} ) .

{v0i(s(n)) =vli}

s1,σ

The max-min value

s

1,σ

Note

: The modified game depends on the initial state.

Slide10

Modified Stochastic Games 2

Let τk be the k-th time in which the limit max-min value of player i changes:τ0 := 1 τk+1 := min{ n > τk : v0i(s(n)) ≠ v0i(s(τk)) }tλ(s1,σ;k) := E [ Σn=1 λ(1-λ)n-11 | H(τk) ]uλi(s1,σ;k) := E[ Σn=1 λ(1-λ)n-1ri(sn,an) 1 | H(τk) ]Γλi (s1,σ) := E[ Σk=0 min{ uλi(s1,σ;k) , v0i(s(τk)) tλ(s1,σ;k) } ]

s1,σ

{τk ≤ n < τk+1}

{τk ≤ n < τk+1}

Slide11

Results

Γλi (s1,σ) := Σl=1L min{uλi(s1,σ;l) , vli tλ(s1,σ;l) } Γλi (s1,σ) := E[ Σk=0 min{ uλi(s1,σ;k) , v0i(s(τk)) tλ(s1,σ;k) } ]

Theorem: In both modified games, V0i (s1) =v0i (s1).

Theorem: The first modified game admits a discounted stationary equilibrium (that depends on the initial state). The second modified game admits a more complex equilibrium.

Question: Does there exist a stationary strategy that is almost optimal for all initial states?

Theorem

: Analog results hold for min-max modification.

Slide12

Monovex Sets

Definition: A set X in Rd is monovex if for every x,y in X there is a continuous monotone path from x to y in X.

Theorem: In the first modified game, if the other players play stationary strategies, then player i has an optimal stationary best response. Moreover, the set of his stationary best responses is monovex.

Question

: Is every monovex set contractible?

Slide13

Monovex Sets

Definition: A set X in Rd is monovex if for every x,y in X there is a continuous monotone path from x to y in X.

Theorem: Every upper semi-continuous set-valued function from a compact convex subset of Rd to itself with monovex nonempty values has a fixed point.

Question

: Is every monovex set contractible?

Slide14

Merci

תודה רבה

Thank you

شكرا

To be continued

IHP, February 15, 2016, 10AM