1 Optimal Multi- A gent - PowerPoint Presentation

Slide1

1

Optimal Multi-Agent Path Findingunder the Sum of Costs Objective

Ariel

Felner

Ben-Gurion UniversityIsraelMuch of the work here is joint withRoni Stern, Guni Sharon, Pavel Surynek, Eli Boyarski

Slide2

Multi-agent path finding (MAPF)

2Input A graph with N states A set of K agents – each with start and goal stateActions

An agent can move or waitTask – a solution

A path for each agent (offline)

ConstraintsPaths shouldn’t conflict Agents cannot be in the same location at the same time(Edge constraints, Following policies)TargetMinimize the cost of the solutionCentralized solverOffline task

Slide3

Motivation

RoboticsVideo gamesTransportation applicationsWarehouse managementProduct assembly3

Slide4

Automated junction

Example from Peter Stone’s web site4

Slide5

Different cost functions

5Cost 1: sum of costs Cost 2: Makepan

=Individual path for agent

a

i µ≤ ξ ≤ m µ

Slide6

Complexity

6The problem was proved to be NP-hard [J. Yu and S. M. LaValle, AAAI-2013]The 15-puzzle is a special case of MAPF

Slide7

Complexity

7

Slide8

Main approaches

MAPF solvers

Suboptimal

Optimal

Search based

Procedure based

Reduction

Solvers

Slide9

Suboptimal solvers

Slide10

Agents are planned individually

Then, conflicts and deadlocks are resolvedAttributes:FastEasy to understand/implementForfeit optimality/completeness

Searched-based suboptimal solvers

Slide11

Cooperative A*

[Silver 2005]Initialize the reservation table TFor each agent do{Find a path (do not conflict with T)Reserve the path in T}Enhancements:Hierarchical CA* (HCA) [Sliver 2005]Windowed-Hierarchical CA* (WHCA*) [Silver 2005]

Conflict Oriented WHCA* [Banya and Felner, ICRA 2014]

Slide12

Relaxing Optimal Solvers

Any optimal solver can be relaxed.WA* of any A*-based algorithmSuboptimal A*+OD+ID [Standley and Korf, IJCAI 2011]Suboptimal ICTS [Aljaloud and Sturtevant, SoCS 2013]CBS-e [

Barrer et al. SoCS-2014]

Slide13

Have specific movement rules

(e.g., go on highway)Complete!Very fast!Far from optimalCan solve very large problems

Procedure-based sub-optimal solvers

Slide14

Procedure-based MAPF solvers

A complete polynomial-time algorithm to the pebble motion problem was already introduced by [Kornhauser, FOCS 1984]It was recently implemented by Surynek.Agents move one at a time. Far from optimal.

Slide15

Procedure-based MAPF solvers

Slidable Multi-Agent Path Planning, [Wang & Botea, IJCAI, 2009]Complete for slidable grids

Push and Swap [Luna &

Bekris, IJCAI, 2011]

Parallel push and swap Push and Rotate [de Wilde et al. AAMAS 2013]Macro-basedComplete for graphs where at least two vertices are always unoccupiedBIBOX [Surynek 2013]Tree-based agent swapping strategy, [Khorshid at el. SOCS, 2011]

Complete for tree type graphs

Slide16

Optimal solvers

Slide17

Main approaches

Optimal MAPF solvers

Search based

Reduction based

A*, M*

ICTS, CBS

SAT

ASP

COP

Slide18

Main approaches

Optimal MAPF solvers

Search based

Reduction based

A*, M*

ICTS, CBS

SAT

ASP

COP

Makespan

Sum-of

costs

Slide19

Reduce MAPF to other known problems in computer science.

SAT [Surynek 2012] Integer Linear Programing [Yu et al. ICRA 2013] Answer Set Programming [Erdem et al, AAAI-2013]

Work extremely fast for small

graphs

May be very slow for large graphsiii. Reduction solvers

Slide20

For

i=1 to infinityCreate a SAT formula that answers: “Is there a solution to the problem of cost i”Solve that formula

Main challenge:

How do we do this for sum-of-costs?

SAT solver for makespan [Surynek]

Slide21

Integer Programing

[Yu and La-Valle (2013a]Model MAPF as a network flow problem.Depths of the flow are associated with the different time steps. Used Integer Linear Programming (ILP) to provided a set of equations and an objective function which yield the optimal solution.Done for Makespan

Slide22

Answer Set Programming

[Erden et al, 2013]Used the declarative programming paradigm of Answer Set Programming (ASP) for optimally solving MAPF. They represent the path finding problem for each agent and the inter-agent constraints as a program P in ASP. The answer sets of P correspond to solutions of the problem.

Slide23

Main approaches

Optimal MAPF solvers

Search based

Reduction based

A*, M*

ICTS, CBS

SAT

ASP

COP

Makespan

Sum-of

costs

Our current work

Very easy

Slide24

Main approaches

Optimal MAPF solvers

Search based

Reduction based

A*, M*

ICTS, CBS

SAT

Makespan

Sum-of

costs

Our current work

TODAY

Slide25

Search-based optimal solvers

Slide26

A*-based algorithm

A*EPEA*A*+OD+IDM*Other search algorithms ICTSCBSMA-CBSOptimal Search-based MAPF solvers

Slide27

A* approaches

State space: Permutations of K agents into N locations=O(NK)

Operators: Locations of all agent in the next time step

Heuristic function: Sum of Individual Costs (SIC)27 SIC = 3+3 = 6Optimal = 3+4 = 7

Wait

Slide28

Problem 1

: State space is too largeSolution: Let’s abstract the underlying graph

Problems with A*

28

Ryan [2008,2011] abstracted the underlying into known shapes such as halls, rings and corridors. Have specific expansion schedule for each of these cases.Sometimes not optimal.

Slide29

Divide the agents into independent groups

Solve each group separatelyi) Independence detection (ID) [Standely

2010]

S

3G3

29

Problem 2

:

The state space is exponential O(

N

K

)

On a 10x10 grid with 10 agents: =

10

20

Solution: let’s reduce the number of agents!

Problems with A*

Slide30

Problem 3

: The branching factor is exponential: bglobal=bK On a grid with 20 agents: bk= 520=

95,367,431,640,625

Solution: let’s reduce the branching factor!

Problems with A*30M* [Wagner 2011]Dynamically change the branching factor based on conflicts.

Works on the global search space but starts with single

moves of agents

When a conflict occurs between two agents M* moves

back

to all ancestors

and generates ALL possible

children.

Slide31

31

S1,S2

A1,B1

C,C

XX

M*

Slide32

32

S1,S2

A1,B1

Am,Bm

…M*

Slide33

Problem 4

: Surplus nodes (those with f>C)Solution: let’s avoid them

Problems with A*

33

i) Operator Decomposition (OD) [Standley AAAI-2010] Intermediate states Each level in the tree moves a single agent Every K levels we have a full state (as A*)ii) Enhanced partial expansion A* (EPEA*) [AAAI-2012]

[Goldenberg et al. 2012] studied combinations of these approaches

Slide34

34

1) The Increasing Cost Tree Search (ICTS) [Sharon et al. IJCAI-2011, AIJ-2012] 2) Conflict-Based Search (CBS) [Sharon et al. AAAI-2012] 3) Meta-agent Conflict-Based Search (MA-CBS) [Sharon et al. SoCS-2012]

New non-A* algorithms

These algorithms are exponential in different parameters

Slide35

Is there a solution with costs

?

High-level

Low-level

NO!101010

Algorithm 1: ICTS

[IJCAI-2011]

two level algorithm

Slide36

What about this?

10

11

10

1031136

High-level

Low-level

YES!

10

Algorithm 1: ICTS

[IJCAI-2011]

two level algorithm

Slide37

∆

No solution

Find a solution

ICTS: High level

37

30

31

32

SIC

Slide38

Experiments: Dragon-Age Origin

[Sturtevant]38

number of agents

#problems solved

under 5 minutes

Slide39

SIC =2

Optimal solution =74Δ=72A*: solved in 51msICTS: solved in 36,688msICTS: pathological case

Slide40

Motivation

: cases with bottlenecks:A* Algorithm 2: Conflict-based Search (CBS) [AAAI-2012]

40

S1,S2

A1,B1A1,B2

Am,Bm

…

C,C

A1,C

C,G2

G1,G2

f=6:

All m

2

combinations

of

(

A

i

,B

j

)

will be generated and expanded -

all will generate

(C,C)

which

is illegal

f=7:

3 states are expanded

Slide41

CBS – underlying idea

CBS plans for single agents but under constraints41

A* and ICTS work in a

K-agent search space

Slide42

Conflict

: [agent A, agent B, location X, time T ]Constraint: [agent A, location X, time T]

Conflict is resolved by adding either [

A,X,T]

or [B,X,T] CBS: general ideaPlan for each agent individuallyValidate plans If the plans of agents A and B conflict Constrain A to avoid the conflict or Constrain B to avoid the conflictConflicts and constraints42

Slide43

The constraint tree

Nodes: A set of individual constraints for each agent A set of paths consistent with the constraintsGoal test

:

Are the paths conflict free.

43

Expand

Expand

Goal

Root

OK

OK

Conflict

Goal Test

Not Goal

Replan 1

Replan 2

{1,2,C,2}

Slide44

A* : m

2+3 = O(m2) statesCBS: 2m+14 = O(m) statesWhen m > 4 CBS will examine fewer states than A*Analysis: example 1

44

Slide45

4 optimal solutions for each agent, each pair of solutions has a conflict

CBS: exponential in #conflicts = 54 statesA*: exponential in #agents = 8 statesWhat if I have both?

Analysis: example 2

Trends seen

In open spaces: use A* In bottlenecks: use CBS45

Slide46

1. Plan for each agent individually

2. Validate plans3. If the plans of agents A and B conflict5

Constrain A to avoid the conflicts

or Constrain B

to avoid the conflictShould merge(A,B) (simple rule)When the number of conflicts already seen between A,B > TAlgorithm 3: Meta-agent CBS (MA-CBS)4 If (should merge(A,B)) merge A and B into a meta-agent

and solve with A* Else

46

T=

∞

(never merge) basic CBS

T=0

(always merge)

Standley’s

ID

MA-CBS

Should merge(A,B): meta-reasoning rules

Slide47

Experiments: choosing T for MA-CBS

Many bottlenecks

Few bottlenecks

Many bottlenecks

 High T (closer to CBS)More agents  Low T (closer to A*)Faster single-agent search  lower T (close to A*)

Slide48

1. Plan for each agent individually

2. Validate plans3. If the plans of agents A and B conflict5

Constrain A to avoid the conflicts

or Constrain B

to avoid the conflictMore Enhancements: Improved-CBS 4 If (should merge(A,B)) merge A and B into a meta-agent and solve with A* Else 48Bypass conflictsPrioritize conflictsMerge and restart

Slide49

Dragon Age: Origins map with many corridors and bottlenecks

Higher is better

Lower is better

ICBS

Experiments: Improved-CBS

Slide50

We observed the following tendencies

:A* - exponential in #agents (NK)best in areas dense with agentsICTS - exponential in Δbest in open areas with few agentsCBS – exponential in #conflictsbest in areas with many bottlenecks

Summary:

No universal winner

MA-CBS continuum

MA-CBS continuum

Slide51

Reduce MAPF to other known problems in computer science.

SAT [Surynek 2012] Integer Linear Programing [Yu et al. ICRA 2013] Answer Set Programming [Erdem et al, AAAI-2013]

Reduction-based

solvers

Slide52

Main approaches

Optimal MAPF solvers

Search based

Reduction based

A*, M*

ICTS, CBS

SAT

Makespan

Sum-of

costs

Our current work

TODAY

Slide53

For

i=1 to infinityCreate a SAT formula that answers: “Is there a solution to the problem of cost i”Solve that formula

Main challenge:

H

ow do we do this for sum-of-costs?SAT solver for makespan [Surynek]

Slide54

Definitions:

()= shortest path for

=

= longest (

)

=

+

Δ

 

Connecting

makespan

to sum of costs

Claim: all solutions of cost

are done within

≤

+

Δ

time

steps

 

(

)=3

,

(

)=3

=

= 3

=7,

Δ

=1

 

=SIC

Slide55

Time-Expansion Graph (TEG)

Slide56

Time-Expansion Graph (TEG)

For each agent Standard edges: at time <

(

)

Extra edges: at time ≥ () 

Slide57

Is there a solution with exactly

Δ extra edges for TEGs of length  

Basic-SAT

Slide58

Using MDDs

We use MDDs instead of TEGs

Slide59

MDD-SAT is much better than basic SAT

MDD-SAT vs basic SAT

Slide60

Experiments

Slide61

Experiments

Slide62

MAP Brc202d

[from Sturtevant 2012]

Slide63

Summary

No universal winner

Search-based methods are faster for easier problems

SAT methods are faster for harder problems

Future: a better comparison is needed

Slide64

New setting for MAPF

Classical setting

Agents: fixed (all)

Time: minimize New settingTime: Fixed Agents: solve as manyAgents may have priorities

Slide65

Everybody needs to exit a dangerous area either to a global goal or to an group-specific goal

BC

A

Evacuation

Slide66

New setting for MAPF

Robots nay have different priorities

Optimize the reward for saving robots

What about priority to people?

People are self interested (morally correct or not?)Will people obey the algorithm? Nearest door policy

Slide67

Fairness, Social welfare

Algorithm (that people will obey)

Simulate the near door policy

Mark those who are saved (lucky)

Lucky people have priority of infinityOther people have priority of 1Save ALL lucky people and as many other peopleIn an A* search prune a node if there is at least one lucky agent that will not reach its goal