CAFÉ: Scalable Task Pool with Adjustable - PowerPoint Presentation

Slide1

CAFÉ: Scalable Task Pool with Adjustable

Fairness and Contention

Dmitry Basin,

Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman

Technion

, Israel Institute of Technology

Slide2

Task Pools

Exist in most server applications:Web Servers, e.g., building block of SEDA ArchitectureHandling asynchronous requests

Ubiquitous programming pattern for parallel programsScalability is essential!

Task Pool

Producers

Consumers

Shared Memory

Slide3

Typical Implementation: FIFO QueueHas inherent scalability problem

Do we really need FIFO ?In many cases no!We would like to:Relax the requirementControl the degree of relaxation

contention points

Shared Memory

Slide4

CAFÉ: Contention and Fairness ExplorerOrdered list of scalable bounded non-FIFO pools

TreeContainer size controls contention-fairness trade-off

garbage

collected

TreeContainer

TreeContainer

TreeContainer

Java VM

Less fairness

Less contention

More fairness

More contention

Tree height

0

Pure FIFO

Slide5

TreeContainer (TC) SpecificationBounded containerA put operation can fail if no free space found

A get operation returns a task or null if TC is emptyRandomized algorithms for put and get operations

Slide6

TreeContainer Data Structure

Complete binary tree

Free node

Used node without a task

Occupied node containing a task

Right sub-tree doesn’t contain tasks

Left sub-tree has tasks

Slide7

Get/Put OperationsStep 1: find target nodeBy navigating the treeStep 2: perform put/get on that nodeNeed to handle races

Step 3: update routesUpdate meta-data on path to node – tricky!

Slide8

Task

Get()

Operation

Get():

Start from the root

TreeContainer

CAS

Close to the root updates are rare

Step3: Update routes up to the root

Step1: Navigate to a task by random walk on arrows graph

Step 2: Extract the task.

Slide9

Put()

Operation

Level 0

Level 1

Level 2

Level 3

TreeContainer

Random node

occupied

Random node

occupied

Random

node

√ free

Task

put()

:

Every level of the tree

implemented by array of nodes

occupied

Random node

Slide10

Put()

Operation

Task

put()

:

Go to the highest free predecessor

CAS

operation

TreeContainer

Level 0

Level 1

Level 2

Level 3

Random

node

√ free

Finished Step 1: found free node

Step 2: occupy the free node

Slide11

Put() Operation

put()

:

Close to the root, updates are rare

true

TreeContainer

Update routes

Slide12

When

Put()

Operation Fails

put()

:

false

TreeContainer

Random

node

k trials fail

Level 0

Level 1

Level 2

Last Level

Slide13

RacesConcurrency issues are not trivial :)Challenge:

guarantee linearizabilityavoid updating all the metadata up to the root upon each operationSee the paper

Slide14

TreeContainer propertiesPut/Get operations arelinearizable

wait-freeUnder the worst-case thread scheduling:Good step complexity of putsWhen N nodes occupied - O(log

2N) whpDoes not depend on TC size

Good tree density (arbitrarily close to 2h whp)

TreeContainer

Slide15

CAFÉ Data Structures

TC

TC

GT

TC

PT

Slide16

CAFÉ Data Structures

TC

TC

TC

PT

TC

TC.Put

(task)

 false

Allocate and connect new TC

TC.Put

(task)

 true

GT

Slide17

TC

CAFÉ:

get

() from Empty TC

TCTC

TC

PT

TC

TC.Get



null

TC.Get

task

Garbage collected

GT

Slide18

CAFÉ: Races

TC

TC

TC

PT

TC

Suspended producer

thread

The task is lost for consumers

GT

Slide19

CAFÉ: Handling Races – Try 1

TC

TC

TC

PT

TC

Move

GT back

Check if GT bypassed TC

GT

Slide20

CAFÉ: Races (2)

TC

TC

TC

PT

TC

TC.Get

 null

Consumer thread

Producer thread

Going to move GT forward

TC.Put

(task)

 true

Consumers can access the task

 can terminate

Task is lost

GT

Slide21

CAFÉ: Handling Races – Try 2

TC

TC

TC

PT

TC

Read

prev

,

If empty read

curr

GT

cur>

<

prev

Lock-Free. To make it wait-free we do additional tricks.

Slide22

CAFÉ: PropertiesSafety:Put()/Get() operations are linearizable

Wait-freedom:Get() operations are deterministically wait-freePut() operations are wait-free with probability 1Fairness:Preserves order among trees

Slide23

Evaluation SetupCompared pools:LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue

Evaluation server:8 AMD Opteron quad-cores  total 32 cores

Slide24

CAFÉ evaluationThroughputCAFÉ-13: CAFÉ with tree height 13

LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue

Throughput as a function of thread number

factor of 30

over lock-free implementations

Slide25

CAFÉ evaluationThroughputCAFÉ: CAFÉ queue

LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue .

CAFÉ throughput as a function of

TreeContainer height

Slide26

CAFÉ evaluationCAS-failures

CAS failures per operation as a function of TreeContainer height

Slide27

SummaryCAFÉ:EfficientWait-FreeWith adjustable fairness and contention

Thank you