Fairness and Contention Dmitry Basin Rui Fan Idit Keidar Ofer Kiselov Dmitri Perelman Technion Israel Institute of Technology Task Pools Exist in most server applications ID: 781063
Download The PPT/PDF document "CAFÉ: Scalable Task Pool with Adjustabl..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CAFÉ: Scalable Task Pool with Adjustable
Fairness and Contention
Dmitry Basin,
Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman
Technion
, Israel Institute of Technology
Slide2Task Pools
Exist in most server applications:Web Servers, e.g., building block of SEDA ArchitectureHandling asynchronous requests
Ubiquitous programming pattern for parallel programsScalability is essential!
Task Pool
Producers
Consumers
Shared Memory
Slide3Typical Implementation: FIFO QueueHas inherent scalability problem
Do we really need FIFO ?In many cases no!We would like to:Relax the requirementControl the degree of relaxation
contention points
Shared Memory
Slide4CAFÉ: Contention and Fairness ExplorerOrdered list of scalable bounded non-FIFO pools
TreeContainer size controls contention-fairness trade-off
garbage
collected
TreeContainer
TreeContainer
TreeContainer
Java VM
Less fairness
Less contention
More fairness
More contention
Tree height
0
Pure FIFO
Slide5TreeContainer (TC) SpecificationBounded containerA put operation can fail if no free space found
A get operation returns a task or null if TC is emptyRandomized algorithms for put and get operations
Slide6TreeContainer Data Structure
Complete binary tree
Free node
Used node without a task
Occupied node containing a task
Right sub-tree doesn’t contain tasks
Left sub-tree has tasks
Slide7Get/Put OperationsStep 1: find target nodeBy navigating the treeStep 2: perform put/get on that nodeNeed to handle races
Step 3: update routesUpdate meta-data on path to node – tricky!
Slide8Task
Get()
Operation
Get():
Start from the root
TreeContainer
CAS
Close to the root updates are rare
Step3: Update routes up to the root
Step1: Navigate to a task by random walk on arrows graph
Step 2: Extract the task.
Slide9Put()
Operation
Level 0
Level 1
Level 2
Level 3
TreeContainer
Random node
occupied
Random node
occupied
Random
node
√ free
Task
put()
:
Every level of the tree
implemented by array of nodes
occupied
Random node
Slide10Put()
Operation
Task
put()
:
Go to the highest free predecessor
CAS
operation
TreeContainer
Level 0
Level 1
Level 2
Level 3
Random
node
√ free
Finished Step 1: found free node
Step 2: occupy the free node
Slide11Put() Operation
put()
:
Close to the root, updates are rare
true
TreeContainer
Update routes
Slide12When
Put()
Operation Fails
put()
:
false
TreeContainer
Random
node
k trials fail
Level 0
Level 1
Level 2
Last Level
Slide13RacesConcurrency issues are not trivial :)Challenge:
guarantee linearizabilityavoid updating all the metadata up to the root upon each operationSee the paper
Slide14TreeContainer propertiesPut/Get operations arelinearizable
wait-freeUnder the worst-case thread scheduling:Good step complexity of putsWhen N nodes occupied - O(log
2N) whpDoes not depend on TC size
Good tree density (arbitrarily close to 2h whp)
TreeContainer
Slide15CAFÉ Data Structures
TC
TC
GT
TC
PT
Slide16CAFÉ Data Structures
TC
TC
TC
PT
TC
TC.Put
(task)
false
Allocate and connect new TC
TC.Put
(task)
true
GT
Slide17TC
CAFÉ:
get
() from Empty TC
TCTC
TC
PT
TC
TC.Get
null
TC.Get
task
Garbage collected
GT
Slide18CAFÉ: Races
TC
TC
TC
PT
TC
Suspended producer
thread
The task is lost for consumers
GT
Slide19CAFÉ: Handling Races – Try 1
TC
TC
TC
PT
TC
Move
GT back
Check if GT bypassed TC
GT
Slide20CAFÉ: Races (2)
TC
TC
TC
PT
TC
TC.Get
null
Consumer thread
Producer thread
Going to move GT forward
TC.Put
(task)
true
Consumers can access the task
can terminate
Task is lost
GT
Slide21CAFÉ: Handling Races – Try 2
TC
TC
TC
PT
TC
Read
prev
,
If empty read
curr
GT
cur>
<
prev
Lock-Free. To make it wait-free we do additional tricks.
Slide22CAFÉ: PropertiesSafety:Put()/Get() operations are linearizable
Wait-freedom:Get() operations are deterministically wait-freePut() operations are wait-free with probability 1Fairness:Preserves order among trees
Slide23Evaluation SetupCompared pools:LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue
Evaluation server:8 AMD Opteron quad-cores total 32 cores
Slide24CAFÉ evaluationThroughputCAFÉ-13: CAFÉ with tree height 13
LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue
Throughput as a function of thread number
factor of 30
over lock-free implementations
Slide25CAFÉ evaluationThroughputCAFÉ: CAFÉ queue
LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree Queue .
CAFÉ throughput as a function of
TreeContainer height
Slide26CAFÉ evaluationCAS-failures
CAS failures per operation as a function of TreeContainer height
Slide27SummaryCAFÉ:EfficientWait-FreeWith adjustable fairness and contention
Thank you