The Data Cyclotron Query Processing Scheme Romulo Goncalves Martin Kersten ACM Trans Database Syst TODS 36427 2011 Outline Introduction Motivations Design Goals Interaction Protocols ID: 254629
Download Presentation The PPT/PDF document "The Data Cyclotron" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The Data Cyclotron
The Data Cyclotron Query Processing Scheme
Romulo GoncalvesMartin KerstenACM Trans. Database Syst. (TODS) 36(4):27 (2011)Slide2
Outline Introduction
MotivationsDesign Goals
Interaction ProtocolsExperimental ResultsConclusionsSlide3
System OverviewSlide4
Motivations High-performance local networks can be faster than disks
Load balancing based on data allocation doesn't perform well with skewed workloadsSlide5
Design Goals
Exploit high-performance local networksLoad balancing using data movements (handle skewed work-loads)Minimize query response time
Maximize query throughputNo single point of failureSlide6
System OverviewSlide7
Motivations: RDMASlide8
Motivations: RDMASlide9
ArchitectureSlide10
The DBMS LayerSlide11
Data Cyclotron LayerSlide12
The Network LayerSlide13
DC Layer: Request Propagation
Request_msg = (SenderID,
ChunkID)If (SenderID ==me) : raise exception (data is missing!)If ChunkID is stored here :It was already loaded to the ring? Ignore the request msg
Is the ring is full ?
YES : add the request to the pending queue
NO : load the data to the ring
If I need the very same
ChunkID
: absorb the request
Else: forward the request to the next node in the ringSlide14
DC Layer: Pending Requests
Check for available space at regular intervalsHandle the older
request that fits the actual available space in the ringSlide15
DC Layer: handling data-chunks
Data-chunk header:Data-chunk-uid
OwnerLOI (Level Of Interest)CopiesHopsCyclesSlide16
DC Layer: handling foreign chunks
Hops++If I was waiting for this
Data-chunk-uid :Wake-up pinned queriesCopies ++Forward the chunkSlide17
DC Layer: handling local chunks
Fragments are loaded/unload into the ring depending on their LOI
How to set the LOI threshold?Slide18
Some questions
How does it scale
?No single point of failure?How does it manage local bandwidth?Resource allocation is fair
?Slide19
Experiments: limited ring capacitySlide20
Experiments: limited ring capacitySlide21
Experiments: limited ring capacitySlide22
Experiments: skewed workloadsSlide23
Experiments: non uniform workloadsSlide24
Possible extensionsUPDATE queries
Pulsating ringsNomadic queriesSlide25
Thank you!Q&A time!