PCAP Project: Probabilistic CAP and Adaptive
Description: PCAP Project: Probabilistic CAP and Adaptive Key-value Stores Indranil Gupta Associate Professor Dept. of Computer Science, University of Illinois at Urbana-Champaign Joint work with Muntasir Raihan Rahman, Lewis Tseng, Son Nguyen, Nitin
Related Topics
Download Presentation
"PCAP Project: Probabilistic CAP and Adaptive" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. PCAP Project: Probabilistic CAP and Adaptive Key-value Stores Indranil Gupta
Associate Professor
Dept. of Computer Science, University of Illinois at Urbana-Champaign Joint work with
Muntasir Raihan Rahman, Lewis Tseng, Son Nguyen, Nitin Vaidya Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 1<br>
slide2. Key-value/NoSQL Storage Systems Key-value/NoSQL stores: $3.4B sector by 2018
Distributed storage in the cloud
Netflix: video position (Cassandra)
Amazon: shopping cart (DynamoDB)
And many others
NoSQL = “Not Only SQL” 2<br>
slide3. Key-value/NoSQL Storage Systems (2) Necessary API operations: get(key) and put(key, value)
And some extended operations, e.g., “CQL” in Cassandra key-value store
Lots of open-source systems (startups)
Cassandra (Facebook)
Riak (Basho)
Voldemort (LinkedIn)
Closed-source systems with papers
Dynamo 3<br>
slide4. Key-value/NoSQL Storage: Fast and Fresh Cloud clients expect both
Availability: Low latency for all operations (reads/writes)
500ms latency increase at Google.com costs 20% drop in revenue
each extra ms $4 M revenue loss
Consistency: read returns value of one of latest writes
Freshness of data means accurate tracking and higher user satisfaction
Most KV stores only offer weak consistency (Eventual consistency)
Eventual consistency = if writes stop, all replicas converge, eventually
Why eventual? Why so weak? 4<br>
slide5. CAP Theorem NoSQL Revolution Conjectured: [Brewer 00]
Proved: [Gilbert Lynch 02]
When network partitioned, system must choose either strong consistency or availability.
Kicked off NoSQL revolution
Abadi PACELC
If P, choose A or C
Else, choose L (latency) or C Consistency Partition-tolerance Availability
/Latency RDBMSs Cassandra, RIAK,
Dynamo, Voldemort HBase, HyperTable,
BigTable, Spanner 5<br>
slide6. Hard vs. Soft Partitions CAP Theorem looks at hard partitions
However, soft partitions may happen inside a data-center
Periods of elevated message delays
Periods of elevated loss rates Data-center 1
(America) Data-center 2
(Europe) Hard partition ToR ToR CoreSw Congestion at switches
=> Soft partition 6<br>
slide7. Our work: From Impossibility to Possibility C Probabilistic C (Consistency)
A Probabilistic A (Latency)
P Probabilistic P (Partition Model)
Probabilistic CAP Theorem
PCAP System to support SLAs (service level agreements) 7<br>
slide8. PCAP Theorem: Impossible to achieve both Probabilistic Consistency and Latency under Probabilistic Partitions if:
tc + ta < tp and pua + pic < α Bad network -> High (α, tp ) To get better consistency -> lower (pic ,tc) To get better latency -> lower (pua ,ta) Probabilistic CAP 8<br>
slide9. 9 Towards Probabilistic SLAs Consistency SLA: Goal is to
Meet a desired freshness probability (given freshness interval)
Maximize probability that client receives operation’s result within the timeout
Example: Google search application/Twitter search
Wants users to receive “recent” data as search
Only 10% results can be more than 5 min stale
SLA: (pic , tc)=(0.1, 5 min)
Minimize response time (fast response to query)
Minimize: pua (Given: ta)<br>
slide10. 10 Towards Probabilistic SLAs (2) Latency SLA: Goal is to
Meet a desired probability that client receives operation’s result within the timeout
Maximize freshness probability within given freshness interval
Example: Amazon shopping cart
Doesn’t want to lose customers due to high latency
Only 10% operations can take longer than 300ms
SLA: (pua, ta) = (0.1, 300ms)
Minimize staleness (don’t want customers to lose items)
Minimize: pic (Given: tc)<br>
slide11. Continuously adapt control knobs to
always satisfy PCAP SLA System assumptions:
Client sends query to coordinator server
which then forwards to replicas (answers reverse path)
There exist background mechanisms to bring
stale replicas up to date Meeting these SLAs: PCAP Systems 11<br>
slide12. Original k-v store PCAP system knobs PCAP Coordinator (1) Inject test operations (read and write) (2) Get estimate of current
consistency or latency
By analyzing operation log (3) Update control knobs to reach target SLA Passive Approach:
Sample ongoing client operations
Non-intrusive to client workloads Active approach PCAP System Control Loop Architecture 12<br>
slide13. Lognormal delay variation Meeting Consistency SLA for PCAP Cassandra (pic=0.135) PCAP system
Satisfies
SLA and close to
optimal Setup
9 server Emulab cluster: each server has 4 Xeon + 12 GB RAM
100 Mbps Ethernet
YCSB workload (144 client threads)
Network delay: Log-normal distribution Mean latency =
3 ms | 4 ms | 5 ms 13<br>
slide14. Summary CAP Theorem motivated NoSQL Revolution
But apps need freshness + fast responses
Under soft partition
We proposed
Probabilistic models for C, A, P
Probabilistic CAP theorem – generalizes classical CAP
PCAP system satisfies Latency/Consistency SLAs
Integrated into Apache Cassandra and Riak KV stores Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 14<br>
slide15. MOOC on “Cloud Computing Concepts” On Coursera
Ran Feb-Apr 2015 (just wrapping up)
120K+ students
Covered distributed systems and algorithms used in cloud computing
Free and Open to everyone
https://www.coursera.org/course/cloudcomputing
Or do a search on Google for “Coursera Cloud Computing” (click on first link) 15 Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu<br>
slide16. Our Posters at GCASR 15 Consistency-Availability Tradeoffs and SLAs
Muntasir Rahman [POSTER HERE]
Online Reconfiguration operations: Morphus project [IEEE ICAC 2015]
Mainak Ghosh [POSTER HERE] 16 Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu<br>
slide17. Summary CAP Theorem motivated NoSQL Revolution
But apps need freshness + fast responses
Under soft partition
We proposed
Probabilistic models for C, A, P
Probabilistic CAP theorem – generalizes classical CAP
PCAP system satisfies Latency/Consistency SLAs
Integrated into Apache Cassandra and Riak KV stores Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 17<br>
Associate Professor
Dept. of Computer Science, University of Illinois at Urbana-Champaign Joint work with
Muntasir Raihan Rahman, Lewis Tseng, Son Nguyen, Nitin Vaidya Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 1<br>
slide2. Key-value/NoSQL Storage Systems Key-value/NoSQL stores: $3.4B sector by 2018
Distributed storage in the cloud
Netflix: video position (Cassandra)
Amazon: shopping cart (DynamoDB)
And many others
NoSQL = “Not Only SQL” 2<br>
slide3. Key-value/NoSQL Storage Systems (2) Necessary API operations: get(key) and put(key, value)
And some extended operations, e.g., “CQL” in Cassandra key-value store
Lots of open-source systems (startups)
Cassandra (Facebook)
Riak (Basho)
Voldemort (LinkedIn)
Closed-source systems with papers
Dynamo 3<br>
slide4. Key-value/NoSQL Storage: Fast and Fresh Cloud clients expect both
Availability: Low latency for all operations (reads/writes)
500ms latency increase at Google.com costs 20% drop in revenue
each extra ms $4 M revenue loss
Consistency: read returns value of one of latest writes
Freshness of data means accurate tracking and higher user satisfaction
Most KV stores only offer weak consistency (Eventual consistency)
Eventual consistency = if writes stop, all replicas converge, eventually
Why eventual? Why so weak? 4<br>
slide5. CAP Theorem NoSQL Revolution Conjectured: [Brewer 00]
Proved: [Gilbert Lynch 02]
When network partitioned, system must choose either strong consistency or availability.
Kicked off NoSQL revolution
Abadi PACELC
If P, choose A or C
Else, choose L (latency) or C Consistency Partition-tolerance Availability
/Latency RDBMSs Cassandra, RIAK,
Dynamo, Voldemort HBase, HyperTable,
BigTable, Spanner 5<br>
slide6. Hard vs. Soft Partitions CAP Theorem looks at hard partitions
However, soft partitions may happen inside a data-center
Periods of elevated message delays
Periods of elevated loss rates Data-center 1
(America) Data-center 2
(Europe) Hard partition ToR ToR CoreSw Congestion at switches
=> Soft partition 6<br>
slide7. Our work: From Impossibility to Possibility C Probabilistic C (Consistency)
A Probabilistic A (Latency)
P Probabilistic P (Partition Model)
Probabilistic CAP Theorem
PCAP System to support SLAs (service level agreements) 7<br>
slide8. PCAP Theorem: Impossible to achieve both Probabilistic Consistency and Latency under Probabilistic Partitions if:
tc + ta < tp and pua + pic < α Bad network -> High (α, tp ) To get better consistency -> lower (pic ,tc) To get better latency -> lower (pua ,ta) Probabilistic CAP 8<br>
slide9. 9 Towards Probabilistic SLAs Consistency SLA: Goal is to
Meet a desired freshness probability (given freshness interval)
Maximize probability that client receives operation’s result within the timeout
Example: Google search application/Twitter search
Wants users to receive “recent” data as search
Only 10% results can be more than 5 min stale
SLA: (pic , tc)=(0.1, 5 min)
Minimize response time (fast response to query)
Minimize: pua (Given: ta)<br>
slide10. 10 Towards Probabilistic SLAs (2) Latency SLA: Goal is to
Meet a desired probability that client receives operation’s result within the timeout
Maximize freshness probability within given freshness interval
Example: Amazon shopping cart
Doesn’t want to lose customers due to high latency
Only 10% operations can take longer than 300ms
SLA: (pua, ta) = (0.1, 300ms)
Minimize staleness (don’t want customers to lose items)
Minimize: pic (Given: tc)<br>
slide11. Continuously adapt control knobs to
always satisfy PCAP SLA System assumptions:
Client sends query to coordinator server
which then forwards to replicas (answers reverse path)
There exist background mechanisms to bring
stale replicas up to date Meeting these SLAs: PCAP Systems 11<br>
slide12. Original k-v store PCAP system knobs PCAP Coordinator (1) Inject test operations (read and write) (2) Get estimate of current
consistency or latency
By analyzing operation log (3) Update control knobs to reach target SLA Passive Approach:
Sample ongoing client operations
Non-intrusive to client workloads Active approach PCAP System Control Loop Architecture 12<br>
slide13. Lognormal delay variation Meeting Consistency SLA for PCAP Cassandra (pic=0.135) PCAP system
Satisfies
SLA and close to
optimal Setup
9 server Emulab cluster: each server has 4 Xeon + 12 GB RAM
100 Mbps Ethernet
YCSB workload (144 client threads)
Network delay: Log-normal distribution Mean latency =
3 ms | 4 ms | 5 ms 13<br>
slide14. Summary CAP Theorem motivated NoSQL Revolution
But apps need freshness + fast responses
Under soft partition
We proposed
Probabilistic models for C, A, P
Probabilistic CAP theorem – generalizes classical CAP
PCAP system satisfies Latency/Consistency SLAs
Integrated into Apache Cassandra and Riak KV stores Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 14<br>
slide15. MOOC on “Cloud Computing Concepts” On Coursera
Ran Feb-Apr 2015 (just wrapping up)
120K+ students
Covered distributed systems and algorithms used in cloud computing
Free and Open to everyone
https://www.coursera.org/course/cloudcomputing
Or do a search on Google for “Coursera Cloud Computing” (click on first link) 15 Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu<br>
slide16. Our Posters at GCASR 15 Consistency-Availability Tradeoffs and SLAs
Muntasir Rahman [POSTER HERE]
Online Reconfiguration operations: Morphus project [IEEE ICAC 2015]
Mainak Ghosh [POSTER HERE] 16 Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu<br>
slide17. Summary CAP Theorem motivated NoSQL Revolution
But apps need freshness + fast responses
Under soft partition
We proposed
Probabilistic models for C, A, P
Probabilistic CAP theorem – generalizes classical CAP
PCAP system satisfies Latency/Consistency SLAs
Integrated into Apache Cassandra and Riak KV stores Distributed Protocols Research Group (DPRG)
http://dprg.cs.uiuc.edu 17<br>