COS 418 Distributed Systems Lecture 12 Wyatt Lloyd Outline Network Partitions Linearizability CAP Theorem Consistency Hierarchy Network Partitions Divide Systems Network Partitions Divide Systems ID: 1003108
Download Presentation The PPT/PDF document "CAP Theorem and Consistency Models" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. CAP Theorem and Consistency ModelsCOS 418: Distributed SystemsLecture 12Wyatt Lloyd
2. OutlineNetwork PartitionsLinearizabilityCAP TheoremConsistency Hierarchy
3. Network Partitions Divide Systems
4. Network Partitions Divide Systems
5. How Can We Handle Partitions?Atomic Multicast?Bayou?Viewstamped Replication?Chord?Paxos?Dynamo?RAFT?
6. How About This Set of Partitions?
7. Fundamental Tradeoff?Replicas appear to be a single machine, but lose availability during a network partitionORAll replicas remain available during a network partition but do not appear to be a single machine
8. CAP Theorem PreviewYou cannot achieve all three of:ConsistencyAvailabilityPartition-TolerancePartition Tolerance => Partitions Can HappenAvailability => All Sides of Partition ContinueConsistency => Replicas Act Like Single MachineSpecifically, Linearizability
9. OutlineNetwork PartitionsLinearizabilityCAP TheoremConsistency Hierarchy
10. Linearizability [Herlihy and Wing 1990]All replicas execute operations in some total orderThat total order preserves the real-time ordering between operationsIf operation A completes before operation B begins,then A is ordered before B in real-timeIf neither A nor B completes before the other begins, then there is no real-time order(But there must be some total order)
11. Real-Time Ordering Examples
12. Linearizability == “Appears to be a Single Machine”Single machine processes requests one by one in the order it receives themWill receive requests ordered by real-time in that orderWill receive all requests in some orderAtomic Multicast, Viewstamped Replication, Paxos, and RAFT provide Linearizability
13. Linearizability is Ideal?Hides the complexity of the underlying distributed system from applications!Easier to write applicationsEasier to write correct applicationsBut, performance trade-offs, e.g., CAP
14. OutlineNetwork PartitionsLinearizabilityCAP TheoremConsistency Hierarchy
15. CAP Conjecture [Brewer 00]From keynote lecture by Eric Brewer (2000)History: Eric started Inktomi, early Internet search site based around “commodity” clusters of computersUsing CAP to justify “BASE” model: Basically Available, Soft-state services with Eventual consistencyPopular interpretation: 2-out-of-3Consistency (Linearizability)AvailabilityPartition Tolerance: Arbitrary crash/network failures
16. CAP Theorem [Gilbert Lynch 02]Assume to contradict that Algorithm A provides all of CAPClient 1Client 1
17. CAP Theorem [Gilbert Lynch 02]Assume to contradict that Algorithm A provides all of CAPPartition Possible (from P)Client 1Client 1
18. CAP Theorem [Gilbert Lynch 02]Assume to contradict that Algorithm A provides all of CAPPartition Possible (from P)Write eventually returns(from A)Client 1w(x=1)okClient 1
19. CAP Theorem [Gilbert Lynch 02]Assume to contradict that Algorithm A provides all of CAPPartition Possible (from P)Write eventually returns(from A)Client 1w(x=1)okClient 1r(x)x=0Read begins after write completesRead eventually returns (from A)
20. CAP Theorem [Gilbert Lynch 02]Assume to contradict that Algorithm A provides all of CAPPartition Possible (from P)Write eventually returns(from A)Client 1w(x=1)okClient 1r(x)x=0Read begins after write completesRead eventually returns (from A)Not consistent (C) => contradiction!
21. CAP Interpretation Part 1Cannot “choose” no partitions2-out-of-3 interpretation doesn’t make senseInstead, availability OR consistency?i.e., fundamental tradeoff between availability and consistencyWhen designing system must choose one or the other, both are not possible
22. CAP Interpretation Part 2It is a theorem, with a proof, that you understand!Cannot “beat” CAP TheoremCan engineer systems to make partitions extremely rare, however, and then just take the rare hit to availability (or consistency)
23. OutlineNetwork PartitionsLinearizabilityCAP TheoremConsistency Hierarchy
24. Consistency ModelsContract between a distributed system and the applications that run on itA consistency model is a set of guarantees made by the distributed systeme.g., LinearizabilityGuarantees a total order of operationsGuarantees the real-time ordering is respected
25. Stronger vs Weaker ConsistencyStronger consistency models+ Easier to write applications More guarantees for the system to ensureResults in performance tradeoffsWeaker consistency models- Harder to write applications+ Fewer guarantees for the system to ensure
26. Strictly Stronger ConsistencyA consistency model A is strictly stronger than B if it allows a strict subset of the behaviors of BGuarantees are strictly strongerLinearizability is strictly stronger than Sequential ConsistencyLinearizability: ∃total order + real-time orderingSequential: ∃total order + process orderingProcess ordering ⊆ Real-time ordering
27. Sequential But Not Linearizable
28. Consistency HierarchyLinearizabilitySequential ConsistencyCausal+ ConsistencyEventual Consistencye.g., RAFTe.g., Bayoue.g., Dynamo
29. Causal+ ConsistencyPartially orders all operations, does not totally order themDoes not look like a single machineGuaranteesFor each process, ∃ an order of all writes + that process’s readsOrder respects the happens-before () ordering of operations+ replicas converge to the same stateSkip details, makes it stronger than eventual consistency
30. Causal+ But Not Sequentialw(x=1)w(y=1)r(x)=0r(y)=0PAPBw(x=1)w(y=1)r(y)=0r(x)=0PA Order: w(x=1), r(y=0), w(y=1)Happens Before OrderProcessOrderingw(x=1)w(y=1)r(y)=0r(x)=0No Total Orderw(x=1)w(y=1)r(y)=0r(x)=0√ Casual+X SequentialPB Order: w(y=1), r(x=0), w(x=1)
31. Eventual But Not Causal+w(x=1)r(y)=1r(x)=0w(y)=1PAPBAs long as PB eventually would see r(x)=1 this is fineHappens BeforeOrderingw(x=1)r(y)=1w(y)=1r(x)=0No Order for PBw(x=1)r(y)=1w(y)=1r(x)=0√ EventualX Causal+
32. Consistency HierarchyLinearizabilitySequential ConsistencyCausal+ ConsistencyEventual Consistencye.g., RAFTe.g., Bayoue.g., DynamoCAP
33. Consistency HierarchyLinearizabilitySequential ConsistencyCausal+ ConsistencyEventual Consistencye.g., RAFTe.g., Bayoue.g., DynamoCAPPRAM 1988(Princeton)
34. PRAM [Lipton Sandberg 88] [Attiya Welch 94]d is the worst-case delay in the network over all pairs of processesSequentially consistent systemread time + write time ≥ dFundamental tradeoff between consistency and latency!
35. OutlineNetwork PartitionsLinearizabilityCAP TheoremConsistency Hierarchy
36.
37. Regrading Request OptionsMark these phrases on the front of your examYou must hand back your exam before you leaveFull Credit:4c,f)“||” okay for real-time – will be applied for all6e) “Close” to 10Partial Credit:10) “Swapped FLP”10) “FLP for X”11b,d) Correctly fixes real problem (not VM-FT) Maybe:4d,g) ”Alternate Universe w/ ½ mistakes and no points”7) “Explicit deletions – all scenarios”11) ”Ill-defined point for backup to start outputting”11) ”VM does not stop”