Paxos COS 418 Distributed Systems Lecture 10 Wyatt Lloyd Let different replicas assume role of primary over time System moves through a sequence of views How do the nodes agree on view primary ID: 1001027
Download Presentation The PPT/PDF document "Consensus, FLP, and" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Consensus, FLP, and PaxosCOS 418: Distributed SystemsLecture 10Wyatt Lloyd
2. Let different replicas assume role of primary over timeSystem moves through a sequence of viewsHow do the nodes agree on view / primary?2Recall the use of ViewsPPPView #1View #2View #3
3. ConsensusDefinition:A general agreement about something An idea or opinion that is shared by all the people in a group
4. ConsensusGiven a set of processors, each with an initial value:Termination: All non-faulty processes eventually decide on a valueAgreement: All processes that decide do so on the same value Validity: The value that has been decided must have proposed by some process
5. Group of servers attempting:Make sure all servers in group receive the same updates in the same order as each other Maintain own lists (views) on who is a current member of the group, and update lists when somebody leaves/fails Elect a leader in group, and inform everybodyEnsure mutually exclusive (one process at a time only) access to a critical resource like a file5Consensus Used in Systems
6. Can We Achieve Consensus?
7. Network model:Synchronous (time-bounded delay) or asynchronous (arbitrary delay)Reliable or unreliable communicationNode failures:Crash (correct/dead) or Byzantine (arbitrary) (Left options indicate an “easier” setting.)7Defining Our System Model
8. Network model:Synchronous (time-bounded delay) or asynchronous (arbitrary delay)Reliable or unreliable communicationNode failures:Crash (correct/dead) or Byzantine (arbitrary) (Left options indicate an “easier” setting.)8Defining Our System Model
9. Consensus is Impossible
10. No deterministic 1-crash-robust consensus algorithm exists with asynchronous communication10“FLP” Result
11. 11FLP’s Weak AssumptionsOnly 1 failure Also impossible for more failuresFor “weak” consensus (only some process needs to decide, not all) Also impossible for real consensusFor reliable communication Also impossible for unreliable communicationFor only two states: 0 and 1 Also impossible for more failuresFor crash failures Also impossible for Byzantine failures
12. FLP’s Strong AssumptionsDeterministic actions at each nodeAsynchronous network communicationAll “runs” must eventually achieve consensus
13. Initial state of system can end in decision “0” or “1”Consider 5 processes, each in some initial state[ 1,1,1,1,1 ] → 1 [ 1,1,1,1,0 ] → ? [ 1,1,1,0,0 ] → ? [ 1,1,0,0,0 ] → ? [ 1,0,0,0,0 ] → 0 13Main Technical ApproachMust exist two configurations here which differ in decision
14. Initial state of system can end in decision “0” or “1”Consider 5 processes, each in some initial state[ 1,1,1,1,1 ] → 1 [ 1,1,1,1,0 ] → 1 [ 1,1,1,0,0 ] → 1[ 1,1,0,0,0 ] → 0 [ 1,0,0,0,0 ] → 0 14Main Technical ApproachAssume decision differs between these two processes
15. Goal: Consensus holds in face of 1 failure [ 1,1,0,0,0 ] → [ 1,1,1,0,0 ] → 15Main Technical ApproachOne of these configurations must be “bi-valent”Both futures possible1 | 00
16. Goal: Consensus holds in face of 1 failure [ 1,1,0,0,0 ] → [ 1,1,1,0,0 ] → Inherent non-determinism from asynchronous networkKey result: All bi-valent states can remain in bi-valent states after performing some work 16Main Technical ApproachOne of these configurations must be “bi-valent”Both futures possible1 | 00
17. Staying Bi-Valent ForeverSystem thinks process p failed, adapts to it…But no, p was merely slow, not failed…(Can’t tell the difference between slow and failed.)System think process q failed, adapts to it…But no, q was merely slow, not failed…Repeat ad infinitum …
18. Consensus is ImpossibleBut, we achieve consensus all the time…
19. FLP’s Strong AssumptionsDeterministic actions at each nodeRandomized algorithms can achieve consensusAsynchronous network communicationSynchronous or even partial synchrony is sufficientAll “runs” must eventually achieve consensusIn practice, many “runs” achieve consensus quicklyIn practice, “runs” that never achieve consensus happen vanishingly rarelyBoth are true with good system designs
20. Consensus is PossibleWith Paxos!
21. ConsensusGiven a set of processors, each with an initial value:Termination: All non-faulty processes eventually decide on a value Good thing that eventually should happenAgreement: All processes that decide do so on the same value Bad thing that should never happenValidity: The value that has been decided must have proposed by some process Bad thing that should never happen
22. Safety vs. Liveness PropertiesSafety (bad things never happen)Liveness (good things eventually happen)
23. PaxosSafetyOnly a single value is chosenOnly chosen values are learned by processes Only a proposed value can be chosenLivenessSome proposed value eventually chosen if fewer than half of processes failIf value is chosen, a process eventually learns itagreementvaliditytermination
24. Paxos’s Safety and LivenessPaxos is always safePaxos is very often liveBut not always live
25. Roles of a ProcessThree conceptual rolesProposers propose valuesAcceptors accept values, where a value is chosen if a majority acceptLearners learn the outcome (chosen value)In reality, a process can play any/all roles25
26. Strawmen3 proposers, 1 acceptorAcceptor accepts first value receivedNo liveness with single failure3 proposers, 3 acceptorsAccept first value received, acceptors choose common value known by majorityBut no such majority is guaranteed26
27. PaxosEach acceptor accepts multiple proposalsHopefully one of multiple accepted proposals will have a majority vote (and we determine that)If not, rinse and repeat (more on this)How do we select among multiple proposals?Ordering: proposal is tuple (proposal #, value) = (n, v)Proposal # strictly increasing, globally uniqueGlobally unique?Trick: set low-order bits to proposer’s ID27
28. Paxos Protocol OverviewProposers:Choose a proposal number nAsk acceptors if any accepted proposals with na < nIf existing proposal va returned, propose same value (n, va)Otherwise, propose own value (n, v)Note altruism: goal is to reach consensus, not “win”Accepters try to accept value with highest proposal nLearners are passive and wait for the outcome28
29. Paxos Phase 1Proposer:Choose proposal number n, send <prepare, n> to acceptorsAcceptors:If n > nhnh = n ← promise not to accept any new proposals n’ < nIf no prior proposal acceptedReply < promise, n, Ø >Else Reply < promise, n, (na , va) >ElseReply < prepare-failed >29
30. Paxos Phase 2Proposer:If receive promise from majority of acceptors, Determine va returned with highest na, if existsSend <accept, (n, va || v)> to acceptorsAcceptors:Upon receiving (n, v), if n ≥ nh,Accept proposal and notify learner(s)na = nh = nva = v30
31. Paxos Phase 3Learners need to know which value chosenApproach #1Each acceptor notifies all learnersMore expensiveApproach #2Elect a “distinguished learner”Acceptors notify elected learner, which informs othersFailure-prone31
32. 32Paxos: Well-behaved Run<accepted, (1 ,v1)>12n...112n...<prepare, 1>1<promise, 1>12n...<accept, (1,v1)>decide v1
33. Intuition: if proposal with value v chosen, then every higher-numbered proposal issued by any proposer has value v.33Paxos is SafeMajority of acceptors accept (n, v): v is chosenNext prepare request with proposal n+1
34. Race Condition Leads to Liveness Problem“FLP Scenario”Completes phase 1 with proposal n034Starts and completes phase 1 with proposal n1 > n0Performs phase 2, acceptors rejectRestarts and completes phase 1 with proposal n2 > n1Process 0Process 1Performs phase 2, acceptors reject… can go on indefinitely …
35. Paxos SummaryDescribed for a single round of consensusProposer, Acceptors, LearnersOften implemented with nodes playing all rolesAlways safeQuorum intersectionOften live“FLP Scenario” prevents it from always being liveAcceptors accept multiple valuesBut only one value is ultimately chosenOnce a value is accepted by a majority it is chosen
36. Flavors of PaxosTerminology is a messPaxos loosely, and confusingly defined…We’ll stick withBasic PaxosMulti-Paxos
37. Flavors of Paxos: Basic PaxosRun the full protocol each timee.g., for each slot in the command logTakes 2 rounds until a value is chosen“FLP Scenario” is dueling proposers
38. Flavors of Paxos: Multi-PaxosElect a leader and have them run the 2nd phase directlye.g., for each slot in the command logLeader election uses Basic PaxosTakes 1 round until a value is chosenFaster than Basic Paxos“FLP Scenario” is dueling proposers during leader electionRarer than Basic PaxosUsed extensively in practice!
39. Consensus TakeawaysConsensus: Terminating agreement on a valid proposalConsensus is impossible to always achieveFLP resultConsensus is possible to achieve in practiceWith Multi-PaxosMostly happens in a single round to the nearest quorumSometimes takes a single round to a further quorumRarely takes multiple rounds to elect a new leader and for that node to get the request acceptedRuns exist where no new leader is ever elected
40.