Declarative Programming over Eventually Consistent Data Stores - PowerPoint Presentation

1. Declarative Programming over Eventually Consistent Data StoresKC SivaramakrishnanGowtham Kaki Suresh Jagannathan

2. HttpHttpHttpHttpAppServerAppServerAppServerCacheCacheCacheCacheStatelessConsistency, Integrity, Durability, Availability, etc.

3. Account balances should be non-negativeUsernames should be uniqueOnly bona fide bids are accepted in an auction.Application invariantsStrong consistency Linearizability & Serializability

4. INTERNET☐ Strongly consistent, but not “always on”☐ Be “always on”, but no strong consistencyEventual Consistency∞ (convergence)

5. INTERNETBasic eventual Read-my-writesCausalMonotonic writesBounded stalenessRead committedParallel Snapshot IsolationStore Consistency LevelsEventually Consistent Data StoresRead-my-writes//init balance = 0deposit(100)?  get_balance()Session 1

6. INTERNETBasic eventual Read-my-writesCausalMonotonic writesBounded stalenessRead committedParallel Snapshot IsolationStore Consistency LevelsEventually Consistent Data StoresRead-my-writes//init balance = 0deposit(100)0  get_balance()Session 1bal=0Replica 2bal=100Replica 1Eventual Consistency//init balance = 0deposit(100)???  get_balance()Session 1bal=100Replica 1Read-my-writes consistencybal=0Replica 2

7. INTERNETBasic eventual Read-my-writesCausalMonotonic writesBounded stalenessRead committedParallel Snapshot IsolationStore Consistency LevelsEventually Consistent Data StoresRead-my-writes//init balance = 0deposit(100)0  get_balance()Session 1bal=0Replica 2bal=100Replica 1Eventual Consistency//init balance = 0deposit(100)100  get_balance()Session 1bal=100Replica 1Read-my-writes consistencybal=100Replica 2

8. INTERNETBasic eventual Read-my-writesCausalMonotonic writesBounded stalenessRead committedParallel Snapshot IsolationStore Consistency LevelsEventually Consistent Data Storesdeposit()withdraw()bid()tweet()Application invariants

9. INTERNETBasic eventual Read-my-writesCausalMonotonic writesBounded stalenessRead committedParallel Snapshot IsolationStore Consistency LevelsEventually Consistent Data Storesdeposit()withdraw()bid()tweet()Application invariantsCan we automate the process of mapping application requirements to store consistency levels?

10. Application requirementsStore consistency guarantees• Unique usernames.• Non-negative balance.• Bona fide bids.• Read-my-writes consistency• Causal consistency• Read committed isolation level• Repeatable read isolation levelMapSpecificationLanguageClassificationSchemeOur solution …• Sound.• OptimalA common medium to express both.An algorithm to …

11. Deposit(200)Withdraw(20)Withdraw(10) ……Replica 1Prelims - System Model……Deposit(200)Withdraw(10) ……Replica nSession 1v1 = getBalance(); ……v2 = getBalance();Session n……Replicated Data StoreSession Order (SO)getBalanceVisibility (Vis)getBalanceVis

12. 12Primitive relationsSpecification LanguageHappens-beforeAxiomatically capture set of valid executionsAssociate with each operation a single abstract effectExpress relationship between effectsVisibility (vis), Session order (so), Same object (sameobj)

13. Replicated Bank Account (1)balance >= 0 violatedSession 2//init balance = 100 withdraw(70);//init balance = 100 withdraw(70);Session 1abvisvis

14. Bank Account Contracts (2)Session 1 deposit(100)aSession 2 withdraw(50)bvisSession 3 -50  A.getbalance () getbalance ()cvisvis 50  getbalance ()

15. Capturing Store Consistency LevelsStrong Consistency CausalConsistency EventualConsistency

16. Decidable  Automatically discharged with the help of Z3 SMT solver.Classification Schemedeposit  ECwithdraw  SCgetBalance  CCStrong Consistency CausalConsistency EventualConsistency

17. Classification Scheme (2)

18. Txn 1 (current)oper1(…)aTxn 2 (committed) oper2(…)bvis oper2(…)oper3(…)cvisRead CommittedTransactionsGeneralizing to transactions is easy.Add single primitive relation - sametxn(a,b)Derived relation: Isolation guarantees of stores can now be specified.

19. Monotonic atomic viewRepeatable ReadRead Committed

20. Haskell library for Eventually Consistent Data Stores (ECDS)Definition language  define operations and transactions on replicated data.Specification language  specify consistency and isolation requirements.+DEFSQuelea Data StoreGHC

21. ApplicationRUBiSTwitter-lite#Tables65#Operations1720#Transactions610Invariants e.g. See all bids placed in current sessionUnique username Results of classification#EC Ops1413#CC Ops26#SC Ops11#RC Txns46#MAV Txns23#RR Txns01Case Studies

22. EvaluationCorrectness with classification vs without classificationHow do they compare in terms of availability?Experimental Setup:Amazon EC2; 5 replicas (StrongRep & Quelea); 1 replica (NoRep)Gradually increased # of concurrent clients from 128 to 1024.NoRep: No ReplicationStrongRep: Strong Replication

23. ConclusionQuelea  Haskell-library for programming ECDSAutomatic classification of operation and transaction contracts through SMT solverLeveraging off-the-shelf ECDSAvoid re-engineering complex systemsMakes it practical!http://kcsrk.info/Quelea

24. Thank you!

25. Summarization is essential to check the unbounded growth of the log.How is summarization done?Ask developer for summarization semantics. Replace (many) original effects with (few) summary effects.State Summarization

26.

27.

28.

29. … like “monotonic reads” (roughly requiring that time doesn’t appear to go backward) .… So, if we want to build an available system providing the monotonic reads session guarantee, we can ensure that read operations only return writes when the writes are present on all servers. Monotonic Reads

30. Read Committed (Atomicity)Txn 1 (current)X.oper1(…)aTxn 2 (committed) X.oper2(…)bvis X.oper2(…)X.oper3(…)cvis

31. Repeatable ReadAtomicity + Repeatable read should return the same value snapshotTxn 1 (current)X.oper1(…)aTxn 2 (committed) X.oper2(…)bvis X.oper2(…)Y.oper4(…)cvisd X.oper1(…)Y.oper3(…)a = d(Atomicity)RRRR =

32. In the paperStronger eventual consistencyHighly available transaction supportSummarization

33. System ModelA  {d10,w2}B  {d9}……Quelea Data StoreA  {d10,w2}B  {d9}A  {d10,w2}B  {d9}R1R2RnSession 1B.deposit($5)B.withdraw($6)Session 2Session n……Session Orderw6d5AliceBobEffects

34. System ModelA  {d10,w2}B  {d9}Quelea Data StoreA  {d10,w2}B  {d9}A  {d10,w2}B  {d9}R1R2RnSession 1B.deposit($5) B.withdraw($6)Session 2Session n……Session Orderw6d5AliceBob……

35. System ModelA  {d10,w2}B  {d9,d5}……Quelea Data StoreA  {d10,w2}B  {d9}A  {d10,w2}B  {d9,w6}R1R2RnSession 1B.deposit($5) B.withdraw($6)Session 2Session n……Session Orderw6w6d5AliceBobd5

36. System ModelA  {d10,w2}B  {d3,d5,w6}……Quelea Data StoreA  {d10,w2}B  {d3,d5,w6}A  {d10,w2}B  {d3,d5,w6}R1R2RnSession 1B.deposit($5) B.withdraw($6)Session 2Session n……Session Orderw6d5AliceBob

37. System ModelA  {d10,w2}B  {d3,d5,w6}Quelea Data StoreR1Session 1B.deposit($5) B.withdraw($6)v1 = B.getBalance()w6d5AliceBob…………gbgbvis

38. System ModelA  {d10,w2}B  {d3,d5,w6}Quelea Data StoreR1Session 1B.deposit($5) B.withdraw($6)v1 = $3+$5–$6 = $2w6d5AliceBob…………gbgbvis

39. Deposit(200)Withdraw(20)Withdraw(10) ……Replica 1System Model……Deposit(200)Withdraw(10)Deposit(10) ……Replica nSession 1getBalance; ……withdraw(6);Session n……Replicated Data StoreSession Order (SO)

40. Replicated Bank Account (2)Session 1 deposit(100)aSession 2 withdraw(50)bvisSession 3 -50  getbalance () getbalance ()cvisvis 50  getbalance ()

41. Evaluation

42. Replicated Bank Account (1)bal = A.getBalance();If (bal ≥ 70) A.withdraw(70);Session 1Session 2 A.withdraw(70);

43. Replicated Bank Account (1)100 = A.getBalance();If (100 ≥ 70) A.withdraw(70);Session 1bSOSession 1Session 2abSession 2 A.withdraw(70);cca

44. Replicated Bank Account (1)Session 2 A.withdraw(70);100 = A.getBalance();If (100 ≥ 70) A.withdraw(70);Session 1bSOSession 1Session 2abaccVIS

45. Replicated Bank Account (1)Required invariant: balance >= 0SOSession 1Session 2abcVIS

46. Replicated Bank Account (1)Session 2//init balance = 100 withdraw(70);//init balance = 100 withdraw(70);Session 1

47. TransactionsAllows composition of operations.Serializable transactions are unavailableHighly available transactions (HAT)Atomic, but relaxed Isolation. Isolation levels: read committed, repeatable read, monotonic atomic view, etc.Express foreign key constraints, secondary indexes etc. Choosing the correct isolation guarantee is an error-prone processAutomate it through specifications and classification!sametxn(a,b)

48. Operations (17)Transactions (6)StockItemnewItemRemoveItemFromStockOpenAuctionAddBidConcludeAuction……An “e-Bay”-like auction siteCase Study - RUBiSApplication Invariants:Canceling a bid must not violate data integrityA bidder must see all bids placed in the current session…

49. INTERNET☐ Strongly consistent, but not “always on”☐ Be “always on”, but no strong consistencyEventual Consistency∞ (convergence)VIDEO_IDCOUNT……VIDEO_IDCOUNT……VIDEO_IDCOUNT……VIDEO_IDCOUNT……