Causal Reliable Broadcast Ali Ghodsi – UC Berkeley
TF
Published · 35 slides · 0 views
1 / 1
Description
Causal Reliable Broadcast Ali Ghodsi UC Berkeley KTH alig(at)cs.berkeley.edu 31413 Ali Ghodsi, alig(at)cs.berkeley.edu 2 Motivation Assume we chat application Whatever written is reliably broadcast to group If you get the following
Related Topics
Share
Embed code
Download this presentation From Below
"Causal Reliable Broadcast Ali Ghodsi – UC Berkeley" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
01
Causal Reliable Broadcast Ali Ghodsi – UC Berkeley / KTH
alig(at)cs.berkeley.edu<br>
alig(at)cs.berkeley.edu<br>
02
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 2 Motivation Assume we chat application
Whatever written is reliably broadcast to group
If you get the following chat output, is it ok?
UserX’s message caused UserY’s message,
UserY’s message caused UserZ’s message [UserZ] Ok
[UserY] Can we push back 1h?
[UserX] Let’s meet at 2pm?<br>
Whatever written is reliably broadcast to group
If you get the following chat output, is it ok?
UserX’s message caused UserY’s message,
UserY’s message caused UserZ’s message [UserZ] Ok
[UserY] Can we push back 1h?
[UserX] Let’s meet at 2pm?<br>
03
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 3 Motivation (2) Does uniform reliable broadcast remedy this? [d]
Causal reliable broadcast solves this
Deliveries in causal order!
Causality is same as happened-before relation by Lamport!<br>
Causal reliable broadcast solves this
Deliveries in causal order!
Causality is same as happened-before relation by Lamport!<br>
04
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 4 Causality Recalled Let m1 and m2 be any two messages:
m1m2 (m1 causally precedes m2) if
C1 (FIFO order).
Some process pi broadcasts m1 before broadcasting m2
C2 (Network order).
Some process pi delivers m1 and later broadcasts m2
C3 (Transitivity).
There is a message m’ such that m1 m’ and m’ m2<br>
m1m2 (m1 causally precedes m2) if
C1 (FIFO order).
Some process pi broadcasts m1 before broadcasting m2
C2 (Network order).
Some process pi delivers m1 and later broadcasts m2
C3 (Transitivity).
There is a message m’ such that m1 m’ and m’ m2<br>
05
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 5 Causal Broadcast Interface Module:
Name: CausalOrder, instance co
Events
Request: co, Broadcast | m
Indication: co, Deliver | src, m
Property:
CB: If node pi delivers m1, then pi must have delivered every message causally preceding () m1 before m1
Is this useful? How can it be satisfied? [d]
It is only safety. Satisfy it by never delivering!<br>
Name: CausalOrder, instance co
Events
Request: co, Broadcast | m
Indication: co, Deliver | src, m
Property:
CB: If node pi delivers m1, then pi must have delivered every message causally preceding () m1 before m1
Is this useful? How can it be satisfied? [d]
It is only safety. Satisfy it by never delivering!<br>
06
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 6 Causality C1 (FIFO order).
Some process pi broadcasts m1 before broadcasting m2 p1 p2 p3 m1 m2 p1 p2 p3 m1 m2<br>
Some process pi broadcasts m1 before broadcasting m2 p1 p2 p3 m1 m2 p1 p2 p3 m1 m2<br>
07
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 7 Causality (2) C2 (Network order).
Some process pi delivers m1 and later broadcasts m2 p1 p2 p3 m1 m2 p1 p2 p3 m1 m2<br>
Some process pi delivers m1 and later broadcasts m2 p1 p2 p3 m1 m2 p1 p2 p3 m1 m2<br>
08
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 8 Causality (3) C3 (Transitivity).
There is a message m’ such that m1 m’ and m’ m2 p1 p2 p3 m1 m2 m3 p1 p2 p3 m1 m2 m3<br>
There is a message m’ such that m1 m’ and m’ m2 p1 p2 p3 m1 m2 m3 p1 p2 p3 m1 m2 m3<br>
09
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 9 Different Causalities Property:
CB: If node pi delivers m1, then pi must deliver every message causally preceding () m1 before m1
CB’: If pj delivers m1 and m2, and m1m2, then pj must deliver m1 before m2
What is the difference? [d]
Indeed, CB implies CB’ p1 p2 p3 m1 m2 m3 p1 p2 p3 m1 m2 m3 Violates CB and CB’ Violates CB, not CB’<br>
CB: If node pi delivers m1, then pi must deliver every message causally preceding () m1 before m1
CB’: If pj delivers m1 and m2, and m1m2, then pj must deliver m1 before m2
What is the difference? [d]
Indeed, CB implies CB’ p1 p2 p3 m1 m2 m3 p1 p2 p3 m1 m2 m3 Violates CB and CB’ Violates CB, not CB’<br>
10
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 10 Reliable Causal Broadcast Interface Module:
Name: ReliableCausalOrder, instance rco
Events
Request: rco, Broadcast | m
Indication: rco, Deliver | src, m
Property:
RB1-RB4 from regular reliable broadcast
CB: If node pi delivers m, then pi must deliver every message causally preceding () m before m<br>
Name: ReliableCausalOrder, instance rco
Events
Request: rco, Broadcast | m
Indication: rco, Deliver | src, m
Property:
RB1-RB4 from regular reliable broadcast
CB: If node pi delivers m, then pi must deliver every message causally preceding () m before m<br>
11
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 11 Uniform Reliable Causal Broadcast Module:
Name: UniformReliableCausalOrder, instance urco
Events
Request: urco, Broadcast | m
Indication: urco, Deliver | src, m
Property:
URB1-URB4 from uniform reliable broadcast
CB: If node pi delivers m, then pi must deliver every message causally preceding () m before m<br>
Name: UniformReliableCausalOrder, instance urco
Events
Request: urco, Broadcast | m
Indication: urco, Deliver | src, m
Property:
URB1-URB4 from uniform reliable broadcast
CB: If node pi delivers m, then pi must deliver every message causally preceding () m before m<br>
12
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 12 Reusing abstractions Reuse RB for CB
Use reliable broadcast abstraction to implement reliable causal broadcast
Use uniform reliable broadcast abstraction to implement uniform causal broadcast<br>
Use reliable broadcast abstraction to implement reliable causal broadcast
Use uniform reliable broadcast abstraction to implement uniform causal broadcast<br>
13
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 13 Towards an implementation Main idea
Each broadcasted message carries a history
Before delivery, ensure causality
First algorithm
History is set of all causally preceding messages<br>
Each broadcasted message carries a history
Before delivery, ensure causality
First algorithm
History is set of all causally preceding messages<br>
14
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 14 Fail-Silent No-Waiting Causal Bcast Each message m carries ordered list of causally preceding messages in pastm
Whenever a node rbDelivers m
coDeliver causally preceding messages in pastm
coDelivers m
Avoid duplicates using delivered<br>
Whenever a node rbDelivers m
coDeliver causally preceding messages in pastm
coDelivers m
Avoid duplicates using delivered<br>
15
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 15 p1 p2 p3 m1 coB(m1) coD(m1) coD(m1) coB(m2) m2 [m1] coD(m2) rbD(m2) m2 [m1] coD(m2) coD(m1) coD(m2) m1 Execution (direct override)<br>
16
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 16 p1 p2 p3 m1 coB(m1) coD(m1) coD(m1) coB(m2) m2 [m1] coD(m2) rbD(m2) m2 [m1] coD(m2) coD(m1) coD(m2) m1 Execution (indirect override)<br>
17
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 17 Fail-silent Causal Broadcast Impl Implements:
ReliableCausalOrderBroadcast instance rco.
Uses: ReliableBroadcast instance rb.
upon event rb, Init do
delivered := ; past := nil
upon event rco, Broadcast | m do
trigger rb, Broadcast | (DATA, past, m)
past := append(past, <pi, m>) Append this message to past history<br>
ReliableCausalOrderBroadcast instance rco.
Uses: ReliableBroadcast instance rb.
upon event rb, Init do
delivered := ; past := nil
upon event rco, Broadcast | m do
trigger rb, Broadcast | (DATA, past, m)
past := append(past, <pi, m>) Append this message to past history<br>
18
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 18 Fail-silent Causal Broadcast Impl (2) upon event rb,Deliver | pi,(DATA, pastm , m) do
if mdelivered then
forall (sn,n)pastm do
if ndelivered then
trigger rco,Deliver|sn, n
delivered := delivered{n}
past := append(past, <sn,n>)
trigger rco,Deliver|pi,m
delivered := delivered{m}
past := append(past, <pi,m>) in ascending order deliver preceding messages append to history deliver current message append to history<br>
if mdelivered then
forall (sn,n)pastm do
if ndelivered then
trigger rco,Deliver|sn, n
delivered := delivered{n}
past := append(past, <sn,n>)
trigger rco,Deliver|pi,m
delivered := delivered{m}
past := append(past, <pi,m>) in ascending order deliver preceding messages append to history deliver current message append to history<br>
19
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 19 Correctness RB1-RB4 follow from use of RB
No creation and no duplication still satisfied
Validity still satisfied
Some messages might be delivered earlier, never later
Agreement directly from RB
CO by induction on prefixes of executions
It is vacuously true for empty executions
Assume it is true for all deliveries of a prefix
Then it is true for any extension with one event<br>
No creation and no duplication still satisfied
Validity still satisfied
Some messages might be delivered earlier, never later
Agreement directly from RB
CO by induction on prefixes of executions
It is vacuously true for empty executions
Assume it is true for all deliveries of a prefix
Then it is true for any extension with one event<br>
20
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 20 Improving the algorithm Disadvantage of algorithm is that the message size (bit complexity) grows
Useful idea
Garbage collect old messages
Implementation of GC
Ack receipt of every message m to all
Use perfect failure detector P
Determine with P when all correct nodes got message m
Delete m from past when all correct nodes got m<br>
Useful idea
Garbage collect old messages
Implementation of GC
Ack receipt of every message m to all
Use perfect failure detector P
Determine with P when all correct nodes got message m
Delete m from past when all correct nodes got m<br>
21
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 21 GC implementation Uses: ReliableBroadcast instance rb, PerfectFD instance P
upon event rco, Init do
delivered := ; past := nil
correct :=
forall m: ack[m] :=
upon event P, crash | pi do
correct := correct \ {pi}
upon event mdelivered and selfack[m] do
ack := ack[m] U {m}
trigger rb, Broadcast | (ACK, m)
upon event rb, Deliver | pi, [ACK, m] do
ack := ack[m] U {pi}
if correctack[m] do
past := remove(past, <x, m>) book keeping of acks called upon coDeliver ack to all When received ack from all, GC m from any x<br>
upon event rco, Init do
delivered := ; past := nil
correct :=
forall m: ack[m] :=
upon event P, crash | pi do
correct := correct \ {pi}
upon event mdelivered and selfack[m] do
ack := ack[m] U {m}
trigger rb, Broadcast | (ACK, m)
upon event rb, Deliver | pi, [ACK, m] do
ack := ack[m] U {pi}
if correctack[m] do
past := remove(past, <x, m>) book keeping of acks called upon coDeliver ack to all When received ack from all, GC m from any x<br>
22
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 22 Towards another implementation Main idea
Each broadcasted message carries a history
Before delivery, ensure causality
First algorithm
History is set of all causally preceding messages
Second algorithm [d]
History is a vector timestamp<br>
Each broadcasted message carries a history
Before delivery, ensure causality
First algorithm
History is set of all causally preceding messages
Second algorithm [d]
History is a vector timestamp<br>
23
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 23 Fail-Silent Waiting Causal Broadcast Represent past history by vector clock (VC)
Slightly modify the VC implementation
At node pi
VC[i]: number of messages pi coBroadcasted
VC[j], ji: number of messages pi coDelivered from pj<br>
Slightly modify the VC implementation
At node pi
VC[i]: number of messages pi coBroadcasted
VC[j], ji: number of messages pi coDelivered from pj<br>
24
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 24 Fail-Silent Waiting Causal Broadcast Upon CO broadcast m
Piggyback VC and RB broadcast m
Upon RB delivery of m with attached VCm
compare VCm with local VCi
Only deliver m once VCm precedes VCi<br>
Piggyback VC and RB broadcast m
Upon RB delivery of m with attached VCm
compare VCm with local VCi
Only deliver m once VCm precedes VCi<br>
25
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 25 p1 p2 p3 b(m1) d(m1) b(m2) d(m2) d(m1) d(m2) d(m2) d(m1) (0,0,0) (1,0,0) m1(0,0,0) m1(0,0,0) (2,0,0) m2(1,0,0) m2(1,0,0) (1,0,0) (0,0,0) (0,0,0) (1,0,0) (2,0,0) (2,0,0) Execution hold m2<br>
26
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 26 Fail-Silent Waiting Causal Impl. Implements:
ReliableCausalOrderBroadcast, instance rco
Uses: ReliableBroadcast, instance rb
upon event rco, Init do
forall pi do VC[i] := 0
upon event rco, Broadcast|m do
trigger rb,Broadcast|(DATA, VC, m)
VC[self] := VC[self] + 1
trigger rco,Deliver|self, m send m with VC VC has only increased, so RCO deliver<br>
ReliableCausalOrderBroadcast, instance rco
Uses: ReliableBroadcast, instance rb
upon event rco, Init do
forall pi do VC[i] := 0
upon event rco, Broadcast|m do
trigger rb,Broadcast|(DATA, VC, m)
VC[self] := VC[self] + 1
trigger rco,Deliver|self, m send m with VC VC has only increased, so RCO deliver<br>
27
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 27 Fail-Silent Waiting Causal Impl. (2) upon event rb,Deliver|pj, (DATA, VCm , m) do
if pj ≠ self then
pending := pending (pj, (DATA, VCm, m))
deliver-pending()
procedure deliver-pending()
while exists x=(sm,(DATA,VCm,m))pending s.t. VCVCm do
pending := pending \ (sm, (DATA, VCm, m)
trigger rco,Deliver | sm, m
VC[ rank(sm) ] := VC[ rank(sm) ] + 1 put on hold Remove on hold deliver and increase local VC<br>
if pj ≠ self then
pending := pending (pj, (DATA, VCm, m))
deliver-pending()
procedure deliver-pending()
while exists x=(sm,(DATA,VCm,m))pending s.t. VCVCm do
pending := pending \ (sm, (DATA, VCm, m)
trigger rco,Deliver | sm, m
VC[ rank(sm) ] := VC[ rank(sm) ] + 1 put on hold Remove on hold deliver and increase local VC<br>
28
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 28 Delivery order isn’t same!
What is wrong? [d] p1 p2 b(m1) d(m1) d(m2) d(m2) d(m1) M1(0,0) m2(0,0) b(m2) Possible execution? (1,0) (1,1) (0,1) (1,1) (0,0) (0,0) Nothing, there is no causality.<br>
What is wrong? [d] p1 p2 b(m1) d(m1) d(m2) d(m2) d(m1) M1(0,0) m2(0,0) b(m2) Possible execution? (1,0) (1,1) (0,1) (1,1) (0,0) (0,0) Nothing, there is no causality.<br>
29
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 29 Other possible orderings Other common orderings
Single-source FIFO order
Total order
Causal order<br>
Single-source FIFO order
Total order
Causal order<br>
30
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 30 Single-Source FIFO order Intuitively
Msgs from same node delivered in order sent
For all messages m1 and m2 and all pi and pj,
if pi broadcasts m1 before m2, and if pj delivers m1 and m2, then pj delivers m1 before m2
Caveat
This formulation doesn’t require delivery of both messages<br>
Msgs from same node delivered in order sent
For all messages m1 and m2 and all pi and pj,
if pi broadcasts m1 before m2, and if pj delivers m1 and m2, then pj delivers m1 before m2
Caveat
This formulation doesn’t require delivery of both messages<br>
31
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 31 Total Order Intuitively
Everyone delivers everything in exact same order
For all messages m1 and m2 and all pi and pj,
if both pi and pj deliver both messages, then they deliver them in the same order
Caveat
This formulation doesn’t require delivery of both messages
Everyone delivers same order, maybe not send order!<br>
Everyone delivers everything in exact same order
For all messages m1 and m2 and all pi and pj,
if both pi and pj deliver both messages, then they deliver them in the same order
Caveat
This formulation doesn’t require delivery of both messages
Everyone delivers same order, maybe not send order!<br>
32
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 32 a Execution Example (1) b single-source FIFO? totally ordered? causally ordered? yes no yes<br>
33
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 33 Execution Example (2) a b single-source FIFO? totally ordered? causally ordered? no yes no<br>
34
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 34 Execution Example (3) a b single-source FIFO? totally ordered? causally ordered? yes no no<br>
35
3/14/13 Ali Ghodsi, alig(at)cs.berkeley.edu 35 Hierarchy of Orderings Stronger implies weaker ordering () best-effort reliable uniform reliable FIFO best-effort reliable FIFO uniform reliable FIFO causal best-effort reliable causal uniform reliable causal<br>