Introduction to Distributed Algorithms - PowerPoint Presentation

1. Introduction to Distributed Algorithms

2. How to represent a distributed algorithm?The notions of atomicity, non-determinism, and fairness are important issues in distributed algorithms, so we will need some notations and examples to clarify them.

3. Syntax & semantics: guarded actions <guard G>  <action A> [ is equivalent to if G then A (Borrowed from E.W. Dijkstra: A Discipline of Programming) 

4. Syntax & semantics: guarded actionsSequential actions S0; S1; S2; . . . ; SnAlternative constructs if . . . . . . . . . . fiRepetitive constructs do . . . . . . . . . od The specification is useful for representing abstract algorithms, not executable codes.

5. Syntax & semanticsAlternative construct if [] … [] fi Note 1. When no guard is true, skip (do nothing). Note 2. When multiple guards are true, the choice of the action to be executed is completely arbitrary. 

6. Syntax & semantics Repetitive construct do [] … [] od Keep executing the actions until all guards are false and the program terminates. When multiple guards are true, the choice of the action is arbitrary. 

7. Example: graph coloring0101 = color of process do neighbor od Use a central scheduler. Will the above computation terminate?There are four processes. Each process has a color 0, 1). The initial state is arbitrary. Here is a program that aims tocolor the processes so that no two neighboring processes have the same color and then terminate. p0p1p2p3

8. Consider another example program uncertain; define : integer; initially = 0 do [] od Question. Will the program terminate? (Our goal here is to understand fairness)  

9. The adversary A distributed computation can be viewed as a game between the system and an adversary. The adversary may come up with feasible schedules to challenge the system (and cause “bad things”). A correct algorithm must be able to prevent those bad things from happening.

10. Non-determinism(Program for a token server - it has a single token}repeat if (req1 ∧ token) then give the token to client1 else if (req2 ∧ token) then give the token to client2 else if (req3 ∧ token) then give the token to client3 foreverNow, assume that all three requests are sent simultaneously.The above program will never let Client 2 or 3 receive the token! The outcome could have been different if the server makes a non-deterministic choice.123Token serverDealing with non-determinism is not artificial, but natural. Why?

11. More examples of non-determinismDeterminism caters to a specific order and is a special case of non-determinism.If there are multiple processes ready to execute actions, then who will execute the action first is nondeterministic.Message propagation delays are arbitrary and the order of message reception is non-deterministic.

12. Atomicity (or granularity)Atomic = all or nothingAtomic actions = indivisible actionsdo red message {red action} [] blue message {blue action}odRegardless of how nondeterminism ishandled, we would expect that the value of x will be an arbitrary sequence of 0's and 7's.Right or wrong? x

13. Atomicity (continued)do red message {red action} [] blue message {blue action}odLet x be a 3-bit integer , so means and means If the assignment is not atomic and the value of x is updatedone bit at a time, then many interleavings are possible, leadingto any possible value of between 0 and 7 xSo, the answer depends on the atomicity of the assignmentX2=0 X2=1✔ X1=0 X1=1 ✔X0=0 X0=1 ✔X2=0 X2=1✔ X1=1 X1=0 ✔X0=0 X0=1 ✔X2=1 X2=0✔ X1=0X1=1 ✔ X0=0X0=1 ✔

14. Atomicity (continued) What happens to x and y here? ☛ Assume each guarded action G → A is an “atomic operation.” Question. Does it make a difference if it is not an atomic operation?Transactions are atomic by definition (in spite of process failures). Also, critical section codes are atomic.Does hardware guarantee any form of atomicity? Yes! (examples?)y=1x=0if fi if fi ⬆︎

15. Atomicity (continued){Program for P}define b: booleaninitially b = true do send msg m to Q [] empty(R,P) receive msg; b := false odSuppose it takes 15 seconds to send the message.After 5 seconds, P receives a message from R. Will it stop sending the remainder of themessage? NO. Atomic action cannot be interrupted PQRbit takes 15 seconds for P tosend the message. After 5 seconds,P receives a message from R.

16. Fairness Defines the choices or restrictions on the scheduling of actions. No such restriction implies an unfair scheduler. For fair schedulers, the following types of fairness have received attention:Unconditional fairnessWeak fairnessStrong fairnessScheduler / demon /adversary

17. FairnessProgram testdefine x : integer {initial value unknown}do [] [] od An unfair scheduler may never schedule the second (or the third actions). So, may always be equal to zero.An unconditionally fair scheduler will eventually give every statement a chance to execute without checking their eligibility. (Example: process scheduler in a multiprogrammed OS.) 

18. Weak FairnessProgram testdefine x : integer {initial value unknown}do [] [] od A scheduler is weakly fair, when it eventually executes every guarded action whose guard becomes true, and remains true thereafterA weakly fair scheduler will eventually execute the second action, but may never execute the third action. Why?

19. Strong fairnessProgram testdefine x : integer {initial value unknown}do [] [] od  A scheduler is strongly fair, when it eventually executes every guarded action whose guard is true infinitely often. The third statement will be executed under a strongly fair scheduler. Why?Study more examples to reinforce these concepts

20. Program correctnessThe State-transition modelGlobal state{ = local state of process k} Each state transition is caused by an action by an eligible process.stateInitial statetransitionWe reason using interleaving semantics, and assume that concurrent actions are serialized in an arbitrary orderA sample computation (or behavior) is ABGHIFL

21. Correctness criteriaSafety propertiesBad things never happenLiveness propertiesGood things eventually happen

22. Testing vs. ProofTesting: Apply inputs and observe if the outputs satisfy the specifications. Fool proof testing can be painfully slow, even for small systems. Most testing are partial.Proof: Has a mathematical foundation, and is a complete guarantee. Sometimes not scalable.

23. Testing vs. Proof To test this program, you have to test all possible interleavings. With n processes p0, p1, … pn-1, and m steps per process, the number of interleavings is How large is this even for moderate values of m and n ? p0p1p2p3step1step1step1step1step2step2step2step2step3step3step3step3

24. Example: Mutual Exclusion Process 0 Process 1 do do Entry protocol Entry protocol Critical section Critical section Exit protocol Exit protocol od od Safety properties (1) There is no deadlock (2) At most one process is in its critical section. Liveness property A process trying to enter the CS must eventually succeed. (This is also called the progress property) CSCS

25. Exerciseprogram mutex 1{two process mutual exclusion algorithm: shared memory model}define busy :shared boolean (initially busy = false} {process 0} {process 1} do do do busy skip od; do busy skip od; busy:= true; busy:= true; critical section; critical section busy := false; busy := false {remaining codes} {remaining codes} od od Does this mutual exclusion protocol satisfy liveness and safety properties? 

26. Safety invariantsInvariant means: something meaningful should always holdExample: Total no. of processes in CS ≤ 1 (mutual exclusion problem)Another safety property is Partial correctness. It implies that “If the program terminates then the postcondition will hold.” Consider the following:Safety invariant: postcondition It does not say if the program will terminate. (termination is a liveness property) Total correctness = partial correctness + termination.do od 

27. ExerciseStarting from the given initial state, devise an algorithm to color thenodes of the graph using the colors 0 and 1, so that notwo adjacent nodes have the same color.program colorme {for process Pi }define color Initially colors are arbitrarydo neighbor(i) : odIs the program partially correct? YES (why?)Does it terminate? NO (why?) 0110p1p2p3p0

28. Liveness properties Eventuality is tricky. There is no need to guarantee when the desired thing will happen, as long as it happens..Some examplesThe message will eventually reach the receiver.The process will eventually enter its critical section.The faulty process will be eventually be diagnosedFairness (if an action will eventually be scheduled)The program will eventually terminate.The criminal will eventually be caught.Absence of liveness cannot be determined from finite prefix of the computation

29. Proving safetydefine c1, c2 : channel; {init c1 = c2 = null} r, t : integer; {init r = 5, t = 5}{program for T} do t > 0 → send msg along c1; t := t -12 [] ¬empty (c2) → rcv msg from c2; t := t + 1 od {program for R} 3 do ¬empty (c1) → rcv msg from c1; r := r+14 [] r > 0 → send msg along c2; r := r-1 od We want to prove the safety property P:P ≡ n1 + n2 ≤ 10transmitterreceivern1= # of messages in c1n2= # of messages in c2tr

30. Proving safetyn1, n2 = # of messages in c1and c2 respectively.We will establish the following invariant:(). Check if I holds after every action.{program for T} do → send msg along 2 [] empty → receive msg from od {program for R} 3 do ¬empty (c1) → rcv msg from 4 [] r > 0 → send msg along od  Use the method of inductionShow that I initially holds, andholds after each action.r=1t=4

31. Proving livenessS1→ S2 → S3 → S4 ↓f ↓ f ↓ f ↓ fw1 w2 w3 w4w1, w2, w3, w4 ∈ WFWF is a well-founded set whose elements can be ordered by » and there is a smallest elementIf there is no infinite chain like w1 » w2 » w3 » w4 .., i.e. f(si) » f(si+1) » f(si+2) .. Global stateGlobal statethen the computation willdefinitely terminate!f is called a variant functionExample?

32. Proof of liveness: an example0n-1321Clock phase synchronizationSystem of n clocks ticking at the same rate.Each clock is 3-valued, i,e it ticks as 0, 1, 2, 0, 1, 2…A failure may arbitrarily alter the clock phases.The clocks need to return to the same phase. .

33. Proof of liveness: an exampleClock phase synchronization{Program for each clock}(= phase of clock , initially arbitrary)do → [] → odShow that eventually all clocks will return to the same phase (convergence), andcontinue to be in the same phase (closure) 0n-1321} 

34. Proof of convergenceLet if no arrow points towards clock ; if a ← points towards clock ; if a → points towards clock ; if both → and ← point towards clock v. By definition, D ≥ 0. Also, D decreases after every step in the system. So the number of arrows must reduce to 0.  020221110122222Understand the game of arrows012n-1