Failing gracefully With the actor model Roger Johansson AkkaNET and ProtoActor Founder Solution Architect Betsson Group Twitter rogeralsing Github rogeralsing Mail rogeralsinggmailcom Githubcomrogeralsingpresentation ID: 765640
Download Presentation The PPT/PDF document "Failing gracefully With the actor model" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Failing gracefullyWith the actor model Roger JohanssonAkka.NET and Proto.Actor FounderSolution Architect - Betsson GroupTwitter: @rogeralsingGithub: rogeralsingMail: rogeralsing@gmail.comGithub.com/rogeralsing/presentationshttp://Proto.Actor
Design for failureDealing with, mitigating and preventing failures.In the small, and at scale.
An escalator can never break: it can only become stairs. You should never see an Escalator Temporarily Out Of Order sign, just Escalator Temporarily Stairs. Sorry for the convenience. -Mitch Hedberg
Bulkheads
Actors
Actor Model Three axioms:Send – an actor can send messages to other actorsCreate – an actor can create new actorsBecome – an actor can decide how to handle it’s next message
”An island of consistency in a sea of concurrency”” Shared nothing”, ”Black box””Location transparent”, ”Distributable by design”Actor Model
Actor Model Implementations:BEAM – Erlang, Elixir, LFEJVM – Akka, Orbit.NET – Akka.NET, Proto.Actor, Microsoft OrleansGo – Proto.Actor
Event-driven thread PIDActorStateSupervision Children Mailbox Behavior Transport
/c2 /b2 /b1 /a1 /a2 /b3 /c3 /c4 /c5 / / c1 /b1 Supervision
ComponentConsumer 2 Consumer 1
Data AccessWebsite Businesslogic
Vendor machine Insert coins Out of beansService guyRefill beansGets coffeeMe Need more coins Out of beans !
Service Request Application errorSupervisorManage failuresResponseValidation errorClient
/c2 /b2 /b1 /a1 /a2 /b3 /c3 /c4 /c5 / Error Kernel / c1 OneForOne supervisor AllForOne supervisor /b1 /b3 99.9999999% uptime ~0.03 sekunder downtime per år
Let it crash child parentProtect state by delegating workPerforms work, can be restarted
Mitigating Failures
Exponential Backoff child parent Attempts Delay
DB Actor DB Query DB Result DB Up DB Query DB Error DB Down Circuit Breaker
Idle ActiveGated QuarantinedAkka Remote Associations
Active Passive Consumers Node 2 Node 1PrimaryQueueSecondary
Consistent Hash Pairs Queue Hash Slot 1 Hash Slot 2 Hash Slot 3
Fan out and quickest reply ? ??? Router Worker 1 Worker 2 Worker 3 Fan out Consumer
Fan out and quickest reply ! !!! Router Worker 1 Worker 2 Worker 3 Return first reply Consumer
N-Modular Redundancy ! !!! Router Worker 2 Worker 3 Require same reply from at least N workers Consumer Worker 1
Dealing with load
Throttling Add a delay after each message
Throttling Limit messages per time slice
Bounded Mailboxes Actor Mailbox
Actor Bounded Mailboxes Mailbox
Limit Concurrency 1 212 3 3 4 4 Router Worker 1 Worker 2 Worker 3 Limit concurrent access to a resource
Work Pulling Worker 1 ProducerWork to be doneSend work Worker 2 Work to be done Send work Work Done Work Done
Virtual ActorsAlways Exists, Never Fails
Virtual Actors
Node A 433 Node B24345Node C82327Hash:989123Node E943772Node D612344 Node F 987234 Grain Id: Roger Names: Roger Names: Roger Actors: Roger : User Virtual Actors
Coordinator Akka Cluster Sharding
The End Roger Johansson Twitter: @rogeralsingGithub: rogeralsingMail: rogeralsing@gmail.comGithub.com/rogeralsing/presentationshttp://Proto.Actor