Reactors: A Case for Predictable, Virtualized - PowerPoint Presentation

Slide1

Reactors: A Case for Predictable, Virtualized Actor Database Systems

Vivek Shah , Marcos Antonio Vaz Salles

University of Copenhagen (DMS@DIKU)Slide2

Increasing NEW oltp application diversity

2Slide3

NEW OLTP APPLICATION TRENDS

Increasing complexity of application logic3

Latency is critical

Scalability mattersSlide4

optimizing for latency in new oltp applications

4

Client

Application Server

Database serverSlide5

NEW OLTP DATABASE TO THE RESCUE

5Modern hardware is fast evolvingNew OLTP databases are being re-designed to harness their performanceIs the programming interface of stored procedures good enough ?Slide6

When are stored procedures not good enough?

Software Engineering Challenges6Performance ChallengesData partitioning not enoughLeverage locality & intra-procedure parallelismNeed low-level control over deploymentRequire modularity and isolation

Need abstractions to reason about performanceSlide7

Actor programming models are desirable

7

Modularity

Asynchronicity

ConcurrencySlide8

Actor relational databases

Relational DatabasesActor Runtimes

8

??Slide9

How can we integrate actor programming models in modern relational databases with high performance?

9Slide10

Talk outline

10Motivation / Problem Statement2. Relational Actor Programming Model (Reactor)3. Relational Actor Database System (ReactDB)4. EvaluationSlide11

Relational actor programming model (REACTOR)

11

2.Slide12

Simplified bitcoin exchange application

12P_EXPOSUREG_RISK5000000023400000NAMERISKTIMEWINDOWCBIT_DK234156918-11-1711:45:6710XBIT_US590986318-11-1711:43:3430

PROVIDER

WALLET

VALUE

SETTLED

CBIT_DK43450NXBIT_US42

1000

Y

XBIT

_US

85

356.23

N

settlement_risk

providers

orders

Providers

WalletsSlide13

What is a Reactor?

13A reactor is an application-defined actor encapsulating state as relationsConcurrent (Single-threaded)IsolatedReactor TypeSchema + MethodsReactor InstanceType + Unique name assigned by the programmerSlide14

reactor

Exchange { Relation settlement_risk { p_exposure float, g_risk float }; Relation provider_names { value varchar(32) };}Examples of reactor types and instances14 void auth_pay(pprovider, pwallet, pvalue) { ... }

reactor

Provider {

Relation

orders {

wallet

int

,

value

float

,

settled

char

(1)

}

Relation

provider_info {

risk

float

,

time

timestamp

,

window

interval

};

}

float

sim_risk(exposure) { … } float calc_risk(p_exposure){ … } void add_entry(wallet, value){ … }Name: OneExName: XBIT_USName: CBIT_DKSlide15

Declarative queries within a reactor

Declarative queries to access isolated relational state within a reactor

Relational

Key/Value

Hybrid

Require modularity and isolationSlide16

Declarative queries within a reactor

16reactor Provider { Relation orders { wallet int, value float, settled char(1) }; Relation provider_info { risk float, time timestamp, window interval }; float calc_risk(p_exposure){ } void add_entry(wallet, value){ } }

INSERT INTO orders VALUES (wallet, value, ‘N’);

SELECT SUM(value) INTO exposure FROM orders WHERE settled = 'N';

if

exposure > p_exposure

then abort

;

SELECT risk, time, window INTO p_risk, p_time, p_window FROM provider_info;

if

p_time < now() - p_window

then

p_risk := sim_risk(exposure);

UPDATE provider_info SET risk = p_risk, time = now();

end if

;

return

p_risk;

Exposure of a provider within threshold, risk simulatedSlide17

Communication with reactors

17Communication across reactors using asynchronous function calls by specifying reactor namesAsync method invocationFuture resultInvocation:-fut_res := fn(params) on reactor_name;Synchronization:-fut_res.get();Need abstractions to reason about performance

Leverage locality & intra-procedure parallelism

Require modularity and isolationSlide18

reactor

Exchange { Relation settlement_risk { p_exposure float, g_risk float }; Relation provider_names { value varchar(32) }; void auth_pay(pprovider, pwallet, pvalue) { } }Communication with reactors18

SELECT

g_risk, p_exposure

INTO

risk,exposure

FROM

settlement_risk;

results := [];

foreach

p_provider

in

(

SELECT

value

FROM

provider_names) {

res := calc_risk(exposure) on reactor p_provider;

results.add(res);

}

total_risk := 0;

foreach

res in results

total_risk := total_risk +

res.get();

if

total_risk + pvalue < risk

then add_entry(pwallet, pvalue) on reactor pprovider; else abort; end if;Total simulated risk across providers within thresholdSlide19

Automatic synchronization with reactors

19ABCFEDGClientMethod invocation on the same reactor are synchronous despite the asynchronous communication model.Parent method completes only when its children methods finishSlide20

Reactors provide transactional guarantees

20All-or-nothing atomicityDurability Serializability guaranteesUnsafe program executions are abortedFormalized equivalence to classical transactional modelABBACAB

DSlide21

In-memory actor relational database system (ReactDB)

21

3.Slide22

In-memory database architecture design problem

22…..

Reactors

Multi-core machine with large main memory

How do we map a set of reactors to the compute and memory resources of a large multi-core machine?Slide23

Delegate to operating system abstractions

23Each reactor is a process vs each reactor is a threadOverheadsLack of scheduling flexibility and controlSlide24

ReactDB architecture building blocks

24Transaction ExecutorsAbstracts a physical coreContainersAbstracts a set of physical cores and the memory shared by themSlide25

25

Map transaction executors to containers (many to one)Map reactors to transaction executors(many to many) such that a reactor belongs to one container only

How do we map a set of reactors to the compute and memory resources of a large multi-core machine?

Specified manually at deployment time

Need low-level control over deploymentSlide26

26

TransactionExecutorRequestQueueTransactionExecutorRequestQueueTransaction CoordinatorTransport DriverTransaction RouterContainerContainerContainerContainer

Multi-core machine

ReactDB

ArchitectureSlide27

ReactDB deployments

27TransactionExecutor(A,B)ContainerSimple RouterTransactionExecutor(C,D)Simple RouterContainershared-nothingSlide28

ReactDB deployments

28TransactionExecutor(A,B,C,D)Load Balancing RouterTransactionExecutor(A,B,C,D)TransactionExecutor(A,B)Affinity based RouterTransactionExecutor(C,D)ContainerContainer

shared-everythingshared-everything

-with-affinity

-without-affinity

Hybrid deployments possible, not explored in this workSlide29

ReactDB implementation overview

29Thread Management in Transaction ExecutorsThreadpool uses cooperative multitaskingMinimize context switchesConfigurable multi-programming levelMaximize resource utilizationSlide30

ReactDB implementation overview

30Concurrency ControlSingle ContainerOCC protocol (Silo, [Tu et al. 2013])Synchronous execution to leverage shared memoryMulti-ContainerOCC + 2PC protocolAsynchronous execution across containersStorage LayerRelations of all reactors in a container stored as primary indices (Masstree [Mao et al. 2012]) in memoryDurability not implemented yetSlide31

Evaluation of Reactors and ReactDB

31

4.Slide32

Can we really leverage asynchronicity for performance gains in new oltp using reactors and ReactDB ?

32Slide33

33

Workload : Bitcoin exchange auth_pay programDeploymentsshared-everything-with-affinity container -> sequential shared-nothing containersParallel exposure calculation followed by sequential risk calculation -> query-parallelismFully parallel exposure and risk calculation (as shown in example code before) -> procedure-parallelismExperimental setupSlide34

34

Experimental setupMachine with 2X AMD Opteron 6274 with 8 cores@2.1 GHz, L1i 64 KB, L2 2MB, L3 shared 6 MB, 125 GB RAM, 64 bit Linux 4.1.151x Exchange reactor, 15X Provider reactors (30,000 orders per of which 800 latest orders used for exposure calculation)Varying complexity of sim_risk by random number generationSingle worker to generate the workload Slide35

35

Asynchronicity gains manifest with increasing parallelizable application logicSlide36

What about the effect of load on gains from asynchronicity ?

Can db architecture virtualization help ? 36Slide37

37

Machine SetupMachine with 2X AMD Opteron 6274 with 8 cores@2.1 GHz, L1i 64 KB, L2 2 MB, L3 shared 6MB, 125 GB RAM, 64 bit Linux 4.1.15Workload100% TPC-C new-order augmented with artificial delay of 300 - 400 𝜇sec to simulate stock replenishment analytics per warehouseImplemented by modeling a warehouse as a reactorDeploymentshared-nothing vs shared-everything-with-affinityScale factor of 8, varying workers to simulate load on the databaseExperimental setupSlide38

38

Asynchronicity gains diminish with increasing loadasynchronicity gains offset by queuingshared-nothing deployment best at low loadcross over happens much earlier without delay @2 workersSlide39

More in the paper

39Slide40

Conclusion

40NEW OLTP + modern databases -> Needs rethink of the existing stored procedure programming modelReactor programming model marries actor programming construct with relational data model and queryingReactDB leverages the Reactor programming model and a containerization mechanism to allow control of database architecture at deployment timePoster session tomorrow at 16:00thanks!Slide41

thanks!

Any questions?You can find me athttp://www.diku.dk/~bonii@bonivivekbonii@di.ku.dk

41