Simba - PowerPoint Presentation

giovanna-bartolotta . @giovanna-bartolotta

400 views
Uploaded On 2016-03-23

Simba - PPT Presentation

Tunable EndtoEnd Data Consistency For Mobile Apps Dorian Perkins Nitin Agrawal Akshat Aranya Curtis Yu Younghwan Go Harsha V Madhyastha ID: 266884

sync simba store data simba sync data store app object tables client apps photo table cloud consistency gateway album

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/266884" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download Presentation The PPT/PDF document "Simba" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

Simba: Tunable End-to-End Data Consistency For Mobile Apps

Dorian Perkins*†, Nitin Agrawal†, Akshat Aranya†, Curtis Yu*, Younghwan Go^†, Harsha V. Madhyastha‡, and Cristian Ungureanu††NEC Labs America, *UC Riverside, ^KAIST, and ‡U MichiganSlide2

Data is king:

apps must be

good at handling dataSlide3

Store and view photos locally

Store album info in SQL tables, photos in local FSApp logic to access both the tables and the FS

Row 1

Obj

hoto info

photo

App components

Activities and GUI

CursorAdapter

Local database I/O

Local file I/O

Unify data model

(

part of app logic)

REST methods (JSON, HTTP)

N/W management

Push notifications

Data caching

Conflict detection

Conflict resolution

Atomic syncing

Provide sync with other users/devices

Design network interaction with a cloud service

May want to notify users of new photos

May cache shared photos to avoid extra transfers

Ensure consistent album info and photo sync

Another user changes the photo tag? conflict

Album gets synced, but no photos? requires atomicity

Writing an App: Photo Sharing

3Slide4

Service components

Mirror functionality for app components +Scalable data storeClient connection managementFailure recoveryNotification engineLoad balancingVersion assignmentConflict handling

Too much for the developer?

Need infrastructure services for managing data

Running a Cloud Service

4Slide5

Writing data-centric apps is onerous!

Partitioned data modelSilo objects & tables; apps forced to manageRequires atomicity across inter-dependent dataEnd-to-end consistencyFailures, disconnections, bandwidthDiverse requirementsDisconnected progress Support offline operationsRequires conflict detection / resolution5Slide6

objects

tables

Emergence of Sync Services

Table

-only or object-only

sync

No atomicity across

object and table data

Single consistency

schemeSlide7

Motivation

StudySimba Evaluation7Slide8

Study of Popular Mobile Apps

How do popular mobile apps behave?Roll-own or sync service (e.g., Dropbox, Parse, Kinvey)Consistency: end-to-end observed by the userConcurrent edits on two devices, disconnectionsStudied various different consistency semanticsDisconnected operationConcurrent + devices onlineOne/both device offline

Client 1

read

Client 2

read

Client 1

update

Client 2

update

8Slide9

App

for sharing notes, embedded images, videosClaims “no dangling pointers” [Evernote Techblog]Guess what we found: dangling pointers!Disconnect during sync: atomicity violation“Dangling pointers” and “half-formed” notes visibleMultimedia notes  only icons appear on client #2Example: Evernote (roll own)9Slide10

App

for shared grocery listsSync semantics oblivious to consistencyLast-writer-wins prevalent; inadequate for collaborationLeads to corruption & resurrection of deleted dataInadequate error propagationAd-hoc behavior when faced with inconsistencyExample: Hiyu (uses Kinvey)10Slide11

Multiple consistency modes within/across apps

Different data types warrant different consistency Sync semantics oblivious to consistencyLast-writer-wins prevalent; inadequate for manyLeads to corruption & resurrection of deleted dataAtomicity violation of granular data“Dangling pointers” and “half-formed” notes visibleLimited offline supportMany apps simply disable mobile offline operationsInadequate error propagationAd-hoc behavior when faced with inconsistencyInadequate Consistency

Data

-sync needs

fixing!

More details in paper

Slide12

objects

Need Better Sync Services

tables

Choose: Eventual

Causal

or Strong

Control over data transfer

Not lose data!

Challenge: provide high-level abstraction and

hide all complexity under the hood

12Slide13

Motivation

StudySimba Evaluation13Slide14

sTable: Simba Table

ConvenienceLocal programming abstractionNice propertiesUnified store & sync: atomic tabular & object dataEnd-to-end tunable consistency across devicesOffline support, Conflict detection & resolution14LogicalPhysicalNamePhotoSnoopy

Lucy

rowID

rowVersion

Name

Photo

row1

Snoopy

[ab1fd, 1fc2e]

row2

Lucy

[x561a,

3f02a]

1fc2e

ab1fd

x561a

3f02a

Object Store

Table StoreSlide15

Store and view photos locally

Store album info in SQL tables, photos in local FSApp logic to access both the tables and the FS

Row 1

Obj

album info

photo

App components

Activities and GUI

CursorAdapter

Local database I/O

Local file I/O

Unify data model (part of app logic)

REST methods (JSON, HTTP)

N/W management

Push notifications

Data caching

Atomic syncing

Conflict detection

Conflict resolution

Provide sync with other users/devices

Design network interaction with a cloud service

May want to notify users of new photos

May cache shared photos to avoid extra transfers

Ensure consistent album info and photo sync

Another user changes photo tag – conflict!

Album gets synced, but no photos? requires atomicity

Writing an App: Photo Sharing

Remember the tedious example?

15Slide16

Create a photo album

Add a new photoRetrieve stored photoRegister read/write syncWriting a Simba Photo AppregisterReadSync

(“album”,600,0,3G) // period=10min,

pref=3GregisterWriteSync(“album”,300,60,WIFI)

// period=5min, pref=WiFi

cursor =

readData

(“album”, {“photo”}, “name=?”, {“Snoopy”})

is =

cursor.getInputStream

().get(0)

inputstream

for object

is.read

(buffer)

// read object data into buffer

objs

writeData

(“album”, {“name=Snoopy”}, {“photo”})

objs

[0].write(

photoBuffer

)

// write object data

createTable

(“album”, “name VARCHAR, photo OBJECT”, Properties)

Strong

Causal

EventualSlide17

Writing a

Simba Photo AppbeginCR(“album”);rows = getConflictedRows(“album”);for (row; rows; next row) resolveConflict(“album”, row, MINE); // choice: mine, their, customendCR(“album”);

Conflict resolutionSlide18

Simba

Sync ProtocolCRUDupcalls

Simba

Client

Mobile App

Simba

SDK

Apps write all data to unified storage API

Simba

handles all data management

Well-defined consistency semantics

Apps can specify sync policy per

sTable

Period, delay tolerance, consistency

Sync objects on

WiFi

, tables on 4G

Writing Apps with

Simba

Activity, GUI, app logic

Simba

Cloud

App components

Activities and GUI

CursorAdapter

Conflict resolution (simplified)

Local database I/O

Local file I/O

Unify data model

(

part of app logic)

REST

methods

(

JSON, HTTP)

Data caching

N/W

mgmt

Push notifications

Atomic syncing

Conflict detection

See FAST’15 paper for more details

“Reliable, Consistent, and Efficient Data Sync for Mobile Apps”Slide19

Simba

CloudS

Authenticator

Load Balancer

Gateway

Simba

Gateway DHT

Store node

sTable

DHT

Table

Store

(Cassandra)

Object

Store

(Swift)

Sync Protocol

Simba

Cloud Store

Sync Protocol

Notifications

Sync Protocol

Notifications

19Slide20

Simba

CloudS

Sync Protocol

Notifications

Sync Protocol

Notifications

Simba

CloudSlide21

Simba Sync

Need to query by versionObject store must perform scanNeed to find rows which match change-set queriesChange-setSet of changes between any past version and the current version21SimbaS3Objectsget(id)/put(id)Change-setssync(v1  v2)Object StoreData-syncStoreSlide22

Swift

Upstream Sync (Update)22ClientGatewaySimba Store

OID1

OID12

Ver

Name

File

Friends

OID1

Retrieve existing row

Increment version

Update tabular data

Write chunks to object store

Write table to table store

Send

notification to

subscribers

Cassandra

Ver

Name

File

Contacts

[c1,c2]

Ver

Name

File

Friends

[c1,c2,

]

sTable

Ver

Name

File

Friends

[c1,c2]

OID1

Subscription

Manager

NSlide23

Downstream Sync (Update)

23ClientGatewaySimba StoreCassandraSwift

sTable

Ver

Name

File

Friends

OID1

Subscription

Manager

Ver

Name

File

Friends

[c1,c2,c3]

Object 1

OID1

Ver

Name

File

Contacts

OID1

Client pulls changes

Send tabular changes

Send object changes

PULL (version=0)

OID1

2Slide24

Swift

Upstream Sync (Update)24ClientGatewaySimba Store

OID1

Retrieve existing row

Increment version

Update tabular data

Write chunks to object store

Write table to table store

Insert change-set into cache

Send

notification to

subscribers

Cassandra

Ver

Name

File

Friends

[c1,c2,

]

sTable

Subscription

Manager

Change

Cache

CS1

NSlide25

Downstream Sync (Update)

25ClientGatewaySimba StoreCassandraSwift

sTable

Ver

Name

File

Friends

OID1

Subscription

Manager

Change

Cache

Ver

Name

File

Friends

[

]

Object 1

OID1

Ver

Name

File

Friends

OID1

Client pulls changes

Lookup change-set

Send tabular changes

Send object changes

PULL (version=0)Slide26

Motivation

StudySimba Evaluation26Slide27

Evaluation

We seek to answer the following questions:How easy is it to write consistent apps with Simba?How does sCloud scale with tables and clients?How lightweight is the Simba Sync Protocol?How does sCloud perform on CRUD operations?How does consistency trade-off latency?27Slide28

Fixing Apps with Simba

28Simba apps easy to write! (~3 hours)Equivalent apps hard with Dropbox (incomplete after 3 days)Universal Password ManagerStore account credentialsUses Dropbox to sync account DBObservation: Concurrent modifications cause silent overwritesFix (2 approaches):Store entire DB as object in a sTable (easiest)Store each account as row in a

sTable

Todo.txt

Interactive task list

Uses

Dropbox

to sync

2 files:

Active tasks

Completed (archived) tasks

Modified to store tasks in 2

sTables

Active (Strong), Completed (Eventual)Slide29

Scalability SetupSusitna cluster at NMC

PRObE32 nodes16 Store/Swift/Clients16 Gateway/CassandraSusitna node specs:4x 16-core AMD Opteron 2.1GHz CPUs128GB DRAM2x 3TB 7200RPM data disksInfiniBand high-speed network29Slide30

Scalability with Clients

3016 Gateways (Cassandra), 16 Stores (Swift) w/ key+data cache128 tables, 9:1 reader-to-writer ratio, 500 operations/sSimba scales under high-client loadSlide31

Scalability with Tables

3116 Gateways (Cassandra), 16 Stores (Swift) w/ key cache only# Clients = 10 x # Tables, 9:1 reader-to-writer ratio, 500 operations/sSimba scales well across tables and objectsMany tables = Bottleneck. Need better sync stores!Slide32

Conclusions

Building data-centric mobile app should be simpleApp developers focus on implementing app core logicsTable abstractionManages unified table and object dataGuarantees per-row atomicityAllows for per-table tunable consistencySimba: scalable cloud data sync service Transparent handling of data sync and failures32Slide33

THANK YOU!

Project: http://tinyurl.com/SimbaServiceCode: https://github.com/SimbaService/Simba33Simba Cloud [EuroSys’15]:http://dl.acm.org/authorize?N95496Simba Client [FAST’15]:https://www.usenix.org/conference/fast15/technical-sessions/presentation/go

Simba - PowerPoint Presentation

Simba - PPT Presentation

Share:

Link:

Embed:

Related Contents