Spring 2009 L 27 Social Networks and Other Stuff Overview Social Networks Multiplayer Games Class Feedback Discussion 2 Background Sybil Attack Sybil attack Single user pretends many fakesybil identities ID: 388396
Download Presentation The PPT/PDF document "15-446 Distributed Systems" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
15-446 Distributed SystemsSpring 2009
L-27 Social Networks and Other StuffSlide2
Overview
Social NetworksMultiplayer Games
Class Feedback Discussion
2Slide3
Background: Sybil Attack
Sybil attack: Single user pretends many fake/sybil identities
Creating multiple accounts from different IP addressesSybil identities can become a large fraction of all identities
Out-vote honest users in collaborative tasks
launch
sybil
attack
honest
malicious
3Slide4
Background: Defending Against Sybil Attack
Using a trusted central authorityTie identities to actual human beings
Not always desirableCan be hard to find such authority
Sensitive info may scare away usersPotential bottleneck and target of attackWithout a trusted central authorityImpossible unless using special assumptions [Douceur’02]
Resource challenges not sufficient -- adversary can have much more resources than typical user 4Slide5
SybilGuard Basic Insight:
Leveraging Social Networks
Undirected graphNodes = identitiesEdges =
strong trust E.g., colleagues, relatives
Our Social Network Definition
5Slide6
SybilGuard Basic Insight
malicious
user
honest
nodes
sybil
nodes
Observation: Adversary cannot create extra edges between honest nodes and sybil nodes
attack edges
n
honest users: One identity/node each
Malicious users: Multiple identities each (sybil nodes)
Sybil nodes may collude – the adversary
6Slide7
SybilGuard Basic Insight
honest
nodes
sybil
nodes
Dis
-proportionally
small cut disconnecting a large number of identities
But cannot search for such cut brute-force…
7Slide8
Goal of Sybil Defense
Goal: Enable a verifier node to decide whether to
accept another suspect node
Accept: Provide service to / receive service from Idealized guarantee: An honest node accepts and only accepts other honest nodes
SybilGuard:Bounds the number of sybil nodes acceptedGuarantees are with high probabilityApproach: Acceptance based on random route intersection between verifier and suspect
8Slide9
Random Walk Review
pick random edge d
pick random edge c
pick random edge e
a
b
d
e
f
c
9Slide10
Random
1 to 1 mapping
between
incoming edge and outgoing edge
Random Route: Convergence
Using routing table gives Convergence PropertyRoutes merge if crossing the same edge
10
a
d
b a
c b
d
cd ee df f
abcde
frandomized routing tableSlide11
Random Route: Back-traceable
Using 1-1 mapping gives Back-traceable Property
Routes may be back-traced
11
a
d
b a
c b
d cd
ee d
f f
a
bc
defIf we know the route traverses edge e, then we know the whole routeSlide12
Random Route Intersection:
Honest Nodes
Verifier accepts a suspect if the two routes intersectRoute length w
:W.h.p., verifier’s route stays within honest regionW.h.p
., routes from two honest nodes intersect
sybil nodes
honest nodes
Verifier
Suspect
12Slide13
Random Route Intersection: Sybil Nodes
SybilGuard bounds the number of accepted
sybil nodes within g
*wg: Number of attack edgesw
: Length of random routesNext …Convergence property to bound the number of intersections within gBack-traceable property to bound the
number of accepted sybil nodes per intersection within w13Slide14
Bound # Intersections Within
gConvergence: Each attack edge gives one intersection
at most
g intersections with g attack edges
sybil nodes
honest nodes
Verifier
Suspect
must cross attack edge to intersect even if sybil nodes do not follow the protocol
same
intersection
Intersection =
(node, incoming edge
14Slide15
Bound # Sybil Nodes Accepted per Intersection within
w
Back-traceable: Each intersection should correspond to routes from
at most
w
honest nodesVerifier accepts at most w nodes per intersectionWill not hurt honest nodes
Verifier
for a given intersection
15Slide16
Summary of SybilGuard Guarantees
Power of the adversary:
Unlimited number of colluding sybil nodes
Sybil nodes may not follow SybilGuard protocol W.h.p., honest node accepts ≤
g*w sybil nodesg: # of attack edgesw: Length of random route
If SybilGuard bounds # accepted sybil nodes within
Then apps can do
n
/
2
byzantine consensus
n
majority voting
not much larger than neffective replication16Slide17
Overview
Social NetworksMultiplayer Games
Class Feedback Discussion
17Slide18
Individual Player’s View
Interactive environment (e.g. door, rooms)
Live ammoMonsters
PlayersGame state
18Slide19
High-Speed, Large-Scale, P2P:
Pick 2
Many console games are peer hosted to save costs
Limits high-speed games to 32 players
1000+ player games need dedicated servers
High-speed
Large-scale
P2P
Question
: Can we achieve all 3?
19Slide20
P2P
Internet
Primary object
Local View
Replica objects
20Slide21
High-Speed
Internet
Local View
Inter-object writes
must be reflected
very quickly
Primary object
Replica objects
21Slide22
High-Speed
Internet
Local View
Replica objects
20 updates/sec
≈ 16 kbps per player
Delay must be < 150ms
[
Beigbeder
‘04]
Primary object
22Slide23
Large-Scale
Internet
Bandwidth per Player
(Mbps)
23Slide24
Area-of-Interest (AOI) Filtering
Large shared world
Composed of map information, textures, etc
Populated by active entities: user avatars, computer AI’s, etcOnly parts of world relevant to particular user/player
Player 1
Player 2
Game World
24Slide25
Object Model
Game world treated as collection of objectsSingle primary copy for each object
Maintained at a single owner node
Serialization point for updatesDetermines actions/behavior of objectSecondary replicas
Primary object may need to examine other objects to determine actionNeed low latency access to object must replicate object in advanceHow to find set of objects to replicate? hardest part
25Slide26
Object Discovery –Solution
Use publish-subscribe to “register” long-lived distributed lookupsPublications created each time object is updated (or periodically when no update is done)
Publication contents = state of object
Player
x
≥
50
x ≤ 150y ≥ 150
y ≤ 250Interests
x 100
y 200Events
(100,200)
(150,150)(50,250)ArenaVirtual World26Slide27
Publish-Subscribe (PubSub) Overview
Key feature
subscription languageSubject/channel-based subscriptions (e.g. all publications on the IBM stock channel)
Rich database-like subscription languages (e.g. all publications with name=IBM and volume > 1000)
State-of-the-artScalable distributed designs with channel-based subscriptionsUnscalable distributed designs with rich subscriptionsGoal scalable distributed design with “richer” subscriptionsExample: x ≤ 200 & y < 100
Support for multi-dimensional range predicates
Subscription
Publications
Publishers produce
events
Subscribers register their
interests
27Slide28
Using DHTs for Range Queries
No cryptographic hashing for key
identifier
Query: 6
x
13key = 6 0xabkey = 7
0xd3…key = 13 0x12
0x00
0x70
0xf0
0x30
0xb0
0x20
0xc0
0xd0
0x100xe00xa0
0x90
0x800x400x500x60
Query: 6 x 13
28Slide29
Using DHTs for Range Queries
Nodes in popular regions can be overloaded
Load imbalance!
29Slide30
DHTs with Load Balancing
Mercury load balancing strategyRe-adjust responsibilities
Range ownerships are skewed!
30Slide31
DHTs with Load Balancing
0x00
0x30
0xa0
0xb0
0xd0
Finger pointers
get skewed!
Popular
Region
0x90
0x80
0xf0
0xe0
Each routing hop may not reduce node-space by half!
no log(n) hop guarantee
31Slide32
Ideal Link Structure
0x00
0x30
0xa0
0xb0
0xd0
Popular
Region
0x90
0x80
0xf0
0xe0
32Slide33
Mercury
Values
Nodes
If we had the above information…
For finger
i
Estimate value v for which
2i th node is responsible
Need to establish links based on node-distance
4
8
v
4
v
8
33Slide34
Mercury
Values
Nodes
Need to establish links based on
node-distance
4
8
v
4
v
8
Values
Node-density
Piece-wise linear approximation
Histogram
34Slide35
Histogram Maintenance
0xf0
0x00
0x70
0xa0
0xb0
0xd0
0x90
0x80
0xe0
0x30
Request sample
(Range, density)
Measure node-density locally
Gossip about it!
Values
Node-density
(Range, density)(Range, density)
35Slide36
Load Balancing
Basic idea: leave-rejoin
Steps
Find average, check if heavy or light
Light nodes perform a leave and rejoin
0
10
20
25
35
45
60
65
707585Average
HeavyLight
1572.5
Load histogram
Load
36Slide37
M
ERCURY
Routing [Sigcomm 2004]
Send subscription to
any one
attribute hubSend publications to
all attribute hubsTunable number of long links can range from full-mesh to DHT-like[0, 80)
[210, 320)
50 ≤ price
≤ 150150 ≤ volume ≤ 250
price 100volume 200
H
price
[240, 320)[80, 160)[160, 240)Hvolume
[0, 105)[105, 210)SubscriptionPublication
Rendezvous point37Slide38
Object Discovery – Basic Design
Use publish-subscribe to “register” long-lived distributed lookupsPublications created each time object is updated (or periodically when no update is done)
Publication contents = state of object
Player
x
≥
50
x ≤ 150y ≥ 150
y ≤ 250Interests
x 100
y 200Events
(100,200)
(150,150)(50,250)ArenaVirtual World38Slide39
Prefetching and Persistence
Basic design has problemsCollecting/updating existing state is too slow
High subscription update rate
1st fix
use Mercury only for object discovery and not state update2nd fix persistent publications/subscriptionsPublication lifetimes subsequent subscriptions would immediately trigger transmissionSubscription lifetimes enabled soft-state approach to subscriptions
3rd fix predict future subscriptionsCreates a new hybrid of persistent storage and pubsub
39Slide40
Colyseus Architecture Overview
Object Location
Replica Management
1. Specify Predicted Interests:
(5 < X < 60 & 10 < y < 200) TTL 30sec
2. Locate Remote Objects: P3 on s2, P4 on s2
3. Register Replicas: R3 (to s2), R4 (to s2)
4. Synch Replicas: R3, R4
Mercuryserver s2
R5
R3
R4
P1P2
Object Store
server s1P3P4Object Placement
5. Infer Interests: R3, R4, P2
5. Optimize Placement: migrate P1 to server s2
P140Slide41
Demo
Synthetic game used
for evaluation
41Slide42
42
View Inconsistency
Observations:
1. View inconsistency is small and gets repaired quickly
2. Missing objects on the periphery
no delay100 ms delay400 ms delaySlide43
Area-of-Interest (AOI) Filtering
Only receive updates from players in your AOI
Colyseus
[Bharambe ‘06]
VON [Hu ‘06]SimMUD [
Knutsson ’04]Problems:Open-area maps, large battlesRegion populations naturally follow a power-law[
Bharambe ‘06, Pittman ‘07]
Requirement
: ~1000 players in
same
AOI
Low population
High population
Quake 3 region popularity
43Slide44
Smoothing Infrequent Updates
Send
guidance
(predictions)
instead of state updatesGuidable AI extrapolates transitions between pointsE.g., game path-finding code
guidance
guidance
Actual path
?
Problem
: Predictions are not always accurate
Interactions appear inconsistent
Jarring if player is paying attention
Replica
object
44Slide45
Donnybrook: Interest Sets
Intuition
: A human can only focus on a constant number of objects at once
[Cowan ‘01, Robson ‘81]Only need a constant number of high-accuracy replicasInterest Set
: The 5 players that I am most interested inSubscribe to these players to receive 20 updates/secOnly get 1 update/sec from everyone else
Me
My Interest Set
45Slide46
Donnybrook: Interest Sets
How to estimate human attention?
Attention(
i
) = how much I am focused on player i
d
1
d
2
θ
1
θ
2
Attention(i) =fproximity(di)faim(θi)finteraction-recency(ti)+
+Player 1Player 246Slide47
= Interest Set
Not in Interest Set
47Slide48
48Slide49
49Slide50
50Slide51
Existing Distributed File Systems
Each server responsible for pseudo-random range of ID spaceObject are given pseudo-random IDs
150-210
211-400
401-513
324
987
110
51Slide52
Preserving File Locality
Use Mercury to store data
Bill
Bob
Docs
1
1
2
6
1
bid
0
…
6
1
bid
1
…61
bid2…useridpath encode
blockid
67570-600601-660
661-700
52Slide53
Benefits
Order preserving placement achieves availability within one ‘9’ of the optimal placement.
Order preserving placement can speedup task latency by 60-100%.
Larger networks see greater speedup.
Availability Improvement
Request Speedup
53Slide54
Continuous Load Balancing
Mercury can balance file system load better than existing systems even under very skewed write patterns
54Slide55
Overview
Social NetworksMultiplayer Games
Class Feedback Discussion
55Slide56
Discussion
Lecture topics?Balance of networking/dist sys/ubiq?
Research/adv topics/practical engineering?Text book vs. papers vs. lectures
ProjectsFree form vs. proposedAndroid vs. no Android?
56