23 rd USENIX Security Symposium August 20 th 2014 Rob Jansen US Naval Research Laboratory John Geddes University of Minnesota Chris Wacek Georgetown University Micah Sherr Georgetown University ID: 632956
Download Presentation The PPT/PDF document "Never Been KIST: Tor’s Congestion Mana..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Never Been KIST: Tor’s Congestion Management Blossoms with Kernel-Informed Socket Transport
23rd USENIX Security SymposiumAugust 20th 2014
Rob
Jansen
US Naval Research Laboratory
John Geddes University of Minnesota
Chris
Wacek
Georgetown University
Micah
Sherr
Georgetown University
Paul
Syverson
US Naval Research LaboratorySlide2
Anonymous Communication: TorSlide3
Tor is Slow!!! Research*
PCTCP: Per-Circuit TCP-over-IPsec Transport for Anonymous Communication Overlay Networks (CCS ‘13)Reducing Latency in Tor Circuits with Unordered Delivery (FOCI ‘13)How Low Can You Go: Balancing Performance with Anonymity in Tor (PETS ‘13)The Path Less Travelled: Overcoming Tor's Bottlenecks with Traffic Splitting (PETS ’13)An Empirical Evaluation of Relay Selection in Tor (NDSS ‘13)LIRA: Lightweight Incentivized Routing for Anonymity (NDSS ‘13)Improving Performance and Anonymity in the Tor Network (IPCCC ‘12)
Enhancing Tor's Performance using Real-time Traffic Classification (CCS
’
12)
Torchestra
: Reducing interactive traffic delays over Tor (WPES ‘12)Throttling Tor Bandwidth Parasites (USENIX Sec ‘12)LASTor: A Low-Latency AS-Aware Tor Client (Oakland ‘12)Congestion-aware Path Selection for Tor (FC ‘12)
*Not a comprehensive listSlide4
Tor is Slow!!! Research*
PCTCP: Per-Circuit TCP-over-IPsec Transport for Anonymous Communication Overlay Networks (CCS ‘13)Reducing Latency in Tor Circuits with Unordered Delivery (FOCI ‘13)
How Low Can You Go: Balancing Performance with Anonymity in Tor (PETS ‘13)
The Path Less Travelled: Overcoming Tor's Bottlenecks with Traffic Splitting (PETS
’
13)
An Empirical Evaluation of Relay Selection in Tor (NDSS ‘13)
LIRA: Lightweight Incentivized Routing for Anonymity (NDSS ‘13)
Improving Performance and Anonymity in the Tor Network (IPCCC ‘12)
Enhancing Tor's Performance using Real-time Traffic Classification (CCS ’12)Torchestra: Reducing interactive traffic delays over Tor (WPES ‘12)Throttling Tor Bandwidth Parasites (USENIX Sec ‘12)LASTor: A Low-Latency AS-Aware Tor Client (Oakland ‘12)Congestion-aware Path Selection for Tor (FC ‘12)
*Not a comprehensive list
Where?Slide5
This Talk
Where is Tor slow?Measure public Tor and private Shadow-Tor networksIdentify circuit scheduling and socket flushing problemsDesign KIST: Kernel-Informed Socket TransportUse TCP snd_cwnd to limit socket writes
Evaluate KIST
Performance and Security
Reduces kernel and end-to-end circuit congestion
T
hroughput attacks unaffected, speeds up latency attacksSlide6
Outline
BackgroundInstrument Tor, measure congestionAnalyze causes of congestionDesign and evaluate KISTPerformanceSecuritySlide7
Relay OverviewSlide8
Relay Overview
TCP
TCP
TCP
TCP
TCP
TCP
TCP
Tor
circuits
are multiplexed over a TCP
transportSlide9
Relay Overview
TCP
TCP
TCP
TCP
TCP
TCP
TCPSlide10
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Opportunities for
traffic
managementSlide11
Outline
BackgroundInstrument Tor, measure congestionAnalyze causes of congestionDesign and evaluate KISTPerformanceSecuritySlide12
Live Tor Congestion -
libkqtimeKernel Input
Kernel Output
Tor Input
Tor Output
Tor CircuitsSlide13
Live Tor Congestion -
libkqtimeKernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
tag
match
tag
matchSlide14
Live Tor Congestion -
libkqtimeKernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
tag
match
tag
match
track cellsSlide15
Shadow Network Simulation
Enhanced Shadow with several missing TCP algorithmsCUBIC congestion controlRetransmission timersSelective acknowledgements (SACK)Forward acknowledgements (FACK)Fast retransmit/recoveryDesigned largest known private Tor network3600 relays and 12000 simultaneously active clients
Internet topology graph: ~700k nodes and 1.3m linksSlide16
Track the UID
Shadow-Tor Congestion – UIDs
UID
UID
Track the UIDSlide17
Track the UID
Shadow-Tor Congestion – UIDs
UID
UID
Track the UID
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor CircuitsSlide18
Tor and Shadow-Tor Congestion
Congestion occurs almost exclusively in
outbound kernel buffers
Shadow-Tor
Live-TorSlide19
Outline
BackgroundInstrument Tor, measure congestionAnalyze causes of congestionDesign and evaluate KISTPerformanceSecuritySlide20
Analyzing
Causes of CongestionKernel Output
Tor Output
Tor Circuits
Queuing delays in kernel output bufferSlide21
Analyzing Causes of Congestion
Kernel Output
Tor Output
Tor Circuits
Queuing delays in kernel output buffer
Problem 1:
Circuit scheduling
Problem 2:
Flushing to SocketsSlide22
Problem 1: Circuit Scheduling
Kernel Output
Tor Output
Tor Circuits
Libevent
schedules one connection at a timeSlide23
Problem 1:
Circuit SchedulingKernel Output
Tor Output
Tor Circuits
Libevent
schedules one connection at a time
Tor only considers a subset of writable circuitsSlide24
Problem 1:
Circuit SchedulingKernel Output
Tor Output
Tor Circuits
Libevent
schedules one connection at a time
Tor only considers a subset of writable circuits
Circuits from
different connections
are not prioritized correctlySlide25
Problem 1: Circuit SchedulingSlide26
Problem 1: Circuit Scheduling
Correctly differentiated
No differentiationSlide27
Problem 1: Circuit Scheduling
Correctly differentiated
No differentiation
99.775%
of any two circuits are
unsharedSlide28
Problem 2: Flushing to Sockets
Kernel Output
Tor Output
Tor Circuits
Queuing delays in kernel output buffer
FIFOSlide29
Problem 2: Flushing to Sockets
Kernel Output
Tor Output
Tor Circuits
Worse priority traffic (high throughput flows)
FIFOSlide30
Problem 2: Flushing to Sockets
Kernel Output
Tor Output
Tor Circuits
Better priority traffic
(low throughput flows)
Worse priority traffic (high throughput flows)
FIFOSlide31
Problem 2: Flushing to Sockets
Kernel Output
Tor Output
Tor Circuits
Better priority traffic
(low throughput flows)
Must wait for kernel to flush socket to network (blocked on TCP
cwnd
)
Worse priority traffic (high throughput flows)
FIFOSlide32
Problem 2: Flushing to Sockets
Kernel Output
Tor Output
Tor Circuits
Better priority traffic
(low throughput flows)
Reduces effectiveness of circuit priority
Worse priority traffic (high throughput flows)
FIFOSlide33
Outline
BackgroundInstrument Tor, measure congestionAnalyze causes of congestionDesign and evaluate KISTPerformance
SecuritySlide34
Ask the kernel, stupid!
Utilize getsockopt and ioctl syscalls
socket_space
=
sndbufcap
–
sndbuflen
tcp_space = (cwnd – unacked) * msssndbuflensndbufcap
unacked
cwnd
Slide35
Kernel-Informed Socket Transport
Don’t write it if the kernel can’t send it;bound kernel writes by: Socket: min(socket_space, tcp_space)Global: upstream bandwidth capacity
Solution to Problem 2Slide36
Kernel-Informed Socket Transport
Don’t write it if the kernel can’t send it;bound kernel writes by: Socket: min(socket_space, tcp_space)Global: upstream bandwidth capacity
Choose globally from
all writable circuits
Solution to Problem 1Slide37
Kernel-Informed Socket Transport
Don’t write it if the kernel can’t send it;bound kernel writes by: Socket: min(socket_space, tcp_space)Global: upstream bandwidth capacity
Choose globally from
all writable
circuits
Try to write
again before kernel starvationSlide38
KIST Reduces Kernel CongestionSlide39
KIST Increases Tor CongestionSlide40
KIST Reduces Circuit CongestionSlide41
KIST Improves Network LatencySlide42
Outline
BackgroundInstrument Tor, measure congestionAnalyze causes of congestionDesign and evaluate KISTPerformance
SecuritySlide43
Traffic Correlation: Latency
Hopper
et.al
. CCS’07
Goal: narrow down potential locations of the client on a target circuitSlide44
Traffic Correlation: Latency
Hopper
et.al
. CCS’07
-Inject redirect or
javascript
-Start timerSlide45
Traffic Correlation: Latency
GET
Hopper
et.al
. CCS’07
Request redirected page or embedded objectSlide46
Traffic Correlation: Latency
GET
Hopper
et.al
. CCS’07
-Stop timer
-Estimate latencySlide47
Latency Attack| estimate – actual |Slide48
Latency Attacknum pings until best estimateSlide49
Traffic Correlation: Throughput
Mittal
et.al
. CCS’11
Goal: find guard relay of the client on a target circuitSlide50
Traffic Correlation: Throughput
Mittal
et.al
. CCS’11
Probe throughput of
all
guard relaysSlide51
Traffic Correlation: Throughput
Mittal
et.al
. CCS’11
Correlate throughput between exit and probesSlide52
Throughput Attack ResultsSlide53
Conclusion
Where is Tor slow?KIST complements other performance enhancements, e.g. circuit priorityNext stepsCurrently exploring various algorithmic optimizationsTest KIST in the wild and deploy in TorSlide54
Questions?
rob.g.jansen@nrl.navy.milrobgjansen.comgithub.com/robgjansen/libkqtimegithub.com/shadow
think like an adversarySlide55
Relay
InternalsKernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Network InputSlide56
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Split data into socket buffersSlide57
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Read data from sockets into TorSlide58
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Process data (encrypt/decrypt)Slide59
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Split cells
into circuit queuesSlide60
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Circuits linked to outgoing connectionSlide61
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Schedule cellsSlide62
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Write data
from Tor into socketsSlide63
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Schedule data for sendingSlide64
Relay Internals
Kernel Input
Kernel Output
Tor Input
Tor Output
Tor Circuits
Opportunities for
traffic
managementSlide65
KIST Improves Network Throughput