Nisarg Raval Animesh Srivastava Ali Razeen Kiron Lebeck Ashwin Machanavajjhala Landon Cox Duke University University of Washington 1 Cameras all around us ID: 629370
Download Presentation The PPT/PDF document "What You Mark is What Apps See" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
What You Mark is What Apps See
Nisarg Raval, Animesh Srivastava, Ali Razeen, Kiron Lebeck, Ashwin Machanavajjhala, Landon CoxDuke University, University of Washington
1Slide2
Cameras all around us
Mobile devicesHome entertainmentIoT and robotics
2Slide3
Protecting visual secrets is hard.
3Slide4
4
http://www.zdnet.com/article/super-bowl-wi-fi-password-credentials-broadcast-in-pre-game-security-gaffe/Slide5
Coarse-grained control
5App gets complete access or noneSlide6
Fine-grained control
6AppPrivacy Framework
Recognizers
Per-app
policies
Detect objects (e.g., faces),
transform imagesSlide7
Fine-grained control
7Privacy Framework
Recognizers
Per-app
policies
Recognizer [
Usenix
Security
2013]
WDAC [CCS 2014]
SurroundWeb
[Oakland 2015]
I-
Pic
[
MobiSys
2016]
Must anticipate objects.
Password recognizer?
Product-roadmap recognizer?Slide8
Want general fine-grained control.
8Slide9
Solution: privacy markersPrivacy markers
Fine-grained access control for visual infoSupport arbitrary objects and surfacesTwo example systems: PrivateEye and WaveOff9PrivateEye
2D surfaces (e.g., whiteboards)
WaveOff
3D Objects (e.g., faces)Slide10
Trust and attacker modelsAssumptions
Recording-device hardware, system software is trustedThird-party apps are untrustedTrusted code isolated from untrusted codeDetermined attackers can still capture secretsGoal is preventing inadvertent leaks by apps10Slide11
Privacy-marker goals
Ease of useMarking surfaces and objects should be convenient, easyReliable recognitionSecrets should be protected in face of motion, lighting, etc.Good performancePreserve legitimate app functionality11Slide12
Privacy-marker goals
Ease of useMarking surfaces and objects should be convenient, easyReliable recognitionSecrets should be protected in face of motion, lighting, etc.Good performancePreserve legitimate app functionality12Slide13
Ease of use
13
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
PrivateEye
WaveOffSlide14
Ease of use
14
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
PrivateEye
Draw special shape around region
(pens, presentation software, etc)Slide15
Ease of use
15
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
PrivateEye
External rectangleSlide16
Ease of use
16
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
PrivateEye
Internal 12-sided polygonSlide17
Ease of use
17
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
PrivateEye
Marked regionSlide18
Ease of use
18
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
WaveOff
Use trusted camera-preview UISlide19
Ease of use
19
Marking must not require extra equipment
No RFID tags, QR codes, etc.
Rely on only device software and common office supplies
WaveOff
Extract features
w/i
bounding boxSlide20
Privacy-marker goals
Ease of useMarking surfaces and objects should be convenient, easyReliable recognitionSecrets should be protected in face of motion, lighting, etc.Good performancePreserve legitimate app functionality20Slide21
Reliable recognition
21
Computer vision often fails
Errors due to motion, occlusion, lighting
Cannot reliably block marked regions
Key design decision
Instead,
whitelist
marked regions
Bias toward security
Track markers to improve app utilitySlide22
Reliable recognition
22Privacy FrameworkApp
PrivateEye
Detect,
whitelist
marked regions
Per-app
policiesSlide23
Reliable recognition
23Privacy FrameworkApp
WaveOff
Per-app
policies
Detect,
whitelist
marked objectsSlide24
Privacy-marker goals
Ease of useMarking surfaces and objects should be convenient, easyReliable recognitionSecrets should be protected in face of motion, lighting, etc.Good performancePreserve legitimate app functionality24Slide25
Good performance
25
Markers designed for quick detection
Detect contours if enough contrast with background
12-sided polygon with many right angles
PrivateEye
Right angles + high-contrast linesSlide26
Good performance
26
Markers designed for quick detection
Builds model for the object instead class of objects
BRISK features for fast and robust matching
WaveOff
Fast recognition and trackingSlide27
Android integration
Hardware
i
ndependent
Hardware
d
ependent
PrivateEye
WaveOff
27Slide28
Android integration
Naïve implementation led to poor performance4 frames-per-second (FPS) video recording on Nexus 5Multiple streams during video recordingKey observations:Different streams have same data with different settingsHigher throughput with non-blocking operationsOptimizations led to a median rate of 22 FPS deliveryMore details in the paper28Slide29
Evaluation
Ease of usePerformed 26-person user studyReliable recognitionPrecision, recall with video benchmarkGood performanceFPS with video benchmark29Slide30
User study
Task: Scan QR code on 2D and 3D surfaces
Used third-party
BarCode
Scanner app
2D surface was a whiteboard
3D surface was a coffee mug
26 participants divided in two groups
Control group - stock camera on unmodified Android
Case group -
PrivateEye
and
WaveOff
enabled Android
30Slide31
User study feedback
31
Case and control rated similarly
(higher is better)Slide32
User study feedback
32
Note:
WaveOff
better rated than control app!Slide33
Recognition evaluation
Video benchmark under different scenarios
10 sec. long videos with resolution 1280x960
Each video combined a setting and motion
Divided video frames into cells, labeled each public or private
Three settings
2D/PrivateEye – Whiteboard, paper, laptop screen
3D/WaveOff – Plain background, nearby object, nearby PC
Three camera motions
Still – simulates image capture
Spin – to understand the impact of change in orientation
Scan – simulates a video recording
33Slide34
PrivateEye recognition
34Near perfect precision(better security)Slide35
PrivateEye recognition
35Drop in recall due to motion blurSlide36
WaveOff recognition
36Irregular shape leads to lower precision and recallSlide37
Performance results
37Median FPS between 20 and 25(4 FPS before optimizations)Slide38
LimitationsRectangular markings are required
May cause leaks at the edgesMarkers should be tightly drawnCamera may zoom inside the marked regionsThe system will block the viewMarked regions are public for all the appsNo apps based policy38Slide39
Summary
Privacy markersGeneral fine-grained controlExample systems: PrivateEye, WaveOffWhitelist approachComputer vision is unreliableBias towards securityPrototype evaluationPositive user-study feedbackGood precision, recall, throughputFutureOther sensing modalities (audio, motion)?More complex policies?
What you mark
What apps see
39