Presentations text content in November 4, 2014 Anupam
Slide1
November 4, 2014
Anupam
Das (UIUC), Nikita Borisov (UIUC),Matthew Caesar (UIUC)
Do You Hear What I Hear? Fingerprinting Smart Devices Through Embedded Acoustic Components
1
CCS 2014
Slide2November 4, 2014
2
Smartphone Usage
How many people use smartphones?Source: Gartner
Shipment of smartphone is increasing every year.
75%
of the mobile phones are smartphones and highly-featured phones
Source: Business Insider
Slide3November 4, 2014
3
A Closer Look at Smartphones
Today, smartphones come with a wide range of sensors. All of which are useful for a variety of tasks.
However, sensors could also be potential source for unique fingerprints.
Motion detection
Gesture detection
Audio Genre detection
Location
detection
Interaction with nearby devices
Compass
Slide4November 4, 2014
4
Why Fingerprint Smartphones?Software based approaches:Browser based features, CookiesDifferent Firmware and Device driversHardware based approaches:
Clock Skew rateRadio TransmitterAccelerometerSmartphones can be fingerprinted for:Targeted AdvertisementSecondary Authentication factor
The main drawback
for software based approaches is that the source of the fingerprints are
not too stable
.
Moreover, device IDs are not always usable. UDID
for
Apple devices has been
removed
since May 1, 2013 and IMEI
for
an Android Device requires explicit permission.
Slide5November 4, 2014
5
Our Goal
Scenario 1: External attacker locally present1. Attacker records audio signal from distance2. Create a fingerprint of the recorded audio and link the fingerprint to a unique smartphone
Scenario 2: Stealthy App
App
1.Play audio
2. Audio signal
3.Record audio
4. Extract Fingerprint
Need access to only:
Microphone
Internet connection
We focus on fingerprinting
smart devices
through their embedded
speakers
and
microphones
.
Requires:
Deployed microphones
Slide6November 4, 2014
6
Top Android Permissions Source: Mario Frank, Ben Dong, Adrienne Porter Felt, Dawn Song. Mining Permission Request Patterns from Android and Facebook Applications, ICDM, 2012 1
Storage : modify/delete USB storage and SD card contents2Network communication : full Internet access
3
Network communication : view network state
4
System tools : prevent device from sleeping
5
Phone calls : read phone state and identity
6
Hardware controls : control vibrator
7
System tools : automatically start at boot
8
Network communication : view Wi−Fi state
9
Your location : fine (GPS) location
10
Your location : coarse (network−based) location
11
System tools : retrieve running applications
12
Your personal information : read contact data
13
Your messages : read SMS or MMS
14
Your messages : receive SMS
15
Hardware controls : take pictures and videos
16
Hardware controls : record audio
17
System tools : modify global system settings
Top 17 Android permissions out of 173 total permissions
Slide7November 4, 2014
7
4,405,876Real World Apps
There are popular apps that people use that require similar permissions. For example: My Talking Tom
Slide8November 4, 2014
8
Closer Look at Microphones
MEMS microphone:Source: STMicroelectronics
Sound Wave
Mechanical Energy
Capacitive Change
The sensitivity of the microphone depends on how well the
diaphragm
deflects to acoustic
pressure
Imperfections
can
arise due to:
Slight
variations in the chemical
composition
of components from one batch to the
next
Wear and tear
in the
manufacturing
machines
Changes
in temperature and humidity
Voltage Change
Slide9November 4, 2014
9
Basic functionality of a speaker :
Closer Look at SpeakersSource: Center Point AudioElectrical current
Magnetic field around voice coil
Mechanical Energy
Sound Wave
The
frequency
of the sound wave produced
is
dictated
by
the
rate
at
which the
voice coil
moves.
Slide10November 4, 2014
10
Experimental SetupMakerModel#AppleiPhone 51HTCNexus One
14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515Sony EricssonW5181
Total
52
Audio Type
Description
Variations
Instrumental
Musical instruments playing together
4
Human Speech
Small
segments of human speech
4
Song
Combination
of human voice & instrumental sound
3
To
emulate an
attacker we use a laptop
Slide11November 4, 2014
11
Fingerprinting Acoustic Components
1. Fingerprinting Speakers
This scenario is suited for the case where the attacker has
deployed
nearby
microphones
to capture
audio signal
(e.g.,
ringtone) from user device.
Slide12November 4, 2014
12
Fingerprinting Acoustic Components
2. Fingerprinting Microphones
3.
Fingerprinting
Both Speakers
and Microphones
In these scenarios the attacker has stealthy convinced the user to provide
access to
microphone.
Slide13November 4, 2014
13
Acoustic Features#FeatureDimension1Root Mean Square12
Zero Crossing Rate13Low-Energy-Rate14Spectral Centroid15Spectral Entropy16Spectral Irregularity1
7
Spectral Spread18
Spectral Skewness1
9
Spectral Kurtosis
1
10
Spectral
Rolloff
1
11
Spectral Brightness
1
12
Spectral Flatness
1
13
Mel
-
Frequency-
Cepstral
-Coefficients
13
14
Chromagram
12
15
Tonal Centroid
6
We investigate
a total of
15
acoustics
features
Slide14November 4, 2014
14
Feature Selection
Are there any dominant features?
Identifying the dominant feature set benefits us in two ways:
Less computation (potentially shifting the feature extraction component into mobile devices)
Improve accuracy as there might be conflicting features
Feature Reduction (Dimensionality reduction)
Feature Extraction
[new features=
function
(old features
)]
e.g., PCA, LDA
Feature Selection
[subset of old features]
Feature selection is preferable to feature extraction
when
dimensionality
and
numerical transformations
of the features are
inappropriate.
Slide15November 4, 2014
15
Sequential Feature Selection
We adopt a well-known machine learning technique called sequential forward selection (SFS). It is a greedy approach where features are added only if it increases accuracy.
We found
Mel-Frequency-
Cepstral-Coefficients (MFCCs) as
the dominating
features.
MFCCs:
Compactly represent the spectrum along the
nonlinear
mel
-scale
of frequency.
Distinguish the
low and fast varying
spectral envelopes of the signal.
Slide16November 4, 2014
16
Evaluation Algorithms & Metrics
We evaluate using two classification algorithm:k-NN: k-nearest neighborsGMM: Gaussian Mixture Model
Evaluation metrics
:
We
compute the average across all
classes. We use a
1:1
ratio for training and testing the classifiers.
TP: True Positive
FP: False Positive
FN: False Negative
Slide17November 4, 2014
17
MakerModel#AppleiPhone 51HTCNexus One14SamsungNexus S
8Galaxy S33Galaxy S410MotorolaDroid A85515Sony EricssonW5181Total52
Different Make & Model Sets
We consider one set from each make & model, giving us a total of 7 different sets.
So we can accurately distinguish smartphones of different make and model.
Slide18Maker
Model
#AppleiPhone 51HTCNexus One14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515
Sony EricssonW5181Total52November 4, 201418
Same Make & Model Sets
Fingerprinting both the speaker and microphone seems to provide better results.
We consider 15 Motorola Droid A855 handsets.
Slide19Maker
Model
#AppleiPhone 51HTCNexus One14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515
Sony EricssonW5181Total52November 4, 201419
Heterogeneous Smartphones
We develop an Android App to collect audio samples from 50 Android sets. In this case the audio clip is played
and
recorded on the phone set.
We were able to correctly classify ~98% of the recorded audio clips.
Slide20November 4, 2014
20
Sensitivity Analysis
We analysis the following factors: Sampling rateTraining set sizeDistance between speaker and microphoneAmbient background noise
We only consider
same make and model
handsets for the following set of experiments (as this is a
harder problem
)
We analyze how different factors impact our fingerprinting accuracy to better understand the feasibility of our approach.
Slide21November 4, 2014
21
Impact of Distance
We vary the distance between the speaker and microphone.
After a distance of
2 meters
the accuracy tend to go down at a faster rate.
We used an Audio-
Technica
ATR-6550
shotgun
microphone (~$45) for this experiment
Slide22November 4, 2014
22
Impact of Background NoiseEnvironmentSNR (dB)GMMShopping Mall15.84 (16%)94.2Restaurant17.77 (13%)91.6City Park15.43 (17%)94.6
Airport Gate14.92 (18%)93.9
We emulate four types of ambient background noise using external speakers.
Even with relative amount of background noise we can successfully fingerprint smartphones.
2 Meters
Played audio from the instrument category
Slide23November 4, 2014
23
Discussions
Providing a counter measure against such a side channel attack is still a open research question. One could attempt to shift the frequencies of the audio signal in a way so as to not deter the quality of the audio stream too much. But a thorough analysis of the impact of such a counter measure is required to fully understand its feasibility as audio streams are used for legitimate purposes too.
Slide24November 4, 2014
24
Conclusion
We see that it is possible to fingerprint smartphones through embedded microphones and speakers.We were able to accurately attribute ~98% of all recorded audio excerpts from 50 different Android devices.
So, the next time you talk to
Tom
keep in mind whether you are giving Tom a
fingerprint of your device
.
http://web.engr.illinois.edu/~
das17/SmartphoneFingerprint.html
More details of the project is available at the following link-
Slide25November 4, 2014
25
The End
November 4, 2014 Anupam
Download Presentation - The PPT/PDF document "November 4, 2014 Anupam" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.