November 4, 2014 Anupam PowerPoint Presentation, PPT - DocSlides

November 4, 2014

Anupam

Das (UIUC), Nikita Borisov (UIUC),Matthew Caesar (UIUC)

Do You Hear What I Hear? Fingerprinting Smart Devices Through Embedded Acoustic Components

1

CCS 2014

November 4, 2014

2

Smartphone Usage

How many people use smartphones?Source: Gartner

Shipment of smartphone is increasing every year.

75%

of the mobile phones are smartphones and highly-featured phones

Source: Business Insider

November 4, 2014

3

A Closer Look at Smartphones

Today, smartphones come with a wide range of sensors. All of which are useful for a variety of tasks.

However, sensors could also be potential source for unique fingerprints.

Motion detection

Gesture detection

Audio Genre detection

Location

detection

Interaction with nearby devices

Compass

November 4, 2014

4

Why Fingerprint Smartphones?Software based approaches:Browser based features, CookiesDifferent Firmware and Device driversHardware based approaches:

Clock Skew rateRadio TransmitterAccelerometerSmartphones can be fingerprinted for:Targeted AdvertisementSecondary Authentication factor

The main drawback

for software based approaches is that the source of the fingerprints are

not too stable

.

Moreover, device IDs are not always usable. UDID

for

Apple devices has been

removed

since May 1, 2013 and IMEI

for

an Android Device requires explicit permission.

November 4, 2014

5

Our Goal

Scenario 1: External attacker locally present1. Attacker records audio signal from distance2. Create a fingerprint of the recorded audio and link the fingerprint to a unique smartphone

Scenario 2: Stealthy App

App

1.Play audio

2. Audio signal

3.Record audio

4. Extract Fingerprint

Need access to only:

Microphone

Internet connection

We focus on fingerprinting

smart devices

through their embedded

speakers

and

microphones

.

Requires:

Deployed microphones

November 4, 2014

6

Top Android Permissions Source: Mario Frank, Ben Dong, Adrienne Porter Felt, Dawn Song. Mining Permission Request Patterns from Android and Facebook Applications, ICDM, 2012 1

Storage : modify/delete USB storage and SD card contents2Network communication : full Internet access

3

Network communication : view network state

4

System tools : prevent device from sleeping

5

Phone calls : read phone state and identity

6

Hardware controls : control vibrator

7

System tools : automatically start at boot

8

Network communication : view Wi−Fi state

9

Your location : fine (GPS) location

10

Your location : coarse (network−based) location

11

System tools : retrieve running applications

12

Your personal information : read contact data

13

Your messages : read SMS or MMS

14

Your messages : receive SMS

15

Hardware controls : take pictures and videos

16

Hardware controls : record audio

17

System tools : modify global system settings

Top 17 Android permissions out of 173 total permissions

November 4, 2014

7

4,405,876Real World Apps

There are popular apps that people use that require similar permissions. For example: My Talking Tom

November 4, 2014

8

Closer Look at Microphones

MEMS microphone:Source: STMicroelectronics

Sound Wave

Mechanical Energy

Capacitive Change

The sensitivity of the microphone depends on how well the

diaphragm

deflects to acoustic

pressure

Imperfections

can

arise due to:

Slight

variations in the chemical

composition

of components from one batch to the

next

Wear and tear

in the

manufacturing

machines

Changes

in temperature and humidity

Voltage Change

November 4, 2014

9

Basic functionality of a speaker :

Closer Look at SpeakersSource: Center Point AudioElectrical current

Magnetic field around voice coil

Mechanical Energy

Sound Wave

The

frequency

of the sound wave produced

is

dictated

by

the

rate

at

which the

voice coil

moves.

November 4, 2014

10

Experimental SetupMakerModel#AppleiPhone 51HTCNexus One

14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515Sony EricssonW5181

Total

52

Audio Type

Description

Variations

Instrumental

Musical instruments playing together

4

Human Speech

Small

segments of human speech

4

Song

Combination

of human voice & instrumental sound

3

To

emulate an

attacker we use a laptop

November 4, 2014

11

Fingerprinting Acoustic Components

1. Fingerprinting Speakers

This scenario is suited for the case where the attacker has

deployed

nearby

microphones

to capture

audio signal

(e.g.,

ringtone) from user device.

November 4, 2014

12

Fingerprinting Acoustic Components

2. Fingerprinting Microphones

3.

Fingerprinting

Both Speakers

and Microphones

In these scenarios the attacker has stealthy convinced the user to provide

access to

microphone.

November 4, 2014

13

Acoustic Features#FeatureDimension1Root Mean Square12

Zero Crossing Rate13Low-Energy-Rate14Spectral Centroid15Spectral Entropy16Spectral Irregularity1

7

Spectral Spread18

Spectral Skewness1

9

Spectral Kurtosis

1

10

Spectral

Rolloff

1

11

Spectral Brightness

1

12

Spectral Flatness

1

13

Mel

-

Frequency-

Cepstral

-Coefficients

13

14

Chromagram

12

15

Tonal Centroid

6

We investigate

a total of

15

acoustics

features

November 4, 2014

14

Feature Selection

Are there any dominant features?

Identifying the dominant feature set benefits us in two ways:

Less computation (potentially shifting the feature extraction component into mobile devices)

Improve accuracy as there might be conflicting features

Feature Reduction (Dimensionality reduction)

Feature Extraction

[new features=

function

(old features

)]

e.g., PCA, LDA

Feature Selection

[subset of old features]

Feature selection is preferable to feature extraction

when

dimensionality

and

numerical transformations

of the features are

inappropriate.

November 4, 2014

15

Sequential Feature Selection

We adopt a well-known machine learning technique called sequential forward selection (SFS). It is a greedy approach where features are added only if it increases accuracy.

We found

Mel-Frequency-

Cepstral-Coefficients (MFCCs) as

the dominating

features.

MFCCs:

Compactly represent the spectrum along the

nonlinear

mel

-scale

of frequency.

Distinguish the

low and fast varying

spectral envelopes of the signal.

November 4, 2014

16

Evaluation Algorithms & Metrics

We evaluate using two classification algorithm:k-NN: k-nearest neighborsGMM: Gaussian Mixture Model

Evaluation metrics

:

We

compute the average across all

classes. We use a

1:1

ratio for training and testing the classifiers.

TP: True Positive

FP: False Positive

FN: False Negative

November 4, 2014

17

MakerModel#AppleiPhone 51HTCNexus One14SamsungNexus S

8Galaxy S33Galaxy S410MotorolaDroid A85515Sony EricssonW5181Total52

Different Make & Model Sets

We consider one set from each make & model, giving us a total of 7 different sets.

So we can accurately distinguish smartphones of different make and model.

Maker

Model

#AppleiPhone 51HTCNexus One14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515

Sony EricssonW5181Total52November 4, 201418

Same Make & Model Sets

Fingerprinting both the speaker and microphone seems to provide better results.

We consider 15 Motorola Droid A855 handsets.

Maker

Model

#AppleiPhone 51HTCNexus One14SamsungNexus S8Galaxy S33Galaxy S410MotorolaDroid A85515

Sony EricssonW5181Total52November 4, 201419

Heterogeneous Smartphones

We develop an Android App to collect audio samples from 50 Android sets. In this case the audio clip is played

and

recorded on the phone set.

We were able to correctly classify ~98% of the recorded audio clips.

November 4, 2014

20

Sensitivity Analysis

We analysis the following factors: Sampling rateTraining set sizeDistance between speaker and microphoneAmbient background noise

We only consider

same make and model

handsets for the following set of experiments (as this is a

harder problem

)

We analyze how different factors impact our fingerprinting accuracy to better understand the feasibility of our approach.

November 4, 2014

21

Impact of Distance

We vary the distance between the speaker and microphone.

After a distance of

2 meters

the accuracy tend to go down at a faster rate.

We used an Audio-

Technica

ATR-6550

shotgun

microphone (~$45) for this experiment

November 4, 2014

22

Impact of Background NoiseEnvironmentSNR (dB)GMMShopping Mall15.84 (16%)94.2Restaurant17.77 (13%)91.6City Park15.43 (17%)94.6

Airport Gate14.92 (18%)93.9

We emulate four types of ambient background noise using external speakers.

Even with relative amount of background noise we can successfully fingerprint smartphones.

2 Meters

Played audio from the instrument category

November 4, 2014

23

Discussions

Providing a counter measure against such a side channel attack is still a open research question. One could attempt to shift the frequencies of the audio signal in a way so as to not deter the quality of the audio stream too much. But a thorough analysis of the impact of such a counter measure is required to fully understand its feasibility as audio streams are used for legitimate purposes too.

November 4, 2014

24

Conclusion

We see that it is possible to fingerprint smartphones through embedded microphones and speakers.We were able to accurately attribute ~98% of all recorded audio excerpts from 50 different Android devices.

So, the next time you talk to

Tom

keep in mind whether you are giving Tom a

fingerprint of your device

.

http://web.engr.illinois.edu/~

das17/SmartphoneFingerprint.html

More details of the project is available at the following link-

November 4, 2014

25

The End