Masquerade Detection - PowerPoint Presentation

Slide1

Masquerade Detection

Mark Stamp

1

Masquerade DetectionSlide2

Masquerade Detection

Masquerader --- someone who makes unauthorized use of a computer

How to detect a masquerader?

Here, we consider…

Anomaly-based intrusion detection (IDS)

Detection is based on UNIX commandsLots and lots of prior work on this problemWe attempt to apply PHMMsFor comparison, we also implement other techniques (HMM and N-gram)

Masquerade Detection

2Slide3

Schonlau Data Set

Schonlau

, et al, collected large data setContains UNIX commands for 50 users50 files, one for each user

Each file has 15k commands, 5k from user plus 10k

for masquerade test dataTest data: 100 blocks, 100 commands eachDataset includes map file100 rows (test blocks), 50 columns (users)0 if block is user data, 1 if masquerade dataMasquerade Detection

3Slide4

Schonlau Data Set

Map file structure

This data set used for many studies

Approximately,

50 published papers

Masquerade Detection4Slide5

Previous Work

Approaches to masquerade detection

Information theoreticText miningHidden Markov models (HMM)Naïve

Bayes

Sequences and bioinformatics

Support vector machines (SVM)Other approachesWe briefly look at each of theseMasquerade Detection5Slide6

Information Theoretic

Original work by Schonlau

included a compression techniqueBased on theory (hope?) that legitimate commands compress more than attack

Results were disappointing

Some additional recent work

Still not competitive with best approachesMasquerade Detection6Slide7

Text Mining

A few papers in this areaOne approach extracts repetitive sequences from training data

Another paper use principal component analysis (PCA)

Method of “exploratory data analysis”

Good

results on Schonlau data setBut high cost during training phaseMasquerade Detection

7Slide8

Hidden Markov Models

Several authors

have used HMMsOne of the best

known approaches

We have implemented HMM detector

We do sensitivity analysis on the parametersIn particular, determine optimal N (number of hidden states)We also use HMMs for comparison with our PHMM results

Masquerade Detection

8Slide9

Naïve Bayes

In simplest form, relies only on command frequencies

That is, no sequence info is usedSeveral papers analyze this approachAmong the simplest approaches

And, results are good

Masquerade Detection

9Slide10

Sequences

In a sense, this is the opposite extreme from naïve

BayesNaïve Bayes only considers frequency

stats

Sequence/bioinformatics focused on sequence-related information

Schonlau’s original work included elementary sequence-based analysisMasquerade Detection10Slide11

Bioinformatics

We

are aware of only one previous paper that uses bioinformatics approach

U

se

Smith-Waterman algorithm to create local alignmentsAlignments then used directly for detectionIn contrast, we do pairwise alignments, MSA, PHMMPHMM is used for scoring (forward algorithm)Our

scoring is much more efficientAlso, our results are at least as strong

Masquerade Detection

11Slide12

Support Vector Machines

Support vector machines (SVM)

Machine learning techniqueSeparate data points (i.e., classify) based on hyperplanes in high dimensional space

Original data mapped to higher dimension, where separation is likely easier

SVMs

maximize separationAnd have low computational costsUsed for classification and regression analysisMasquerade Detection12Slide13

SVMs

& Masquerade Detection

SVMs have been applied to masquerade detection problemResults are goodComparable to naïve

Bayes

Recent work using

SVMs focused on improved efficiencyMasquerade Detection13Slide14

Other Approaches

The following have also

been studiedDetect using low frequency commandsDetect using high frequency commands

Hybrid

Bayes

“one step Markov”Natural to consider hybrid approachesMultistep MarkovMarkov process of order greater than 1None of these particularly successfulMasquerade Detection

14Slide15

Other Approaches (Continued)

Non-negative matrix factorization (NMF)

At least 2 papers on this topicAppears to be competitive

Other

hybrids that attempt to combine several approachesSo far, no significant improvement over individual techniquesMasquerade Detection15Slide16

HMMs

See previous presentation

Masquerade Detection

16Slide17

HMM for Masquerade Detection

Using the

Schonlau data set we…Train HMM for each userSet thresholds

Test the models and plot results

Note that this has been done before

Here, we perform sensitivity analysisThat is, we test different number of hidden states, NAlso use it for comparison with PHMMMasquerade Detection

17Slide18

HMM Experiments

Plotted as

“ROC” curvesCloser to origin is betterUseful region

That is, false

positives below 5%

The shaded regionMasquerade Detection18Slide19

HMM Conclusion

Number of hidden states does not

matterSo, use N=2

Since most

efficient

Masquerade Detection19Slide20

PHMM

See previous presentation

Masquerade Detection

20Slide21

PHMM Experiments

A problem with

Schonlau data…For given user, 5000 commandsNo begin/end session markers

So,

must split it up to obtain multiple sequencesBut where to split sequence?And what about tradeoff between number of sequences and length of each sequence?That is, how to decide length/number???

Masquerade Detection

21Slide22

PHMM Experiments

Experiments done for following cases:

See next slide…

Masquerade Detection

22Slide23

PHMM Experiments

Tests various numbers of sequences

Best results5 sequences, 1k commands each seq.

This case in

next slide

Masquerade Detection23Slide24

PHMM Comparison

Compare PHMM to “weighted

N-gram” and HMMHMM is

best

PHMM

is competitiveMasquerade Detection24Slide25

PHMM Detector

PHMM at disadvantage

on Schonlau data

PHMM uses positional information

Such info not available

for Schonlau dataWe have to guess the positions for PHMMHow to get fairer comparison between HMM and PHMM?We need different data setOnly option is

simulated data set

Masquerade Detection

25Slide26

Simulated Data

We generate

simulated data as followsUsing Schonlau data, construct Markov chain for each user

Use resulting Markov chain to generate sequences

representing user behavior

Restrict “begin” to more common commandsWhat’s the point?Simulated seqs have sensible begin and end

Masquerade Detection

26Slide27

Simulated Data

Training data and user data

for scoring generated using Markov chainAttack data taken from Schonlau

data

How much data to generate?

First test, we generate same amount of simulated data as is in Schonlau setThat is, 5k commands per user Masquerade Detection27Slide28

Detection

with Simulated Data

PHMM vs

HMM

Round 2

It’s close, but HMM still wins!Masquerade Detection28Slide29

Limited Training Data

What if less training data is available?

In a real application, initially, training data is limitedCan’t detect attacks until sufficient training data has been accumulated

So, less data required, the better

Experiments, using simulated data,

limited training dateUsed 200 to 800 commands for trainingMasquerade Detection

29Slide30

Limited Training Data

PHMM

vs HMMRound 3With 400 or less,

PHMM wins big!

Masquerade Detection

30Slide31

Conclusion

PHMM

is competitive with best approaches

PHMM likely to do better, given better training data (begin/end info)

PHMM much better than HMM when limited training data available

Of practical importanceWhy does it make sense that PHMM would do better with limited training data?Masquerade Detection31Slide32

Conclusion

Given current state of research…

Optimal masquerade detection approachInitially, collect small training setTrain PHMM and use for detection

N

o attack, then continue to collect data

When sufficient data available, train HMMFrom then on, use HMM for detectionMasquerade Detection32Slide33

Future Work

Collect better real data set!!

!Many problems/limitations with Schonlau data

Improved data set could be basis for lots and lots of research

Directly compare PHMM/bioinformatics approaches with previous work (HMM, naïve

Bayes, SVM, etc., etc.)Consider hybrid techniquesOther techniques?Masquerade Detection33Slide34

References

Masquerade detection using profile hidden Markov models, L. Huang and M. Stamp, to appear in

Computers and SecurityMasquerading user data, M. Schonlau

Masquerade Detection

34