/
Challenges and Opportunities Challenges and Opportunities

Challenges and Opportunities - PowerPoint Presentation

sherrill-nordquist
sherrill-nordquist . @sherrill-nordquist
Follow
401 views
Uploaded On 2018-01-12

Challenges and Opportunities - PPT Presentation

Andrew Wicker Machine Learning for Cloud Security Security is a top concern when migrating to the cloud Attacks can cause irreparable damage Different industries with targeted attacks Types of Attacks ID: 622825

attacks security triage domain security attacks domain triage data detections challenge signals outliers events automated attack risk rules compromise user evolving results

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Challenges and Opportunities" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Challenges and OpportunitiesAndrew Wicker

Machine Learning for Cloud SecuritySlide2

Security is a top concern when migrating to the cloud

Attacks can cause irreparable damage

Different industries with targeted attacksTypes of Attacks:Data BreachesLeaked CredentialsMalicious InsidersAPI VulnerabilitiesAdvanced Persistent Threats…

Cloud SecuritySlide3

Detecting attacks is nontrivial

Tremendous effort to maintain current state of security

Even more to detect new attacksRed Queen’s Race

Red Team

Blue TeamSlide4

No longer assume we are immune!We can not prevent human error

Phishing is still incredibly effective

Assume BreachSlide5

What can we do to make progress?Slide6

Challenge 1: Outliers to Security EventsSlide7

Finding statistical outliers is easyFinding anomalies requires a bit more domain knowledge

Making the leap to security event is challenging

Outliers to Security EventsSlide8

Simple changes in behavioral patterns are insufficient

Typically lead to high false positive rate

File access activity:User accesses one team’s files exclusivelySuddenly accessing team files from different division within companyRisky? Compromise?Uninteresting Behavioral AnomaliesSlide9

Use domain experts to make the leap“Tribal knowledge”

Credential scanning patterns

Storage compromise patternsSpam activity patternsFraudulent account patternsDomain ExpertiseSlide10

Use threat intelligence data to improve signalsBenefits

Indicators of Attack

Indicators of CompromiseIndustries targetedIP reputationThreat IntelligenceSlide11

Rules help filter noise from interesting security events

Sources:

Domain expertsTI feedsEasy to understandDifficult to maintain!Be careful relying too much on rulesEmbrace RulesSlide12

Top-level:

Bottom-level:

Incorporating Rules

Action

OS

IP

App

IsHighRisk

AccessFile

Windows 10

102.13.19.54

Excel

No

ModifyFile

Windows 8.1

23.12.16.65

Browser

No

AddGroup

OS X

74.23.76.12

Browser

No

UploadFile

Windows 10

91.25.46.5

SyncClient

No

AddAdmin

Windows 10

104.43.23.7BrowserYes

If Action is in RiskyActions, then Flag as HighRisk.Slide13

Basic

Advanced

Sophistication of Signals

Security Domain Knowledge

Usefulness of Alerts

Outliers

Anomalies

Security Events

Less Useful

More UsefulSlide14

Challenge 2: Everything is in FluxSlide15

Frequent/irregular deploymentsNew services coming online

Usage spikes

Evolving LandscapeSlide16

Constantly changing environments leads to constantly changing attacks

New services

New features for existing servicesFew known instances of attacksLack of labelled dataEvolving AttacksSlide17

Performance fluctuations of training/testing

Important for RT/NRT detections

Concept DriftData distributions affected by service changesMonitorsUnderstand the “health” of security signalsML ImplicationsSlide18

Don’t throw out your old detections

Old attacks can be reused, esp. if attackers know monitoring is weak

Signals are never “finished”Must update to keep up with the evolving attacksMake New Detections, But Keep the Old!Slide19

Challenge 3: Model ValidationSlide20

Recap:Lack of labeled dataFew known compromises, if any

Changing infrastructure

Service usage fluctuationsSo, how do we validate our models?Model ValidationSlide21

As always, metric selection is critical

Precision-Recall curve vs ROC curve

How do we define “false positive”?Augment dataWhat’s Your Precision and Recall?Slide22

Domain experts provide:Known patterns

Insights into what potential attacks might look like

Inject automated attack dataEvaluate metrics against this injected dataAttack AutomationSlide23

Do not naïvely optimize for automated attacksPrecision vs Recall

Many events generated by automated attacker may be benign

Be careful if labeling all automated attack events as positive labelLean toward precision instead of recallAttack Automation - CaveatSlide24

Human analysts provide feedback that we can use to improve our models

Feedback LoopSlide25

Challenge 4: Understanding DetectionsSlide26

Surfacing a security event to an end-user can be useless if there is no explanation

Explainability

of results should be considered at earliest possible stage of developmentBest detection signal with no explanation might be dismissed/overlookedUnderstanding DetectionsSlide27

Results without Explanation

UserId

Time

EventId

Feature1

Feature2

Feature3

Feature4

Score

1a4b43

2016-09-01 02:01

a321

0.3

0.12

3.9

20

0.2

73d87a

2016-09-01 03:15

3b32

0.4

0.8

0

11

0.09

9ca231

2016-09-01 05:10

8de20.80.349.27…0.95e91232016-09-01 05:3291de2.50.85

7.62.1…0.71e6a7b2016-09-01 09:122b4a3.10.833.66.2…0.1

33d693

2016-09-01 14:43

3b89

4.1

0.63

4.7

5.1

0.019

7152f3

2016-09-01 19:11

672f

2.7

0.46

3.9

1.4

0.03

Good luck!Slide28

Textual description“High speed of travel to an unlikely location”

Supplemental data

Rank ordered list of suspicious processesVariable(s)Provide one or more variables that impacted score the mostAvoid providing too many variablesHelpful ExplanationsSlide29

Detections must results in downstream actionGood explanation without being actionable is of little value

Examples

Policy decisionsReset user passwordActionable DetectionsSlide30

Challenge 5: Burden of TriageSlide31

Someone must triage alertsMore signals => More triaging

Many cloud services

And each must be protected against abuse/compromiseBurden of TriageSlide32

Flood of uncorrelated detectionsLack of contextual information

Dashboards!Slide33

Consolidate SignalsSlide34

Reduce burden of triage via integrated risk score

Combine relevant signals into a single risk score for account

Allows admin to set policies on risk score instead of triaging each signalIntegrated RiskSlide35

Outliers to Security EventsEverything is in Flux

Model Validation

Understanding DetectionsBurden of TriageSummarySlide36