An Interview Study on Third-Party Cyber Threat
Description: An Interview Study on Third-Party Cyber Threat Hunting Processes in the U.S. Department of Homeland Security James C. Davis Purdue University William P. Maxam III US Coast Guard To be presented at USENIX Security 2024 Paper! Lab website
Related Topics
Download Presentation
"An Interview Study on Third-Party Cyber Threat" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. An Interview Study on Third-Party Cyber Threat Hunting Processes in the U.S. Department of Homeland Security James C. Davis
Purdue University William P. Maxam III
US Coast Guard To be presented at USENIX Security 2024 Paper! Lab website<br>
slide2. Talk overview 2 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide3. Talk overview 3 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide4. If hacked, hopefully you discover the break-in 4 Adversary infiltrates the network Adversary is discovered Adversary is evicted<br>
slide5. Dwell time: How long did it take you to discover the adversary? 5 Adversary infiltrates the network Adversary is discovered Adversary is evicted Dwell Time<br>
slide6. Average dwell times are ~200 days 6 2020 2019 2018 2017 2016 2015 Dwell time (days) Eviction time (days) 0 50 100 150 200 250 Time to detect (blue) and evict (purple) adversaries<br>
slide7. Three common paths to adversary discovery (3) Detection (4) Response (2) Compromise (1) Prevention<br>
slide8. Third-party Threat Hunt in private sector and government 8<br>
slide9. Existing Process Models of Threat Hunt: TaHiTI 9<br>
slide10. Example Threat Hunt Frameworks: Kill Chain, ATT&CK 10 … …<br>
slide11. Talk overview 11 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide12. Gap #1: In practice, TH is ad hoc 12 Around half of surveyed organizations use “ad hoc” hunting (No defined process)<br>
slide13. Gap #2: Ad hoc is ineffective if your hunters keep leaving 13 Time in role for cyber security analysts (US data)<br>
slide14. Theme 1: Process
What processes are currently used by 3rd-party government TH teams?
What challenges do they observe in these processes?
Theme 2: Addressing turnover
What factors indicate expertise?
How are new members integrated? Research questions 14<br>
slide15. Talk overview 15 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide16. Interview protocol 16<br>
slide17. Subject recruitment and demographics 17<br>
slide18. Talk overview 18 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide19. Range of TH process diagrams 19<br>
slide20. (Simplified) Unified TH model 20<br>
slide21. (Full) Unified TH model<br>
slide22. Challenges: Process 22<br>
slide23. Challenges: New members 23<br>
slide24. Talk overview 24 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide25. Lessons learned for 3rd-party government TH teams in the DHS Improve planning
Revisit the automated alert loop
Formalize apprenticeship
Many open questions remain
Especially, sharing across DOD, Sandia, etc.? Paper Lab website<br>
slide26. Bonus slides<br>
slide27. NB: Dwell times: Averages != medians – Still, not good 27 Attacker dwell time (days from compromise to detection) Inverse cumulative percentage (N=65) Mean (Avg): 125 days (18 weeks)
Median: 7 days<br>
slide28. The data is saturated (so we could stop interviewing) 28 Subject ID Cumulative
unique codes observed Saturation # Codes Unique codes by subject # Codes Subject ID<br>
slide29. No observed variation in themes by job role 29 # mentions by subjects<br>
slide30. Bianco’s Pyramid of Pain – Another framework for TH<br>
slide31. Threat Hunt maturity model (Bianco) 31<br>
slide32. Closest military equivalent: cognitive modeling for cyber protection teams/CPTs (Trent et al.) 32<br>
Purdue University William P. Maxam III
US Coast Guard To be presented at USENIX Security 2024 Paper! Lab website<br>
slide2. Talk overview 2 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide3. Talk overview 3 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide4. If hacked, hopefully you discover the break-in 4 Adversary infiltrates the network Adversary is discovered Adversary is evicted<br>
slide5. Dwell time: How long did it take you to discover the adversary? 5 Adversary infiltrates the network Adversary is discovered Adversary is evicted Dwell Time<br>
slide6. Average dwell times are ~200 days 6 2020 2019 2018 2017 2016 2015 Dwell time (days) Eviction time (days) 0 50 100 150 200 250 Time to detect (blue) and evict (purple) adversaries<br>
slide7. Three common paths to adversary discovery (3) Detection (4) Response (2) Compromise (1) Prevention<br>
slide8. Third-party Threat Hunt in private sector and government 8<br>
slide9. Existing Process Models of Threat Hunt: TaHiTI 9<br>
slide10. Example Threat Hunt Frameworks: Kill Chain, ATT&CK 10 … …<br>
slide11. Talk overview 11 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Method
Results
Takeaways<br>
slide12. Gap #1: In practice, TH is ad hoc 12 Around half of surveyed organizations use “ad hoc” hunting (No defined process)<br>
slide13. Gap #2: Ad hoc is ineffective if your hunters keep leaving 13 Time in role for cyber security analysts (US data)<br>
slide14. Theme 1: Process
What processes are currently used by 3rd-party government TH teams?
What challenges do they observe in these processes?
Theme 2: Addressing turnover
What factors indicate expertise?
How are new members integrated? Research questions 14<br>
slide15. Talk overview 15 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide16. Interview protocol 16<br>
slide17. Subject recruitment and demographics 17<br>
slide18. Talk overview 18 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide19. Range of TH process diagrams 19<br>
slide20. (Simplified) Unified TH model 20<br>
slide21. (Full) Unified TH model<br>
slide22. Challenges: Process 22<br>
slide23. Challenges: New members 23<br>
slide24. Talk overview 24 Threat Hunt and the landscape of cyber ops
Knowledge gaps and research questions
Methods
Results
Takeaways<br>
slide25. Lessons learned for 3rd-party government TH teams in the DHS Improve planning
Revisit the automated alert loop
Formalize apprenticeship
Many open questions remain
Especially, sharing across DOD, Sandia, etc.? Paper Lab website<br>
slide26. Bonus slides<br>
slide27. NB: Dwell times: Averages != medians – Still, not good 27 Attacker dwell time (days from compromise to detection) Inverse cumulative percentage (N=65) Mean (Avg): 125 days (18 weeks)
Median: 7 days<br>
slide28. The data is saturated (so we could stop interviewing) 28 Subject ID Cumulative
unique codes observed Saturation # Codes Unique codes by subject # Codes Subject ID<br>
slide29. No observed variation in themes by job role 29 # mentions by subjects<br>
slide30. Bianco’s Pyramid of Pain – Another framework for TH<br>
slide31. Threat Hunt maturity model (Bianco) 31<br>
slide32. Closest military equivalent: cognitive modeling for cyber protection teams/CPTs (Trent et al.) 32<br>