Does Security Compliance Make Any Difference? A Case Study - PowerPoint Presentation

Does Security Compliance Make Any Difference?A Case StudySAIAF Meeting – April 26, 2019 Robert Stiles IT Auditor, Texas Department of Family Protective Services Robert.stiles@dfps.state.tx.us

Introduction Robert StilesCISA, CISSP, CFE, Certified FAIRMSc. of Identity Management & Security from University of Texas at Austin

Objectives How do organizations trust each other’s information security?Organizations as Complex Systems A Case Study Results How do we make things better?

The Good and Bad of Information Security ComplianceAnd a hint at the ugly.

The Good Organizations need a method to trust one another when exchanging data or sharing IT resourcesA Trust Framework helps this happen with common rules of behavior. Increases efficiencies, enables processes Examples: Credit Card participants – Payment Card Industry Digital Security Standard Cloud service providers to US Government – FEDRAMP Health care business associates – HITRUST Catalogues of control make for efficient validations, and assessment economies of scale

The BadOne organization may have to comply with multiple frameworks with different standardsControl catalogues mooovvveee sllloooooowwwwwllllyyy (the eight character password) The Market for Lemons in Assessments Unable to judge quality, it’s a race to the bottom

The Ugly They don’t work

Organizations as Complex Adaptive Systems

Complex Adaptive SystemsSelf-similarity, chaos, and the butterfly effect.Every human in an organization has their own set of schema or rules of behavior So does every team / department / division They communicate with each other and establish “shadow systems” These systems are non-linear. Everything is headed toward chaos - and through chaos, adaptation .

Heierarchical v. Complex How we think it works How it actually works

Evolution to the Edge of Chaos“The behavior of some simple, deterministic systems can be impossible, even in principle , to predict in the long term.” Complexity: A Guided Tour , M. Mitchell (2009) “The concurrency of multiple, and often conflicting performance measures and reward structures, which define the goals that decision makers attend to is a central characteristic of real organizations.” Models of complex adaptive systems in strategy and organization research (O. Baumann, Mind & Society , 2015) It’s not always a bad thing. But do it wrong and you die.

The Case StudyXYZ Corp faces some new challenges.

History of XYZFounded around 40 years agoStable management, stable governance BOD appointed by government body Oversight from State & Federal government. Provides services to Federal Agency Culture was compliance-focused No so much risk-adverse, but risk-”unaware” Federal law passes that will lead to slow decline in XYZ’s revenue (at that time around $50 million annual) (2010) Headcount around 700, 30% of headcount IT (2014) Stability, then a forced change

IT & Security BackgroundInternet interfaces to institutional and individual customersPrior “laptop lost by contractor” data breach incident (2006) fading from institutional memory InfoSec team – 7 staff, 1 CISO who reported to the executive management leader for IT. Implementing ISO & SP 800-53 controls as guidelines, but started to prepare serious FISMA compliance in advance of federal contracting initiative. IT Departments include Application Development, Enterprise Architecture, and Project Management office, as well as Network, Desktop, and other infrastructure support.

The Big ChangeChange in governance from public to private non-profitBoard re-writes by-laws, reduces transparency. The CEO leaves, and the new CEO restructures Creates New Division – the planned source of new revenue Lay offs target the IT, eliminating PMO, EA, and most application support Security team cut in half; staff freshly trained in FISMA gone IT now reports to new CFO. Spring 2015

Surprise!The Federal Agency says that XYZ must now comply with FISMA while it service out the legacy program. New XYZ management does not allocate more money to FISMA compliance. IT Performance Goal is “Reduce Legacy Support Budget” Summer 2015

So what happens next? Will XYZ’s security outcome improve or decline? Will it become FISMA compliant and get an ATO? Should XYZ’s business partners and the Federal Agency have more or less TRUST in XYZ’s data security after it completes the FISMA assessment?

How can we tell?Even though there were no security metrics in place, there’s always more data than you think.

Money. Corporate RevenueCorporate Expenses IT Budget Headcount Spending on Cloud Calculated IT Intensity (Percentage of revenue budgeted for IT) And people too.

IT & Security Outcomes StatsPenetration Test ResultsInternal Audit FindingsSecurity and Privacy Incidents SPAM & Phishing reports Security Quiz Results

Cutting Budget vs. Pen Test ResultsThe drop in IT Intensity corresponds to a rise in pen-test findings Does this indicate a real change in security outcomes? The rate of incidents doesn’t follow the same pattern. Let look at the data a little closer.

The Pen Test ResultsBounced back, but some warning signs.

  2012 2013 2014 2015 2016 2017 E xt e r n a l C r i t i ca l a nd H igh Findings   0   0   2   1   6   3 We b C r iti ca l a nd H igh Findings   0   2   1   1   3   3 A ll C r iti ca l a nd H igh Findings   0   2   3   2   9   6 A ll We b Findings   12   5   43   46   54   42 A ll E xt e r n a l Findings   19   25   46   49   53   38 A ll P e n e t r a tion T e s t Findings   31   30   89   95   107   80

Incidents

SIDEBAR:Does Awareness Work?The total number of incidents reported hit its lowest level the year of the big layoff. The Business Email Compromise Attempt & Information Security’s response.

The IntangiblesReview of Board Meeting MinutesInterviews:Head of IT Head of Internal Audit Security Staff HR Staff Cloud Strategy Chaos engines

The Intangibles & the Complex System: IT in a constant battle with business operationsCan’t meet the level of service it used to deliver The Board is getting interested in information security Only two agenda items on infosec between 2010 to 2014 (one was approval of a consultant contract, but a regular agenda item beginning in 2016. IT communication to board on infosec is controls-based No real institutional understanding of IT risk Risk as general not a part of board or managmenet conversations. Attempt at establishing an ERM stalled. All new IT to the Cloud But on-premises is cheaper than FEDRAMP compliance

How Compliant Are They?

The FISMA Assessments XYZ, on suggestion from Department, hired a 3rd Party Assessment team to evaluate the level of FISMA Compliance 2016 Assessment of all MODERATE controls listed in Appendix F of NIST SP 800-53 rev 4. 2017 Assessment of 80 “Key Controls” (as defined by assessment firm) + 1/3 of remaining controls

Any guess on what the assessment said?Hint: It was about 87% banana pants nonsense

    2016 A ss e ss m e nt   2016 V u l n era b ilit y C ount R a ti o of C on t r o l s A ss e ss e d t o R i sks   2017 A ss e ss m e nt   2017 V u l n era b ilit y C ount R a ti o of C on t r o l s A ss e ss e d t o R i sks C on t r o l s re v i ewe d   262       143     Ri sks – H i gh   23   23   9%   1   264   0 . 6 % Ri sks - M e d i um   15   7   6%   4   218   3 % Ri sks – L ow   14   0   5%   3   101   2% A l l R i sks 52 30 20 % 8 583 5% P OA M I t e m s   54       8     R e p ea t P OA M I t e m s   N /A       5    

Second Assessment Made Nobody HappyThe second assessment team was different, and less experienced, than the first team. “They took our internal scan, and threw it back at us.” – CISO The format of reports between 2016 and 2017 were different, with different uses of the terms “risk” and ”vulnerability.” Rating of “risk” was made based on CVSS score, or “judgment” without regard to asset value or compensating controls. Security team admits they should have more closely monitor and coordinate with the secpmd assessment team.

Does compliance improve outcomes or trust?

OutcomesPositives Pen Test Results Recover, and new processes emerge New Engagement from Board of Directors Problems Employees disengaged from security (fewer reporting incidents) XYZ Internal Audit tested some of the common controls – Some controls existed (and marked good by assessor) but were not effective.

TrustPositives The Federal Agency never asked for the reports, and never used them as a criteria for an ATO. Problems Inconsistent from year to year Did not identify or report relevant gaps in security Reported “HIGH” vulnerabilities that were likely at most medium

So what can audit do?

Control catalogue-based compliance is a small part of the story.Controls are only a component of a risk equation. Measure the value of the assets, the strength of the threat, and the frequency of attack

Manage and monitor 3PAs of your org closelyBe ready to support compensating controlsBe ready to explain your agency’s approach to IT risk Don’t rely on their work unless you can see their workpapers Monitor the skill quality of 3PA personnel (may change from year to year)

Quantify risk whenever possible. OK Better

Thanks. See you in 2043!Robert.Stiles@dfps.state.tx.us