Andrew Ruef Michael Hicks Dave Levin James Parker Atif Memon Jandelyn Plane P eter Mardziel Whats the motivation What goes into secure software development How could we measure and contrast different styleslanguages ID: 705375
Download Presentation The PPT/PDF document "Build It Break It Fix It" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Build It Break It Fix It
Andrew Ruef
, Michael Hicks, Dave Levin, James Parker, Atif Memon, Jandelyn Plane, P
eter MardzielSlide2
What’s the motivation?
What goes into secure software development?
How could we measure and contrast different styles/languages?
How do we teach people to write secure code?
What exercise will let people observe both building and breaking?Slide3
What’s our idea?
A contest where contestants
Build
some secure software according to a specification
Break
the software written by other contestants
Fix
the bugs found in their software by other
Organizers provide the specification
Spread the contest over three weekends
Each phase takes one weekend
Announce two winners, one for best software, one for most bugs foundSlide4
Challenge specifications
Needs to be at least a little fun
Have high and low level security properties
Writing in Java or Python should not win by default
Judge implementations on both correctness and performance
Capable of unambiguously testing features
Should be somewhat complicated, but doable in 72 hoursSlide5
Fall 2014, alarm system
Two programs,
logappend
,
logread
, manipulate a secure log file to either add events or query events
Both programs authenticate to each other via a single shared symmetric key
Programs that run faster are better
Smaller log file size is better
Slide6
Fall 2015, bank and ATM
Two programs,
bank
,
atm
, communicate over a TCP stream to manage an account at a bank
Programs authenticate
accounts
via an opaquely specified card file
Programs that run faster are better
Less data in transit is better
Slide7
Three different types of bugs
Correctness
– The program didn’t meet some part of the specification, or crashes
Integrity
– Data can be modified to attest to a false fact
Confidentiality
– Data can be analyzed to determine a protected fact
We can automatically judge correctness and integrity bugs
Integrity, confidentiality, and a correctness bug that produces a crash are counted as
exploitsSlide8
Infrastructure
This is still a hacking competition, it would be nice to not be compromised by our contestants
Interface with contestants
A Haskell webapp
Run contestant code
An EC2 backend to run every test in its own container
Slide9
What were the results?
We ran the contest over September 2014
Out of 90 registered
teams
with over 180 registered
individuals
, we had
20
teams attempt to
submit
something
11
teams submit code that passed core tests
Successful submissions in Go, Haskell, Python, Java, C, and C++
Some failed submissions in RubySlide10
Scores over timeSlide11
Break-It roundSlide12
Overall winners Fall 2014
First place build-it languages
Python
Haskell
First place break-it team wrote in Go (and was third in build-it)Slide13
Bug finding strategies
First place break-it winner did everything with manual auditing
Second place used some fuzzing
One team reported repurposing testing infrastructure they used during build-itSlide14
What do we think about it?
Memory safety helped but was not sufficient
This is an important property for the competition
Strong static typing helped but was not enough
Python still wound up beating Haskell and Go
There might be some interesting properties in the programs we already have Slide15
Conclusion
Our contestants had fun and learned about security
We measured peoples ability to both find bugs and write code
We amplified one CTF problem into N
We’ll do it again