Presentations text content in Type-Based Data Structure Verification
Slide1
Type-Based Data Structure Verification
Ming Kawaguchi,
Patrick
Rondon
, Ranjit Jhala
University of California, San Diego
Slide2Goal: Static Software Verification
Verify absence of run-time errors
Buffer overflows
Deadlocks
Assertion failures
Requires precise data structure verification
Slide3The Problem With Structures
1
2
5
3
4
1
2
5
3
4
1
2
4
3
5
Unbounded Size
Need
Universally Quantified
Properties
“
Every element
has property
P
”
Slide4Contributions
Precise properties
of individual cells
Types
Predicates
Lift properties to
invariants on structures
Slide5Contributions
Recursive Structures
(Lists, Trees)
Table Structures
(Fields, Arrays, Hash Maps)Supports InferenceExpressive
Sorted, Duplicate-Free, Height-Balanced, Acyclic, …Practical Sorting Algorithms, Splay Heaps, Binary Heaps,
AVL Trees, Red-Black Trees, Vectors, Union-Find, BDDs, … Predicate
-Type Mechanisms
Type
Mechanisms
Refined
Slide6Plan
Contributions
Types
&
Structures
Refined Types
& Data StructuresExpressivenessResults
Slide7Types
&
Structures
How do
types
handle
structures?
Represent Universal
Properties
Algorithm for
Instantiating
Properties
Algorithm for
Generalizing
PropertiesAlgorithm for
Inference
Slide81. Representation: Recursive Types
1
2
3
1:
:(
2::(3::[]))
type
int
list
=
|
[]
|
:: of
x:int * int list
[]
::
x
:
int
Slide9[]
::
x
:
int
[]
::
h
1
:
int
[]
::
x
:
int
[]
::
h
2
:
int
[]
::
x
:
int
[]
::
h
3
:
int
[]
::
x
:
int
1. Representation:
Recursive Types
Type Unfolding
Universal
Property:
For all
x
in
l
,
x
is an
int
Universal
Property:
h
1
:
int
,
h
2
:
int
,
h
3
:
int
, …
Slide10Types
&
Structures
How do
types
handle
structures?
Represent Universal
Properties
Algorithm for
Instantiating
Properties
Algorithm for
Generalizing
PropertiesAlgorithm for
Inference
Slide112. Instantiation Algorithm:
Unfold
[]
::
x
:
int
Unfold
::
h
:
int
[]
::
x
:
int
l
:
int
list
h:
int
t:
int
list
Instantiate
t
l
l
=
h
::
t
Slide123. Generalization Algorithm: Fold
[]
::
x
:
int
Fold
::
h
:
int
[]
::
x
:
int
h:
int
t:
int
list
Generalize
t
l
l
=
h
::
t
l
:
int
list
Slide13Types
&
Structures
How do
types
handle
structures?
Represent Universal
Properties
Algorithm for
Instantiating
Properties
Algorithm for
Generalizing
PropertiesAlgorithm for
Inference
Slide14let
rec
insert(
x,l
) =
match l
with | [] ->
x::[] | h::t
->
if
x<h
then
x::l
else h::insert(
x,t)
insert :: (x:
int, l:
int list
) ! int list
Verification = Generalization + Instantiation
Ex: Typecheck Insertion Into List 1.2.
3.
Slide15Assume Input Type
Output Checks!
[]:
int
list
x:int
let
rec
insert(
x,l
) =
match
l
with
| [] ->
x::[] |
h::t -> if
x<h then x::l else
h::insert(x,t)
Generalize
x::[]:int list
Ex: Typecheck Insertion Into List 1/3
G
insert
::
(x:
int,
l: int list) !
int list1.
2.3.
Slide16Assume Input Type
Output Checks!
let
rec
insert(
x,l
) =
match
l with
|
[]
->
x::[]
|
h::t -> if
x<h then
x::l else
h::insert(x,t
)Ex: Typecheck Insertion into List
2/3Generalize
G
insert
:: (x: int
, l: int list)
! int
list
x:
int
l:int list
x::l:int
list1.2.
3.
Slide17Input Assumption
Output Checks!
let
rec
insert(
x,l
) =
match
l with
|
[]
->
x::[]
|
h::t -> if
x<h then
x::l else
h::insert(x,t
)
h::t:int
listEx: Typecheck Insertion into List 3/3
h:
intt:int list
I
Generalize
h
::
insert(
x,t
):int list
h:intinsert(
x,t):int list
G
Instantiate
insert
::
(x:
int
,
l:
int
list
)
!
int
list
x:
int
2.
3.
1.
Slide18let
rec
insert(
x,l
) =
match l
with | [] ->
x::[] | h::t
->
if
x<h
then
x::l
else h::insert(
x,t)
I
G
insert
::
(x:
int, l:
int list) !
int list
G
G
Verification = Generalization + Instantiation
Gen
when
adding to
struct. Ins when taking from struct.
Slide19Types
&
Structures
How do
types
handle
structures?
Represent Universal
Properties
Algorithm for
Instantiating
Properties
Algorithm for
Generalizing
PropertiesAlgorithm for
Inference
Slide20Plan
Contributions
Types
&
Structures
Refined
Types & Data StructuresExpressivenessResults
Slide21Idea:
“Piggyback”
Predicates
over
Type
Representation for
Universal Data
Properties
Algorithm for
Instantiating
Data
Properties
Algorithm for Generalizing Data PropertiesAlgorithm for
Inference
Refined Types
& Data Structures
Slide220<x
[]
::
x
:
int
[]
::
h
:
int
1. Representation:
Refined
RecTypes
Refined
Type Unfolding
[]
::
x
:
int
0<x
0<h
Slide230<x
0<x
[]
::
x
:
int
0<x
[]
::
h
1
:
int
[]
::
x
:
int
[]
::
x
:
int
[]
::
x
:
int
1. Representation:
Refined
RecTypes
0<h
1
0<h
2
0<h
3
0<x
[]
::
h
2
:
int
[]
::
h
3
:
int
Universal
Data
Property:
For all
x
in
l
,
0<x
Refined
Type Unfolding
Universal
Data
Property:
l:
{x:
int
| 0<x}
list
Slide241. Representation: RecRefined
RecTypes
[]
::
x
:
int
[]
::
x
1
:
int
[]
::
x
:
int
x<V
0<x
V
Refers to Each Element in Tail
Refines
all
elements
Refines
tail’s
elements
Slide25x<V
[]
::
x
:
int
[]
::
h
:
int
1. Representation:
RecRefined
RecTypes
RecRefined
Type Unfolding
[]
::
x
:
int
x<V
h<V
Push
Edge Predicate
Into
Node
h<x
Instantiate
V
Slide26h
2
<x
h
1
<x
h1<x
x<Vx<V
[]
::
x
:
int
x<V
[]
::
h
1
:
int
[]
::
x
:
int
[]
::
x
:
int
1. Representation:
RecRefined
RecTypes
h
1
<V
h
3
<x
[]
::
h
2
:
int
[]
::
h
3
:
int
RecRefined
Type
Unfolding
h
1
<h
2
h
2
<V
h
1
<h
3
[]
::
x
:
int
h
2
<h
3
h
3
<V
h
1
<x
h
2
<x
x<V
Universal
Recursive Data Property
h
1
<
h
2
<
h
3
< …
Universal
Recursive Data Property
l:
sorted
list
Slide272. Instantiation Algorithm:
Unfold
[]
::
x
:
int
Unfold
::
h
:
int
[]
::
x
:
int
l:
sorted
list
h:
int
t:
sorted
list
&
{h<x}
list
Instantiate
t
l
l
=
h
::
t
x<V
x<V
h<x
Slide283. Generalization Algorithm:
Fold
[]
::
x
:
int
Fold
h
:
int
[]
::
x
:
int
::
l:
sorted
list
h:
int
t:
sorted
list
&
{h<x}
list
Generalize
t
l
l
=
h
::
t
x<V
x<V
h<x
Slide29Refined
Types
&
Data
StructuresIdea:
“Piggyback” Predicates over Type
Representation for
Universal
Data
Properties
Algorithm for
Instantiating
Data Properties
Algorithm for Generalizing Data
PropertiesAlgorithm for Inference
Slide30Refinement Type Inference
[]
::
x
:
int
x<V
Refinements Determine Property
1.
Unknown
refinements are
variables
2.
Constraints
over
variables
3.
Solve
to find
refinements
0<x
K2K1
Slide31Hints for Type Inference
Apply at Refinement Points
Keep Only Valid Refinements
Refinement Type Inference
[]
::
x
:
int
Refinements Determine Property
1.
Unknown
refinements are
variables
2.
Constraints
over
variables
3.
Solve
to find
refinements
K
2K
1*<*0<**=*…
x<V
0<xx=V
…
x<V0<x
x<V0<xx=V…
Instantiate With Program Variables
Slide32Refined
Types
&
Data
StructuresIdea:
“Piggyback” Predicates over Type
Representation for
Universal
Data
Properties
Algorithm for
Instantiating
Data Properties
Algorithm for Generalizing Data
PropertiesAlgorithm for Inference
Free
Representation
Free Algorithms
Slide33let
rec
insert(
x,l
) =
match l
with | [] ->
x::[] | h::t
->
if
x<h
then
x::l
else h::insert(
x,t)
I
G
G
G
Verification = Generalization + Instantiation
insert
::
(x:
int
,
l:
sorted list
) ! sorted list
Generalize, Instantiate at same place as typechecker!
Gen when adding to struct. Ins when taking from
struct. Verify Insertion Into List
insert
:: (x:
int, l:
int list
) !
int list
Verify Insertion Into
Sorted
List
Slide34Plan
Contributions
Types
&
Structures
Refined
Types & Data StructuresExpressivenessResults
Slide35[]
::
x
:
int
x<V
::
h
1
:
int
Property:
h
1
<
h
2
<
h
3
< …
Refined
Unfolded
::
h
2
:
int
h1<h2
::
h
3:int
h1<h3
h2
<h3
h
3<x
[]
::
x
:
int
h
1
<x
h
2
<x
x<V
Property:
sorted
list
Type
Refinements Determine Property
Slide36[]
::
x
:
int
x
V
::
h
1
:
int
Property:
h
1
h
2
h3 …
Refined Unfolded
::
h
2
:
int
h1h2
::
h
3:int
h1
h3
h2
h3
h
3
x
[]
::
x
:
int
h
1
x
h
2
x
x
V
Property:
duplicate-free
list
Type
Non-aliasing in Collections
e.g. list of
distinct
addresses
Collections
Slide37Trees: Type
Leaf
Node
x
:
int
x<V
V<x
Trees:
Refined
Type
Slide38Node
Leaf
Node
x
:
int
Leaf
Node
x
:
int
Unfold
Refined
Type
Node
x:int
Leaf
x<V
V<x
x<V
V<x
r<V
V<r
Push
edge predicate
inside
LHS nodes < root
< RHS nodes
Property:
binary-search
tree
r
<V
V<x
r:int
V<
r
x
<V
x<r
r<x
Slide39Refined
Type
Node
|
Hl
–
Hr|<
2
Leaf
l
r
Refine
Node
l , r
=
Names
of left, right trees
H
l,
H r = Heights of left, right trees
Slide40Node
|
Hl
–
Hr
|
<
2
Leaf
l
r
Refined
Type
Unfold
Node
|
H
l
1
–
H
r1|<
2
l1r1
Node
|Hl –
Hr|< 2
Leaf
l
r
Node
|
Hl
–
Hr
|
<
2
Leaf
l
r
Refined
Type
Unfold
Height balanced at each level
Property:
balanced
tree
Slide41Plan
Contributions
Types
&
Structures
Refined
Types & Data StructuresExpressivenessResults
Slide42Our Inference Tool
Specification
Unsafe
Safe
Dsolve
Hints
OCaml
Program
Slide43Data Structures
Program
Lines
List-sort
111
Map
98
Redblack
106
Stablesort
124
Vec
343
BinHeap
122
SplayHeap
134
Malloc
71
Bdd
206
UnionFind
65
SubvSolve
264
Total
1736
Slide44Data Structures
Program
Lines
Property
List-sort
111
Sorted,
Outputs Permutation of Input
Map
98
Balance, BST, Implements a Set
Redblack
106
Balance, BST, Color
Stablesort
124
Sorted
Vec
343
Balance, Bounds Checking, …
BinHeap
122
Heap,
Returns
Minimum, …
SplayHeap
134
BST,
Returns Minimum, Implements a Set
Malloc
71
Used and Free Lists Are Accurate
Bdd
206
Maintains Variable Order
UnionFind
65
Acyclic
SubvSolve
264
Acyclic
Total
1736
Slide45Data Structures
Program
Lines
Property
List-sort
111
Sorted,
Outputs Permutation of Input
Map
98
Balance, BST, Implements a Set
Redblack
106
Balance, BST, Color
Stablesort
124
Sorted
Vec
343
Balance, Bounds Checking, …
BinHeap
122
Heap,
Returns
Minimum, …
SplayHeap
134
BST,
Returns Minimum, Implements a Set
Malloc
71
Used and Free Lists Are Accurate
Bdd
206
Maintains Variable Order
UnionFind
65
Acyclic
SubvSolve
264
Acyclic
Total
1736
Slide46Program
Lines
Property
List-sort
111
Sorted,
Outputs Permutation of Input
Map
98
Balance, BST, Implements a Set
Redblack
106
Balance, BST, Color
Stablesort
124
Sorted
Vec
343
Balance, Bounds Checking, …
BinHeap
122
Heap,
Returns
Minimum, …
SplayHeap
134
BST,
Returns Minimum, Implements a Set
Malloc
71
Used and Free Lists Are Accurate
Bdd
206
Maintains Variable Order
UnionFind
65
Acyclic
SubvSolve
264
Acyclic
Total
1736
Data Structures
Slide47Program
Lines
Property
List-sort
111
Sorted,
Outputs Permutation of Input
Map
98
Balance, BST, Implements a Set
Redblack
106
Balance, BST, Color
Stablesort
124
Sorted
Vec
343
Balance, Bounds Checking, …
BinHeap
122
Heap,
Returns
Minimum, …
SplayHeap
134
BST,
Returns Minimum, Implements a Set
Malloc
71
Used and Free Lists Are Accurate
Bdd
206
Maintains Variable Order
UnionFind
65
Acyclic
SubvSolve
264
Acyclic
Total
1736
Data Structures
Slide48Program
Lines
Property
List-sort
111
Sorted,
Outputs Permutation of Input
Map
98
Balance, BST, Implements a Set
Redblack
106
Balance, BST, Color
Stablesort
124
Sorted
Vec
343
Balance, Bounds Checking, …
BinHeap
122
Heap,
Returns
Minimum, …
SplayHeap
134
BST,
Returns Minimum, Implements a Set
Malloc
71
Used and Free Lists Are Accurate
Bdd
206
Maintains Variable Order
UnionFind
65
Acyclic
SubvSolve
264
Acyclic
Total
1736
Data Structures
Slide49Data Structures
Program
Lines
List-sort
111
Map
98
Redblack
106
Stablesort
124
Vec
343
BinHeap
122
SplayHeap
134
Malloc
71
Bdd
206
UnionFind
65
SubvSolve
264
Total
1736
Slide50Data Structures
Program
Lines
Hints
List-sort
111
7
Map
98
14
Redblack
106
2
Stablesort
124
1
Vec
343
9
BinHeap
122
6
SplayHeap
134
3
Malloc
71
2
Bdd
206
3
UnionFind
65
2
SubvSolve
264
2
Total
1736
54
3%
of code size
Slide51Data Structures
Program
Lines
Hints
Time (sec)
List-sort
111
7
5
Map
98
14
25
Redblack
106
2
29
Stablesort
124
1
4
Vec
343
9
87
BinHeap
122
6
33
SplayHeap
134
3
6
Malloc
71
2
2
Bdd
206
3
80
UnionFind
65
2
5
SubvSolve
264
2
20
Total
1736
54
300
Slide52Data Structures
Program
Lines
Hints
Time (sec)
List-sort
111
7
5
Map
98
14
25
Redblack
106
2
29
Stablesort
124
1
4
Vec
343
9
87
BinHeap
122
6
33
SplayHeap
134
3
6
Malloc
71
2
2
Bdd
206
3
80
UnionFind
65
2
5
SubvSolve
264
2
20
Total
1736
54
300
Slide53Vec: Extensible Arrays (317 LOC)
“Python-style” arrays for
OCaml
find, insert, delete, join
etc.
Efficiency via
balanced trees
Balanced Height difference between siblings ≤ 2
Dsolve found balance violation
Slide54fatal off-by-one error
Recursive Rebalance
Slide55Debugging via Inference
Using
Dsolve
we found
Where imbalance occurred
(specific path conditions)
How imbalance occurred (left tree off by up to 4)Leading to test
and fix
Slide56Plan
Contributions
Types
&
Structures
Refined Types
& Data StructuresExpressivenessResults
Slide57http://pho.ucsd.edu/liquid
source, papers, demo, etc.
Slide58Data
Precision
Types
Predicates
Lifting to
Structures
Data
Structure
Verification
Conclusion
Slide59(Finite)
Maps
1
2
4
3
5
n
1
.succs
=[
n
2
;
n
3
;
n
4
]
(node,
node
list) Map
Field Read/Get
Field Write/Set
n.succs
n.succs
:=
eset succs
n eget succs n
Data
Key
Slide60Refined Maps
1
2
4
3
5
(node,
node
list) Map
(
n:
node
,
{
x
:
node
|
n<x
}
list) Map
P(x0), P(x1
),…
8x. P(x)
How to Generalize?
How to Instantiate?
Refine poly-type for
set
Refine poly-type for get
When getting data from
keyWhen setting
key to data
Acyclic Graph!
Slide61Textual Representation
μ
t. [] + ::(x:
int
, t)
[]
::
x
:
int
x<V
0<x
<<>, <
0<x
,
true
>>
type
int
list =
| [] | ::
of x:int *
int list
[] has no parameters
Refines Elements
Refines Tail
[] + ::(x:
{0 < x}
,
μ
t. [] + ::(x:
int
, <…>t))
μ
t. [] + ::(x:
int
, <<>, <
x<V
,
true
>>t)
Slide62Insertion Sort Type and Hint
let
rec
ins l x = match l
with | [] -> x :: [] | h ::
xs -> if x < h then
x :: h :: xs else h :: (ins xs x)
let insert_sort lst =
List.fold_left
ins []
lst
hint:
* =< *
sorted =
μt. [] + ::(x: int
, <<>, <V >= x, true
>>t)insert_sort
: sorted !
sorted
Type-Based Data Structure Verification
Download Presentation - The PPT/PDF document "Type-Based Data Structure Verification" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.