Automated Developer Testing:

Automated Developer Testing: Automated Developer Testing: - Start

2016-06-13 67K 67 0 0

Automated Developer Testing: - Description

Achievements and Challenges. Tao . Xie. North Carolina State University. In collaboration with Nikolai Tillmann. , . Peli. de . Halleux. , Wolfram Schulte . @Microsoft Research . and students . @NCSU ASE. ID: 360170 Download Presentation

Download Presentation

Automated Developer Testing:




Download Presentation - The PPT/PDF document "Automated Developer Testing:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentations text content in Automated Developer Testing:

Slide1

Automated Developer Testing:

Achievements and Challenges

Tao

XieNorth Carolina State UniversityIn collaboration with Nikolai Tillmann, Peli de Halleux, Wolfram Schulte @Microsoft Research and students @NCSU ASE

Slide2

Why Automate Testing?

Software testing is

important

Software errors cost the U.S. economy about $59.5 billion each year (0.6% of the GDP)

[NIST 02]

Improving testing infrastructure could save 1/3 cost

[NIST 02]

Software testing is

costly

Account for even half the total cost of software development

[

Beizer

90]

Automated testing

reduces

manual testing

effort

Test execution:

JUnit

,

NUnit

,

xUnit

, etc.

Test generation:

Pex

,

A

gitarOne

,

Parasoft

Jtest

,

etc.

Test-behavior checking:

Pex

,

A

gitarOne

,

Parasoft

Jtest

,

etc.

Slide3

Automation in Developer Testing

Developer testing

http://www.developertesting.com/

Kent Beck’s 2004 talk on “Future of Developer Testing”

http://www.itconversations.com/shows/detail301.html

This talk focuses on tool automation in

developer

testing (e.g., unit testing)

Not system testing etc. conducted by testers

Slide4

Software Testing Setup

=

?

Outputs

Expected Outputs

Program

+

Test

inputs

Test Oracles

Slide5

Software Testing Problems

=

?

Outputs

Expected Outputs

Program

+

Test

inputs

Test Oracles

Test Generation

Generating high-quality test inputs (e.g., achieving high code coverage)

Slide6

Software Testing Problems

=

?

Outputs

Expected Outputs

Program

+

Test

inputs

Test Oracles

Test Generation

Generating high-quality test inputs (e.g., achieving high code coverage)

Test Oracles

Specifying high-quality test oracles (e.g., guarding against various faults)

Slide7

The Recipe of Unit Testing

var list = new List(); list.Add(item); var count = list.Count;

Assert.AreEqual(1, count);}

Three essential ingredients:DataMethod SequenceAssertions

void Add() {

int

item = 3;

Slide8

The (problem with) Data

Which value matters?Bad choices cause incomplete test suites.Hard-coded values get stale when product code changes.Why pick a value if it doesn’t matter?

list.Add(3);

Slide9

Parameterized Unit Testing

void Add(List list, int item) { var count = list.Count; list.Add(item); Assert.AreEqual(count + 1, list.Count);}

Parameterized Unit Test = Unit Test with ParametersSeparation of concernsData is generated by a toolDeveloper can focus on functional specification

[

Tillmann&Schulte

ESEC/FSE 05

]

Slide10

Parameterized Unit Tests areAlgebraic Specifications

A Parameterized Unit Test can be read as a universally quantified, conditional axiom.

void

ReadWrite(string name, string data) { Assume.IsTrue(name != null && data != null); Write(name, data); var readData = Read(name); Assert.AreEqual(data, readData);}

string name, string data:

name

null ⋀ data

null

equals(

ReadResource

(

name,WriteResource

(

name,data

)),

data)

Slide11

Parameterized Unit Testingis going mainstream

Parameterized Unit Tests (PUTs) commonly supported by various test frameworks.NET: Supported by .NET test frameworkshttp://www.mbunit.com/http://www.nunit.org/…Java: Supported by JUnit 4.Xhttp://www.junit.org/Generating test inputs for PUTs supported by tools.NET: Supported by Microsoft Research Pexhttp://research.microsoft.com/Pex/Java: Supported by Agitar AgitarOnehttp://www.agitar.com/

Slide12

Test Generation

HumanExpensive, incomplete, …Brute ForcePairwise, predefined data, etc…Random:Cheap, Fast“It passed a thousand tests” feelingDynamic Symbolic Execution: Pex, CUTE,EXEAutomated white-boxNot random – Constraint Solving

Slide13

Dynamic Symbolic Execution

Code to generate inputs for:

Constraints to solve a!=null a!=null &&a.Length>0 a!=null &&a.Length>0 &&a[0]==1234567890

void CoverMe(int[] a){ if (a == null) return; if (a.Length > 0) if (a[0] == 1234567890) throw new Exception("bug");}

Observed constraintsa==nulla!=null &&!(a.Length>0)a!=null &&a.Length>0 &&a[0]!=1234567890a!=null &&a.Length>0 &&a[0]==1234567890

Datanull{}{0}{123…}

a==null

a.Length

>0

a[0]==123…

T

T

F

T

F

F

Execute&Monitor

Solve

Choose next path

Done: There is no path left.

Negated condition

Slide14

Loops Fitnex [Xie et al. DSN 09]Generic API functions e.g., RegEx matching IsMatch(s1,regex1) Reggae [Li et al. ASE 09-sp]Method sequences MSeqGen [Thummalapenta et al. ESEC/FSE 09]Environments e.g., file systems, network, db, …Parameterized Mock Objects [Marri AST 09]OpportunitiesRegression testing [Taneja et al. ICSE 09-nier]Developer guidance (cooperative developer testing)

Challenges of DSE

Slide15

Loops Fitnex [Xie et al. DSN 09]Generic API functions e.g., RegEx matching IsMatch(s1,regex1) Reggae [Li et al. ASE 09-sp]Method sequences MSeqGen [Thummalapenta et al. ESEC/FSE 09]Environments e.g., file systems, network, db, …Parameterized Mock Objects [Marri AST 09]ApplicationsTest network app at Army division@Fort Hood, TexasTest DB app of hand-held medical assistant device at FDA

NCSU Industry Tech Transfer

Slide16

Pex on MSDN DevLabsIncubation Project for Visual Studio

Download counts

(20 months)

(Feb. 2008 - Oct. 2009 )

Academic:

17,366

Devlabs

:

13,022

Total: 30,388

Slide17

Loops Fitnex [Xie et al. DSN 09]Generic API functions e.g., RegEx matching IsMatch(s1,regex1) Reggae [Li et al. ASE 09-sp]Method sequences MSeqGen [Thummalapenta et al. ESEC/FSE 09]Environments e.g., file systems, network, db, …Parameterized Mock Objects [Marri AST 09]ApplicationsTest network app at Army division@Fort Hood, TexasTest DB app of hand-held medical assistant device at FDA

NCSU Industry Tech Transfer

Slide18

There are decision procedures for individual path conditions, but…Number of potential paths grows exponentially with number of branchesReachable code not known initiallyWithout guidance, same loop might be unfolded foreverFitnex search strategy [Xie et al. DSN 09]

Explosion of Search Space

Slide19

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Path condition:!(x == 90) ↓New path condition:(x == 90) ↓New test input:TestLoop(90, {0})

Test input:

TestLoop

(0, {0})

DSE Example

Slide20

Path condition:(x == 90) && !(y[0] == 15) ↓New path condition:(x == 90) && (y[0] == 15) ↓New test input:TestLoop(90, {15})

Test input:

TestLoop

(90, {0})

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

DSE Example

Slide21

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Test input:TestLoop(90, {15})

Path condition:(x == 90) && (y[0] == 15) && !(x+1 == 110) ↓New path condition:(x == 90) && (y[0] == 15) && (x+1 == 110) ↓New test input:No solution!?

Challenge in DSE

Slide22

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Path condition:(x == 90) && (y[0] == 15) && (0 < y.Length) && !(1 < y.Length) && !(x+1 == 110) ↓New path condition:(x == 90) && (y[0] == 15) && (0 < y.Length) && (1 < y.Length) Expand array size

Test input:

TestLoop(90, {15})

A Closer Look

Slide23

We can have infinite paths! (both length and number)Manual analysis  need at least 20 loop iterations to cover the target branchExploring all paths up to 20 loop iterations is practically infeasible: 220 paths

Test input:

TestLoop(90, {15})

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

A Closer Look

Slide24

Fitnex: Fitness-Guided Exploration

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Test input:

TestLoop

(90, {15, 15})

Our solution: Prefer to flip nodes on the most promising pathPrefer to flip the most promising nodes on pathUse fitness function as a proxy for promising

Key observations: with respect to the coverage target,

not all paths are equally promising for flipping nodes

not all nodes are equally

promising to flip

Slide25

Fitness Function

FF computes fitness value (distance between the current state and the goal state)Search tries to minimize fitness value

[Tracey et al. 98, Liu at al. 05, …]

Slide26

Fitness Function for (x == 110)

public bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Fitness function: |110 – x |

Slide27

Compute Fitness Values for Paths

(90,

{0}) 20(90, {15}) 19(90, {15, 0}) 19(90, {15, 15}) 18(90, {15, 15, 0}) 18(90, {15, 15, 15}) 17 (90, {15, 15, 15, 0}) 17(90, {15, 15, 15, 15}) 16(90, {15, 15, 15, 15, 0}) 16(90, {15, 15, 15, 15, 15}) 15…

FitnessValue

(x, y)

Give preference to flip a node in paths with better fitness values.We still need to address which node to flip on paths …

public

bool TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

Fitness function: |110 – x |

Slide28

Fitness

Value

Compute Fitness Gains for Branches

public

bool

TestLoop(int x, int[] y) { if (x == 90) { for (int i = 0; i < y.Length; i++) if (y[i] == 15) x++; if (x == 110) return true; } return false;}

(90, {0}) 20(90, {15})  flip b4 19(90, {15, 0})  flip b2 19(90, {15, 15})  flip b4 18(90, {15, 15, 0})  flip b2 18(90, {15, 15, 15})  flip b4 17 (90, {15, 15, 15, 0})  flip b2 17(90, {15, 15, 15, 15})  flip b4 16(90, {15, 15, 15, 15, 0})  flip b2 16(90, {15, 15, 15, 15, 15})  flip b4 15…

(x, y)

Fitness function: |110 – x |

Branch b1:

i < y.LengthBranch b2: i >= y.LengthBranch b3: y[i] == 15Branch b4: y[i] != 15

Flipping branch node of b4 (b3) gives us average 1 (-1) fitness gain (loss)Flipping branch node of b2 (b1) gives us average 0 (0) fitness gain (loss)

Slide29

Let p be an already explored path, and n a node on that path, with explored outgoing branch b.After (successfully) flipping n, we get path p’ that goes to node n, and then continues with a different branch b’.Define fitness gains as follows, where F(.) is the fitness value of a path.Set FGain(b) := F(p) – F(p’)Set FGain(b’) := F(p’) – F(p)Compute the average fitness gain for each program branch over time

Compute Fitness Gains for Branches

Slide30

Implementation in Pex

Pex: Automated White-Box Test Generation tool for .NET, based on Dynamic Symbolic Execution Pex maintains global search frontierAll discovered branch nodes are added to frontierFrontier may choose next branch node to flipFully explored branch nodes are removed from frontierPex has a default search frontierTries to create diversity across different coverage criteria, e.g. statement coverage, branch coverage, stack traces, etc.Customizable: Other frontiers can be combined in a fair round-robin scheme

Slide31

Implementation in Pex

We implemented a new search frontier “Fitnex”:Nodes to flip are prioritized by their composite fitness value:F(pn) – FGain(bn),wherepn is path of node nbn is explored outgoing branch of n Fitnex always picks node with lowest composite fitness value to flip.To avoid local optimal or biases, the fitness-guided strategy is combined with Pex’s search strategies

Slide32

A collection of micro-benchmark programs routinely used by the Pex developers to evaluate Pex’s performance, extracted from real, complex C# programs

Ranging from string matching like

if (

value.StartsWith

("Hello") &&

value.EndsWith("World!") && value.Contains(" ")) { … }to a small parser for a Pascal-like language where the target is to create a legal program.

Evaluation Subjects

Slide33

Pex with the Fitnex strategyPex without the Fitnex strategyPex’s previous default strategyRandoma strategy where branch nodes to flip are chosen randomly in the already explored execution treeIterative Deepeninga strategy where breadth-first search is performed over the execution tree

Search Strategies Under Comparison

Slide34

#runs/iterations required to cover the target

Pex

w/o

Fitnex: avg. improvement of factor 1.9 over RandomPex w/ Fitnex: avg. improvement of factor 5.2 over Random

Evaluation Results

Slide35

Object Creation

Pex normally uses public methods to configure non-public object fieldsHeuristics built-in to deal with common typesUser can help if needed

void (

Foo foo) { if (foo.Value == 123) throw …

[

PexFactoryMethod

]

Foo

Create(Bar

bar

) {

return new

Foo

(bar);

}

Slide36

QuickGraph Example

A graph example from QuickGraph library

36

36

interface

IGraph{ /* Adds given vertex to the graph */ void AddVertex(IVertex v); /* Creates a new vertex and adds it to the graph */ IVertex AddVertex(); /* Adds an edge to the graph. Both vertices should already exist in the graph */ IEdge AddEdge(IVertex v1, Ivertex v2); }

Slide37

Method Under Test

Desired object state for reaching targets 1 and 2: graph object should contain vertices and edges

Class

SortAlgorithm{ IGraph graph; public SortAlgorithm(IGraph graph) { this.graph = graph; } public void Compute (IVertex s) { foreach(IVertex u in graph.Vertices) { //Target 1 } foreach(IEdge e in graph.Edges) { //Target 2 } }}

m

ethod

sequence

Slide38

Method Under Test

VertexAndEdgeProvider v0 = new VertexAndEdgeProvider();Boolean v1 = false;BidirectionalGraph v2 = new BidirectionalGraph((IVertexAndEdgeProvider)v0, v1);IVertex v3 = v2.AddVertex();IVertex v4 = v0.ProvideVertex();IEdge v15 = v2.AddEdge(v3, v4);

Achieved 31.82% (7 of 22) branch coverageReason for low coverage: Not able to generate graph with vertices and edges

Applying Randoop, a random testing approach that constructs test inputs by randomly selecting method calls

Example sequence generated by Randoop

v4

not in the graph, so edge cannot be added

to graph.

Slide39

New MSeqGen Approach

VertexAndEdgeProvider

v0;bool bVal;IGraph ag = new AdjacencyGraph(v0, bVal);IVertex source = ag.AddVertex();IVertex target = ag.AddVertex();IVertex vertex3 = ag.AdVertex();IEdge edg1 = ag.AddEdge(source, target);IEdge edg2 = ag.AddEdge(target, vertex3);IEdge edg3 = ag.AddEdge(source, vertex3);

Use mined sequences to assist Randoop and PexBoth Randoop and Pex achieved 86.40% (19 of 22) branch coverage with assistance from MSeqGen

Mine sequences from existing code bases Reuse mined sequences for achieving desired object states

A Mined sequence from an existing codebase

Graph object

includes both vertices and edges

Slide40

Challenges Addressed by MSeqGen

Existing codebases are often large and complete analysis is expensive  Search and analyze only relevant portionsConcrete values in mined sequences may be different from desired values  Replace concrete values with symbolic values and use dynamic symbolic executionExtracted sequences individually may not be sufficient to achieve desired object states Combine extracted sequences to generate new sequences

Slide41

MSeqGen: Code Searching

Problem: Existing code bases are often large and complete analysis is expensive Solution: Use keyword search for identifying relevant method bodies using target classesAnalyze only those relevant method bodies

Target classes: System.Collections.Hashtable QuickGraph.Algorithms.TSAlgorithmKeywords: Hashtable, TSAlgorithm

Shortnames

of target classes are used as keywords

Slide42

MSeqGen: Sequence Generalization

Problem: Concrete values in mined sequences are different from desired values to achieve target statesSolution: Generalize sequences by replacing concrete values with symbolic values

Class A {

int f1 { set; get; } int f2 { set; get; } void CoverMe() { if (f1 != 10) return; if (f2 > 25) throw new Exception(“bug”); }}

Method Under Test

A

obj = new A();obj.setF1(14);obj.setF2(-10);obj.CoverMe();

Mined Sequence for A

Sequence cannot help in exposing bug since desired values are f1=10 and f2>25

Slide43

MSeqGen: Sequence Generalization

Replace concrete values 14 and -10 with symbolic values X1 and X2

A

obj = new A();obj.setF1(14);obj.setF2(-10);obj.CoverMe();

Mined Sequence for A

int

x1 = *, x2 = *;A obj = new A();obj.setF1(x1);obj.setF2(x2);obj.CoverMe();

Generalized Sequence for A

Use DSE for generating desired values for X1 and X2

DSE explores

CoverMe

method and generates desired values (X1 = 10 and X2 = 35)

Slide44

Improvement of State-of-the-Art

44

44

Randoop

Without assistance from

MSeqGen

: achieved

32%

branch coverage

achieved

86%

branch coverage

In evaluation, help

Randoop

achieve 8.7% (maximum 20%) higher branch coverage

Pex

Without assistance from

MSeqGen

: achieved

45%

branch coverage

achieved

86%

branch coverage

In evaluation, help

Pex

achieve 17.4% (maximum 22.5%) higher branch coverage

Slide45

Test Oracles

Write assertions and Pex will try to break themWithout assertions, Pex can only find violations of runtime contracts causing NullReferenceException, IndexOutOfRangeException, etc.Assertions leveraged in product and test codePex can leverage Code Contracts

Slide46

Summary:Automated Developer Testing

=

?

Outputs

Expected Outputs

Program

+

Test

inputs

Test Oracles

Test Generation

Test inputs for PUT generated by

tools

(e.g.,

Pex

)

Fitnex

: guided exploration of paths [DSN 09]

MSeqGen: exploiting real-usage sequences [ESEC/FSE 09]Test OraclesAssertions in PUT specified by developers

Division of Labors

Slide47

Thank you

http://research.microsoft.com/pex

http://pexase.codeplex.com/

https://sites.google.com/site/asergrp/

Slide48

Code Contracts

http://research.microsoft.com/en-us/projects/contracts/Library to state preconditions, postconditions, invariantsSupported by two tools:Static CheckerRewriter: turns Code Contracts into runtime checksPex analyses the runtime checksContracts act as Test OraclePex may find counter examples for contractsMissing Contracts may be suggested

Slide49

Example: ArrayList

Class invariant specification:

public class ArrayList { private Object[] _items; private int _size; ... [ContractInvariantMethod] // attribute comes with Contracts protected void Invariant() { Contract.Invariant(this._items != null); Contract.Invariant(this._size >= 0); Contract.Invariant(this._items.Length >= this._size); }

Slide50

Parameterized

Models

Slide51

Unit test: while it is debatable what a ‘unit’ is, a ‘unit’ should be small.Integration test: exercises large portions of a system.Observation: Integration tests are often “sold” as unit testsWhite-box test generation does not scale well to integration test scenarios.Possible solution: Introduce abstraction layers, and mock components not under test

Unit Testing vs.

Integration Testing

Slide52

AppendFormat

(null, “{0} {1}!”, “Hello”, “World”);  “Hello World!” .Net Implementation:public StringBuilder AppendFormat( IFormatProvider provider, char[] chars, params object[] args) { if (chars == null || args == null) throw new ArgumentNullException(…); int pos = 0; int len = chars.Length; char ch = '\x0'; ICustomFormatter cf = null; if (provider != null) cf = (ICustomFormatter)provider.GetFormat(typeof(ICustomFormatter)); …

ExampleTesting with Interfaces

Slide53

Introduce a mock class

which implements the interface. Write assertions over expected inputs, provide concrete outputspublic class MFormatProvider : IFormatProvider { public object GetFormat(Type formatType) { Assert.IsTrue(formatType != null); return new MCustomFormatter(); }}Problems:Costly to write detailed behavior by exampleHow many and which mock objects do we need to write?

Stubs / Mock Objects

Slide54

Parameterized

Mock Objects - 1

Introduce a mock class which implements the interface. Let an oracle provide the behavior of the mock methods.public class MFormatProvider : IFormatProvider { public object GetFormat(Type formatType) { … object o = call.ChooseResult<object>(); return o; }}Result: Relevant result values can be generated by white-box test input generation tool, just as other test inputs can be generated!

54

Slide55

Chosen values can be shaped by assumptions

public class MFormatProvider : IFormatProvider { public object GetFormat(Type formatType) { … object o = call.ChooseResult<object>(); PexAssume.IsTrue(o is ICustomFormatter); return o; }}(Note: Assertions and assumptions are “reversed” when compared to parameterized unit tests.)

Parameterized Mock Objects - 2

55

Slide56

Models from Choices

Choices to build parameterized models

class PFileSystem : IFileSystem { // cached choices PexChosenIndexedValue<string,string> files; string ReadFile(string name) { var content = this.files[name]; if (content == null) throw new FileNotFoundException(); return content; }}

Slide57

57

57

Subjects: QuickGraph FacebookResearch Questions: RQ1: Can our approach assist Randoop (random testing tool) in achieving higher code coverages? RQ2: Can our approach assist Pex (DSE-based testing tool) in achieving higher code coverages?

MSeqGen

Evaluation

Slide58

58

RQ1: Assisting Randoop

Slide59

59

RQ2: Assisting Pex

Legend:

#c: number of classes

P: branch coverage achieved by

Pex

P + M: branch coverage achieved by

Pex

and

MSeqGen

Slide60

Assumptions and Assertions

void PexAssume.IsTrue(bool c) {

if (!c) throw new AssumptionViolationException();}void PexAssert.IsTrue(bool c) { if (!c) throw new AssertionViolationException();}Assumptions and assertions induce branchesExecutions which cause assumption violations are ignored, not reported as errors or test cases

Slide61

How to test this code?

(Actual code from .NET base class libraries)

Motivation: Unit Testing HellResourceReader

61

Slide62

Motivation: Unit Testing Hell

ResourceReader

Slide63

ResourceReader

PUT

[

PexClass

,

TestClass

]

[

PexAllowedException

(

typeof

(

ArgumentNullException

))]

[

PexAllowedException

(

typeof

(

ArgumentException

))]

[

PexAllowedException

(

typeof

(

FormatException

))]

[

PexAllowedException

(

typeof

(

BadImageFormatException

))]

[

PexAllowedException

(

typeof

(

IOException

))]

[

PexAllowedException

(

typeof

(

NotSupportedException

))]

public partial class

ResourceReaderTest

{

[

PexMethod

]

public unsafe void

ReadEntries

(byte[] data) {

PexAssume.IsTrue

(data != null);

fixed (byte* p = data)

using (

var

stream = new

UnmanagedMemoryStream

(p,

data.Length

)) {

var

reader = new

ResourceReader

(stream);

foreach

(

var

entry in reader) { /* reading entries */ }

}

}

}

Slide64

Creating Complex Objects

Exploration of constructor/mutator method sequencesTesting with class invariants

Slide65

Write class invariant as boolean-valued parameterless methodRefers to private fieldsMust be placed in implementation codeExploration of valid states by setting public/private fieldsMay include states that are not reachable

Object Creation:

Class Invariants

Slide66

Example: ArrayList

Class invariant specification:

public class ArrayList { private Object[] _items; private int _size; ... [ContractInvariantMethod] // attribute comes with Contracts protected void Invariant() { Contract.Invariant(this._items != null); Contract.Invariant(this._size >= 0); Contract.Invariant(this._items.Length >= this._size); }

Slide67

Example: ArrayList

PUT:

[PexMethod]public void ArrayListTest(ArrayList al, object o){ int len = al.Count; al.Add(o); PexAssert.IsTrue(al[len] == o);}

Slide68

Example: ArrayList

Generated Test:

[TestMethod] public void Add01() { object[] os = new object[0]; // create raw instance ArrayList arrayList = PexInvariant.CreateInstance<ArrayList>(); // set private field via reflection PexInvariant.SetField<object[]>(arrayList, "_items", os); PexInvariant.SetField<int>(arrayList, "_size", 0); // invoke invariant method via reflection PexInvariant.CheckInvariant(arrayList); // call to PUT ArrayListTest(arrayList, null); }

Slide69

Slide70

Slide71


About DocSlides
DocSlides allows users to easily upload and share presentations, PDF documents, and images.Share your documents with the world , watch,share and upload any time you want. How can you benefit from using DocSlides? DocSlides consists documents from individuals and organizations on topics ranging from technology and business to travel, health, and education. Find and search for what interests you, and learn from people and more. You can also download DocSlides to read or reference later.