Assurance Based Development of Dependable Systems Patrick J Graydon John C Knight University of Virginia Certification By Safety Argument 2 System Requirements and Goals Does the evidence imply ID: 637844
Download Presentation The PPT/PDF document "Software Process Synthesis in" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Software Process Synthesis inAssurance Based Developmentof Dependable Systems
Patrick J. GraydonJohn C. KnightUniversity of VirginiaSlide2
Certification By Safety Argument2
System Requirements
and Goals
Does the
evidence imply
the goals?
Standard Development Process
Rigorous
Argument
“I believe that
the evidence
I have obtained
justifies the claim
that the goals
are met”
Evidence
Goals
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable SystemsSlide3
Can We Exploit Argument?3
?
Argument tells us what is
important
Artifacts provide services and evidence
Is there a mechanism?
ArgumentArtifact2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
Mechanism goal: generate the necessary artifacts (including
software) and evidenceSlide4
Turning the Knobs to MaximumExtreme Programming
“turned up the knobs”ABD turns up the knobs on rational development 2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
4
Good
BetterSafety argumentsBroader argument“Early and often”Continuous argument updateRational choicesA completely rational processSlide5
2010 April 29Comprehensive Argument
Problem:Argument must address all goals of all stakeholdersThere will be tradeoffsMust appear explicitly in the argumentMust be addressedSeen to be addressedImpact of each must be clear
Without this, consequences could be severe
As a driver, safety argument does not do this
Software Process Synthesis in Assurance Based Development of Dependable Systems
5Slide6
Software That Is Fit For UseNot just safety, but
fitness for purpose:Adequate safety, andAdequate security, andDesired functionality, andEverything else
Main fitness claim
:
The system is adequately fit for use in the
context(s
) in which it will be operated.2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems6SafetyFunctionalitySecuritySlide7
The Merit Of Envelope Protection7 of 68
“Darn, the wings
did
fall off.”
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable SystemsSlide8
Successful DevelopmentThe fitness argument
captures data about the productThe success argument captures data about the process2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
8
Fitness argument
Software
Success argumentDevelopment process
Development schedule, andBudget and other
resource constraints, and
Pragmatic development constraintsSlide9
9 of 68Assurance Based Development
Success Argument
Fitness Argument
Development
Synthesized
Development
ProcessObligationSupportSupport
Obligation
Shrinks
Grows2010 April 29Software Process Synthesis in Assurance Based Development of Dependable SystemsSlide10
Process Synthesis:
Select Goals to Address2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems
10
Fitness
argument
Success
argumentExperience (both personal and that of colleagues)Pattern library and other literatureOption nOption 1Option 2
…
Assurance
obligationsAssuranceobligationsOptionsOptions Area of expertise Perceived risk of infeasibility Minimize interdependencyFirst step: select a goal or goals to addressSlide11
Process Synthesis: Gather Options2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems11
Fitness
argument
Success
argument
Experience (both personal and that of colleagues)Pattern library and other literatureOption nOption 1Option 2
…
Assurance
obligationsOptionsOptionsGathering options takes timeMust balance effort spent versus perceived risk of making a poor choiceSecond step: gather options that meet the selected obligationsAssurance obligationsSlide12
Example OptionsObligations:Success: Timing goals can be met demonstrably
Fitness: Real-time requirements metOptions considered:Analyze WCET using a tool to be chosen laterUtilize a watchdog timer to re-issue last frame’s control outputs if deadline would be missed2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
12Slide13
Process Synthesis: Evaluating Options
Consider:FunctionalityRestrictions on laterchoicesCostFeasibilityApplicable standardsNon-functional requirements
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
13
Option
nOption 1Option 2…
Arg.
frag
. nArg. frag. 1Arg. frag. 2…Fitness argumentSuccess argument
EvaluationSlide14
Evaluation of Example OptionUse WCET analysisWould supply strong evidence that the hard real-time deadlines would be met
Re-issue last frame’s control outputs if lateUnacceptable — how would we demonstrate:re-issuing the last frame’s outputs would be raredoing so rarely would keep the system safe2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
14Slide15
Process Synthesis: Recording A Choice2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems15
Updated success argument
Evaluation
Arg.
fragment(s
)Process elementsUpdated fitnessargument
Planned process description
If a choice is expected to produce evidence, the evidence added to the argument is marked as forthcoming (e.g. using a diamond in Goal Structuring Notation)
Fragments associated with chosen optionSlide16
Process Execution2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems16
Process synthesis mechanism
Process description
Process
execution
ExperienceArtefacts
Confirmation or exception
EvidenceSlide17
A Case Study of ABDControlled, replicated experiment impractical
Instead: conduct carefully a case studyLook for evidence of four kinds of problems:Infeasibility: developers might be incapable; additional effort might be infeasibleObligations are not appropriate choice driversCan’t judge options by argument impact aloneNo decision criteria is missing or superfluousStudy protocol includes questionnaire
23 questions answered per synthesis step
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
17Slide18
The UVA LifeFlow LVAD
Left Ventricular AssistDeviceContinuous-flowaxial designMagnetic bearingsLess blood damagethan currentmodels!
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
18Slide19
Magnetic Bearing ControlCompute control updates in hard-real-time (5 kHz)
State-space control model, 16 statesNo more than 10-9 failures per hour of operation2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
19Slide20
Resulting Synthesized ProcessFormal specification in PVS
Design a cyclic executive to manage the real-time tasksDesign bearing control task routinesImplement MBCS in SPARK Ada (2,510 lines)Implement bootstrap (106 assembly instructions)Use AdaCore's
GNAT
Pro High-Integrity Edition
compiler
Formally verify the implementation
Used Echo approach, PVS, and SPARK ToolsAnalyze Worst-Case Execution Time (WCET) and stack usageRequirements-based functional testing to Modified Condition / Decision Coverage (MC/DC) (not completed)2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems20Slide21
Fitness argument
2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems
21Slide22
Fitness argument
348 GSN elementsWidest step: 5 child elementsLongest path: 26 elementsGeneral form:Integration testing and
Appeal to satisfied requirements
Timing demonstrated by WCET analysis
Functionality demonstrated by testing
and
formal proof2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems22Slide23
Success Argument
2010 April 29Software Process Synthesis in Assurance Based Development of Dependable Systems
23Slide24
Success Argument
49 GSN elementsWidest step: 10 child elementsLongest path: 9 elementsGeneral form:Appeal to planning and
Argument over enumerated development risks
Evolution:
Starts small, grows early, becomes moot
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems24Slide25
Case Study Results2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems25
Feasibility
No difficulties observed
Obligations are not appropriate choice drivers
28 choices:
19 followed direct reasoning6 others addressed an obligation2 addressed unnoted development risk1 remaining case was an implicit choiceCan’t judge options by argument impact aloneWe observed no value that could not be represented in the fitness or success argumentNo decision criteria is missing or superfluousWe observed that impact on schedule was not covered in the choice criteria; it has been addedSlide26
Next StepsCase study reimplementation of Tokeneer
NSA security challenge problemCommon CriteriaEvaluating ABD repair mechanismEvaluating ABD pattern libraryCollecting developer effort metricsDeveloping argument-based certification mechanismWorking with philosophy department on argument quality
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
26Slide27
Questions?Software Process Synthesis in
Assurance Based Developmentof Dependable SystemsPatrick J. GraydonJohn C. Knight
University of Virginia
2010 April 29
Software Process Synthesis in Assurance Based Development of Dependable Systems
27