Pavan Rajagopal GeoControl Systems James B Dabney UHCL Gary Barber GeoControl Systems 1 Spacecraft FSW Workshop 2015 Objectives of Work IVampV Background Importance of early defect identification ID: 741635
Download Presentation The PPT/PDF document "Risk-Driven Spacecraft Flight Software I..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Risk-Driven Spacecraft Flight Software Independent Verification and Validation
Pavan Rajagopal, GeoControl SystemsJames B. Dabney, UHCLGary Barber, GeoControl Systems
1
Spacecraft FSW Workshop 2015Slide2
Objectives of WorkIV&V Background
Importance of early defect identificationConventional Strategies for Targeting IV&VUsing Critical Events for Targeting IV&VCritical Event IdentificationAssurance case overviewRisk TreeAdjectival and Probabilistic Scoring of Risk
Detailed Analysis & ScoringBenefits of ApproachConclusions and Future work
Spacecraft FSW Workshop 2015
2
OverviewSlide3
Spacecraft FSW is prone to defects
IV&V identifies and resolves defectsObjectives of methodologies:Accurately scope & target IV&VEffectively perform IV&V to identify and resolve defects Measure the risk reduction achieved
Spacecraft FSW Workshop 2015
3
ObjectivesSlide4
Evaluates system and software for Correctness & Completeness
Technically, Organizationally and Financially IndependentMost effective when applied throughout lifecycleKey information sourcesDeveloper artifactsIV&V Technical ReferenceSpacecraft FSW Workshop 2015
4
IV&V BackgroundSlide5
Analyze technical artifactsAssess adequacy of verification activities and environments
Perform Independent testingAlgorithmsComplex or High-risk code fragmentsOff-nominal scenariosSpacecraft FSW Workshop 2015
5
Typical IV&V ActivitiesSlide6
Value of Early Defect Identification
Source: Ref [1]
6
Spacecraft FSW Workshop 2015Slide7
Criticality Analysis and Risk Assessment (CARA)
Identify critical functionPrioritize using risk (likelihood of problem) and criticality (consequences)Portfolio-based risk assessment [2]Based on hardware and software entitiesAlso uses risk-criticality matrixBoth methods result in broad IV&V targets
Spacecraft FSW Workshop 2015
7
Strategies for Targeting IV&VSlide8
Based on flow of mission eventsMission timelines
Concept of operationsBenefitsPermits early lifecycle IV&V participationNarrows analysis targets and enables prioritizationEnables cross cutting analysisSpacecraft FSW Workshop 2015
8
Targeting using Critical EventsSlide9
Based on risk categories
Human safetyLoss of missionDamage to assetLoss of key mission objectivesScoringEvents scored for each category of riskComposite score used to rank events Highest ranked events get priority in analysis
Critical Event Identification
9
Spacecraft FSW Workshop 2015Slide10
Structured argument [3]
Based on safety casesUses logical flow (decomp) fromClaimsSupporting claimsEvidenceHigh-level claim is successful performance of system function or objective
Supporting claims deal withSystem Configuration
Environment
Procedures
HW/SW functionality
Assurance Case Overview
10
Spacecraft FSW Workshop 2015
Evidence examples
Documentation
Testing
AnalysesSlide11
Uses assurance case structureOverall risk for top-level claim depends on
Risk of lower level supporting claimsStrength of influence of lower level supporting claimsCompleteness and correctness of evidence determines risk for lowest level supporting claim Score rollup optionsAdjectival (stoplight chart)Numerical weightingProbabilistic (requires extensive calibration)
Rollup can feed into project risk management tool
Spacecraft FSW Workshop 2015
11
Risk TreeSlide12
Uses assurance case structureOverall risk for top-level claim depends on
Risk of lower level supporting claimsStrength of influence of lower level supporting claimsCompleteness and correctness of evidence determines risk for lowest level supporting claim Score rollup optionsAdjectival (stoplight chart)Numerical weightingProbabilistic (requires extensive calibration)
Rollup can feed into project risk management tool
Spacecraft FSW Workshop 2015
12
Risk TreeSlide13
Staging failure
Docking failureFailure of Trajectory and orbit maneuversExamples of risks related to Mission-critical events
13
Spacecraft FSW Workshop 2015Slide14
System is not configured for event
Precursor events do not successfully completeFailed Event TriggersMissed or failed Execution stepsFailure to confirm correct completionExamples of Risk Sub-claims
14
Spacecraft FSW Workshop 2015Slide15
Requirements
DesignTestingAnalysisPrior use of subsystemFormal methods analysisExamples of evidence used to mitigate risks
15
Spacecraft FSW Workshop 2015Slide16
Spacecraft FSW Workshop 2015
16
Partial Critical Event Risk Tree Example
Deorbit
Burn Fails
Requirements
Evidence
Incorrect Computation of Burn Parameters
Miscompute Delta V
Incorrect Execution of Burn
Miscompute Ignition Time
Design Evidence
Flight Control Failure
Uncompensated HW Failure
Requirements
Evidence
Requirements
Evidence
Requirements
Evidence
Design Evidence
Design Evidence
Design EvidenceSlide17
Suitable structure and format for stoplight risk management process
Spacecraft FSW Workshop 201517
Adjectival Score Rollup
Adapted from Ref [4]Slide18
Uses Dempster
-Shafer belief functionsBased on historical data from similar projectsCorrelated to project characteristics and activitiesComputes belief that claims will be realized based on:Confidence in EvidenceRelative importance of supporting claims
Spacecraft FSW Workshop 2015
18
Probabilistic Scoring and RollupSlide19
Spacecraft FSW Workshop 2015
19Probabilistic Rollup ScoringSlide20
Analysis
Performed on lowest level supporting claimsInvolvesTraditional IV&V inspection and analysisSimulationIndependent testingScoringPerformed at the lowest level supporting claim Based on subjective assessment of evidence by qualified IV&V analyst
This assessment is fit into a range of defect densities from historical like projectsTree structure used to establish score at mid level and top nodes
Detailed Analysis & Scoring
20
Spacecraft FSW Workshop 2015Slide21
Drives cross cutting analysis across multiple participating subsystems (HW and SW)Analysis provides insight
Points out omissions or errors in evidenceIdentifies issues and defectsEnhances objectivity in evaluating risk Spacecraft FSW Workshop 2015
21
Benefits of approachSlide22
Critical event, risk-driven approach is effective
Allows relatively fine grained analysis targetingProvides solid support for scope and analysis decisionsConstruction of risk tree aids and documents system understandingRecords Analysis decisionsFacilitates change impact analysis
Conclusions
22
Spacecraft FSW Workshop 2015Slide23
Increase insight into scoring by tracking defects vs risk assessment
Integrate methods into workflow toolsetAutomate tracking and reporting of risk scoreFuture Work
23
Spacecraft FSW Workshop 2015Slide24
[1] J. B. Dabney and G. Barber, “
Direct return on investment of software independent verification and validation: Methodology and initial case studies,” Assurance Technology Symposium, 5 June 2003[2] N. Alvaro and S. Raque, “Portfolio based risk assessment and risk-based assessment process,” Technical Report, NASA IV&V Center, Fairmont, WV, 2012[3] S. Blanchette
, Jr., “Assurance cases for analysis of complex system of systems software,” Software Engineering Institute, June, 2010[4] G. Barber, “Risk reduction demonstration pilot,” NASA IV&V
Workship
, Morgantown, WV, September 13 – 15, 2011
References
24
Spacecraft FSW Workshop 2015