1 Jacob Aubertine 2 Vidhyashree Nagaraju and 1 Lance Fiondella 1 Department of Electrical and Computer Engineering University of Massachusetts Dartmouth MA USA 2 Tandy School of Computer Science ID: 1042432
Download Presentation The PPT/PDF document "A Metrics-based Software Tool to Guide T..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. A Metrics-based Software Tool to Guide Test Activity Allocation1Jacob Aubertine, 2Vidhyashree Nagaraju, and 1Lance Fiondella1Department of Electrical and Computer EngineeringUniversity of Massachusetts Dartmouth, MA, USA2Tandy School of Computer ScienceThe University of Tulsa, OK, USA1
2. OutlineIntroductionDiscrete Cox Proportional Hazard NHPP SRGMSoftware ArchitectureTool OverviewTab 1: Data Upload and Model SelectionTab 2: Model Results and PredictionsTab 3: Model ComparisonTab 4: Effort AllocationConclusion and Future Research2
3. 3IntroductionCritical systems increasingly software intensiveFailure of software lead to Significant loss of human life Causes environmental and economic damageReliability, security, availability of software to ensure proper operationGovernment agencies and industries that produce software canBenefit significantly from methods to quantify software reliability and security
4. Introduction (2)Non-homogeneous Poisson process (NHPP) software reliability growth models (SRGM)Quantify improvement in software achieved during testingOveremphasized progressively more complex mathematical formsDo not identify specific test activities underlying fault discoveryCovariate models are recent alternativeExplicitly link test activities to model parametersNeed structured environment to balance focus4
5. Introduction (3)Covariate models enabled formulation of optimal test activity allocation problemMaximize fault discovery within budget constraint Minimize budget required discover specified number of additional faults.5
6. Introduction (4)Previous NHPP SRGM toolsComputer-Aided Software Reliability Estimation Tool (CASRE)Software Failure and Reliability Assessment Tool (SFRAT)Existing covariate toolsNot open sourceLack graphical user interfaceNo practical optimization problems6
7. ContributionCovariate Software Failure and Reliability Assessment Tool (C-SFRAT)Implements our recent research onCovariate models Optimal allocation problem Model selection based on multiple goodness-of-fit measuresOpen sourceFlexible architecture enables extensionsNumerical and symbolic algebra used7
8. C-SFRAT Functionality8 hazard functionsPrimary functions of C-SFRAT:Import covariate failure data from spreadsheetDisplay model fit and failure intensity plotsPrediction of future failures and failure intensityComparison of fitted models using user-defined weightingTest activity allocation recommendations8
9. Example of Covariate Data9DS1 data with 17 observations …
10. Discrete Cox Proportional Hazard NHPP SRGMMean value functionwhere Discrete Cox Proportional Hazard modelNumber of faults that would be detected with indefinite testingBaseline hazard functionVector of Cox modelparameters Vector of model parameters Covariates associated with at the time of testing 10
11. Baseline Hazard FunctionsGeometric (GM):where Negative binomial of order two (NB):where and 2 indicates orderDiscrete Weibull of order two (DW):where and 2 indicates order 11
12. Software ArchitectureImplemented in Python 3, GUI in PyQt5GitHub link: https://github.com/LanceFiondella/C-SFRATExample data sets: https://lfiondella.sites.umassd.edu/research/software-reliability12
13. Software Architecture (2)13
14. Tool Overview14
15. Load failure dataExcel spreadsheet or csvSelect sheet (Excel)15Tab 1: Importing Data
16. Tab 1: Cumulative Plot of Imported Data16
17. Tab 1: Plot Views17
18. Tab 1: Failure Intensity Plot18
19. Tab 1: Model and Covariate SelectionHazard functions and covariatesBegin model fitting19
20. Tab 2: Model Results and Predictions20
21. 21Tab 2: Model Results and Predictions
22. Tab 2: Effort Specification22Enables prediction assuming operational profile
23. Tab 2: Plotting Options23
24. Tab 3: Model Comparison24Critic values close to 1 indicates best overall model fit
25. Tab 4: Effort AllocationTwo types of allocationMaximize failures given a set budgetMinimize effort to detect k additional failuresDetermine optimal allocation of resources (covariates)25
26. Tab 4: Allocation 1 (Maximize Defect Discovery)26
27. 27Tab 4: Allocation 2 (Minimize Budget)
28. Conclusion and Future ResearchPresented Covariate Software Failure and Reliability Assessment Tool (C-SFRAT)An open source software reliability tool to apply covariate modelsEight hazard rate functions and five goodness-of-fit measuresFlexible architecture allows for additional hazard functionsModels used to predict future defects and To determine optimal allocation of future testing resourcesFuture researchFormulate additional practical test activity allocation problems28
29. AcknowledgmentThis material is based upon work supported by the National Science Foundation under Grant Number #174963529
30. Resourceshttps://github.com/LanceFiondella/C-SFRAT Contact Information:Lance FiondellaAssociate ProfessorUniversity of Massachusetts Dartmouth, MA, USAEmail: lfiondella@umassd.edu 30