Metrics review

Metrics review Metrics review - Start

Added : 2016-08-06 Views :65K

Download Presentation

Metrics review




Download Presentation - The PPT/PDF document "Metrics review" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.



Presentations text content in Metrics review

Slide1

Metrics review

Claudio (SA1), Lars, Duarte, Eamonn and Maria (SA2)

Slide2

Feedback: general remarksMetrics in the Metrics ReportSuccessful buildsSLOC countBacklog managementPriority PMD/checkstyle violation densityFindbugs error densityMetrics in the SQAPNew metrics

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

2

Outline

Bug density distribution

Time to close bugsCumulative time to close bugsOpen bugsUntouched bugsFixed bugs

Process metricsPriority bug (time to handle a bug)Fixed bugs Open bugsUntouched bugsBug severity distributionBacklogSuccessful buildIntegration test effectivenessUp-to-date documentationDelay on release schedule

Product metrics

Unit test coverage

Supported platforms

Total bug density

Bug density per release

Static analysis

Cyclomatic

complexity

C/C++

Java

Python

Slide3

UNICORE: Code metrics are based on ETICS but not all code is built with ETICS.UNICORE: Metrics should have a priority so that PTs know where to put more effort. It’s not the same checkstyle than findbugs.ARC: It’s not clear who are the consumers of the metrics report. Funding bodies? SA2? JRA1? All? Current report is too long and too frequent, not good for any of the above.dCache: only make sense to PT if they know implications derived from metrics, and what metrics are exactly needed for.gLite: dashboard with metrics results needed.SA1 QC: prioritise metrics.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

3

Feedback: general remarks

Slide4

Successful buildsSLOC countBacklog managementPriority PMD/checkstyle violation densityFindbugs error densityBug density distributionTime to close bugsCumulative time to close bugsOpen bugsUntouched bugsFixed bugs

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

4

Metrics in the Metrics Report

Slide5

For each metric:Feedback from developersOur commentsAssociated Action

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

5

Proposed analysis

Slide6

No comments.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

6

Successful builds

Slide7

ARC: Doesn’t need to be calculated every night. Not even every week. No useful information for daily development.dCache: little variations thoughout EMI, so what does it really mean?

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

7

SLOC

Slide8

ARC: Wrong definition. backlog is not the open_bugs – closed_bugs, but number of non expected open bugs after the grace period, which can vary per component.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

8

Backlog management

Slide9

ARC: useful to keep code clean but doesn’t need to be evaluated nightly or weekly. Not helpful in bug fixing.dCache: % of comments is not really useful since it doesn’t check the quality of the comments.gLite: priority to Javadoc.UNICORE: PMD lower importance for Java. Use CPD. Filter warnings. Too many! Different type of checks; Checkstyle minimal importance for Java. It doesn’t help bug detection. Rules not clear. Too many errors of little importance. No worth the effort of fixing them. Use javadoc or remove metric.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

9

Error or Style violation density

Slide10

dCache: they already run it and improve code when feasible. So metric report doesn’t bring anything extra.gLite: focus on java configuration that developers find useful. Same for other languages.UNICORE: no threshold given. Sometimes the give false-positives. Proposed formula is incomplete. Output is quite short. Proposal to change metric.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

10

Findbugs

Slide11

ARC: doesn’t need to be calculated nightly. When to start evaluating it? Better time to resolve a bug. But does resolve mean the same for all middlewares?dCache: manual intervention is needed to explain certain deviations. Is this feasible?

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

11

Time to close bugs

Slide12

ARC: not useful.gLite: is the meaning of closed bug the same for all mw? Better differentiate: open to fix and to certified and then the time to go to production.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

12

Cumulative time to close bugs

Slide13

ARC: open bugs over period of time and time to solve useful but definition contradictory. Is open how can you calculate time to solve? Comparison with previous periods may be interesting. Cumulative time useless.UNICORE: bug density (bugs per KLOC) not useful.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

13

Open bugs

Slide14

ARC: useful. Moreover it helps to monitor whether priorities are properly assigned.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

14

Untouched bugs

Slide15

ARC: is it properly calculated? No double counting?

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

15

F

ixed bugs

Slide16

Process metricsPriority bug (time to handle a bug)Fixed bugs Open bugsUntouched bugsBug severity distributionBacklogSuccessful buildIntegration test effectivenessUp-to-date documentationDelay on release schedule

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

16

Metrics in the SQAP

Slide17

Product metricsUnit test coverageSupported platformsTotal bug densityBug density per releaseStatic analysisCyclomatic complexityC/C++JavaPython

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

17

Metrics in the SQAP

Slide18

Integration test effectiveness: Impossible to calculate in UNICORE.Bug density per release: difficult to calculate.Cyclomatic complexity: how is it going to be normalised? As it is now doesn’t make sense.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

18

UNICORE feedback

Slide19

dCache: functionality duplication in EMI.Testing coverage results.

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

19

New metrics

Slide20

Inform PEB which metrics can’t be calculated because of limitations in trackers, etc.Separate reports for EC, SA2 and JRA1.GGUS has to be used!

EMI Technical Forum - April 2011

SA2.2 - Software Quality Assurance in EMI

20

QA feedback

Slide21


About DocSlides
DocSlides allows users to easily upload and share presentations, PDF documents, and images.Share your documents with the world , watch,share and upload any time you want. How can you benefit from using DocSlides? DocSlides consists documents from individuals and organizations on topics ranging from technology and business to travel, health, and education. Find and search for what interests you, and learn from people and more. You can also download DocSlides to read or reference later.
Youtube