Steven Borg Principal ALM Consultant Northwest Cadence StevenBorgnwcadencecom Every best in class company measures software quality There are no exceptions If your company does not do this it is not an industry leader and there is a good chance that your software quality level ID: 722083
Download Presentation The PPT/PDF document "Metrics That Matter Real Measures to Imp..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Metrics That MatterReal Measures to Improve Software Development
Steven Borg, Principal ALM Consultant
Northwest Cadence
Steven.Borg@nwcadence.comSlide2
Every ‘best in class’ company measures software quality. There are no exceptions. If your company does not do this it is not an industry leader and there is a good chance that your software quality levels are marginal at best. Capers Jones, Applied Software MeasurementSlide3
You get what you measure.If you don’t measure it, you can’t manage it.You cannot improve what you can’t measure.Garbage in, garbage out.If you don’t measure it, it’s just a hobby."You can't manage what you can't control, and you can't control what you don't measure. " Tom DeMarcoSlide4
Metrics MatterWithout metrics you can’t predictyou can’t judge qualityyou can’t accurately estimateyou can’t measure impactsyou can’t improve consistentlySlide5
Characteristics of Effective Metrics
Comparable
Simple
Actionable
Honest Slide6
True Metrics vs Proxy MetricsTrue MetricWhat you SHOULD measureProxy MetricWhat you CAN measureTip #1Focus on the true metric,not proxy metricsSlide7
“The companies that wish to improve but do not measure are at the mercy of fads and chance. Progress may not be impossible, but it is certainly unlikely.” Capers JonesTip #2Only introduce one or two new metrics at a timeSlide8
Deming Cycle1st
2
nd
3
rd
4
th
Slide9
ITIL Seven Step Improvement ProcessDefine what you should measure.Define what you can measure.Gather the data. Who? How? When? Integrity of the data.Process the data.
Frequency, format, system, accuracy
Analyze the data.
Relationships, trends, according to plan, targets met, corrective action
Present and use the information.
Implement the corrective action.
ITIL
:
Information Technology Infrastructure LibrarySlide10
Tip #3Standardize by setting triggers to alert you to slips in prior metricsA few effective metrics:Quality MetricsSizing MetricsProductivity MetricsUser MetricsBusiness MetricsSlide11
Quality MetricsDefect Removal EfficiencyWarning: Not always “simple”Code CoverageWarning: Not always “honest”Performance ProfilingCode CoverageTest Cases per featureBugs per feature (bug density metrics)Slide12
Sizing MetricsLines of CodeWarning: Not always “honest”Function PointsWarning: Not “simple”Effort (in hours, days, weeks, etc)Warning: Not always “honest” or “comparable”
Story Points
Warning
: Rarely “comparable”Slide13
Productivity MetricsVelocityWarning: Not always “simple”ThroughputLines of Code / Developer / Day (???)Quality*Note: Defect Removal Efficiency is highly correlated with Productivity
So highly correlated it can often act as proxy metric for productivitySlide14
User MetricsUser satisfaction Warning: Not always “comparable”Post-release bug countNumber of Help Desk calls Warning: Not always “honest”Slide15
Business MetricsSchedule Estimation AccuracyWarning: Not always “honest”Cost Estimation AccuracyWarning: Not always “honest”Scope Estimation AccuracyWarning
: Rarely “honest”
ROI Estimation Accuracy
Warning
: Rarely “simple”, Not always “honest”Slide16
Tip #4First identify your problem, THEN identify the metricTip #5Balance introduction of new metrics across different categories
He broke the Build!!!
The problemSlide17
Tip #5Balance introduction of new metrics across different categoriesThe problem today is not a deficiency in software measurement technology itself; rather, it is cultural resistance on the part of software management and staff.The resistance is due to the natural human belief that measures might be used against them. This feeling is the greatest barrier to applied software measurement. Capers Jones, Applied Software MeasurementSlide18
Tip #6NEVER use metrics to evaluate individualsTip #7Watch your metrics closely; adjust when necessarySlide19
Estimation and Planning“There is a perfect correlation between measurement accuracy and estimation accuracy: Companies that measure well can estimate well; companies that do not measure accurately cannot estimate accurately either.” Capers Jones"Optimism is an occupational hazard of programming: feedback is the treatment." Kent Beck“Planning and estimation are the mirror images of measurement. ”Capers JonesSlide20
ALM (Application Lifecycle Management Tooling)
Next SlideSlide21
Tip #8Count first, Calculate next, use Judgment lastWhy ALM (Application Lifecycle Management Tooling)?
"Tools are very important element of defining a path of least resistance.
If I can set up a tool so that it’s easier for a developer to do something the way that I want the developer to do it, and harder for the developer to do it some other way, then I think it’s very likely the developer is going to do it the way I want them to, because it’s easier.
It’s the path of least resistance."
Steve McConnellSlide22
Application Lifecycle Management ToolingTip #9When possible, use an integrated ALM Tool to gather metricsTip #10Start using the built-in reports right away!Slide23
Cultural issues with MetricsThe problem today is not a deficiency in software measurement technology itself; rather, it is cultural resistance on the part of software management and staff. The resistance is due to the natural human belief that measures might be used against them. This feeling is the greatest barrier to applied software measurement. Capers Jones, Applied Software MeasurementSlide24
"The problem of software process change are often complicated by the fact that no one is responsible to make it happen. If software process improvement isn't anybody's job, it is not surprising that is doesn't get done! If it is important enough to do, however, someone must be assigned the responsibility and given the necessary resources. Until this is done, software process development will remain a nice thing to do someday, but never today." Watts HumptheySlide25
Tip #11Put someone in charge of process improvement… use metrics to show change"If it ain't broke, don't fix it," the saying goes. Common software development practices are seriously broken, and the cost of not fixing them has become extreme. Traditional thinking would have it that the change represents the greatest risk. In software's case, the greatest risk lies with not changing - staying mired in unhealthy, profligate development practices instead of switching to practices that were proven more effective many years ago.
Steve McConnellSlide26
Tip #12Start collecting metrics now. Next year, you shouldn’t be saying “I wish I had some metrics”Steven Borg