Software Process Dynamics - PowerPoint Presentation

Dollface . @Dollface

343 views
Uploaded On 2022-08-04

Software Process Dynamics - PPT Presentation

Dr Raymond Madachy Naval Postgraduate School rjmadachnpsedu INCOSE San Diego Chapter Meeting March 21 2018 2 Agenda Introduction Overview Process models and system dynamics Example applications ID: 935239

process software system model software process model system dynamics systems modeling concurrence feedback development quality project simulation cost processes

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/935239" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download Presentation The PPT/PDF document "Software Process Dynamics" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

Software Process Dynamics

Dr. Raymond Madachy

Naval Postgraduate School

rjmadach@nps.edu

INCOSE San Diego Chapter Meeting

March 21, 2018

Slide2

Agenda

Introduction

Overview

Process models and system dynamicsExample applicationsBrooks’s Law modelInspection modelValue-based product modelSpiral hybrid process model for Software-Intensive System of Systems (SISOS)Backup slidesIntroduction, background and examples

Slide3

Research Background

The evaluation of process strategies for the architecting and engineering of complex systems involves many interrelated factors.

Effective systems and software engineering requires a balanced view of technology, mission or business goals, and people.

System dynamics is a rich and integrative simulation framework used to quantify the complex interactions and the strategy tradeoffs between cost, schedule, quality and risk.

Slide4

Systems and Software Engineering Challenges

What to build? Why? How well?

Stakeholder needs balancing, mission/business case

Who to build it? Where?Staffing, organizing, outsourcingHow to build? When; in what order?Construction processes, methods, tools, components, incrementsHow to adapt to change? In user needs, technology, marketplaceHow much is enough? Functionality, quality, specifying, prototyping, test

Slide5

Software Process Dynamics

Table of Contents

Part 1 - Fundamentals

Chapter 1 – Introduction and Background Chapter 2 – The Modeling Process with System Dynamics Chapter 3 – Model Structures and Behaviors for Software Processes Part 2 – Applications and Future Directions Chapter 4 – People Applications Chapter 5 – Process and Product Applications Chapter 6 – Project and Organization Applications Chapter 7 – Current and Future Directions

Appendices and References

Appendix A- Introduction to Statistics of Simulation

Appendix B- Annotated Bibliography

Appendix C- Provided Models

Slide6

Agenda

Introduction

Overview

Slide7

Why Model Systems?

A system must be represented in some form in order to analyze it and communicate about it.

The models are abstractions of real or conceptual systems used as surrogates for low cost experimentation and study.

Models allow us to understand systems/processes by dividing them into parts and looking at how the parts are related. We resort to modeling and simulation because there are too many interdependent factors to be computed, and truly complex systems cannot be solved by analytical methods.

Slide8

Software Process Models

Used to quantitatively reason about, evaluate and optimize the software process.

Demonstrate effects of process strategies on cost, schedule and quality throughout lifecycle and enable tradeoff analyses.

Can experiment with changed processes via simulation before committing project resources.Provide interactive training for software managers; “process flight simulation”.Encapsulate our understanding of development processes (and support organizational learning).Benchmark process improvement when model parameters are calibrated to organizational data.Process modeling techniques can be used to evaluate other existing descriptive theories/models.Force clarifications, reveal discrepancies, unify fields

Slide9

System Dynamics Principles

Major concepts

Defining problems dynamically, in terms of graphs over time

Striving for an endogenous, behavioral view of the significant dynamics of a systemThinking of all real systems concepts as continuous quantities interconnected in information feedback loops and circular causalityIdentifying independent levels in the system and their inflow and outflow rates Formulating a model capable of reproducing the dynamic problem of concern by itselfDeriving understandings and applicable policy insights from the resulting modelImplementing changes resulting from model-based understandings and insights.Dynamic behavior is a consequence of system structureThe continuous view

Individual events are not tracked

Entities are treated as aggregate quantities that flow through a system, and can be described through differential equations

Discrete approaches usually lack feedback, internal dynamics

Slide10

Software Processes and System Dynamics

Software development and evolution are dynamic and complex processes

Interrelated technology, business, and people factors that keep changing

E.g. development methods and standards, reuse/COTS/open-source, product lines, distributed development, improvement initiatives, increasing product demands, operating environment, volatility, resource contention, schedule pressure, communication overhead, motivation, etc.System dynamics featuresProvides a rich and integrative framework for capturing process phenomena and their relationshipsComplex and interacting process effects are modeled using continuous flows interconnected in loops of information feedback and circular causalityProvides a global system perspective and the ability to analyze combined strategiesCan model inherent tradeoffs between schedule, cost and qualityAttractive for schedule analysis accounting for critical path flows, task interdependencies and bottlenecks not available with static models or PERT/CPM methods

Enables low cost process experimentation

System dynamics is well-suited to deal with the complexities of software processes and their improvement strategies

Slide11

System Dynamics Notation

System represented by x’(t)= f(x,p).

x: vector of levels (state variables), p: set of parameters

Legend:

Example system:

Slide12

Model Elements

Slide13

Model Elements (continued)

Slide14

Example Levels and Rates

Levels

Rates

Slide15

Example Auxiliaries

Slide16

Software Product Chain

Cycle time per task = transit time through relevant phase(s)

Cycle time per phase =

start time of first flowed entity - completion time of last flowed entity

Slide17

Error Detection and Rework Chain

Cost/schedule/quality tradeoffs available when defects are represented as levels with the associated variable effort and cycle time for rework and testing as a function of those levels.

Slide18

Personnel Chain

Slide19

Feedback Loops

A feedback loop is a closed path connecting an action decision that affects a level, then information on the level being returned to the decision making point to act on.

Slide20

Software Production Structure

Combines task development and personnel chains.

Production constrained by productivity and applied personnel resources.

Slide21

Example Delay Structure and Behavior

Delays are ubiquitous in processes and important components of feedback systems

outflow rate = level / delay time

Slide22

Typical Behavior Patterns

Slide23

Agenda

Introduction

Overview

Slide24

Brooks’s Law Modeling Example

“Adding manpower to a late software project makes it later” [Brooks 75].

We will test the law using a simple model based on the following assumptions:

New personnel require training by experienced personnel to come up to speed

More people on a project entail more communication overhead

Experienced personnel are more productive then new personnel, on average.

Model illustrates software management lessons

Slide25

Brooks’s Law Dynamics

Causal Loop Diagram with Feedback

Time Trends

Slide26

Model Diagram and Equations

Slide27

Model Output for Varying Additions

Sensitivity of Software Development Rate to Varying Personnel Allocation Pulses

(1: no extra hiring, 2: add 5 people on 100th day, 3: add 10 people on 100th day)

Days

Function points/day

Slide28

Brooks’s

Law

Model Online

Run or clone the model

https://

insightmaker.com/insight/106111/brookss-law

Slide29

Agenda

Introduction

Overview

Slide30

Inspection Model Example

Research problem addressed

What are the dynamic effects to the process of performing inspections?

Model used to evaluate process quantitativelyDemonstrates effects of inspection practices on cost, schedule and quality throughout lifecycleCan experiment with changed processes before committing project resourcesBenchmark process improvementSupport project planning and managementModel parameters calibrated to Litton and JPL data

Error generation rates, inspection effort, efficiency, productivity, others

Model validated against industrial data

Slide31

System Diagram

Slide32

System Diagram (continued)

Slide33

Effects of Inspections

3:18 PM 10/21/28

0.00

75.00

150.00

225.00

300.00

Days

0.00

13.00

26.00

1: total manpower rate

2: total manpower rate

task graphs: Page 7

1: with inspections, 2: without inspections

Qualitatively matches generalized effort curves for both cases from Michael Fagan,

Advances in software inspections

, IEEE Transactions on Software Engineering, July 1986

Slide34

Inspection Policy Tradeoff Analysis

Varying error generation rates shows diminishing returns from inspections:

Slide35

Derivation of Phase Specific Cost Driver

Slide36

Agenda

Introduction

Overview

Slide37

Value-Based Model Background

Purpose

: Support software business decision-making by experimenting with product strategies and development practices to assess

real earned valueDescription: System dynamics model relates the interactions between product specifications and investments, software processes including quality practices, market share, license retention, pricing and revenue generation for a commercial software enterprise

Slide38

Model Features

A Value-Based Software Engineering (VBSE) model covering the following VBSE elements:

Stakeholders’ value proposition elicitation and reconciliation

Business case analysisValue-based monitoring and controlIntegrated modeling of business value, software products and processes to help make difficult tradeoffs between perspectivesValue-based production functions used to relate different attributesAddresses the planning and control aspect of VBSE to manage the value delivered to stakeholdersExperiment with different strategies and track financial measures over timeAllows easy investigation of different strategy combinations Can be used dynamically before or during a projectUser inputs and model factors can vary over the project duration as opposed to a static modelSuitable for actual project usage or “flight simulation” training where simulations are interrupted to make midstream decisions

Slide39

Model Sectors and Major Interfaces

Software process and product sector computes the staffing and quality over time

Market and sales sector accounts for market dynamics including effect of quality reputation

Finance sector computes financial measures from investments and revenues

Slide40

Software Process and Product

product defect flows

effort and schedule calculation

with dynamic COCOMO variant

Slide41

Finances, Market and Sales

investment and revenue flows

software license sales

market share dynamics

including quality reputation

Slide42

Quality Assumptions

COCOMO cost driver

Required Software Reliability

is a proxy for all quality practicesResulting quality will modulate the actual sales relative to the highest potentialPerception of quality in the market mattersQuality reputation quickly lost and takes much longer to regain (bad news travels fast) Modeled as asymmetrical information smoothing via negative feedback loop

Slide43

Market Share Production Function

and Feature Sets

Cases from Example 1

Slide44

DoD Analog:

Mission Effectiveness

Production Function and Feature Sets

Added Mission Effectiveness Percent

Slide45

Sales Production Function and Reliability

Cases from Example 1

Slide46

DoD Analog:

Product Illity

Production Function

Reliability or Other Product Illity Rating

Slide47

Example 1

: Dynamically

Changing Scope and Reliability

Shows how model can assess the effects of combined strategies by varying the scope and required reliability independently or simultaneouslySimulates midstream descoping, a frequent strategy to meet time constraints by shedding featuresThree cases are demonstrated:Unperturbed reference caseMidstream descoping of the reference case after ½ yearSimultaneous midstream descoping and lowered required reliability at ½ year

Slide48

Control Panel and Simulation Results

Unperturbed

Reference Case

Case 2

Case 1

Descope

Descope +

Lower Reliability

Slide49

Case Summaries

Case

Delivered

Size

(Function

Points)

Delivered

Reliability

Setting

Cost

($M)

Delivery

Time

(Years)

Final

Market

ROI

Reference Case:

Unperturbed

700

1.0

4.78

2.1

28%

1.3

Case 1:

Descope

at Time = ½ years

550

1.0

3.70

1.7

28%

2.2

Case 2:

Descope and

Lower Reliability

at Time = ½ years

550

.92

3.30

1.5

12%

1.0

Slide50

Example 2

: Determining the

Reliability Sweet Spot

Analysis processVary reliability across runsUse risk exposure framework to find process optimumAssess risk consequences of opposing trends: market delays and bad quality losses Sum market losses and development costsCalculate resulting net revenueSimulation parametersA new 80 KSLOC product release can potentially increase market share by 15%-30% (varied in model runs)75% schedule accelerationInitial total market size = $64M annual revenueVendor has 15% of market

Overall market doubles in 5 years

Slide51

Cost Components

3-year time horizon

Slide52

To achieve real earned value, business value attainment must be a key consideration when designing software products and processes

Software enterprise decision-making can improve with information from simulation models that integrate business and technical perspectives

Optimal policies operate within a multi-attribute decision space including various stakeholder value functions, opposing market factors and business constraints

Risk exposure is a convenient framework for software decision analysisCommercial process sweet spots with respect to reliability are a balance between market delay losses and quality lossesModel demonstrates a stakeholder value chain whereby the value of software to end-users ultimately translates into value for the software development organization

Value-Based Model Conclusions

Slide53

Value-Based Model Future Work

Enhance product defect model with dynamic version of COQUALMO to enable more constructive insight into quality practices

Add maintenance and operational support activities in the workflows

Elaborate market and sales for other considerations including pricing scheme impacts, varying market assumptions and periodic upgrades of varying qualityAccount for feedback loops to generate product specifications (closed-loop control) External feedback from users to incorporate new featuresInternal feedback on product initiatives from organizational planning and control entity to the software processMore empirical data on attribute relationships in the model will help identify areas of improvementAssessment of overall dynamics includes more collection and analysis of field data on business value and quality measures from actual software product rollouts

Slide54

Agenda

Introduction

Overview

Slide55

Spiral Hybrid Process Introduction

The spiral lifecycle is being extended to address new challenges for Software-Intensive Systems of Systems (SISOS), such as coping with rapid change while simultaneously assuring high dependability

A hybrid plan-driven and agile process has been outlined to address these conflicting challenges with the need to rapidly field incremental capabilities

A system-of-systems (SOS) integrates multiple independently-developed systems and is very large, dynamically evolving, unprecedented, with emergent requirements and behaviorsHowever, traditional static approaches cannot capture dynamic feedback loops and interacting phenomena that cause real-world complexity (e.g. hybrid processes, project volatility, increment overlap and resource contention, schedule pressure, slippages, communication overhead, slack, etc.)A system dynamics model is being developed to assess the incremental hybrid process and support project decision-makingBoth the hybrid process and simulation model are being evolved on a very large scale incremental project for a SISOS (U.S. Army Future Combat Systems)

Slide56

Future Combat Systems (FCS) Network

Slide57

Scalable Spiral Model Increment Activities

Organize development into plan-driven increments with stable specs

Agile team watches for and assesses changes, then negotiates changes so next increment hits the ground running

Try to prevent usage feedback from destabilizing current increment

Three team cycle plays out from one increment to the next

Slide58

Spiral Hybrid Model Features

Estimates cost and schedule for multiple increments of a hybrid process that uses three specialized teams (agile re-baseliners, developers, V&V’ers) per the scalable spiral model

Considers changes due to external volatility and feedback from user-driven change requests

Deferral policies and team sizes can be experimented withIncludes tradeoffs between cost and the timing of changes within and across increments, length of deferral delays, and others

Slide59

Model Input Control Panel

Slide60

Model Overview

Built around a cyclic flow chain for capabilities

Arrayed for multiple increments

Each team is represented with a level and corresponding staff allocation rate

Changes arrive a-periodically via the

volatility trends

time function and flow

into the

level for

capability changes

Changes are

processed by the agile team and

allocated to increments per the deferral policies

Constant or variable staffing for the agile team

For each increment the

required capabilities

are developed into

developed capabilities

and then V&V’ed into

V & V’ed capabilities

Productivities and team sizes for development and V&V calculated with a Dynamic COCOMO variant and continuously updated for scope changes

Flow rates between

capability changes

and

V & V’ed capabilities

are bi-directional for capability “kickbacks” sent back up the chain

User-driven changes from the field are identified as

field issues

that flow back into the

capability changes

Slide61

Volatility Cost Functions

The

volatility effort multiplier

for construction effort and schedule is an aggregate multiplier for volatility from different sources (e.g. COTS, mission, etc.) relative to the original baseline for increment

The

lifecycle timing effort multiplier

models increased development cost the later a change comes in midstream during an increment

Slide62

Sample Response to Volatility

An unanticipated change occurs at month 8 shown as a

volatility trend [1]

pulseIt flows into capability changes [1] which declines to zero as the agile team processes the changeThe change is non-deferrable for increment 1 so total capabilities [1] increasesDevelopment team staff size dynamically responds to the increased scope

* [1] refers to increment #1

Slide63

Sample Test Results

Test case for two increments of 15 baseline capabilities each

A non-deferrable change comes at month 8 (per previous slide)

The agile team size is varied from 2 to 10 peopleIncrement 1 mission value also lost for agile team sizes of 2 and 4

Slide64

Sample Test Results (cont.)

Slide65

System dynamics is a convenient modeling framework to deal with the complexities of a SISOS

A hybrid process appears attractive to handle SISOS dynamic evolution, emergent requirements and behaviors

Initial results indicate that having an agile team can help decrease overall cost and schedule

Model can help find the optimum balanceWill obtain more empirical data to calibrate and parameterize model including volatility and change trends, change analysis effort, effort multipliers and field issue ratesModel improvementsAdditional staffing optionsRayleigh curve staffing profilesConstraints on development and V&V staffing levelsMore flexible change deferral options across incrementsIncrement volatility balancing policiesProvisions to account for (timed) business/mission value of capabilitiesAdditional model experimentation

Include capabilities flowing back from developers and V&V’ers

Vary deferral policies and volatility patterns across increments

Compare different agile team staffing policies

Continue applying the model on a current SISOS and seek other potential pilots

Spiral Hybrid Model

Conclusions and Future Work

Slide66

Agenda

Introduction

Overview

Slide67

Backup Slide Outline

Research introduction

Processes and system dynamics

Example model structures and system behaviors Software Process Dynamics book chaptersExamplesInspection model supplementSoftware cost and quality tradeoff simulation tool (NASA)Process concurrence modeling

Slide68

System

: a grouping of parts that operate together for a common purpose; a subset of reality that is a focus of analysis

Open, closed

Software Process

: a set of activities, methods, practices and transformations used by people to develop software.

Model

: an abstract representation of reality.

Static, dynamic; continuous, discrete

Simulation

: the numerical evaluation of a mathematical model.

System dynamics

: a simulation methodology for modeling continuous systems. Quantities are expressed as levels, rates and information links representing feedback loops.

Terminology

Slide69

Process Modeling Characterization Matrix

and Examples

Example Litton studies in [Madachy et al. 2000]

Slide70

Systems Thinking

A way to realize the structure of a system that leads to it’s behavior

Systems thinking involves:

Thinking in circles and considering interdependenciesClosed-loop causality vs. straight-line thinkingSeeing the system as a cause rather than effectInternal vs. external orientationThinking dynamically rather than staticallyOperational vs. correlational orientation Improvement through organizational learning takes place via shared mental modelsThe power of models increase as they become more explicit and commonly understood by people A context for interpreting and acting on dataSystem dynamics is a methodology to implement systems thinking and leverage learning efforts

Slide71

Software Process Control System

Software Process

Software Artifacts

Requirements, resources etc.

internal project feedback

external feedback from operational environment

Software Development or Evolution Project

Slide72

A Software Process

Slide73

Modeling Process Overview

Iterative, cyclic

policy implementation

system understandings

problem definition

model conceptualization

model formulation

simulation

policy analysis

Slide74

Modeling Stages and Concerns

problem definition

model conceptualization

model formulation

simulation

evaluation

context; symptoms

reference behavior modes

model purpose

system boundary

feedback structure

model representation

model behavior

reference behavior modes

Slide75

Backup Slide Outline

Research introduction

Processes and system dynamics

Slide76

Error Co-flows

Slide77

Learning Curve

Slide78

General System Behaviors

Behaviors are representative of many known types of systems.

Knowing how systems respond to given inputs is valuable intuition for the modeler

Can be used during model assessmentuse test inputs to stimulate the system behavioral modes

Slide79

System Order

The

order

of a system refers to the number of levels contained.A single level system cannot oscillate, but a system with at least two levels can oscillate because one part of the system can be in disequilibrium.

Slide80

Example System Behaviors

Delays

Goal-seeking Negative Feedback

First-order Negative FeedbackSecond-order Negative FeedbackPositive Feedback Growth or DeclineS-curves

Slide81

Delays

Time delays are ubiquitous in processes

They are important structural components of feedback systems.

Example: hiring delays in software development. the average hiring delay represents the time that a personnel requisition remains open before a new hire comes on board

Slide82

Third Order Delay

A series of 1st order delays

Graphs show water levels over time in each tank

tank 1 starts full

Slide83

Delay order Pulse input Step input

Infinite

(pipeline)

Delay Summary

input

output

Slide84

Negative Feedback

Negative feedback exhibits goal seeking behavior, or sometimes instability

May represent hiring increase towards a staffing goal. The change is more rapid at first and slows down as the discrepancy between desired and perceived decreases. Also a good trend for residual defect levels.

zero goal

positive goal

Analytically

Level = Goal + (Level

- Goal)e

-t/tc

rate = (goal - present level)/time constant

Slide85

Orders of Negative Feedback

First-order Negative Feedback

Second-order Negative Feedback

Oscillating behavior may start out with exponential growth and level out. It could represent the early sales growth of a software product that stagnates due to satisfied market demand, competition or declining product quality.

Slide86

Positive Feedback

Positive feedback produces a growth process

Exponential growth may represent sales growth (up to a point), Internet traffic, defect fixing costs over time

rate = present level*constant

Analytically

exponential growth:

Level

exponential decay:

Level

-t/TC

Slide87

S-Curves

S-curve

: graphic display of a quantity like progress or cumulative effort plotted against time that exhibits an s-shaped curve. It is flatter at the beginning and end, and steeper in the middle. It is produced on a project that starts slowly, accelerates and then tails off as work tapers off

S-curves are also observed in the ROI curve of technology adoption, either time-based return or in production functions that relate ROI to investment.

Slide88

Backup Slide Outline

Research introduction

Processes and system dynamics

Slide89

INTRODUCTION AND BACKGROUND

Foreword by Barry BoehmPreface1.1 Systems, Processes, Models and Simulation1.2 Systems Thinking1.3 Basic Feedback Systems Concepts Applied to the Software Process1.4 Brooks’s Law Example1.5 Software Process Technology Overview1.6 Challenges for the Software Industry1.7 Major References1.8 Chapter 1 Summary

1.9 Exercises

Slide90

THE MODELING PROCESS WITH SYSTEM DYNAMICS

2.1 System Dynamics Background2.2 General System Behaviors2.3 Modeling Overview2.4 Problem Definition2.5 Model Conceptualization2.6 Model Formulation and Construction2.7 Simulation2.8 Model Assessment2.9 Policy Analysis2.10 Continuous Model Improvement

2.11 Software Metrics Considerations

2.12 Project Management Considerations

2.13 Modeling Tools

2.14 Major References

2.15 Chapter 2 Summary

2.16 Exercises

Slide91

3. MODEL STRUCTURES AND BEHAVIOR FOR SOFTWARE PROCESSES

3.1 Introduction

3.2 Model Elements

3.3 Generic Flow Processes3.4 Infrastructures and Behaviors3.5 Software Process Chain Infrastructures3.6 Major References3.7 Chapter 3 Summary3.8 Exercises

Slide92

4. PEOPLE APPLICATIONS

4.1

INTRODUCTION

4.2

OVERVIEW OF APPLICATIONS

4.3

PROJECT WORKFORCE MODELING

4.4

EXHAUSTION AND BURNOUT

4.5

LEARNING

4.6

TEAM COMPOSITION

4.7

OTHER APPLICATION AREAS

4.8

MAJOR REFERENCES

4.9

CHAPTER 4 SUMMARY

4.1

EXERCISES

Slide93

5. PROCESS AND PRODUCT APPLICATIONS

5.1

INTRODUCTION

5.2

OVERVIEW OF APPLICATIONS

5.3

PEER REVIEWS

5.4

GLOBAL PROCESS FEEDBACK (SOFTWARE EVOLUTION)

5.5

SOFTWARE REUSE

5.6

COMMERCIAL OFF-THE-SHELF SOFTWARE (COTS) - BASED SYSTEMS

5.7

SOFTWARE ARCHITECTING

5.8

QUALITY AND DEFECTS

5.9

REQUIREMENTS VOLATILITY

5.1

SOFTWARE PROCESS IMPROVEMENT

5.11

MAJOR REFERENCES

5.12

PROVIDED MODELS

5.13

CHAPTER 5 SUMMARY

5.14

EXERCISES

Slide94

6. PROJECT AND ORGANIZATION APPLICATIONS

6.1

INTRODUCTION

6.2

OVERVIEW OF APPLICATIONS

6.3

INTEGRATED PROJECT MODELING

6.4

SOFTWARE BUSINESS CASE ANALYSIS

6.5

PERSONNEL RESOURCE ALLOCATION

6.6

STAFFING

6.7

EARNED VALUE

6.8

MAJOR REFERENCES

6.9

PROVIDED MODELS

6.1

CHAPTER 6 SUMMARY

6.11

EXERCISES

Slide95

CURRENT AND FUTURE DIRECTIONS

7.1 Introduction7.2 Simulation Environments and Tools7.3 Model Structures and Component-Based Model Development7.4 New and Emerging Trends for Applications7.5 Model Integration7.6 Empirical Research and Theory Building7.7 Mission Control Centers and Training Facilities7.8 Chapter 8 Summary7.9 Exercises

Slide96

Appendices

Appendix A: Introduction to Statistics of Simulation

A.1 RISK ANALYSIS AND PROBABILITY

A.2 PROBABILITY DISTRIBUTIONSA.4 ANALYSIS OF SIMULATION INPUTA.5 EXPERIMENTAL DESIGNA.6 ANALYSIS OF SIMULATION OUTPUTA.7 MAJOR REFERENCESA.8 APPENDIX A SUMMARYA.9 EXERCISESAppendix B: Annotated System Dynamics BibliographyAppendix C: Provided Models

Slide97

Examples of

Provided Models (Ch. 6 Only)

Slide98

Backup Slide Outline

Research introduction

Processes and system dynamics

Slide99

Validation to Empirical Data

Using 329 Litton inspections and 203 JPL inspections

Project/Test Case

Test Effort Reduction

Test Schedule Reduction

Litton Project a compared to previous project

50%

25%

Test case 11 with Litton productivity constant and job size compared to test case 1.3 with Litton parameters

48%

19%

Test case 1.1 compared to test case 1.3

48%

21%

Project/Test Case

Effort Ratio of Rework to Preparation and Meeting

Litton project

.47

JPL projects

.45

Test case 1.1

.49

Simulated ROI within 15% of actual ROI

Slide100

100

Sample Project Progress Trends

From [Madachy 94]

8:18 AM 11/3/28

0.00

75.00

150.00

225.00

300.00

Days

0.00

266.65

533.30

0.00

266.63

533.27

0.00

266.63

533.27

0.00

0.50

1.00

0.00

130.12

260.25

1: cum tasks design…

2: cum tasks coded

3: tasks tested

4: fraction done

5: actual completio…

task graphs: Page 1

Slide101

101

Error Multiplication Effects

Slide102

102

Risk Analysis

A deterministic point estimate from a simulation run is only one of many actual possibilities

Simulation models are ideal for exploring risk

test the impact of input parameters

test the impact of different policies

Monte-Carlo analysis takes random samples from an input probability distribution

Slide103

103

Monte-Carlo Example

Results of varying inspection efficiency:

Slide104

104

Contributions of Inspection Model

Demonstrated dynamic effects of performing inspections.

Validated against empirical industry data

New knowledge regarding interrelated factors of inspection effectiveness.Demonstrated complementary features of static and dynamic models.Techniques adopted in industry.

Slide105

105

Backup Slide Outline

Research introduction

Processes and system dynamics

Example model structures and system behaviors Software Process Dynamics book chaptersExamplesInspection model supplementSpiral hybrid process model for Software-Intensive System of Systems (SISOS)Software cost and quality tradeoff simulation tool (NASA)Process concurrence modeling

Slide106

106

Backup Slide Outline

Research introduction

Processes and system dynamics

Slide107

107

Software Cost/Quality Tradeoff Tool (NASA)

Orthogonal Defect Classification (ODC) COQUALMO system dynamics model working prototype

ODC defect distribution pattern per JPL studies [Lutz and Mikulski 2003]

Includes effort estimationIncludes tradeoffs of different detection efficiencies for the removal practices per type of defect

Slide108

108

Software Cost/Quality

Simulation Tradeoff Tool Demo

Slide109

109

Backup Slide Outline

Research introduction

Processes and system dynamics

Slide110

110

Process Concurrence Introduction

Process concurrence

: the degree to which work becomes available based on work already accomplished

represents an opportunity for parallel worka framework for modeling constraint mechanics Increasing task parallelism is a primary RAD opportunity to decrease cycle timeSystem dynamics is attractive to analyze schedulecan model task interdependencies on the critical path

Slide111

111

Trying to Accelerate Software Development

development rate

software tasks

restricted channel flow

tasks to

develop

completed

tasks

personnel

(partially adapted from

Putnam 80)

Slide112

112

Limited Parallelism of Software Activities

There are always sequential constraints independent of phase:

analysis and specification; figure out what you're supposed to do

development of something (architecture, design, code, test plan, etc.)assessment: verify/validate/review/debugpossible rework recycle of previous activitiesThese can't be done totally in parallel with more applied peopledifferent people can perform the different activities with limited parallelism, but downstream activities will always have to follow some of the upstream

Slide113

113

Lessons from Brooks in

The Mythical Man-Month

Sequential constraints imply tasks cannot be partitioned.

applying more people has no effect on scheduleMen and months are interchangeable only when tasks can be partitioned with no communication among them.

Slide114

114

Process Concurrence Basics

Process concurrence describes interdependency constraints between tasks

can be an internal constraint within a development stage or an external constraint between stages

Describes how much work becomes available for completion based on previous work accomplished Accounts for realistic bottlenecks on work availabilityvs. a model driven solely by resources and productivity that can finish in almost zero time with infinite resources Concurrence relations can be sequential, parallel, partially concurrent, or other dependent relationships

Slide115

115

Internal Process Concurrence

Internal process concurrence relationship shows how much work can be done based on the percent of work already done.

The relationships represent the degree of sequentiality or concurrence of the tasks aggregated within a phase.

Slide116

116

Internal Concurrence Examples

Simple conversion task where tasks

can be partitioned with no communication

Complex system development where

tasks are dependent due to required

inter-task communication.

initial work on important segments; other segments have to wait until these

are done

region of parallel work

less parallel

integration

Slide117

117

External Process Concurrence

External process concurrence relationships describe constraints on amount of work that can be done in a downstream phase based on the percent of work released by an upstream phase.

See examples on following slide

More concurrent processes have curves near the upper left axes, and less concurrent processes have curves near the lower and right axes.Tasks can be considered to be the number of function points demonstrable in their phase-native form

Slide118

118

1 - a linear lockstep concurrence in the

development of totally independent modules

2 - S-shaped concurrence for new, complex

development where an architecture core is

needed first

3 - highly leveraged instantiation like COTS

with some glue code development

4 - a slow design buildup between phases

External Concurrence Examples

Slide119

119

Roles Have Different Mental Models

Differing perceptions upstream and downstream (Ford-Sterman 97)

Group visualization helps identify disparities to improve communication and reduce conflict.

Slide120

120

RAD Modeling Example

One way to achieve RAD is by having base software architectures tuned to application domains available for instantiation, standard database connectors and reuse.

The next two slides contrast the concurrence of an HR portal development using two different development approaches 1) from scratch and 2) with an existing HR base architecture.

Slide121

121

Example: Development from Scratch

Slide122

122

Architecture Approach Comparison

Opportunity

for increased task

parallelism and

quicker

elaboration

Slide123

123

Rayleigh Curve Applicability

Rayleigh curve was based on initial studies of hardware research and development

projects resemble traditional waterfall development for unprecedented systems

Rayleigh staffing assumptions don’t hold well for COTS, reuse, architecture-first design patterns, 4th generation languages or staff-constrained situationsHowever an “ideal” staffing curve is proportional to the number of problems ready for solution (from a product perspective).

= *

Slide124

124

Process Concurrence Advantages

Process concurrence can model more realistic situations than the Rayleigh curve and produce varying dynamic profiles

Can be used to show when and why Rayleigh curve modeling doesn’t apply

Process concurrence provides a way of modeling constraints on making work available, and the work available to perform is the same dynamic that drives the Rayleigh curvesince the staff level is proportional to the problems (or specifications) ready to implement

Slide125

125

External Concurrence Model

the time profile of tasks

ready to elaborate ~

“ideal” staffing curve shape

Slide126

126

Simulation Results and Sample Lessons

flat Rayleigh COTS pulse

at front

N/A

Critical customer decision delays

slow progress

- e.g. can’t design until timing

performance specs are known

Early stakeholder concurrence

enables RAD

- e.g. decision on architectural

framework or COTS package

Slide127

127

Additional Considerations

Process concurrence curves can be more precisely matched to the software system types

COTS by definition should exhibit very high concurrence

Mixed strategies produce combined concurrence relationships

E.g. COTS first then

new development:

Slide128

128

Process Concurrence Conclusions

Process concurrence provides a robust modeling framework

a method to characterize different approaches in terms of their ability to parallelize or accelerate activities

Gives a detailed view of project dynamics and is relevant for planning and improvement purposesa means to collaborate between stakeholders to achieve a shared planning vision Can be used to derive optimal staffing profiles for different project situationsMore generally applicable than the Rayleigh curveMore empirical data needed on concurrence relationships from the field for a variety of projects

Slide129

129

References

Abdel-Hamid T, Madnick S,

Software Project Dynamics

, Englewood Cliffs, NJ, Prentice-Hall, 1991 Boehm B, Huang L, Jain A. Madachy R, “ The ROI of Software Dependability: The iDAVE Model”, IEEE Software Special Issue on Return on Investment, May/June 2004Boehm B, Software Engineering Economics. Englewood Cliffs, NJ, Prentice-Hall, 1981Boehm B and Huang L, “Value-Based Software Engineering: A Case Study, IEEE Computer, March 2003Boehm B., Abts C., Brown A.W., Chulani S., Clark B., Horowitz E., Madachy R., Reifer D., Steece B., Software Cost Estimation with COCOMO II, Prentice-Hall, 2000 Boehm B., Turner R.,

Balancing Agility and Discipline

, Addison Wesley, 2003

Boehm B., Brown A.W., Basili V., Turner R., “Spiral Acquisition of Software-Intensive Systems of Systems”, CrossTalk. May 2004

Boehm B., “Some Future Trends and Implications for Systems and Software Engineering Processes”, USC-CSE-TR-2005-507, 2005

Brooks F,

The Mythical Man-Month

, Reading, MA, Addison-Wesley, 197

Chulani S,

Boehm B, “

Modeling Software Defect Introduction and Removal: COQUALMO (COnstructive QUALity MOdel)”, USC-CSE Technical Report 99-510, 1999

Forrester JW,

Industrial Dynamics

. Cambridge, MA: MIT Press, 1961

Kellner M, Madachy R, Raffo D,

Software Process Simulation Modeling: Why? What? How?,

Journal of Systems and Software, Spring 1999

Slide130

130

References (cont.)

Madachy R,

A software project dynamics model for process cost, schedule and risk assessment, Ph.D. dissertation

, Department of Industrial and Systems Engineering, USC, December 1994Madachy R, System Dynamics and COCOMO; Complementary Modeling Paradigms, Proceedings of the Tenth International Forum on COCOMO and Software Cost Modeling, SEI, Pittsburgh, PA, 1995 Madachy R, System Dynamics Modeling of an Inspection-Based Process, Proceedings of the Eighteenth International Conference on Software Engineering, IEEE Computer Society Press, Berlin, Germany, March 1996Madachy R, Tarbet D, Case Studies in Software Process Modeling with System Dynamics, Software Process Improvement and Practice, Spring 2000Madachy R, Simulation in Software Engineering, Encyclopedia of Software Engineering, Second Edition, Wiley and Sons, Inc., New York, NY, 2001

Madachy R,

Integrating Business Value and Software Process Modeling

, Proceedings of SPW/ProSim 2005, Springer-Verlag, May 2005

Madachy R, Boehm B, Lane J,

Spiral Lifecycle Increment Modeling for New Hybrid Processes

, Journal of Systems and Software, 2007 (to be published)

Madachy R.,

Software Process Dynamics

, Wiley-IEEE Computer Society, 2008

Reifer D.,

Making the Software Business Case

, Addison Wesley, 2002