Software Classic Disasters

Software Classic Disasters

CS 577b Software Engineering II

Supannika Koolmanojwong

“Insanity: doing the same thing over and over again and expecting different results.” — Albert Einstein

2

Outline

IT Project Management: Infamous Failures, Classic Mistakes, and Best PracticesTop 10 Worst Practices

3

IT Project Management: Infamous Failures, Classic Mistakes, and Best Practices R. Ryan Nelson , MIS Quarterly Executive Vol. 6 No. 2 / June 2007

Retrospectives by project postmortems or post-implementation reviews 99 retrospectives conducted in 74 organizations over the past 7 years “Insanity: doing the same thing over and over again and expecting different results.” — Albert Einstein

4

10 of the most infamous IT project failures

Large magnitudeOver $100 millionOne-half come from the public sectorwasted taxpayer dollars lost servicesthe other half - the private sectorbillions of dollars in added costslost revenueslost jobs.

5

1. Internal Revenue Service (IRS)1999

PROJECT: Business Systems Modernization; Launched in 1999 to upgrade the agency’s IT infrastructure and more than 100 business applications$8 billion modernization project , team of vendorsa complex project overwhelms the management capabilities of both vendor and client. the most expensive systems development “fiasco” in history, with delays costing the U.S. Treasury tens of billions of dollars per year. ability to collect revenue, conduct audits, and go after tax evaders was severely compromised

http://www.cio.com/article/32197/No_Easy_IT_Fix_for_the_IRS?page=1&taxonomyId=3195

6

2. Federal Aviation Administration, 1996

PROJECT: Advanced Automation System (AAS); FAA’s effort to modernize the nation’s air traffic control system. Estimated to cost $2.5 billion ( $1.5 billion is wasted)Numerous delays and cost overruns, which were blamed on both the FAA and the primary contractor, IBM. Technical complexity of the effort, bad resource estimation, ineffectively requirements control "For example, they wanted the system to have only 3 seconds of downtime a year. But to get the data to prove that requirement had been met would have taken about 10 years” (later on change to 5 minutes downtime) Instead of admitting the problem, IBM turned AAS into a research projectThe project collapsed

http://www.baselinemag.com/c/a/Projects-Processes/The-Ugly-History-of-Tool-Development-at-the-FAA/

7

3. Federal Bureau of Investigation, 20004

PROJECT: “Trilogy;” Four-year, $500M overhaul of the FBI’s antiquated computer system. Ill-defined requirements, changed dramatically after 9/11 (agency mission switched from criminal to intelligence focus)$170 million project was abandoned altogether400 problems with early versions of the troubled software, but never told the contractorThe bureau went ahead with a $17 million testing program even the software would have to be scrapped

8

4. McDonalds, 2001

PROJECT: “Innovate;” Digital network for creating a real-time enterprise planned to spend $1 billion over five yearsObjective:  to better serve customers by using information and communications technologies to monitor the quality of products and servicesExecutives in company headquarters would have been able to see how soda dispensers and frying machines in every store were performing, at any moment. Would need $1billion for infrastructure, and $zillions to maintain and upgradeAfter two years and $170M, the fast food giant threw in the towel.

9

5. Denver International Airport 1994

PROJECT: Baggage-handling system. It took 10 years and at least $600 million to figure out big muscles, not computers, can best move baggage The baggage system, designed and built by BAE Automated Systems Inc., launched, chewed up, and spit out bags so often that it became known as the “baggage system from hell.”

http://www.computerworld.com/s/article/102405/United_axes_troubled_baggage_system_at_Denver_airport?taxonomyId=73&pageNumber=1

10

6. AMR Corp., Budget Rent A Car Corp., Hilton Hotels Corp., Marriott International Inc, 1992

PROJECT: “Confirm;” Reservation system for hotel and rental car bookings After four years and $125 million in development, when it became clear that Confirm would miss its deadline by as much as two years. Was supposed to be a leading edge comprehensive travel industry reservation program combining airline, rental car and hotel informationMajor problems surfaced when Hilton tested the system, then 18 months delay and the problems could not be resolved

http://sunset.usc.edu/classes/cs510_2004/notes/confirm.pdf

11

7. Bank of America, 1988

PROJECT: “MasterNet;” Trust accounting system. hardware problems caused the Bank of America (BofA) to lose control of several billion dollars of trust accounts. All the money was eventually found in the system, but all 255 people in the entire Trust Department were fired, as all the depositors withdrew their money. This is a classic case study on the need for risk assessment, including people, process, and technology-related risk. BofA spent $60M to fix the $20M project before deciding to abandon it altogether. BofA fell from being the largest bank in the world to No. 29CRACK stakeholders problems, bad modular design, focusing in competing with competitors-but ready for transition

12

8. Kmart, 2000

PROJECT: IT systems modernization $1.4 billion IT modernization effortaimed at linking its sales, marketing, supply, and logistics systems. 18 months later, cash-strapped Kmart cut back on modernization, writing off the $130 million it had already invested in IT. Four months later, it declared bankruptcyFailing to allocate enough money and manpower to not clearly establishing the IT project's relationship to the organization's business

13

9. London Stock Exchange, 1993

PROJECT: “Taurus;” Paperless share settlement system. £800 million, original budget £6 million Abandoned after 10 years of developmentBy Vista Concepts, US, for database management. Although being very good for on-line real time processing, it could not handle distributed data processing or batch processingLSE tried to modify Vista by rewriting almost 60% of it, hence hidden bugs and long delaysGrew from a settlement only system, to become a full “share registration and transfer system”.

http://www.it-cortex.com/Examples_f.htm

14

10. Nike, 2000

PROJECT: Integrated enterprise software$400 million installing ERP, CRM, and SCM—the full complement of analyst-blessed integrated enterprise software. Caused major inventory glitch, over-produced some shoe models and under-produced others profits drop by $100 million

15

Classic Mistakes

Behind scheduleAdd more peopleWant to speed up developmentCut testingA new version of OS becomes available during the project, Time for an upgrade! Key contributors aggravating the rest of the team? Wait until the end of the project to fire him!

16

Classic Mistakes: People

Undermined motivationproductivity and qualityIndividual capabilities of the team members or the working relationshipsFailure to take action to deal with a problem employeeAdding people to a late projectpouring gasoline on a fire

17

Classic Mistakes: Process

BDUF – Big Design Up FrontUnderestimate, overly optimistic schedules, under scoping it, undermining effective planning, and shortchanging requirements determination and/or quality assurancePoor estimation also puts excessive pressure on team members, leading to lower morale and productivity.Insufficient risk managementcontractor failure - outsourcing and offshoring

18

Classic Mistakes: Product

FAA’s modernization effort, where the goal was 99.99999% reliability, which is referred to as “the seven nines.”Requirements gold-platingFeature creepaverage project experiences about a +25% change in requirements over its lifetime.Developer gold-plating - new technology that are required in the product.Research-oriented developmentSilver-bullet syndromeOverestimated savings from new tools or methodsSwitching tools in the middle of a project

19

A Meta-Retrospective of 99 IT Projects

process mistakes (45%), people mistakes (43%) product mistakes (8%) or technology mistakes (4%).project managers should be experts in managing processes and people.Scope creep didn’t make the top ten mistakesAs long as project manager pays attention to itContractor failure has been climbing in frequency in recent yearsIf the project managers had focused their attention on better estimation and scheduling, stakeholder management, and risk management, they could have significantly improved the success of the majority of the projects studied.

20

21

Avoid classic mistakes through best practices

Avoiding Poor Estimating and/or SchedulingCost overrun, 1994-180%, 2003-43%, Schedule overrun, 2000- 63%, 2007-82%.cone of uncertainty by multiplying the “most likely” single-point estimate by the optimistic factor lower bounds - optimistic estimateupper bounds - pessimistic estimate.Capital One100% cushion - beginning of the feasibility phase75% cushion in the definition phase50% cushion in design25% cushion at the beginning of construction

22

Avoiding Poor Estimating and/or Scheduling

Valuable approaches to improving project estimation and scheduling Timebox development shorter, smaller projects are easier to estimate, creating a work breakdown structure to help size and scope projectsretrospectives to capture actual size, effort and time data for use in making future project estimatesa project management office to maintain a repository of project data over time.

23

Avoiding Ineffective Stakeholder Management

ineffective stakeholder management is the second biggest cause of project failure Have to knowwho has influence over otherswho has direct control of resourcesstakeholder level of intereststakeholder degree of support/resistance

24

Avoiding Insufficient Risk Management

risk identification, analysis, prioritization, risk-management planning, resolution, and monitoring.Methods/ toolsa prioritized risk assessment tablea top-10 risks list, interim retrospectivesappointing a risk officer

25

Avoiding Insufficient Planning

Ensure the followingsClear roles and responsibilitiesResource allocationSchedule / timelineFollow project policies, plans, and procedures

26

Avoiding Shortchanging Quality Assurance

When a project falls behind schedule, the first two areas that often get cut are testing and training.Cut corners by eliminating test planning, eliminating design and code reviews, and performing only minimal testingSuggestions:agile development, joint application design sessions, automated testing tools, and daily build-and-smoke tests.

27

Avoiding Weak Personnel and/or Team Issues

get the right people assigned to the project from the beginningBetween 1999 and 2006, the retrospectives reported an increasing number of problems with distributed, inter-organizational, and multi-national teams.reduction in face-to-face team meetings, time-zone barriers, and language and cultural issues

28

Avoiding Insufficient Project Sponsorship

Not only getting top management support, but identifying the right sponsor From the beginning !!!

29

30

Outline

IT Project Management: Infamous Failures, Classic Mistakes, and Best PracticesTop 10 Worst Practices

31

There are two kinds of failures:

those who thought and never did and those who did and never thought.

Worst Practices Capers Jones, "Our Worst Current Development Practices," IEEE Software, vol. 13, no. 2, pp. 102-104, Mar. 1996

Project failuresterminated because of cost or schedule overrunexperienced schedule or cost overruns in excess of 50 percent of initial estimatesresulted in client lawsuits for contractual noncompliance

33

Worst Practice #1 No historical software-measurement

Lack of historical data makes stakeholders blind to see the realities of software developmentNeed to check on schedule, cost, progress, performance

34

Worst Practice #2 Rejection of accurate estimates

No accurate estimate is the root cause for the rest of the worst practices including:inability to perform return-on-investment calculationssusceptibility to false claims by tool and method vendors software contracts that are ambiguous and difficult to monitor.

35

Worst Practice #3 & 4Failure to use automated estimating tools and automated planning tools.

50 commercial software-cost estimating tools Checkpoint, COCOMO, Estimacs, Price-S, or Slim100 project-planning tools on the marketMicrosoft Project, Primavera, Project Manager’s Workbench, or TimelineCombination of estimating and planning tools leads to accurate and realistic outcomes not easily overridden by clients or executive

36

Worst Practices

5 & 6 - Excessive, irrational schedule pressure and creep in users’ requirements7 & 8 - Failure to monitor progress and to perform risk management“90 percent completion”9 & 10 - Failure to use design reviews and code inspections.

37