Version 3.0 (Draft) Last Updated: 10 November 2011 Visit our CBA Website for more information - PowerPoint Presentation

Version 3.0 (Draft)Last Updated: 10 November 2011 Visit our CBA Website for more information regarding locations, signing up, upcoming training sessions, and more https://cpp.army.mil AMERICA’S ARMY: THE STRENGTH OF THE NATION Cost-Benefit Analysis (CBA)Four-Day Training BriefingStep 7: Compare AlternativesStep 8: Report Results and Recommendations 1

Step 7: Compare Alternatives 2 7a. Compare alternatives using the selection criteria to identify the preferred alternative. 7b. Identify the billpayer if there is a bill associated with the recommended alternative.7c. Identify the positive and negative impacts of the 2nd and 3 rd order effects. What must be done to manage the negative impacts?7d. Determine the robustness of the conclusions.If anything changes (i.e., assumptions, costs, benefits) would the recommendation change? 7e. Identify the high-risk aspects of the recommended alternative and define appropriate risk mitigation measures.

Compare alternatives by:Concepts Comparisons of costs with benefits Decision support tools and methods (bringing the CBA together) Sensitivity analysis and Risk assessmentStep 7: Compare Alternatives 3

The CBA process is essentially comparing the costs with the benefits between all alternatives The preferred alternative provides the greatest amount of valueRisk and second- and third-order effects should be included in the comparison of alternativesConcepts for Comparing Costs with Benefits 4 Value = Benefit – Costs

Aid for Completing - Step 7a CostsBenefitsSelection Criteria EqualUnequalAlternative that provides greatest benefits for given cost Equal Subjective reasoning and a fortiori analysisUnequalUnequal Alternatives ranked in order (based on benefit/cost ratio, net present value, or other relevant criterion)EqualLeast costly alternative 5 Most likely CBA Scenario CBA may not be appropriate

Overly Complex Final Recommendation6 ProductionRisk of shutdownLocal TaxesCompetition for Employees CostAlternative 120 widgets/yr5% 8%30 competitors$25 Alternative 215 widgets/yr8%6% 20 competitors$15Alternative 310 widgets/yr12%7%10 competitors$10

Comparative Methods 7 Method Description When UsedNet Present Value (NPV)Converts future cash flows into present equivalent values and then adds them togetherWhen alternatives have the same economic life Benefit/cost ratio (BCR)Compares present value (PV) of benefits with present value of costsWhen competing alternatives have unequal costs and unequal benefitsBreak-even PointIdentifies point at which cumulative cost of two alternatives equal the cumulative benefitsWhen projects are high-risk, to show when investment costs need to be recovered quicklySubjective reasoningApplies professional judgment as a complement to, or to the exclusion of, quantitative data.When professional judgment is considered to be more importantPoint SystemApplies objective values to subjective criteria.When decision makers wish to narrow the list of alternative solutions to the few that are most suitableDecision MatrixAllows for multiple criteria to be used to compare alternativesIt is a very flexible tool that can be used under many circumstances. It can even account for other decision support methods described in this table.A Fortiori AnalysisDetermines whether a strongly favored alternative is still the best choice even when assumptions are formulated that put that alternative at a disadvantage. When generally accepted intuitive judgment strongly favors one alternative

Practices: Compare the incremental costs with the incremental benefitsCompare the one or two most relevant benefits with the costsFind the “knee in the curve” or optimal value solutionBest Practices 8

“Over-averaging” for sake of simplification:Example:“Fort Hood data is representative of all bases.”Assuming away cost:Examples: “Year-end funds will be available,” when that’s not the case“Higher headquarters will pay for it.” “Other organizations will pay for it.”“Military personnel are free.”Assuming away the problem:Example: “Unused office space is available.”“Chief of Staff said we need this.” Adding a layer of oversight will increase process efficiencyCommon Mistakes 9These are examples of flawed assumptions that should have been rejected in step 2.

Cost ($M) Benefit Status Quo30GoodAdded capability (Alternative 1)45 Very GoodGreatly added capability (Alternative 2)55 ExcellentCommon Mistakes: Over SimplificationWhich alternative has the best value? 10

“Green ( fill in blank, i.e., funding, status…) for $55M is the preferred alternative” is the value statement.The Cost of a Green Chiclet Cost ($M)Benefit Alternative 1 55GoodAlternative 245Average Alternative 330Critical 11 This should help in making a decision.

What happens when using a per cost or per benefit metric?Effects of Over simplification 12MetricBenefit Score/Rank CostAlternative 120 widgets/yrGood 3$25Alternative 2 15 widgets/yrAverage2$15Alternative 3 10 widgets/yrBad1$10CBIBCIAlternative 18.30.12Alternative 27.50.13Alternative 3100.10 Best Case: COA 2 is best?

In some cases, new programs or requirements will be supported entirely by resources that are already funded. This could be described as “cost free” in the sense that there no additional costs in terms of the budget. Nonetheless, a “cost free” analysis must still consider cost in terms of opportunity cost: what the already-funded resource could have been used for if it had not been applied to the new requirement.“Cost Free” Analysis 13

Example: the Organization of the Joint Chiefs of Staff is planning a holiday party for itself. The Chairman of the JCS must assign three current employees to serve on the Holiday Party Planning Committee, which requires a commitment of 16 hours of work per member. CJCS is presented with two courses of action:COA1: Appoint the Chief of Naval Operations, the Chief of Staff, Army, and the Chief of Staff, Air Force, to the Holiday Party Planning Committee. COA2: Appoint Intern #1, Intern #2, and Intern #3—three GS-05 recent college graduates—to the Holiday Party Planning Committee.The additional budgetary cost of this new Holiday party requirement is equal for both COAs: it is “cost free” because the salaries of current employees will be paid regardless of whether the Holiday party gets planned. Determine which COA has the lower opportunity cost.“ Cost Free” Analysis14

All COAs proposed within a CBA must be solutions to the problem/opportunity statement. Assumptions should not solve the problem. Common Mistakes: Assuming Away the Problem15

Subjective Reasoning Uses informal criteria to rank alternativesA Fortiori AnalysisUsed when intuitive judgment strongly favors an alternativeAssumptions are the basis in choosing which alternatives are preferred Non-Quantitative Methods 16

Decision Matrix: A tool that compares benefits with costs to produce a single value scoreDecision Matrix 17 One of the best ways to elucidate the resource-informed decision to senior leadership is to include a Decision Matrix in the Decision Brief.

Pros Easy to useNormalizes costs and benefitsIs a familiar toolFlexibleCons Error proneHighly judgmentalLoss of information via normalizationResults not definitiveScoring is subjective Decision Matrix Merits 18

Example - Decision Matrix 19 COA-1 highest benefit COA-2 best value COA-3 lowest cost  Decision Matrix Rating or Ranking Benefit Criteria Weight COA-1 COA-2 COA-3 Lethality 30% 9 6 2 Safety 45% 4 6 6 Survivability 25% 6 5 3   Score : 6.0 5.8 4.1 Cost $M in BY-2011 $20 $ 16 $12 Cost = $ quantifiable cost – $ quantifiable benefit or saving Benefit = $ non-quantifiable benefit and $ non-quantifiable risk Cost per Benefit $3.33 $2.78 $2.96 COA-1 COA-2 COA-3 COA-1 COA-2 COA-3 Rating: 1 (worst) to 9 (best)

Decision Matrix20 Note: If calculating Cost-Benefit index, the benefits criteria for each COA should be rated, not ranked. Otherwise, the Cost-Benefit index calculated would be meaningless.

How sensitive is the recommendation to possible changes in costs, benefits, assumptions, etc.? If the recommendation is highly sensitive, can deeper analysis be done?Which elements of the CBA require sensitivity analysis?Only test elements with significant uncertainty or risk.Elements may include assumptions, constraints, costs, benefits, weighting of selection criteria, etc. Address sensitivity from one or both perspectives:What is the impact of a change of such and such magnitude? How large a change can occur before the recommendation changes?Have all probable risks and their respective impacts been identified?Can the risks decrease future benefits? Can the risks increase future costs?Questions for Steps 7c and 7d 21

The goal of risk assessment is to answer questions like: What risks may occur? What is the likelihood that risks will occur? Are the source of these risks internal or external? What causes these risks? What are the consequences if the risks go unresolved? What assets, operations, activities, functions, etc. will be affected as a result? How much risk is tolerable? What should be done to anticipate and limit risks? Risk Assessment and Mitigation 22 Always measure the risk by the potential adverse impacts on alternatives.

Business/Programmatic Risk:Affect the budget and viability of a program Operational Risk:Affect the ability to perform a missionProcess Risk:Associated with failing to meet standards and performance benchmarks in a newly established processTechnical Risk:Associated with failing to develop or implement technologySchedule Risk:Associated with allocating time to perform and manage tasksOrganizational Risk:Associated with management changesAll risks need to be reflected in Costs and/or Benefits of COAsRisks must factor into the decision-making process Types of Risks 23

Methods of measuring and/or addressing risk in a CBA include, but are not limited to:Effective MeanCost of Risk Mitigation Sensitivity AnalysisWays to Measure/Address Risk 24

The effective mean, or expected value, of a measurable quantity is the sum of all possible outcomes multiplied by their corresponding probabilities.Example: if the cost of a new ground combat vehicle is judged to be $1.4M with 50% probability, $2M with 25% probability, and $1.2M with 25% probability, then the effective mean (expected cost) is $1.4M x 0.5 + $2M x 0.25 + $1.2M x 0.25 = $1.5M. Example: if inter-theater transit time for a sustainment brigade is projected to be 5 days with 90% probability and 4 days with 10% probability, then the expected transit time is 5 x 0.9 + 4 x 0.1 = 4.9 days. Effective Mean (Expected Value)25

Cost of Risk Mitigation26 If a cost can be associated with reducing risk, then risk can be measured by that monetary value.Example: If for $22K extra, the risk of a schedule over-run can be reduced from 15% to 3%, then $22K can be a measure for the difference in risk.

Sensitivity Analysis:Done mainly on the most important and least certain selection criteria, assumptions, and cost estimatesAddresses the impacts of risks comprehensively Identifies the impact on a recommendation when elements of the analysis change, such as:AssumptionsCostsBenefitsConstraints ScopeWeighing of selection criteriaRisk probabilitiesSensitivity Analysis 27

Divide analysis into two groups of factors: Those outside an agency’s control (i.e., assumptions)Those agencies that have a degree of influence or controlChoose several elements (costs, assumptions, benefits, etc.) that have greatest impact and are most likely to changeVary each element over a reasonable set of values while holding the other variables constant relative to each other Determine the impact of these changes on:Total costTotal benefitRanking of alternatives Some factors that may warrant sensitivity analyses are:  The effects of a shorter or longer economic life.  The effects of variation in the estimated volume, mix, or pattern of workload; for example, the production rate or learning curve.  The effects of potential changes in requirements resulting from either congressional mandate or changes in functional responsibilities.  The effects of potential changes in requirements resulting from changes in organizational responsibility at the site, installation, base, or Army command/direct reporting unit/Army service component command level.  The effects of changes in configuration of hardware, software, data communications, prime support equipment, and other facilities.  The effects of alternative assumptions on areas such as project operations, inflation rate, residual value of equipment, and length of development.  The effects of changing the fielding strategy. 28 Suggested Steps to Conduct Sensitivity Analysis

Crystal Ball software may be used to perform sensitivity analyses.In the example to the right, “Cost of Revenues %” has the greatest effect on Net Present Value, while “Year 4 Units Sold” is the least sensitive. Example: Sensitivity Analysis29

Perform a sensitivity analysis on the weight of Lethality in this Decision matrix. Test for weights of16%44%58% 86%Determine if the recommended COA changes.Exercise: Sensitivity Analysis30COA-1 highest benefit COA-2 best value COA-3 lowest cost  Decision Matrix Rating or Ranking Benefit Criteria Weight COA-1 COA-2 COA-3 Lethality 30% 9 6 2 Safety 45% 4 6 6 Survivability 25% 6 5 3   Score : 6.0 5.8 4.1 Cost $M in BY-2011 $20 $ 16 $12 Cost = $ quantifiable cost – $ quantifiable benefit or saving Benefit = $ non-quantifiable benefit and $ non-quantifiable risk Cost per Benefit $3.33 $2.78 $2.96 COA-1 COA-2 COA-3 COA-1 COA-2 COA-3

Defense funding is a zero-sum game: every dollar awarded to one program comes at the expense of another competing program that will not be funded. The CBA team rarely will have the authority to identify a bill-payer. This is a task for the organization’s comptroller/resource manager (the G-3 for most organizations)Don’t offer a “gold watch” as a bill-payer … a program that is so important to the Army that it’s certain higher headquarters will reject it.Alternatives will compete with other developments for funding if no billpayers are identified Paying for Alternatives 31 Note : This does not guarantee funding. Identify billpayers to fund alternatives!

Comparing the costs with benefits of each alternative is a required part of the CBA methodology. Sensitivity analysis is a technique that analyzes whether changes in assumptions, quantitative values, or priorities will affect the recommendation. Billpayers are the sources that will fund the costs of alternatives. Quick Review 32

In a rented garage at 58 Bagley Street in Detroit, Henry Ford completed his first gas-powered car on the morning of June 4, 1896. He had spent $250 FY1896 dollars on equipment to build the car, and $100 on labor. The garage cost $15 per month to rent. After the car was assembled, it was clear that it would not be able to fit through the door of the rented garage. The car was to be used as a model for the two-year long test-driving phase of his R&D process, with the final goal of arriving at a model for mass production and sale. 33Review Exercise: Henry Ford

Identify selection criteria and compare each of the following Courses of Action:COA1 (Status Quo): Leave the car in the garage, do nothing else.COA2: Disassemble the car and reassemble it outside the garage.COA3: Obtain permission of garage landlord to widen the garage door. 34Review Exercise: Henry Ford

CBA Decision Package: Preferred format is a narrative or PowerPointAccompany with a decision briefingWrite it in such a way that a layman can understand the topicResults and Recommendations:Summarizes the analysisMakes conclusive statements about each of the alternativesThe recommended alternative should followResults address how the alternatives were ranked using the criteria developed in Step 6 Document all data and information used in Steps 1-8Provide supporting information so reviewers can understand how Steps 1-8 were developedStep 8: Report Results and Recommendations Questions for the Reviewer:- Does the package contain all key elements that are accompanied with supporting documentation? - Does the recommended alternative address the problem, and is it consistent with the assumptions and constraints?- Does the analysis explain how the recommended alternative is the best one to satisfy the selection criteria? 35

The CBA’s quality and comprehensiveness may be enhanced by the following supplementary content: GlossaryDefines unfamiliar abbreviations, acronyms, and terms used in the CBATimelineDisplays key dates and actions associated with the CBA in terms of its development and/or implementation Coordination SheetIdentifies who will need/have to review the CBA 36 Key Elements for Documentation

Briefing Format: This is a suggested but not required format.This is just a suggestion. The actual briefing for a CBA (or any briefing, for that matter) should be determined based on the briefer’s style, the known preferences of the individual being briefed, and the subject matter.An example has been provided in the CBA guideIt is not a substitute for a well documented CBA in narrative form.The CBA coordinator, the decision makers, and stakeholders should collaborate to determine the content and format of the CBA decision brief as they will have differing preferences. Briefing the Results of the CBA 37

End Requirements Approval Resource Approval Requirement approval forums use CBA to: Make resource-informed decisions Validate and approve requirements Resource approval forums use CBAs to make resource- informed decisions Identify tradeoffs and billpayers Decide funding approval decision Proponent organizations: Register CBA online Develop and submit CBAs online with value proposition linked to command’s strategic priorities Proponent leadership endorses submission online Organization validates the CBA methodology Accuracy and completeness CBA Workflow Automation Tool 38 Draft Submitted Lead Analyst Review CBARB Review Final Review Completed CBA Registration, Development and Submission CBA Validation Start (Capability Gap) https://cpp.army.mil/ Register and submit your CBA online

https://cpp.army.mil/39

CBA Workflow Tool40

Document the CBA narrative and share the results with stakeholders for approval. CBA templates are optional tools that can be used to brief the results of the CBA.Include all documentation and calculations in back up.Ensure that the CBA clearly describes the value proposition, and highlight how the benefits outweigh the cost, risks, and trade offs. Quick Review 41

42 FOUR DAYS FOUR

U.S. Army Special Operations Command maintains a specialized communications link between the Commander, USASOC, the Chief of Staff and Vice Chief of Staff, Army, and Team Leaders in the field during highly sensitive counter-terrorism missions. The link provides video and voice transmissions that are typically delayed by 30 seconds. Currently, $2M per month is spent on operating and maintaining the communications link: mostly consisting of payments to the National Security Agency for “satellite bandwidth.” The NSA has recently launched into orbit a new satellite that promises shorter transmission delays: typically 10 seconds, but with a 20% risk of 20 second delays. The satellite takes advantage of recently developed technology: costs to the Army for the first month are set at $10M, and are projected to decrease based on an 85% learning curve ($10M for the first month, 0.85 x $10M per month for the next two months, 0.85 x 0.85 x $10M per month for the next four months, 0.85^3 x $10M per month for the next eight months, 0.85^4 x $10M per month for the next sixteen months, etc.), though there is a 40% risk that costs will only decrease according to a 90% learning curve. Because time delay is the overwhelming factor in communications for counter-terrorism operations, Army planners have assigned a 100% benefit weight to signal delay time. The benefit score will be determined by the function , where t is the expected time delay in seconds.Mini-case Exercise #743 or “can you hear me now?” or “what’s the expected value?”

Currently, a contractor provides 12 translators to a SIGINT battalion, each contracted for 1885 hours annually, on a government contract costing $1.2M per year. The battalion commander would like to evaluate the option of insourcing the labor to Army civilians. The total burdened cost per civilian is $85,209 and the estimated annual productivity is estimated at 1740 hours each. Determine whether insourcing would be more cost effective than the status quo.Mini-case Exercise #844 or “to insource or not to insource, that is the question”

OA22 has invited a contracting team to the Pentagon to inspect its centralized HVAC system. Following the inspection, the contractor reported minor damage to the HVAC system, and offers to replace it. Despite the damage, the HVAC system continues to function at the same capacity, although with risk of damage to the building. You assess that in any one year, there is a 15% chance of damage being done. In the event that there is damage, you assess that the average amount of damage will be $200K if occurring within the first year, $300K if occurring during the second year, $400K if occurring during the third year, etc. (increasing by $100K each year). If you decide to replace the HVAC system, the contractor has offered a payment program of $256K for the first year of ownership, $128K for the second year, $64K for the third year, etc. (reduce by 50% each year). The expected life cycle of this HVAC system is 6 years. Alternatively, you may buy the same HVAC system from a separate contractor at a cost of $100K for the first year, and $90K for every subsequent year. Under this special program, the contractor will replace the HVAC system at the end of its life cycle without any additional material cost (operations & maintenance, replacement costs not included). All costs are given in constant FY12 dollars. Perform all steps of the CBA process to support a course of action. Mini-case Exercise #945 or “I’ll gladly pay you Tuesday for air conditioning today”

The Captain of the Pequod has ordered a cost-benefit analysis to determine the optimal number of rowboats to deploy to pursue a whale sighting. As defined by the Commander, the selection criteria are manpower, stealth, and ease of sustainment, with weights of 0.5, 0.3, and 0.2, respectively. You have been told as a rule of thumb that the total mission cost can be estimated at $2200 for each rowboat involved in the hunt. Each rowboat can transport 15 fishermen. Mini-case Exercise #1046