TEACHING MULTI-OBJECTIVE MULTI-STAKEHOLDER

slide1. TEACHING MULTI-OBJECTIVE MULTI-STAKEHOLDER DECISION MODELING WITH CASES L. Robin Keller*, Jay Simon*** University of California, Irvine, USA President, INFORMS (INFORMS.org)** Defense Resources Management Institute, USA 11TH International Workshop on Operations Research
OR & Human Welfare: Health, Environment, and Education
Havana, Cuba, March 10-13, 2015 1<br>
slide2. 2 Decision Analyst
Ralph Keeney
advises us to practice
Value-Focused Thinking

Thinking about what we value as expressed in our objectives

Keeney, R. L. 1992. Value-Focused Thinking—A Path to Creative Decision Making. Harvard University Press, Cambridge, MA.

Hammond, J. S., R. L. Keeney, H. Raiffa. 1999. Smart Choices: A Practical Guide to Making Better Decisions. Harvard Business School Press.<br>
slide3. 3 Keeney’s Personal Objectives Maximize my quality of life

Enjoy life
Be intellectually fulfilled
Enhance the lives of family and friends
Contribute to society

Keeney (1992), Value Focused Thinking<br>
slide4. MULTIPLE OBJECTIVE DECISIONS UNDER CERTAINTY Model
Objectives hierarchies of stakeholder(s)Additive “weight & rate” multiple objective measurable value function
Software
Use Excel with sliders to input swing weights
Show sensitivity analysis in real time as bar graphs change

Experiential Learning
Applicable to business, personal, social/charity projects
Apply in class, on homework, and in term project
Students able & willing to “take away” & use in future

Much of this material is at http://faculty.sites.uci.edu/lrkeller/classes/. LR Keller, JSimon, Y Wang. "Multiple objective decision analysis involving multiple stakeholders," Ch. 7 in M. R. Oskoorouchi (ed.) Tutorials in Operations Research- Decision Technologies and Applications. INFORMS. (2009). [faculty.sites.uci.edu/lrkeller/files/2011/06/multiple-objective-decision-analysis-involving-multiple-stakeholders.pdf] 4<br>
slide5. 5 MERGER DECISION ANALYSIS OF POTENTIAL MERGER OF OPERATIONS RESEARCH SOCIETY OF AMERICA  (ORSA)  AND
THE INSTITUTE OF MANAGEMENT SCIENCES
(TIMS)

L. Robin Keller and Craig W. Kirkwood, “The Founding of INFORMS: A Decision Analysis Perspective”, Operations Research. 47(1), Jan.-Feb. 1999, 16-28. [faculty.sites.uci.edu/lrkeller/files/2011/06/The-Founding-of-Informs-Decision-Analysis.pdf]
Powerpoint: http://faculty.sites.uci.edu/lrkeller/classes/<br>
slide6. 6 ORSA/TIMS COOPERATION ALTERNATIVES SEP: SEPARATION OF ORSA & TIMS
SQ: STATUS QUO PARTNERSHIP
SM: SEAMLESS MERGER
M2: MERGE WITH ORSA/TIMS AS SUB-UNITS
M3: MERGE WITH NO ORSA/TIMS SUB-UNITS;  SUB-UNITS ARE REPRESENTED ON BOARD<br>
slide7. 7 ORSA/TIMS MERGER OBJECTIVES FIVE MAIN CATEGORIESIMPROVE COST EFFICIENCYENHANCE QUALITY OF PRODUCTSESTABLISH STRONG EXTERNAL IMAGEMAINTAIN SCOPE/DIVERSITY OF FIELDIMPROVE OPERATIONS Elicited stakeholders’ objectives & combined them into 1 hierarchy<br>
slide8. 8 ADD BRANCHES TO MAIN CATEGORIES IMPROVE COST EFFICIENCY

MAINTAIN ALLOCATE WELL MAINTAIN
EFFICIENT REVENUES AND EFFICIENTUSE OF FUNDS EXPENSES USE OF
TIME

EXPLOIT BALANCE DUES REMOVE
ECONOMIES RATE & FEE- DOUBLED
OF SCALE FOR-SERVICE DUES<br>
slide9. 9 VALUE RATING SCALE 2: SEEN BY AVERAGE MEMBER AS IMPROVED
1: SEEN BY OFFICERS AS IMPROVED BUT NOT BY AVERAGE MEMBER
0: NO CHANGE
-1: SEEN BY OFFICERS AS WORSE
-2: SEEN BY AVERAGE MEMBER AS WORSE<br>
slide10. 10 INTERPRETATION OF “MEASURABLE” VALUE RATINGS STRENGTH OF PREFERENCES IS REFLECTED IN DIFFERENCES OF VALUES
DEGREE OF IMPROVEMENTFROM 0 TO 1 IS THE SAME AS  FROM 1 TO 2<br>
slide11. 11 JUDGED VALUE RATING SCORES<br>
slide12. 12 WEIGHTS FOR OBJECTIVES SUM OF WEIGHTS IS 1OO% FOR ALL LOWEST LEVEL OBJECTIVES
OBJECTIVE’S WEIGHT DEPENDS ON RANGE ATTAINABLE ON OBJECTIVE
Use a SWING WEIGHT Interpretation
Assume a weighted Additive Model
DECISION MAKER JUDGES WEIGHTS ON OBJECTIVES<br>
slide13. 13<br>
slide14. 14 COMPUTE WEIGHTED AVERAGE OF VALUE RATINGS MULTIPLY OBJECTIVE’S WEIGHT TIMES VALUE RATING ON EACH OBJECTIVE
SUM UP OVER ALL OBJECTIVES
(Use SUMPRODUCT function in Excel)

RECOMMENDED OPTION IS ONE WITH
HIGHEST OVERALL VALUE<br>
slide15. 15<br>
slide16. 16 RESULTS OFFICERS PREFERRED MERGER3 ALTERNATIVE

VOCAL OPPONENTS COMPROMISED ON SEAMLESS MERGER, AS LONG AS NEW NAME included “OPERATIONS RESEARCH”<br>
slide17. 17 OUTCOME MEMBERS VOTED TO MERGE IN
SEAMLESS MERGER
on JAN. 1ST, 1995
into
INSTITUTE FOR OPERATIONS RESEARCH AND
THE MANAGEMENT SCIENCES<br>
slide18. 18 Perspectives of Multiple Stakeholders can help… -identify mutually agreeable alternatives
-foresee opposition to decisions

-design new & better alternatives

-understand the evolution of past decisions from  multiple perspectives<br>
slide19. 19 Multiple-Stakeholder Decision Making The StarKist Tuna Fishing Decision Stakeholders





San Diego, CA USA
Tuna Fishing Fleet
http://www.sandiegohistory.org/journal/81fall/images/piva.jpg http://www.earthisland.org/index.php/donate/



Monika I. Winn and L. Robin Keller, “A Modeling Methodology for Multi-Objective Multi-Stakeholder Decisions: Implications for Research”, Journal of Management Inquiry. 10(2), June 2001, 166-181. [faculty.sites.uci.edu/lrkeller/files/2011/06/A-Modeling-Methodology-for.-Multiobjective-Multistakeholder-Decisions.-Implications-for-Research.pdf]
Much of this material is at http://faculty.sites.uci.edu/lrkeller/classes/<br>
slide20. Problem: Purse seine nets from boats can catch dolphins along with tuna fishimage source http://www.crownprince.com/nets-tuna.htm 20<br>
slide21. 21 DECISION ALTERNATIVES

Legal Quota
Maintain current practices and stay within legal limits

Limited Mortality
Step up efforts to reduce the number of dolphins killed

Zero-Mortality
No fishing associated with setting nets on dolphins<br>
slide22. 22 Decision Alternatives Rated for Fishing Fleet + favorable 0 neutral/balanced ? insufficient info. - unfavorable<br>
slide23. 23 Decision Alternatives Rated for
Environmental Interest Groups<br>
slide24. 24 StarKist’s “Crisis Mode” Objectives Hierarchy StarKist’s (1991) Dolphin Safe Policy
"StarKist will not buy any tuna caught in association with
dolphins in the Eastern Tropical Pacific."<br>
slide25. 25 Home Depot Case Feng, T., L. R. Keller, X. Zheng. 2008. Modeling Multi-Objective Multi-Stakeholder
Decisions: A Case-Exercise Approach. INFORMS Transactions on Education 8(3) 103-114,
(http://ite.pubs.informs.org/, http://pubsonline.informs.org/doi/abs/10.1287/ited.1080.0012
supplemental files: HomeDepotTeachingNote.pdf (for instructors), Excel file. Files also at
http://faculty.sites.uci.edu/lrkeller/classes/<br>
slide26. 26 Background Home Depot proposed to open a
retail building supply store
in San Juan Capistrano, California USA

The new store would be on 15 acres in a strip of industrial land.

Home Depot owned two acres of this land.
The rest of the land was owned by the city, and would need to be bought.<br>
slide27. 27 Background The city would get $9 Million if it sells Home Depot the 13 acres.

Many were concerned that a “big box store” would destroy its historical small town feeling.

Nearby residents also worry that a Home Depot would cause traffic jams, pollute the air, produce noise and block ocean breezes.<br>
slide28. Home Depot Case Alternatives

Build Home Depot
Don’t develop the land
Build a recreational vehicle park
Build specialty retail facilities Stakeholders (assign 6 student groups)
City of San Juan Capistrano
Competing Local Small Businesses
Complementary Local Small Businesses
Home Depot
Nearby Residents
Other Area Residents 28<br>
slide29. Case Instructions Ask the groups to:
Brainstorm the objectives of the stakeholder. Create a hierarchy of objectives by grouping related objectives.
Put the objectives in the spreadsheet.
Rate the options’ performance on each objective on a scale from 0 to 10.
Make their own judgment of the “raw swing weights” to put on the lowest level objectives.
Answer questions and determine the best option based on the analysis.
Post completed spreadsheet file to share. 29<br>
slide30. Spreadsheet Structure for Each Stakeholder 30<br>
slide31. Identify group’s objectives Promote convenience of shopping Promote convenience of shopping 31<br>
slide32. 32 Complementary Local Small Businesses-
Representative Hierarchy of Objectives<br>
slide33. 33 Home Depot in San Juan Capistrano? A Sample Spreadsheet
to Evaluate the Home Depot Case

Excel file (HomeDepotCase.xls)
Make sure to choose "enable the macros" when you open the spreadsheet. If you still have the problem of adjusting the sliders due to the security level after that, please go to the menu of "tools->macro->security", switch the security level from high to medium, save the file, then close the file and finally reopen the file and it should work.<br>
slide34. 34 Moving Sliders on Weights Dynamically Changes Graph<br>
slide35. 35 Moving Sliders on Weights Dynamically Changes Graph<br>
slide36. Case Discussion While comparing the results from different stakeholders, the instructor inputs the calculated overall values for each option from each group into a summary file to create bar charts showing results.

Sample results from all the six stakeholders

Take a class vote among the options to predict the actual vote.

“This approach can help decision makers understand the perspectives of different stakeholders, and provide a way to design more acceptable alternatives.” 36<br>
slide37. 37 What do you think: Yes or No? (City voters voted on this issue in November 2002.)<br>
slide38. 38 Example Home Depot Case Perspectives Data from Executive Education session, February 2009. UC Irvine Merage<br>
slide39. 39 Each Alternative from Different Stakeholders’ Viewpoints<br>
slide40. 40 Each Stakeholder’s View of Different Alternatives<br>
slide41. Appendix 1. Hospital capital budgetsDon Kleinmuntz, former INFORMS President (now at Univ. of Notre Dame)(http://mendoza.nd.edu/research-and-faculty/directory/don-kleinmuntz/) started Strata Decision Technology (http://www.stratadecision.com/ ) to create Excel-based (or bigger database) software to aid hospital administrators in capital budgeting (choosing a set of expensive projects to fund), w/ an additive multiple attribute measurable value function + linear programming (Excel Solver or LINDO for knapsack problem) Their original capital budgeting software was StrataCap , new product is (cloud-based) Strata Jazz  http://www.stratadecision.com/our-solutions/capital-and-equipmentVideo, in 1 out of 5 US hospitals:  http://www.stratadecision.com/our-company/our-history 41<br>
slide42. Weights on objectives
(from Kleinmuntz)

Identify most important objective(s)
Score of 100
Rate others objectives relative to 100
90, 80, 50, …
Divide by total to get weights that add to
100% 100 100

60

75 100

80

65

60 42<br>
slide43. Hospital Capital Budgeting
Objectives Hierarchy (from Kleinmuntz) 43 Max. Project Benefits
Financial
Quality
Improve patient outcomes
Enhance patient/family satisfaction
Enhance physician satisfaction
Enhance facility quality
Strategy
Increase market share
Enhance information integration
Promote operating efficiency Weights on objectives
(from Kleinmuntz)

Identify most important objective(s)
Score of 100
Rate others objectives relative to 100
90, 80, 50, …
Divide by total to get weights that add to
100% 100

100
80
65
60

100
60
65<br>
slide44. Appendix 2. Planning for potassium iodide (KI) distribution for thyroid risk from radioactive iodine exposureEvaluate plans for distribution of potassium iodide (KI) to protect against thyroid cancer, when there will be radioactive iodine exposure as a result of an incident at a U.S. nuclear power plant. The types of KI distribution plans include the following:• Predistribute to households, schools,hospitals, etc. —Via mail —Via voluntary pickup• Stockpile at evacuation reception centers• Do not predistributeT. Feng, L. R. Keller, “A Multiple-Objective Decision Analysis for Terrorism Protection: Potassium Iodide Distribution in Nuclear Incidents”, Decision Analysis, (June 2006), 3 (2): 76-93.http://pubsonline.informs.org/doi/abs/10.1287/deca.1060.0072(supplement has Excel file) Much of this material is at http://faculty.sites.uci.edu/lrkeller/classes/Based on book: http://www.nap.edu/catalog/10868/distribution-and-administration-of-potassium-iodide-in-the-event-of-a-nuclear-incident 44<br>
slide45. KI study Objectives Minimize Radioactive Iodine Risk To Thyroid Maximize KI Availability Optimize Ability To Take KI On Time Minimize Harm From Inappropriate KI AdministrationMinimize Harm From Other Aspects Of Incident KI Procedures Don’t Impede Evacuation Avert Mortality/Morbidity From Radiation Or AccidentsMinimize Panic/Anxiety Due To KI ProceduresKI Procedures’ Resource Use Not ExcessiveSimple KI Procedures Before/During IncidentEducate Public To Respond To Incidents 45<br>
slide46. Appendix 3. BiologicalBiological clock multi-objective utility model with weights on objectives changing over time(maximizing family life quality, social life quality, career life quality) CAREER WOMEN MIGHT WANT TO HAVE CHILDREN ASAPLiveScience, Nov. 9, 2007 -- A new mathematical model developed by professor Ralph Keeney and doctoral student Dinah Vernik of Duke's Fuqua School ofBusiness could help women decide the optimal time in their lives to have kids…http://www.livescience.com/health/071109-women-children.html Video of authors talking about paper:https://www.youtube.com/watch?v=mZxXf1W6FxM“Analysis of the Biological Clock Decision”, RL Keeney, D Vernik, http://dx.doi.org/10.1287/deca.1070.0094 2007 , 4(3), 114 - 135 (supplement has the Excel file and a user guide) 46<br>
slide47. Appendix 4. Multi-objective Prostate Cancer Treatment ChoiceJay Simon worked for a firm that had a prostate cancer decision analysis website to help potential patients make their treatment decision.Side effects reduce quality of life score multiplicativelySurvival from prostate cancer without impotence or incontinence = 90Survival from prostate cancer with incontinence = 90(80%) =72Survival from prostate cancer with impotence = 90(60%) = 54Survival from prostate cancer with impotence and incontinence = 90(80%)(60%) = 43.2(new site, with more focus on info. :  http://www.prostatesmart.info/)“Decision Making with Prostate Cancer: A Multiple-Objective Model with Uncertainty”, Jay Simon, 2009 39(3), 218 - 227, http://pubsonline.informs.org/doi/abs/10.1287/inte.1080.0406 47<br>
slide48. 48 Appendix 5. Andy Grove’s Prostate Cancer In the fall of 1994, Andy Grove- the former CEO of Intel- was faced with a difficult problem to solve. Initially, he was presented with an abnormal screening PSA (Prostate Specific Antigen) test that could represent the presence of cancer. His first reaction was to ask what to do with that information. At this point, he may or may not have had cancer.

So, to better define if there was a required decision, he chose to gather further information. Some basic facts he obtained gave him a first understanding of the probabilities and outcomes he might face, finding that 200,000 men were diagnosed with prostate cancer in 1994 and that 38,000 would be expected to die, making prostate cancer the second leading cause of cancer deaths in men.

Since his PSA result was just over the upper limit of normal, he elected to repeat the test in early 1995 in case his results were within the error margin of the test. The results suggested more strongly the presence of a tumor and he visualized a sugar-cube-sized tumor in his prostate. These tests results convinced him of the need to see the urologist for a biopsy to define if the test result was a true positive or a false positive.<br>
slide49. 49 Andy Grove’s Prostate Cancer The biopsy results indicated his PSA result was a true positive.
He did have prostate cancer.
This led to the formulation of his decision problem.
What type of treatment should Mr. Grove pursue for treatment of his prostate cancer? There appeared to be four main decision alternatives. One option was to have the tumor and prostate gland surgically removed. This alternative can increase the survival rate and decrease the recurrence rate as well, however it will lead to a greater chance of being impotent after the surgery. Another alternative was to receive radiation treatments in the form of “seed” implantation to destroy the cancer cells. This option can increase the survival rate, but it also has serious side effects. A third alternative was cryosurgery, or freezing the tumor cells. Regarding this option, there was not enough information available to make an informed decision. The last option was to do nothing, taking the “wait and see” approach, which also carried much risk of losing his life if the cancer grew very quickly.
Apparently, none of these alternatives was perfect. Furthermore, several other stakeholders were also very concerned with Andy Grove’s situation. Andy’s dilemma will be modeled as a multi-stakeholder decision problem.

Those stakeholders are: Andy Grove, Urologist, Oncologist,
Andy’s Family/Wife and Andy’s Company – Intel

This case was written by L. Robin Keller and Tianjun Feng, of the UCI MSB, building upon the article by Andy Grove published in Fortune (1996), several case study reports by UCI HCEMBA students: L. Jeff Koh, Kenneth Rich, Suehei Lee, H. Ena Leo and others. See TAKING ON PROSTATE CANCER by Andy Grove with reporter associate Bethany McLean, FORTUNE, May 13, 1996.<br>
slide50. 50 Andy Grove’s Prostate Cancer Divide into 5 groups.
For your stakeholder group, rate each alternative on each objective.
Determine weights on objectives.
Compute the overall weighted score for each alternative.
Andy Grove Case Excel file at http://faculty.sites.uci.edu/lrkeller/classes/<br>
slide51. Added background info. on this talk

Talk Abstract Many know about the use of decision analysis to decide among alternative investments (such as pharmacological research and development) using decision trees with chance nodes to compute expected monetary value of different alternatives. Such an analysis aims to maximize a single evaluation measure for a single decision maker.

We demonstrate less widely known decision analysis techniques using spreadsheet models of the multiple objective perspectives of the decision stakeholders.

We show how to teach students to analyze real-life decision problems using case examples and discuss specific skills students are expected to learn, such as dynamic sensitivity analysis using sliders in Excel on objectives’ weights, and typical student questions and errors during case discussion. 51<br>
slide52. Talk Abstract, continued

Taught in business courses for both MBAs (including health care executive MBAs) and undergraduates.

Sometimes, one objectives hierarchy is suitable for a set of stakeholders, and differences in opinions across stakeholders can be characterized by differences in the multiple objectives’ weights:

-Merger of the Operations Research Society of America &
The Institute of Management Sciences (INFORMS)

-Protection against radioactive iodine in nuclear incidents 52<br>
slide53. Talk Abstract, continued

In other cases, an objectives hierarchy will be constructed for each stakeholder because their objectives are so different that construction of separate hierarchies better represents their divergent perspectives.

-Tuna fish supplier source selection decision
(StarKist, environmentalists, San Diego tuna fishing fleet)

-Siting of a new Home Depot building supply store

-Prostate cancer treatment decision  (former Intel CEO Andy Grove, his family, company, doctors) 53<br>
slide54. 54 Keeney’s Professional Objectives Maximize the contribution of professional activities to…
my quality of life
Max. enjoyment
Max. learning
Provide service
Enhance professional career
Max. economic gain
Build good professional relationships
Min. the time required
Min. time required where I live
Min. time required away from home

Keeney (1992), Value Focused Thinking<br>
slide55. 55 Objectives for Keeney’s son’s name 1. Single spelling
2. Not a unisex name
3. Reasonable initials
4. Understandable pronunciation
4.2. With last name
4.3. With middle and last name
5. No obvious “unwanted” nickname
6. Not unique
7. Not extremely common<br>
slide56. 56 8. Not religious
9. Not named after anyone
10. Has a nice rhythm
10.1. With last name
10.2 With middle and last names
11. Nice-sounding in foreign languages
12. Appealing (i.e., you feel predisposed to talk to or
meet the person)
13. No “ee” sounds Objectives for Keeney’s son’s name<br>
slide57. 57 8. Not religious
9. Not named after anyone
10. Has a nice rhythm
10.1 With last name
10.2 With middle and last names
11. Nice-sounding in foreign languages
12. Appealing (i.e., you feel predisposed to talk to or
meet the person)
13. No “ee” sounds

Keeney (1992), Value Focused Thinking Objectives for Keeney’s son’s name The Winning Name is
Gregory<br>
slide58. Methodology A Multi-objective Multi-stakeholder Decision Analysis Methodology Identify Stakeholders Identify Alternatives Develop the
Objectives
Hierarchy Develop
the
Weights Rate Alternatives over Objectives Is There a Dominant Alternative? Compute Overall Values of Alternatives Make
the
Recommendations Yes No Conduct Dynamic Sensitivity Analysis Using Sliders in Excel 58<br>
slide59. Teaching Notes: Skills Students Can Learn Learn to assign value ratings to how well each option satisfies each objective

Learn to creatively generate objectives and structure them into a hierarchy of objectives

Learn to use the swing weight approach to generate importance weights on objectives 59<br>
slide60. Teaching Notes: Skills Students Can Learn Learn to do sensitivity analysis in decisions under certainty, using “sliders” created in the Excel software.

Learn to compare the overall values of options, using the sumproduct function in Excel.

Learn to compare and contrast results from different stakeholder groups. 60<br>
slide61. Teaching Notes: Typical Student Questions & Errors Students might not understand the difference between ratings and weights.

The same weights assigned to different subobjectives are allowed.

Students might generate wrong or redundant subobjectives for one specific objective.

Students questioned whether they should start with the lowest or highest level subobjectives when computing swing weights. 61<br>
slide62. Case Objectives and Pedagogical Benefits Enrich the content of the typical undergraduate/masters level decision analysis or management science course
Focus on multi-objective multi-stakeholder decisions
Link creative problem structuring with analytical tool

Introduce the methodology to the students
In-class exercises and/or homework
Decision making tool to tackle real-life context-rich decision problems
Applied to corporate strategic decision making for a facility location problem 62<br>
slide63. Stakeholders The city of San Juan Capistrano: interested in the potential revenue, but concerned with interests of multiple stakeholders
Competing local small businesses: will be influenced by the arrival of Home Depot in terms of profit, etc.
Complementary local small businesses: will definitely be affected in terms of profit, etc.
Home Depot
Nearby residents: concerned with the possible adverse impacts on their quality of life
Other area residents: will enjoy the convenience, but may suffer from the possible increased traffic flow 63<br>
slide64. 64<br>
slide65. 65 Decision Alternatives Rated with
StarKist’s “Business-As-Usual” Objectives Hierarchy<br>
slide66. 66 StarKist’s “Strategic Planning” Objectives Hierarchy<br>