Choice Design and Max Difference Design/ Analysis in Optimizing High School Laptop Purchase - PowerPoint Presentation

1. Choice Design and Max Difference Design/ Analysis in Optimizing High School Laptop Purchase (STEAMS Project)(2022-EU-45MP-1014)Mason Chen, Charles Chen, and Patrick Giuliano01/30/2022STEAMS Organization, Stanford OHS01/31/202212022 JMP Europe Discovery Summit

2. STEM Project CharterOpportunity Statement: each year, Most SOHS G9 students need to take the Core MSB Course and do one MSB project and AP Statistics Students need do one AP Stat Project (100-150 Projects/year). >90% projects used Survey to collect data. Most MSB projects are using Survey to collect data. This project introduce a modern JMP Choice Design to optimize the Survey Methodology (avoid Sampling Bias), and use Choice Model to analyze Survey Data Project Scope: use JMP 16 Choice Design and Choice Model Platforms to study the Consumer Research and help MSB STEAMS Projects and assist with Laptop Selection for optimal student usage. Project Objective: demonstrate a JMP case study on how computer manufacturer could design a new laptop based on customer preferences before beginning an expensive development process. Derive a simple model for consumer to assess the Extended Warranty Protection offers01/31/20222022 JMP Europe Discovery Summit2

3. STEM DiagramScienceConsumer/ Marketing Research(10 Respondents)TechnologyLaptop Electronics (4 Attributes)EngineeringChoice Design of SurveyMathematicsChoice Model/Warranty(Purchasing Probability)• size of hard drive disk (40 GB or 80 GB)• speed of processor (1.5 GHz or 2.0 GHz)• battery life (4 Hrs or 6 Hrs)• cost of computer ($1000, $1200 or $1500)01/31/20222022 JMP Europe Discovery Summit3

4. Science/Statistics: Consumer/Marketing ResearchConsumer Research (also called Marketing Research) provides a full suite of tools for analyzing consumer behavioral research.Collect information how customers use products or servicesHow satisfied they are with your offeringsWhat new features they might desire. The resulting insights create better products and services, happier customers, and more revenue for your organization. Tools for analyzing these consumer research activities are located in the JMP Consumer Research menu. 01/31/20222022 JMP Europe Discovery Summit4

5. Science/Engineering: Consumer/Marketing ResearchCan we develop a consumer/marketing research methodology to help SOHS MSB projects by conducting JMP Choice Design of Survey?01/31/20222022 JMP Europe Discovery Summit5

6. Technology/Statistics: Laptop ElectronicsVOC: the manufacturer decides to construct a design consisting of two sets of profiles that will be administered to ten respondents. The goal of the JMP choice design is to understand how potential laptop purchasers view the advantages of a collection of four attributes: • Size of hard drive disk (40 GB or 80 GB)• Speed of processor (1.5 GHz or 2.0 GHz)• Battery life (4 Hrs or 6 Hrs)• Cost of computer ($1000, $1200 or $1500)01/31/20222022 JMP Europe Discovery Summit6

7. 7Engineering/Technology: Choice Design of SurveyUse JMP16 Choice Design Platform to determine the 8-Choice Set Survey for 10 respondents01/31/20222022 JMP Europe Discovery SummitLook for the sample data file and include a reference here so they can reproduce the analysis if they want.

8. 8Statistics/Engineering: JMP Choice Model PlatformUse JMP16 Choice Model Platform to run Statistical Analysis on the collected SOHS MSB Survey data01/31/20222022 JMP Europe Discovery Summit4 AttributesChoice DesignGrade, SexMissing ValueStacked 10 RespondentsNo personality selection. Limited to choice model only. How come?

9. 9Statistics/Science: Effect Summary and Parameter Est.The Effect Summary report shows the effects in order of significance: Speed and Price are top two factorsThe Parameter estimates report shows the Estimated Main Effect (Logistic Regression Model)The negative estimate values mean the probability of purchasing chance is lower at that particular attribute level: buyers prefer larger HDD Size, faster Speed, Longer Battery Life, and Cheaper Laptop01/31/20222022 JMP Europe Discovery SummitTalk about how this doesn’t handle interaction term. Why?? Maybe it’s not practical. But what’s happening with the model itself (not a lack of DOF problem).

10. 10Statistics/Engineering: Effect Marginals AnalysisThe Effect Marginals report shows the marginal probability of each four attribute at their different levelsFor example, 71% buyers may choose 80GB over 40GB HDD Size, 68% Buyers may choose $1,000 Price over $1,200 and $1,500 Price01/31/20222022 JMP Europe Discovery Summit

11. 11Statistics/Science: Utility Profiler AnalysisThe Utility Profiler report shows the buyers’ satisfaction at the particular scenarioHigher Utility means happier buyer: above 1 is very happy, below 0 is unhappyFor example: buyers will be very happy if they can spend $1,000 to get a laptop with 80GB HDD, 2.0 GHz, 6 Hrs Battery Life based on the Choice Model01/31/20222022 JMP Europe Discovery SummitMaximize Desirability = Maximize Utility in the context of this model. How happy are they to purchase? BONUS: How is utility related to Probability? BONUS: What is the historical context for “Utility?”Potential scenario is for optimization (highest utility)

12. 12Statistics/Engineering: Probability Profiler AnalysisThe Probability Profiler report shows the purchasing probability at the particular scenarioProbability range is from 0 to 1For example: 12% buyers will consider to spend $1,500 to get a laptop with 40GB HDD, 1.5GHz Speed, and 4 Hrs Battery LifeProfiler Model is limited to 2 levels of each 4 Attributes: for example, we can not set up Disk Size at 50GB 01/31/20222022 JMP Europe Discovery SummitFor any special condition, you want to know what the probability is.Run to ground why Probability profiler vs Utility profiler

13. 13Engineering/Statistics: Warranty Consideration01/31/20222022 JMP Europe Discovery SummitSold Price $1,000Extended Warrantee Protection Plans2 Years: $1023 Years: $141Original Limited Warranty1 year Depot or Carry-inProfiler at Purchase ConditionRight after purchase or as part of purchase, do you purchase extended warranty or not? If higher price, warranty protection plan price is correspondingly higher.

14. Technology: Original Limited Warranty01/31/20222022 JMP Europe Discovery Summit14Warrants that each hardware product that you purchase is free from defects in materials and workmanship under normal use during the limited warranty period. The warranty period for the product starts on the original date of purchase specified on your sales receipt or invoice unless the Producer informs you otherwise in writing. The warranty period and type of warranty service that apply to your product are designated below under the section titled “Warranty Information.” Types of Limited Warranty Service Customer Replaceable Unit (“CRU”) Service: your Service Provider will ship CRUs to you for installation by you.On-site Service: your Service Provider will either repair or exchange the product at your location.Courier or Depot Service: your product will be repaired or exchanged at a designated service center, with shipping arranged by your Service Provider. Customer Carry-In or Mail-In Service: your product will be repaired or exchanged at a designated service center, with delivery or shipping arranged by you. You are responsible to deliver or mail, as your Service Provider specifies, (prepaid by you unless specified otherwise) the product, suitably packaged to a designated location.Product Exchange Service: the service provide will ship a replacement product to your location. You are responsible for its installation and verification of its operation. In a future presentation: might make sense to simplify slide and put extra detail in notes.Note: warranty policy information leveraged from Dell website

15. Engineering/Statistics: JMP Reliability Forecast ModelThe Reliability Forecast helps you predict the number of future failures. JMP estimates the parameters for a life distribution using production dates, failure dates, and production volume. Using the interactive graphs, you can adjust factors such as future production volumes and contract length to estimate future failures. Repair costs can be incorporated into the analysis to forecast the total cost of repairs across all failed units.Based on the failure rate, repair cost, and life distribution model, the risk of setting Original Limited Warranty and the Extended Protection Warranty Policy can be estimated.01/31/20222022 JMP Europe Discovery Summit1515

16. Engineering: Reliability Bathtub Curve and Warranty01/31/20222022 JMP Europe Discovery Summit16Limited WarrantyExtended Warranty Protection

17. Statistics: Sold and Return Qty Database01/31/20222022 JMP Europe Discovery Summit17Field return Qty information on 04/2011 (in the first 22 Months) was available for the items sold in the first 6 months (07/2009-12/2009).Sold Qty was recoded by month and Return Qty was by Quarter Not real data, and only used to build reliability model for explanatory and illustrative purpose (from JMP sample data directory).

18. 01/31/20222022 JMP Europe Discovery Summit18Statistics: Fit Reliability Life DistributionWeibull distribution fit has shown the best goodness among others across AICc, Likelihood, and BIC

19. 01/31/20222022 JMP Europe Discovery Summit19Statistics: Weibull Failure Probability and Survival ProbabilityWeibull distribution β shape is at 1.76 (after 22 months field return monitoring).The above may indicate that reliability has passed beyond EOL and enter the wear-out maintenance stage since β >1Use JMP Estimate Probability to report the Survival Probability at the months after sold If you don’t purchase warranty, what is probability of failure?It’s not black-and-white clear whether we should purchase warranty or not in this case. We need more information.Survival, Hazard or Failure Function (Life Distribution).

20. 01/31/20222022 JMP Europe Discovery Summit20Engineering: Extended Warranty Protection Decision for ConsumerHere, authors will derive a simple decision model for consumers whether to purchase the extended Warranty Protection offers (2 years or 3 years) beyond current 1-year original warrantyThe consumer decision model will consider the following factors:The survival probability at each month between one year and three yearsThe failure probability at each month between one year and three yearsThe market laptop value at each month between one year and three yearsThe Monetary loss if not purchasing the extended warranty protection offersCompare the Monetary loss vs. the expense of purchasing the extended warranty protection offers

21. 01/31/20222022 JMP Europe Discovery Summit21Engineering: Extended Warranty Protection Decision for ConsumerThe survival probability at each month between one year and three yearsThe failure probability at each month between one year and three yearsUse the previous Weibull Estimate of the Survival Probability at each month to calculate the Conditional Failure Probability at each month between 1 year and 3 yearsLag( :Survival Probability at that Month, 1 )(:Survival Probability at Previous Month - :Survival Probability at that Month) * :Survival Probability at Previous Month

22. 01/31/20222022 JMP Europe Discovery Summit22Engineering/Statistics: Market Laptop ValueThe market value of any laptop product would be gradually degraded after the initial releaseLinear regression fit of the market value over monthsCalculate the remaining market value% at each month after sold1.0016474 - 0.0132825 * :Months after Purchased

23. 01/31/20222022 JMP Europe Discovery Summit23Cost of not Purchasing the Extended Protection:Conditional Failure Probability at that Month * :" Market Value% at that Month"n *1000Calculate the Cost for each month if not purchasing the extended warranty protection offersCost is $48 if not purchasing the 2-years protection planCost is $95 if not purchasing the 3-years protection planExtended Warranty Protection Plans2 Years: $1023 Years: $141There is about $50 gap between the Warranty plan and the estimated Cost which may be attributed to the Services and other Fixed Costs

24. 01/31/20222022 JMP Europe Discovery Summit24Reliability Forecast ModelThe Reliability Forecast Model can predict the field return Qty for Service Provider to prepare the rework activities.Long term repair forecast tells what the value proposition is. You don’t want to make money in beginning but lose money in the end.Question for Live Q&A: Any other company use Reliability for calculating warranty policy??

25. STEM Take Away Learnings and Future WorkLearning STEM could decompose each STEM element and combine it in an interdisciplinary framework.JMP 16 Choice Design can help Design Survey method, and Choice Model can analyze the survey data.Engineering Reliability analysis can be used to enhance decision-making by looking at the cost vs benefit of warranty purchase at different stages of ownership.2021-2022 SOHS Core Team Leaders will introduce this JMP Choice Design Survey Method to SOHS MSB Students and Teachers as their class tutors.More motivated to learn JMP software and statistics through STEM projects.Future Work: Increase Choice Levels to make it more a continuous model (to further epand upon the current Choice Model framework).2501/31/20222022 JMP Europe Discovery Summit

26. References on Statistical DetailsBurnham, K. P., and Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. 2nd ed. New York: Springer-Verlag Firth, D. (1993). “Bias Reduction of Maximum Likelihood Estimates.” Biometrika 80:27–38.  Heinze, G., and Schemper, M. (2002). “A Solution to the Problem of Separation in Logistic Regression.” Statistics in Medicine 21:2409–2419. Kaminskiy, M. P., and Krivtsov, V. V. (2005). “A Simple Procedure for Bayesian Estimation of the Weibull Distribution.” IEEE Transactions on Reliability 54:612–616. Kessels, R., Jones, B., Goos, P. (2011). “Bayesian Optimal Designs for Discrete Choice Experiments with Partial Profiles.” Journal of Choice Modelling 4:52–74. Louviere, J. J., Flynn, T. N., and Marley, A. A. (2015). Best-Worst Scaling: Theory, Methods and Applications. Cambridge: Cambridge University Press. McFadden, D. (1974). “Conditional Logit Analysis of Qualitative Choice Behavior.” in Frontiers in Econometrics edited by P. Zarembka, 105–142. New York: Academic Press. Accessed April 25, 2016. Meeker, W. Q., and Escobar, L. A. (1998). Statistical Methods for Reliability Data. New York: John Wiley & Sons. Nelson, W. B. (1982). Applied Life Data Analysis. New York: John Wiley & Sons. Rossi, P. E., Allenby, G. M., and McCulloch, R. (2005). Bayesian Statistics and Marketing. Chichester, UK: John Wiley & Sons. Sall, J. (2008). “Choice Experimental Designs Are Different.” Accessed January 22, 2016.  Train, K. E. (2009). Discrete Choice Methods and Simulation. 2nd ed. Cambridge University Press.2601/31/20222022 JMP Europe Discovery Summit

27. Backup Slides (not for presentation)7/14/20212021 ASQ/IEOM/SOHS High School Competition Event27

28. 28Engineering/Statistics: Collect Survey DataStratified Sampling (Avoid Bias)Population: SOHS High SchoolersSplit Population to 5 Clusters by Grade (G8-G12)Collect 2 Simple Random Sampling (1 Boy and 1 Girl) per each 5 clustersProvide $10 Starbucks Card eachData Collection:Survey Duration < 5 mins through Google FormEach Survey: 8 Choice Sets with 2-levels Four Attributes in each Choice SetRespondents can use (0,1) to type in their choice in Response Indicator Columns: 1 is their choice01/31/20222022 JMP Europe Discovery SummitStacked 10 Respondents

29. 29Statistics/Science: Consumer Choice Model PlatformChoice modeling, pioneered by McFadden (1974), is a powerful analytic method used to estimate the probability of individuals making a particular choice from presented alternatives. Choice modeling is also called conjoint choice modeling, discrete choice analysis, and conditional logistic regression.The Choice Modeling platform uses a form of conditional logistic regression (Categorical Response) to estimate the probability that a configuration is preferred. Unlike simple logistic regression, choice modeling uses a linear model to model choices based on response attributes and not solely upon subject characteristics. In choice modeling, a respondent might choose between two levels that are described by a combination of four attributes in our case studySegmentation: Market researchers sometimes want to analyze the preference structure (subject Effects: Grade, Sex) for each subject separately in order to see whether there are groups of subjects that behave differently.01/31/20222022 JMP Europe Discovery Summit

30. 01/31/20222022 JMP Europe Discovery Summit30Extended Warranty Protection Cost Model for Service ProviderExtended Warranty Protection Cost Model for Service Provided can be derived based on Reliability Life Distribution