Benefit-Risk Assessment in Medical Product Development - PowerPoint Presentation

1. Benefit-Risk Assessment in Medical Product DevelopmentPresented by:Tommi Tervonen, PhDMartin Ho, MSISPOR 2023 | Sunday, 7 May 2023

2. Copyright, Trademark, and ConfidentialityThis course was developed by ISPOR for members and other interested parties. Unless referenced, it is the property of ISPOR and confidential. No part of this document may be disclosed or repurposed in any manner without the prior written consent of ISPOR – The professional society for health economics and outcomes research.

3. Antitrust Compliance StatementISPOR has a policy of strict compliance with both United States, and other applicable international antitrust laws and regulations.Antitrust laws prohibit competitors from engaging in actions that could result in an unreasonable restraint of trade. ISPOR members (and others attending ISPOR meetings and/or events) must avoid discussing certain topics when they are together including, prices, fees, rates, profit margins, or other terms or conditions of sale.Members (and others attending ISPOR meetings and/or events) have an obligation to terminate any discussion, seek legal counsel’s advice, or, if necessary, terminate any meeting if the discussion might be construed to raise antitrust risks.The Antitrust policy is available on the ISPOR website.The Antitrust policy is available on the ISPOR website at ispor.org/antitrust.

4. Welcome!Tommi TervonenChief ScientistKielo ResearchMartin HoBiostatisticianASA Fellow

5. Learning objectivesAfter the completion of this course, the participants will be able to:Recognize how to use benefit-risk assessment techniques in medical product developmentFollow the steps required for conducting a benefit-risk assessmentIncorporate patient preference data into benefit-risk assessments

6. AgendaTimeAgendaPresenter8.00-8.051. Opening and introductionTommi8.05-8.502. History and real-life examples of benefit-risk assessmentMartin8.50-9.00Break9.00-10.003. Qualitative and quantitative benefit-risk assessment: methodsTommi10.00-10.15Break10.15-11.154. Case study in plaque psoriasis: introduction and problem structuring exerciseMartin11.15-11.305. Preference elicitation for the case study: interactive sessionTommi11.30-11.456. Hands-on analysisTommi11.45-12.007. ClosingTommi

7. HousekeepingMake sure you can connect to wifi / mobile data with some device (computer/tablet/mobile phone) – you will need that at the preference elicitation exerciseQuestions during the course? Please interrupt us!QR CODE(to be added)

8. History and real-life examples of benefit-risk assessment  208.05-08.50

9. “Because all drugs can have adverse effects, the demonstration of safety requires a showing that the benefits of the drug outweigh its risks.”-- U.S. FDA Benefit-Risk Assessment for New Drug and Biological Products Draft Guidance (2021) *Available at https://www.fda.gov/media/152544/download

10. Benefit-Risk Assessment (BRA)BRA is a process of weighing potential favorable and unfavorable effects associated with a medical product to make an informed decision. Decision: Make a choice out a given set of options by weighing each option’s associated effects relative to the effects of the other options.Effects: Key performance of a medical product derived from available data from various sources, often with uncertainty.Weighing: Favorable and unfavorable effects rarely occur naturally on the same scale and never with the same certainty

11. What are typical benefits and risks?Who benefits?Clinical importance of the benefitSurvivalSymptomatic improvementCurePreventionQOLModified progression of conditionMagnitude and chance of benefitsWho is harmed?Clinical importance of harmSeriousnessSeverityFrequencyReversibilityTolerabilityIncreased severity of conditionMagnitude and chance of benefits

12. Structured BRAFor many years, BRA was usually an informal processMany decisions are ‘no brainers’ but some are hardFor hard decisions, informal BRA can lead to institutional biasesPatients and physicians may opt for interventionRegulators may tend to avoid riskPayors may tend to minimize short-term costsOver past decade, push from several directions to have formal, structured BRA for critical decision pointsAlways includes quantitative aspectsMay have quantitative foundation

13. Regulatory milestones of benefit-risk and patient preferences20152012201620202022FDA CDRH draft guidance on benefit-risk assessmentExplicit mention of factors for considerations: benefits, risks, uncertainty & patient perspectives (Final: 2016)EMA Benefit-risk projectMCDA as the preferred technique for quantitative benefit-risk assessmentEMA regulatory science to 2025Highlights importance of including patient preferences in benefit-risk assessmentFDA CDER & CBER draft benefit-risk guidanceExplicit mention of patient preferencesNICE scientific advice on patient preferencesFirst formal adviceIMI PROTECTReview of potential methodologies for benefit-riskEffects table in EPARFirst formal structure for reporting key benefit-risk data in regulatory reportsFDA CDRH final guidance on patient preference Detailed guidance also referenced by CDER and CBER for evaluating patient preference studiesAbbreviations: EMA, European Medicines Agency; MCDA, multi-criteria decision analysis; EPAR, European Public Assessment Report; FDA, Food and Drug Administration; CDRH, Center for Devices and Radiological Health; CDER, Center for Drug Evaluation and Research; CBER, Center for Biologics Evaluation and Research; MDIC, medical device innovation consortium; NICE, National Institute for Health and Care ExcellenceIMI PREFERReview and case studies of key patient preference methodologiesEMA qualification of patient preference methodologiesFour methods tested by PREFER qualified for regulatory useISPOR Task Force on Quantitative Benefit-Risk Assessment: Good Practice GuidanceDetailed guidance for developing benefit-risk modelsMDIC Benefit-Risk Framework & Catalog of Methods Incorporate patient preference info. re benefit and risk into the regulatory assessments of med. tech.

14. What are preferences?“Qualitative or quantitative statements of the relative desirability or acceptability of attributes that differ among alternative interventions.”Medical Device Innovation Consortium (PCBR Framework Report 2015)Stories from individualsEvidence representative of target populationDefined by what people are willing to give uporhealth statescare processesreduce burdenotherCharacteristics or featuresOften obtained from surveys

15. Key Regulatory Guidance – ICH“A descriptive approach that explicitly communicates the interpretation of the data and the benefit-risk assessment will generally be adequate.An applicant may choose to use methods that quantitatively express the underlying judgments and uncertainties in the assessment.Analyses that compare and/or weigh benefits and risks using the submitted evidence may be presented.”

16. Key Regulatory Guidance – FDA / Benefit-Risk“Benefit-risk planning by the sponsor, beginning early in development, can add value by helping to ensure that the clinical trial data and other supporting information collected are best suited to support the benefit-risk assessment.Various qualitative structured approaches and supporting tools […] may be useful to support sponsors’ benefit-risk planning, assessments, and communications with FDA.Patient preference information may be useful to sponsors at various stages of drug development, including […] informing benefit-risk assessment.

17. Key Regulatory Guidance – FDA / Patient Preferences“Patient preference information may be particularly useful in evaluating a device’s benefit-risk profile when patient decisions are “preference sensitive.”Patient decisions regarding treatment options are preference sensitive when:multiple treatment options exist and there is no option that is clearly superior for all patients;when the evidence supporting one option over others is considerably uncertain or variable; and/orpatients’ views about the most important benefits and acceptable risks of a technology vary considerably within a population, or differ from those of healthcare professionals”

18. Current Opinion: European Medicines Agency Qualification of Patient Preferences“There is a shared interest in structuring patient involvement, including patient preference studies, in regulatory decision-making processes.[…]Overall, it is agreed that the framework is suitable for informing on objectives, design and conduct, and reporting of patient preference studies.”

19. Source: Benefit-Risk Assessment for New Drug and Biological Products Guidance for Industry. FDA, September 2021.FDA CDER B-R FrameworkStructured qualitative approach for B-R assessment and communicationNot a complete structured BRA, reflecting reality of FDA’s BRA practices May need to add an explicit place for reporting on the relative importance of outcomesTherapeutic context for weighing benefits and risksProduct-specific assessments based on available evidence

20. EMA Effects TableIntroduced as “mandatory” component for EPARs in 2015Structured description of key benefits (“favourable effects” and risks (“unfavourable effects”)“Two steps” from a full quantitative assessmentAbbreviations: EPAR, European Public Assessment ReportSource: Lenvima EPAR, European Medicines Agency, 26 March 2015.

21. Outline of Qualitative Benefit-Risk AssessmentTreatment ATreatment BTreatment CProblem structuringMeasure treatment performanceEvaluate and compare alternativesPerformance metrics

22. Outline of Quantitative Benefit-Risk AssessmentAbbreviations: MCDA, Multi-Criteria Decision Analysis; SMAA, Stochastic Multicriteria Acceptability AnalysisSame as in qualitative BRATreatment ATreatment BTreatment C   Overall B-R scoreElicit stakeholder preferencesProblem structuringMeasure treatment performanceEvaluate and compare alternativesMCDA65%30%5%SMAAPerformance metricsStakeholder preferences

23. Case Study 1: EMA BRA of NatalizumabActive drugNatalizumab (Tysabri)IndicationRelapsing-remitting multiple sclerosis Regulatory historyApproved 2004Licence withdrawn 2005Reintroduced due to patient demand 2006The Committee for Medicinal Products for Human Use (CHMP) reassesses benefit risk and continues approval in 2009ComparatorsAvonex, Copaxone, Movectro ChallengeTreatments are effective at reducing disease progression and relapse rateBut they also have frequent or serious adverse eventsHow to judge if the benefits are worth the risks?

24. Case Study 1: EMA BRA of Natalizumab

25. Ticagrelor was originally approved for secondary prevention of cardiovascular events.Applicant submitted structured benefit-risk assessment using the Benefit-Risk Action Team Framework, but no quantitative benefit-risk assessment, to support label extension to primary prevention.The FDA conducted independently a Multi-Criteria Decision Analysis to assess benefit-risk of ticagrelor in the new indication.Case Study 2: Quantitative BRA by FDASource: Lackey LG, Garnett CE, Senatore F. Applying Decision Analysis to Inform the US Food and Drug Administration’s Benefit–Risk Assessment of Ticagrelor for Primary Prevention of Myocardial Infarction or Stroke Based on THEMIS. Circulation 2021;144(8):655-658

26. Case Study 2: FDA’s Quantitative BRA (Steps 1 to 3)Source: Lackey LG, Garnett CE, Senatore F. Applying Decision Analysis to Inform the US Food and Drug Administration’s Benefit–Risk Assessment of Ticagrelor for Primary Prevention of Myocardial Infarction or Stroke Based on THEMIS. Circulation 2021;144(8):655-658

27. Case Study 2: FDA’s Quantitative BRA (Steps 4 to 5)Source: Lackey LG, Garnett CE, Senatore F. Applying Decision Analysis to Inform the US Food and Drug Administration’s Benefit–Risk Assessment of Ticagrelor for Primary Prevention of Myocardial Infarction or Stroke Based on THEMIS. Circulation 2021;144(8):655-658

28. Case Study 2: FDA’s Quantitative BRA (Steps 6 to 8)Source: Lackey LG, Garnett CE, Senatore F. Applying Decision Analysis to Inform the US Food and Drug Administration’s Benefit–Risk Assessment of Ticagrelor for Primary Prevention of Myocardial Infarction or Stroke Based on THEMIS. Circulation 2021;144(8):655-658

29. Structured BRA submitted by applicant“First, the applicant considers a number of composite endpoints.Second, […] by including the composites, the Applicant has thereby introduced triple counting of clinical outcomes.Third, the Applicant used the full analysis set for all measures. In contrast, the FDA statistical reviewer used the full analysis set for key benefits and the safety analysis set for key risks.”Highlights the importance of planning for quantitative B-R assessment even if you do not submit it.Case Study 2: Applicant’s Quantitative Benefit-Risk Assessment

30. NxStage hemodialysis machine is cleared for home hemodialysis with a partner and nocturnal use.Sponsor seek a label expansion to include solo home dialysis. Patient groups also conveyed their interest in reducing burden on patients despite potential risks of home dialysis without a partner overnight.The regulator's quantitative BRA for specific to this decision: What is patients' maximum acceptable risk (MAR) in exchange for the reduced burden?If patients' MAR close to the regulator's, *and* additional safety measures are deemed satisfactory, then the machine's proposed label expansion might seem reasonableCase Study 3: Maximum Acceptable Risk (1/3)Source: Todd Snell presentation at the ISPOR FDA workshop "Using Patient Preference Information in Medical Device Regulatory Decisions" September 2020; see ISPOR Value & Outcome Spotlight at https://t.ly/KmHL.

31. The machine has an established safety profile for in-clinic and home dialysis.Risk #1: Dialysis-induced hypotension (approx. 1 in 20 treatments)Risk #2: Needle dislodgment, leading to blood loss, loss of consciousness, or death (approx. 1 in 450 patients per year)MAR #1: Identify risk tolerance threshold of death for which experienced HHD/in-center self-care patients remain willing to choose to perform Solo HHD.MAR #2: Identify risk tolerance threshold of needle dislodgement for which experienced HHD/in-center self-care patients remain willing to choose to perform Solo HHD.Case Study 3: Maximum Acceptable Risk (2/3)

32. 16%16%20%24%Case Study 3: Maximum Acceptable Risk (3/3)Results49% of patients with ≥ 1 yr. experience selected Solo if the risk of death was 30% (vs. 16% for in-center hemodialysis). 53% of patients with ≥ 1 yr. experience selected Solo if the risk of needle-dislodgement-led serious injury was 11% (vs. 0.2%)

33. Traditionally, treating young children with middle ear inflammation required tympanostomy under general anesthesia in an operating room, with a success rate ~ 100%. Tusker Medical Tubes Under Local Anesthesia (TULA) System is a procedure in a doctor’s office using local anesthesia but with a lower success rate - “Toddlers typically do not like you to do things to them.”The regulator's quantitative BRA for specific to this decision: What is patient guardians’ minimum acceptable benefit (MAB) of TULA in exchange for reduced burden?“The primary effectiveness endpoint of Procedural Success was based on the results of a Patient Preference study” --TULA’s Summary Of Safety And Effectiveness (https://t.ly/skRa)Case Study 4: Minimum Acceptable Benefit as Performance GoalSource: Dan Harfe’s presentation at the ISPOR FDA Workshop "Using Patient Preference Information in Medical Device Regulatory Decisions" September 2020; see ISPOR Value & Outcome Spotlight at https://t.ly/KmHL.

34.

35. Qualitative and quantitative benefit-risk assessment: methods309.00-10.00

36. Classification of Benefit-Risk FrameworksBRAT, FDA B-R framework, EMA effects tableNNT/NNH, NCBMCDA, SMAAAbbreviations: BRAT, Benefit-Risk Action Team; FDA, Food and Drug Administration; EMA, European Medicines Agency; PROACT-URL, Problems, Objectives, Alternatives, Consequences, Trade-offs, Uncertainty, Risk attitudes, and Linked decisions; NNT, Number Needed to Treat; NNH, Number Needed to Harm; NCB, Net Clinical Benefit; MCDA, Multi-Criteria Decision Analysis; SMAA, Stochastic Multicriteria Acceptability Analysis

37. BRAT FrameworkDeveloped by PhRMA Other qualitative frameworks are similarAbbreviations: PhRMA, Pharmaceutical Research and Manufacturers of America; BRAT, Benefit-Risk Action TeamSource: BS Levitan et al. Application of the BRAT Framework to Case Studies: Observations and Insights. Clin Pharm Ther 2011

38. Define decision context (Step 1)

39. Identify key benefit and risk outcomes (Step 2) Identify the set of key benefit and risk outcomesValue attributes can be generated top-down or bottom-upTop down: ‘Value focused thinking’ Bottom up: identifying characteristics that distinguish alternativesTop-down approach preferred and is generally believed to lead to better analyses*Develop a value tree*Keeney: Value-focused thinking. Harvard University PressFigure source: P Coplan et al. Development of a Framework for Enhancing the Transparency,Reproducibility and Communication of the Benefit–Risk Balance of Medicines. Clin Pharm Ther 2011

40. Identify key benefit and risk outcomes (Step 2): Composite Endpoints and Grouped Adverse EventsOften benefit (favorable effect) outcomes are composite endpointsE.g., risk of MACEAdverse events are often groupedE.g., CTCAE Grade 3-4 AEsWhether to use individual or composite/grouped events depends on the contextMirror clinical rationaleUse primary endpoint(s), if possibleHowever, benefit-risk assessment does not need to use same endpoints – e.g. to avoid double-counting of eventsOften there is focus on a limited number of key safety concernsAbbreviations: MACE, Major Adverse Cardiac Event; CTCAE, Common Terminology Criteria for Adverse Events; AE, Adverse Event

41. Identify and extract source data (Step 3)Identify data sourcesRegulatory submissions typically based on pivotal trialsConsider compatibility of trial populations and need for indirect treatment comparisonsExtract and organize source dataSource detailsEMA effects table format appropriate

42. Display and interpret benefit-risk metrics (Step 6)Develop summary table of key dataVisualize with appropriate measuresIf primary data sources report relative measures, include visualization of absolute measuresAssess preference sensitivityFigure source: BS Levitan et al. Application of the BRAT Framework to Case Studies: Observations and Insights. Clin Pharm Ther 2011

43. ISPOR Good Practice Guidance for Quantitative Benefit-Risk Assessment

44. Research Question (Step 1) Identify needs of decision makersDecision maker view of the preference sensitivity Identify requirements for preference dataSpecify source of preference data Determine what data is needed: single trade-off (MAR) or trade-ofs among multiple attributes? Specify role of external expertsClinical, subject area, eg, regulatory or HTA, or patient experts are often involved in beyond providing preference data

45. Model Development – Value function format (Step 2)Preferential independence implies that the value is additiveResults in unitless ‘benefit-risk score’ v(x)Preference parameters to be elicited include and  Additive multi-attribute value functionWhere partial value functions are normalized so, thatAnd weights express relative importance of scale swings  

46. Model Development – Value function format (Step 2): example 1Two attributesResponse rate (from 0 to 20%)Risk of an adverse event (from 0 to 40%)Weights imply that:Increase of response rate from 0 to 20% is as important as decrease of AE rate from 40 to 0%1% decrease in risk of AE is same value anywhere from 0 to 40% (preferences are linear)Preferences for improvements in efficacy are not linear  

47. Model Development – Value function format (Step 2): example 2Two attributesResponse rate (from 0 to 20%)Risk of an adverse event (from 0 to 40%)Weights imply that:Increase of response rate from 0 to 20% is three times as important as decrease of AE rate from 40 to 0%Weight meaning depends onOther weights (=ratio of weights)Sizes of scale “swings”  

48. Model Development – Value function use (Step 2): example 2  Treatment ATreatment BResponse10%20%Adverse events20%40%Treatment ATreatment BV(Response)0.61.0V(Adverse events)0.50Treatment ATreatment BV0.6 * 0.75 +0.5 * 0.25=0.5751.0 * 0.75 +0 * 0.25=0.75Performance matrixCalculatePartial valuesWeightPartial valuesTreatment B Treatment A 

49. Model Development (Step 2): Select benefit and safety endpointsNumerous efficacy endpoints are typically reported in any clinical trial, and often many endpoints are causally dependent and reflect the same underlying eventNeed to select key end-pointsExample: MI counted in or may cause events of most endpoints

50. Model Development (Step 2): Select benefit and safety endpointsThe criteria for a “key” endpoint typically includeEvent frequencyAbsolute risk difference between treatmentsClinical importanceTime courseVariability in subgroupsReversibilityTolerabilityRegulatory relevanceAdverse event severityDepend on the research question

51. Model Development (Step 2): Eliminate double-countingA common issue in qualitative benefit-risk assessmentsDouble-counting: an event occurring in more than one endpoint in the effects tableStrategies to deal with double-counting:Include subset of key outcomes in the modelRe-structure endpointsAccount for alternative sets of endpoints in structural sensitivity analyses (e.g., vary primary outcome measure)

52. Model Development (Step 2): Consider Attribute value dependenceAdditive value model requires the attributes to be preferentially independent: E.g., stakeholders’ preferences for an increase in survival time may depend on the level of quality of life experiencedSuspected preferential dependence can be tested with qualitative workStrategies to deal with preferential dependence:Re-structuring the set of outcomesUse value model with multiplicative termsDifficult to interpret and elicit – not widely used

53. Model Development (Step 2): Consider Attribute value dependence: examplePreferential dependence is not equal to distributional dependenceTwo outcomes can be highly correlated for all treatments and still measure different conceptsOutcomes measured on probability scales are mostly preferentially independentHigh salaryLow salary(Unaffordable leisure)(Affordable leisure)

54. Preference Elicitation (Step 3): Choose preference elicitation methodThree main methods:Discrete choice experiment (DCE) or Best-worst scaling case 3Threshold techniqueSwing weightingEach method provides information on trade-offs (i.e., HOW important risks and benefits are relative to each other)NOTE: ‘Weights’, ‘marginal utility’, or ‘relative attribute importance’ can only be interpreted in relation to changes across scales

55. Preference Elicitation (Step 3): Choose preference elicitation methodDiscrete Choice ExperimentDescribes 2 or more full treatment profiles with varying levels of included attributesRequires careful development, pre- and piloting, and a complex experimental designChoice task format needs to mimic with clinical situationSimons G, Veldwijk J, DiSantostefano RL, et al. Preferences for preventive treatments for rheumatoid arthritis: discrete choice survey in the UK, Germany and Romania. Rheumatology (Oxford). 2023 Feb 1;62(2):596-605. doi: 10.1093/rheumatology/keac397. PMID: 36068022; PMCID: PMC9891433.

56. Preference Elicitation (Step 3): Choose preference elicitation methodThresholding TechniqueSimple technique, little design properties, pre-testing required, results in close-to exact individual trade-offsDescribes a full treatment profile usually compared to standard of care Iterative process to elicit a single trade-off (multi-dimensional thresholding possible extension) Simons G, Janssen EM, Veldwijk J, et al. Acceptable risks of treatments to prevent rheumatoid arthritis among first-degree relatives: demographic and psychological predictors of risk tolerance RMD Open 2022;8:e002593. doi: 10.1136/rmdopen-2022-002593 

57. Preference Elicitation (Step 3): Choose preference elicitation methodSwing weightingSimple technique, little design properties, pre-testing required, results in exact trade-off estimate for each subjectAsks respondents to indicate their preferences for 'swings' in attributes (possible to extent with point allocation task)Less suitable for online survey setting: time intense and requires skilled moderator

58. Preference Elicitation (Step 3): Choose preference elicitation methodResearch questionRequirements for preference dataPractical considerations Feasible sample sizeAvailable timeBudgetAvailable expertise DCE and BWS Case 3Threshold TechniqueSwing WeightingTaxonomy of methodDiscrete choice-basedIndifferenceRatingDesign ConsiderationsQuestion Format; type of dataChoice; ordinal dataChoice; ordinal data Rating, cardinal dataNumber of trade-offs ≥21≥1Number of attribute levels variedAllOneVariesCan capture interaction effectsYesNo NoDirect vs indirect elicitationIndirectDirectDirectImplementation ConsiderationsSample sizeTypically >100Typically <100Typically <100Study durationTypically > 12 monthsTypically < DCEVariesAnalytical ConsiderationsPreference parameters elicitedFull value function (, )Single trade-off (Attribute weights ()Allows for status quo alternativeYesYesNo DCE and BWS Case 3Threshold TechniqueSwing WeightingTaxonomy of methodDiscrete choice-basedIndifferenceRatingDesign ConsiderationsQuestion Format; type of dataChoice; ordinal dataChoice; ordinal data Rating, cardinal dataNumber of trade-offs ≥21≥1Number of attribute levels variedAllOneVariesCan capture interaction effectsYesNo NoDirect vs indirect elicitationIndirectDirectDirectImplementation ConsiderationsSample sizeTypically >100Typically <100Typically <100Study durationTypically > 12 monthsTypically < DCEVariesAnalytical ConsiderationsPreference parameters elicitedAllows for status quo alternativeYesYesNo

59. Preference Elicitation (Step 3): Ensure appropriate framing of attributesAssessing benefits and risks is cognitively demandingMany probabilistic attributes and small probabilitiesFraming and number of included attributes on a probability scales can induce heuristics and measurement biasPositive and negative framing as well as graphical presentationPrioritize non-probability representation of attributesWhen attributes using a probability scale are used:Present as absolute instead of relative changes such as relative riskUse same denominators within and across attributesInclude graphics such as icon arraysAlways pre-test preference elicitation instrument

60. Preference Elicitation (Step 3): Assess quality of preference dataData quality measures include:Validity assessmentsTime to complete surveyResponse to comprehension questionsThese may be interpreted based on:Educational levelHealth literacy and numeracyTable source: Tervonen et al. Maintenance inhaler therapy preferences of patients with asthma or chronic obstructive pulmonary disease: a discrete choice experiment. Thorax 2020;75:735-43.

61. Analysis (Step 4): Normalize Preference weightsStandard analysis pipeline requires separation of weights from partial value functionsIn DCE partial value functions need to be rescaled to extract “weights”Weights sum to constant e.g., 1Partial value functions on [0, 1] rangeDepending on DCE coding strategy (effects / dummy coding, exact specification), may need to re-center and/or invert value functions“Weight” of attribute Mortality risk

62. Analysis (Step 4): Conduct base-case analysisCharacterization of benefit-risk scores of all treatment optionsContribution of individual attributes to the benefit-risk scoresBenefit-risk score differences between treatmentsIf preference data coming from a DCE, then include also predicted choice probabilities

63. Analysis (Step 4): Conduct base-case analysis: Reading net benefit-risk chartsRelative contribution of effect to the net B-R score depends on scale rangeFor safety attributes, lower risk levels “give” value

64. Analysis (Step 4): Sensitivity analysesOne-way sensitivity analysis of clinical effect estimatesOne-way sensitivity analysis of preference weights

65. Analysis (Step 4): Sensitivity analysesMulti-way sensitivity (SMAA) analysis of clinical effect estimatesStructural sensitivity analysesDifferent outcomesDifferent outcome measurementsDifferent timeframes (e.g. shortvs. long-term safety)

66. Analysis (Step 4): Analyze preference heterogeneityStructural sensitivity analyses using preference data from key subgroupsIn case preference heterogeneity is captured in a continuous formatPresent distributions of treatments’ benefit-risk scores, or use in SMAA analysis

67. Communication of results (Step 5): fill out reporting checklistISPOR Task Force checklist includes 27 items grouped by the different stepsChecklist can also be used in design of a benefit-risk studyItemName of itemDescription of itemStep 1: Reporting the Research Question1 TitleIdentify the study as a quantitative benefit-risk assessment used to weigh preferences and describe the interventions including (if relevant) the decision context.2AbstractProvide a structured summary of objectives, setting, methods used to weigh preferences (including study design, participants and input data), results (including base-case and sensitivity analyses) and conclusions.3Research questionExplain its relevance to inform, for example, regulatory decisions or health technology assessment.4ComparatorsDescribe the interventions or strategies compared and state why they were chosen by describing the decision context.5PerspectiveDescribe the rationale for the selection of whose preferences were used to address the research question.Step 2. Reporting the process of model development6 Defining risks and benefitsDefine and describe the relevant benefits and risks (harms) for the interventions compared and/or decision context.7Selection of risks and benefitsDescribe the process used to select the included benefits and risks (harms) from the list of potential benefits and risks (harms) for the interventions or strategies compared. 8Time frameDefine and describe the relevant time frame for the quantitative benefit risk assessment of interventions or strategies compared.

68. Communication of results (Step 5): Ensure efficient communication of results to decision makersMost decisions informed by a benefit-risk assessment involve considerations beyond the results of quantitative BRAEmphasize results of sensitivity analyses to increase confidence to the resultsExpress results using decision-maker relevant scales

69.

70. Case study in plaque psoriasis: introduction and problem structuring exercise410.15-11.15

71. Case Study: Benefit-risk assessment of brodalumabBrodalumab is a biologic to treat moderate-to-severe plaque psoriasis, in non-responsive patients to non-biologic systemic treatmentsYou are tasked to develop benefit-risk assessment of brodalumab taking into account current evidence at regulatory approvalTwo doses (140 mg and 210 mg). Which dose should be approved (if any)?Figure source: Psoriasis: LSCMMG Biologic and High Cost Drug Commissioning Pathway. Version 1.7, July 2020.

72. Case Study – Group workWe would like you to split into groups of 3-5 peopleIn this task, you are to develop a value tree, and answer the following questions:Why did you choose the outcomes?What other outcomes you considered but ended excluding?

73. Benefit endpoints (favourable effects) –Clinical dataPASI: Psoriasis Area Severity IndexsPGA: static Physician Global AssessmentPSI: Psoriasis Symptom Inventory

74. Treatment risks (unfavourable effects) – Clinical data

75. Further materialDetails of the studies in Lebwohl et al. Phase 3 Studies Comparing Brodalumab with Ustekinumab in Psoriasis. NEJM 2019.Supplementary material contains further details of the adverse eventshttps://www.nejm.org/doi/full/10.1056/nejmoa1503824

76. Preference elicitation for the case study: interactive session511.15-11.30

77. Preference elicitation with multi-dimensional thresholdingPhase 1: Ranking of scale swingsPhase 2: Thresholding of individual trade-offs (for 4 attributes, thresholding on 3 pairs)

78. Preference elicitation for the case studyLink to be addedResults presented on-line using Valorem.health dashboardQR CODE(to be added)

79. Hands-on analysis611.30-11.45

80. Results to be presented

81. Closing7Section

82. Thank you for attending!ISPOR will distribute course evaluation form electronically, please do complete itFor contacting the faculty directly:tommi.tervonen@kieloresearch.commho@martinhk.us

83. Please complete your Course Evaluation Link located in the Community in “Announcements” Your Certificate of Course Completion can be downloaded from your ISPOR member profile Thank You!