What is RWE RWE realworld evidence 1 Food and Drug Administration RWE https wwwfdagovscienceresearchscienceandresearchspecialtopicsrealworldevidence Accessed July 2019 ID: 935779
Download Presentation The PPT/PDF document "R eal-world evidence Value, methodologic..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Real-world evidence
Value, methodological strengths and weaknesses
Slide2What is RWE?
RWE, real-world evidence
.
1. Food and Drug Administration. RWE. https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence (Accessed July 2019).
RWE is a term for clinical evidence relating to the usage and potential benefits or risks of medical products derived from analyses of real-world data. It can be generated via a range of different study designs, including:1Pragmatic trialsObservational studies (prospective or retrospective)
Real-world
data
(e.g. electronic medical records, insurance claims data)
Analyzed in real-world
studies
(e.g. retrospective claims review)
Real-world evidence
Context
Slide3Growing recognition of RWE within medicine
*Search algorithm: “real-world” OR “real world” OR “real-life” Or “real life”.
RWE, real-world evidence.
1. US National Library of Medicine National Institutes of Health. PubMed. https://www.ncbi.nlm.nih.gov/pubmed/ (Accessed April 2019).
ContextThere has been increasing recognition of the potential value of RWE in recent years as reflected by its increasing presence within the medical literature1Year
Slide4RWE complements RCT evidence
RWE can:
Confirm whether RCT results are observed in everyday clinical practice
1 Provide complementary insights from more varied settings1Help to translate RCT findings into evidence that can guide routine clinical practice1*e.g. different age, race, comorbidities, co-medications, adherence.
RCT, randomized controlled trial; RWE, real-world evidence.1. Nallamothu BK, et al. Circulation 2008;118(12):1294–303.
RCTs: required
for
regulatory
approval
Tightly controlled patient population
Real-world
studies
enable evaluation of:
Different
settings
Different
outcomes
Different treatments
Different populations*
Context
Slide5Real-world data source example: claims data
Healthcare service
(hospital, clinic, pharmacy)
Physical examination
Laboratory procedures
Pharmacy prescriptions
Insurance claim
Data abstraction CodingAdjudication Health databasePrivacy protectionIntegrationStandardization Codes: disease, procedures, drugs
Healthcare/pharmacy encounters RWE dataData
Slide6Real-world data source example: EHRs
Physician office records
Medical history
Physical examinationDiagnosisLaboratory
resultsPrescription informationHospital admissionsSpecialist referrals
Patient enrolled in database
Hospital visits (typically outpatient)
EHRs: electronic health records.
RWE dataData
Slide7Examples of real-world data sources and the oncology-related questions they can address
Research question
Insurance
claimsPragmatic trialCancer registriesChart reviewElectronic health records*Incidence, prevalence
Patient demographics
Cancer diagnosis
Clinical characteristicsCancer staging
Survival, mortalityBiomarkers, mutations
Laboratory resultsProcedures: surgeries, resections, radiation
Disease progressionTreatment duration
Treatment sequencingNon-oncological comorbidities
Safety events
*
Cancer-specific (i.e. not linked to wider health
records).
This slide
was developed
based
upon
Boehringer Ingelheim's experience of real-world data collection and
analysis.
Generally:
Evaluable
;
Unevaluable
;
Feasible to
infer,
or subject to data
collection
.
RWE data
Data
Slide8AE, adverse events; RCT, randomized
controlled
trial.
In addition to the two cited references, this slide was developed based upon Boehringer Ingelheim's experience of real-world data limitations.1. Southworth
MR, et al. N Engl J Med 2013;368(14):1272; 2. Larsen TB, et al. J Am Coll Cardiol 2013;61(22):2264–73. Real-world data limitationsReal-world data sourceExamples
of potential limitations (not exhaustive)
GeneralMay include off-label use Limited comorbidities/dosing information Other unmeasured confounding factors and/or methodological limitations (selection and assessment bias)
Safety/AE reporting databasesEvent reporting is not always mandatory (i.e. true number of events not accurately reflected)New drugs more likely to have AEs reported (‘Weber effect’)1,2Influenced by whether an AE is described in a drug label, publicity about the event/safety concern, class action lawsuits
Claims databasesHealth claims codes not well defined, may be inaccurate and without source data verificationOutcome definitions not always clearMissing information on comparator treatment can affect outcome interpretation
Observational data from cohort studiesInconsistent definitions of diagnoses and coding practice (e.g. grading of disease severity) Seldom
multi-national RegistriesOften lack comparator arm
Case reportsSmall sample size, limited generalizability No comparator Endpoints seldom clearly defined
Publication
bias results in greater editorial weighting towards
difficult cases
and/or
extreme outcomes (positive
or negative)
RWE data
Data
Slide9RCT, randomized
controlled
trial.
1. Khozin S, et al. J Natl Cancer Inst 2017;109(11); 2. Roche N, et al. Ann Am Thorac
Soc 2014;11(Suppl. 2):S99–S104; 3. Nallamothu BK, et al. Circulation 2008;118(12):1294–303. RCTs vs real-world studies: characteristicsRCT1–3
Real-world studies
1–3 PurposeEvaluate the efficacy and safety of therapies
Post-marketing surveillance, cost-effectiveness evaluation, comparative effectiveness, service implementation, center audit, hypothesis generationPopulation characteristicsPredefined, HIGHLY SELECTED population
(stringent inclusion/exclusion criteria to assess causality and to minimize potential confounding)More DIVERSE, ROUTINE CARE patients, including those with characteristics that would prohibit their RCT eligibility (e.g. comorbidities, advanced disease stage)Intervention or treatmentRandomized treatment allocation; c
learly defined, controlled and monitored regimensNon-randomized regimen; physician judgement as informed by label and other considerationsComparator (typically)Placebo or current gold standard
Routinely prescribed treatment alternativesOutcomeEfficacy, safety
Effectiveness, side-effect profile, patient and clinical behaviors, natural history of diseaseTime/observation period Usually short termVariable with potential for long-term evaluation
Treatmen
t adherence
Usually high
(monitored
and reinforced)
Variable, reflecting routine usage
Methods
Slide10*The approval of an extended indication of palbociclib (
for the treatment of men with HR+, HER2- metastatic breast cancer) was based predominantly on RWE.
4
**Through patient selection and optimized standards of care.HCP, healthcare professional; RCT, randomized controlled
trial; RWE, real-world evidence.1. Khozin S, et al. J Natl Cancer Inst 2017;109(11); 2. Roche N, et al. Ann Am Thorac Soc 2014;11(Suppl. 2):S99–S104; 3. Nallamothu BK, et al. Circulation 2008;118(12):1294–303; 4. Wong GWK, et al. Ann Am Thorac Soc 2014;11(Suppl. 2):S85–91; 4. Pfizer. Press Release. https://www.pfizer.com/news/press-release/press-release-detail/u_s_fda_approves_ibrance_palbociclib_for_the_treatment_of_men_with_hr_her2_metastatic_breast_cancer (Accessed July 2019). RCTs vs real-world studies: characteristics1–4
RCT
Real-world studiesAccepted by regulatory authorities
YesNot routinely*Internal validityHigh
LowGeneralizability/external validityLowHighPotential for confounding
LowHighMaximizes potential treatment effect**Yes
NoPotential to:Detect rare events
LowHigherEvaluate long-term outcomesLow
High
Evaluate
aspects of HCP/patient behaviors
Low
High
Evaluate aspects
of healthcare delivery
Low
High
Methods
Slide11RWE/RCT terminology differences
This slide was developed based upon Boehringer Ingelheim's
experience of terminology used in real-world studies and randomized clinical trials.
PFS, progression-free survival; RCT, randomized
controlled trial; RWE, real-world evidence; TTP, time to progression.Commonly used terminology often differs for real-world studies and RCTs:RCT Real-world studyTrialStudyAllocated treatment
Prescribed treatment
EndpointsOutcomesTreatment arm Treatment group/cohortTrial resultsRWEEfficacy Effectiveness
SafetySide-effect profilePFSTTPMethods
Slide121. Roche N, et al. Lancet Respir
Med
2013;1(10):
e29–30.Describing study designs in real-world terms
MethodsThe framework links various types of studies based on:population characteristics and ecology of care The typical position of common study designs are illustrated, but can be moved in any direction depending on the specifics of its study design
Conceptual framework of therapeutic
research1STUDY DESIGN ECOLOGY OF CAREFreeNarrowBroad
POPULATIONConstrained
Slide13RCT, randomized controlled trial.
1. Thorpe KE, et al. J Clin Epidemiol 2009;62(5):464–75.
PRECIS wheel: describing study designs using ten explanatory pragmatic axes
1
Methods
Explanatory studies:
Tests a causal research hypothesis: ‘Does a treatment work under ideal conditions?’
Pragmatic studies:
Helps to choose between care options: ‘Does a treatment work under usual conditions?’
Slide14Interpreting RCT findings and RWE
Results
Real-world studies often make use of
proxy endpoints
if specific variables are not routinely availableRCT endpoints and real-world outcomes can differ; methodological review is required before comparing their findings: RCT endpointTypical
RCT definition
Disaggregated componentsChallenges of real-world evaluationPFSTime from randomization* until objective tumor progression
by RECIST or deathDeathPDMonitoring and follow-up varies across healthcare settings, centers and clinicians, which limits
(or prevents) evaluation of RECIST TTPTime from randomization* until clinical progression, or deathDeath
Clinical progressionPDRoutine care methods of assessing and monitoring clinical progression are difficult to standardize for real-world evaluationTTFTime from randomization* to discontinuation for any reason
DeathClinical progressionPDAEs**
Pragmatic routine care management approaches may result in treatment through progression resulting in “time on treatment” being used as a proxy for TTF*Time from study enrollment for single-arm trials; **TTF may discriminate between discontinuations due to AEs and discontinuations for other
reasons.
This slide was developed based upon Boehringer Ingelheim's experience analyzing data from randomized controlled trials and real-world studies.AE, adverse event; PD, progressive disease;
PFS, progression-free survival; RCT, randomized controlled trial; RECIST, Response Evaluation Criteria in Solid Tumors; RWE, real-world evidence; TTF, time to treatment failure; TTP, time to progression.
Slide15Endpoints – PFS, TTP, TTF: RCT scenario 1
Conceptual scenario
:
The effect of treatment beyond progression results in TTP being longer than mPFS, and outweighs the effect of treatment discontinuation for other reasons
PDPDPDDeathPD
Results
PDPD
PDDeathPDClinical PD
Clinical PDClinical PDPDPDPDDeath
PDClinical PDClinical PDClinical PD
AEAEAEAE
AE, adverse events; mPFS, median progression-free survival; RCT, randomized controlled trial; TTF, time to treatment failure; TTP, time to progression.
Slide16Conceptual scenario
:
The effect of treatment discontinuation for other reasons has more effect than prolongation of
mPFS by
treating beyond RECIST progression
Endpoints – PFS, TTP, TTF: RCT scenario
2AE, adverse events; mPFS, median progression-free survival; RCT, randomized controlled trial; RECIST, Response Evaluation Criteria in Solid Tumors; TTF, time to treatment failure; TTP, time to progression.PD
PDPDDeathPD
ResultsPDPDPDDeath
PDClinical PDClinical PDClinical PD
PDPDPDDeathPD
Clinical PDClinical PDClinical PDAE
AE
AE
AE
AE
AE
Slide17Outcomes – TTP, ToT (TTF/TTD): Real-world scenario
Conceptual scenario
:
Routine care patients treated as long as clinician deems beneficial. A greater number of events (e.g. PD, withdrawal, discontinuation due to tolerability) contribute to ToT (TTF) analyses, making it shorter than TTP
AE, adverse events; PD, progressive disease; RECIST, Response Evaluation Criteria in Solid Tumors; ToT, time on treatment; TTF, time to treatment failure; TTP, time to progression.
Progression-free survival
mPFS: 21.5 monthsContributing events:
PDPDPDDeath
PD
Time to progressionTTP: 23.7 monthsContributing events:
Time on treatment / time to treatment failureToT/ TTF: 20.2 monthsContributing events:
PDPDPDDeathPD
Clinical
PD
Clinical
PD
Clinical
PD
PD
PD
PD
Death
PD
Clinical
PD
Clinical
PD
Clinical
PD
AE
AE
AE
AE
AE
AE
Not evaluable: in routine care, patients do not stop according to RECIST and do not have a standardi
z
ed imaging schedule
Results
Slide18Interpreting RWE: comparing vs RCT results
Scenario
Interpretation
and considerations
RCT efficacy
shown, andreal-world effectiveness shown
RWE validates RCT findings in routine care patients and/or settingsIncreases confidence in the finding, e.g. real-world studies of adjuvant chemotherapy have shown OS results comparable with those reported in several pivotal RCTs1
RCT efficacy shown, but real-world effectiveness not shown
Result raises questions about the extent to which a novel therapy can offer benefit in the real world:Does treatment only work in selected populations?
Should (particularly toxic) treatments be prescribed in older and/or sicker patients than those who were eligible for the RCT?RCT efficacy marginal, butreal-world effectiveness shown
May suggest RCT was underpowered
and real-world studies
can provide further confirmatory evidence
RCT efficacy
not shown
, and
real-world effectiveness
shown
X
The demonstration of real-world effectiveness in the absence of any RCT efficacy signal requires
close statistical scrutiny to assess potential sources of bias
*Through
patient selection and
optimized
standards of
care.
OS,
overall
survival;
RCT
,
randomized
controlled
trial; RWE, real-world evidence.1. Karim S, Booth CM. J Clin Oncol 2019:37(13):1047–50.
Results
Slide19Side-by-side comparison: real-world effectiveness vs RCT efficacy
Objective:
Measure the relationship between RCT efficacy and real-world effectiveness for oncology treatments, in:
Selected real-world patients who meet routine RCT eligibility criteria (
baseline cohort) All real-world patients with evaluable records (full cohort)Methods:RCT data: abstracted from 21 Phase III oncology RCTs reporting OS, PFS or TTP Real-world data: EHRs – SEERs Medicare dataPrimary outcome: real-world OS, estimated as MHRCox proportional hazard regression model used to calibrate difference between real-world MHR and:RCT MHRRCT PPS/TTP SHR
Predicting
Real-World Effectiveness of Cancer Therapies Using Overall Survival and Progression-Free Survival from Clinical Trials: Empirical Evidence for the ASCO Value Framework1
Author conclusions: Baseline cohort: in real-world patients eligible for RCTs, real-world OS benefits were:Similar to those observed in RCTs based on OS endpoints
16% less than RCTs based on surrogate endpoints (PFS, TTP)Full cohort: in non-selected real-world populations, treatment benefit was predicted to be even less than in the baseline, RCT-eligible real-world cohortResults
*Not reported in Ladawalla et al; included for illustrative purposes to exemplify the direction of the observed differences. EHR, electronic health record; HR, hazard ratio; MHR, mortality hazard ratio; OS, overall survival; PFS, progression-free survival; RCT, randomized controlled trial; SEERS, Surveillance and Epidemiology End Results;
SHR, surrogate hazard ratio; TTP, time to progression.1. Lakdawalla DN, et al. Value in Health 2017;20(7):866–75.
Results:
Slide20Pragmatic real-world endpoints in NSCLC studies: considerations
Methods:
18 metastatic NSCLC RCTs:
Initiated after 2007
Submitted to the FDA Involved 8,947 patientsCompared TTD to PFS and OSTTD: defined as date of randomization to date of discontinuation or death Results: Overall: TTD was more closely associated with PFS (r=0.87, 95% CI 0.86–0.87) than with OS (r=0.68, 95% CI 0.67–0.69)Oncogene-targeted subgroups: mTTDs exceeded mPFS:EGFR+: 13.4 months vs 11.4 months, respectively ALK+
: 14.1 months vs 11.3 months, respectively
Analysis of time-to-treatment discontinuation of targeted therapy, immunotherapy, and chemotherapy in clinical trials of patients with NSCLC1
Author conclusions: mTTD exceeds mPFS for oncogene-targeted subgroups due to treatment beyond RECIST progression. The ability to treat beyond progression may reflect real-world treatment tolerability and the need to continually
suppress driver mutations despite emergence of alternative resistance mutationsWith ICI therapy, mPFS (4.2 months) was slightly longer than mTTD (3.5 months), with cases of both early and late TTD. This may reflect some: cases of early termination due to immune-mediated AEs; some instances of durable treatment benefit, and others of treatment beyond conventional progressionFurther research is needed to validate TTD as a measure for pragmatic RCTsSelected KM analysis of TTD and PFS: (A) EGFR TKI vs doublet ChT with maintenance EGFR mutant population; (B) EGFR TKI vs doublet ChT (no maintenance) in EGFR mutant population; (C) EGFR TKI vs EGFR TKI in EGFR mutant population; (D) ICI vs monotherapy ChT
Results (cont’d):
PFS ChT double + maint.PFS EGFR TKI
TTD ChT double + maint.TTD EGFR TKI
PFS ChT (no maint.)
PFS EGFR TKI
TTD ChT
(no
maint.)
TTD EGFR TKI
PFS EGFR TKI Arm 1
PFS EGFR TKI Arm
2
TTD EGFT TKI Arm
1
TTD EGFT TKI Arm 2
PFS ChT monotherapy
PFS ICI
TTD Cht monotherapy
TTD ICI
Results
AE
, adverse
event;
ALK
,
anaplastic lymphoma
kinase;
ChT,
chemotherapy;
EGFR
, epidermal growth factor receptor; FDA, Food and Drug Administration; ICI, immune checkpoint inhibitor; maint., maintenance; mPFS, median PFS; mTTD, median TTD; NSCLC, non-small cell lung cancer; OS, overall survival; PFS, progression-free survival; RCT, randomized controlled trial; RECIST, Response Evaluation Criteria in Solid Tumors; TKI, tyrosine kinase inhibitor;
TTD, time to discontinuation.1. Blumenthal GM, et al. Ann Oncol 2019;30(5):830–8.
Slide21RCT,
randomized
controlled
trial; RWE, real-world evidence.1. Wong GWK, et al. Ann Am Thorac Soc 2014;11(Suppl. 2):S85–91; 2. Nallamothu BK, et al. Circulation 2008;118(12):1294–303; 3. Roche N, et al. Ann Am Thorac Soc 2014;11(Suppl. 2):S99–S104
; 4. Karim S, Booth CM. J Clin Oncol 2019:37(13):1047–50; 5. Lakdawalla DN, et al. Value Health 2017;20(7):866–75. RWE summaryBackground: Real-world studies complement RCTs – they provide evidence from everyday patient populations managed in routine care settings, including those with characteristics that would preclude their participation in RCTs1–3 Real-world data sources: There is a wide range of real-world data sources available; each has specific strengths and limitationsReal-world data source examples include: electronic medical record databases (primary care ± secondary care ± pharmacy ± other linked data); insurance claims databases; patient access schemes; registries
Real-world studies vs RCTs:
1,3Conducted for different purposes, e.g. drug registration (RCTs) vs long-term surveillance (real-world studies) Differ according to how much their design reflects (i) real-world patients and (ii) real-world clinical management Different strengths and weaknesses, e.g. real-world studies offer high external validity, but low internal validityReal-world vs RCT findings:Often evaluate different outcomes; real-world studies make use of proxy measures where specific variables are unavailableRCTs may over-estimate treatment benefits through their selection of ‘ideal’ populations4
Real-world studies and RCTs provide complementary evidence; a judicious approach combining evidence from both can provide a more complete view of the evidence base1–5