Pharmacy Practice Jarrod King Pharm D PGY1 UAMS Managed Care Resident Ref Bastian Hilda Statistical Significance and Its Part in Science Downfalls Bias Science Communication ID: 1009840
Download Presentation The PPT/PDF document "Evidence Based Medicine In" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Evidence Based Medicine In Pharmacy PracticeJarrod King, Pharm. D. PGY1 UAMS Managed Care Resident
2. Ref: Bastian, Hilda. “Statistical Significance and Its Part in Science Downfalls.” Bias, Science Communication. (2013) http://blogs.plos.org/absolutely-maybe/2013/11/11/statistical-significance-and-its-part-in-science-downfalls/
3. Pharmacist Objectives:1. Understand the medical literature hierarchy.2. Describe a p-value and a 95% confidence interval and tell how they are similar and how they are different. 3. Calculate a number needed to treat (NNT) and number needed to harm (NNH) from a clinical trial and describe what the result means. 4. Differentiate between statistical and clinical significance and provide an example. 5. Give an example of disease-oriented evidence that DID and DID NOT translate to a meaningful clinical outcome in a trial.
4. Technician Objectives:1. Explain how to calculate a number needed to treat (NNT) and number needed to harm (NNH) from a clinical trial and discuss what the result means. 2. Compare and contrast statistical and clinical significance and provide an example. 3. Discuss examples of disease-oriented evidence that DID and DID NOT translate to a meaningful clinical outcome in a trial.
5. The Medical Hierarchy of Evidence
6. Case ReportsCase seriesCohort studiesNon-randomized controlledTrialsRandomized, controlled trialsMeta-analyses Systematic ReviewsCase Control Study
7. .Experimental vs Observational Studies
8. Systematic Review and Meta AnalysisFigure: Olopatadine vs ketotifen. Itching at 14 days.Castillo, Mayret, et al. "Topical antihistamines and mast cell stabilisers for treating seasonal and perennial allergic conjunctivitis." The Cochrane Library(2015).
9. Question 1:Which of the following statements are false?Meta analyses provide more statistical power than individual trials alone.Randomized controlled trials always hold more statistical power over cohort and case-controlled studies.Cohort studies compare outcomes between groups that share a common characteristic (drug exposure) to those without the characteristic by following them over time.Case Control studies look back retrospectively at people with or without an outcome (disease) to compare exposure to a risk factor.
10. Reporting the Results
11. There are a number of ways to report clinical trial data:1. P - value2. Confidence interval (CI)3. Risk ratio (RR)4. Hazard ratio (HR)5. Odds ratio (OR)6. Absolute risk reduction (ARR)7. Number needed to treat (NNT)8. Number needed to harm (NNH)Ref: Bastian, Hilda. “Statistically Funny Commenting on the science of unbiased health research with cartoons.” (2015) http://statistically-funny.blogspot.com/2015/07/arr-or-nnt-whats-your-number-needed-to.html
12. P - valueProbability that the results found in a trial were by chance.Typically 0.05. Also stated, there is < 5% probability that the result occurred by chance.Provides no information about size of the observed effect or clinical significance.P = 0.01 vs p = 0.00001.Tiotropium + budesonide/formoterol(n=287)Tiotropium Monotherapy(n=286)DifferenceP valueMean Post dose change in FEV1 at 60 min (%)16.4%7.5%8.6(6.2, 11.1)<0.0001Lee, Sang‐Do, et al. "Efficacy and tolerability of budesonide/formoterol added to tiotropium compared with tiotropium alone in patients with severe or very severe COPD: a randomized, multicentre study in East Asia." Respirology21.1 (2016): 119-127.
13. Confidence IntervalRange of values that have a given probability of encompassing the “true value.”CI provide statistical significance, an estimation of effect size, and variability.Difference data (mean diff, % diff) = statistically significant if the CI DOES NOT cross 0 (0.001 – 3.153).Ratio data (HR, RR, OR) = statistically significant if the CI DOES NOT cross 1 (0.34 – 0.85).Adjusted RateRate Ratio95% CI, (p value)Budesonide + formoterol inhalerFluticasone + salmeterol inhalerCOPD exacerbation rate0.880.861.020.96 – 1.09, (0.5637)Ref: Kern, David M., et al. "Comparative effectiveness of budesonide/formoterol combination and fluticasone/salmeterol combination among chronic obstructive pulmonary disease patients new to controller treatment: a US administrative claims database study." Respiratory research 16.1 (2015): 52.
14. Confidence IntervalsFigure: Olopatadine vs ketotifen. Itching at 14 days.
15. Risk Ratio or Relative Risk (RR)Probability of an unfavorable event occurring BY THE END OF A CLINICAL TRIAL in the tx grp vs. control grp.Major shortcoming: overstatement of treatment value (look at absolute difference too).Calculations:Risk = # of subjects w/ unfavorable event# of subjects in that armRelative Risk = risk in tx arm risk in control armInterpretation: RR = 1 = no difference in riskRR < 1 fewer events occurring in tx grp.RR > 1 more events occurring in tx grp.Beta Blocker(n = 1,000)Placebo(n=1,000)Relative Risk# w/ HF exacerbation (%)160 (16%)200 (20%)0.8 # of Deaths (%)15 (1.5%)37 (3.7%)0.41
16. Relative Risk Reduction (RRR)Quantifies how much the risk is reduced in the tx group vs. control group.1-RRRR = 0.93 means tx subjects were 93% or 0.93 times AS LIKELY to experience an event compared to controlRR = 0.93RRR = 1-0.93 = 0.07Means tx subjects were 7% LESS likely to experience an event compared to control.Again, neither RR or RRR express importance or magnitude of the treatment effect. Can overexpress tx results.
17. Hazard Ratio (HR)Calculated and interpreted (numerically) the same as RR Hazard rate in tx grp Hazard rate in control grpUsed for survival or time-to-event analysis.Often expressed with Kaplan-Meier curves.RR = relative likelihood of an event occurring at the end of the trial.HR = relative likelihood of an event occurring at any given time during a clinical trial. Weighted RR over the entire study.
18. Odds Ratio (OR)Probability of the event occurring compared to probability it will not.For Case Control studies, typically only result calculated.RR is the probability of someone w/o event developing the event.Beta Blocker(n = 1,000)Placebo(n=1,000)Relative RiskOdds Ratio# w/ HF exacerbation160 (16%)A200 (20%)C0.8 0.76# w/o HF exacerbation840 (84%)B800 (80%)DCalculations: OR = A * D B * COdds of exacerbation w/ BB Odds of exacerbation w/ placebo
19. Absolute Risk Reduction (ARR)Beta Blocker(n =1,000)Placebo(n=1,000)Relative RiskOdds RatioAbsolute Risk Reduction# w/ HF exacerbation (%)160 (16%)200 (20%)0.8 (80%)0.76 (76%)4%Difference in event rate between control and treatment groups.(% events in control) – (% events in treatment)For every 100 patients treated, 4 fewer will experience HF progression.
20. Usual insulinIntensive InsulinRRARRTrial 10.00096%0.00028%71%0.00068%Trial 29.6%2.8%71%6.8%Trial 396%28%71%68%COMPARISON OF RR VS. ARROccurrence of neuropathy
21. NNT and NNHTx duration x 2 yearsBeta Blocker(n = 1,000)Placebo(n=1,000)Relative RiskOdds RatioAbsolute Risk Reduction# w/ HF exacerbation (%)160 (16%)200 (20%)0.8 (80%)0.76 (76%)4%NNT = 25# w/ bradycardia episode180 (18%)50 (5%) 13% NNH = 7NNT or NNH = 1 / ARR (expressed as decimal)NNT is always rounded up 1 / 0.04 = 25NNH is always rounded down. 1 / 0.13 = 7.7 = 7 Meaning, 25 people have to be treated with a beta blocker for 2 years to prevent 1 person from having a heart failure exacerbation.
22. Question 2:Which answer combinations are correct?The Risk Ratio can overexpress the treatment effect.Confidence Intervals provide statistical significance, estimation of effect size, and variance of the estimate.A p value can give a direct determination of treatment effect size.A NNT = 5 means for every patient treated with study medication, there is a reduction of 5 events.1 and 21,2 and 41,2,3 and 41 and 42 and 3
23. Statistical vs. Clinical Benefit and the Analysis of Surrogate Endpoints
24. Ref: Bastian, Hilda. “Statistical Significance and Its Part in Science Downfalls.” Bias, Science Communication. (2013) http://blogs.plos.org/absolutely-maybe/2013/11/11/statistical-significance-and-its-part-in-science-downfalls/
25. Disease-oriented evidenceSurrogate markerClinical outcomes that matterHCTZ controls hypertensionHCTZ decreases BPHCTZ decreases mortality in patients with HTNAtorvastatin treats hypercholesterolemiaAtorvastatin substantially reduces LDL-CAtorvastatin reducesHeart attack, stroke, mortalityAlendronate treats osteoporosisAlendronate improves BMDAlendronate decreases hip fx & vertebral fxMedications for diabetes lower blood sugarDecreased A1CReduced CV risk and mortalitySurrogate vs. Clinical Endpoints
26. Question 3:Which is a surrogate endpoint that may or may not translate to a clinical endpoint?Nonfatal myocardial infarctionOverall SurvivalStrokeHemoglobin A1cHip Fracture
27. Does a statistically significant reduction of a lipid panel result in a clinically significant reduction in CV events?
28. Lipid lowering drugs and cardiovascular outcomesThe ACC/AHA 2013 guideline concluded that non-statin therapies for cholesterol lowering are NOT RECOMMENDED unless statins are not tolerated.Specifically, ezetimibe was the agent of choice for additional add on therapy with statins.Fibrates have a minimal effect on the lipid profile.Can increase risk of myopathy, especially when given with ezetimibe or statins, and increase the risk of cholelithiasis. Only gained the “expert opinion” strength of recommendation for their place in therapy in patients with TG > 500 mg/dL.Decrease TG ~ 20-50%Increase HDL ~ 15%Decrease LDL ~ 5-20% (but can increase LDL when TG are high)Cannon, Christopher P., et al. "Ezetimibe added to statin therapy after acute coronary syndromes." New England Journal of Medicine 372.25 (2015): 2387-2397.Jacobson, Terry A., et al. "National lipid association recommendations for patient-centered management of dyslipidemia: part 1—full report." Journal of clinical lipidology 9.2 (2015): 129-169.Stone, Neil J., et al. "2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults." Circulation(2013): 01-cir.
29. Fenofibrates and cardiovascular outcomesThe ACCORD Lipid Trial - Effects of Combination Lipid Therapy in Type 2 Diabetes MellitusTreatment groups:simvastatin + fenofibrate 160 mg (adjusted according to GFR w/ MDRD).simvastatin + placebo.Similar average daily simvastatin dose between groups.Mean follow up – 4.7 yearsPrimary Outcome – first occurrence of major CV event including nonfatal MI, nonfatal stroke, or death from CV cause.Ref: ACCORD Study Group. Effects of combination lipid therapy in T2DM. N Engl J Med. 2010;362:1563-74.
30. RESULTS FROM ACCORD LIPID TRIALRef: ACCORD Study Group. Effects of combination lipid therapy in T2DM. N Engl J Med. 2010;362:1563-74.
31. Subgroup Analysis of Primary Endpoint:ACCORDLipid TrialRef: ACCORD Study Group. Effects of combination lipid therapy in T2DM. N Engl J Med. 2010;362:1563-74.
32. Fibrates and CV outcomes: a systemic review and meta analysisConclusions:Fibrates reduce risk of CV disease largely by preventing coronary events.There was no difference in all cause or cardiovascular mortality.“In high risk individuals with combined dyslipidemia, clinically meaningful reductions in risk could be achieved.”Jun M, Foote C, Lv J, Neal B, et al. Effects of fibrates on CV outcomes: a systematic review and meta-analysis. Lancet 2010:375:1875-84.
33. Figure: Effects of fibrates on the risk of coronary events.The following trials showed a statistically significant reduction in coronary events:TrialStudy DrugWHO CO-OP TrialClofibrateHelsinki heartGemfibrozilBECAITBezafibrateSENDCAPBezafibrateVA-HITGemfibrozil
34. Does an increase in progression free survival result in an increase in overall survival?
35. Progression Free Survival in OncologyProgression Free Survival (PFS): Common primary endpoint in chemotherapy clinical trials.Establishes the time after treatment of a disease until the patient’s disease progresses.Overall Survival (OS) Clinically important outcome.Commonly a secondary endpoint, because it takes longer to establish. PFS has become a surrogate for OS, but there are a number of examples when a significant increase in PFS did not correlate to a meaningful increase in OS.
36. Miller, Kathy, et al. "Paclitaxel plus bevacizumab versus paclitaxel alone for metastatic breast cancer." New England Journal of Medicine 357.26 (2007): 2666-2676.BEVACIZUMAB (AVASTIN) AND PFS IN METASTATIC BREAST CANCERResults: PFS - Paclitaxel plus bevacizumab significantly prolonged PFS compared with paclitaxel aloneMedian OS - similar in the group receiving combined therapy and in the group receiving paclitaxel alone
37. BEVACIZUMAB (AVASTIN) AND PFS IN METASTATIC BREAST CANCER
38. Avastin and PFS in Metastatic Breast CancerD'Agostino Sr, Ralph B. "Changing end points in breast-cancer drug approval—the Avastin story." New England Journal of Medicine 365.2 (2011): e2.
39. Avastin and PFS in Metastatic Breast CancerRef: Sasich LD, Sukkari SR. The US FDAs withdrawal of the breast cancer indication for Avastin (bevacizumab). Saudi Pharmaceutical Journal : SPJ. 2012;20(4):381-385. doi:10.1016/j.jsps.2011.12.001.Robert, Nicholas J., et al. "RIBBON-1: randomized, double-blind, placebo-controlled, phase III trial of chemotherapy with or without bevacizumab for first-line treatment of human epidermal growth factor receptor 2–negative, locally recurrent or metastatic breast cancer." Journal of Clinical Oncology29.10 (2011): 1252-1260.Pivot, Xavier, et al. "Efficacy and safety of bevacizumab in combination with docetaxel for the first-line treatment of elderly patients with locally recurrent or metastatic breast cancer: results from AVADO." European journal of cancer 47.16 (2011): 2387-2395.
40. Does a reduction in A1c result in a reduction in CV events?
41. Surrogate endpoints in Diabetes: sitagliptan (Januvia)Current standard for FDA approval of diabetes meds is reduction in A1c compared to placebo.sitagliptan (Januvia) consistently ranks in the top 25 selling drugs in the US.The overall least-squares mean difference in A1C during the median 3 year follow up was −0.29% in the sitagliptin group compared to placebo (95% confidence interval [CI], −0.32 to −0.27).Primary composite CV outcome occurred in 839 patients in the sitagliptin group (11.4%) and 851 in the placebo group (11.6%). Noninferiority of sitagliptan CAUSING the composite CV outcome vs placebo was established.Green, Jennifer B., et al. "Effect of sitagliptin on cardiovascular outcomes in type 2 diabetes." New England Journal of Medicine 373.3 (2015): 232-242.
42. Surrogate endpoints in Diabetes: DPP-4 InhibitorsOutcome/TrialsComparator - NResultsAll cause mortality/43Placebo – 31,228Active – 20,182RR = 1.04, 95% CI 0.93–1.16, P = 0.54RR = 0.65, 95% CI 0.39–1.09, P = 0.10CV mortality/40Placebo – 30,485Active – 18,094RR = 0.97, 95% CI 0.85–1.12, P = 0.70RR = 0.98, 95% CI 0.44–2.17, P = 0.95ACS outcomes/44Placebo -33,227Active – 20,235RR = 1.00, 95% CI 0.89–1.12, P = 0.98RR = 0.63, 95% CI 0.41–0.96, P = 0.03Stroke Outcomes/29Placebo – 28,187Active – 14,550RR = 1.05, 95% CI 0.86–1.28, P = 0.64RR = 0.48, 95% CI 0.26–0.89, P = 0.02Heart Failure Outcomes/24Placebo – 27,818Active – 12,563RR = 1.17, 95% CI 1.01–1.34, P = 0.03RR = 0.80, 95% CI 0.35–1.81, P = 0.59Summary: No statistically significant decrease compared to placebo or active for all cause mortality or mortality related to CV event. Statistically significant increase in Heart Failure outcomes compared to placebo.Ref: Wu, Shiying, et al. "Dipeptidyl peptidase‐4 inhibitors and cardiovascular outcomes: meta‐analysis of randomized clinical trials with 55,141 participants." Cardiovascular therapeutics 32.4 (2014): 147-158.
43. Final thoughtsA poorly conducted study does not necessarily have statistical power over a well done study lower on the medical literature hierarchy.There are a number of ways to display clinical trial data, each with specific shortcomings and parameters for use.A number of medications have gained FDA approval based off surrogate endpoints. After further testing, several medications, other than those described, failed to show a meaningful clinical benefit. EBM uses the best current evidence to guide treatment decisions. Some situations will never have adequate evidence at the top of the pyramid including,:Orphan diseasesCancer trials that allow crossoverAs healthcare professionals, we must challenge the utilization of medications without known clinical benefit as drug prices continue to inflate in order to create a more sustainable healthcare system.
44. QUESTIONS???