Learning Objective 5.2 Explain what is meant by “publication bias” and how it might - PowerPoint Presentation

Slide1

Learning Objective 5.2Explain what is meant by “publication bias” and how it might affect the response to emergencies

WHO Training Manual

Ethics in

epidemics

, emergencies and

disasters

: Research, surveillance and patient care

OutlineIntroductionReadingTamiflu videoCase study and discussion

Suggested time

0-20

(20 min)

21-40

(20 min)

41-55

(15 min)

56-85

(30 min)

86-90

(5 min)

Activity

Introduction

Reading

Video

and group discussion

Case study

and discussion

Summary and conclusion

BackgroundDecisions about how to prepare for and respond to public health emergencies are likely to be informed by evidence generated from research.What is published and therefore what makes up the evidence base upon which decisions are informed has the potential to be biased.

BackgroundBias means “[a]ny process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth” (Sackett, 1979, pg. 60). Publication bias refers to what occurs when what is published is systematically unrepresentative of what is available to be published (Rothstein et al., 2005).

BackgroundThe effect of publication bias could be that published studies do not accurately represent all data (in the case of publishing results from a single study) or all studies (in the case of publishing a systematic review or meta-analysis).Ultimately, the consequence of publication bias is the distortion of the evidence base (Eggar et al., 1995). If decisions are to be informed based on this evidence, those decisions may ultimately be based on incomplete, suppressed, or inaccurate information.

Forms of Publication BiasPositive results biasPositive results bias occurs when statistically significant positive results from a study are more likely to be published than results that support the null hypothesis or results that are inconclusive (i.e. ‘negative’ results) (Hopewell et al., 2009). Hot stuff bias / lack of interest biasThis bias occurs when a topic is popular (or not) within the scientific community or in the broader public domain, leading to an increased (or decreased) interest for publication on that topic

Forms of Publication BiasConfirmation biasCan exist when the results of research tend to match or support the interests, expectations, or hypothesis of the researcher or study sponsor (Mahoney, 1977).

Case StudyWatch “New doubts over Tamiflu”: http://www.channel4.com/news/articles/science_technology/new+doubts+over+tamiflu/3454737.html

Case StudyIn 2003, a paper was published reporting the results of a study sponsored by F. Hoffmann-La Roche Ltd. about the impact of oseltamivir (brand name: Tamiflu) treatment on influenza-related lower respiratory tract complications (LRTCs) and hospitalizations. The paper reported that oseltamivir treatment of influenza reduces LRTCs, antibiotic use, and hospitalizations both for at-risk and healthy adults (Kaiser et al., 2003).

Case StudyThis study involved the analysis of 10 separate phase three randomized control trials (RCTs) sponsored by Roche, of which only 2 have been published in peer-reviewed journals. In a subsequent Cochrane review it was claimed that there is insufficient evidence to answer the question about whether oseltamivir is effective in reducing LRTCs, antibiotic use, or hospitalization without including the data from the 8 unpublished studies included in the initial paper (Jefferson et al., 2009).

Case StudyIrrespective of this, the original study’s evidence has been used by public health decision-makers to justify recommending oseltamivir as a treatment option in combating influenza, including pandemic strains of influenza (Godlee and Clarke, 2009). As a result, this has also led to the stockpiling of oseltamivir for use during an influenza pandemic.The authors of the Cochrane review conclude, “[i]t is possible that there is a publication bias, especially as we know of eight trials that are unpublished and inaccessible…Its direction might be in favour of exaggerating the treatment effect” (Jefferson et al. 2009, pg. 6).

Case Study QuestionsCould this case study indicate a publication bias? Why or why not? What kind of publication bias might exist in this case?What could be done to prevent or mitigate the potential publication bias in this case?How could this case affect the response to a public health emergency?

Discussion questionsWhat is it that fundamentally drives publication? Is this motivation any different during a public health emergency?If there is questionable evidence of effectiveness for a public health intervention (for example, the use of quarantine in response to an infectious disease like severe acute respiratory syndrome), what does that mean in terms of the justifiability of using that public health intervention? See Bensimon and Upshur, 2007 for discussion on this topic.

Discussion questionsCould a publication bias occur for the publication of papers on the ethics of public health emergency preparedness and response? Could papers that come to conclusive answers about difficult ethical questions get published more easily or quickly?If the trustworthiness of the evidence base upon which decisions are informed has been diminished due to publication bias, how should decisions be made in light of this?Should peer-reviewers or journal editors (or some other body) have the ability to review study data themselves?

SummaryWhat is published (therefore what makes up evidence upon which decisions are informed) may be biasedIncreased appetite for studies to inform planning and response efforts during public health emergenciesMay affect quality, nature and even findings of studies, affecting what is publishedInterests of all stakeholders must be examined to try to mitigate biasData should also be publically available and evaluatedThere is a responsibility to work in solidarity in a public health emergency

SourcesBensimon CM, Upshur REG. Evidence and effectiveness in decisionmaking for quarantine. American Journal of Public Health, 2007, 97:S44–S48.Egger M, Smith GD. Misleading meta-analysis. British Medical Journal, 1995;310: 752-4.Godlee F, Clarke M. Why don’t we have all the evidence on oseltamivir? British Medical Journal, 2009; 339: b5351.Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin. Publication bias in clinical trials due to statistical significance or direction of trial results (review). The Cochrane Library, 2009;1: 1-26.Jefferson T, Jones M, Doshi P, Del Mar C. Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and metaanalysis. British Medical Journal, 2009; 339: b5106.Kaiser L,

Wat

C, Mills T, Mahoney P, Ward P, Hayden F. Impact of

oseltamivir

treatment on influenza-related lower respiratory tract complications and hospitalizations.

Archives of Internal Medicine

, 2003; 163: 1667-1672.

Mahoney

MJ. Publication prejudices: an experimental study of confirmatory bias in the peer review system.

Cognitive Therapy and Research

, 1977;1(2): 161-75.

Rothstein

HR, Sutton AJ,

Borenstein

M. Publication bias in meta-analysis. In Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments,

eds

Rothstein HR, Sutton AJ,

Borenstein

M. John Wiley & Sons, Ltd: Chichester, UK. 2006.

Sackett

DL. Bias in Analytic Research.

Journal of Chronic Diseases

, 1979;32: 51-63.

AcknowledgementsChapter authorSmith, Maxwell, Dalla Lana School of Public Health and Joint Centre for Bioethics, University of Toronto, Toronto, Ontario, Canada