Systematic reviews how we use - PDF document

Systematic reviews: how we use them in health care research and how they can be used in other disciplines Dr Mona Nasser DDS, MSc, PhD If,sometimessupposed,scienceconsistednothingbutthelaboriousaccumulationfacts,wouldsooncomestandstill,crushed,were,underitsownweightTwoprocessesarethusworksideside,thereceptionnewmaterialthedigestionassimilationtheTheworkwhichdeserves,butafraiddoesnotalwaysreceive,themostcreditthatwhichdiscoveryexplanationhand,whichnotonlyarenewfactspresented,buttheirrelationoldonespointedout(Rayleigh1885The

James Lind Library 4.2 Preparing and maintaining systematic reviews of all the relevant evidence https://www.jameslindlibrary.org/essays/4preparingandmaintainingsystematicreviewsofallthe relevantevidence/ ) "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials"Archie Cochrane, 1979 Systematicreviewsseekcollateevidencethatfitsprespecifiedeligibilitycriteriaorderanswerspecificresearchquesti

onTheyaimminimizebiasusingexplicit,systematicmethodsdocumentedadvancewithprotocolChandler J, CumpstonM, Thomas J, Higgins JPT, DeeksJJ, Clarke MJ. Chapter I: Introduction. In: Higgins JPT, Thomas J, Chandler J, CumpstonM, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook.Steps involved in conducting systematic reviewsDefine and Frame a review questionDeveloping and Running a search

strategy Data Screening and Data Extraction Quality Assessment of individual studiesSynthesis of the data (qualitative and/or quantitative synthesis)Judging the quality of the body of the evidenceDeveloping recommendation for future practice and research (Certain steps might involve consulting stakeholders) . expdentistry/2. expdental facilities/3. infection control, dental/4. expdentists/5. dental staff/6. expdental auxiliaries/7. (dental or dentist$ or hygienist$).8. ((oral or maxillofacial) adj5 (care$ or procedure$ or surg

ery or surgical or medicine)).9. orthodonti$.mp.10. periodont$.mp.11. (tooth or teeth or gum$ or endodont$ or plaque$ or pulpotom$ or pulpectom$ or "cavity prep$" or molar$ or bicuspid$ or premolar$ or premolar$ or incisor$ or canine$ or eyetoothor eyeteeth or cuspid$).12. ((scal$ adj2 polish$) or "root canal" or (root adj6 resect$) or (root$ adj3 planing) or apicectom$ or apicoectom$).13. ((root$ or periodont$ or dental or subgingiv$ or gingiv$ or supragingiv$) adj5 (scale or scaling or scaler$ or curettage)).14. Dental high

speed equipment/15. ("high speed air rotor$" or "low speed handpiece$" or "low speed hand piece$" or micromotor$ or "turbine handpiece$" or "electrosurgeryunit" or "air polisher$" or "prophyangle$" or "airwater syringe$" or "high speed hand piece$" or "high speed handpiece$" or "threeway air syringe$" or "threewayair syringe$" or "ultrasonic scaler$" or "hardtissue laser$" or "dental drill$" or "piezo unit$" or "piezo hand piece$" or "piezo handpiece$" or "rotary instrument$" or "air abrasion" or "water spray$").16. or/117. Ai

r microbiology/18. Air pollution, indoor/19. Aerosols/20. Inhalation exposure/21. (aerosol$ or bioaerosol$).22. (droplet$ or splatter$ or spatter$ or microbe$ or bacillus or germ$ or microorganism$ or virus$ or viral or coronavirus$ or COVID$ or "middle east? respiratory syndrome$" or MERS or MERSCoVor "camel flu" or SARS or "sudden acute respiratory syndrome$" or "Wuhan virus$" or 2019nCoV or SARSCoV2 or SARSCoVor SARSCoV1 or SARS1).23. (air adj5 (pollut$ or quality or impur$)).24. or/1725. Decontamination/26. ("high volume e

vacuat$" or HVE or "high volume aspirat$").27. Rubber dams/28. ((rubber adjdam$) or (oral adjdam$) or (dental adjdam$) or (latex adjdam$) or Kofferdam29. ("Optra Dam" or "OptraDamPlus" or OptiDamor FlexiDamor "HygenicFiesta").30. Suction/31. ("saliva ejector" or "low volume aspirat$" or (suction adj2 saliva)).32. Air filters/33. (air adj5 (filter$ or filtration or purif$ or clean$)).34. ((HEPA or "High Efficiency Particulate Air" or "High Efficiency Particulate Arrestance") adj5 filter$).35. Air ionization/36. (ionis$ or ioniz

$).37. Ozone/38. (ozonis$ or ozoniz$).39. Ultraviolet rays/40. (ultraviolet or UV or ultraviolet or actinic).41. ((aerosol$ or bioaerosol$ or droplet$ or spatter or splatter) adj2 reduc$).42. Fumigation/43. (fog$ or fumigat$ or decontaminat$ or "smoke out" or smokeoutor depollut$ or depurat$).44. or/2545. 16 and 24 and 44This subject search will be linked to the Cochrane Highly Sensitive Search Strategy (CHSSS) for identifying randomised trialsMEDLINE: sensitivitymaximising version (2008 revision) ( Lefebvre 2019 1. randomized

controlled trial.pt.2. controlled clinical trial.pt.3. randomized.ab4. placebo.ab5. drug therapy.fs6. randomly.ab7. trial.abnot humans.sh.11. 9 not 8. groups.ab9. or/110. expanimals 1. expdentistry/2. expdental facilities/3. infection control, dental/4. expdentists/5. dental staff/6. expdental auxiliaries/7. (dental or dentist$ or hygienist$).8. ((oral or maxillofacial) adj5 (care$ or procedure$ or surgery or surgical or medicine)).9. orthodonti$.mp.10. periodont$.mp.11. (tooth or teeth or gum$ or endodont$ or plaque$ or pulp

otom$ or pulpectom$ or "cavity prep$" or molar$ or bicuspid$ or premolar$ or premolar$ or incisor$ or canine$ or eyetoothor eyeteeth or cuspid$).12. ((scal$ adj2 polish$) or "root canal" or (root adj6 resect$) or (root$ adj3 planing) or apicectom$ or apicoectom$).13. ((root$ or periodont$ or dental or subgingiv$ or gingiv$ or supragingiv$) adj5 (scale or scaling or scaler$ or curettage)).14. Dental high speed equipment/15. ("high speed air rotor$" or "low speed handpiece$" or "low speed hand piece$" or micromotor$ or "turbine

handpiece$" or "electrosurgeryunit" or "air polisher$" or "prophyangle$" or "airwater syringe$" or "high speed hand piece$" or "high speed handpiece$" or "threeway air syringe$" or "threewayair syringe$" or "ultrasonic scaler$" or "hardtissue laser$" or "dental drill$" or "piezo unit$" or "piezo hand piece$" or "piezo handpiece$" or "rotary instrument$" or "air abrasion" or "water spray$").16. or/1 1. randomized controlled trial.pt.2. controlled clinical trial.pt.3. randomized.ab4. placebo.ab5. drug therapy.fs6. randomly.ab7.

trial.ab/ not humans.sh.11. 9 not 10.8. groups.ab9. or/110. expanimals We found 16 studies that involved a total of 425 people. Studies involved between one and 80 participants, who were aged between 5 and 69 years. Six studies were conducted in the USA, five in India, two in the UK and one each in Egypt, the Netherlands and the United Arab EmiratesThe studies evaluated one or more of the following devices highvolume evacuator (7 studies); handsfree suction device (2 studies); saliva ejector (1 study); rubber dam (3 studies);

rubber dam with a highvolume evacuator (1 study); or air cleaning system (1 studyNone of the studies evaluated the risk infectious disease transmission. Nor did they evaluate cost, acceptability or ease of implementation. Publication Bias We care both about sources of biases (systematic error) in individual studies but also in the body of the evidence. One well known bias is publication bias. We use empirical evidence in the form of metaepidemiological studies to inform our approach to assess the risk of bias in studies or rev

iews.Nonreporting biases lead to bias due to missing results in a systematic review. BoutronI, Page MJ, Higgins JPT, Altman DG, LundhA, HróbjartssonA. Chapter 7: Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, CumpstonM, Li T, Page MJ, Welch VA (editors).Cochrane Handbook for Systematic Reviews of Interventionsversion 6.1(updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook . Page MJ, Higgins JPT, Sterne JAC. Chapter 13: Assess

ing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, CumpstonM, Li T, Page MJ, Welch VA (editors).Cochrane Handbook for Systematic Reviews of Interventionsversion 6.1(updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook . Hosseini S, TurhanB, A Systematic Literature Review and Metaanalysis on Cross Project Defect Prediction. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. X, NO. Y, DECEMBER 2016. https://bura.brunel.ac.uk/bitstream/2438/15341/1/Fu

lltext.pdf KitchemanB. MadeyskiL, Brereton P, Metaanalysis for families of experiments in software engineering: a systematic review and reproducibility and validity assessment. Empirical Software Engineering. Issue 1/2020GrimstadS. Jorgensen M, MolokkenOstvoldKM. The Clients’ Impact on Effort Estimation Accuracy in Software Development Projects. 11th IEEE International Software Metrics Symposium (METRICS 2005). https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1509288 KitchenhamB, Pretorius R, BudgenD, et al. Sys

tematic literature reviews in software engineering A tertiary study. Information and Software Technology. Volume 52, Issue 8, August 2010, Pages 792 Systematic mapping/Evidence mapping/ScopingConfigurative versus aggregating resultsEmerging concepts versus estimates of magnitudes and precision Metaanalysis (and other quantativesynthesis methods)Thematic AnalysisNarrative synthesisNetwork metaanalysisUmbrella reviews or overview of reviewsRealist reviewMixed methods reviewMetanarrativesMetaetnographyRapid review Littell, J.H. (

2018), Conceptual and practical classification of research reviews and other evidence synthesis products. Campbell Systematic Reviews, 14: 121. doi: 10.4073/cmdp.2018.1 Network Meta analysis “When multiple interventions have been used and compared for the same disease and outcomes, network metaanalysis (also commonly referred to as a multiple treatment comparison metaanalysis or mixed treatment metaanalysis) offers a set of methods to visualize and interpret the wider picture of the evidence and to understand the relative

merits of these multiple interventions” Mills EJ, ThorlundK, Ioannidis JP. Demystifying trial networks and network metaanalysis. BMJ. 2013 May 14;346:f2914. doi: 10.1136/bmj.f2914. PMID: 23674332. S. Ghanavati, D. Amyotand L. Peyton, "A systematic review of goaloriented requirements management frameworks for business process compliance,"Fourth International Workshop on Requirements Engineering and Law, Trento, 2011, pp. 25They did not use a quantativesynthesis to answer the question, they had a predefined criteria (categ

orised the question further into requirements engineering framework for managing compliance, goal modelling approach, business process compliance, legal requirements extraction, law complaint business process templates, legal requirements prioritisation, tool support, other legal compliance issues.They categorised data based on the framework and then summarised and compared the studies and data they identified specific areas that required further work in relation to prioritization to improve compliance, templates for generatin

g law compliant processes, general links between legal requirements, goal models, and business processes, and semiautomation of legal compliance and analysis Systematic mapping “Systematic mapping does not aim to answer a specific question as does a systematic review, but instead collates, describes and catalogues available evidence (e.g. primary, secondary, quantitative or qualitative) relating to a topic of interest. The included studies can be used to develop a greater understanding of concepts, identify evidence for p

olicyrelevant questions, knowledge gaps (topics that are underrepresented in the literature that would benefit from primary research), and knowledge clusters (subsets of evidence that may be suitable for secondary research, for example using systematic review).”James, K.L., Randall, N.P. & Haddaway, N.R. A methodology for systematic mapping in environmental sciences.Environ Evid7 (2016). https://doi.org/10.1186/s13750 Cheng, S.H., MacLeod, K., Ahlrothet al.A systematic map of evidence on the contribution of forests to pov

erty alleviation.Environ Evid(2019). https://doi.org/10.1186/s13750 Configurative versus Aggregative reviews Focus on aggregating or adding up data versus focus on configuring data from the included studies They address different questions but it is more a continuum (not a dichotomy) Example: Do students in classes where there is a classroom teaching assistant get higher or lower scores on test scores? VERSUS How can we conceptualise the way that the presence of classroom assistants changes relationships between students and

teachers and between teachers in class? Gough D, Oliver S, Thomas J, Introducing systematic reviews. Sage Publicadtions The first part of the PhD was a systematic review focused on the question ‘How do referrals operate within the UK primary dental care setting?’. It was a configurative systematic that used critical interpretive synthesis. Overview of reviews Overviews are known by a variety of different names, all potentially reflecting different aspects and aims of the syntheses. Terms used include: overview; um

brella review; metareview; (systematic) review of (systematic) reviews; synthesis of systematic reviews; and summary of systematic reviews. The common feature of the methods associated with all of these terms is the fundamental process of synthesising evidence which is derived, often exclusively, from systematic reviews. The systematic review forms the primary ‘unit of analysis’ and is the basis upon which an overview is builtHunt, H., Pollock, A., Campbell, P. et al. An introduction to overviews of reviews: planning

a relevant research question and objective for an overview. Syst Rev 7, 39 (2018). https://doi.org/10.1186/s13643 What do we know about evidence - informed priority setting processes to set a population - level health - research agenda: An overview of reviews Audrey tan, Sumanth Kumbargere, Mona Nasser, Tarang Sharma, Tanja Kuchenmuller ObjectiveThis overview aimed to synthesize existing systematic reviews to produce a draft framework of evidenceinformed health priority setting processes that can be considered by WHO Europ

ean Region countries to support national research agendas and highlight guidance to choose appropriate methodologies for conducting such exercises. Study design and settingWe searched Ovid MEDLINE® and the WHO Institutional Repository for Information Sharing from 20102020 for critical or systematic reviews that evaluated research priority setting exercises. We adapted the AMSTAR checklist to assess the quality of included reviews and used adapted frameworks for data extraction and analysis. ResultsThe search resulted in 1918

titles, of which 26 were synthesized. All of the included systematic reviews, were of low quality. We categorized the included studies into three broad categories based on the primary focus of the review: engaging with stakeholders; methods; context; or health area. We appraised and synthesized these prioritysetting reviews to propose a draft evidenceinformed framework for countries to consider while developing their national research agendas.ConclusionThe draft framework could support countries in answering questions that are

of value to society and whose outcomes inform policies that improve the health and wellbeing of populations. What’s the future Living systematic reviews or living maps Rapid reviewsthinking the evidence ecosystem or evidence universe Evidence synthesis 2.0 model Evidence for future research? Cummulativemetaanalysis showing proven benefit of aprotininon need for blood transfusion during cardiac surgery (Fergusson et al 2005) Are applicants who seek support for new research required to refer to systematic reviews of exi

sting evidence?Funding AgenciesCountryNational Institute for Health Research (NIHR)UK/EnglandPatientCenteredOutcomes Research Institute (PCORI)USAMedical Research Council (MRC)Deutsche Forschungsgemeinschaft(DFG)GermanyThe Netherlands Organisation for Health Research and Development (ZonMwNetherlandsCanadian Institutes of Health Research (CIHR) CanadaFrench Ministry of Health (FMoH) Francel’Agence Nationale de la Recherche (ANR)FranceDanske Regioner(DR )DenmarkRegional Health Authorities (RHA )NorwayNational Health and M

edical Research Council (NHMRC)AustraliaNational Institutes of Health (NIH)USA Nasser M, Clarke M, Chalmers I, BrurbergKG, NykvistH, Lund H, Glasziou P. What are funders doing to minimise waste in research? Lancet. 2017 Mar 11;389(10073):10061007. doi: 10.1016/S01406736(17)306578. Nasser , Clarke M, Chalmers I, BrurbergKG, NykvistH, Lund H, Glasziou P. How can research funders add value to research as part of a special session on responsible research conduct for funding agencies. Work Research integrity conference, Amsterdam,