Steps for Conducting Systematic Reviews - PowerPoint Presentation

1. Steps for Conducting Systematic ReviewsIrena Bond,Associate Professor/Library ManagerMassachusetts College of Pharmacy and Health Sciences (MCPHS)irena.bond@mcphs.eduJune 28, 2023

2. GoalsTo provide a step-by-step approach for properly conducting Systematic Reviews (SR) for beginners and junior researchersTo provide links to accepted and widely used international templates, checklists, and guidelines used in conducting SR

3. What is a SR?A systematic review is a review that uses a systematic method to summarize evidence on questions with a detailed and comprehensive plan of studyOpposite - Unsystematic narrative review tends to be descriptive, in which the authors select articles based on their point of view which leads to its bias and poorer quality

4. Why SR?Abundance of studies – clinical medicine is more complexStudies are often heterogeneous in their design, quality, subjects…A well-conducted SR is a feasible solution to keep up-to-date of current Evidence-Based Medicine (EBM)Understanding of SR steps is important for its conduction. It is not easy to be doneAll steps of SR should be done independently by 2–3 reviewers’ discussion, to ensure data quality and accuracy

5. Steps in Conducting SR (1) Development of research questionPreliminary research/Forming inclusion/exclusion criteriaSearch strategySearching databasesProtocol registrationTitle Abstract Full-text screeningManual searching Extracting dataQuality assessmentData checkingStatistical analysis (there are more steps and variety of meta analysis)Double data checkingManuscript writing and submission

6. Steps for conducting SR – Flow chart (1)Fig. 1 Detailed flow diagram guideline for systematic review and meta-analysis steps. Note: Star icon refers to “2–3 reviewers screen independently”

7. Research QuestionShould be feasible, interesting, novel, ethical, and relevant. It should be clear, logical, and well-definedPICO question framework (usually used for SR of clinical trial study) – Patient/Population, Intervention, Comparison, Outcome PICO for an observational study - use without intervention or comparator, in many cases it is usually enough to use P (Patient) and O (outcome) only or PEO-patient, exposure, outcome

8. ExampleWe want to know whether the Ebola vaccine is safe and can induce sufficient immunogenicity to the subjects.Question: How is the safety and immunogenicity of Ebola vaccine in human? P: healthy subjects (human)I: vaccinationC: placeboO: safety or adverse effects

9. Preliminary Research and Idea Validation (1)Start by doing a simple search in PubMed or Google Scholar with search terms Ebola AND vaccineWhy? to identify relevant articles, ensure the validity of the proposed idea, avoid duplication of previously addressed questions, and assure that we have enough articles for conducting its analysisWe find a SR of determinant factors influencing antibody response from vaccination of Ebola vaccine in non-human primate and human, which is a relevant paper to read to get a deeper insight and identify gaps for better formulation of our research question or purposeWe can still conduct systematic review and meta-analysis of Ebola vaccine because we evaluate safety as a different outcome and different population (only human)

10. Inclusion and Exclusion CriteriaEligibility criteria - based on the PICO approach, study design, and date. Exclusion criteria - unrelated, duplicated, unavailable full texts, or abstract-only papers. Exclusions should be stated in advance to refrain the researcher from biasInclusion criteria - articles with the target patients, investigated interventions, or the comparison between two studied interventions. They contain information answering our research question. Most importantly there should be clear and sufficient information, including positive or negative, to answer the question

11. Example – Inclusion & Exclusion CriteriaInclusion Criteria:any clinical trial evaluating the safety of Ebola vaccine no restriction regarding country, patient age, race, gender, publication language, and dateExclusion criteria:study of Ebola vaccine in non-human subjects or in vitro studiesStudy with data not reliably extracted, duplicate, or overlapping dataabstract-only papers as preceding papers, conference, editorial, and author response theses and booksarticles without available full text available;case reports, case series, and systematic review studies.

12. Search StrategyA standard search strategy is used in PubMed first Later the PubMed search strategy is modified according to each specific database to get the best relevant resultsThe basic search strategy is built based on the research question formulation (i.e.,PICO). Search strategies are constructed to include free-text terms (e.g., in the title and abstract) and any appropriate subject indexing (e.g., MeSH) expected to retrieve eligible studies, with the help of an expert in the review topic field or an information specialistOutcome terms are usually not searched because outcome is not mentioned obviously in the articles The improvement of the search term is made while doing a trial search and looking for another relevant term within each concept from retrieved papersA detailed search strategy for each database is developed next

13. Example:The final search strategy for PubMed is as follows: (ebola OR ebola virus OR ebolavirus OR ebola virus disease OR EVD) AND (vaccine OR vaccination OR vaccinated OR immunization) AND (“clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH Terms] OR “clinical trial”[All Fields])Because the study for this topic is limited, we do not include outcome term (safety and immunogenicity) in the search term to capture more studies.

14. PRISMA Flow Diagram Fig. 2 PRISMA flow diagram of studies’ screening and selection

15. Search databases, import all results to a library, and export to an excel sheetAccording to the AMSTAR guidelines, at least two databases have to be searched in the SR, but as you increase the number of searched databases, you get much yield and more accurate and comprehensive resultsThe ordering of the databases depends mostly on the review questions; being in a study of clinical trials, you will rely mostly on Cochrane, mRCTs, or International Clinical Trials Registry Platform (ICTRP)Proposed databases for our topic: PubMed, Scopus, Web of Science, EMBASE, GHL, VHL, Cochrane, Google Scholar, Clinical trials.gov, mRCTs, POPLINE, and SIGLE (grey literature)Finally, all records are collected into one Reference Manager (e.g. Zotero, Mendeley, etc. ) library in order to delete duplicates and then to export records into an excel sheet with essential information for screening

16. Google Scholar Advanced Search Example1. With all of the words: ebola virus With at least one of the words: vaccine vaccination vaccinated immunization Where my words occur: in the title of the article2. With all of the words: EVD With at least one of the words: vaccine vaccination vaccinated immunization Where my words occur: in the title of the article

17. Protocol Writing and RegistrationProtocols describe the rationale, hypothesis, and planned methods of the project before the review has begun and allow readers of completed reviews to identify deviations from planned methods and determine whether the bias impacts the interpretation of review results and conclusions (e.g. criteria you will use to screen literature, how you will search for literature, QA tools you chose etc.)Protocol registration is important at an early stage It guarantees transparency in the research process, feedback from other researchers, and protects from duplication problemsIt is considered a documented proof of team plan of action, research question, eligibility criteria, intervention/exposure, quality assessment, and pre-analysis plan It is recommended that researchers send it to the principal investigator (PI) to revise it, then upload it to registry sitesThere are many registry sites available for SR like by Cochrane and Campbell collaborations; PROSPERO is recommended as it is easier. There is a PROSPERO (https://www.crd.york.ac.uk/prospero/) protocol template (1)

18. Title and Abstract ScreeningDecisions to select retrieved articles for further assessment are based on eligibility criteria, to minimize the chance of including non-relevant articlesWhen there is a doubt about an article decision, the team should be inclusive rather than exclusive, until the main leader or PI makes a decision after discussion and consensusAll excluded records should be given exclusion reasons

19. Full text Downloading and ScreeningReserach4Life ResourcesMany search engines provide links for free to access full text articles In case not found, we can search in some research websites as Research Gate, which offer an option of direct full-text request from authors.Contact your librarian to explore other options

20. Manual searchSearching reference lists from included studies/reviewsContacting authors and expertsLooking at cited articles in PubMed and Google Scholar (citation tracking- track all the articles that cite each one of the included articles)Looking at relevant (similar) articles All the possible relevant article must undergo further scrutiny against the inclusion criteria, after following the same records yielded from electronic databases, i.e., title/ab abstract and full-text screeningEach reviewer has a “tag” and a distinct method to compile all the results at the end for comparison of differences and discussion and to maximize the retrieval and minimize the biasThe number of included articles has to be stated before addition to the overall included records

21. Data Extraction and Quality AssessmentCollect data from included full-texts in a structured extraction excel sheetData presented in graphs is extracted by software tools, e.g. Web plot digitizer (1)The sheet is classified into the study and patient characteristics, outcomes, and quality assessment (QA) In the EBOLA example, extract: name of authors, country of patients, year of publication, study design (case report, cohort study, or clinical trial or RCT), sample size, the infected point of time after EBOLA infection, follow-up interval after vaccination time, efficacy, safety, adverse effects after vaccinations, and QA sheet To evaluate the risk of bias, reviewers should rate each of the 14 items into dichotomous variables: yes, no, or not applicable (NA). An overall score is calculated by adding all the items scores as yes=1, while no and NA = 0. A score will be given for every paper to classify them as poor, fair, or good, where a score from 0–5 was considered poor, 6–9 as fair, and 10–14 as goodQA tools (relevance, reliability, validity, applicability) (4). Variety of tools are available for the QA, depending on the design (1)

22. Data CheckingEvery included article is compared with its counterpart in an extraction sheet, to detect mistakes in data2–3 independent reviewers should do the data checking ideally not the ones who performed the extraction of those articles. When resources are limited, each reviewer is assigned a different article than the one he extracted in the previous stage

23. Statistical analysisInvestigators use different methods for combining and summarizing findings of included studies. Before analysis, there is an important step called cleaning of data in the extraction sheet, where the analyst organizes extraction sheet data in a form that can be read by analytical software The analysis consists of 2 types namely qualitative and quantitative analysis. Qualitative analysis mostly describes data in SR studies, while quantitative analysis consists of two main types: MA and network meta-analysis (NMA) (1)

24. Double Data CheckingFor more assurance on the quality of results, the analyzed data should be rechecked from full-text data by evidence photos, to allow an obvious check for the PI of the study.

25. Manuscript writing, revision, and submission to a journalWriting based on four scientific sections: introduction, methods, results, and discussion, conclusions.Having a characteristic table for study and patient characteristics is a mandatory step (see template in paper) After finishing the manuscript writing, characteristics table, and PRISMA flow diagram, the team should send it to the PI to revise it well and reply to his commentsFinally, choose a suitable journal for the manuscript which fits with considerable impact factor and fitting field. Pay attention by reading the author guidelines of journals before submitting the manuscript

26. References: 1. Tawfik, G.M., Dila, K.A.S., Mohamed, M.Y.F. et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health 47, 46 (2019). doi.org/10.1186/s41182-0190165-62. UNC Libraries. Forming Focused Questions with PICO and other Question Frameworks. Available at: https://guides.lib.unc.edu/pico/about 3. Morgan RL, Whaley P, Thayer KA, Schünemann HJ. Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environ Int. 2018;121(Pt 1):1027-1031. doi:10.1016/j.envint.2018.07.015 4. UNC Libraries. Systematic Reviews: Step 6: Assess Quality of Included Studies. Available at: https://guides.lib.unc.edu/systematic-reviews/assess-quality 5. Carnegie Melon University. Free Systematic Review and Mata analysis Course. Available at: https://oli.cmu.edu/courses/systematic-reviews-and-meta-analysis-o-f/ 6. Cochrane. Students 4 Best Evidence. Available at: https://s4be.cochrane.org/

27.