/
Introduction to Systematic Reviews Introduction to Systematic Reviews

Introduction to Systematic Reviews - PowerPoint Presentation

debby-jeon
debby-jeon . @debby-jeon
Follow
515 views
Uploaded On 2017-09-06

Introduction to Systematic Reviews - PPT Presentation

Dawn Craig Institute of Health amp Society dawncraignclacuk dawncraig Aims of this session Outline what a systematic review is To discuss scope and the formulation of a review question ID: 585646

review data systematic studies data review studies systematic question information reviews findings extraction study health quality results appraisal research evidence protocol synthesis

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Introduction to Systematic Reviews" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Introduction to Systematic Reviews

Dawn Craig

Institute of Health &

Society

dawn.craig@ncl.ac.uk

@

dawn_craigSlide2

Aims of this session….

Outline what a systematic review is

To discuss scope and the formulation of a review question

The purpose and format of a protocol

Systematic searching/screening of

studies

Data extraction/quality appraisal and intro to evidence synthesisSlide3

What is a systematic review?

SYSTEMATIC

: Done or acting according to a fixed plan or system: methodical

REVIEW:

A critical appraisal of a book, play or other workSlide4

What is a systematic review?

“A

systematic review

is a review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesized according to a predetermined and explicit method.”*

(*Klassen et al. Guides for reading and interpreting systematic reviews. Arch

Pediatr

Adolesc

Med 1998;152:700-704.)

A

systematic review

attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made

(

Antman

1992,

Oxman

1993)Slide5

Why we need systematic reviews

Minimise the impact of bias/errors

Can help to end confusion

Highlight where there is not sufficient evidence

Combining findings from different studies can highlight new findings

Can mitigate the need for further trialsSlide6

Why we need systematic reviews

Facilitate

rational decision making

Health care providers, researchers and policy makers are inundated with unmanageable amounts of information

Over 20 million citations in PubMed

Approx. 75 to 100 RCTs published daily

Usually impossible to consider all relevant individual primary research studies in a decision making context

Enable practitioners to keep up to date with evidence accumulating in field and to practice evidence-based medicineSlide7

Why not traditional

reviews

Un

scientific’

rarely pre-specify or make methods explicit

Rarely transparent or reproducible

Usually

qualitative

, subjective,

opinions of individual

Of

ten incomplete

,

filing cabinet

or MEDLINE

review

Difficult

to make sense across groups of studies, especially when conflicting based on qualitative reading aloneSlide8

Hierarchy of evidenceSlide9

Who undertakes systematic reviews?

Cochrane/Campbell Collaboration

NICE/Regulatory bodies

Health Technology Assessment

Academics/researchers/Clinicians

MSc/PhD studentsSlide10

Who undertakes systematic reviews?

Multidisciplinary teams

Clinicians

Health services researchers

Information scientists

Statisticians

Health Economists

Patient and public involvement – particularly for guidelinesSlide11

Key Stages in a Systematic Review- the process

Define research/review question

In consultation/collaboration with the clinical community, commissioners and patient/public representatives

Identify relevant studies

Develop a comprehensive search strategy and undertake systematic searches of the literature

Assess eligibility

Select those studies which meet the pre-defined inclusion criteria

Data extraction /checking

Develop data extraction from into which study

information

and

outcome data

can be extracted, checked

& verified

Synthesis

Narratively and/or statistically summarise/describe the data, exploring similarities

and differences

between studies.

Develop review protocol

Pre-specify

the type

of studies to be included

, the

methods of

collating, appraising and analysing data

Knowledge translation

Review details and results are disseminated to relevant target audiences using appropriate formats

Study assessment/appraisal

Assess the quality

and

validity of the included studies using the pre-defined method. Slide12

Define research/review question

Questions may be broad or narrow

Well-formulated questions will guide many aspects of the review process

Searching strategy

Inclusion/exclusion criteria

Data extraction

Choice of synthesis method

Presentation/dissemination of findingsSlide13

Quick Activity

Discuss a very broad question and how you might narrow it? (10 mins)

Discuss the potential limitations of your review questions

If time and resource were not a limitation – consider how useful would the answer to your review question be?Slide14

Current guidance

a clear and concise statement of a review's objectives (or questions) is critical and should begin with a precise statement of the primary objective, including the interventions reviewed and the targeted problem; ideally, this would be presented in a single sentence

Cochrane &

Prisma

StatmentSlide15

Current guidance

“To assess the effects of [

intervention or comparison

] for [

health problem

] in [

types of people, disease or problem, and setting if specified

].”

Several criteria/frameworks proposed to help guide question

developmentSlide16

Question formulation

Determining

the scope

is

a decision dependent upon multiple

factors:

P

erspectives

regarding a question’s relevance and potential impact;

Supporting

theoretical, biologic and epidemiological information

;

The

potential generalizability and validity of answers to the

questions;

Available resources;

The wider literature base – has a recent high-quality SR been conducted?Slide17

Question formulation

Advantages

and disadvantages to both broad and narrow

questions

The

validity of very

broad question may

be criticized for ‘mixing apples and

pears’; but advantages might include

Comprehensive summary of the

evidence

Generalizability

of findings

Most obvious advantage of narrow focus is clarity of objectives and ease of reading; but disadvantages might include

Sparse

evidence may limit findings/usefulness

Generalizability of findingsSlide18

Question formulation

Often dealing with complex interventions

Might be a need to develop working definitions of the intervention of interest

Several options on how to do this (pragmatic real world v theoretical, logic models, etc.)

Use content experts outside the review team to ensure that the resulting definitions are likely to be robust and meaningfulSlide19

Protocol Development

A protocol is an essential component of the systematic review process

Helps to ensure careful

a priori

planning

Consistency

Transparency

Integrity

Integral part of the process for leading organisations/publication processSlide20

Protocol Development

One of the features that distinguish a systematic review from a narrative review is the pre-specification of criteria

Inclusion

Exclusion

Methods

Outcomes to be synthesised

Etc.Slide21

PROSPERO – CRD initiative

Search for existing current reviews

Register their planned review online

Publish protocol online

Update record on Prospero website as the review progresses

Avoids duplication of reviewsSlide22

Searching for Information

Types of Studies (RCTs, non-RCTs, cohort/case-controlled)

Population and setting

Interventions

Outcome measures

Cochrane Handbook and CRD Guidelines

Both provide explanations re the difference study designs, likely biases and issues to consider when including them

www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf http://handbook.cochrane.org/Slide23

Searching for Information

MESH terms and key words/synonyms Medical Subject Heading – controlled vocabulary thesaurus used for indexing articles

young;

adoles

*; teen*; child*...................

*

end of the ‘stem’ of the word it will automatically search for all the endings for that word stem

Child* will also return children, childbearing, childbirth and so on…Slide24

Searching for Information

Word variantsAIDSacquired immunodeficiency syndromeacquired immuno-deficiency syndrome

acquired immune deficiency syndrome

acquired immune-deficiency syndrome

Synonyms

e.g. Newborn: infant, toddler, baby, etc.

Plurals

e.g. child : children OR teenager : teenagersSpelling variants (UK vs US) e.g. randomise/randomizeSlide25

Searching for Information

Where to searchElectronic databases: Medline, Embase, Cochrane, PsycInfo, etc.

Grey literature, dissertations, theses, conference proceedings, national bodies (NICE, HTA), clinical trial database (

www.clincialtrails.gov/

)

Look at the databases own guidance for searching they vary!Slide26

Boolean operatorsSlide27

Searches in medline

2006-2010 database

Alcohol – subject (all subheadings)

5565

Alcohol – subject or keyword

12052

([young or

adoles

* or teen*] [review or system*]

19180

Rows above combined with

or

6791Slide28

More specific – get help!Slide29

Selection of Studies

Reference manager software packageEndnote – RefMan – ProCite – MendeleyImport results and screenAssess titles/abstracts against your predetermined criteria

If in doubt include

Retrieve full text articles of initial selections

Assess full text for inclusion

Requires judgement (>1 reviewer)

Check reviewer agreement (3

rd review to resolve)Use a selection form to ensure consistency and record decisionsSlide30

Data

Extraction/Quality AppraisalSlide31

Data Extraction

Be clear what information you want

from the

studies:

Study details

Data for

your analysis

Information will need to be collected relating to:

Methodology

Population

Interventions being compared

Outcomes evaluated Slide32

Give consideration to….

What effect measures

you are

you going to

calculate

What data do you need to do this?

How

are

you planning to group studies for

the analysis?

By intervention?

By study design?

What

information do you need to

extract to enable you to organise

and analyse

the

way you want

?

REMEMBER YOUR PROTOCOL – IT IS YOUR ROADMAP, FOLLOW IT!Slide33

How much to extract??

Level of judgement is required

Sufficient to describe studies

Sufficient to allow you to undertake the planned analysis

Sufficient so you do not need to return to the full text papers

However

You need to limit unnecessary detailSlide34

Data extraction software?

There is a wide selection of software to choose from

Selection depends on a number of factors

Main

considerations are

probably

What are you

are

familiar with?

What package best suits your data?

How many included studies do you have?Slide35

Which software?

Word

Excel

Access

EPPI reviewer

COEVIDENCE

REVMAN

????Slide36

Consistency/Standardisation

We all have to be doing the same thing

Essential >one

reviewer is extracting

data

Data must be interpreted in the same way by all reviewers

Independent piloting of

data

extraction

forms –

always

one standardised form

Regular discussion of progress/disagreements

Regular comparison of

data

extraction – don’t wait till the endSlide37

Efficient data extraction

Once data extraction is complete you may need to:

Sort/search

your data

Filter data

Calculate frequencies

Transform data (e.g. SE to SD)

Categorising/coding data will make these tasks

easier

Needs to be implemented with consistency by the whole team

A database can be designed to have this

functionalitySlide38

Things to consider

Are you including more

than one study

design?

You may need separate forms for each study design

However, you

are still answering the same

question, so make

sure the core information extracted is the same

Have one or a few studies reported data differently from the others?

Will the data still be

useful?

Should you include it?

Make sure the core information extracted is the same

You

may

need to update the form, or have more than one

form

Any changes need to be agreed and made consistentlySlide39

Stay on track……

Be careful about collecting ‘extra’

data

It is very tempting

to collect data that are not directly relevant to the review question

The data needed to answer the review question should have already been

decided (REMEMBER YOUR PROTOCOL)

Collect data for good reasons – stay focused and don’t get side-tracked

Time and

effort to collect, only to find it is not usefulSlide40

Quality Assessment & Critical Appraisal

Why bother????

What are we trying to achieve?

Not all published and unpublished literature is

rigorous!

being in a journal doesn’t mean it is good

Quality

may be used as an explanation for differences in study results

or to guide interpretation

of

findings, strength

of inferencesSlide41

Quality Assessment & Critical Appraisal

Quantitative studies

Internal validity

Bias: selection

;

performance

;

detection

;

attrition

;

reporting

External validity

Move away from checklists/numerical scores to domain based assessment

Cochrane Risk of Bias - RCTs

QUADAS 2 – diagnostic accuracy

ROBIS for systematic reviewsSlide42

Quality Assessment & Critical Appraisal

Qualitative studies

Three broad categories

Rigour

: has a thorough and appropriate approach been applied to key research methods in the study?

Credibility

: are the findings well presented and meaningful?

Relevance

: how useful are the findings to you and your organisation? Slide43

CASP appraisal checklist

Clear

aims of research (goals, why it is important, relevance)

Appropriate methodology

Sampling strategy

Data collection

Relationship

between researcher and participants

Ethical

issues

Data

analysis

8

. Findings

9

.

Value

of research (context dependent) Slide44

Data Synthesis

Building up; putting together; making a whole out of the parts; the combination of separate elements of thought into a whole; reasoning from principles to a conclusionSlide45

Data Synthesis

Results from different studies need to be synthesisedAre studies and results similar enough to be combined into a single numerical result?NO – qualitative descriptive/narrative summaryYES – quantitative meta-analysis

Heterogeneity

Difference in results can arise due to differences in study design, population, selection, intervention delivery

How similar is similar? Results from heterogeneous studies should not be pooledSlide46

Narrative synthesis

Instead of/alongside meta-analysisPotential bias in presentationLack of a take home messageSlide47

Tools for narrative synthesis

Partly informed by methodological work in qualitative synthesisTabulationGroupings and clustersVote counting as a descriptive tool

Examination of moderator variables (elements of e.g. setting, population)

Rodgers et al Evaluation 2009 15 49-72Slide48

Meta-analysis/forest PlotSlide49

Most important thing:

Be organised!!!Slide50

Gantt chartsSlide51

Reference management

Use a reference manger to sift and storeKeep all citations retrieved

Add in those you can’t download

Use to de-duplicate results

Sift citations for inclusion/exclusion

Can use codes/notesSlide52

Version control

Dates – YYYYMMDDVersion numbering v0.1 = first draft v1.0 = final versionv1.1 = minor amendments to final version

v2.0 = major revision

Avoid using draft, draft 1, final final, etc.

Clear naming convention

E.g. Date_project_title

20/11_Autism HTA_resultsv1.2.docSlide53

Other help available

FMS Systematic review group Informal monthly session where methods are discussed and issues can be raised

fiona.beyer@ncl.ac.uk

or

jenni.hislop@ncl.ac.uk

alternatively you have my contact details

MSc in Public Health and Health Services Research (~October 2016)

10 credit module ‘Introduction to systematic reviewing and critical appraisal’pghealth@newcastle.ac.ukSlide54

Thanks for your attention!

Contact details:

Dawn Craig

Institute

of Health &

Society

dawn.craig@ncl.ac.uk @dawn_craig