Objective Understand the philosophy of metaanalysis and its contribution to epidemiology and science Understand the limitations of metaanalysis Introduction Systematic quantitative integration of results ID: 919822
Download Presentation The PPT/PDF document "Simon Thornley Meta-analysis: pooling st..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Simon Thornley
Meta-analysis: pooling study results
Slide2Objective
Understand the philosophy of meta-analysis and its contribution to epidemiology and science.
Understand the limitations of meta-analysis
Slide3Introduction
Systematic quantitative integration of results
several
independent
studies
D
istinct
from a narrative
review
“expert”
Synthesis
of published information.
Usually
considered only appropriate for RCTs
Still controversial
even in this context
.
Google search on “meta-analysis”
8 million hits
!
Slide4Criticism
“statistical alchemy” for the 21st
Century
“
The intellectual allure of making mathematical
models and
aggregating collections of studies has been used
as
an escape from the more fundamental scientific challenges”
-Feinstein.
Slide5Purposes of meta analysis
Inefficiency
of traditional narrative reviews
.
Allow researchers to keep abreast of
accumulating evidence
Resolution of uncertainty when r
esearch
disagree
s?
Increase statistical power
, enhances precision of
effect estimates
– especially small
effects
Allows exploratory analysis (subgroups)
Slide6Inadequate sample size? (Deal with type-2 error)
Single trials too
small to detect moderate effects
(low power
– high chance of
Type-2 error (
false-negative
))
Investigators
often
over enthusiastic about
size of treatment effects and sample
size
Meta-analysis doesn’t deal with other threats to study validity (bias, measurement error; in fact, may increase)
e
.g. CVD death vs. total mortality
Slide7Accept H
0
Reject H
0
Statistical Test result
H
0
True
False
OK
OK
Type-1
error
Type-2
error
Prob
of a type 1 error = alpha
a
(
usually fixed, say 0.05
)
Prob
of a type 2 error = beta
b
= 1-power
Slide8Random error lecture
Average odds ratio is 21?? Consistency??
Slide9Which studies?
Need defined question, state MESH terms
Reproducible
Exhaustive search
Unpublished and published studies
Variety of databases.
Slide10Difference in means,
Standardized differences in means
Survival measures
Relative
risk
Odds
ratio
Risk
difference
NNT [=1/RD]
Incidence rate ratios (person time data)
Typical summary outcome
measuresBinary: Continuous:
Slide11Assume distribution of true effects
Aim is to measure mean of distribution of true effects
Greater heterogeneity --> greater variation
Gives greater weight to small studies than fixed effect method of analysis.
More conservative (wider confidence interval around effect estimate, compared to fixed effect method)
Mantel-
Haenszel
method
treat
each trial as a “stratum”
take
weighted average of effects
.O-E (Peto) methodBinary
outcome (e.g. death) Oi =observed # deaths on treatment in trial i
Ei=expected # deaths (assuming no treat effect)
look at average of Oi - Ei over all trialsAssumes underlying true effect for each study and differences only due to random errorMethods of analysisFixed effectRandom effect
Slide12Dietary fat and cholesterol
Slide13Reduced or modified dietary fat and all-cause mortality
Slide14Publication bias
Slide15When meta-analysis goes bad…
In CVD drug research, CVD outcomes often favoured over total mortality
Which would you prefer????
Slide16Publication
bias:
other
methods
Ioannidis JPA,
Trikalinos
TA. An exploratory test for an excess of significant findings.
Clin
. Trials
2007;4(3):245-53
.
Calculate expected number of positive studies, given:
Sample size of individual studiesNumber of events in controlsSummary effect (assumed true)
Slide17Statin meta-analysis
Slide18Problems
Combining heterogeneous studies (apples
and
oranges
)
Combining good and bad studies (good
and
bad apples
) (
study quality
)
Publication bias (tasty apples only
)The "Flat Earth" criticism – reductionism –(Braeburns only)Combin
ing data (individual v summary data stewed apples have different character to raw)
Application to randomized studies only? Type-2 error only one problem with epi studies
Slide19Meta analysis in observational
studies
MA often applied
in
observational studies
As
often as RCTs (Egger et al)
…. with
controversy
….
Confounding and bias unlikely to “cancel out
”Publication bias and “research initiation bias” (i.e. studies only done when there is an association)Different ways of reporting/analysing result (e.g different outcome measures, confounders, models, exposure levels)
Slide20Summary
Meta-analyses increasingly used
Logical only for RCTs?
Summarise medical literature
Reduce type-2 error by increasing sample size.
Don’t deal with other types of epidemiological error (confounding/measurement error)
Prone to unique type of error (Publication bias)
Can be difficult to detect