Metaanalysis pooling study results Objective Understand the philosophy of metaanalysis and its contribution to epidemiology and science Understand the limitations of metaanalysis Introduction ID: 357419
Download Presentation The PPT/PDF document "Simon Thornley" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Simon Thornley
Meta-analysis: pooling study resultsSlide2
Objective
Understand the philosophy of meta-analysis and its contribution to epidemiology and science.
Understand the limitations of meta-analysisSlide3
Introduction
Systematic quantitative integration of results
several
independent
studies
D
istinct
from a narrative
review
“expert”
Synthesis
of published information.
Usually
considered only appropriate for RCTs
Still controversial
even in this context
.
Google search on “meta-analysis”
8 million hits
!Slide4
Criticism
“statistical alchemy” for the 21st
Century
“
The intellectual allure of making mathematical
models and
aggregating collections of studies has been used
as
an escape from the more fundamental scientific challenges”
-Feinstein.Slide5
Purposes of meta analysis
Inefficiency
of traditional narrative reviews
.
Allow researchers to keep abreast of
accumulating evidence
Resolution of uncertainty when r
esearch
disagree
s?
Increase statistical power
, enhances precision of
effect estimates
– especially small
effects
Allows exploratory analysis (subgroups)Slide6
Inadequate sample size? (Deal with type-2 error)
Single trials too
small to detect moderate effects
(low power
– high chance of
Type-2 error (
false-negative
))
Investigators
often
over enthusiastic about
size of treatment effects and sample
size
Meta-analysis doesn’t deal with other threats to study validity (bias, measurement error; in fact, may increase)
e
.g. CVD death vs. total mortality Slide7
Accept H
0
Reject H
0
Statistical Test result
H
0
True
False
OK
OK
Type-1
error
Type-2
error
Prob
of a type 1 error = alpha
a
(
usually fixed, say 0.05
)
Prob
of a type 2 error = beta
b
= 1-powerSlide8
Random error lecture
Average odds ratio is 21?? Consistency??Slide9
Which studies?
Need defined question, state MESH terms
Reproducible
Exhaustive search
Unpublished and published studies
Variety of databases.Slide10
Difference in means,
Standardized differences in means
Survival measures
Relative
risk
Odds
ratio
Risk
difference
NNT [=1/RD]
Incidence rate ratios (person time data)
Typical summary outcome
measuresBinary: Continuous:Slide11
Assume distribution of true effects
Aim is to measure mean of distribution of true effects
Greater heterogeneity --> greater variation
Gives greater weight to small studies than fixed effect method of analysis.
More conservative (wider confidence interval around effect estimate, compared to fixed effect method)
Mantel-
Haenszel
method
treat
each trial as a “stratum”
take
weighted average of effects
.O-E (Peto) methodBinary
outcome (e.g. death) Oi =observed # deaths on treatment in trial i
Ei=expected # deaths (assuming no treat effect)
look at average of Oi - Ei over all trialsAssumes underlying true effect for each study and differences only due to random errorMethods of analysisFixed effectRandom effectSlide12
Dietary fat and cholesterolSlide13
Reduced or modified dietary fat and all-cause mortalitySlide14
Publication biasSlide15
When meta-analysis goes bad…
In CVD drug research, CVD outcomes often favoured over total mortality
Which would you prefer????Slide16
Publication
bias:
other
methods
Ioannidis JPA,
Trikalinos
TA. An exploratory test for an excess of significant findings.
Clin
. Trials
2007;4(3):245-53
.
Calculate expected number of positive studies, given:
Sample size of individual studiesNumber of events in controlsSummary effect (assumed true)Slide17
Statin meta-analysisSlide18
Problems
Combining heterogeneous studies (apples
and
oranges
)
Combining good and bad studies (good
and
bad apples
) (
study quality
)
Publication bias (tasty apples only
)The "Flat Earth" criticism – reductionism –(Braeburns only)Combin
ing data (individual v summary data stewed apples have different character to raw)
Application to randomized studies only? Type-2 error only one problem with epi studiesSlide19
Meta analysis in observational
studies
MA often applied
in
observational studies
As
often as RCTs (Egger et al)
…. with
controversy
….
Confounding and bias unlikely to “cancel out
”Publication bias and “research initiation bias” (i.e. studies only done when there is an association)Different ways of reporting/analysing result (e.g different outcome measures, confounders, models, exposure levels)Slide20
Summary
Meta-analyses increasingly used
Logical only for RCTs?
Summarise medical literature
Reduce type-2 error by increasing sample size.
Don’t deal with other types of epidemiological error (confounding/measurement error)
Prone to unique type of error (Publication bias)
Can be difficult to detect