2 Study Summary of Methodology and Results Quantifying Effect Sizes William Sones Tuesday 16 th May 2017 PSI 2017 Conference The Grange Tower Bridge Hotel London Introduction Target difference estimation ID: 592364
Download Presentation The PPT/PDF document "The DELTA" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The DELTA
2
Study:
Summary of
Methodology and Results
Quantifying
Effect Sizes
William
Sones
Tuesday 16
th
May 2017
PSI 2017 Conference
The Grange Tower Bridge Hotel, LondonSlide2
IntroductionSlide3
Target difference estimation
Target difference (effect size) estimation has been established as a key area of uncertainty in
RCTs
Although research has been performed examining techniques applied, little guidance exist for practical implementation
DELTA - Guidance for
a standard superiority two-arm parallel group trial
(Cook
at al. 2015)
Research has been performed to
determine techniques applied (but this may not cover best
approaches
)Slide4
Target difference estimation
The
MRC and NIHR kindly offered funding
to:
Determine scope and design required
Write and publish guidance
The
DELTA
2
study aims to establish
guidance required
and publish guidance on target difference estimationSlide5
The DELTA2 StudySlide6
The DELTA2 Study
Review existing guidance
UK and International funding programmes and regulatory bodies
Identify key methodological developments and changes
Systematic review of recent methodological developments
Consensus process
Delphi study
Consensus meetings
Draft guideline feedback
Construction and
dissemination
of final
guidelinesSlide7
Review of existing guidanceSlide8
Review of existing guidance
Funding bodies, regulatory authorities, and advisory groups reviewed include:
Arthritis Research UK, BHF, HRA, MRC, RDS, Cancer Research UK,
Wellcome
Trust, NIHR, FDA, Health Canada, European Commission (H2020), NIHR Statistics Group, CIHR, PCORI, NIH, NHMRC, and AHRQ
Majority of sites contained little, if any guidance
Guidance observed generally provided information on the role of the target difference, and little on implementationSlide9
Key methodological developments and changesSlide10
Key methodological developments and changes
Systematic review was performed upon 22 Journals between the dates 2011/01/01 and 2016/03/31
1395 publications were returned using search terms:
sample size, target difference, effect size, important difference, detectable difference, power calculation, value of information, value of perfect information, value of partial perfect information, value of sampling information, expected net gain
Review of title and abstract reduced the number to 85
Full
text investigation
showed 41 publications
containing information of relevanceSlide11
Consensus process Slide12
Consensus process
Delphi study
Consensus meetings
Stakeholder engagement
Multiple online surveys
(c
ore stakeholders with interest and expertise in the design of RCTs)
Outcome
of each survey is
fedback
to stakeholders, influencing opinion
E
stablish consensus (guidance scope and level of contents)
2-day consensus meeting of experts representing different aspects of RCTs
Session at SCT with potential scope and content of guidance
Establish scope
Session at PSI presenting draft guidanceSlide13
Delphi surveySlide14
Delphi survey
Aimed to determine a wide opinion upon the required scope of guidance
Invitation to participate was sent to:
Funding panel Chairs/Directors, CTU directors, support groups (e.g. Hubs, NIHR stats group, RDS), statisticians
162 invitations were sent. 78 (48%)
participated
The online survey investigated topics including:
Type of study, methods for specifying target difference, approaches to consider, complex designs, value of existing guidanceSlide15
Delphi results - Participant details
Your role in RCTs (select all that apply):Slide16
Delphi results - Participant details
Your role in RCTs (select all that apply):Slide17
Delphi results - Participant details
Primary RCT related affiliation:Slide18
Delphi results - Participant details
Primary RCT related affiliation:Slide19
Delphi results - Participant details
Where do you work? If you work across Europe or Internationally please choose the category in which the majority of your work is performed.Slide20
Delphi results - Participant details
Where do you work? If you work across Europe or Internationally please choose the category in which the majority of your work is performed.Slide21
Delphi Results - Types of studies
Guidance for specifying the target difference for a phase III/IV (often called definitive or confirmatory) trial needs to be dealt with separately from early phase, pilot or feasibility trials.Slide22
Delphi results - Methods for specifying target difference
Should the following approaches be considered a formal method and covered within the guidance?Slide23
Delphi results - Methods for specifying target difference
Should the following approaches be considered a formal method and covered within the guidance?Slide24
Delphi results - Special topics
Degree of coverage required for the following special
topicsSlide25
Delphi results - Special topics
Degree of coverage required for the following special
topicsSlide26
Delphi results - Special topics
Additional
comments under special
topics include:
Additional topics of interest
E.g. absolute
vs relative risks
Different perspectives
E.g. provider
and regulatory agency perspectivesSlide27
Delphi results - Complex designs
Degree of coverage for trials with more complex
designsSlide28
Delphi results - Complex designs
Degree of coverage for trials with more complex
designsSlide29
Delphi results - Existing guidance
The existing paper (DELTA; Cook at al
. 2015)
is
usefulSlide30
Delphi results - Existing guidance
Criticism of DELTA guidance:
Guidance style (e.g. more examples/example to provide greater coverage of trials/different approach-less descriptive and more practical guidance
)
Increase coverage
for
different forms of
trials
Improvement of guidance within 2-arm
superioritySlide31
Delphi results - Further suggestions
Views
covered to greater
extent/importance include:
More complicated designs beyond those
addressed exist
Wider
range of
guidance/Support
for a wider audience
Will DELTA
2
provide greater support for what is wanted rather than trials performed?Slide32
Delphi surveySummarySlide33
Summary
Majority of respondents had statistical experience, worked in academia, and were UK based
Suggestions:
Phase III/IV should be reviewed separately from early phase, pilot or feasibility
Standardised effect size should be covered in guidance
VoI
should be covered (perhaps to a lesser extent)
Alternative research questions and multiple primary outcomes deserved greater attention
than
other
topics
Support for a wider
audienceSlide34
Summary (cont.)
Suggestions
(cont.):
Adaptive designs and cluster randomised trials deserved greater attention than within-subject paired and cross-over designs
Existing DELTA guidance is
usefulSlide35
Acknowledgments
The project was jointly funded by the Medical Research Council (MRC) and National Institute for Health Research (NIHR)
Survey was performed using BOS, University of
Bristol
Email: William.sones@ndorms.ox.ac.uk