PITFALLS in the CONDUCT of - PowerPoint Presentation

Slide1

PITFALLS in the CONDUCT of MULTINATIONAL RANDOMIZED CLINICAL TRIALS

Gilles DELLA CORTE, MDDELLMED Consulting

IntroductionMultinational Randomized Clinical Trials (MRCTs) are challenging

But, despite the challenge, the need to conduct clinical research internationally is stronger than everThe increasing shift from expert opinion towards evidence-based medicine enhances the importance of MRCTsIn addition, the demand of Health Authorities (HAs) for high quality data is also increasing, leading to favor MRCTs

DELLMED Consulting

2

Pitfalls levels in MRCTsUnreasonable protocol design

Wrong outsourcing choiceNon optimal feasibility assessmentRegulatory naivetyBlurring border between Marketing and Research

Optimistic patients recruitment plan

Inadequate communication and monitoring

Deficiency in Data Management

Deficient statistics hypothesis and analysis methods

DELLMED Consulting

3

Unreasonable protocol design

Study investigated hypothesis is not clearly definedThe Null-Hypothesis under investigation must be clearly statedIn rule, limit the objectives to only one primary objective

Study design and investigations too much demanding

Too many visits or investigations leading to poor compliance from the patients and investigators

Investigations methods too tricky or not meaningful of the actual efficacy of the compound (e.g. use of of non-validated surrogate endpoint) or with a large inter-examiner variation

Placebo group not always acceptable (e.g. life-threatening disease)

Wrong inclusion/exclusion criteria

A right balance between recruitment needs and scientific constrains must be achieved

The population must be selected according to the compound characteristics and study goal and relevant to demonstrate “efficacy” of the compound but not too selective compared to real-life recruitment in order to demonstrate “effectiveness”

DELLMED Consulting

4

Unreasonable protocol design (cont)

Critical literature review is keyParticular interest in looking at the discussion section of paper regarding similar studies, the authors often speculate on what was the weaknessess and what needs to be accomplishedThe support of an Experts Steering Committee is keyChoice of the experts is challenging, aiming to obtain a right balance between scientific input and pragmatism

Include medical responsible and guests at request (statistician, monitor …)

Endorse Steering Committee proposals only if scientifically validated (literature, guidelines …)

The Investigators Meetings are key

To ensure full understanding and acceptation of the protocol and detect country-linked discrepancies

DELLMED Consulting

5

Wrong outsourcing choice

There is an inflationary trend for oursourcing at least part of services requested in MRCTsThe Contract Research Organisation (CRO) choice must focus on:Previous experience of the provider in this indication with a pool of « ready to use » sites

Significant monitors presence in the target regions (be mistrustful with CROs subcontracting locally or using local freelance CRAs, with just a local « storefront »)

Experience of CRAs involved (Minimal seniority must be defined with CVs provided)

Proposal describing precisely the services provided (optimally provide a proposal template to ensure comparibility of the proposals) with number of units and costs (including passthrough costs evaluation)

Previous experience with this provider (Preferred provider) is ofen an asset preventing the « learning period »

The CRO proposal for feasibility assessment (a « light » plan is not a good signal)

Pre-select at least 3 CROs based on a proposal draft and then organize individual bid meetings with them before getting their final proposal

Also consider the use of Site Management Organizations (SMOs)

Be careful if you give both the monitoring and the data management to the same provider as data quality concerns could be minimized

A provider coud be used for data management and analysis separately from that responsible for clinical operations and data collection. Moreover this allow to have the same provider responsible for data from all studies (homogeneity) when different providers are used for clinical operations depending on the study specific needs

DELLMED Consulting

6

Non optimal feasibility assessment

Feasibility study prior to trial start is keyShould not be limited to an e-questionnaire sent to sites, direct contacts is preferable (site qualification visits particularly for unknown sites)Recruitment potential is important but also verifyEquipment and drug storage facilities

Availability of CT dedicated staff

Competitive CTs ongoing or planned

Familiarity with electronic data capture (EDC)

Ability of the site to comply with protocol constrains, often different from the regular practice

Use of a non final protocol is a risk if important features are changed

Site selection is important but also regulatory frame per country and standard of care in the target indication must be assessed

Re-conduct site feasibility if > 3 months from selection to activation

DELLMED Consulting

7

Regulatory naivety

Local constrains must be taken into account:Agencies but also Institutional Review Boards/Ethics Committees structure, demands and approval timelinesCustoms clearance, taxation, licenses, documentationNeed for documents translation in local languages (sometimes « certified » translations can be obtained)

Local healthcare system reimbursment

Timelines must be evaluated per country leading to define a set-up strategy plan (stagger submissions)

Some proactive actions are useful:

Involve local Regulatory experts if not existing inhouse

A best case and a worst case must be set, probably the actual timelines will be in between

Finalize the protocol before submission

Ask for a pre-IND meeting is often useful

Make sure if some countries have data acceptance policies (e.g. acceptance of 100% foreign data for local marketing approval)

Establish a check list of required documents

Applied for more patients and sites than needed if possible

DELLMED Consulting

8

Regulatory naivety (cont)

Frequent naivety if to think approval is possible with only few doses testedDuring Phase II, in most of the cases, at least 3 doses must be tested because, to confirm the chosen dose is that with the best efficacy / safety ratio you need to demonstrate that a lower dose showed less efficacy and that a higher dose showed either not more efficacy or is less toleratedSometimes more than 3 doses tested in Phase II are necessary to reach this conclusion. Be conscious that numerous Phase III programs failed because the Phase II hadn’t been properly performed

A doubt can lead to test still 2 doses in Phase III

DELLMED Consulting

9

Blurring border between Marketing and Research

Key opinion Leaders (KOLs) involvement is important for commercial purpose but only if they have actually the capacity to perform the trial optimally from recruitment potential to quality data generation or if their poor performace is forecasted in the recruitment planAlways mix with lesser-known but highly motivated and competent investigatorsThe

Study Principal Investigator

(PI) must belong to both categories as he will have to adjudicate controversies, enforces the decisions, sign the Clinical Study Report and be the main author of the publication. As such, his legitimacy must be recognized by the sites PIs

If

Ancillary Studies

are implemented, they must really bring interesting data (e.g. pharmaco-economic) or be a motivation tool (specific publication) and not just « please » a local KOL and it must be sure they do not compromise the core MRCT (additional complexity that jeopardizes recruitement)

DELLMED Consulting

10

Optimistic patients recruitment plan

Investigators are often optimistic when they evaluate their patients recruitement potential; their statement must be ponderatedAnalyze disease prevalence and incidence per regionConsider catchment area of patients around site

Ensure the local standard of care makes your protocol acceptable

Competitive trials ongoing or planned must be identified

Proper site training must be delivered

Fair allocation of site grants must be ensured

Recruitment facilitation and patient retention methods are useful (after confirming acceptance for local regulation):

performance incentive with bonus to « good recruiters »

ads in newpaper or local radio

direct phone call from the investigator to the patient

involvment of GPs as referral network

« motivation » visits by the CRA

patients travel costs to hospital reimbursed …

electronic tools for recruitment tracking and boosting

DELLMED Consulting

11

Optimistic patients recruitment plan (cont)A CAPA Plan must be ready in case of low recruitment rate

Back-up sites identified from scratch and ready to operateMid-study rejuvenation meetingsprotocol amendment

« motivation » visits by the sponsor itself …

Buffers must exist i the Clinical Development Plan (CDP) for unexpected delay and costs

DELLMED Consulting

12

Inadequate communication and monitoring

Face to face interaction is useful (at least video conferencing) and must not be replaced by only phone call or e-mailsUse a Clinical Trial Management System (CTMS) or at least a technology like SharepointWhether on-site or remotely, monitoring must ensure data quality

Adequete source data verification (with pre-defined %) has to be set-up

e-CRF with integrated on-line validation program is of great help

Build strong relationship with site PIs is key

Investigator meetings involving the sponsor (pre-study but also during it when long-term follow-up is planned)

monthly study newsletter, phone calls

fees difference between countries not too high (max 30% and always the same within a given country)

Only one point of contact in case of problem (usually the Study Medical Responsible of the sponsor who works closely with the Study PI and the CRO Project Leader)

DELLMED Consulting

13

Deficiency in Data Management

Lack of preservation or concealment of randomizationShould be prevent by internet-based randomizationIntent-to-treat analysis is keyRandomization is the only method that can balance unknown confounders between groups

Lack of blinding

Placebo must be identical to active drug

If not possible double dummy method must be used

If blinding of the investigator or patient is not possible (e.g. surgery versus medical treatment), use, as much as possible, an independant assessor to assess the main evaluation criterion

Wrong management of Data Safety Monitoring Board (DSMB) meetings and decisions

Independence from the study investigators is key (use of external experts)

Involvement of the sponsor’s Medical Responsible

Involvement

of a Statistician (independant from the study statistician)

Possible involvement of a CRO Representative (e.g. Project Leader)

DELLMED Consulting

14

Deficient statistics hypothesis and analysis methods

Alpha Error (Type I) is usually set as p < 0,05Evaluation parameters must be defined before the trial startsP must be diminished in case of multiple comparisons to guard against the risk of false positive findings (e.g. p=0,05/N, N= number of comparisons)Be aware that not statistically significant just means that no conclusion can be withdrawn and not that there is no difference

Confident Intervals (CIs) are more informative than p-values

DELLMED Consulting

15

Deficient statistical hypothesis and analysis methods (cont 1)

Beta Error (Type II) is usually set as max 20% (power ≥ 80%)Meaning up to 20% false negative is acceptedThe patient sample size must be calculated accordingly based on the primary evaluation parameter.Sometimes meta-analysis is used to reduce beta error but they are limited by the heterogeneity of the pooled studies

Concerning the primary evaluation parameter:

the magnitude of the difference expected between groups must not be overestimated but it must be large enough to be « clinically relevant »

the anticipated standard deviations in each groups must not be underestimated. Variability can be linked with the patient response to treatment but also with the method used to assess this response

results from previous studies (from Proof Of Concept and Phase II studies, from literature about similar compounds) must be carefully assessed

DELLMED Consulting

16

Deficient statistical hypothesis and analysis methods (cont 2)

Neglect of site differencesSite x Treatment Interaction must be taken into account (even if the sample size necessary to have adequate power to detect an interaction is typically many times larger than that to detect a main effect of treatment)Potential site effect must be assessed (when recruitment is much larger in one site than in the others)

Not well defined expected drop-out rate

How the project deals with patients drop-out (and missing data) must be included in the statistical section

Post-hoc analysis must be avoided

Plan Ad-hoc sub-group analysis in the initial protocol as much as possible

Be aware that subgroup analysis introduces multiple testing and is not relevant to assess interaction between factors

Do not overemphasize the results of subgroups analyses, they are just indicative for further confirmatory trials

DELLMED Consulting

17

ConclusionAll stages of MRCTs set up and follow-up must be proactively assessed (protocol, CROs choice, recruitment, monitoring and communication, datamanagement and statistics) to prevent the numerous pitfalls that jeopardize the production of relevant conclusions

« Luck favors the well prepared »

DELLMED Consulting

18