Collaboration and Consensus Standards Around Data Definitions and Operational Workflow Matt Wilson RN Director CECSSACL DCRI Objectives Standards B est P ractices in adjudication DataDefinitions ID: 764064
Download Presentation The PPT/PDF document "Collaboration and Consensus –" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Collaboration and Consensus – Standards Around Data, Definitions and Operational Workflow Matt Wilson, RN Director CEC-SS-ACL DCRI
Objectives: Standards B est Practices in adjudication?Data-DefinitionsOperational workflowGuiding principlesStart to FinisheCRFCharterTriggersQuality Metrics that matter
The CEC MissionP rovide HIGH quality adjudicated endpoint data with scientific rigor, efficiency and innovation by coordinating and conducting S ystematic, Consistent, Unbiased, Blinded, and Independent Clinical Events Adjudication.
Are we saying the same things? What should we adjudicate? Primary, Secondary, Events of Interest, Causality, Eligibility, etc.How should we identify (trigger) events for adjudication? Outcome pages, Lab data, Medical records, Free text, Hospitalizations, AE’s, SAE’s, etc. What should we collect? eCRF (patient profile report), DC summary, H&P, ER report, Lab slips, Image reports, EKG’s, Death Certificate, Autopsy report, Consult notes, MARS, etc.Who are our partners? Sponsor, CRO, ARO, Ops, Data, Stats, Reviewers, Sites, FDA, IRB, EC, SC, DSMB etc. What is quality?Key Risk Indicators, Currency, Traceability, Major Discordance, Missing data, Missing events, etc.Who is monitoring the eCRF?Site Management, CEC, Stats, Sponsor, Safety, Technology, etc.
Do we have standards?
Guiding Principles - Defining Quality Were all suspected events triggered as planned?Were all events processed per charter procedures? Were all adjudications data based accurately?Were all QC procedures performed/documented with identified issues addressed?Did any major SOP/WI/HIPAA/GCP/IRB violations occur?
Start to Finish! Protocol +Efficacy vs Safety? +Endpoint driven?Charter +Definitions? +SD tableeCRF design +QB/EVENT Pages Triggers +Clinical Trigger spec Edit C hecks/DVC +Stats/DM partners
Definitions2014 FDA Hicks KA et al: Circulation 2014 – http:// www.onlinejacc.org/content/66/4/403 2012 Key Data Elements and Definitions for PAD – http://www.onlinejacc.org/content/59/3/294Fourth Universal Definition MI – http://www.onlinejacc.org/content/early/2018/08/22/j.jacc.2018.08.1038?_ga=2.89629597.726480529.1537383130-267648101.1537383130TAVR VARC-2 Consensus document – https://www.ncbi.nlm.nih.gov/pubmed/23026738
Source Documentation When is it too Much? When is it not enough? “Fine balance between getting enough information and getting ALL the information to review a case”
Which way do we go? Source documentation vs. eCRF data Source Documents eCRF data ComprehensiveReduce site burdenData Security (PHI considerations)Efficiency (cycle times, cost)TranslationsConsistency (same questions)
Big Picture – what, when, where… Recommended Event-Specific eCRF data/Source Documents Death and Hospitalization Events eCRFClinical Event Narrative NT-pro-BNP results (if performed) Procedure reports; PCI/CABG (if performed) Right heart catheterization results (if performed) Cardiac Markers with ULN (if performed) Source Documents (tracked by CEC) Discharge Summary (if hospitalized) Autopsy Report (if performed) Event ECG’s pre, during, post (if performed) Neurology Consult notes (if performed) Urgent Heart Failure Visits eCRF Clinical Event Narrative Exam report Treatment plan Labs to confirm evidence of new or worsening HF Source Documents (tracked by CEC) ER or clinic visit note “We are quickly approaching an era of limited source data and a dependency on CRF data…the necessity of a good clinical narrative…is REALLY important… conventions that the sponsor and/or FDA agree upon re: adjudication from CRF data only will be important”
Designing the Data Collection Process
eCRF Best Practice: Quarterback PageCaptures Key Components/ Critical Variables Yes/No response required for each event typeDynamically creates event page to collect further details when event is answered ‘Yes’Through telephone or clinic visit Recommend collecting no less frequently than every 3 monthsImportant in later clinical phases
Support the Endpoint DefinitionsMeasurable and ObjectiveStrategic “Net” to identify eventsComplements source documents Reduces source document collectionPertinent Information Consistent same questions Chronology of careTrigger on this page Death, Stroke, Myocardial Infarction, Bleeding, Congestive Heart Failure, Acute Pancreatitis, Neoplasms…. eCRF Best Practice: Event/Endpoint Detail Page
eCRF Best Practice: CEC Relevant PagesHospitalization Page Strategic monitoring strategyLab Value PagesCardiac Markers, Troponin I/T, CK & CK-MBs Labs of Interest: HGB/HCTTransfusionsProcedural Pages Cardiac Catheterization (CATH)Percutaneous Coronary Intervention (PCI)SAE/AE Pages
No matter how good (or bad) your CRF is, if it’s flawed — you won’t get intended results.
Design and Implement an effective CRF (data capture and edit checks) Routine QC checks (involve Stats, DM, CEC, Site Management) Cross checks Lab data suspicious of event, no event reportedDuplicate events (e.g. >2 in 1 week) Reconcile SAE/AE (coded terms) to CEC triggered events Negatively adjudicated events (follow up for potential SAE/AE) Data that you cannot algorithmically assess: Free text review; Hosp , SAE/AE pages, SAE labs) 2 Large ACS trials, DCRI methodology applied (possible Missed MI’s) 41,965 lab records, 2,247 possible triggers, 29 triggered & confirmed 10,269 hosp records, 370 possible triggers, 49 triggered & confirmed Source document review (Medical records submitted for adjudication, review for SAEs/additional endpoints)
Operational Workflow – CEC project staffing (Clinical team)? CEC PI, Members, PL, CTC, CTA, CDSWho trains who? CEC PI, CTC What are we training our CEC teams on? Protocol, Charter, Conventions, etc.How do we adjudicate? Phase 1, Phase 2, Single Clinician Review, Third Reviewer What qualifies a reviewer to adjudicate? TA/Clinical experience, CEC experience What % of cases reach committee? >30% retrain, assess for issuesWhat if CEC is receiving more triggers than expected? Check your net, patient demographicAre the sites submitting the right SD in a timely manner? 2 weeks Are the sites responding to queries in a timely manner? 2 days Query escalation plan in place ? Swat Team approach with CRAs, CEC, Sites, Sponsor Are there calls between site management and the CEC coordinators to discuss what’s needed on the more difficult cases ? Who is responsible for cleaning the CRF? Site management
Metrics that matterCurrency % Query (10-15%)% Complete (80-90%)Cycle TimesQuery to Clean (10-30 days)Clean to Complete (5-10 days) Discordance (Major disagreements)Site vs CEC (10-20%)CEC vs CEC (10-20%)
Adjudication c ycle times – how are we doing? 30 days total * The event date to triggered can be dependent on visit windows Set targets: Query to Adjudication <30 days Query @site x-days, escalate Adjudication to Complete <10 days Case @review x-days, escalate
Adjudication Completeness “Currency” – Gold 80%, Platinum 90%, Executive Platinum 95% Troubleshoot problems: queries, translations, reviewer delays. Know where to push! Event Types # Events Triggered # at Query # at Phase 1 Physician Review # Ready for Committee Review # at QC # at Re-Review # at No Action Needed # at Complete + # at No Action Needed # at Complete Bleed 219 23 15 3 0 0 4 178 174 Death 36 8 2 1 0 0 0 25 25 MI 25 6 1 0 0 0 2 18 16 Stent Thrombosis 16 1 1 0 0 0 0 14 14 Stroke 12 1 1 1 0 0 0 9 9 Totals 308 39 20 5 0 0 6 244 238
Do I have enough to adjudicate the case? Recommend a joint Ops/Reviewer strategy1.) Ops tracking Quantify what is in the casesAssist with query process Active/End of trial metrics “what cases were missing x?”2.) Reviewer assessmentAnswers the key question – “did I have enough?”
Insufficient Source Documentation question and follow-up: Entered by reviewers “reviewer form” Entered by querying team “tracking page”
CEC Key Risk Indicators (KRI’s)time between event occurred and event triggered time between "in query" to “ready for adjudication"time between Phase 1 and a) final outcome or b)phase 2 [overall]time between Phase 1 and a) final outcome or b)phase 2 [by adjudicator] time between Phase 1 disagreement and Phase 2ratio of # agreement/#disagreement by adjudicator # documents received/# documents expected by site & event type # agreement events [sites and CEC]/#total events [sites and CEC] and event type
Design/Implement plan early on, include key stakeholders (FDA, Sponsor, EC)Methods used require validation stepsDocumentation is key (project plan, trigger spec, charter) Involve SMEs (CEC, Stats, DM, SM) Develop and Implement plan early on (startup/FPI) to avoid the following: Missed eventsDifficulty collecting data retrospectivelyLimit free text fields and train sites on effectively capturing clinical context in eCRF narrativeIntegrate your CEC and Safety teams for “Cross Talk”Real time reconciliation of reporting (CEC/Safety)