Simplifying Data Quality Assessment - PowerPoint Presentation

Slide1

Simplifying Data Quality Assessment through Audit SoftwareStephany N. Duda, PhD; Firas H. Wehbe, MD, PhD; Catherine C. McGowan, MDVanderbilt University, Nashville, TN, USA

CONCLUSIONS

We

believe audit software with these attributes has the potential to simplify the audit process for investigators wishing to implement an internal data quality program to ensure the validity of their research data and study findings.

METHODSThe CCASAnet data center conducts on-location, Good Clinical Practice-based audits of datasets submitted by member sites.2,3 We reviewed the audit documentation and reports of auditor experi-ences collected during 14 paper-based data audits and subsequent communications with study sites.

BACKGROUND Source document verification audits are the gold standard for assessing data quality in clinical research.1 During such audits, external auditors compare submitted research datasets to the original documentation of patient care, which may include paper clinical charts, laboratory reports, or the contents of electronic medical record and laboratory systems at the study sites.

IMPLEMENTATIONWe developed a prototype system to demonstrate the significance of these key attributes as well as the feasibility of implementing them in a simple web-based application.

Funding for this work was provided by US NIAID (NIH U01 AI069923-06)

1

Developing and implementing a comprehensive clinical QA audit program. Li H, Hawlk S, Hanna K, Klein G, Petteway S. The Quality Assurance Journal. 2007; 11: 128–137. 2 ICH Harmonized Tripartite Guideline: Guideline for Good Clinical Practice E6. 1996; Available from: www.ich.org3 Measuring the Quality of Observational Study Data in an International HIV Research Network. Duda SN, Shepherd BE, Gadd CS, Masys DR, McGowan CC. PLoS One. In press.4 Desiderata for a computer-assisted audit tool for clinical data source verification audits. Duda SN, Wehbe FH, Gadd CS. Stud Health Technol Inform. 2010;160(Pt 2):894-8.

Disorganized Paper Form

FINDINGS

We identified

five key attributes that a computer-

Key Software Attributes

Multi-user support through

networking

Simple file formats for importing and exporting research and audit dataStandard error assessment protocolsAudit decision support for record selectionTools for calculating and displaying audit results

We tested this application during audits of twelve cancer and tuberculosis datasets in nine locations and found the system drastically improved the audit process bydecreasing audit preparation time from over six hours, on average, to less than one hour,enforcing consistent, on-site error coding for all audited data,eliminating difficulties reading other auditors’ completed forms, andsimplifying the manual error tallying required for generating a post-audit report.

Audit Tool Prototype

Audit Data Entry Screen

Improved Audit Results Form

Most

audits use paper

forms, which can be difficult to standardize, share between auditors, and interpret post-audit.

The available audit software packages focus on analyzing an

existing, elec-tronic dataset for errors and unusual patterns, rather than facilitating the compar-ison between the dataset

and a physical source document. We believe a computer-assisted audit tool specifically designed for research data auditing can improve the process of source document verification.

assisted audit tool should possess in order to simplify and standardize the audit process. 4

Home Screen