Table 1.

Obstacles encountered during clinical data audits and corresponding desiderata for computer-assisted audit tools.

Obstacles Encountered during Audits	Solutions/Desiderata for a Computer-Assisted Audit Tool
Challenge: Collaboration	Solution: Networking
Auditors need to work collaboratively on the same copy of a record.	Real-time Collaboration: networked laptops for auditors, shared data-bases, web-based systems
Audit sites may have no network infrastructure.	Portable Network Infrastructure: peer-to-peer networking, portable server and router
Challenge: Audit Data	Solution: Audit Data Management
Paper audit forms take a long time to prepare and validate.	Import Functionality: one-click import of data and data descriptions (metadata) from research database to CAAT, instant generation of basic electronic audit forms
Copying audit results from paper forms into a spreadsheet for analysis is time-consuming.	Export Functionality: export of audit results into structured data formats such as XML
Datasets may contain different medical content (e.g., HIV, Tuberculosis, or cancer data).	Metadata Management: customizable import interface, customizable display of data on screen, data dictionaries for special topic areas (HIV, TB, Cancer)
Data may violate syntactic rules; auditors may need to record corrected values.	Reasoning About Data Types: representing simple and complex data types, data syntax rules, codification of mismatch between research data and native records, handling malformed data
Challenge: Types of Errors	Solution: Standardized Assessment of Errors
Errors are not categorized and described clearly on paper forms, making it difficult to analyze and report error types and rates.	Representation of Error Types: hierarchical ontology of errors, clear operational descriptions of error types, specification of domain of error types (applies to specific variables within the audit record or applies to entire record), specification of error labels and default values and whether closed or open world assumptions apply
Auditors discover new and unexpected types of errors during the audit process.	Error Scheme Evolution: support for versioning and collaborative authoring of error schemes, interface to edit error schemes while in use
Some audits require different error classification schemes that are better suited to the data.	Error Scheme Management: storage, import, and export of audit-specific error terminologies
Challenge: Audit Design	Solution: Audit Decision Support
Auditors are unsure how many records should be audited to produce meaningful results.	Statistical Dashboard: guidance for sample size calculations, identification of grossly problematic records, pre-selection of records via statistical sampling
Challenge: Analyzing and Presenting Results	Solution: Results Reporting Tools
Tallying and tabulating errors by hand is a time-consuming and error-prone task for auditors.	Automatic Report Generation: software support for generating tables and graphs
Manual approaches may miss subtle patterns of data error.	Real-time Trend Detection: automatic checks for patterns of error suggesting data falsification, systematic errors