Utility of electronic medical record for recruitment in clinical research: from rare to common disease

Abstract

Background

Recruitment for clinical trials is a major challenge. Movement disorders, which do not have associated diagnostic laboratory tests, may be especially prone to inaccuracy in coding. Our objective was to evaluate the accuracy of diagnostic codes such as cervical dystonia (CD) and PD in an electronic medical record.

Methods

Retrospective chart review was performed to confirm the ICD-9 diagnoses of PD, CD and diabetes mellitus type 2 (DM-2), using published clinical diagnostic criteria (PD, CD) and hemoglobin A1c ≥ 6.5 (DM-2).

Results

421 charts (n=129, n=142, n=150 for PD, CD and DM-2, respectively) were reviewed. The accuracy rate was different between all diseases examined with an overall p<0.001. In post hoc pairwise comparisons, the accuracy of DM-2 diagnosis by ICD-9 (96.6%) was greater than CD (88.0%) and both greater than PD (55.0%) (p≤0.003).

Conclusions

Using an electronic medical record based screening of clinically diagnosed diseases such as CD may be more accurate than previously thought and may identify potential clinical trial participants even without confirmatory lab tests available.

Introduction

Recruitment in clinical trials remains a major challenge. In one study, nearly half (45%) of clinical trials requested an extension to attempt to reach their recruitment target.¹ Even with this extension, these trials were no more likely to recruit their target sample size. Overall, 78% of trials recruited just 80% of their target.¹ Broad use of the electronic medical record (EMR) has the potential to reduce patient recruitment times by allowing for participant pre-screening.² Even further, EMR databases could be potentially linked to active clinical trials databases to speed recruitment.³ Utility of EMR-based administrative data such as ICD-9 codes for clinical trials remains unclear due to the concerns regarding the accuracy of these codes.^4,5 Further, “honest broker” systems have been touted to balance the preservation of patient privacy concerns with access of clinical researchers to potential participants. These systems range from health linkage tools to institutional review board mandated non-study personnel who can contact potential participants identified through the EMR to assess their interest in participating while maintaining their privacy. However, challenges persist in both difficulty of communicating across different EMR platforms and also in how accurate the diagnostic codes really are. Movement disorders such as Parkinson Disease (PD) and cervical dystonia (CD) may be especially prone to these inaccuracies as these are clinical diagnoses without any confirmatory laboratory or radiological tests.⁴

The aim of this study was to evaluate the accuracy rates of ICD-9 codes for CD and PD, which are clinical diagnoses, and then to compare these accuracy rates with a common medical disorder, diabetes mellitus (DM-2), which has confirmatory laboratory testing. This was performed within one EMR system and at one institution. We hypothesized that accuracy rates for clinical diagnoses would be lower than the rates of diseases with a confirmatory laboratory diagnostic test, and that accuracy rates for CD will be lower than PD due to the rarity of the diagnosis.⁶

Methods

A retrospective chart review was performed to confirm the ICD-9 diagnoses of PD, CD and DM-2. 150 patient charts with ICD-9 diagnoses of CD (333.83), PD (332 and 332.0) and DM-2 (250*) were randomly selected for review. In querying the EMR, the following criteria were used for all three disease codes: 1) the ICD-9 code of interest must be the primary diagnosis; 2) ages 18 and over; and 3) date range from beginning of 2005 (when electronic billing began in our institution) up to present day. In addition, for DM-2, the 150 patients randomly selected from 29,000 candidates in the EMR with ICD-9 code of 250*, were evaluated against a hemoglobin A1c of ≥ 6.5. All the records were obtained from our institution’s EMR system. The study was determined to be exempt by the University of New Mexico Human Research Review Committee.

After identifying the 150 charts for each ICD-9 code of interest, a manual chart review was performed to confirm the ICD-9 diagnosis. For CD, all clinical documentation in the EMR was evaluated and compared with the clinical criteria for dystonia proposed by Albanese et al.⁷ Likewise, for PD, the UK Parkinson’s disease Society Brain Bank Clinical Diagnostic Criteria was compared to the available documentation through the complete EMR chart.⁸ Finally, as described above, the EMR was queried for the ICD-9 code for DM-2 and cross-referenced with hemoglobin A1c ≥ 6.5 to confirm the diagnosis of DM-2.⁹ The statistical analysis of the accuracy rates found for these ICD-9 codes was performed using Fisher’s exact test.

Results

Initially, 150 charts each for PD, CD and DM-2, were included in the study for review for a total of 450 charts. However, 21 charts (PD patients) and 8 charts of patients with CD were excluded due to lack of sufficient documentation to comment about the accuracy of these diagnoses. In these cases, there was no direct documentation (e.g. progress notes, history and physical, or clinic notes) that addressed the coded diagnoses and these charts were excluded from further review.

421 charts (n=129, n=142, n=150 for PD, CD and DM-2, respectively) were reviewed. The accuracy rate was different between all diseases examined with an overall p<0.001. In post hoc pairwise comparisons, the accuracy of DM-2 diagnosis by ICD-9 (96.6%) was greater than CD (88.0%) and both greater than PD (55.0%) (p≤0.003) (Table 1). The accuracy rate of diagnosis was different between all diseases examined with an overall p<0.001. In post hoc pairwise comparisons, the accuracy of DM-2 diagnosis by ICD-9 (96.6%) was greater than CD (88.0%) and both greater than PD (55.0%) (p≤0.003)

Table 1.

Diagnostic accuracy based on ICD-9 diagnosis and confirmed through chart review. Raw number of charts listed with percentage in parentheses.

	DM (%)	CD (%)	PD (%)	TOTAL (%)
Diagnosis Confirmed	145 (96.6)	125 (88)	71 (55)	341 (81)
Diagnosis Not confirmed	5 (3.3)	17 (12)	58 (45)	80 (19)
TOTAL	150	129	142	421

Discussion

In this study, we showed that ICD-9 diagnoses for disorders like DM-2 with clearly defined laboratory criteria are more accurate compared to clinically diagnosed movement disorders, like CD and PD. We were able to identify novel information about the accuracy rate of CD, which had a high diagnostic accuracy rate in our study. We were able to confirm the findings seen previously that the ICD-9 code for parkinsonism is ambiguous, combining both PD and parkinsonism.⁴

As coding systems such as ICD-9 and ICD-10 were designed primarily as administrative databases for use in medical billing, etc., the validity of using EMR-based queries based on ICD is of uncertain validity in many diseases. A recent paper sought to review the validity data on common neurologic diagnoses.¹⁰ They looked at PD/parkinsonism and revealed the same ambiguity seen in our study with a positive predictive value ranging from 38.6 to 81.0.¹⁰ The authors concluded that the interpretation of EMR or other health database studies is dependent on the accuracy of the case definition in that particular database.¹⁰ And, for diseases with clinically-based diagnosis without a confirmed laboratory test, the accuracy rate may vary greatly depending on the database and coding system employed.

CD had a high rate of ICD-9 accuracy in our database—though still with 12% of cases not meeting diagnostic criteria on chart review. The higher accuracy rate may be due to its relative rarity in neurologic disease overall, but with dystonia as the third most common diagnosis seen in sub-specialty movement disorders clinics, the diagnosis of CD was entered into our EMR by predominantly movement disorders specialists. This was in contrast to PD, which had a wide range of neurologists and non-neurologists assigning the code for PD.

Limitations of this study include the retrospective nature of the study and the dependency on documentation to evaluate for the accuracy of diagnosis. Both the quality and thoroughness of the documentation are provider dependent, and as such, this may bias our accuracy rates especially for the PD patients. Another limitation is that there are other candidate control conditions that could be looked at which are typically made by clinical diagnosis but have diagnostic tests for confirmation, such as peripheral neuropathy, which may have added contrast to our findings. Further, the United States will be converting to the use of ICD-10 diagnostic codes in the near future. This may certainly allow for increased granularity in diagnoses such as PD; however, clinically diagnosed neurologic diseases will still depend on physician expertise and optimal documentation to see increased accuracy in the new system. Finally, these coding accuracy rates are valid for our institution and our EMR and we do not know the generalizability of this finding across sites and other EMR systems.

Despite these limitations, these findings contribute to the validation of neurologic disease in administrative databases. Based on our results, ICD-9 based screening of clinically diagnosed diseases such as CD may be more accurate than previously thought.