Abstract
Background
Recruitment for clinical trials is a major challenge. Movement disorders, which do not have associated diagnostic laboratory tests, might be especially prone to inaccuracy in coding. Our objective was to evaluate the accuracy of diagnostic codes such as cervical dystonia (CD) and Parkinson's disease (PD) in an electronic medical record (EMR).
Methods
Retrospective chart review was performed to confirm the International Classification of Diseases, Ninth Revision (ICD‐9) diagnoses of PD, CD, and diabetes mellitus type 2 (DM‐2), using published clinical diagnostic criteria (PD, CD) and hemoglobin A1c ≥ 6.5 (DM‐2).
Results
A total of 421 charts (n = 129, n = 142, n = 150 for PD, CD, and DM‐2, respectively) were reviewed. The accuracy rate was different among all diseases examined, with an overall P < 0.001. In post hoc pairwise comparisons, the accuracy of DM‐2 diagnosis by ICD‐9 (96.6%) was greater than CD (88.0%) and both were greater than PD (55.0%) (P ≤ 0.003).
Conclusions
The use of an electronic medical‐record–based screening of clinically diagnosed diseases such as CD might be more accurate than previously thought, and it may identify potential clinical trial participants even without the availability of confirmatory laboratory tests.
Keywords: dystonia, Parkinson's disease, clinical research
Recruitment in clinical trials remains a major challenge. In one study, extension requests were made in nearly half (45%) of the clinical trials in an attempt to reach recruitment targets.1 Even with this extension, these trials were no more likely to recruit their target sample sizes. Overall, 78% of trials recruited just 80% of their targets.1 Broad use of the electronic medical record (EMR) has the potential to reduce patient recruitment times by allowing for participant prescreening.2 Even further, EMR databases could be potentially linked to active clinical trial databases to speed recruitment.3 Use of EMR‐based administrative data, such as ICD‐9 codes for clinical trials, remains unclear because of the concerns regarding the accuracy of these codes.4, 5 Further, “honest broker” systems have been touted to balance the preservation of patient privacy concerns with access of clinical researchers to potential participants. These systems range from health linkage tools to institutional review‐board–mandated nonstudy personnel who can contact potential participants identified through the EMR to assess their interest in participating while maintaining their privacy. However, challenges persist in both difficulty of communicating across different EMR platforms and also in how accurate the diagnostic codes really are. Movement disorders such as Parkinson's disease (PD) and cervical dystonia (CD) may be especially prone to these inaccuracies, as these are clinical diagnoses without any confirmatory laboratory or radiological tests.4
The aim of this study was to evaluate the accuracy rates of ICD‐9 codes for CD and PD, which are clinical diagnoses, and then to compare these accuracy rates with a common medical disorder, diabetes mellitus (DM‐2), which has confirmatory laboratory testing. This was performed within one EMR system at one institution. We hypothesized that accuracy rates for clinical diagnoses would be lower than the rates of diseases with a confirmatory laboratory diagnostic test, and that accuracy rates for CD would be lower than PD because of the rarity of the diagnosis.6
Patients and Methods
A retrospective chart review was performed to confirm the ICD‐9 diagnoses of PD, CD, and DM‐2. A total of 150 patient charts with ICD‐9 diagnoses of CD (333.83), PD (332 and 332.0), and DM‐2 (250*) were randomly selected for review. In querying the EMR, the following criteria were used for all three disease codes: (1) the ICD‐9 code of interest must be the primary diagnosis; (2) ages 18 and older; and (3) date range from the beginning of 2005 (when electronic billing began in our institution) until present day. In addition, for DM‐2, the 150 patients randomly selected from 29,000 candidates in the EMR, with ICD‐9 code of 250*, were evaluated against a hemoglobin A1c of ≥6.5. All the records were obtained from our institution's EMR system. The study was determined to be exempt by the University of New Mexico Human Research Review Committee.
After identifying the 150 charts for each ICD‐9 code of interest, a manual chart review was performed to confirm the ICD‐9 diagnosis. For CD, all clinical documentation in the EMR was evaluated and compared with the clinical criteria for dystonia proposed by Albanese and Lalli.7 Likewise, for PD, the UK Parkinson's disease Society Brain Bank Clinical Diagnostic Criteria was compared to the available documentation through the complete EMR chart.8 Finally, as described earlier, the EMR was queried for the ICD‐9 code for DM‐2 and was cross‐referenced with hemoglobin A1c ≥ 6.5 to confirm the diagnosis of DM‐2.9 The statistical analysis of the accuracy rates found for these ICD‐9 codes was performed using Fisher's exact test.
Results
Initially, 150 charts each for PD, CD, and DM‐2 were included in the study for a review of a total of 450 charts. However, 21 charts (PD patients) and 8 charts of patients with CD were excluded because of a lack of sufficient documentation to comment about the accuracy of these diagnoses. In these cases, there was no direct documentation (e.g, progress notes, history and physical notes, or clinic notes) that addressed the coded diagnoses, and these charts were excluded from further review.
A total of 421 charts (n = 129, n = 142, and n = 150 for PD, CD, and DM‐2, respectively) were reviewed. The accuracy rate was different among all diseases examined with an overall P < 0.001. In post hoc pairwise comparisons, the accuracy of DM‐2 diagnosis by ICD‐9 (96.6%) was greater than CD (88.0%), and both were greater than PD (55.0%) (P ≤ 0.003) (Table 1). The accuracy rate of diagnosis was different among all diseases examined with an overall P < 0.001. In post hoc pairwise comparisons, the accuracy of DM‐2 diagnosis by ICD‐9 (96.6%) was greater than CD (88.0%), and both were greater than PD (55.0%) (P ≤ 0.003).
Table 1.
Diagnostic Accuracy based on ICD‐9 Diagnosis and Confirmed through Chart Reviewa
| DIAGNOSIS | DM (%) | CD (%) | PD (%) | Total (%) |
|---|---|---|---|---|
| Diagnosis confirmed | 145 (96.6) | 125 (88) | 71 (55) | 341 (81) |
| Diagnosis not confirmed | 5 (3.3) | 17 (12) | 58 (45) | 80 (19) |
| Totals | 150 | 129 | 142 | 421 |
Raw number of charts listed with percentage in parentheses.
DM, diabetes mellitus; CD, cervical dystonia; PD, Parkinson's disease.
Discussion
In this study, we showed that ICD‐9 diagnoses for disorders such as DM‐2 with clearly defined laboratory criteria were more accurate compared with clinically diagnosed movement disorders, such as CD and PD. We were able to identify novel information about the accuracy rate of CD, which had a high diagnostic accuracy rate in our study. We were able to confirm the findings seen previously that the ICD‐9 code for parkinsonism was ambiguous, combining both PD and parkinsonism.4
As coding systems such as ICD‐9 and ICD‐10 were designed primarily as administrative databases for use in medical billing and so forth, the utility of using EMR‐based queries based on ICD is of uncertain validity in many diseases. In a review, the validity data on common neurologic diagnoses were documented.10 The authors looked at PD/parkinsonism and discovered the same ambiguity seen in our study with a positive predictive value ranging from 38.6 to 81.0.10 The authors concluded that the interpretation of EMR or other health database studies is dependent on the accuracy of the case definition in that particular database.10 And, for diseases with clinically based diagnosis without a confirmed laboratory test, the accuracy rate might vary greatly depending on the database and coding system used.
CD exhibited a high rate of ICD‐9 accuracy in our database—although 12% of cases still did not meet diagnostic criteria on chart review. The higher accuracy rate may be a result of its relative rarity in neurologic disease overall, but with dystonia as the third most common diagnosis seen in subspecialty movement disorders clinics, the diagnosis of CD was entered into our EMR predominantly by movement disorders specialists. This was in contrast to PD, which had a wide range of neurologists and nonneurologists assigning the code for PD.
Limitations of this study include the retrospective nature of the study and the dependency on documentation to evaluate for the accuracy of diagnosis. Both the quality and thoroughness of the documentation are provider dependent, and, as such, this may bias our accuracy rates especially for the PD patients. Another limitation is that there are other candidate control conditions that could be looked at that are typically made by clinical diagnosis but have diagnostic tests for confirmation, such as peripheral neuropathy, which may have added contrast to our findings. Further, the United States will be converting to the use of ICD‐10 diagnostic codes in the near future. This may certainly allow for increased granularity in diagnoses such as PD; however, clinically diagnosed neurologic diseases will still depend on physician expertise and optimal documentation to see increased accuracy in the new system. Finally, these coding accuracy rates are valid for our institution and our EMR, and we do not know the generalizability of this finding across sites and other EMR systems.
Despite these limitations, these findings contribute to the validation of neurologic disease in administrative databases. Based on our results, ICD‐9–based screening of clinically diagnosed diseases such as CD might be more accurate than previously thought.
Author Roles
1. T.T.: Research project: C. Execution; Statistical analysis: C. Review and Critique; Manuscript: A. Writing of the first draft, B. Review and Critique; 2. A.R.W.: Research project: C. Execution; Statistical analysis: C. Review and Critique; Manuscript: B. Review and Critique; 3. S.P.R.: Research project: A. Conception, B. Organization C. Execution; Statistical analysis: A. Design, B. Execution, C. Review and Critique; Manuscript: B. Review and Critique.
Disclosures
Funding Sources and Conflicts of Interest: This study was supported (in part or in full) by National Center for Research Resources and the National Center for Advancing Translational Sciences of the National Institute of Health through Grant Number KL2 TR0000089 and Grant Number UL1 TR0000041. The authors report no conflicts of interest.
Financial Disclosures for Previous 12 Months: The authors declare that there are no additional disclosures to report.
Acknowledgment
We thank Clifford Qualls, PhD, for his statistical expertise and Dr. Mark Burge for his assistance with diagnostic codes for diabetes. We also thank Ms. Laurie McPherson for her expertise with the electronic medical record database.
Relevant disclosures and conflicts of interest are listed at the end of this article.
References
- 1. Sully B, Julious S, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials 2013;14:166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. De Moor G, Sundgren M, Kalra D, et al. Using electronic health records for clinical research: the case of the EHR4CR project. J Biomed Inform 2015;53:162–173. [DOI] [PubMed] [Google Scholar]
- 3. Pfiffner PB, Oh J, Miller TA, et al. ClinicalTrials.gov as a data source for semi‐automated point‐of‐care trial eligibility screening. PLoS One 2014;9:e111055. doi: 10.1371/journal.pone.0111055. eCollection 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Swarztrauber K, Anau J, Peters D. Identifying and distinguishing cases of parkinsonism and Parkinson's disease using ICD‐9 CM codes and pharmacy data. Mov Disord 2005;20:964–970. [DOI] [PubMed] [Google Scholar]
- 5. Geraci JM, Ashton CM, Kuykendall DH, et al. International classification of diseases, 9th revision, clinical modifications codes in discharge abstracts are poor measures of complication occurrence in medical inpatients. Med Care 1997;35:589–602. [DOI] [PubMed] [Google Scholar]
- 6. Defazio G, Jankovic J, Giel JL, Papapetropoulos S. Descriptive epidemiology of cervical dystonia. Tremor Other Hyperkinet Mov (NY) 2013;3: tre‐03‐193‐4374.2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Albanese A, Lalli S. Is this dystonia? Mov Disord 2009;24:1724–1731. [DOI] [PubMed] [Google Scholar]
- 8. Hughes AJ, Ben‐Shlomo Y, Daniel SE, et al. What features improve the accuracy of clinical diagnosis in Parkinson's disease: a clinicopathologic study. Neurology 1992;42:1142–1146. [DOI] [PubMed] [Google Scholar]
- 9. American Diabetes Association . (2) Classification and diagnosis of diabetes. Diabetes Care 2015;38(Suppl):S8–S16. [DOI] [PubMed] [Google Scholar]
- 10. St. Germaine‐Smith C, Metcalfe A, Pringsheim T, et al. Recommendations for optimal ICD codes to study neurologic conditions: a systematic review. Neurology 2012;79:1049–1055. [DOI] [PMC free article] [PubMed] [Google Scholar]