The Impact of Documentation Workflow on the Accuracy of the Coded Diagnoses in the Electronic Health Record

Abstract

Objective

To determine the impact of documentation workflow on the accuracy of coded diagnoses in electronic health records (EHRs).

Design

Cross-sectional study.

Participants

All patients who completed visits at the Casey Eye Institute Retina Division faculty clinic between April 7, 2022 and April 13, 2022.

Main Outcome Measures

Agreement between coded diagnoses and clinical notes.

Methods

We assessed the rate of agreement between the diagnoses in the clinical notes and the coded diagnosis in the EHR using manual review and examined the impact of the documentation workflow on the rate of agreement in an academic retina practice.

Results

In 202 visits by 8 physicians, 78% (range, 22%–100%) had an agreement between the coded diagnoses and the clinical notes. When physicians integrated the diagnosis code entry and note composition, the rate of agreement was 87.9% (range, 62%–100%). For those who entered the diagnosis codes separately from writing notes, the agreement was 44.4% (22%–50%, P < 0.0001).

Conclusion

The visit-specific agreement between the coded diagnosis and the progress note can vary widely by workflow. The workflow and EHR design may be an important part of understanding and improving the quality of EHR data.

Financial Disclosure(s)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

The use of large-scale, real-world data holds promise to improve our understanding of a wide range of conditions, both rare and common.¹ However, concerns about the quality of data from electronic health records (EHRs) may limit the validity and acceptance of these studies.² For example, a recent paper highlighted how institutions spend significant resources manually verifying and adjusting the coded diagnoses for quality measurement and reimbursement to avoid possible financial penalties.³ If the EHR data as-is is not good enough to make accurate inferences about quality outcomes, it is possible that it is not good enough for research.

The accuracy of coded diagnoses in the EHR is of fundamental importance in Big Data research,^1,4 and there is a need to understand the factors that influence it. Studies of EHR data quality have typically assessed the usefulness of the data for specific use cases, such as disease surveillance,⁵ and predictive models for specific outcomes.⁶ Some studies have assessed the accuracy of coded data,⁷ but as this often requires a manual review,⁸ there are few studies assessing the concordance between the coded data and the clinical documentation or the factors that influence it, such the physician workflow and EHR design.⁹ These factors, if associated with improved data quality, may be helpful in informing the best practices for EHR use and design. This study examines the encounter-specific agreement between the International Classification of Disease 10 (ICD10) diagnoses in the EHR and the diagnoses found in the progress note of an academic retina clinic and its relationship to the documentation workflow.

Methods

This study adhered to the Declaration of Helsinki, and the Oregon Health and Science University’s Institutional Review Board approved the study. A requirement for participants' consent was waived by the Institutional Review Board. A reviewer (T.S.H.) extracted the diagnoses from the progress notes of visits to the faculty retina clinic at the Casey Eye Institute between April 7, 2022, and April 13, 2022, and compared them to the ICD10 diagnosis codes in the Epic EHR system from the same visits. In addition to agreement with the progress note, the diagnosis codes were assessed for the appropriate specificity, severity, and laterality according to the Center for Medicare and Medicaid Services guidelines,¹⁰ which stipulate that (1) coded diagnoses are to reflect documented diagnoses or the reason for visit; (2) if a diagnosis exists that explains the reason for visit, it should be recorded instead of the symptom; (3) although nonspecific codes, such as retinal edema, are allowed when there is clinical uncertainty, the most specific clinically documented code should be used when possible; and (4) laterality and severity should be specified when clinically known and available for the specific code.

The reviewer then categorized the agreement between the progress note and the diagnosis codes into TRUE or FALSE. The reasons for disagreement were categorized as follows: (1) wrong severity (e.g., severe nonproliferative diabetic retinopathy was coded as mild nonproliferative diabetic retinopathy), (2) wrong or missing laterality, (3) incorrect codes unrelated to any diagnosis in the note, (4) missing codes (a documented diagnosis was not coded), and (5) nonspecific codes, meaning a nonspecific code, such as H35.81 (retinal edema), was used when a more specific diagnosis was evident in the progress note, such as branch retinal vein occlusion with macular edema.

The documentation workflows were categorized into integrated charting, in which the physician used the coded diagnoses to create the narrative section of the note, and independent charting, in which the physician entered the diagnosis codes separately from composing the note. Integrated charting includes problem-based charting, which creates the progress note by compiling narratives notes attached to specific diagnoses in the problem list section and using a function within Epic called “SmartLink” that pulls the diagnoses selected for the encounter as the starting point to create the assessment and plan notes. We compared the rate of agreement between integrated and independent charting using Generalized Estimating Equations logistics regression models with exchangeable correlation matrices.

Results

The study included 202 encounters with 8 physicians. The physicians were on the same EHR system for an average of 12.1 years (range, 6–16 years). The encounters included 388 unique ICD10 codes. Age-related macular degeneration-related codes (H35.30-H35.32XX) were the most frequent (n = 77), followed by diabetic retinopathy-related codes (E10.3XXX and E11.3XXX, n = 50).

Of these encounters, 158 (78.2%) had accurate ICD10 codes that reflected the diagnoses listed in the progress note with the specificity that met the Center for Medicare and Medicaid Services standard. Of the 44 encounters with mismatches, 15 (34.1%) were because of wrong disease severity, 15 (34.1%) were because of missing codes, 5 (11.4%) were because of incorrect codes, 5 (11.4%) were because of the use of nonspecific codes, and 4 (9.1%) were because of wrong laterality.

Per physician (Fig 1), the rate of agreement ranged from 22.2% to 100%. Of the 8 physicians, 6 physicians used integrated charting, of which 5 used the problem-based charting function, and 1 used a SmartLink that pulled the coded visit diagnoses. Two of the 8 physicians used an independent workflow. Those that used the integrated charting method had an agreement rate of 87% (range, 62%–97%), and those that used independent charting had a match rate of 44% (range 22%–50%; P < 0.0001).

Figure 1.

Agreement rate per physician. Physicians 1-5 used problem-oriented charting (green), physician 6 used the visit-diagnosis SmartLink (blue), and physicians 7-8 used independent charting (orange).

Discussion

Our study found that the quality of coded diagnosis data using the Center for Medicare and Medicaid Services standard varied widely from physician to physician. The visit-level agreement between coded diagnoses and the progress notes was significantly better when physicians used integrated charting by using the coded diagnoses to compose the progress note. This finding suggests that the EHR coding errors may be distributed in a nonrandom fashion, and the understanding of the workflow and EHR design may be important in evaluating the data quality.

To our knowledge, this study is the first to suggest that a certain EHR design or documentation workflow could affect the data quality in the outpatient setting. Li et al⁹ reported that problem-based charting can improve the quality of coded data in the EHR in the inpatient setting. Electronic health records systems offer variable levels of integration of structured data entry and note composition. Epic, the system studied here, offers a variety of available workflows with different levels of constraints in documentation, including problem-oriented charting, SmartLink-based charting, as well as free-form progress notes. Other systems, such as the Computerized Patient Record System that the Veterans Administration hospitals use, rely mostly on free-text progress notes. In contrast, NextGen’s EHR, by default, requires integrated charting, in which coded diagnoses are the required headers for the assessment and plan section. These user-interface level design choices could affect the quality of data in different systems and should be considered when assessing the data quality.

We found that an integrated workflow was associated with a higher rate of agreement but did not ensure it. Mismatches with integrated workflow occurred when physicians documented additional or contradictory diagnoses in the narrative sections of the note. This reflects the reality of individual physician practices and the nature of the integrated workflow, which requires more “clicks” to enter the discrete diagnoses compared with simply typing in the narrative. However, there are times when the available diagnosis codes do not sufficiently capture the clinical uncertainty or the interplay between different conditions. Some flexibility may be necessary for good clinical documentation, even if it comes at the cost of data quality.¹¹

One reason for the heterogeneity of workflows is the lack of defined best-practice patterns for EHR documentation and coding. Outside of critiques of recognized undesirable practices, such as copy forward¹² and note bloat,¹³ few EHR best-practice guidelines exist. Although Weed’s¹⁴ classic discourse on problem-oriented charting outlines the ideal of “medical records that guide and teach,” this is not universally accepted as the best practice because there are disagreements about how to address its shortcomings, such as working with multiple related problems and diagnostic uncertainty.¹⁵ In addition, problem-oriented charting per se does not necessarily address data quality for secondary use because the principles of problem-oriented charting were used by those physicians who use free-form progress notes with separate coding practices. It appeared that the specific workflow that integrates the diagnostic code entry with composition of the note was essential to improve the data quality.¹⁶ Furthermore, other goals of EHR best practices, such as physician wellness,¹⁷ regulatory compliance, efficiency, and communication with patients and other providers¹⁸ may not be met with this approach. Evidence-based EHR best-practice guidelines that balance these goals are needed.

Another possible reason for disagreement between coded diagnosis and the progress note is the use of copy forward. It is a common practice to copy forward the assessment and plan section from visit to visit and use it as a “parking lot” for relevant historical information, even if not all of the issues on the list are addressed at each visit. The physician may only code the diagnoses that they addressed in the visit, but may not make it clear in the note that there were diagnoses that were not addressed, resulting in discrepancy. However, even with a longitudinal review of the notes, it may not be possible to determine what the physician’s intent was unless the note was specifically edited to indicate which of the copy-forwarded diagnoses were addressed at the visit. This, in addition to avoiding the propagation of incorrect information, may be another good reason to discourage this practice.

This study is limited by its small size and nonrandomized, retrospective design. There may also have been individual physician-specific factors unrelated to workflow, such as familiarity with and commitment to coding requirements and facility with the system that influenced the results of the study. However, all the clinical practices had similar patient mix and complexity. The physicians all had used the same system for ≥ 6 years, minimizing the effect of experience and training. Also, the study relied on a single reviewer to determine the accuracy of the diagnosis. Although the rubric was straightforward, it nevertheless requires some judgment. This may have made the evaluations more subjective than a study performed with multiple reviewers. We also did not examine if the integrated workflow, which is more structured, could have had a negative impact on the efficiency and wellness of the physician.¹⁹ Also, the study was not designed to detect any differences in patient outcomes. Finally, this study was conducted in a single EHR system in a single subspecialty clinic in an academic center. A study examining the effect of workflow on data quality across EHR systems and different subspecialties is necessary to generalize our findings.

In conclusion, there is considerable variability in agreement between the progress note and the coded diagnostic codes. The physicians that used an integrated documentation workflow had a significantly higher rate of agreement between the notes and the coded diagnoses. The EHR workflow and design may be an important part of understanding and improving the quality of EHR data.

Manuscript no. XOPS-D-23-00193R1.