Minimizing electronic health record patient-note mismatches

Abstract

We measured the prevalence (or rate) of patient-note mismatches (clinical notes judged to pertain to another patient) in the electronic medical record. The rate ranged from 0.5% (95% CI 0.2% to 1.7%) before a pop-up window intervention to 0.3% (95% CI 0.1% to 1.1%) after the intervention. Clinicians discovered patient-note mismatches in 0.05–0.03% of notes, or about 10% of actual mismatches. The reduction in rates after the intervention was statistically significant. Therefore, while the patient-note mismatch rate is low compared to published rates of other documentation errors, it can be further reduced by the design of the user interface.

Introduction

The nation-wide adoption of electronic health records through programs like HITECH¹ makes it important to understand both the unintended consequences of clinical utilization of these systems as well as the paper record vulnerabilities that persist in electronic record systems. A number of unintended consequences have been identified,^2–4 but there is little knowledge of the rate at which such events occur.⁵ A better knowledge of rates would allow designers and users to prioritize their response to these events, which will likely consist of studies, redesign, and training.

We addressed a particular health-record-related adverse event, the presence of a patient note in another patient's record, which we will refer to here as a patient-note mismatch. Patient-note mismatches can potentially lead to errors in diagnosis or therapy if another health worker reads the note and takes an action that is inappropriate for the patient. There is a risk of patient-note mismatches with paper records, and the problem may be increased or decreased with electronic health records. There is currently little in the literature on this particular adverse event.

Based on our use of electronic health records at a teaching hospital, we estimated the rate of patient-note mismatches. We implemented an intervention in the middle of the study period that was intended to minimize the mismatch rate, and we measured its effect.

Case description

We estimated the prevalence of patient-note mismatches in two steps: first we quantified the rate of clinician-discovered mismatches between the note and the patient record, and then we inferred the total rate of mismatches by looking at inconsistencies between notes and the rest of the patient record. While the clinician-discovered mismatches were likely to underestimate the total mismatch rate, they provided a subset from which we could estimate correction factors for the full sample, and they provided a directly measurable parameter that could be tracked to infer changes in the overall rate.

During the study period, clinicians used a clinical information system to write electronic notes. If a clinician realized or believed that a note (usually their own note) was present in the electronic health record pertaining to another patient, he or she reported it via a feedback button, and the note was blocked from view by information technology staff (users could not block notes by themselves). We measured the rate at which clinicians reported patient-note mismatches.

To estimate the total rate of patient-note mismatches including those which were unreported, we could not rely on straightforward manual review because the expected rate was low. We therefore created an enriched sample to estimate the rate. We noted that cases where note and the electronic health record's patient demographics differ in gender will occur frequently in patient-note mismatches but rarely in the general population. That is, if gender mismatch and patient-note mismatch are uncorrelated, then about half the patient-note mismatches should have the wrong gender. There are two effects that need to be accounted for, however. First, a user may be more likely to notice a mismatch between the true subject of the note and the health record if the gender does not match. Second, the note may be on the right patient but simply have a gender error in the note or in the patient record (or, less frequently, there could be a true gender change). Therefore, we used the set of clinician-discovered mismatches to estimate the correction from the gender mismatch rate to the patient-note mismatch rate.

During the study period, we introduced an intervention to minimize the patient-note mismatch rate. Before December 2007, the user interface showed the patient's name, gender, birthdate, and medical record number at the top of the screen that was used to write the note, and the user completed the note by clicking a submit button. In December 2007, a pop-up window was added to reiterate the patient's name and medical record number (figure 1).

Figure 1.

Intervention pop-up window. The top of the note editing screen always displayed the medical record number, name, gender, and date of birth. The intervention added a pop-up window (center) that reiterated the name and medical record number when a note was submitted. (A fictional patient is shown.)

Method of implementation

Our detailed methods for each step are as follows. For clinician-discovered mismatches, we reviewed note block requests from January to October 2007, and from January to October 2008 (November–December 2007 was unavailable). These two cohorts were sampled and reviewed manually to determine the rate at which the reason for the request was a patient-note mismatch.

To estimate the full mismatch rate, we analyzed all admission notes written between 2001 and 2009 (not just those that were reported). We chose admission notes because a high proportion of them are likely to specify gender. We developed a regular expression parser to extract the gender of the patient from the notes and compared that gender to what was coded in the health record. Those whose gender did not match were considered parser positive for gender mismatch, denoted G. For the admission notes that were identified by clinicians as having a patient-note mismatch, we estimated the following:

$$\text{P}\left(\text{G}|\text{I}\cap \text{A}\right)\text{,}$$ (1)

where I signifies a clinician-identified patient-note mismatch, and A signifies an admit note. We made the assumption that the probability in equation 1 was approximately equal to the corresponding probability for all patient-note mismatches, denoted M, where I ⊂ M:

$$\text{P}\left(\text{G}|\text{M}\cap \text{A}\right)\approx \text{P}\left(\text{G}|\text{I}\cap \text{A}\right)$$ (2)

That is, we assumed that the clinician-identified set was not biased with respect to gender mismatches. We sampled the set of all parser positive cases, G, and estimated the probability of a patient-note mismatch:

$$\text{P}\left(\text{M}|\text{G}\cap \text{A}\right)$$ (3)

We counted the number of parser positive cases out of all admission notes that we attempted to parse in order to estimate the following probability:

$$\text{P}\left(\text{G}|\text{A}\right)$$ (4)

The desired probability of a patient-note mismatch in an admission note can then be derived from the probabilities in equations 2–4, and equation 1 estimates 2:

$$\begin{array}{l}\text{P}\left(\text{M}|\text{A}\right)=\frac{\text{P}\left(\text{M}\cap \text{A}\right)}{\text{P}\left(\text{A}\right)}\\ =\frac{\text{P}\left(\text{M}\cap \text{G}\cap \text{A}\right)}{\text{P}\left(\text{G}\cap \text{A}\right)}\frac{\text{P}\left(\text{G}\cap \text{A}\right)}{\text{P}\left(\text{A}\right)}\frac{\text{P}\left(\text{M}\cap \text{A}\right)}{\text{P}\left(\text{M}\cap \text{G}\cap \text{A}\right)}\\ =\text{P}\left(\text{M}|\text{G}\cap \text{A}\right)\times \text{P}\left(\text{G}|\text{A}\right)÷\text{P}\left(\text{G}|\text{M}\cap \text{A}\right)\\ \approx \text{P}\left(\text{M}|\text{G}\cap \text{A}\right)\times \text{P}\left(\text{G|A}\right)÷\text{P}\left(\text{G}|\text{I}\cap \text{A}\right)\end{array}$$ (5)

Thus the patient-note mismatch rate for admission notes is estimated by a combination of measurable parameters. Note that by using this approach, it is not necessary for all patient-note mismatches to have a gender mismatch or for all gender mismatches to have a patient-note mismatch. The gender mismatches merely supply an enriched (but imperfect) sample that can be used to estimate the patient-note mismatch rate.

To assess the effect of the intervention, we assessed the change in rate of clinician-discovered mismatches and the change in the estimated rate of gender mismatches for January–October 2007 versus January–October 2008. The institutional review board approved the study.

Example and observations

Table 1 shows the number of clinician feedback emails in the two 10-month periods in 2007 and 2008. Approximately half were sampled, and about half of those were note block requests for patient-note mismatches. The rates of clinician-reported patient-note mismatches per note written were 0.0005 and 0.0003 for 2007 and 2008, respectively, and the difference was statistically significant. The gender mismatched on 30% of patient-note mismatches, and it did not differ significantly between the two periods.

Table 1.

Clinician-initiated note block requests

	Jan–Oct 2007	Jan–Oct 2008	Overall	Significance 2007 vs 2008
Number of emails	313	219	532
Number sampled	137	106	243
Number of patient-note mismatches	74	51	125
Total notes	324857	358921	683778
Reported patient-note mismatch rate (95% CI)	0.00051 (0.00037 to 0.00060)	0.00029 (0.00021 to 0.00038)	0.00040 (0.00033 to 0.00047)	p<0.004

From 2001 to 2009, there were 199 158 admission notes, and 126 186 were successfully parsed; 375 were parser positive for gender mismatch. We found that the parser had a recall of 60% and a precision of 98% for finding true gender mismatches (these numbers were not used in the calculations below because the parser merely created an enriched set). There were 16 admission notes in the set of clinician-discovered patient-note mismatches, three of which were parser positive for gender mismatch. Fifty-seven of the 375 parser positive cases were sampled, and 25 were found to be patient-note mismatches. Therefore, using equation 5, the overall patient-note mismatch rate was as follows (95% CI in parentheses):

$$\begin{array}{l}\text{P}\left(\text{M}|\text{A}\right)\approx \text{25}/\text{57}\text{*}375/199158\text{*}16/\text{3}\\ =0.0044\left(0.0019-0.0155\right)\end{array}$$

We repeated the analysis using the parser positive gender mismatch rates for 2007 and 2008, where A₂₀₀₇ and A₂₀₀₈ signify the admission notes in the years 2007 and 2008, respectively:

$$\begin{array}{l}\text{P}\left({\text{M}|\text{A}}_{2007}\right)\approx 25/57\text{*}87/41300\text{*}16/3\\ =0.0049\left(\text{0.0021}-\text{0.0173}\right)\end{array}$$

$$\begin{array}{l}\text{P}\left({\text{M}|\text{A}}_{2008}\right)\approx 25/57\text{*}58/43080\text{*}16/3\\ =\text{0.0031}\left(\text{0.0014}-\text{0.0111}\right)\end{array}$$

The rates for 2007 and 2008 differed significantly at p<0.001 (the individual confidence intervals are wide, but the differences are significant because the numbers are correlated). Therefore, both the clinician-reported mismatch rate (p<0.004 in table 1) and the gender mismatch rate fell after the intervention. Figure 2 shows the monthly rate before and after the intervention both for clinician-reported mismatches and for gender mismatches. Even after a year, neither measure appears to return to baseline.

Figure 2.

Monthly rate of mismatches before and after intervention. The rate of clinician-reported patient-note mismatches (block note requests) and the estimated rate of gender mismatches (parser positive rate) are shown for the months preceding and following the intervention.

Given the estimated total patient-note mismatch rate and the rate of clinicians reporting mismatches, one can calculate the proportion of mismatches discovered by clinicians. It was approximately 10%.

Discussion

Our study demonstrates that our patient-note mismatch rate ranged from 0.5% to 0.3%, depending on the user interface. The confidence intervals were very wide because there were only 16 admission notes in the clinician-discovered set of patient-note mismatches. While we could not find analogous patient-note mismatch rates reported in the literature, we found rates related to documentation errors. Nurse documentation errors were found in 6% of documented procedures,⁶ 1.3% of anesthetic records had documentation errors that prevented billing,⁷ 2.0% of radiation treatments had a documentation error,⁸ and 62% of neonatal ICU resident documentation had some kind of error.⁹ Therefore, the patient-note mismatch rate appears to make a small contribution to the total error rate. In the neonatal ICU resident documentation study, there was no mention of a patient-note mismatch in the 339 notes that were reviewed⁹; the 95% CI of this result (0 of 339) is 0 to 1.1% and thus corroborates our findings.

We also demonstrated that a pop-up window reduced the patient-note mismatch rate by about 40%, as evidenced by the drop in the gender mismatch rate (37%) and in the clinician-reported patient-note mismatches (43%). This confirms that user interface design is important and can affect error rates substantially. While we do not know the patient-note mismatch rate in paper records, we do know that pop-up window interventions are only possible in the electronic record.

We did not determine the cost-benefit tradeoff of the pop-up window intervention, in terms of user time spent per note corrected. Assuming 1 s to clear the pop-up window, about 8 min of user time are required to avoid one patient-note mismatch. During the pre-intervention period of the study, users simply submitted a note by clicking a submit button but did not receive a pop-up window. Now that notes are signed by re-entering a password in a pop-up window anyway, the marginal cost of adding the patient identification information to that window is minimal.

The pop-up window may have two components important to reducing the mismatches. First, it interrupted the user action enough to improve the review of information, and second, it made the information more accessible to the user. In our system, documentation tasks are separate from data review tasks in the user interface. However, more advanced, workflow-friendly documentation systems that are integrated into the data review may be more effective at reducing patient-note mismatches. These systems would have more complete information about the patient accessible to the user during note creation, and be better integrated into the workflow so as to not require interruption to access that information. We are currently investigating how such systems could be used to improve the overall documentation process.

While the patient-note mismatch rate is low, it may be possible to reduce it further. For example, on-the-fly natural language processing can detect inconsistencies related to gender and age and warn the user. A patient photograph in the pop-up window might trigger the user's recognition that it is the wrong patient.

This study has several limitations. First, it was carried out at a single teaching hospital on a single clinical information system. The rate is somewhat sensitive to the design of the electronic health record, as demonstrated by the effect of the intervention. Nevertheless, the estimated rates (before and after intervention) offer a rough range that is likely to be near those of other electronic health records, as long as they do not use extraordinary measures (eg, enforce barcode matching for each note, or pop up the patient's photograph).

Second, the patient-note mismatch rate is based on estimates from clinician-discovered errors. The discovery process may have its own unique gender mismatch rate, rendering the assumption invalid. For some clinical specialties that are at least somewhat gender-specific (eg, gynecology or urology), the gender rate assumptions may not be correct, and this will limit the ability to detect mismatches by comparing gender. Similarly, we note that the term in equation 3 may differ by year because it is a posterior probability for mismatches and sensitive to mismatch prevalence. We had insufficient data to estimate it by year, but we note that the bias is toward the null hypothesis (no change due to intervention), so our aggregation across years will tend to yield a more conservative result, and the true effect of the intervention may be higher. This does not affect the estimated overall rate.

Third, the rate for admission notes may differ from that for other notes. We believe, however, that our estimates at least approximate the overall rate. Finally, our method is convenient for patient-note mismatches, but it is not generalizable to most other unintended consequences of electronic health records. Extensive manual review is likely to be needed to vastly expand knowledge of error rates.

In conclusion, our study demonstrates that the prevalence of finding a note in another patient's record, referred to here as the patient-note mismatch rate, is approximately 0.3%. We have demonstrated that the rate depends significantly on the design of the user interface.