Critical Gaps in the World’s Largest Electronic Medical Record: Ad Hoc Nursing Narratives and Invisible Adverse Drug Events

Abstract

The Veterans Health Administration (VHA), of the U.S. Department of Veteran Affairs, operates one of the largest healthcare networks in the world. Its electronic medical record (EMR) is fully integrated into clinical practice, having evolved over several decades of design, testing, trial, and error. It is unarguably the world’s largest EMR, and as such it makes an important case study for a host of timely informatics issues. The VHA consistently has been at the vanguard of patient safety, especially in its provider-oriented EMR. We describe here a study of a large set of adverse drug events (ADEs) that eluded a rigorous ADE survey based on prospective EMR chart review. These numerous ADEs were undetected (and hence invisible) in the EMR, missed by an otherwise sophisticated ADE detection scheme. We speculate how these invisible nursing ADE narratives persist and what they portend for safety re-engineering.

INTRODUCTION

Patient safety has long been on the minds of clinicians and managers at the Veterans Health Administration. Years before the widely acclaimed Institute of Medicine Report, To Err is Human, was released in 1999¹ the VHA had active grant solicitations for studies in patient safety, and had already laid the groundwork for the VA’s National Center for Patient Safety (NCPS), the most comprehensive national network of safety coordinators in healthcare today. Our research team in Salt Lake was interested in establishing a local baseline rate for adverse drug events (ADEs) so that we could better assess future patient safety initiatives aimed at reducing this common safety problem.

At the outset we thought that establishing the ADE incidence baseline would be a simple thing. We turned to the VA for internal statistics and were surprised to find that in a survey of sixty-six VA medical centers conducted in 1996, ADE incidence (by percent of admissions) varied from less than 3% to more than 18%.² We suspected that these incidences might be a VA anomaly. We surveyed the general ADE literature back well into the 1980s, and found an even more discordant picture: ADE incidence (again by percent of admission) varied from less than 1%³ to well over 32%.⁴ Regrettably, there was no consistent definition of terms (even for obvious terms like adverse drug event or adverse drug reaction), let alone any consistency in the methods used to detect and classify the events described in these studies. So we mounted our own multiple-method detection and classification study of inpatient ADEs.

That work and its results are described in detail elsewhere (submitted to The Lancet). The important point for the present discussion is that we found prospective chart review by clinical pharmacists to be the most sensitive means of detecting ADEs. This reaffirms previous reports.^5–7 By contrast patient interviews, ICD9 codes, and spontaneous reports were of marginal comparative value. All providers at our center hand type their own notes and use provider order entry to generate drug and other orders, so our prospective chart review relied on the EMR. The pharmacists conducting the review had access to progress notes (all disciplines), inpatient and outpatient medication orders, allergy lists, problem lists, consult reports, laboratory values, radiology and other diagnostic evaluations, and eventually the discharge summary, all on-line. In short, we took full advantage of the world’s largest EMR and of the information it put at our fingertips.

While conducting that study, our pharmacist reviewers began noticing that some critical information for confirming the causality for certain types of ADEs was coming from nursing notations made in the bar coded medication administration record (a system used to match a patient’s coded wristband to a coded unit dose medication dispensed by the pharmacy). These notes were short, cryptic notations made in the dispensing record, and they were available only to a limited number of users (i.e., dispensing nurses and pharmacists). Although electronic, they were not part of the clinical EMR used by clinicians and researchers (including by our study pharmacists). The discovery that there was important ADE diagnostic information not readily visible in the EMR started us off in search of other hidden narratives.

METHODS

To avoid confusion between the original ADE study and this follow-on study of nursing narratives, we will use the phrase ‘EMR-based’ study to refer to the original study.

When we finished the EMR-based study and turned to its analysis, we noted that the results were very physician-centric. Physicians were the most likely by far to be involved in describing or treating an ADE. For those ADEs that were caused in error, fully three-quarters were attributed to a physician. This gave us pause. The classic model for an inpatient pharmacy intervention runs something like this: patient manifests symptom or disease ? physician orders drug ? pharmacist prepares drug ? drug is delivered to floor nurse ? floor nurse verifies and administers drug to the patient. Given that nurses participate at key points in the drug delivery process, and then again in subsequent monitoring of the patient’s functional status after a drug is delivered, it seemed unlikely that the nurse would not be more intimately involved in detecting adverse events.

We assessed the source of ADE signals (i.e., the trigger that starts a pharmacist down the investigative trail of a suspected ADE). Looking just at progress note signals, nursing notes were the signal source 14.5% of the time, whereas physician notes were the source 80% of the time. These rates are comparable to other studies.⁸ We were convinced that vital nursing documentation relevant to ADE surveillance was not making its way into the EMR. So we assigned a clinical pharmacist from the EMR-based study to review all paper and electronic sources of nursing text related to ADEs, which we call collectively ‘nursing narratives.’

We defined two major narrative categories in addition to nursing progress notes:

• Incident Reports: the widely-accepted mechanism of officially logging incidents that resulted in real or potential harm to patients in the inpatient setting (these were not reviewed in the original EMR-based study).

• “Bedside” notes : all handwritten or electronic annotations that were made (while caring for the patient at the bedside) onto a medium not generally available to readers of the EMR (these were not reviewed in the original EMR-based study).

The study pharmacist systematically reviewed both of these categories using charts from the original EMR-based project. A second pharmacist would have provided a more reliable review, but it was beyond our means to hire one. Note that the study pharmacist did have a great of experience with ADE chart review. The clinical pharmacist was looking for new ADEs missed in the first study, but she applied the same ADE definition: “a unique event that is a discrete clinical syndrome associated a drug and that is significant if (1) it was an untoward medical occurrence AND (2) there was at least a possible causal link between event and pharmaceutical AND (3) it was linked to a change in treatment plan OR linked to a symptom/sign noted in the medical record.” This is an operationalized version of the current WHO ADE definition.⁹ It was augmented by a table of dozens of specific inclusion and exclusion criteria meant to eliminate the guess work for special cases like titration effects, treatment failure, laboratory parameters (e.g., what level of potassium constitutes hypokaelmia?), etc. The virtues of using a rigorous and explicit definition for ADEs was described in a previous article.¹⁰

As in the EMR-based study, all suspected ADEs brought forward by the pharmacist were verified by a multidisciplinary committee composed of two MDs, two PharmDs, one PhD nurse, and a Masters-level study coordinator who had also conducted the patient interviews in the EMR-based study.

RESULTS

Incidence Reports

Nurses filed 179 incidence reports (IRs) for the 2,306 admits covered in the original study period (the last 20 weeks of 2000), for a crude incidence of 78 per 1,000 admissions. Of these, 19 were related to an ADE under our definition (10.6% of all the IRs, 8.2 per 1,000 admissions). Of the 19 ADEs described in the incident reports, 16 were novel. They had not been detected in the EMR-based study. So 9% of all incident reports contained ADE-signals not otherwise detected in the EMR . If the IR-based ADEs had been included in the original EMR-based study, the overall incidence would have risen about 3%.

The utility of IRs for ADE surveillance is controversial. Some authors believe IRs make poor ADE signals,¹¹ while others find that they are quite useful.¹² Our study shows a modest contribution, but clearly this will vary according to local nursing culture.

Bedside Notes

Due to constraints on time, the pharmacist investigator only reviewed 303 charts from the EMR-based study, essentially her half of the original study charts minus those charts that we were unable to physically locate. This category provided a fascinating glimpse into the variety of nursing narratives that are normally invisible in the EMR. The various narratives types are shown in Table 1.

Table 1.

Types of Handwritten “Bedside” Nursing Narratives

Narrative Types (all are paper documents unless noted*)	Number of Novel ADEs (not already found in EMR-based study)
Medical floor flow sheets	20
Surgical floor flow sheets	71
All ICU flow sheets	50
	(Telemetry 12, MICU 15, SICU 23)
Recovery room	4
Bar code medication administration comment*	12
Medication administration record	5
Code Blue sheet	1
Rehab evaluation form	0
ER nursing assessment	0
RN admission assessment	5
MD admission certificate	2
Patient-controlled analgesia note	1
Combinations (two or more documents together comprise signal)	15
Total	186

By way of comparison, the EMR-based study found 406 ADEs in the same set of charts. Of those, 109 overlapped the nursing bedside narratives, while 186 were novel. That means that the overall ADE incidence would have risen by 31% if the bedside narratives had been included.

The bedside narratives largely took the form of scribbled annotations on flow sheets or very brief clinical notes on special-purpose forms (like the ER assessment form). They were used to convey important but fragmentary information between shifts or as self-reminders . By themselves, they were rarely rich enough to allow the pharmacist to directly infer an ADE, but they did provide a signal that frequently lead to a verified event. Unfortunately, we did not log bedside narratives that proved to be false positives, so the overall predictive value of bedside narratives is unknown. Table 2 shows several example narratives.

Table 2.

Sample Bedside Narratives

Narrative Types	Text Samples
Flow sheets	“Nausea/constipation/urinary retention from spinal narcotic”
	“very sedated from lorazepam”
	“hallucinations from lortab, MS”
	“edema/wheezing from IVF”
Bar code medication administration comment	“Nausea from cefazolin”
	“urinary retention from MS, spinal narc”
	“constipation from calcium”
ER nursing assessment	“mental status change/hypotension from oxycontin, MS conti{sic}, percocet, and lorazepam”

DISCUSSION

Without the nursing narrative data, our EMR-based study found a surprisingly high incidence density of adverse drug events, 76 per 1000 patient days (this over a period of 6,100 patient days). To put that figure into perspective, typical reported incidence densities range from 6 to 30 per 1000 patient days.^8,11,13–16 The results from the EMR-based study are not the focus here, but we mention these figures to illustrate that the original study was already highly sensitive. In spite of that sensitivity, a significant number of ADEs were found in a careful review of data that routinely fails to make it into the EMR. These data are invisible to most practitioners.

An EMR is a double-edged sword.¹⁷ It offers wide and instantaneous availability, but its very convenience lulls healthcare providers into a false sense of security. It is far too easy to assume that the EMR is portraying the “complete picture” of a patient stay. The results described here suggest that a large, informal set of data are in routine use at this healthcare facility but that they remain invisible to most providers because they remain invisible to the electronic record.

Based on the clinical EMR alone, we found that nurses detected only about 14% of ADEs in our original study. However informal, often non-electronic nursing notes increased overall ADE incidence by nearly a third. An important implication of these data is that nurses are an excellent source of signals for ADEs. They should be part of any rigorous ADE study because they play an important role in ADE surveillance. Their role in this regard is easy to underestimate, especially in studies that rely on “classic” chart review. We know from experience that chart reviewers tend to skim, or skip altogether, non-normative annotations (e.g., text that is scribbled on a flow sheet or that is a terse fragment in a tight computer field like “comments” in an electronic medication administration record). Finally, these data underscore the need for a re-engineering of EMRs to take seriously the informal and often invisible communication that takes place at the nursing level. Nursing narratives, especially bedside notes, are a rich (albeit difficult to process) source of ADE surveillance data.