Design and implementation of an automated email notification system for results of tests pending at discharge

Abstract

Physicians are often unaware of the results of tests pending at discharge (TPADs). The authors designed and implemented an automated system to notify the responsible inpatient physician of the finalized results of TPADs using secure, network email. The system coordinates a series of electronic events triggered by the discharge time stamp and sends an email to the identified discharging attending physician once finalized results are available. A carbon copy is sent to the primary care physicians in order to facilitate communication and the subsequent transfer of responsibility. Logic was incorporated to suppress selected tests and to limit notification volume. The system was activated for patients with TPADs discharged by randomly selected inpatient-attending physicians during a 6-month pilot. They received approximately 1.6 email notifications per discharged patient with TPADs. Eighty-four per cent of inpatient-attending physicians receiving automated email notifications stated that they were satisfied with the system in a brief survey (59% survey response rate). Automated email notification is a useful strategy for managing results of TPADs.

Introduction

The transition to the ambulatory setting after hospital discharge is susceptible to communication failure of vital patient information.^1–4 Roy et al determined that 41% of patients were discharged before all test results were finalized, and, although inpatient and ambulatory providers caring for these patients had access to a shared, clinical data repository via an integrated, electronic medical record (EMR), they were aware of only 38% of the results.⁵ Failure to reliably follow-up results of tests pending at discharge (TPADs) can lead to readmissions, delays in diagnosis and treatment, and sometimes patient harm.⁶

Computerized applications have been used to notify clinicians of test results as they are finalized.^7–10 However, few institutions have implemented standardized systems to manage TPADs, and there are scant data with regard to successful technological strategies. Such strategies face an array of obstacles, as the provider who follows the patient after discharge is often different from the provider who ordered the test(s) as an inpatient. Consequently, the responsibility for acting on the result(s) may be unclear. To be successful, such systems must conform to the workflow of both inpatient and ambulatory providers, support coordination-of-care, and promote a seamless transition in responsibility.

Previous studies have demonstrated barriers to the adoption and optimal usage of information systems to manage TPADs.^{11 12} A ‘results manager’ application (available within our EMR) was activated for TPADs and evaluated at our institution; however, it was not accepted by inpatient physicians because it required them to actively access the information and did not conform to inpatient workflow.¹¹ Many users of this system commented that an alerting mechanism would have been useful if directed at the provider responsible for the test result. We describe the design and implementation of an automated email notification system that pushes the finalized results of TPADs to the responsible inpatient-attending physician at discharge and facilitates communication with the primary care physician (PCP).

Methods

Setting and participants

The system was developed at the Brigham and Women's Hospital (BWH), a 750-bed acute care hospital affiliated with Partners HealthCare. The study was approved by the Partners Institutional Review Board. We piloted the system on the inpatient general medicine and cardiology services (both house staff and non-house staff), which together discharge 11 800 patients per year. Randomly selected inpatient-attending physicians (hospitalists, traditional internists, and subspecialists staffing these services) and network PCPs received email notifications generated by the system. All Partners physicians have a network email address and can access their accounts from any Partners workstation or a fully encrypted personal computer (desktop, laptop, mobile device) connected to the Partners Microsoft Exchange Server via a secure internet connection. All email messages sent within Partners is encrypted. All Partners physicians have access to a web-based EMR which uses a shared, clinical data repository containing both inpatient and ambulatory test results and has internal clinical messaging functionality.

Overview of system

The notification system leverages functionalities available or enhanced from existing, internally developed information systems at BWH. These included the Brigham Integrated Clinical Information System (BICS), BWH inpatient bed-management system, and BWH admitting databases. We defined TPADs to include a test ordered by any provider or service from the time the patient presented to the emergency room (or admitted directly) until the time of discharge. In other words, we included tests ordered during the entire episode of acute care within our institution (eg, ordered in the emergency room, intensive care unit, etc) with a status of pending, received, processing, or preliminary.

The system (1) is triggered by a patient's electronic discharge time stamp (entered by the unit clerk into the BWH inpatient bed management system as part of routine care), (2) identifies the discharging inpatient-attending physician and network PCP from central administrative databases, (3) files all non-finalized tests in a queue, (4) updates the status of non-finalized tests at 12:01 AM each day, and (5) sends an email of the patient's result(s) to the inpatient-attending physician. A carbon copy of the email is sent to the PCP if within the network when newly finalized results are available; if the patient has an out-of-network PCP, or if no PCP is listed, only the inpatient-attending physician receives the notification email. The system continues to update the status of TPADs for each discharged patient until all are finalized. At the time TPADs are filed, an interim step suppresses selected TPADs based on configurable rules (figure 1).

Figure 1.

Overview of system. The system coordinates a sequence of electronic events initiated by the discharge time entered by the unit clerk and culminating in automated email notifications to the identified providers. For any discharged patient with tests pending at discharge (TPADs), providers receive no more than one email per notification type per day.

The system sends separate notifications for (1) chemistry and hematology, (2) radiologyⁱ and pathology, and (3) microbiology test results.ⁱⁱ We chose this grouping for technical and logistical reasons. First, it largely coincides with the classification system used by our central clinical data repository. Second, we wished to optimize the length of the email transcript to improve readability (particularly beneficial for patients discharged with multiple TPADs of different types). Third, this allowed us to modify the frequency of email notifications to providers based on volume of TPADs by test type. For example, we could minimize the volume of microbiology notification emails (compared with other test type emails) given the relatively high volume of pending microbiology culture results.¹²

Design features and considerations

Central design considerations included: integration within existing clinical information systems; addressing workflow constraints; defining a method for assigning responsibility; and minimizing alert fatigue. Moreover, pushing email alerts to the inboxes of responsible providers seemed an obvious and potentially successful target to facilitate communication of TPAD results given the robust culture of email utilization by providers at BWH and within the Partners network for inpatient care activities.¹¹

Discharge time stamp

To alert the admissions office of bed availability, unit clerks typically enter the time when a patient physically leaves an inpatient bed, a reasonably accurate means of electronically capturing the triggering discharge event. We contemplated using the electronic discharge order entered by physicians, but because it could be entered at variable times (eg, early in the morning) and often more than once (eg, as discharge instructions are modified), we felt that it was not the most reliable or accurate trigger.

Identity of providers

Our admissions office routinely updates the identity of the inpatient-attending physician in central administrative databases. The identity of the PCP is typically updated upon request. We used these data to capture the identity (internal provider ID, email address) of the inpatient-attending physician and PCP. Guided by our institutional policies and the patient safety leadership at Partners, we designated the discharging inpatient-attending physician as responsible for results of TPADs. We decided against using the ordering provider (typically a trainee or mid-level provider) as a surrogate because of issues of multiple hand-offs, transient/weekend coverage, variable supervision, and transfers between services or different levels of care. Finally, to improve post-discharge communication, we sent a carbon copy of the notification email to the PCP's inbox. We crafted wording with the intent of facilitating a dialog between the inpatient-attending physician and the PCP in order to acknowledge the result, consider subsequent actions, and facilitate the transfer of responsibility from inpatient to outpatient provider (figure 2).

Figure 2.

Example of a chemistry/hematology email notification. All emails are addressed to the responsible inpatient attending and states whether a carbon copy was sent to the primary care provider (PCP). All emails specify the date the email was sent, identifies the patient (name, discharge date, Brigham and Women's Hospital (BWH) medical record number). Finalized test results appear at the top and pending tests appear at the bottom. Abnormal results are highlighted. For this patient, Dr Hospitalist received a total of three chemistry/hematology emails on three separate days (the first email is shown here).

Alert fatigue

To minimize alert fatigue, we incorporated configurable rules to suppress selected tests from notification emails for each test type. We assumed that both normal and abnormal results could be actionable depending on the perspective and practices of individual physicians. For example, receipt of a negative culture result may prompt a physician to discontinue antibiotics sooner if the indications were not compelling and the risk of adverse drug events or medication interactions were high. With regard to which results to include (or exclude) from notification emails, we explored several possibilities: (a) we could include abnormal only, or both normal and abnormal results; (b) we could exclude inpatient-specific results (eg, arterial blood gas), commonly ordered inpatient tests (eg, complete blood counts, basic metabolic panels), or inpatient tests with a fast turnaround time likely seen by inpatient providers on the day of discharge (eg, prothrombin time). During the initial pilot period (described below), we suppressed a small number of tests,ⁱⁱⁱ as we were uncertain about which types of results individual providers would find actionable.

Notification volume

The system was configured to send no more than one email per patient per day per notification type. This was achieved by a routine job which updates the status of TPADs at 12:01 AM each day. Given the high volume and variability in post-discharge finalization times for microbiology results, after the initial email was sent (for either normal or abnormal results), subsequent notifications were only sent on abnormal results and after all results were finalized.

Notification email content

Based on feedback received from a focus group of potential users (hospitalists and PCPs), we iteratively refined content to convey appropriate meaning (eg, to avoid transferring responsibility to the PCP until explicit communication takes place), minimize length, and optimize format. We created a prioritization hierarchy to display results by status (finalized > still pending), type (abnormal > normal), and date (most recent > least recent), and created a mechanism to flag abnormal results (highlighted in red). We incorporated phone contact information for both providers (additional fields pulled from administrative databases). Figure 2 illustrates a typical chemistry/hematology email notification.

Implementation and measurements

We activated the system for patients with TPADs discharged by randomly selected inpatient-attending physicians from general medicine and cardiology services over a 6-month period. Patients discharged without TPADs were excluded. During phase I (October 26, 2010 through November 30, 2010, activated for chemistry/hematology notification type), we measured the volume of tests processed, effect of initially configured suppression rules, and reliability of discharge time entry and provider identification for any patient discharged with TPADs (regardless of whether emails were sent). In phase II (March 15, 2011 through April 15, 2011, activated for all notification types), we measured the volume of automated email notifications received by physicians stratified by notification type for patients discharged with TPADs. Patients with TPADs whose inpatient-attending physician was not randomized to receive automated email notifications were excluded. We sent a brief survey to these randomly selected inpatient-attending physicians approximately 3 days after all TPADs had been finalized on their discharged patient. We asked them to rate their satisfaction with receiving automated email notification(s) on a five-point Likert scale and to provide any feedback or suggestions in an open-ended comment box.

Results

Phase I

Volume of tests processed and effect of suppression rules

Eighty-three patients with TPADs were discharged during phase I. For 82 correctly identified patients, the system automatically detected 264 chemistry and 141 hematology TPADs (4.9 per patient), triggered 136 emails (1.7 per patient) with two or more emails triggered on 28 patients (34%), and flagged 73 abnormal results (18%). Nineteen (4.7%) tests (all hematology) were suppressed on the basis of initially configured suppression rules.

Reliability of discharge time stamp and provider identification

The system triggered emails on one patient incorrectly (1.2%). In this case, a unit clerk inadvertently ‘discharged’ the patient on day 4 of a 10-day hospital stay. The system detected 510 TPADs (249 chemistry, 261 hematology) and triggered nine emails to the responsible inpatient-attending physician on this patient. Regarding correct identification of providers, we received three responses from physicians stating that the email(s) were sent to the wrong provider in error (3.6%), two from the inpatient-attending physician and one from a PCP.

Phase II

Email notifications

Ninety-five patients of randomly selected physicians were discharged with non-suppressed TPADs. Inpatient-attending physicians and network PCPs received 153 and 80 email notifications, respectively (figure 3). During this 1-month period, inpatient-attending physicians received approximately 1.6 notifications per discharged patient with TPADs, and anywhere from one to 32 emails in total (at most one email per patient per day per notification type). The majority of emails sent to inpatient-attending physicians were microbiology notifications (58%). Emails were not copied to a PCP in 48% of cases, indicating that the PCP was unknown or out of network. Seventy-two abnormal results were reported to physicians in these emails. The majority of abnormal results (53%) were in chemistry/hematology notifications.

Figure 3.

Email notifications sent to providers of 95 discharged patients with all tests pending at discharge (TPADs) finalized. The majority of emails are of the microbiology notification type. For patients with a non-network primary care physician (PCP) (or no identified PCP), the inpatient-attending physician alone receives the alert. Chem, chemistry; Heme, hematology; Micro, microbiology; Path, pathology; Rad, radiology.

User satisfaction

A total of 119 surveys were sent to 36 distinct inpatient-attending physicians who received automated email notifications of TPAD results on patients they discharged. We received 70 completed surveys from 29 distinct physicians. The overall survey response rate was 59%, and the response rate from distinct physicians was 81%. In survey responses, 84% (59/70) stated that they were satisfied or very satisfied with receiving automated email notifications of TPAD results on their discharged patient(s). Some users were not aware that the test had been ordered, some varied with regard to type(s) of results they wished to receive, and some thought specialist providers would benefit from receiving the notification as well (box 1).

Box 1. Selected comments from inpatient-attending physician users.

“I find this extremely useful, knowing the final results of tests, both test results that are positive as well as negative.”

“Was unaware of this test even being ordered—had it not been for auto-notification, would never have known about test or result. No call to PCP as test is in normal range and will not affect management.”

“The concept is great. All the notifications I have received are for negative results. Might be more worthwhile for blood tests if it was only for abnormal results.”

“Test was not needed and was not ordered by me… ”

“It is best to send these pathology results not just to the ordering physician but also the GI physician performing the biopsy.”

Discussion

We describe an approach for identifying and highlighting the results of TPADs by alerting the responsible physician as they are finalized during a patient's transition from the inpatient to ambulatory setting. Although inpatient and ambulatory providers affiliated with our healthcare system have access to a common clinical data repository and integrated EMR, our experience suggests that this subset of test results still ‘fall through the cracks.’⁵ Our system clarifies the primary responsibility for these results as well as communication practices upfront by (1) identifying the responsible physician (in our case, the discharging inpatient-attending physician) and (2) emphasizing this responsibility but facilitating communication with the primary outpatient provider. For these results, reliable, timely, and clear communication with key follow-up provider(s) is important to ‘close the loop’ (ie, to acknowledge the result, provide clinical context, and outline subsequent actions to be taken). Our pilot validates the approach whereby the discharging attending physician is considered responsible (reinforced by the language in the email) until communication acknowledging and conferring transfer of responsibility occurs. Finally, we demonstrate the utility of incorporating logic and configurable rules to minimize the risk of overalerting physicians; these are potentially valuable to other systems attempting to develop similar solutions for tackling this patient safety concern. Box 2 outlines key lessons from our experience.

Box 2. Suggestions for developing (or enhancing) an automated push notification strategy for results of tests pending at discharge (TPADs).

Use electronic events which precisely and reliably capture discharge time (or a clinical status change from the inpatient to ambulatory setting) to accurately identify TPADs.

Institute processes to accurately and reliably identify the key team members (ie, the responsible inpatient-attending physician and primary care provider at the minimum).

Clearly assign responsibility for the TPAD results to one provider according to institutional policies/best practices, particularly if the notification is sent to multiple providers.

Incorporate logic and configurable rules to reduce the volume of notifications to minimize risk of alert fatigue. These could be manipulated ‘on-the-fly’ for different physicians, services, test types (eg, microbiology vs pathology), or type of result (eg, normal vs abnormal).

Use language that facilitates communication between inpatient and outpatient providers regarding TPADs (eg, clinical context, interpretation of the results, possible actions to be taken, and subsequent transfer of responsibility for taking action).

Because most hospitals do not have a reliable system for managing these test results, most providers rely on documentation in the discharge summary, direct communication with the PCP (verbal or electronic correspondence), or other individual approaches.¹³ Many of these ad hoc systems are faulty for a variety of reasons, predominately because they are not failsafe. Conversely, even those healthcare systems with integrated EMRs that can place results of TPADs in the ‘inbox’ of designated inpatient and outpatient providers have their limitations because they do not highlight these particular tests as having increased potential for ‘falling through the cracks,’ do not clearly assign responsibility to a single physician the instant TPAD results are finalized, and do not give physicians a facile mechanism for communicating clinical context (including additional actions) to the provider who subsequently cares for the patient. Also, these systems may not work well if the PCP is outside the healthcare system and does not use the EMR. By recognizing these deficiencies, leveraging electronically captured events from existing clinical information systems, and adapting to institutional culture and workflow, we were able to develop and implement an automated system that reliably identifies patients discharged with pending tests, updates the status on a daily basis, notifies the responsible provider of the finalized results, and facilitates communication with key follow-up providers (our system notifies the discharging attending physician alone when the PCP is either out of network or not identified, with language meant to encourage communication with that PCP if needed).

In pilot testing, we demonstrated the accuracy and reliability of our system with regard to discharge time entry and identification of providers. We attribute our success in part to identifying reliable processes that were being electronically captured as part of routine care. Although entry of the discharge time stamp by unit clerks is susceptible to inadvertent human error (namely inaccurate entry due to inadvertent discharges, ‘gaming’ of discharge time, and variability by unit), we observed this to be infrequent. With regard to provider identification, our hospital admitting office had devised a generally reliable process of updating the inpatient-attending physician on the basis of centralized schedules. In fact, the few anomalies we observed were due to either weekend attending physician coverage (not updated by the admissions office unless specifically requested) or services with two attending physicians (as for three of seven general medicine services).

The high volume of TPADs and emails sent underscores the importance of a mechanism to suppress certain results and logic to limit notification volume and frequency to minimize alert fatigue. Based on our current configuration, inpatient-attending physicians receive approximately 1.6 alerts per discharged patient with pending tests. On a 12-day rotation under full implementation of this service, s/he would typically discharge 50 patients (41% with TPADs⁵), and therefore receive 33 notifications (∼2.75 emails per day). Of these, 58% (88/153) would be microbiology notifications (∼19 emails), which would include abnormal results 20% (18/88) of the time. Having a mechanism to reconfigure and/or modify test suppression rules ‘on-the-fly’ mitigates the risk of overalerting clinicians about clinically insignificant results. For example, in the future, we may suppress email notifications of negative culture results in order to reduce the overall volume for any individual provider.

Most users were satisfied with the notification system primarily because they do not have to actively remember or create a task list to track results of TPADs. Nonetheless, there was variability with regard to the type of results they wished to receive (abnormal results vs both normal and abnormal results) and to whom results should be sent (eg, consultants). These comments suggest that, although physicians value the system, they desire some degree of influence over which results they receive. Future versions could include user-configurable settings to suppress certain test results and to activate the notification system for other key providers involved in the patient's care.

Our system has several limitations. First, it was developed at a single academic center with home-grown, proprietary information systems. However, other centers (ie, those using commercially available EMRs) may consider adapting existing functionality to identify this specific subset of test results, alert the responsible provider once their results are finalized, and facilitate communication with key follow-up providers. These measures go beyond simply placing test results in a relevant provider's inbox; they recognize the special status of TPADs as occurring during a care transition. Institutions using a commercial system's internal clinical messaging capabilities may consider enhancements to specifically assign responsibility for TPAD results according to institutional policies and to automatically alert the responsible provider of their availability even when s/he is not logged-on (eg, via an encrypted page, sms, email to provider's institutional email inbox, or ‘push’ notification to a mobile version of the EMR). Such enhancements should heighten awareness of TPAD results while minimizing the diffusion of responsibility that can occur if the result is viewed by multiple providers (Singh et al¹⁴ observed that, in cases of dual notification of results within an EMR, the odds that the alert would be left unacknowledged doubled and was associated with less timely follow-up actions).

Second, it was implemented within a large integrated healthcare network which provides email access to all network physicians and has a robust culture of email use for communicating inpatient clinical information. Third, it is highly dependent on accurate and reliable entry of the discharge time stamp and identification of responsible providers from central administrative databases; variability in these processes could have unintended consequences. Fourth, we did not create a tool that facilitates electronic acknowledgment of individual test results by specific physicians (therefore accepting responsibility for subsequent actions). However, such tools could be incorporated into future versions. Fifth, although patients can access their results via Patient Gateway (the web-based patient portal for Partners), they are not specifically alerted about TPAD results. Although such a strategy could provide an additional safety net, simple receipt of the test result by both the patient and the PCP may not be enough to promote appropriate action. Additional work would need to be carried out to fully utilize patient portals regarding results of TPADs. Finally, because of a variable lag between the time when the discharge order is entered by the physician and when the discharge is processed by the unit clerk, certain tests that were pending according to the discharging physician could have been finalized before the patient actually left, unbeknownst to the physician. However, it is an uncommon occurrence and only tends to happen when physicians order tests with an intermediate turnaround time on the day of discharge (eg, drug levels that take several hours to return).

We successfully developed and implemented an automated email notification system to ‘push’ the finalized results of TPADs to the responsible inpatient-attending physician and facilitate communication with the primary outpatient provider. Future studies should evaluate the effect on awareness and satisfaction of both inpatient and ambulatory patient physicians, analyze downstream actions taken in response to notifications, and further elucidate desired features (including optimal ‘tuning’ of logic and suppression rules) to maximize utility.