Computerized Alerts Improve Outpatient Laboratory Monitoring of Transplant Patients

Abstract

Authors evaluated the impact of computerized alerts on the quality of outpatient laboratory monitoring for transplant patients. For 356 outpatient liver transplant patients managed at LDS Hospital, Salt Lake City, this observational study compared traditional laboratory result reporting, using faxes and printouts, to computerized alerts implemented in 2004. Study alerts within the electronic health record notified clinicians of new results and overdue new orders for creatinine tests and immunosuppression drug levels. After implementing alerts, completeness of reporting increased from 66 to >99 %, as did positive predictive value that a report included new information (from 46 to >99 %). Timeliness of reporting and clinicians' responses improved after implementing alerts (p <0.001): median times for clinicians to receive and complete actions decreased to 9 hours from 33 hours using the prior traditional reporting system. Computerized alerts led to more efficient, complete, and timely management of laboratory information.

Background

Especially for patients on anticoagulation, immunosuppression, and other therapies requiring ongoing monitoring, laboratory testing provides key information for clinical decision making. National patient safety efforts have targeted the need to improve communication about laboratory results. ^1–3 The 2005 Ambulatory Care National Patient Safety Goals of the Joint Commission stated that organizations should “assess and, if appropriate, take action to improve the timeliness of reporting, and the timeliness of receipt by the responsible licensed caregiver, of critical test results.” The Agency for Healthcare Research and Quality advises patients, “If you have had a test, don't assume no news is good news.”³ New strategies for patient safety include communicating results of tests directly to patients. ⁴

The plethora, diversity, and de-centralized nature of laboratories, data storage systems, and communication modes all complicate laboratory result communication. Clinicians communicate with multiple laboratories using faxes, phone, mail, and electronic systems that passively or actively report results. The heterogeneous nature of laboratory sources and reporting methods often leads to lost or delayed results and may impact patient outcomes. In a recent study, Smith et al. found that completed laboratory results were unavailable during 6% of all primary care outpatient visits. ⁵ Clinicians indicated that missing information resulted in delayed care or duplicative medical services in 60% of the visits with missing clinical information. In another study, Lin et al. found that 46% of the resident physicians working in an outpatient clinic had seen a patient's medical condition worsen due to a delay in test result follow-up. ⁶ Several studies document physicians' concerns about the availability and timeliness of results, the ability to detect patients overdue for follow-up, and the ability to prioritize and respond to abnormal (not critical) results. ^5–11

Clinicians at the liver transplantation program at LDS Hospital in Salt Lake City, Utah, have had similar concerns. LDS Hospital is part of the Intermountain Healthcare (Intermountain) enterprise. Liver transplant patients undergo periodic outpatient testing to identify clinical complications (such as deteriorating renal function and organ rejection) at the earliest possible time, and to monitor appropriateness of medication dose. At LDS Hospital, transplant patients have “standing orders” for laboratory testing to be performed at any laboratory from three times a week to every three months, depending on the patient's status and the time from transplantation. To obtain laboratory results, nurses had to telephone the (possibly remote) laboratory that performed the tests, or log into a patient's electronic health record (EHR), ¹² and also find any results that arrived on the fax/printer from a variety of laboratories internal and external to the Intermountain network. Since 2001, the Intermountain EHR included all laboratory results generated from five Intermountain clinics and 17 hospitals (including LDS Hospital) located throughout Utah and southern Idaho, as well as other electronic patient data from multiple inpatient and outpatient sources. The clinicians, often unaware of new results saved to the EHR, depended on a phone call from the patient, a paper report, and their own memory to track all the liver transplant patients. The LDS transplant office staff and medical assistant manually sorted every laboratory report received. The medical assistant and the transplant nurses reviewed and transcribed results to paper flowcharts. A medical assistant often spent more than two hours each day sorting and transferring laboratory results from faxed reports to the flowchart. Clinicians at LDS Hospital used the paper flowchart as the primary longitudinal record for decision-making because it integrated laboratory results and medication dosages from all sources side by side. While clinicians had meticulously implemented this process, they feared they were missing results, and requested a better system for tracking laboratory data. ^13,14

Research Question

Information technology has long been used to manage result reporting in the outpatient setting. ^8,15–18 Computerized alerts have improved the timeliness and completeness of critical value reporting for hospitalized patients. ^19–23 However, a paucity of published studies examine the impact of computerized alerts on the quality of outpatient transplant patients' laboratory result reporting-related care, including the timeliness, completeness, and predictive value that the latest report would include new results. This study assessed differences between the traditional result reporting-related clinical processes that were dependent on faxes and printed documents, and improvements in quality of results reporting and timeliness of clinician responses after implementing computerized alerts.

Methods

The study involved laboratory reporting for all liver transplant patients actively managed by the LDS Hospital transplant team between August 2003 and March 2005. We selected reporting attributes that are commonly used to evaluate surveillance systems. ²⁴ We assessed the following: 1) completeness and timeliness of reporting creatinine levels, tacrolimus levels, and cyclosporin A drug levels collected in the outpatient setting from any Intermountain laboratory; 2) positive predictive value (PPV) that a report included one or more new results not previously reported to the transplant office; 3) and timeliness and type of clinical actions in response to the reported information. The study focused on results reported by laboratories internal to the Intermountain network because it was possible to identify for those tests the set of results that should have been reported to the transplant clinicians. The study presents limited information about results from laboratories external to the Intermountain network. Laboratories external to the network were located throughout the United States and included one hospital laboratory that were owned by Intermountain Healthcare but not yet integrated with the Intermountain EHR. Institutional Review Boards from the University of Utah and Intermountain approved this study.

Assessment of Traditional Reporting Process

The study assessed traditional reporting processes during a three-week period between February 20 and March 11, 2004. The LDS transplant clinic staff tracked all laboratory reports that arrived in the transplant office by fax, printer, and US mail. Data collection involved a standardized, self-coding form that allowed nurses and office staff to track the flow and status of information easily as it traveled throughout the office. Project team members attached a data collection form to each report when it arrived. The nurses and office staff documented whether the report arrived spontaneously or was requested. In addition, the nurses and staff identified results that were new to the nurse (classified as “new”) and those that had already been sought by other means from the EHR or a laboratory, and already transcribed to the flowchart. The latter results presumably already had been used for decision making (classified as “new, but too late”). The nurses and staff also documented the date and time the results arrived in the office, time of transcription to the paper flowchart, and time of first review by the clinic nurse. Finally, the nurses documented actions they performed in response to the results but did not document the time when actions were completed.

Completeness of reporting was assessed by comparing the reports received in the office by March 12, 2004 with all creatinine, tacrolimus, and cyclosporin A test results collected at relevant Intermountain laboratories between February 20 and March 11, 2004. All results during pre-intervention and post-intervention study periods were available in the EHR. Positive Predictive Value (PPV) was assessed by determining the number of times unique creatinine, tacrolimus and cyclosporin A results from specimens collected between February 20 and March 11 were reported to the transplant office by March 12, 2004. Timeliness was assessed by calculating the time between specimen collection and the first time the result arrived in the office (reporting time) and time between arrival in the office and first review by the nurse (response time).

Description of Computerized Alerts

Using Intermountain's computerized decision support system infrastructure, ²⁵ authors created a new set of rules in 2004 to monitor the liver transplant patients' laboratory results. The decision support system logic activated whenever data saved to the EHR involved selected study results on patients enrolled in the study protocol. To enroll in the study protocol, the patient's unique identifier was added to the protocol ‘patient list’ (▶). A message log application integrated with the Intermountain EHR delivered the alerts to a generic electronic in-box that was accessible to all members of the liver transplant team. Each liver transplant clinician signed-on individually but could view and manage alerts for all liver transplant patients. The list of alerts could be sorted by severity, date, patient name, and type of alert, and was updated in real-time as new alerts were generated. The clinicians divided responsibility for patients using the first letter of the patient's last name, but provided cross-coverage when needed.

Figure 1.

Decision support system for the liver transplant laboratory alerts.

The liver transplant team defined the logic in the liver transplant protocol. Alerts notified the clinicians when any creatinine result or immunosuppression drug (tacrolimus, cyclosporin A, or sirolimus) level was reported for a liver transplant patient. The alerts also: 1) specified if a new creatinine result represented an acute elevation or increasing trend in creatinine of 0.3 mg/dL or more; 2) specified whether tacrolimus levels were within, above, or below the target range defined by the physicians, based on the time since transplantation; 3) identified abnormal potassium and magnesium values; 4) included the date of transplantation and the time since transplantation; 5) indicated if the patient was currently hospitalized; and 6) identified patients overdue for creatinine or immunosuppression drug level testing (▶).

Table 1.

Table 1 Logic Used to Trigger the Overdue Alerts for the Liver Transplant Patients

	Time since Transplant
	0 - < 3 months	3 - < 6 months	6 mo – < 4 years	> = 4 years
Routine lab testing schedule	At least weekly	Every 2 weeks	Every month	Every 3 months
Alert if it has been at least ____ since last result	14 days	21 days	45 days	120 days
Repeat overdue alert every ___days after the initial alert until new lab result obtained	3 days	3 days	14 days	14 days

Note: For example, patients that underwent transplantation during the previous three to six months are expected to get their blood tested every two weeks. If a patient had no new immunosuppression drug level for 21 days (i.e., two weeks and a seven-day buffer), then an “overdue for immunosuppression drug level” alert was triggered. Alerts continued to be delivered every 3 days until a new result was saved to the EHR. The same logic applied for creatinine results.

Nurses initiated the following processes to handle alerts: they sorted and triaged the list of alerts; “opened” a single alert (▶); viewed the messages; navigated to the laboratory data that triggered the alert and reviewed all new laboratory results; transcribed the laboratory results onto the paper flowchart if the results were not present; reviewed the laboratory and drug dosage information side by side on the paper flowchart; reviewed other pertinent clinical information for the patient in the EHR; contacted the patient or physician when indicated; specified whether the nurses accepted or rejected the alert; and documented their response to the alert. Additionally, the nurses reviewed the overdue alerts, and either called the patient directly or asked a medical assistant to create and send a pre-populated letter using a reporting tool integrated with the EHR. Alerts were generated in real-time as new results were saved to the EHR or as overdue triggers were activated. Therefore, the nurses used the list of alerts to triage patient management activities and create a work list of patients with new, abnormal and overdue results.

Figure 2.

Alert for a liver transplant patient.

The alerts prompted the liver transplant program medical assistants to initiate the following processes: they reviewed new and ‘cleared’ alerts generated during a specific time period and identified laboratory data to transcribe to the paper flowchart; located relevant flowcharts; transcribed the results; and gave the updated flowcharts to the nurses for review.

Using a data entry application integrated within the Intermountain EHR, ^25,26 transplant office support staff hand-entered laboratory results generated external to the Intermountain network into the Intermountain EHR in a timely manner, so that the results could be used by the decision support logic. ^25,26 External results were important, particularly for resetting the timers for overdue logic, because approximately 40% of the transplant patients used a laboratory external to Intermountain for at least some of their laboratory testing. ^25,26 The nurses verified the hand-entered external laboratory results when they reviewed the paper flowcharts and the paper laboratory reports.

Starting in the spring of 2004, information about the date, description, and status of each patient's transplantation, and any new external laboratory results, were hand entered into the EHR. The alerts were implemented on May 28, 2004, and fully integrated into the clinician's workflow by November 1, 2004. ^13,14

Assessment of Reporting Process Using Computerized Alerts

The study evaluated laboratory result reporting processes using computerized alerts during the four-month period between November 1, 2004, and February 28, 2005. Using the EHR, study team members determined the date and time alerts were reported and “cleared” by the transplant nurse, the patient's hospitalization status, and the source of the laboratory results. The study calculated PPVs by identifying duplicate alerts triggered by each laboratory result. We assessed the completeness of reporting from an Intermountain facility by comparing the alerts received by March 1, 2005 with all creatinine, tacrolimus, and cyclosporin A test results collected between November 1, 2004, and February 28, 2005. All results were available in the EHR. We calculated timeliness as both the time between specimen collection and delivery of the alert (reporting time) and between delivery and acknowledgement of the alert (response time). We could not determine when nurses first viewed an alert; however, the nurses used the EHR to document when they completed their alert-related actions (e.g., “contacted patient to change medication dose”).

Data Analysis

All analyses were performed using SAS 9.1 (SAS Institute, Inc., Cary, NC). Descriptive statistics and tests of proportions (Pearson chi-square) were used to assess the completeness and accuracy of reporting and charting. When assessing timeliness, we stratified by the type of test (creatinine, cyclosporin A, and tacrolimus) and the source of the report (LDS Hospital laboratory or other Intermountain laboratory integrated with the Intermountain EHR). We used nonparametric statistics to compare groups. We interpreted “p ≤ 0.05” as indicating statistical significance; all p values were two-sided. We report the p value for a Wilcoxon rank sum test and a Kruskal-Wallis test when comparing two and multiple samples, respectively.

Results

Study Population

The LDS Hospital transplant program managed a similar number of liver transplant outpatients during the pre-intervention and post-intervention periods in 2004 (n=300) and 2005 (n=336). The intensity of follow-up required to manage the patients did not change during this time. A similar number of patients received transplants (30 and 24 patients, respectively) and a similar number died (1 and 3, respectively) during the six months preceding the end of the data collection periods in 2004 and 2005.

Reporting of Outpatient Intermountain Results Using the Traditional Process

During the three week assessment of the traditional reporting process, 348 creatinine, tacrolimus, and cyclosporin A results were collected from outpatient Intermountain settings. Among the 348 results, 34% were not reported to the transplant office, 31% were reported once, 20% were reported twice, and 15% were reported 3 to 11 times. Therefore, the completeness of reporting by fax and printing was 66%. All results were available in the EHR, but the clinicians used the faxes and printouts to identify most new results and expected all results to be sent to the transplant office. The unreported results included 65 (36%) creatinine results, 6 (54%) cyclosporin A results, and 49 (31%) tacrolimus results. The proportion of unreported results was similar for each week (29, 38, and 35%; χ² = >0.29). The 270 duplicate results received in the transplant office gave a positive predictive value of 46% that a given result reported was in fact a new result for the clinician to review (228 unique creatinine, tacrolimus and cyclosporine A results/498 total creatinine, tacrolimus and cycolosporin A reported).

In assessing timeliness, the study used the first report received for each result. Among the 223 first reports received, 78 (35%) were “new, but too late” and had already been viewed in the computer and transcribed to the flowchart by the time the paper report arrived. There was no difference in the timeliness of reporting for “new” and “new, but too late” results after stratifying by the source and type of test (p > 0.1). Among the remaining 145 (65%) “new” results, the paper report was the first source of information for the nurse. The timeliness of reporting varied by the type of test and source of the report but not by the day of the week the specimen was collected (▶). Once a laboratory report arrived in the office, a median of 22 hours elapsed before the nurse first reviewed the report. There was no significant difference in response time after stratifying for the source of the report, type of test, day of the week the specimen was collected, or day of the week the report arrived. The total time between specimen collection and nursing review ranged from approximately 23.5 hours for creatinine results from LDS Hospital to 50 hours for cyclosporin A results from other Intermountain facilities.

In response to the 145 “new” results, over half (56%) of the results caused a nurse to contact the patient, and one-fourth (25%) of the results required a nurse to contact a physician. The remaining results did not require immediate contact with the patient or physician. Patients were contacted for a variety of reasons, including notification of their results (48%), to make medication changes (8%), to repeat laboratory tests (5%), or to schedule biopsies (5%).

Reporting of Outpatient Intermountain Results with Computerized Alerts

During the four month assessment of the computerized alerts, 2,123 creatinine, tacrolimus, and cyclosporin A results were collected from an outpatient Intermountain setting. Among the 2,123 results, 17 (0.8%) results were not reported, 2,106 (99.2%) were reported once, and 1 (<0.1%) result was reported twice. All results were available in the EHR by March 1, 2005. Therefore, both the completeness of reporting and the predictive value that a reported results was new for the clinician to review were >99%. Several factors contributed to missing 17 alerts for 10 patients. Nine alerts were missing for three patients who were not immediately enrolled in the protocol after they were transferred into the program. Three alerts were missing for two newly transplanted patients who had no transplantation date entered into the EHR; the nurses were alerted that lab data were available but the transplant date needed to be charted to make the alerts function properly. Finally, five alerts were missing for five patients who had immunosuppression drug level results reported in a batch and saved to the EHR during a five-minute interval on February 23, 2005 when the computer system was unstable. Alerts were not processed during the rare, unstable interval caused by a system upgrade.

Comparison of Timeliness between the Two Processes

After implementing the alerts, there was a significant decrease in the reporting and response time for both creatinine and tacrolimus results, whether or not the source of the report was LDS Hospital or another Intermountain facility (p < 0.001). Overall, using the alerts, the nurses received, reviewed, and completed actions faster (median = 9.2 hours) than they were able to receive and review lab reports using the traditional reporting process (median = 33.4 hours) (▶).

Table 2.

Table 2 Timeliness of Traditional Process and Computerized Alerts for Reporting Outpatient Intermountain Healthcare Laboratory Results

	Intermountain facility	Traditional reporting with faxes, printouts and mailed reports			Computerized alerts
		N	median # hrs	range ∗	n	median # hrs	range ∗	p value
		Specimen collected → nurse started reviewing results			Specimen collected → nurse completed actions
	Total Time	145	33.4	(25.8 – 58)	2106	9.2	(4.3 – 28)
Reporting Time		Specimen collected → Report arrived †			Specimen collected → Alert arrived §
Creatinine	LDS Hospital	48	1.5	(0.9 – 7.6)	567	0.8	(0.6 – 1.1)	<.001
	Other	29	7.2	(3.9 – 24.2)	475	1.3	(0.9 – 1.8)	<.001
Tacrolimus	LDS Hospital	38	7	(3.6 – 24)	572	3.5	(2.4 – 4.9)	<.001
	Other	25	25.6	(22.7 – 47.3)	452	17.4	(12 – 24)	<.001
Cyclosporin A	LDS Hospital	2	14.8	(12.0 – 17.5)	21	3.4	(2.0 – 4.8)	0.07
	Other	3	28	(21.4 – 42.8)	19	26.1	(23.8 – 28)	0.39
Response Time		Report arrived→ nurse reviewed result ‡			Alert arrived → nurse completed actions §
Creatinine	LDS Hospital	48	22.4	(5.0 – 31.9)	567	3.5	(1.4 – 7.7)	<.001
	Other	29	19.9	(6.7 – 34.7)	475	5.6	(1.7 – 24)	.004
Tacrolimus	LDS Hospital	38	24.9	(12.2 – 34.1)	572	4.1	(1.9 – 22)	<.001
	Other	25	11.6	(6.7 – 44.8)	452	13.5	(5.1 – 35)	0.72
Cyclosporin A	LDS Hospital	2	31.1	(16.3 – 45.9)	21	4.5	(1.9 – 22)	0.3
	Other	3	32.7	(3.8 – 80)	19	4.4	(0.8 – 51)	0.6

^∗ Reported ranges are interquartile ranges.

^† For creatinine and tacrolimus tests, there was a significant difference in the paper-based reporting time from LDS Hospital vs. other Intermountain hospitals (p < 0.002).

^‡ There was no significant difference in the response time during paper-based reporting, by source of the report or type of lab test.

^§ Using computerized alerts, there was a significant difference in the reporting time from LDS Hospital vs. other Intermountain hospitals for each lab test (p < 0.001) and there was a significant difference in the response time for creatinine and tacrolimus results (p<0.001).

Reporting of Inpatient Intermountain Results

Liver transplant patients are frequently admitted for reasons other than transplantation, and the transplant clinicians needed to monitor their care. Prior to implementing alerts, the transplant clinicians were not always informed when their patients were hospitalized, and they never received reports about inpatient tests. During the three-week assessment of the traditional reporting process, 150 creatinine, 96 tacrolimus, and one cyclosporine A result were saved to the EHR for the liver transplant patients. None of these results were reported to the transplant office. After implementing alerts, the clinicians were notified of all creatinine, tacrolimus, and cyclysporin A results, and all abnormal magnesium and potassium results, collected in the inpatient setting. The alerts included the statement, “Patient is hospitalized” and prompted the transplant team to proactively monitor the hospitalized patients. During the four-month assessment period, the nurses received alerts concerning 18 patients hospitalized for transplantation and an additional 33 patients hospitalized in four Intermountain hospitals for other reasons. Thus, the completeness of laboratory result reporting for patients hospitalized in any one of the 17 Intermountain facilities increased from 0 to 100% after alert implementation.

Reporting of External Outpatient Results

During the three-week assessment period of the fax-based reporting process, 64 creatinine, 29 tacrolimus, and 2 cyclosporin A results were reported to the transplant office by 33 laboratories external to the Intermountain network. An additional 4 results were discovered at a later date, but the completeness of reporting could not be determined because we could not determine the total number of results that should have been reported. The positive predictive value that a given report included information not previously reported was 63% (95 unique results out of 150 results reported); unique creatinine, tacrolimus, and cyclosporine A results were reported from one to six times.

After implementing alerts, it was important to enter results from external laboratories into the EHR because both external and internal results are needed to reset the time-driven triggers for the overdue alerts. Likewise, the overdue alerts impacted the reporting of results from external laboratories because they helped the transplant team identify patients that underwent testing at laboratories that were not sending results to the transplant office.

Impact on Workflow

The computerized alerts increased workload but improved workflow. With the traditional reporting process, the transplant office received approximately 150 sets of laboratory reports each week on over 1,000 sheets of paper. To identify new results, the nurses and assistants had to sort the papers and locate and review flowcharts to determine whether the results had already been charted and observed by a clinician. Only half the reports received included new information (PPV = 46% for Intermountain and 63% for external results). In addition, the nurses spent time “chasing down” unreported results.

With computerized alerts, the transplant office no longer received paper reports from laboratories integrated with the Intermountain EHR. The transplant office continued to receive paper reports from external laboratories. The nurses received approximately 130 alerts for new creatinine results each week, and these alerts were an indicator that there were additional new hepatic function and chemistry results. In addition, most of the creatinine alerts were accompanied by an immunosuppression drug level alert. In addition, each week, the nurses received 12 alerts for patients newly recognized as overdue for testing and 30 alerts for patients that continued to be overdue for testing. The nurses no longer “chased down” unreported results. They responded to alerts for new results or alerts that identified patients overdue for testing. The overdue alerts created a mechanism for shifting the responsibility for managing overdue patients from nursing to the support staff.

Discussion

Laboratory results are an important source of information for managing patients on immunosuppressant medications. The traditional laboratory reporting system did not meet the LDS transplantation clinic provider' needs for information. The practice of receiving laboratory results by fax and printout is slow, incomplete, and inefficient. Missing, delayed, and duplicate reporting events were common, as were delays in processing information after it arrived in the office. One-third of the “new” results that arrived spontaneously were “new, but too late” and had already been sought by the clinician for decision-making. Only about half of all lab reports received from Intermountain and external laboratories contained new information. Recipients must handle every report as if it contains new information. Intermountain laboratories sent multiple copies of results from a single blood draw or sent reports that included results from multiple blood draws. When sorting duplicate reports, pulling flowcharts, and finding information that had already been charted and handled by a clinician, the clinicians and support staff were performing an unproductive task that took them away from patient care.

Delays in reporting and processing increase the risk that new results may supersede results being reviewed by the clinician, especially when patients are tested two or three times per week. When clinicians are not using the most recent results, they are making decisions based on the wrong information and may harm the patient. In addition, when results are missing or delayed, patients incur additional costs, inconvenience, and discomfort for repeat testing without accruing a benefit. In addition, the decision-making by their clinical team is delayed.

During the evaluation of the computerized alerts, anecdotal observations identified positive changes in workflow. The computerized alerts could be sorted by severity, type, date, and last name. The nurses could quickly find patients with abnormal values and trends and could review laboratory results online and respond from the hospital setting where physicians were accessible for consultation. Previously, the nurses could only manage laboratory reports while physically in the transplant office viewing the paper reports and would need to page the physicians to discuss new results. Second, the nurses were alerted when previously reported results were corrected. Third, all the nurses managed one list of alerts for all the liver transplant patients and could view all new alerts and actions taken by others in response to previous alerts. This design was essential for the nurses to provide cross-coverage for one another and for the support staff to transcribe information to the paper flowcharts. Fourth, the repeated overdue alerts prevented loss to follow-up and continued to be received until a new result was saved to the EHR or the patient was removed from the protocol.

The computerized alerts improved the quality of data available for decision-making on the paper flowchart. The paper flowchart was used throughout the study because clinicians needed to view laboratory results and drug dosage side by side to make many, but not all, dose adjustment decisions. Medications are not yet stored in the EHR in a manner to support this view. The alerts improved the efficiency of “pulling charts” and improved the completeness of results recorded on the flowchart. Among results collected within 7 days prior to the chart review, the proportion of results missing on the paper flowchart decreased from 33% to 16% after implementing alerts. ¹³ Among results collected 14 days to 4 months prior to the chart review, the proportion of missing results decreased from 12% to 5% after implementing alerts. ¹³ The change was significant for both time intervals (Pearson chi-square ≤ 0.02). ¹³

Several factors potentially limited the study. First, the study team assumed that all routine chemistry laboratory results would include a creatinine level result. Creatinine was used as a proxy for all routine chemistry results. If, for some reason, a creatinine test was not included in a chemistry panel, then an alert for other transplantation-related chemistry tests would only be triggered if there were abnormal magnesium or potassium results. Second, we assumed that clinicians would review all new laboratory results in response to an alert related to a creatinine result, because the project did not want to inundate clinicians with alerts for each individual laboratory test. The study used creatinine as a proxy for other routine tests. Third, we assumed that the EHR would provide a reliable method for delivering computerized alerts. Data in the EHR were more accessible and reproducible than in the paper record. Paper records can be misplaced and are only available in one place at one time. We found that the <0.1% reporting failure rate of the computerized system was much lower than the rate of unreported lab reports using traditional reporting methods (33%). Faxed and printed reports may have been missing due to incomplete requisition forms, incomplete registration at the time of phlebotomy, or incomplete transmission of results. Fourth, we assumed that liver transplant patients would be enrolled in the liver transplant protocol in order to generate alerts. Enrollment could be delayed or accidentally discontinued; however, the nurses were motivated to enroll patients soon after transplantation and the nurses immediately received an alert if a patient's enrollment were discontinued. Fifth, the three-week time period used to assess the traditional reporting process may not be representative; however, the assessment spanned the beginning of the month when test frequency is greatest, and no unusual events or staffing deficits occurred. Sixth, the end-point for calculating response time was different for the two reporting systems; however, the comparison was conservative. The difference in timeliness would have been greater if we had used the same end point. Finally, improvements in laboratory reporting could have been due to faster specimen transport and processing during the data collection period in 2005. The time to finalize tacrolimus tests from an Intermountain laboratory other than LDS Hospital decreased by seven hours between March 2004 and February 2005 (p = 0.01). However, there was no change in the number of hours to finalize creatinine results from any laboratory, or tacrolimus results from LDS Hospital (p > 0.1). Despite these potential study limitations, the study found the computerized reporting system to provide more complete, timely, efficient, stable, and flexible information than the traditional system of faxes and printouts. The changes in workflow, and the ability for the entire team to efficiently manage new, abnormal and overdue results, are expected to have a positive impact on patient safety. As of December 2007, the liver transplant team continues to use the system daily to manage their patients and receive 50–60 alerts each day.

Laboratories are required to report results in a timely, accurate, and reliable manner, ensure that specific content is included with each report, and monitor post-analytic variables between the completion of testing and receipt by the requesting physician. ^27,28 Laboratories are urged to help physicians identify and respond to clinically important laboratory results, assess the timeliness of reporting from a clinician's perspective, and recognize that improving timeliness is a duty to patients and clinicians. ^29,30 However, reporting requirements do not specify the mode of transmission or any limits on the number of reports sent. Laboratories should consider the negative impact of duplicate reporting. Problems with incomplete, late, and duplicate reporting may be experienced by any outpatient clinician using the traditional reporting processs. In 2005, a national survey of 2,569 medical group practices found that laboratory result management systems varied by practice size, implementation of an EHR, and geographic region. ³¹ Overall, 47% of all medical group practices used only a manual system of paper documents, fax and telephone requests to manage laboratory results and an additional 30% use a mix of manual and computerized systems. ³¹ Only 21% of the practices reported that they primarily used a computerized laboratory management system. ³¹ Currently, most laboratories mail or fax reports to outpatient clinicians because 1) most outpatient settings do not have an EHR to receive the results; ^31,32 2) laboratories in the United States are not required to electronically report results to clinicians; and finally, 3) the laboratory is often external to the outpatient setting and there are logistical, economic, and regulatory issues that need to be resolved before laboratories can directly send their results to an EHR outside their network. ^33,34

Computerized outpatient result management systems are starting to be developed for advanced EHR systems ⁸ and the percent of outpatient clinics with EHRs are expected to increase over time. ^31,35 Therefore, there are opportunities in the future to develop decision support systems to manage laboratory result reporting. Computerized alerts efficiently identify the occurrence, trends, and omissions in test results routinely reviewed by clinicians. Computerized alerts have an added benefit of being able to identify critical values relevant to the patient population, not simply the standard critical values.

Conclusion

The computerized alerts met LDS Hospital and Intermountain Healthcare objectives to inform the nurses when liver transplant patients had new, abnormal, or overdue laboratory results. Computerized alerts were timely, complete, and efficient in reporting information for the management of liver transplant patients. The time required to respond to routinely collected outpatient laboratory results was reduced from a median of 33 to 9 hours after specimen collection.