A Computerized Alert Screening for Severe Sepsis in Emergency Department Patients Increases Lactate Testing but does not Improve Inpatient Mortality

Abstract

Objective

This study tested the hypothesis that lactate testing in ED sepsis patients could be increased using a computer alert that automatically recognizes systemic inflammatory response syndrome (SIRS) criteria and recommends lactate testing in cases of sepsis defined as ≥2 SIRS criteria plus physician suspicion of infection. Secondary outcomes included the effect of the alert on lactate testing among admitted sepsis patients, the proportion of admitted patients with lactate ≥4.0 mmol/L identified and the in-patient mortality difference before and after alert implementation.

Methods

After a 6 month pre-alert phase, a computer alert was implemented that computed and displayed abnormal vital signs and white blood cell counts for all patients with >2 SIRS criteria and recommended testing lactate if an infection was suspected. Data for admitted patients was collected electronically on consecutive patients meeting sepsis criteria for 6 months before and 6 months after implementation of the alert.

Results

There were a total of 5,796 subjects enrolled. Among all septic patients, lactate testing increased from 5.2% in the pre-alert phase to 12.7% in the alert phase, a 7.5% (95% CI 6.0 to 9.0%) absolute increase in lactate testing, p<0.001. Among the 1,798 admitted patients with sepsis, lactate testing increased from 15.3% to 34.2%, an 18.9% (95% CI 15.0 to 22.8%) absolute increase, p<0.001. Among admitted patients with sepsis, there was a 1.9% (95% CI 0.03 to 3.8%, p = 0.05) increase in absolute number of patients with elevated lactate levels identified and a 0.5% (95% CI -1.6 to 2.6%, p=0.64) decrease in mortality.

Conclusion

The proportion of ED patients who had lactate tested and the number of admitted patients identified with a lactate level ≥4.0 mmol/L improved significantly after the implementation of a computer alert identifying sepsis patients with >2 SIRS criteria while mortality among admitted sepsis patients remained unchanged.

Introduction

Background

Sepsis is a major health concern with a substantial impact on health care utilization in the US [1]. Sepsis is the systemic spread of an infection with manifestations such as fever, tachycardia, tachypnea and an acute elevation in white blood cell count. As sepsis progresses, severe sepsis, defined as infection with acute organ dysfunction without persistent hypotension, may develop. The diagnosis of severe sepsis is supported by evidence of acute failure in one or more of a number of organ systems during times of stress related to infection. One commonly used biomarker is serum lactate, which is a surrogate for metabolic dysfunction and decreased organ tissue perfusion [2-7]. Septic shock, the most serious degree of disease, is sepsis with persistent hypotension (mean arterial blood pressure <65 mmHg) despite aggressive efforts at fluid resuscitation (20 to 30 cc/kg intravenous crystalloid fluid challenge). Severe sepsis and septic shock account for approximately 571,000 annual U.S. Emergency Department (ED) visits and 215,000 deaths [1].

Severe sepsis is more challenging to diagnose than septic shock in the ED largely because of the lack of blood pressure abnormalities [5]. Therefore it is important to utilize lab tests to assist in recognition of organ dysfunction in septic patients who are not hypotensive. Hyperlactatemia (serum lactate ≥4.0 mmol/L) is associated with an increased risk of short-term mortality in emergency department (ED) patients with suspected infection and has been found to herald poor clinical outcomes in patients with infections regardless of blood pressure [3, 5-7]. During systemic infection, hyperlactatemia is believed to be caused by decreased end-organ perfusion, leading to anaerobic glycolysis and lactate accumulation [7-11]. As such, serum lactate is an important biomarker for detecting organ dysfunction in severe sepsis.

Recently, early goal directed therapy (EGDT) has emerged as a validated protocol, recommended by the Surviving Sepsis Campaign to treat patients with severe sepsis or septic shock. Implementation of EGDT has demonstrated significant improvements in survival at 28 and 60 days and a 34.4% relative reduction in in-hospital mortality [4, 12-16]. The guidelines recommend a 6 hour care bundle in the emergency department with resuscitation measures based on optimizing hemo-dynamic parameters of central venous pressure and mean arterial pressure as well as the central venous oxygen saturation (ScvO₂) [4]. A recent study has demonstrated that in-hospital mortality was not significantly different among ED patients with septic shock randomized to testing and optimizing lactate clearance to at least 10% within 6 hours when compared to optimizing ScvO₂ to at least 70% using costly invasive central venous catheters. The results of this study advance our understanding of the use of serum lactate levels in the care of patients with septic shock, highlighting its use as a diagnostic test and as a guide to resuscitation [17].

Despite the demonstrated advantages of EGDT, evidence suggests that ED implementation of this therapy remains inconsistent. A national survey of ED directors and nurse managers reported identification of patients with sepsis to be one of the most commonly cited barriers to implementation of EGDT in the ED [18]. Measurement of lactate is of great importance for identification of severe sepsis in ED patients with suspected infection who do not have persistent hypotension.

The challenge associated with detecting severe sepsis in ED patients can potentially be addressed by adapting computerized technology that already exists in the ED. The work flow of most ED physicians is tied to computer work stations where a variety of clinical patient data are brought together and made available. Clinical decision support (CDS) is a form of information system designed to improve clinical decision making by providing computer guidance to clinicians at the point of care [19]. For the identification of sepsis, CDS systems can synthesize data from vital signs and lab tests already collected electronically and physicians can be prompted to order a lactate level if certain criteria are met.

A recent systematic review of over one hundred studies of CDS systems found that 64% of 97 studies on the effect of CDS on practitioner performance demonstrated improved diagnosis, preventive care, disease management and drug management [20]. Among these studies, a number were carried out in the ED, but none examined the ability of CDS to assist in screening patients for severe disease by incorporating vital signs, lab data and physicians’ clinical judgment [21-31]. ED care, by its very nature, lends itself to cognitive errors due to high volumes of severe acuity patients, a need for rapid and concise decision making and limited staff. Therefore CDS may be particularly valuable in this environment.

The primary goal of this investigation was to determine if the addition of a sepsis alert to a computer provider order entry (CPOE) system could improve the test rate for serum lactate used to screen for severe sepsis in ED patients presenting with sepsis.

Methods

Design

The study used a quasi-experimental pre-post interventional design. Data on the proportion of ED sepsis patients with serum lactate testing performed during their ED stay was collected in six-month periods before and after the implementation of an automatically generated computerized alert reminding clinicians to order a serum lactate for patients meeting criteria for sepsis. Data were collected from 2/15/09 through 2/14/10. The study received an expedited approval by the institutional review board of the Albert Einstein College of Medicine (AECOM).

Selection of participants

Computerized records of all visits by ED patients were included in the study if patients met the following criteria: age 19 years and older with sepsis defined as meeting two or more systemic inflammatory response syndrome (SIRS) criteria [temperature >38°C or <36°C; heart rate >90 beats per minute; respiratory rate >20 breaths per minute; white blood cell count >12,000 cells/mm³ or <4000 cells/mm³, or >10% immature bands], and clinical suspicion of infection. Subjects could be included more than once as the study was designed specifically to evaluate clinician response to the computerized alert. Three of the four principal investigators (TB, AB and PG) were working clinically at the study site during the course of the project. Data collected during their ED shifts were included as they were not directly involved in ordering of laboratory tests during the study period and typically discussed medical management with residents after labs were ordered. Furthermore, the identity of the study investigators and the purpose of the alert were not public knowledge.

Setting

The study was performed at an urban ED, Jacobi Medical Center, a tertiary care level 1 trauma center with an established EM residency program and an annual adult volume of 70,000. The ED has a common commercially available CPOE system called Quadramed CPR™ that allows for automated electronic ordering of lab and radiologic testing, drug ordering, results retrieval and site specific modification of order entry, alert development and results display. Vital signs are entered directly into the system by the ED triage nurse. All lab results, including complete blood count results are entered by lab staff at the time of reporting. The only vital signs evaluated for study entry were those collected at ED triage and the lab results used were the first obtained in the ED. All lab results utilized in this study were extracted from the CPOE system only when the electronic time stamp associated with the results fell between the time of ED triage and the closure of the ED chart ensuring that they were ordered and reported during the ED visit. All lab tests utilized for this study were run in the hospital’s clinical lab and lactate blood specimens were drawn from venous samples into gray top tubes specifically designed to inhibit glycolysis and false elevations in serum lactate levels. Lactate samples are not routinely transported on ice as they are immediately sent to the lab via the pneumatic tube system.

Intervention

An IT consultant developed a CDS alert, working in concert with the study authors, which was incorporated into the ED CPOE system. This alert presented ED clinicians with a prominent message if patients met SIRS criteria based on the electronically entered triage vital signs and white blood cell (WBC) count, stating: “Alert: This patient may have severe SEPSIS. Consider ordering a LACTIC ACID LEVEL if you suspect infection.” This message was followed by the specific vital signs and, if appropriate, the WBC count obtained by the CDS from the electronic record (►Figure 1). The alert was immediately triggered when SIRS criteria were met and remained on display for the duration of the patient’s ED stay. The purpose of the alert was to encourage the appropriate ordering of lactate levels to screen sepsis patients for the presence of severe sepsis. If the alert was triggered for a non-septic patient and a lactate was tested, the lactate level would presumably be sufficiently low enough to preclude aggressive EGDT intervention. The alert was not powered to detect sepsis in patients without SIRS criteria, therefore those patients’ lactate screening was not influenced by a CDS intervention. There were no other efforts made by the research group or clinical department to increase lactate screening at the institution during the study period.

Fig. 1.

Example of the CDS Alert display as it appeared in an electronic ED patient chart during the study period

Methods of measurement

Sepsis

Sepsis was defined as the presence of two or more SIRS criteria plus suspected infection. In order to identify patients with SIRS criteria in both phases of the study, temperature, respiratory rate and heart rate were obtained automatically by the CDS system from the triage vital signs. The WBC count was automatically retrieved from the electronic documentation of lab results. To obtain clinician suspicion for infection, the closure procedure for all electronic ED charts was configured by the IT department with a mandatory query asking the treating provider: “Do you suspect that this patient has an infection? (yes/no)”. The query, in fact, had no impact on the alert and was included at the closure of each ED chart to capture and ‘tag’ every patient who was suspected of having an infection. This provided the authors with a denominator of ED patients with the common element of suspected infection necessary to consider sepsis as a potential concern in the ED. Then every patient with a suspected infection was screened retrospectively for SIRS criteria and those with both suspected infections and SIRS criteria comprised the sepsis cohort analyzed in the study. The query was placed at the closure point of the ED chart because this was the only point during ED care that this data element could be universally captured for all study patients. (►figure 2). All clinicians in the department were asked by email, prior to the study period, to respond to the query with “yes” only if they suspected that an acute infection was associated with the chief complaint for the patient’s ED visit.

Fig. 2.

Conceptual flow of the CDS alert and patient selection. (Patients with sepsis were included in the study); Numbered vital signs and WBC count represent SIRS criteria cutoffs for each parameter.

Outcomes

The primary outcome measure was the proportion of ED sepsis patients with lactate tested in the two periods. Patients meeting SIRS criteria with an affirmative response to a physician query regarding suspected infection were identified and the presence or absence of an ED lactate test was retrieved from electronic patient data. Only the first serum lactate level obtained in the ED prior to closure of the ED chart for either admission or discharge was included in the study.

Secondary outcomes included the proportion of admitted patients with sepsis who had a serum lactate level obtained before and after implementation of the alert, the in-patient mortality among all admitted patients with sepsis and the proportion of admitted sepsis patients who had a serum lactate ≥4.0 mmol/L identified in both phases of the study. Admission data were obtained by query of the electronic ED record and mortality data for admitted patients were provided separately by the medical records department.

The decision was made to separately assess the effect of the alert on the admitted subgroup based on the assumption that these patients were likely to be the group with more serious infections who would derive the most benefit from lactate testing and recognition of severe sepsis within the 6 hour window for aggressive care. Thus, separately quantifying the response to the alert for admitted patients was considered an important part of the analysis. Mortality data was not available for discharged patients.

The difference in the proportion of patients with serum lactate ≥4.0 mmol/L among admitted patients was measured because increasing the identification of severe sepsis has the potential to lead to increased EGDT performance in patients who might otherwise have entered the hospital with the extent of their disease undetected and less than optimal clinical care provided in the ED.

Mortality is reported as an outcome among admitted patients to demonstrate whether an improvement in process (lactate ordering) can translate into a measurable change in clinical outcomes. This outcome was chosen based on the assumption that increased lactate testing may increase the number of patients identified with severe sepsis and thus lead to an increase in EGDT performance and a decrease in mortality among patients with severe sepsis.

Data collection and processing

Data were collected by the IT and medical records departments of the institution and reports were sent in Microsoft Excel spreadsheet format via secure electronic transfer.

Data analysis

All statistical analysis was performed using STATA version 10, (College Station, TX). Descriptive statistics were calculated to characterize the samples in the two time periods. Differences in percentages of reported outcomes between pre and post intervention phases were reported with 95% confidence intervals. Chi square tests were used to compare the percentages between the two periods. Normally distributed continuous variables were compared using t-tests taking into account equality of variance and non-normally distributed continuous variables were compared using Mann Whitney U tests. Binary variables were also compared using chi square tests.

Results

5,796 subjects met SIRS criteria and had suspected infection during the study period. These subjects were defined as having sepsis and comprised our main study sample. 1,798 of these 5,796 patients were admitted to the hospital. Characteristics of study subjects are presented in ►table 1.

Table 1.

Demographics Characteristics, Presenting Vital Signs, and Laboratory Data Stratified by Alert Phase

		All patients with sepsis		Admitted patients with sepsis
		Pre-Alert	Alert	Pre-Alert	Alert
		N = 2903	N = 2893	N = 908	N = 890
Demographic Characteristics
Age in years mean (standard deviation)		42.2 (17.6)	42.4 (18.2)	53.6 (18.3)	54.3 (18.8)
Sex female N (%)		1721 (59.3)	1703 (58.9)	450 (49.6 )	462 (51.9)
Race n (%)	Hispanic	1368 (47.1)	1395 (48.2)	399 (43.9)	353 (39.7)
	African American	939 (32.4)	810 (28.0)	300 (33.0)	249 (28.0)
	White	265 (9.1)	321 (11.1)	82 (9.0)	122 (13.7)
	Other	331 (11.4)	367 (12.7)	127 (14.0)	166 (18.7)
Vital Signs mean (standard deviation)
Temp (°F)		99.2 (2.0)	98.8 (2.0)	99.1 (2.2)	98.9 (2.2)
Heart Rate (beats/min)		104 (14.2)	104 (15.6)	107 (16.6)	107 (18.1)
Respiratory Rate (breaths/min)		19 (5.3)	19 (6.8)	20 (7.0)	20 (6.9)
Laboratory Data mean (standard deviation)		N = 1610†	N = 1592†	N = 903†	N = 884†
White Blood Cell Count (cells/mm3)		11.3 (6.2)	12.0 (6.9)	12.5 (7.1)	12.9 (6.8)

† Reflects the number of subjects out of the total in each column who had a WBC count tested

As shown in ►table 2, ED lactate testing increased significantly after implementation of the alert in all patients and in the subset of patients who were admitted to hospital. The magnitude of the change was greater for hospitalized patients, than for the entire group of patients. The absolute increase pre and post-alert in lactate orders was 18.9% for hospitalized patients and 7.5% for all patients. The proportion of sepsis patients who had lactate levels obtained in the ED for each month of the study is presented in ►figures 3 and 4. The proportions of lactates obtained for patients who were admitted are shown in ►figure 4.

Table 2.

Change in frequency of lactate tests ordered before and after the alert

			Lactate tests
		n	n (%)	% Difference	95% CI of difference
All Patients	Pre-alert	2903	151 (5.2)	7.5	(6.0 to 9.0)
	Post alert	2893	366 (12.7)
Admitted Patients	Pre-alert	908	139 (15.3)	18.9	(15.0 to 22.8)
	Post-alert	890	304 (34.2)
Discharged Patients	Pre-alert	1995	12 (0.6)	2.5	(1.7 to 3.3)
	Post alert	2003	62 (3.1)

Fig. 3.

Percent of subjects with sepsis who had a lactate tested by month among all sepsis patients. An analysis of the proportion of lactate tests among all sepsis patients in the first two months following alert implementation (months 7 and 8, n= 87/836, 10.4%) compared with the last two months of the alert phase (months 11 and 12, n= 134/1033, 13.0%) demonstrated no decrease in lactate testing.

Fig. 4.

Percent of subjects with sepsis who had a lactate tested by month among all admitted sepsis patients. An analysis of the proportion of lactate tests among all admitted sepsis patients in the first two months following alert implementation (months 7 and 8, n= 70/267, 26.2%) compared with the last two months of the alert phase (months 11 and 12, n= 108/300, 36.0%) demonstrated no decrease in lactate testing.

After implementation of the alert, screening for severe sepsis increased the absolute number of patients identified with hyperlactatemia from 33 to 54. However the proportion of patients who had their lactates tested and had elevated lactates decreased from the pre-alert to the alert phase of the study as shown in Table 3 due to the increased number of patients tested in the alert phase.

Table 3.

Change in frequency of hyperlactatemia in patients with lactate tested before and after the alert

		n	n (%)	% Difference	95% CI of difference
All Patients	Pre-alert	151	33 (21.9)	7.1	(-0.4 to 14.6)
	Post alert	366	54 (14.8)
Admitted Patients	Pre-alert	139	33 (23.7)	7.6	(-0.6 to 15.8)
	Post-alert	304	49 (16.1)
Discharged Patients	Pre-alert	12	0	8.1	(1.3 to 14.9)
	Post alert	62	5 (8.1)

Fifty-two of 908 admitted patients with sepsis died (5.7%) during the pre-alert phase compared with 46 of 890 (5.2%) during the alert phase, a 0.5% (95% CI -1.6 to 2.6%, p = 0.64) absolute decrease in deaths.

Discussion

Results of this study demonstrate that implementation of an emergency department CDS alert was associated with improved clinician adherence to sepsis screening guidelines as measured by increased testing of serum lactate. This effect was obtained in the absence of any specific lectures, emails or conferences aimed at increasing ED sepsis screening. The effect was small and this was likely due to the absolute lack of implemented education interventions or clinical reminders in the ED to support the alert.

The CDS alert was most effective at increasing the absolute percentage of ED lactate ordering among patients admitted to the hospital with sepsis. The results suggest that clinicians may have incorporated clinical judgment in their response to this alert and that alerts of this type may have the greatest impact when they are most necessary in the sickest patients. It is also important to note that the alert’s effect did not decrease over the 6 month alert phase. (►figures 3 and 4).

The absolute number of patients with elevated lactate levels was higher in the alert phase of the study. However the proportion of patients tested who had high lactate levels was lower in the alert phase. This reflects the trade-off between the ability to uncover occult severe sepsis through utilization of an alert to increase lactate testing as a screening tool versus the expense of testing a greater number of lactate levels among ED patients with sepsis. The mortality benefit of EGDT in the treatment of patients with severe sepsis may make it worthwhile to cast a wide net and screen patients liberally to identify those who qualify for enrolment.

Despite the increase in lactate testing, alert implementation was not associated with a reduction in sepsis mortality in the study. There are several possible explanations for this. First, despite the increase in lactate testing, the rate of testing was low in both phases of the study, thus limiting its potential impact on mortality. Secondly, the initiation of EGDT is unknown following identification of elevated lactate levels. The assumption that increasing early lactate testing as a screening tool in the ED would be expected to increase the number of patients with severe sepsis identified and, thus increase the number of patients who would benefit from EGDT may not be correct. This study was not designed to measure the influence of the CDS intervention on EGDT performance in the ED. However, this is a direction for future studies and is the next logical step in evaluating the overall clinical impact of CDS implementation to improve ED severe sepsis management.

This study brought together evidence based clinical methodology for sepsis screening and attempted to use the CPOE system to enhance awareness of sepsis as a potential syndrome in patients with SIRS criteria. This study utilized an automatic alert because prior studies of CDS systems have demonstrated better performance of systems that automatically prompt clinicians when compared with those that require users to actively initiate the system [20]. The ED is a potential environment that may benefit from CDS implementation. Future work should focus on the ability of alerts to improve EGDT performance and should examine the impact of CDS on clinical response at the point of care to illuminate the effect that the alert has on all aspects of sepsis management. The addition of educational interventions as well as reminders to follow the recommendations of clinical alerts may also be of importance in evaluating the impact of CDS alerts of this kind.

This study contributes to the previous body of work that has demonstrated process improvements in clinical care through CDS alerts and demonstrates the importance of continuing to explore the true clinical impact of electronic decision support in the ED.

Limitations

This study had several important limitations. First, there was no randomization. Randomization of this type of intervention by individual patients was not considered feasible due to the likelihood of a contamination effect of an alert so ubiquitously present in the ED. Future studies might address this issue through the use of a multicenter design with cluster randomization, or by randomization of providers within institutions.

The current study design also does not allow for control of secular trends in sepsis management that may have occurred during the study period. The authors were aware of no major shifts in sepsis management in the clinical department during the duration of the study. It is possible that knowledge of the purpose of the study may have influenced clinician ordering behavior. Efforts were made to minimize this by limiting discussion of the purpose of the study and did not provide clinicians with any official guidance on use of the alert.

There was no evaluation of the extent of protocolized care provided to patients during the study period as this was not within the scope of the study. As such, this study was unable to address the effect that the alert may have had on individual components of EGDT for severe sepsis.

The decision was made to inquire about suspected infection electronically at the close of the ED chart because this was the single common point identified at which physician input could be captured. As a result , there was no way to incorporate a documented suspected infection into the alert logic, therefore the alert had to specify that if the clinicians suspected infection, they should check a lactic acid level.(►figures 1 and 2). Display of the alert in cases in which SIRS was not related to sepsis may have attenuated the effect of the alert on clinician behavior. An ideal alert of this type would synthesize clinician response to a suspected infection query and limit alert display to only those patients with clinician suspicion for infection.

Finally, the study was conducted in a single hospital. The results (effect of the alert on clinician behavior) may not be generalizable to other sites with different types of practitioners and different baseline compliance with sepsis guidelines.

Conclusions

The results of this study demonstrate the ability of a CDS alert to modestly improve emergency department lactate screening for severe sepsis without showing signs of fatigue over time. Inpatient mortality was not changed by implementation of this alert. The impact of this alert on lactate screening behavior was relatively small and demonstrates the limitations of using alerts of this type in isolation without associated education interventions or clinical reminders.

Future studies should focus on CDS interventions in the ED with an effort to study and improve clinical outcomes through their use and to increase the intervention effect size by including education interventions and reminders.

Conflicts of Interest

None of the listed authors had any financial or personal relationships with any person or organization that might influence or bias the objectivity of the submitted content of this study or its acceptance for publication.

Protection of Human Subjects

The procedures used in this study have been reviewed in compliance with ethical standards of the responsible committee on human experimentation at AECOM and with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.

Funding Sources

This publication was made possible by the CTSA Grant UL1 RR025750 and KL2 RR025749 and TL1 RR025748 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessary represent the official view of the NCRR or NIH. This research was partially funded by an Empire State ECRIP grant.