Abstract
Introduction
The aim of this systematic review was to determine the impact of automated patient monitoring systems (PMSs) on sepsis recognition and outcomes.
Methods
Systematic searches were conducted using CINAHL, MEDLINE, and Cochrane, for articles published from 2008 through 2018. English-language, peer-reviewed articles that reported the impact of PMS on sepsis care were included. For selected articles, the authors abstracted information, with the study designed to be compliant with Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.
Results
Nineteen articles were identified for inclusion: 4 systematic reviews and 15 individual studies. Study design and quality varied, with some randomized controlled trials and quasiexperimental studies, as well as many observational studies. Study results for outcome measures (e.g., mortality, intensive care unit [ICU] length of stay, ICU transfer) were mixed, with more than half of the studies showing a significant improvement in at least one measure. Evidence for process measure (e.g., time to antibiotic administration, lactate measurement, etc.) improvement was of moderate strength across multiple types of hospital units, and evidence was most consistent outside the ICU.
Conclusions
Automated sepsis PMSs have the potential to improve sepsis recognition and outcomes, but current evidence is mixed on their effectiveness. More high-quality studies are needed to understand the effects of PMSs on important sepsis-related process and outcome measures in different hospital units.
Key Words: sepsis, sepsis screening, decision support systems, computerized alert, informatics
Sepsis is a syndrome of life-threatening organ dysfunction due to a person’s systemic, dysregulated response to infection.1 It is a common reason for hospital admission and readmission, with an estimated incidence of 6% of all hospital admissions or more than 1 million admissions in the United States every year.2,3 Sepsis also has one of the highest mortality rates of any hospital condition, estimated at 15% to 30%.3,4 In addition to its prevalence and mortality, $24 billion was spent treating sepsis in 2013, more than any other condition treated in U.S. hospitals.5
In response to this, international organizations such as the Society for Critical Care Medicine have focused on addressing the 2 problems that sepsis presents: delay in recognition and diagnosis of sepsis, and delay in start of treatment, which combined contribute to the high mortality rate for sepsis.6 This article focuses on an emerging solution to the former problem: identifying signs of sepsis in a patient as early as possible so that treatment can be started, which is critical to averting organ failure and risk of death.1
The symptoms of sepsis are shared by many other conditions (e.g., high temperature, low blood pressure, etc.), making sepsis difficult to diagnose, especially in the early stages.7 As a result, sepsis does not have a simple diagnostic test or specific symptoms that unambiguously indicates onset. International organizations have developed diagnostic criteria and recommend screening patients at risk of sepsis using these criteria.2
Although manual screening has been the most common approach in the past, automated electronic patient monitoring (i.e., surveillance) for signs of emerging sepsis is becoming more widespread, especially in hospitals, which have sophisticated technology infrastructures. Such systems automatically and continuously monitor data from telemetry devices and/or electronic health record entries, and alert a clinician if set criteria for sepsis are met. The alerting system can take many forms depending on the study and setting, from paging a specific team member to displaying a noninterruptive alert on a patient dashboard. If, after alert and evaluation, a clinician determines that the patient has sepsis, the clinician must start treatment immediately to reduce mortality and improve patient outcomes.2 The goal is to decrease the time to treatment initiation for sepsis, which has been shown to increase survival.3,4 This article reviews the current literature on the effectiveness of these patient monitoring systems (PMSs) for sepsis, as well as implementation facilitators and barriers.
METHODS
We conducted a search of 3 databases (CINAHL, MEDLINE, and Cochrane) for articles published from 2008 to 2018 to identify those that described the use of a PMS to improve sepsis recognition and care. Search terms included “sepsis” and related synonyms, as well as “monitoring,” “surveillance,” and other similar terms. English-language, peer-reviewed articles that reported the impact of PMS on sepsis care were included.
The lead author reviewed the title and abstract for the retrieved articles to determine relevance to the study objectives. Articles were excluded if the outcomes were not relevant, if the article was out of scope (including not quantitative), or if the study design was insufficiently described. If an article was deemed relevant or additional information was needed to make that determination, the full text was obtained and reviewed by the lead author. Through a consensus process, the 2 coauthors selected the articles for inclusion based on relevance, quantified outcomes, and sufficiently described study design. If a study was referenced within a selected systematic review, it was not included as a separate study. For selected articles, the authors abstracted information into an evidence table. The study was designed to be compliant with Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.8
RESULTS
The initial search yielded 345 results; after duplicates were removed and additional articles added, 350 were screened for inclusion and 55 full-text articles were retrieved. Of those, 15 individual studies and 4 systematic reviews were selected for inclusion in this review.
Results of the studies are reviewed by measure type (process and outcome) and setting. Of note, the sepsis diagnostic criteria as well as the sensitivities and specificities of PMSs are not examined because the algorithms within PMSs that scan for sepsis can be constantly adjusted to fit the needs of the setting and optimize performance, as opposed to a static manual screening tool. Most included studies based their diagnostic criteria on those developed by consensus-based professional organizations such as the systemic inflammatory response syndrome criteria and Modified Early Warning Score, but some studies used other indicators and thresholds. Upon designing and implementing a sepsis PMS, the clinicians/administrators typically test the system performance and adjust variable thresholds to best balance speed, sensitivity, and specificity for their setting.
All included individual studies took place in the hospital setting, 5 in the intensive care unit (ICU), 5 in the emergency department (ED), 3 in general units, 1 in a telemetry unit, and 1 in multiple hospital units (ICU, pediatric intensive unit [PCU], and medical/surgical units).
Effect on Outcome Measures
The patient outcomes used to determine the effectiveness of the studies of automated PMS included mortality, ICU transfer rate, hospital length of stay (LOS), and ICU LOS. Outcome measures were reported in 12 studies: 3 in the ED,9–11 5 in the ICU,12–16 2 in general units,17,18 1 in a telemetry unit,19 and 1 in multiple hospital units (ICU, PCU, and medical/surgical units).20
Eight of the 12 studies found a significant effect of a sepsis PMS in improving at least one outcome measure, and others showed absolute but not statistically significant improvements. The studies that showed a significant improvement included 2 randomized controlled trials (RCTs), 1 quasiexperimental study, and 5 observational studies. Six of the 12 studies that reported mortality showed a statistically significant decrease after implementing a PMS. For example, Manaktala and Claypool18 found that after implementing the sepsis PMS, screened patients had a 2.1 times lower risk of death (odds ratio, 0.474; 95% confidence interval, 0.228–0.988). A study in 9 neonatal ICUs across the United States showed a significant reduction in mortality (8.1% versus 10.2%, P = 0.04) after implementing a neonatal sepsis PMS.16
Nine studies reported on hospital LOS, and 4 found a significant effect of the sepsis PMS. For example, McCoy and colleagues20 found a 9.55% decrease in hospital LOS after the implementation of a machine learning–based PMS in multiple hospital units (ICU, PCU, and medical/surgical units) in a 242-bed regional community hospital. In contrast, Manaktala and Claypool,18 described previously, showed a significant decrease in mortality but did not find a significant decrease in hospital LOS.
Only 1 of the 4 studies that reported on ICU LOS found a significant effect from a PMS. This was an observational study of a PMS implemented in a 34-bed surgical ICU in a large academic medical center.13 The studies that found no effect on ICU LOS varied in setting, with 1 implemented in the ED, 1 in a medical ICU, and 1 in all noncritical care units.21 One study attributed lack of impact on ICU LOS to a PMS with poor predictive value,9 and one credited the already vigilant ICU staff12; the third was underpowered to detect modest changes in ICU LOS. Two studies reported on ICU transfer rate, and neither found a significant effect on this or any other outcome measure.9,17 Several studies that showed no significant effects on outcome measures showed significant effects on process measures, for example, Umscheid and colleagues.17
Effect on Process Measures
Although assessing PMSs for effects on outcome measures (e.g., mortality) is the ultimate goal of this patient safety practice, it is also important to evaluate whether a PMS improves sepsis care processes. Process measures are typically based on evidence-based clinical recommendations, and an improvement in the measures would indicate that patients are receiving care that has been shown to lead to better outcomes. Processes that are commonly targeted for improvement are the timely administration of antibiotics, lactate measurement, blood culture draw, and fluid administration. One or more process measures for sepsis PMSs were reported in 9 studies: 4 in the ED,9–11,22 3 in the ICU,12–14 and 2 in noncritical care units.17,23 Studies had various designs including 2 RCTs,12,14 1 quasiexperimental study,10 and 6 observational pre/post studies.9,11,13,17,22,23 In addition, 4 systematic reviews covered this topic to some degree.21,24–26 The most commonly reported process measure was time to antibiotic administration (n = 8), followed by time to lactate measurement and blood culture draw (n = 5 each), and time to fluid administration (n = 3).
A systematic review by Warttig and colleagues,24 which included RCTs conducted in the ICU through September 2017, determined that there is very low-quality evidence for any improvement in time to antibiotic administration after implementation of a PMS. None of the reviewed studies showed a significant improvement. Three other systematic reviews found mixed results on improvement in sepsis process measures (including time to antibiotic administration among others mentioned previously).21,25,26 Despins25 searched for automated sepsis detection in the hospital setting (ICUs, EDs, and general wards) from 2005 to 2015; Makam and colleagues26 searched for electronic sepsis recognition systems in ICUs, EDs, and general wards through June 2014; and Alberto and colleagues21 searched for sepsis screening systems in only general hospital wards through June 2016. Several studies these authors reviewed (all observational and all outside the ICU) reported that PMSs significantly improved time to administration of antibiotics, lactate draw, blood culture draw, and/or fluid administration. For example, Narayanan and colleagues,11 after implementing a PMS in the ED of an academic medical center, found that average time to antibiotic administration decreased from 61.5 to 29.0 minutes (P < 0.001). The authors of one systematic review hypothesized that PMSs in the ICU may not be as effective as those outside the ICU because clinicians in the ICU are already vigilant for signs of patient deterioration, so a sepsis PMS in this setting may be redundant, among other reasons.23
Of the 6 studies we reviewed that were published after the systematic reviews were conducted, 5 found a significant effect of a PMS on at least one process measure. Of these 5, one was an RCT and the others were observational studies. An RCT in 2 ICU units with a total of 32 beds at an urban medical center found that patients with automated sepsis monitoring received antibiotics an average of 2.76 hours earlier than did patients in the control group and had blood cultures drawn an average of 2.79 hours earlier than did patients in the control group.14 Austrian et al9 was the only new study that found no effect of a PMS on time to first lactate measurement or antibiotic administration before blood cultures in the ED and urgent care units of an urban academic medical center. This was a pre/post observational study with control of possible cofounders, and the authors suggested that alert fatigue from a tool with low positive predictive value contributed to the lack of impact on process measures.
DISCUSSION
Evidence for the effectiveness of sepsis PMSs was mixed but suggests that a PMS can improve sepsis care processes and outcomes in some environments. Many hospitals are currently implementing or considering implementing PMSs for sepsis, primarily because of its advantages over manual screening for sepsis. An automated surveillance system is less time-consuming for staff than manual screening, and alerts clinicians in near real time to a patient’s deteriorating condition more quickly than most manual screening strategies. However, some units across the hospital might benefit more from a sepsis PMS than others. Patient monitoring systems seem to be less effective in ICUs than in the ED or general units, perhaps because of the existing vigilance for sepsis in ICUs.
Several other themes emerged from the studies. First, as with manual screening tools, a PMS will only be effective if the system has a high level of sensitivity and specificity, to engender clinician trust and reduce false-positive alerts. However, the nonspecific nature of sepsis makes achieving a highly predictive system difficult, whether on paper or in an automated PMS. To overcome this, some prospective studies iteratively revised thresholds for key values, with input from the clinicians, to optimize tool performance.20,27 Some more recent studies used machine learning to optimize system performance.14,27 This is an approach that may improve the performance of PMS in the future. Although not discussed by the studies, another possible strategy to combat the nonspecific nature of sepsis is engaging patients and families in the sepsis monitoring process, such as providing them with a stoplight-style symptom checker or allowing them to trigger a sepsis alert. These strategies could improve early sepsis recognition by capturing some of the “softer” information that goes into an accurate diagnosis, as well as improve qualitative patient experience measures. Studies on other patient safety topic have shown positive effects from patient and family engagement practices, and future studies could investigate how these practices impact sepsis PMSs.28
Second, if the electronic monitoring and alerting system is difficult to use and/or is not regarded as helpful to clinicians, it can lead to confusion, to frustration, and possibly to worse patient care.29 For example, if the alert physicians receive contains too little information (or too much), or if the action required is not clear, physicians may find the system too difficult or burdensome to use.29 To improve system usability, input from clinicians was solicited in some studies, followed by adaptations. Examples of clinician-driven adaptations included allowing a nurse to “snooze” an alert for 6 hours if the patient is already under assessment for sepsis, or implementing a “traffic light” system on a dashboard to visually show clinicians which patients are in a warning zone (yellow) or need urgent attention (red).20,30
Finally, the cost of designing and implementing a de novo PMS can be prohibitive for smaller hospitals. Several PMS systems are now available as an add-on electronic health record or telemedicine module, but this may result in less customizable functionality. Also, after a system is implemented, refining the algorithm and updating it based on changing sepsis criteria and clinician needs requires close work with the facility’s IT department, which can be resource and time intensive.
Limitations
This review has several limitations, some relating to the study design and some relating to the studies found. First, the inclusion of only English language studies could have introduced a language bias. The exclusion of gray literature (conference proceedings, theses, government reports, etc.) may have also affected the review conclusions.
Many of the included studies had a moderate potential for bias, primarily because of observational designs. The studies also use a variety of criteria and thresholds to identify patients with sepsis, which makes cross-study comparison more problematic. In addition, it is difficult to attribute effects on outcome measures, or lack thereof, to a PMS intervention, because many patients who develop sepsis are older, have multiple comorbidities, and may have advance directives for end-of-life care, any of which also affects the outcomes of interest. Although some studies attempt to control for these factors, they nonetheless present a challenge for evaluating sepsis interventions. In addition, reasons for ICU transfer and ICU LOS are multifactorial and not necessarily correlated with sepsis or the PMS.31 Finally, the broad increase in the capture of sepsis cases in recent years, many of which are less severe cases, can cause sepsis mortality to seem lower over time. When claims or diagnosis codes are used as diagnostic criteria for sepsis in a study, this phenomenon can lead to falsely concluding that an improvement in mortality was caused by an intervention.18
CONCLUSIONS
A sepsis PMS aims to reduce the time to recognition of sepsis so that treatment can be initiated quickly, with associated improvement in important patient outcomes. Evidence for PMSs in the hospital setting showed some improvement in both process and outcome measures, especially in non-ICU units. Because of mixed results, more high-quality studies are needed to help to understand the effects of sepsis PMS on important process and outcome measures in different hospital units.
In addition, the emergence of machine learning technology has the potential to improve the accuracy, consistency, and customizability of PMS. Rather than rules-based patient monitoring with predetermined thresholds, machine learning can continually learn from sepsis and nonsepsis cases and be able to better and more quickly predict when a patient is at risk of sepsis.11 More studies testing the effect of these systems on processes and outcomes are needed.
Footnotes
This work was funded by the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract numbers HHSP233201500013I and HHSP23337002T).
The authors disclose no conflict of interest.
REFERENCES
- 1.Singer M Deutschman CS Seymour CW, et al. . The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801–810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Pfuntner A, Wier LM, Stocks C. Most Frequent Conditions in U.S. Hospitals, 2010: Statistical Brief #148. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville, MD: Agency for Healthcare Research and Quality; 2006. [PubMed] [Google Scholar]
- 3.Rhee C Dantes R Epstein L, et al. . Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA. 2017;318:1241–1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hatfield KM Dantes RB Baggs J, et al. . Assessing variability in hospital-level mortality among U.S. Medicare beneficiaries with hospitalizations for severe sepsis and septic shock. Crit Care Med. 2018;46:1753–1760. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Torio CM, Moore BJ. National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2013: Statistical Brief #204. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville, MD: Agency for Healthcare Research and Quality; 2006. [PubMed] [Google Scholar]
- 6.Dantes RB, Epstein L. Combatting sepsis: a public health perspective. Clin Infect Dis. 2018;67:1300–1302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Seymour CW Kahn JM Martin-Gill C, et al. . Delays from first medical contact to antibiotic administration for sepsis. Crit Care Med. 2017;45:759–765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Liberati A Altman DG Tetzlaff J, et al. . The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Austrian JS Jamin CT Doty GR, et al. . Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay. JAMIA. 2018;25:523–529. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Berger T Birnbaum A Bijur P, et al. . A computerized alert screening for severe sepsis in emergency department patients increases lactate testing but does not improve inpatient mortality. Appl Clin Inform. 2010;1:394–407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Narayanan N Gross AK Pintens M, et al. . Effect of an electronic medical record alert for severe sepsis among ED patients. Am J Emerg Med. 2016;34:185–188. [DOI] [PubMed] [Google Scholar]
- 12.Hooper MH Weavind L Wheeler AP, et al. . Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit*. Crit Care Med. 2012;40:2096–2101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jung AD Baker J Droege CA, et al. . Sooner is better: use of a real-time automated bedside dashboard improves sepsis care. J Surg Res. 2018;231:373–379. [DOI] [PubMed] [Google Scholar]
- 14.Shimabukuro DW Barton CW Feldman MD, et al. . Effect of a machine learning–based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4:e000234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Croft CA Moore FA Efron PA, et al. . Computer versus paper system for recognition and management of sepsis in surgical intensive care. J Trauma Acute Care Surg. 2014;76:311–317. [DOI] [PubMed] [Google Scholar]
- 16.Moorman JR Delos JB Flower AA, et al. . Cardiovascular oscillations at the bedside: early diagnosis of neonatal sepsis using heart rate characteristics monitoring. Physiol Meas. 2011;32:1821–1832. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Umscheid CA Betesh J VanZandbergen C, et al. . Development, implementation, and impact of an automated early warning and response system for sepsis. J Hosp Med. 2015;10:26–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Manaktala S, Claypool SR. Evaluating the impact of a computerized surveillance algorithm and decision support system on sepsis mortality. J Am Med Inform Assoc. 2017;24:88–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.McRee L Thanavaro JL Moore K, et al. . The impact of an electronic medical record surveillance program on outcomes for patients with sepsis. Heart Lung. 2014;43:546–549. [DOI] [PubMed] [Google Scholar]
- 20.McCoy A, Das R. Reducing patient mortality, length of stay and readmissions through machine learning–based sepsis prediction in the emergency department, intensive care unit and hospital floor units. BMJ Open Qual. 2017;6:e000158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Alberto L Marshall AP Walker R, et al. . Screening for sepsis in general hospitalized patients: a systematic review. J of Hosp Infect. 2017;96:305–315. [DOI] [PubMed] [Google Scholar]
- 22.Nelson JL Smith BL Jared JD, et al. . Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Ann Emer Med. 2011;57:500–504. [DOI] [PubMed] [Google Scholar]
- 23.Sawyer AM Deal EN Labelle AJ, et al. . Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011;39:469–473. [DOI] [PubMed] [Google Scholar]
- 24.Warttig S Alderson P Evans DJ, et al. . Automated monitoring compared to standard care for the early detection of sepsis in critically ill patients. Cochrane Database Syst Rev. 2018;6:CD012404. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Despins LA. Automated detection of sepsis using electronic medical record data: a systematic review. J Healthc Qual. 2017;39:322–333. [DOI] [PubMed] [Google Scholar]
- 26.Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: a systematic review. J Hosp Med. 2015;10:396–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Amland RC, Hahn-Cover KE. Clinical decision support for early recognition of sepsis. Am J Med Qual. 2016;31:103–110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Park M, Giap TTT. Patient and family engagement as a potential approach for improving patient safety: a systematic review. J Adv Nurs. 2020;76:62–80. [DOI] [PubMed] [Google Scholar]
- 29.Pertiwi AAP Fraczkowski D Stogis SL, et al. . Using heuristic evaluation to improve sepsis alert usability. Crit Care Nurs Clin North Am. 2018;30:297–309. [DOI] [PubMed] [Google Scholar]
- 30.Cruz AT Williams EA Graf JM, et al. . Test characteristics of an automated age- and temperature-adjusted tachycardia alert in pediatric septic shock. Pediatr Emerg Care. 2012;28:889–894. [DOI] [PubMed] [Google Scholar]
- 31.Hunter CL Silvestri S Stone A, et al. . Prehospital sepsis alert notification decreases time to initiation of CMS sepsis core measures. Am J Emerg Med. 2019;37:114–117. [DOI] [PubMed] [Google Scholar]