Moving Beyond Single Parameter Early Warning Scores For Rapid Response System Activation

Early warning systems provide an assessment of a patient’s likelihood of developing critical illness and thus requiring additional critical care resources. The groundwork for these systems was laid millennia ago, with the Hippocratic “Book of Prognostics.” The statement attributed to Hippocrates that “it is bad if he has dyspnoea, and urine that is thin and acrid, and if sweats come out about the neck and head” includes clinical variables (respiratory rate and urine output) still used in early warning systems today (1). These systems now form the foundation for activating Rapid Response and Medical Emergency Teams.

Traditionally, early warning systems have come in two primary configurations: single parameter criteria and aggregated weighted scores (2). The former originated in Australia over two decades ago as a set of equally weighted abnormal physiologic thresholds (e.g., respiratory rate >36), the presence of any of which would trigger the system (3). In contrast, aggregated weighted scoring systems, such as the Modified Early Warning Score (MEWS), which arose in the United Kingdom around the same time, involve summing up points from multiple parameters based on the degree of derangement (e.g., 2 points for a respiratory rate of 21–29 and 3 points for ≥30) (4, 5).

In the current issue of the Journal, Professor Smith and colleagues provide important evidence regarding the comparative accuracy of the National Early Warning Score (NEWS), an aggregated weighted score similar to the MEWS, which was developed by the Royal College of Physicians as a uniform method of identifying clinical deterioration in patients across the National Health Service (NHS) in the United Kingdom (6). Using data from an NHS District General Hospital, the authors compared NEWS to 44 distinct single parameter tools and found it to be superior for predicting death, cardiac arrest, and/or unanticipated intensive care unit transfer. This study is limited by the fact that this is a single center study using a population arising from the same hospital in which the VitalPAC™ Early Warning Score (ViEWS), its immediate precursor, was originally derived (7). However, these concerns are largely mitigated by the fact that there is no overlap between the ViEWS derivation cohort and the current study population, and that the findings are consistent with independent studies demonstrating the superiority of aggregated weighted scoring systems over single parameter criteria (2, 8).

From a statistical modeling perspective, the finding that an aggregate weighted scoring system is more accurate than single parameter criteria is not surprising. Single parameter tools are generally based on single cut-points of continuous variables, which result in the loss of valuable information. For example, respiratory rates of 18 and 30 count similarly if they are both below the activation threshold. Furthermore, these criteria will miss subtle abnormalities in multiple vital signs, which have been shown to be more important for predicting outcomes than more dramatic elevations in a single vital sign (9). Aggregate weighted scores, which include several gradations of derangement and allow high scores to occur from both individual and combinations of vital sign abnormalities, do not suffer from these limitations. The NEWS has the added benefit of being informed by the dataset used to derive the ViEWS, rather than having been developed solely on the basis of expert opinion, upon which the vast majority of single parameter and many commonly used aggregated weighted scores were, including the MEWS. This is evident in the heavier weighting of subtler respiratory rate derangements, for example, which has been shown to be the vital sign with the strongest correlation to clinical deterioration (10, 11). In fact, the use of patient data in its development is the likely rationale for the superiority of NEWS to MEWS in prior head-on comparisons.

However, the improvements in accuracy need not stop there. Additional variables like laboratory data can be added and the full range of values can be utilized with logistic regression models and other similar models (12, 13). The use of vital sign trends can also increase accuracy, although accounting for these is more complicated than initially thought (14). Furthermore, the advent of machine learning tools, such as random forests, enable even more accurate models for predicting clinical deterioration (11).

If one believes that accuracy matters, and any hospital that has ever struggled with false alarms or missed opportunities would be hard pressed to argue that it doesn’t, each hospital system owes it to its providers and patients to implement the most accurate activation tool it can. For those hospitals still using paper charts, that should be one of the aggregated weighted scores, of which the NEWS appears to be one of the stronger contenders. However, for those hospitals that have transitioned to the computer age, it’s time to start thinking beyond paper based screening tools and make our expensive computers and electronic health records (EHRs) do the work they were designed to do. Retrofitting them with less accurate paper-based tools makes little sense.

Although results like the paper by Smith and colleagues suggest that this could and should be the beginning of the end for single parameter tools, it is becoming clear that sometime in the future we will be saying the same thing about simple aggregated weighted scores, like the NEWS, at least in their current form. EHRs are already ubiquitous in the United States, and are becoming more common in Europe, Australia, and other parts of the world as well. The EHR can harness the promise of “big data,” with countless variables and high power computing to automatically calculate complex and accurate algorithms in real-time. The future will belong to comprehensive and complex scores that are more accurate than NEWS, examples of which are already up and running in several hospitals today (13, 15). For hospitals that have already fully transitioned to using EHRs, it’s time to make this future a reality. At a minimum, it’s time to retire the single-parameter activation criteria once and for all.