Abstract
Background.
Severity scoring systems are commonly used in critical care and, when applied to the populations for whom they were developed and validated, these tools can inform mortality prediction and risk stratification, resource utilization, and optimization of patient outcomes.
Methods.
Original articles published in the English language were identified through MEDLINE literature searches conducted for the years 1980 to 2020. A list of terms associated with critical care scoring systems were used alone or in combination for the literature search.
Results.
This article appraises the characteristics and applications of the scoring systems most frequently applied to critically ill patients: those that predict risk of in-hospital mortality at time of ICU admission (APACHE, SAPS, and MPM), and those that assess and characterize current degree of organ dysfunction (MODS, SOFA, and LODS). Variable type and collection timing, score calculation, patient population, and comparative performance data of these systems are detailed.
Conclusion:
Awareness of the strengths, limitations, and specific characteristics of severity scoring systems commonly used in ICU patients is vital for critical care nurses to effectively employ these tools in clinical practice and to critically appraise research findings based on their usage.
Keywords: intensive care, scoring systems, risk assessment, hospital mortality
Background
Critically ill patients, families, and providers are concerned with recovery likelihood, but prognostication in the intensive care unit (ICU) can be difficult. Scoring systems are standard tools used in critical care research as study inclusion criteria and to demonstrate equivalence of patient groups. Clinically, they are used to objectively quantify condition severity, risk stratify patients for clinical prognostication and, at the unit level, they can serve as a tool for assessing the impact of quality variables (staffing, organization, management, protocol changes) on patient outcomes.1-3
Scoring systems commonly applied to critically ill patients can be broadly classified into disease specific scoring systems (such as the CHA2DS2-VASc score to predict thromboembolic risk in atrial fibrillation4), injury scoring systems (such as the Glasgow Coma Scale [GCS]) and severity scoring systems (Table 1). The latter category can be further subdivided into: 1) severity scoring systems which predict risk of in-hospital mortality based on degree of physiologic derangement at time of ICU admission (such as the Acute Physiology and Chronic Health Evaluation [APACHE] system, the Simplified Acute Physiology Score [SAPS], and the Mortality Probability Models [MPM]), and 2) severity scoring systems developed to assess and characterize current degree of organ dysfunction (such as the Multiple Organ Dysfunction Score [MODS], the Sequential Organ Failure Assessment [SOFA], and the Logistic Organ Dysfunction Score [LODS]).
Table 1.
Categories of clinical scoring systems commonly used in the Intensive Care Unit
| Category | Intent | Examples | Outcome Measure | |
|---|---|---|---|---|
| Disease Specific | Determine risk of a single specific disease | CHADS2-DS2-VASc Score for stroke risk in patients with atrial fibrillation | Risk and/or severity of a specific disease | |
| Injury Scoring Systems | Characterizes degree of injury severity | Glasgow Coma Scale (GCS) | Severity of acute brain injury | |
| Illness Severity Scoring Systems | Outcome Risk Prediction | Provides an indication of risk of in-hospital death of groups of ICU patients based on the degree of physiologic derangement on ICU admission | Acute Physiology and Chronic Health Evaluation (APACHE) | Risk of in-hospital death |
| Simplified Acute Physiology Score (SAPS) | ||||
| Mortality Probability Model (MPM) | ||||
| Organ Dysfunction | Define degree of organ failure. Can be used as sequential scores to assess effect of new therapies | Sequential organ failure assessment (SOFA) | Degree of organ dysfunction | |
| Multiple organ dysfunction score (MODS) | ||||
| Logistic organ dysfunction score (LODS) | ||||
Outside of the ICU, there are a variety of early warning system (EWS) scores (e.g., the Modified Early Warning Score [MEWS], the Cardiac Arrest Risk Triage [CART], the quick Sepsis-related Organ Failure Assessment [qSOFA]) which have been developed to support earlier identification of clinical deterioration and need for initiation of a rapid response team visit to bring critical care expertise to patients in hospital wards and step-down units.5-7 Because this article focuses on ICU patient illness severity scoring systems, EWS scores are not included in this discussion.
Purpose.
ICU severity scoring systems can be useful tools to objectively quantify patient condition severity and help inform clinical care.8-12 When applied to the populations for whom they were developed and validated, severity scoring systems can inform mortality prediction, risk stratification, resource utilization and optimization of patient outcomes.13,14 Inaccurate application, however, can produce erroneous scores, leading to wasted time and resources, and may contribute to patient mismanagement. By equipping critical care nurses with the information needed to accurately and effectively utilize severity scoring systems to risk stratify high-acuity patients, to assess ICU care quality, and to critically appraise research findings based on their usage, this article fills a critical knowledge gap. We first describe the methodology of how risk scoring systems are developed and evaluated, and then discuss the characteristics, applications, strengths, and limitations of the two categories of severity scoring systems most frequently applied to ICU patients: outcome risk prediction and organ dysfunction.
Methods
Original articles with abstract and full text availability published in the English language were identified through a MEDLINE literature search conducted for the years 1980 to 2020. The following terms were used alone or in combination for the literature search: adult, severity, scoring systems, prognostic, critically ill, intensive care, outcome prediction, risk prediction, APACHE, SAPS, SOFA, and MPM.
Results
Development and Evaluation of Risk Prediction Scoring Systems.
Outcome risk prediction scoring systems are developed through analysis of large repositories of patient data, and are generally comprised of two parts: a score and a probability model. The score, a number based on the sum and weights of specific variables, quantifies the patient’s illness severity (morbidity); a higher number usually indicates a more severe condition. The probability model is a calculated equation or algorithm that provides the probability of in-hospital death (mortality) based on the variable score. The model refines the ability of illness severity scores to be used in comparing various groups of patients.15 Accordingly, these systems are not designed to provide for individual patient prognostication; but rather to allow the calculated scores to be used to compare groups of patients based on their predicted risk of in-hospital mortality. According to the literature, severity scoring systems that predict outcome risk should include the following characteristics13:
They should be based on routinely recorded variables, enabling ability to efficiently calculate scores from readily available information for use in probability modeling.
The probability model should be well calibrated, meaning observed mortality numbers are close to the numbers predicted by the model. Changes over time in the prevalence of major diseases and improvements in diagnostics and treatments are associated with outcome prediction scoring models overestimating mortality risk (calibration deterioration).16,17 Thus, periodic review and updating of established scoring systems (i.e., APACHE I, II, III, IV) is important.18
The probability model should have a high level of discrimination, meaning the model should be able to accurately differentiate patients who will survive (sensitivity) from patients who will die (specificity).
The probability model should be widely applicable to all patient populations, therefore, the data used in model development should include data from a wide variety (case-mix) of patients from multiple ICU types, hospitals, and countries.
The outcome risk prediction scoring systems most commonly used in ICU patients are APACHE, SAPS, and MPM. All were developed based on the outcome measurement of risk of in-hospital mortality and specifically excluded burn and cardiac surgery patients from the case-mix of patients used for model development as the physiologic and clinical variables predictive of mortality in those populations are considerably different than those in a general ICU population.
The first generations of these systems were developed and incorporated by intensivists in the 1980’s, and over the past three decades, through the use of larger data sets and more sophisticated statistical analyses, updated versions of these three scoring systems have been developed, validated and tested. These systems differ with regard to variables included, timing of score measurement, intent of use, and population used in model development (Table 2).
Table 2.
Comparison of outcome prediction scoring systems
| Scoring system | APACHE IV (2006) | SAPS 3 (2005) | MPM0-III (2007) |
|---|---|---|---|
| Assessment Timing | Within 24 hours of ICU admission | Within first hour of ICU admission | Within first hour of ICU admission, followed by 24 h, 48 h and 72 h assessment scores |
| Number of Variables | 27 | 17 | 16 |
| Sample size | 110,558 | 16,784 | 124,855 |
| Countries | 1 (US) | 35 | 5 (US, Canada, Brazil, Australia, Puerto Rico) |
| Number of Hospitals/ICUs | 45/104 | 303 | 96/135 |
| Advantages | Performs well in CABG patients; Algorithm for LOS prediction; Model coefficients regularly updated; Less prone to case-mix limitations | Low burden of abstraction; Development sample comprised of 35 countries; Customized equations for seven geographic regions; Less prone to inter-observer variability | Lowest burden of abstraction; Less prone to inter-observer variability |
| Limitations | Developmental sample limited to US population; Requires complex data collection; High abstraction burden | No estimation of LOS | More susceptible to case-mix limitations |
APACHE, acute physiology and chronic health evaluation; CABG, coronary artery bypass graft; LOS, length of stay; MPM mortality prediction model; SAPS simplified acute physiology score
Acute Physiology and Chronic Health Evaluation System (APACHE) I-IV.
Developed in 1981, from a sample of 582 ICU patients in the United States (US), APACHE was constructed on the hypothesis that illness severity can be measured by quantifying the degree of abnormality of different physiologic variables.19 Employing variables subjectively selected by an expert panel, APACHE used a combination of pre-admission health indicators and 34 weighted physiologic variables, gathered within the first 24 hours of ICU admission, to develop a score shown to correlate with subsequent risk of in-hospital death.19 Although risk was based on correlational analyses from a relatively small sample of patients, APACHE’s usefulness in analyzing ICU utilization and patient outcomes was immediately apparent.
APACHE II, developed four years later from a larger sample (n=5815), reduced the number of subjectively selected variables from 34 to 12 and required specification of a principal ICU admission diagnosis.20 Using a point score (0-71) based on the variables’ most deranged values collected within the first 24 hours of ICU, the APACHE II score and ICU diagnosis are used in an equation to predict hospital mortality risk.21 APACHE II has shown good calibration and discrimination across a range of disease and diagnostic groups, is easy for clinicians to use, and despite the development of newer APACHE models, it remains the most commonly used severity scoring system worldwide,14 likely due to variable parsimony and simplicity.
APACHE III, designed to refine APACHE II and released in 1991, was developed using a significantly larger data repository from 40 US hospitals and utilized multivariate logistic regression (MLR) and advanced statistical modeling techniques, rather than an expert panel, to determine selection and weighting of over 100 variables.22,23 Treatment location immediately prior to ICU admission, and major disease categories were added, providing APACHE III the ability to calculate in-hospital mortality risk based on specific disease groups.24,25 However, the time burden of manually extracting complex patient information and the trademarked, proprietary ownership of the scoring system’s predictive algorithm, have contributed to its limited acceptance.13
Released in 2006, APACHE IV was developed from a more diverse case-mix of patients than prior versions and added mechanical ventilation, thrombolysis, GCS sedation impact, and arterial oxygen tension and fractional concentration of inspired oxygen ratio (PaFiO2).25 Notably, APACHE IV also provides prediction equations to estimate ICU length of stay (LOS), which can serve as a proxy for ICU resource utilization.26 APACHE IV’s algorithm is not proprietary, is available in the public domain, and model coefficients are routinely updated.
Simplified Acute Physiology Score (SAPS) I, II and 3.
SAPS was developed in 1984, from a sample of 679 patients from eight different ICUs in France, as an alternative to APACHE.27 The original SAPS score calculation included the assessment of abnormal deviation of 14 physiological variables (determined by subjective expert clinical judgment) routinely collected during the first 24 hours of ICU admission. The correlation of severity score value to risk of in-hospital death was almost identical for both SAPS and APACHE, however, SAPS was lauded as being quicker and less expensive to compute.
Updated in 1993, using a larger sample (n=13,152) from 12 different countries, SAPS II variable selection and weighting was derived through statistical modeling and included 12 physiologic variables, age, type of admission (medical, surgical scheduled or surgical unscheduled) and three underlying disease variables (AIDS, hematologic cancer, and metastatic cancer). MLR was used to transform the severity score into a probability model predicting risk of in-hospital death. Advantages of SAPS II included ease of use and public availability.
SAPS 3 was developed in 2005, from a worldwide database comprising 35 countries. MLR was used to guide variable selection and weighting, resulting in 20 variables from three different categories: 1) patient characteristics before ICU admission (i.e., age, co-morbidities), 2) circumstances surrounding the ICU admission (i.e., readmission), and 3) degree of physiologic derangement (i.e., vital signs, GCS).16 SAPS 3 score data is collected during the first 60 minutes of ICU admission. Advantages of the SAPS 3 probability model include public availability and equations to customize mortality prediction based on seven global geographic regions, rendering it useful internationally.3
Mortality Probability Models (MPM) I-III.
First described in 1985, using data obtained from 755 ICU patients in a single hospital, Lemeshow et al.,28 employed multivariate techniques to identify seven physiologic variables that predicted in hospital mortality with a high degree of accuracy. MPM differs from APACHE and SAPS in that mortality probability is directly calculated from input data without producing an intermediate physiology score. Two models were derived: 1) MPM0-I, containing seven treatment-independent variables collected within the first hour of ICU admission (similar to SAPS 3), and 2) MPM24-I, containing seven variables collected 24-hours after admission, reflecting patient conditions and treatment interventions in the ICU.
MPM-II models were developed and validated on the same international database used to develop SAPS 3. MPM-II incorporates ongoing assessment by adding modeling points at 48 and 72 hours after ICU admission.29 Age and chronic health status, along with 15 other variables are contained in the admission model (MPM0-II). The 24-hr model (MPM24-II), for application to patients with an ICU stay > 24 hours, contains five admission variables plus an additional eight specific to the ICU stay.14 Variables not obtained are assumed to be normal, rather than missing, simplifying the scoring process for clinicians. MPM-II also includes a weighted hospital-days scale (WHD-94) that can be used to measure ICU resource utilization.30
MPM0-II was updated using Project IMPACT data: an international, diverse case-mix database developed by the Society of Critical Care Medicine to measure and describe the care of ICU patients.31 The resulting MPM0-III model uses 16 variables obtained within 60 minutes of ICU admission and includes two new variables strongly associated with a very low mortality risk: a zero-factor variable (absence of all variables except age) and full code status on ICU admission, i.e., a patient admitted post-operatively after a long duration of anesthesia.32
Discussion of Outcome Risk Prediction Scoring Systems
The APACHE II, APACHE IV, SAPS 3 and MPM0-III are the most commonly used outcome risk prediction scoring systems developed, tested, and validated to estimate probability of in-hospital mortality in critically ill patients recently admitted to the ICU and to stratify patients for inclusion in research studies. Although the outcome measurement for all models is the same (in-hospital death due to disease severity), the timing of score calculation varies: ICU admission but pre-intervention (MPM0-III), within the first hour of ICU admission (SAPS 3), within 24 hours of admission (APACHE II and IV), and 24 hours after ICU admission (MPM24-II). Application of these scoring systems outside of the validated time frames, or to non-ICU populations, is inaccurate and reliability can no longer be presumed.
As newer generations of these severity scoring systems demonstrate improvement over earlier versions, using the most current version is recommended. With the constant development of new science and knowledge to inform and change diagnostic processes, disease management, and patient care, it is expected that model performance will deteriorate over time and require updating and recalibration to reflect those advances. C
SAPS 3 and MPM0-III are unique in that they were developed using variables obtained within the first hour of admission to the ICU, intending to provide a mortality risk prediction prior to any ICU interventions. Retrospective review of these data could be valuable for ICU benchmarking and evaluating unit practices such as patient triage and nurse-to-patient ratios, based on mortality risk at time of admission. However, lead-time bias (discussed in limitations below) can impact this aim. MPM’s Weighted Hospital Days scale (WHD-94) and APACHE IV’s ICU LOS equation enable their use as audit tools to assess ICU care quality at the unit level.
Although APACHE IV, SAPS 3 and MPM0-III were developed from large databases with a varied case-mix, research demonstrates certain scoring systems are superior to others when applied to specific types of patients:
Post-cardiac arrest, coronary and cardiac surgery patients: SAPS 3 performs poorly in patients with acute coronary syndrome and post-cardiac arrest. Despite a trend to overestimate mortality, APACHE IV has had good model performance and calibration with this population.33-35
Burn patients: Burn surface area, thickness, and inhalation injury are mainstays of mortality prognostication in thermally injured patients and a variety of mortality risk scoring systems specific to this population have been developed and validated.36,37 The widely used FLAMES (Fatality by Longevity, APACHE II score, Measured Extent of Burn, and Sex) score uses the APACHE II score as a component of its mortality prediction score.38
Patients with cancer and solid-organ transplant: The SAPS 3 scoring system has been validated in several studies including patients with cancer, predicting mortality with an accuracy that far exceeded APACHE IV and MPM0-III.39 No one scoring system has shown superior performance in patients with solid-organ transplant.26
Patients requiring renal replacement therapy and extra-corporeal membrane oxygen (ECMO): All scoring systems performed poorly when estimating in-hospital mortality risk at ICU admission in these groups.26
Patients enrolled in clinical research studies: Clinical research studies have strict protocols regarding timing of data collection. If used in association with a study, choosing a system where timing of variable acquisition and score measurement aligns with the timing and operationalization of the study protocol is imperative.
In highly specialized ICUs, (i.e., burn units, liver transplant units, units with extracorporeal membrane oxygenation (ECMO) programs) population specific scoring systems are recommended in place of those discussed here. Examples include the FLAMES score for thermally injured patients, the model for end-stage liver disease (MELD) score for patients with chronic liver disease, 40 and the respiratory ECMO survival prediction (RESP) score for ECMO candidates.41 Finally, patient ethnicity and global location should be considered when choosing a severity scoring system. The APACHE II, IV and MPM0-III systems were developed on a largely North American patient population. SAPS 3 incorporated a global patient case-mix in its development and its scoring equations can be customized for seven geographic locations, making it a good choice for populations outside of North America.
Limitations to consider for outcome risk prediction scoring systems.
Post-ICU admission functional status and quality of life are ideal patient outcome metrics, however, at present, no such scoring system exists. Current models have been developed with the intent and capability to predict probability of in-hospital mortality alone. The fact that in-hospital mortality can be impacted by factors across the entire hospital system, pre- and post-ICU admission,42 is an important consideration when using these systems to assess ICU quality performance.
These scoring systems are designed to compare similar groups of critically ill patients with regard to risk stratification, comparative analysis, and treatment effectiveness, and they provide accurate estimates of the number of patients expected to die among a group of similar patients. They should not be used to make firm predictions for an individual or as a singular reason to deny escalation of care. Additionally, these systems are not based on a linear scale; a score of 70 does not indicate a patient is twice as sick or has twice the mortality risk as a patient with a score of 35. Nor can those values be compared with scores from other models developed on a different scale with different numbers of variables.
Lead-time bias, defined as variations in care prior to ICU admission, can impact severity score values,42,43 with the variables of heart rate (HR), blood pressure (BP), respiratory rate (RR), serum pH, oxygenation, and blood glucose accounting for the most bias.13,44 For example, a patient with severe sepsis admitted from an emergency department (ED) capable of initiating optimal early treatment (e.g., fluid resuscitation, antibiotic therapy, vasopressor support) will likely exhibit less vital sign derangement at ICU admission than a similar patient, emergently transferred from a general hospital unit, who has yet to be stabilized. A severity score suggestive of low in-hospital mortality risk does not definitively correlate with low patient acuity, nor does it negate the impact close monitoring and intensive nursing care can have on patient survival.32 In contrast, some conditions associated with a high degree of physiological derangement (e.g., diabetic ketoacidosis, post-operative patient admitted to the ICU before full anesthesia reversal) that are either self-limiting or quickly reversible with routine management, can generate exceedingly high severity scores which do not accurately predict actually in-hospital mortality risk. While these tools can provide valuable prognostic information, clinician judgment should supersede a severity score calculated at a single point in time.
Variable definitions, timing of data collection, and scoring rules when data is missing must align with the requirements of each scoring system to ensure accuracy. Quality and completeness of input data are limiting factors. Increasing availability of electronic health records and ICU clinical information systems can mitigate clinician burden of manual data abstraction and risk of scoring errors, but they do not address data completeness and accuracy issues.42 Finally, severity scale score models must be applied to the population and time setting for which they were developed and validated. Incorrect score calculations can result in wasted time, resources and misinform care decisions.
Organ Dysfunction Scoring Systems
Organ dysfunction scoring systems are designed to assess and characterize a patient’s current degree of organ dysfunction. While degree of dysfunction often correlates with mortality, these scoring systems are not developed or intended for outcome risk prediction. Rather, they are useful in quantifying disease severity, and can be used on a daily basis to assess disease progression and response to interventions. The organ dysfunction scoring systems most commonly used in adult ICU patients are outlined in Table 3 and discussed below.
Table 3.
Comparison of Organ Dysfunction Scoring Systems
| Scoring System Authors, Date |
Variables | Score and Timing | Clinical Pearls |
|---|---|---|---|
|
MULTIPLE ORGAN DYSFUNCTION SCORE (MODS)
Marshall, J.C., et al., 1995 |
-Glasgow Coma Score -PAR (HR x RAP/MAP) -Creatinine -PaO2 mmHg /FiO2 -Platelets -Bilirubin |
-First daily value used -Variables ranked on a 0-4 scale. -Score range 0-24 -Daily use |
-Relatively simple to use, however, cardiovascular variable requires calculation. -When used as a sequential scoring system, accurately characterizes disease/dysfunction progression over time in |
|
SEQUENTIAL ORGAN FAILURE ASSESSMENT (SOFA)
Vincent, J.L., et al., 1996 |
-Glasgow Coma Score -Hypotension -Creatinine -Urine output/day -PaO2 mmHg /FiO2 -Platelets -Bilirubin |
-Worst daily value used -Variables ranked on a 0-4 scale -Score range 0-24 -Daily use |
-Simplest to use -When used as a sequential scoring system, accurately characterizes disease/dysfunction progression over time in -Developed for use in patients with sepsis. Subsequently tested and validated in a variety of critically ill patient populations. |
|
LOGISTIC ORGAN DYSFUNCTION SCALE (LODS)
Le Gall, J.R., et al., 1996 |
-Glasgow Coma Score -Heart rate -Systolic Blood Pressure (SBP) -Blood urea nitrogen (BUN) -Creatinine -Urine output/day -PaO2 mmHg /FiO2 -White blood cell count -Platelets -Bilirubin -Prothrombin time |
-Worst daily value used -Variables ranked on a 0,1,3,5 scale -Score range 0-22 -First 24 hours* |
-Most complex -Reflects severity level within each organ system and the relative severity among organ systems -Includes an equation that can convert the score into a mortality probability |
PAR pressure adjusted heart rate, HR heart rate, RAP right arterial pressure, MAP mean arterial pressure, PaO2 partial pressure of oxygen, FiO2 fraction of inspired oxygen.
Developed for use in first 24 hours, but has subsequently been validated for daily use
Multiple Organ Dysfunction Score (MODS).
MODS was developed in 1995, to objectively quantify organ dysfunction and provide a reliable and meaningful index of syndrome severity in individual ICU patients. Variables included were determined by a literature review and then refined using MLR to produce a score that includes five basic physiologic variables and a composite variable for the cardiovascular system called “pressure-adjusted heart rate” (PAR; assumed to be normal in patients without a central line).45 The first measurement of the day for each variable is ranked on a five-point scale to produce a total score ranging from zero (normal) to 24 (maximum dysfunction). Outcome prediction is not a goal of MODS, but progressive organ dysfunction correlates with ICU and hospital mortality.45,46
Sequential Organ Failure Assessment (SOFA).
SOFA was developed to be a more simplified scoring tool to objectively define degree of organ failure and risk stratify groups of patients with sepsis. Six variables were chosen by consensus by the European Society of Intensive Care Medicine’s Working Group on Sepsis Related Problems, and then prospectively tested in a multi-center study.47 The worst daily values for each variable are scored on a five-point scale to produce a total score ranging from zero (normal) to 24 (maximum dysfunction). While similar to the MODS, SOFA score variables do not require calculation, making it a more efficient tool for bedside clinicians. Increasing SOFA scores are highly correlated with mortality when applied to patients with sepsis.14,47,48 SOFA has been validated in other patient populations and while an association between increasing SOFA scores and mortality has been suggested, the SOFA scoring system intent is not to predict, but to define the degree of present organ failure, assess the effect of new therapies, and describe a sequence of complications in critically ill patients.
Logistic Organ Dysfunction System (LODS).
LODS used MLR to objectively derive a set of 12 individually weighted variables that reflect severity level within an organ system as well as the relative severity among organ systems.49 The worst values within 24 hours of ICU admission are ranked on a scale that considers abnormally high and low values for each system. The LODS score ranges from zero (normal) to 22 (maximum dysfunction) and includes an equation that can convert the score into a mortality probability, making it a hybrid organ dysfunction and outcome risk prediction scoring system. Although originally developed for use at ICU admission, the LODS has been shown to be accurate in characterizing disease progression when used daily.50,51
Discussion of Organ Dysfunction Scores
The MODS, SOFA, and LODS scoring systems scores can be quickly calculated at the bedside using routinely collected patient information, and they provide clinicians with valuable information regarding patient morbidity, disease progression, and patient response to interventions. Additionally, they offer clinicians a snapshot of individual organ function. All three have been validated for daily use, however the timing of variable collection and methods for score calculation vary (Table 3). Many studies have demonstrated no significant difference in the strong descriptive performance of these three scoring systems.46,52-55 Nevertheless, the SOFA continues to be the most commonly used score due to its simplicity. Although the LODS was developed to also provide mortality risk estimates, its prognostic ability has not been found to be superior to the other systems.48,50
Implications for Nurses
Clinical Considerations.
Severity scoring systems are widely used in critical care settings and the two types of severity scoring systems, outcome risk prediction and organ dysfunction scoring systems, complement, rather than compete with each other. Outcome risk prediction scoring systems can provide an assessment of disease severity and risk stratification between groups of patients and can assist clinicians in objectively quantifying individual patient disease severity, disease progression, and response to therapy. These tools can help nurses identify individuals requiring higher levels of nursing care and inform nursing workload and staffing decisions, which can positively impact care quality, care costs, and patient outcomes.
In addition to understanding the application and limitations of each system, nurse clinicians must also consider the logistics of using each and determine the best fit for their institution and patient population. If a severity score requires manual calculation, how much time will be required of nurses or other designated personnel to collect variable data and calculate the score? Are the scoring system variables easily obtained in your institution? Some models are more reliant on laboratory values than others and some require weighting of past medical history information. A scoring system where many of the fields are left blank due to data that is either unobtainable or so burdensome to collect that fields are left with missing information will not produce accurate results. Since scoring system performance can vary by disease type, individual ICU case-mix is an important consideration; the use of a scoring system that performs well and has been appropriately validated in your population of interest is critical. Finally, costs associated with the acquisition of copyright-protected scoring systems need to be considered.
Quality Considerations.
Awareness of the strengths and limitations associated with ICU severity scoring system data provide critical care nurses engaged in evaluating ICU care processes, resource utilization, and ICU patient outcome data with the insight needed to implement robust data collection and quality control protocols that drive quality improvement and reduce health care costs.
Research Considerations.
Severity scoring system data are commonly used in clinical research and quality improvement studies and the information provided in this review provides critical care nurses with the knowledge needed to employ these tools in clinical research studies, critically evaluate study results and to inform recommendations for the translation of evidence into practice.
Conclusion
In summary, clinical severity scoring systems are commonly used in critical care and when applied to the populations for whom they were developed and validated, these tools can inform mortality prediction and risk stratification, resource utilization, and optimization of patient outcomes. Critical care nurses, as valued members of the multidisciplinary care team, are exposed to the information provided by severity scoring systems in their practice and in their evaluation of research. This article equips nurses with the knowledge needed to understand and effectively utilize severity scoring systems in critically ill patient populations.
Acknowledgments
Funding: Tiffany Purcell Pellathy Jonas Nurse Scholar Program Scholarship and NIH F31NR018102. Michael R. Pinsky and Marilyn Hravnak NIH, R01NR013912
Footnotes
Tiffany Pellathy is an acute care nurse practitioner and pre-doctoral fellow at the University of Pittsburgh School of Nursing.
Funding: Jonas Nurse Scholar Program Scholarship and NIH F31NR018102
Dr. Pinsky is a Professor of Critical Care Medicine (primary), Bioengineering, Cardiovascular Diseases, Clinical & Translational Science, and Anesthesiology at the University of Pittsburgh.
Funding: NIH, R01NR013912
Dr. Hravnak is a Professor of Nursing and the Director of the Nursing PhD Program at the University of Pittsburgh School of Nursing.
Funding: NIH, R01NR013912
Contributor Information
Tiffany Purcell Pellathy, University of Pittsburgh School of Nursing, 3500 Victoria Street, Suite 336, Pittsburgh, PA, 15213.
Michael R. Pinsky, University of Pittsburgh School of Medicine, 638 Scaife Hall, 3550 Terrace Street, Pittsburgh, PA 15261.
Marilyn Hravnak, University of Pittsburgh School of Nursing, 3500 Victoria St., 336 Victoria Hall, Pittsburgh, PA 15261.
References
- 1.Miranda DR, Ryan DW, Schaufeli W, Fidler V Organisation and management of intensive care: a prospective study in 12 European countries. Vol 29: Springer Science & Business Media; 2012. [Google Scholar]
- 2.Moreno R, Miranda D, Matos R, Fevereiro T Mortality after discharge from intensive care: the impact of organ system failure and nursing workload use at discharge. Intensive care medicine. 2001;27(6):999–1004. [DOI] [PubMed] [Google Scholar]
- 3.Metnitz PG, Moreno RP, Almeida E, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med. 2005;31(10):1336–1344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Gage BF, van Walraven C, Pearce L, et al. Selecting patients with atrial fibrillation for anticoagulation: stroke risk stratification in patients taking aspirin. Circulation. 2004;110(16):2287–2292. [DOI] [PubMed] [Google Scholar]
- 5.Singer M, Shankar-Hari M qSOFA, cue confusion. Annals of internal medicine. 2018;168(4):293–295. [DOI] [PubMed] [Google Scholar]
- 6.Smith ME, Chiovaro JC, O'Neil M, et al. Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Annals of the American Thoracic Society. 2014;11(9):1454–1465. [DOI] [PubMed] [Google Scholar]
- 7.McGaughey J, Alderdice F, Fowler R, Kapila A, Mayhew A, Moutray M. Outreach and Early Warning Systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database of Systematic Reviews. 2007(3). [DOI] [PubMed] [Google Scholar]
- 8.Rafferty AM, Clarke SP, Coles J, et al. Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional analysis of survey data and discharge records. International journal of nursing studies. 2007;44(2):175–182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kelly L, Vincent D. The dimensions of nursing surveillance: a concept analysis. Journal of advanced nursing. 2011;67(3):652–661. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. Jama. 2003;290(12):1617–1623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Clarke SP, Aiken LH. Failure to Rescue: Needless deaths are prime examples of the need for more nurses at the bedside. AJN The American Journal of Nursing. 2003;103(1):42–47. [DOI] [PubMed] [Google Scholar]
- 12.Stanton MW, Stanton MW. Hospital nurse staffing and quality of care. Agency for Healthcare Research and Quality; Rockville, MD; 2004. [Google Scholar]
- 13.Bouch DC TJ. Severity Scoring Systems in the Critically Ill. Continuing Education in Anaesthesia, Critical Care & Pain. 2008;8(5):181–185. [Google Scholar]
- 14.Vincent J-L, Ferreira F, Moreno R. Scoring systems for assessing organ dysfunction and survival. Critical care clinics. 2000;16(2):353–366. [DOI] [PubMed] [Google Scholar]
- 15.Rapsang AG, Shyam DC. Scoring systems in the intensive care unit: A compendium. Indian J Crit Care Med. 2014;18(4):220–228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Moreno RP, Metnitz PG, Almeida E, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–1355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Alba AC, Agoritsas T, Walsh M, et al. Discrimination and Calibration of Clinical Prediction Models: Users' Guides to the Medical Literature. Jama. 2017;318(14):1377–1384. [DOI] [PubMed] [Google Scholar]
- 18.Haniffa R, Isaam I, De Silva AP, Dondorp AM, De Keizer NF. Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review. Critical Care. 2018;22(1):18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med. 1981;9(8):591–597. [DOI] [PubMed] [Google Scholar]
- 20.Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. [PubMed] [Google Scholar]
- 21.Vincent J-L, Moreno R. Clinical review: scoring systems in the critically ill. Critical care. 2010;14(2):207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957–2963. [DOI] [PubMed] [Google Scholar]
- 23.Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619–1636. [DOI] [PubMed] [Google Scholar]
- 24.Becker RB, Zimmerman JE, Knaus WA, et al. The use of APACHE III to evaluate ICU length of stay, resource use, and mortality after coronary artery by-pass surgery. J Cardiovasc Surg (Torino). 1995;36(1):1–11. [PubMed] [Google Scholar]
- 25.Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34(5):1297–1310. [DOI] [PubMed] [Google Scholar]
- 26.Salluh JI, Soares M. ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care. 2014;20(5):557–565. [DOI] [PubMed] [Google Scholar]
- 27.Le Gall JR, Loirat P, Alperovitch A, et al. A simplified acute physiology score for ICU patients. Crit Care Med. 1984;12(11):975–977. [DOI] [PubMed] [Google Scholar]
- 28.Lemeshow S, Teres D, Pastides H, Avrunin JS, Steingrub JS. A method for predicting survival and mortality of ICU patients using objectively derived weights. Critical care medicine. 1985;13(7):519–525. [DOI] [PubMed] [Google Scholar]
- 29.Higgins TL, Teres D, Nathanson B. Outcome prediction in critical care: the Mortality Probability Models. Current opinion in critical care. 2008;14(5):498–505. [DOI] [PubMed] [Google Scholar]
- 30.Rapoport J, Teres D, Lemeshow S, Gehlbach S. A method for assessing the clinical performance and cost-effectiveness of intensive care units: a multicenter inception cohort study. Critical care medicine. 1994;22(9):1385–1391. [DOI] [PubMed] [Google Scholar]
- 31.Cook SF, Visscher WA, Hobbs CL, Williams RL. Project IMPACT: results from a pilot validity study of a new observational database. Critical care medicine. 2002;30(12):2765–2770. [DOI] [PubMed] [Google Scholar]
- 32.Higgins TL, Kramer AA, Nathanson BH, Copes W, Stark M, Teres D. Prospective validation of the intensive care unit admission Mortality Probability Model (MPM0-III). Crit Care Med. 2009;37(5):1619–1623. [DOI] [PubMed] [Google Scholar]
- 33.Donnino MW, Salciccioli JD, Dejam A, et al. APACHE II scoring to predict outcome in post-cardiac arrest. Resuscitation. 2013;84(5):651–656. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Exarchopoulos T, Charitidou E, Dedeilias P, Charitos C, Routsi C. Scoring Systems for Outcome Prediction in a Cardiac Surgical Intensive Care Unit: A Comparative Study. Am J Crit Care. 2015;24(4):327–334; quiz 335. [DOI] [PubMed] [Google Scholar]
- 35.Salciccioli JD, Cristia C, Chase M, et al. Performance of SAPS II and SAPS III scores in post-cardiac arrest. Minerva Anestesiol. 2012;78(12):1341–1347. [PMC free article] [PubMed] [Google Scholar]
- 36.Sheppard N, Hemington-Gorse S, Shelley O, Philp B, Dziewulski P. Prognostic scoring systems in burns: a review. Burns. 2011;37(8):1288–1295. [DOI] [PubMed] [Google Scholar]
- 37.Halgas B, Bay C, Foster K. A comparison of injury scoring systems in predicting burn mortality. Annals of burns and fire disasters. 2018;31(2):89. [PMC free article] [PubMed] [Google Scholar]
- 38.Gomez M, Wong DT, Stewart TE, Redelmeier DA, Fish JS. The FLAMES score accurately predicts mortality risk in burn patients. Journal of Trauma and Acute Care Surgery. 2008;65(3):636–645. [DOI] [PubMed] [Google Scholar]
- 39.Soares M, Salluh JI. Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med. 2006;32(11):1839–1844. [DOI] [PubMed] [Google Scholar]
- 40.Kamath PS, Kim WR. The model for end-stage liver disease (MELD). Hepatology. 2007;45(3):797–805. [DOI] [PubMed] [Google Scholar]
- 41.Brunet J, Valette X, Buklas D, et al. Predicting survival after extracorporeal membrane oxygenation for ARDS: an external validation of RESP and PRESERVE scores. Respiratory care. 2017;62(7):912–919. [DOI] [PubMed] [Google Scholar]
- 42.Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1—interpretation and accuracy of outcome prediction scoring systems. Chest. 2012;141(1):245–252. [DOI] [PubMed] [Google Scholar]
- 43.Breslow MJ, Badawi O. Severity Scoring in the Critically III: Part 2: Maximizing Value From Outcome Prediction Scoring Systems. Chest. 2012;141(2):518–527. [DOI] [PubMed] [Google Scholar]
- 44.Tunnell R, Millar B, Smith G. The effect of lead time bias on severity of illness scoring, mortality prediction and standardised mortality ratio in intensive care—a pilot study. Anaesthesia. 1998;53(11):1045–1053. [DOI] [PubMed] [Google Scholar]
- 45.Marshall JC, Cook DJ, Christou NV, Bernard GR, Sprung CL, Sibbald WJ. Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Critical care medicine. 1995;23(10):1638–1652. [DOI] [PubMed] [Google Scholar]
- 46.Ferreira AMP, Sakr Y. Organ dysfunction: general approach, epidemiology, and organ failure scores. Paper presented at: Seminars in respiratory and critical care medicine. 2011. [DOI] [PubMed] [Google Scholar]
- 47.Vincent J-L, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. In: Springer-Verlag; 1996. [DOI] [PubMed] [Google Scholar]
- 48.Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. Jama. 2001;286(14):1754–1758. [DOI] [PubMed] [Google Scholar]
- 49.Le Gall J-R, Klar J, Lemeshow S, et al. The Logistic Organ Dysfunction system: a new way to assess organ dysfunction in the intensive care unit. Jama. 1996;276(10):802–810. [DOI] [PubMed] [Google Scholar]
- 50.Timsit J-F, Fosse J-P, Troché G, et al. Calibration and discrimination by daily Logistic Organ Dysfunction scoring comparatively with daily Sequential Organ Failure Assessment scoring for predicting hospital mortality in critically ill patients. Critical care medicine. 2002;30(9):2003–2013. [DOI] [PubMed] [Google Scholar]
- 51.Pettilä V, Pettilä M, Sarna S, Voutilainen P, Takkunen O. Comparison of multiple organ dysfunction scores in the prediction of hospital mortality in the critically ill. Critical Care Medicine. 2002;30(8):1705–1711. [DOI] [PubMed] [Google Scholar]
- 52.Arts DG, de Keizer NF, Vroom MB, De Jonge E. Reliability and accuracy of sequential organ failure assessment (SOFA) scoring. Critical Care Medicine. 2005;33(9):1988–1993. [DOI] [PubMed] [Google Scholar]
- 53.Johnson S, Saranya A. Comparison of different scoring systems used in the intensive care unit. J Pulm Respir Med. 2015;5(276):2. [Google Scholar]
- 54.Luo X, Wang H, Hu S, et al. Comparison of three different organ failure assessment score systems in predicting outcome of severe sepsis. Zhonghua wai ke za zhi [Chinese journal of surgery]. 2009;47(1):48–50. [PubMed] [Google Scholar]
- 55.Viglino D, Maignan M, Debaty G. A modified sequential organ failure assessment score using the Richmond agitation-sedation scale in critically ill patients. Journal of thoracic disease. 2016;8(3):311. [DOI] [PMC free article] [PubMed] [Google Scholar]
