Accuracy of a Modified qSOFA Score for Predicting Critical Care Admission in Febrile Children

The LqSOFA, a rapid bedside assessment, has greater discrimination than the qSOFA in predicting CC admission in febrile children in the ED.

Abstract

Download video file ^{(13.3MB, mp4)}

DOI: 10.1542/6176719900001

Video Abstract

BACKGROUND AND OBJECTIVES:

The identification of life-threatening infection in febrile children presenting to the emergency department (ED) remains difficult. The quick Sequential Organ Failure Assessment (qSOFA) was only derived for adult populations, implying an urgent need for pediatric scores. We developed and validated a novel, adapted qSOFA score (Liverpool quick Sequential Organ Failure Assessment [LqSOFA]) and compared its performance with qSOFA, Pediatric Early Warning Score (PEWS), and National Institute for Health and Care Excellence (NICE) high-risk criteria in predicting critical care (CC) admission in febrile children presenting to the ED.

METHODS:

The LqSOFA (range, 0–4) incorporates age-adjusted heart rate, respiratory rate, capillary refill, and consciousness level on the Alert, Voice, Pain, Unresponsive scale. The primary outcome was CC admission within 48 hours of ED presentation, and the secondary outcome was sepsis-related mortality. LqSOFA, qSOFA, PEWS, and NICE high-risk criteria scores were calculated, and performance characteristics, including area under the receiver operating characteristic curve, were calculated for each score.

RESULTS:

In the initial (n = 1121) cohort, 47 CC admissions (4.2%) occurred, and in the validation (n = 12 241) cohort, 135 CC admissions (1.1%) occurred, and there were 5 sepsis-related deaths. In the validation cohort, LqSOFA predicted CC admission with an area under the receiver operating characteristic curve of 0.81 (95% confidence interval [CI], 0.76 to 0.86), versus qSOFA (0.66; 95% CI, 0.60 to 0.71), PEWS (0.93; 95% CI, 0.90 to 0.95), and NICE high-risk criteria (0.81; 95% CI, 0.78 to 0.85). For predicting CC admission, the LqSOFA outperformed the qSOFA, with a net reclassification index of 10.4% (95% CI, 1.0% to 19.9%).

CONCLUSIONS:

In this large study, we demonstrate improved performance of the LqSOFA over qSOFA in identifying febrile children at risk for CC admission and sepsis-related mortality. Further validation is required in other settings.

What’s Known on This Subject:

The quick Sequential Organ Failure Assessment has been shown to more accurately predict mortality or ICU transfer than systemic inflammatory response syndrome or the quick Pediatric Logistic Organ Dysfunction-2 in an emergency department population, but with only moderate prognostic accuracy.

What This Study Adds:

In this retrospective study of >12 000 febrile children, the Liverpool quick Sequential Organ Failure Assessment outperforms the quick Sequential Organ Failure Assessment in predicting critical care admission. Liverpool quick Sequential Organ Failure Assessment is a rapid bedside tool that should undergo implementation testing.

Acute febrile illness is one of the most common reasons for presentation to the pediatric emergency department (ED).¹ Within this heterogeneous group of patients, only a small minority will have sepsis. However, for these children, mortality rates remain high,² and delays in recognition and treatment of sepsis lead to an increased risk of mortality.³ To date, a sensitive and specific tool to identify patients with sepsis in the ED remains elusive. In adults, the quick Sequential Organ Failure Assessment (qSOFA) score is proposed as a screening tool for clinicians outside the ICU to promptly identify patients with infection who are more likely to have poorer outcomes.^4,5 The presence of ≥2 of the 3 qSOFA components, altered mentation, raised respiratory rate (RR), and low systolic blood pressure (BP), was associated with an increased risk of mortality, but the derivation and validation of Sepsis-3 and the qSOFA did not involve pediatric data.

The use of the qSOFA to identify pediatric sepsis has many limitations. First, researchers in several studies in both adult and pediatric populations have reported poor sensitivity, ranging from 37% to 70% in predicting in-hospital mortality^6–8 and only 36% to 54% in predicting ICU admission.^7–10 Second, researchers in many qSOFA studies have used mortality as their primary outcome,^4,6–8 but mortality in pediatric ED settings is <1%, therefore critical care (CC) admission is a more suitable outcome because this allows assessment of whether the score can identify those patients requiring additional support, regardless of survival.¹⁰ Third, hypotension is a late sign of pediatric septic shock.³ Furthermore, BP is infrequently measured in the pediatric ED.¹¹

Schlapbach et al⁶ compared the prognostic performance of an age-adjusted qSOFA with systemic inflammatory response syndrome (SIRS) criteria in PICU patients with infection and demonstrated that the qSOFA was not substantially superior to SIRS criteria at predicting mortality and prolonged ICU stay. Researchers of a small pediatric ED study comparing the qSOFA, quick Pediatric Logistic Organ Dysfunction-2, and SIRS criteria for their ability to predict mortality or ICU admission found the qSOFA had the greatest predictive validity but achieved only moderate prognostic accuracy.⁹

To address these limitations, we amended the age-adjusted qSOFA to create a novel score known as the Liverpool quick Sequential Organ Failure Assessment (LqSOFA). Although different scoring systems vary in relation to the number of items considered, and the cutoffs applied, past literature indicates that respiratory, cardiovascular, and central nervous system alterations are the most consistent features of severe illness in children and adults. Accordingly, the qSOFA in adults is based on RR, systolic BP, and Glasgow Coma Scale (GCS).⁴ In contrast to adults, hypotension represents a late sign of pediatric septic shock.³ Therefore, a predictive score that incorporates earlier signs of shock, such as tachycardia and prolonged capillary refill time (CRT),^3,12 could improve on the sensitivity demonstrated by the qSOFA. Researchers of a study assessing various vital signs for their ability to predict severity of infection in an ED population reported heart rate (HR) to be the most sensitive.¹³ Researchers in several studies in both adults and children have found a prolonged CRT acts as a marker for sepsis^12,13 and is associated with a greater degree of organ dysfunction (as measured by Sequential Organ Failure Assessment score)¹⁴ and risk of mortality.^15,16 The Alert, Voice, Pain, Unresponsive (AVPU) scale is the standard assessment of neurologic state in pediatric EDs in the United Kingdom. It is quick and simple to measure and has been shown to correlate well with GCS.¹⁷ For these reasons, we a priori selected RR, HR, CRT, and AVPU for the LqSOFA. A recent secondary analysis of the Fluid Expansion as Supportive Therapy study demonstrated the relevance of respiratory, cardiovascular, and central nervous system dysfunction in identifying patients at a greater risk of mortality, further supporting this approach.¹⁸ The 2005 Pediatric Sepsis Consensus¹⁹ thresholds for abnormal HR and RR, used in both SIRS and the age-adjusted qSOFA, were based on values that were likely determined through clinical consensus.²⁰ In contrast, evidence-based sets of reference ranges have since been developed, on ED²¹ and hospitalized patients,²² which may be more discriminative in the febrile ED population. Therefore, we compared these 2 sets of reference ranges to identify the most suitable for use in the LqSOFA.

Our aim with the current study was to develop and validate a novel, rapid, bedside scoring system for use in febrile children attending the ED to predict CC admission within 48 hours. We compared the performance of our score with the age-adjusted qSOFA, the Pediatric Early Warning Score (PEWS) in routine use at our hospital, and National Institute for Health and Care Excellence (NICE) high-risk criteria.²³

Methods

Study Population and Definitions

The study was conducted at a single center: Alder Hey Children’s Hospital, Liverpool, United Kingdom. This is one of Europe’s largest pediatric hospitals, managing ∼60 000 ED attendances annually.²⁴ The initial cohort used to amend the qSOFA into the LqSOFA (referred to as amendment cohort) was used in a previously published cohort from a prospective data set of febrile children (<16 years) attending the ED between November 2010 and April 2012.²⁴ Children were eligible if the treating clinician decided to perform blood tests. Of the 1872 patients recruited to this study, 670 were excluded because of a lack of capacity for study recruitment in the ED, 15 declined consent, and 4 were excluded because of a primary immunodeficiency. The remaining cohort of 1183 patients was used to develop the LqSOFA score by using age-adjusted percentile ranges and rapid-assessment adaptations of other components of the qSOFA score (Supplemental Information). For the validation cohort, children were identified retrospectively from the hospital electronic patient record if they presented to the ED between September 1, 2015, and August 31, 2017. Eligible patients were identified by reviewing ED and CC admissions during this period. Fever was defined as a temperature ≥38°C recorded in the ED or a history of fever reported in the previous 3 days. Exclusion criteria were missing observation data, no history of fever, and transfer from another hospital. Missing observation data were defined as missing ≥2 components required to calculate any of the scoring systems. If only 1 component was missing (ie, not measured), this was deemed to be normal. Ethical approval was granted for both cohorts (research ethics committee reference: 10/H1014/53 and 16/LO/1684).

Outcome Definitions

The primary outcome was CC admission within 48 hours of ED attendance. The secondary outcome was sepsis-related mortality. This was defined as CC admission within 48 hours of ED presentation with suspected sepsis, which led directly or indirectly to in-hospital death within 28 days of admission, as determined by a review of the clinical notes.

Score Calculations

The variables for LqSOFA were developed by using adapted qSOFA components (Supplemental Information). Two evidence-based sets of reference ranges for abnormal HR and RR (O’Leary et al²¹ and Bonafide et al²²) were compared in the amendment cohort, with the highest performing reference range selected for use in the LqSOFA (Supplemental Information). Scores for LqSOFA, qSOFA, PEWS, and NICE high-risk criteria were calculated by using the worst observations recorded in the ED. Each scoring system differs in both the included vital signs and the age-thresholds used; these can be seen in full in the Supplemental Information. PEWS was in routine use clinically and was therefore calculated electronically in real-time, whereas scores for the other 3 systems were calculated retrospectively for study purposes only, by using individually documented vital sign information in the electronic patient record. As with the qSOFA, each component in the LqSOFA could be scored as 0 or 1, making a total possible score of 4 (Table 1). If blood tests were performed, the worst lactate and worst C-reactive protein (CRP) recorded in the ED were collected.

TABLE 1.

Scoring Thresholds for Each Component of LqSOFA

Criterion	Points Allocated
	1 Point	0 Points
CRT	≥3 s	<3 s
AVPU	VPU	Alert
HR	>99th centile Bonafide et al22 age-specific thresholds	≤99th centile Bonafide et al22 age-specific thresholds
RR	>99th centile Bonafide et al22 age-specific thresholds	≤99th centile Bonafide et al22 age-specific thresholds

VPU, Voice, Pain, Unresponsive.

Statistical Analysis

We assessed the prognostic ability of each scoring system using the area under the receiver operating characteristic curve (AUC). Sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratio, odds ratio, and accuracy were calculated for each score alongside an asymptotic 95% confidence interval (CI). LqSOFA and qSOFA were compared by using net reclassification improvement (NRI).²⁵ Statistical analyses were performed by using SPSS version 25 (IBM SPSS Statistics, IBM Corporation) and R version 3.6.1.²⁶ Graphics were generated by using ggplot2²⁷ and eulerr.²⁸

Results

Study Population

A total of 62 (5.2%) patients were excluded from the amendment cohort because of missing observation data, leaving 1121 patients for the amendment cohort (Supplemental Information). Of 14 121 patients identified for the validation cohort, 1880 were excluded: 1249 (8.8%) because of missing observation data, 454 (3.2%) because of no history of fever, 96 (0.7%) because of duplicates, 77 (0.5%) because patients were ≥16 years of age, and 4 (0.03%) because of transfer from another hospital (Fig 1). Importantly, 848 (75.6%) and 8354 (68.2%) patients in the amendment and validation cohorts, respectively, did not have any BP measurements recorded while in the ED. Demographic characteristics, clinical outcomes, and scores of the amendment and validation cohorts are summarized in Table 2.

FIGURE 1.

Flow diagram summarizing the composition of the validation cohort.

TABLE 2.

Summary of Demographic Characteristics, Clinical Outcomes, and Scores of the Amendment and Validation Cohorts

Characteristics	Amendment Cohort	Validation Cohort
No. patients	1121	12 241
Age in y, median (IQR)	2.3 (4.7)	2.5 (3.8)
Male sex, n (%)	621 (55.4)	6527 (53.3)
Admitted to ward, n (%)	817 (72.9)	1481 (12.1)
CC admission within 48 h, n (%)	47 (4.2)	135 (1.1)
Hospital LOS, d, median (IQR)	3 (3)	1.04 (2.73)
CC LOS, d, median (IQR)	2 (5)	2.23 (4.33)
In-hospital mortality, n (%)	1 (0.09)	10 (0.08)
Sepsis-related mortality, n (%)	1 (0.09)	5 (0.04)
LqSOFA ≥2, n (%)	79 (7.0)	155 (1.3)
LqSOFA ≥1, n (%)	369 (32.9)	1919 (15.7)
qSOFA ≥2, n (%)	31 (2.8)	149 (1.2)
qSOFA ≥1, n (%)	1027 (91.6)	11 579 (94.6)
PEWS ≥3, n (%)	—	1465 (12.0)
PEWS ≥2, n (%)	—	5211 (42.6)
NICE high-risk criteria ≥2, n (%)	199 (17.8)	1357 (11.1)
NICE high-risk criteria ≥1, n (%)	737 (65.7)	6787 (55.4)

PEWS not in use at the hospital when amendment cohort data were collected. IQR, interquartile range; LOS, length of stay; —, not applicable.

Amendment Cohort

By using the 99th centile cutoffs from Bonafide et al,²² the LqSOFA score achieved an AUC of 0.78 (95% CI, 0.71 to 0.85) in predicting CC admission within 48 hours, compared with the qSOFA (AUC, 0.61; 95% CI, 0.52 to 0.70) and NICE high-risk criteria (AUC, 0.73; 95% CI, 0.65 to 0.81). The performance of each score and biomarkers at different cutoffs are shown in Fig 2 and Supplemental Table 10.

FIGURE 2.

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the amendment cohort. PEWS data are not available because it was not in use at the hospital when amendment cohort data were collected.

Validation Cohort

LqSOFA was then applied to the independent validation cohort. A total of 135 children (1.1%) were admitted to CC within 48 hours: 115 to the high-dependency unit (HDU), 28 to the ICU, and 8 to both units. Of those 135 admitted to CC, 116 (85.9%) met Sepsis-3 diagnostic criteria on day 1 of admission, whereas 99 (73.3%) met 2005 Pediatric Sepsis Consensus criteria.^19,29 Lactate was measured in 451 (3.7%) patients; of these, 186 (41.2%) had a lactate ≥2 and 27 (6.0%) of ≥4. CRP was measured in 1628 (13.3%) patients. Ten children (0.08%) died, 5 of which (0.04%) were directly attributable to sepsis.

Comparison of qSOFA, LqSOFA, PEWS, and NICE High-Risk Criteria

For predicting CC admission in the validation cohort, PEWS revealed the greatest discriminative ability (AUC, 0.93; 95% CI, 0.90 to 0.95), followed by LqSOFA (AUC, 0.81; 95% CI, 0.76 to 0.86) and NICE high-risk criteria (AUC 0.81; 95% CI, 0.78 to 0.85), with qSOFA (AUC, 0.66; 95% CI, 0.61 to 0.71) the least discriminative (Table 3, Fig 3). For predicting sepsis-related mortality, PEWS revealed the greatest discrimination (AUC, 0.96; 95% CI, 0.92 to 1), followed by LqSOFA (AUC, 0.87; 95% CI, 0.65 to 1), qSOFA (AUC, 0.81; 95% CI, 0.55 to 1), and NICE high-risk criteria (0.72; 95% CI, 0.58 to 0.86) (Table 3, Supplemental Fig 6). Performance of the scores at various cutoffs as binarized scores are shown in Tables 3 and 4. When comparing LqSOFA with qSOFA by using a cutoff of ≥2 for both scores, the NRI for those admitted to CC within 48 hours was 10.37% (95% CI, 0.90 to 19.84) and for those not admitted 0.07% (95% CI, −0.15 to 0.29), yielding an overall NRI of 10.44% (95% CI, 0.96 to 19.91). Figure 4 reveals the percentage of patients admitted to CC within 48 hours, by score, for each of the 4 systems; in Supplemental Fig 7, we demonstrate this for sepsis-related mortality. In Supplemental Fig 8, we show the number of CC admissions predicted by each scoring system or combination of systems.

TABLE 3.

AUCs for Primary and Secondary Outcomes for LqSOFA, qSOFA, PEWS, NICE High-Risk Criteria, Lactate, and CRP in the Validation Cohort

Predictor and Definition	Primary Outcome: CC Admission Within 48 h	Secondary Outcome: Sepsis-Related Mortality
	AUC (95% CI)	AUC (95% CI)
LqSOFA
LqSOFA score	0.81 (0.76 to 0.86)	0.87 (0.65 to 1)
LqSOFA ≥2	0.69 (0.64 to 0.75)	0.79 (0.53 to 1)
LqSOFA ≥1	0.78 (0.74 to 0.83)	0.82 (0.62 to 1)
qSOFA
qSOFA score	0.66 (0.61 to 0.71)	0.81 (0.55 to 1)
qSOFA ≥2	0.64 (0.58 to 0.70)	0.79 (0.53 to 1)
qSOFA ≥1	0.53 (0.48 to 0.57)	0.53 (0.29 to 0.77)
PEWS
PEWS	0.93 (0.90 to 0.95)	0.96 (0.92 to 1)
PEWS ≥3	0.88 (0.84 to 0.91)	0.94 (0.91 to 0.97)
PEWS ≥2	0.75 (0.72 to 0.78)	0.79 (0.68 to 0.90)
NICE high-risk criteria
NICE high-risk criteria	0.81 (0.78 to 0.85)	0.72 (0.58 to 0.86)
NICE high-risk criteria ≥2	0.75 (0.70 to 0.80)	0.55 (0.28 to 0.81)
NICE high-risk criteria ≥1	0.70 (0.66 to 0.73)	0.72 (0.58 to 0.86)
Lactate
Lactate	0.63 (0.56 to 0.69)	0.89 (0.71 to 1)
Lactate ≥4 mmol/L	0.58 (0.51 to 0.65)	0.85 (0.59 to 1)
Lactate ≥2 mmol/L	0.61 (0.55 to 0.68)	0.66 (0.41 to 0.92)
CRP	0.55 (0.49 to 0.61)	0.83 (0.68 to 0.98)

FIGURE 3.

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the validation cohort.

TABLE 4.

Comparison of Prognostic Performance for LqSOFA, qSOFA, PEWS, NICE High-Risk Criteria, and Lactate in the Validation Cohort

Scoring System	Odds Ratio (95% CI)	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Positive Predictive Value, % (95% CI)	Negative Predictive Value, % (95% CI)	Positive Likelihood Ratio (95% CI)	Negative Likelihood Ratio (95% CI)	Accuracy, % (95% CI)
Primary outcome: CC admission within 48 h
LqSOFA ≥2	76.1 (51.2 to 113.1)	39.2 (31.0 to 48.0)	99.2 (99.0 to 99.3)	34.2 (28.1 to 40.9)	99.3 (99.2 to 99.4)	46.6 (35.0 to 62.0)	0.6 (0.5 to 0.7)	98.5 (98.3 to 98.7)
LqSOFA ≥1	14.4 (9.9 to 21.0)	71.9 (63.4 to 79.3)	85.0 (84.3 to 85.6)	5.1 (4.5 to 5.6)	99.6 (99.5 to 99.7)	4.8 (4.3 to 5.4)	0.3 (0.3 to 0.4)	84.8 (84.2 to 85.4)
qSOFA ≥2	44.3 (29.2 to 67.2)	28.9 (21.4 to 37.3)	99.1 (98.9 to 99.3)	26.2 (20.4 to 32.9)	99.2 (99.1 to 99.3)	31.8 (23.0 to 43.9)	0.7 (0.6 to 0.8)	98.3 (98.1 to 98.5)
qSOFA ≥1	15.7 (1.0 to 252.4)	100 (97.3 to 100)	5.5 (5.1 to 5.9)	1.2 (1.2 to 1.2)	100	1.1 (1.1 to 1.1)	0	6.5 (6.1 to 7.0)
PEWS ≥3	51.9 (31.5 to 85.5)	86.7 (79.8 to 91.9)	88.9 (88.3 to 89.4)	8.0 (7.4 to 8.6)	99.8 (99.7 to 99.9)	7.8 (7.1 to 8.5)	0.2 (0.1 to 0.2)	88.8 (88.3 to 89.4)
PEWS ≥2	17.3 (9.1 to 32.9)	92.6 (86.8 to 96.4)	58.0 (57.1 to 58.9)	2.4 (2.3 to 2.5)	99.9 (99.7 to 99.9)	2.2 (2.1 to 2.3)	0.1 (0.1 to 0.2)	58.4 (57.5 to 59.2)
NICE high-risk criteria ≥2	13.1 (9.3 to 18.7)	60.7 (52.0 to 69.0)	89.5 (88.9 to 90.0)	6.0 (5.3 to 6.9)	99.5 (99.4 to 99.6)	5.8 (5.0 to 6.7)	0.4 (0.4 to 0.5)	89.2 (88.6 to 89.7)
NICE high-risk criteria ≥1	13.0 (6.3 to 26.5)	94.1 (88.7 to 97.4)	45.0 (44.1 to 45.9)	1.9 (1.8 to 2.0)	99.9 (99.7 to 99.9)	1.7 (1.6 to 1.8)	0.1 (0.1 to 0.3)	45.5 (44.6 to 46.4)
Lactate ≥4 mmol/L	8.2 (3.6 to 18.9)	17.8 (10.9 to 26.7)	97.4 (95.2 to 98.8)	66.7 (48.1 to 81.2)	80.4 (78.9 to 81.8)	6.9 (3.2 to 15.0)	0.8 (0.8 to 0.9)	79.6 (75.6 to 83.2)
Lactate ≥2 mmol/L	2.1 (1.3 to 3.3)	55.5 (45.2 to 65.3)	62.9 (57.6 to 67.9)	30.1 (25.7 to 35.0)	83.0 (79.5 to 86.0)	1.5 (1.2 to 1.9)	0.7 (0.6 to 0.9)	61.2 (56.5 to 65.7)
Secondary outcome: sepsis-related mortality
LqSOFA ≥2	119.3 (19.8 to 718.8)	60.0 (14.7 to 94.7)	98.8 (98.6 to 99.0)	1.9 (0.9 to 4.0)	100	48.3 (23.2 to 100.1)	0.4 (0.1 to 1.2)	98.7 (98.5 to 98.9)
LqSOFA ≥1	21.6 (2.4 to 193.0)	80.0 (28.4 to 99.5)	84.4 (83.7 to 85.0)	0.2 (0.1 to 0.3)	100	5.1 (3.3 to 7.9)	0.2 (0.0 to 1.4)	84.4 (83.7 to 85.0)
qSOFA ≥2	124.2 (20.6 to 748.9)	60.0 (14.7 to 94.7)	98.8 (98.6 to 99.0)	2.0 (1.0 to 4.0)	100	50.3 (24.1 to 104.7)	0.4 (0.1 to 1.2)	98.8 (98.6 to 99.0)
qSOFA ≥1	0.6 (0.0 to 11.4)	100 (47.8 to 100)	5.4 (5.0 to 5.8)	0.04 (0.04 to 0.04)	100	1.1 (1.1 to 1.1)	0	5.5 (5.1 to 5.9)
PEWS ≥3	81.2 (4.5 to 1468.7)	100 (47.8 to 100)	88.1 (87.5 to 88.6)	0.3 (0.3 to 0.4)	100	8.4 (8.0 to 8.8)	0	88.1 (87.5 to 88.6)
PEWS ≥2	14.9 (0.8 to 268.7)	100 (47.8 to 100)	57.5 (56.6 to 58.3)	0.1 (0.1 to 0.1)	100	2.4 (2.3 to 2.4)	0	57.5 (56.6 to 58.4)
NICE high-risk criteria ≥2	2.0 (0.2 to 18.0)	20.0 (0.5 to 71.6)	88.9 (88.4 to 89.5)	0.1 (0.0 to 0.4)	100	1.8 (0.3 to 10.4)	0.9 (0.6 to 1.4)	88.9 (88.3 to 89.4)
NICE high-risk criteria ≥1	8.8 (0.5 to 160.0)	100 (47.8 to 100)	44.6 (43.7 to 45.5)	0.1 (0.1 to 0.1)	100	1.8 (1.8 to 1.8)	0	44.6 (43.7 to 45.5)
Lactate ≥4 mmol/L	52.9 (5.3 to 527.5)	75.0 (19.4 to 99.4)	94.6 (92.1 to 96.5)	11.1 (5.9 to 19.9)	99.8 (98.7 to 100)	14.0 (7.0 to 27.8)	0.3 (0.1 to 1.4)	94.5 (91.9 to 96.4)
Lactate ≥2 mmol/L	4.3 (0.4 to 41.9)	75.0 (19.4 to 99.4)	59.1 (54.3 to 63.7)	1.6 (0.9 to 2.8)	99.6 (98.0 to 99.9)	1.8 (1.0 to 3.3)	0.4 (0.1 to 2.3)	59.2 (54.5 to 63.8)

FIGURE 4.

Percentage of validation cohort patients admitted to CC by LqSOFA score, qSOFA score, NICE high-risk criteria, and PEWS, annotated with raw figures above each bar.

Lactate and CRP

For the primary outcome, poor discrimination was seen with both lactate (AUC, 0.63; 95% CI, 0.56 to 0.69) and CRP (AUC, 0.55; 95% CI, 0.49 to 0.61) (Table 3, Fig 3). For the secondary outcome, both lactate (AUC, 0.89; 95% CI, 0.71 to 1) and CRP (AUC, 0.83; 95% CI, 0.68 to 0.98) revealed good predictive ability (Table 3, Supplemental Fig 6). Several standard performance metrics of CRP and lactate, as well as each of the 4 scoring systems, are presented in Tables 3 and 4.

Discussion

With our data, we show that a simple 4-item score reveals improved discriminant ability compared with the age-adjusted qSOFA and NICE high-risk criteria. Our novel, rapid, bedside score for children, the LqSOFA score, has revealed good discriminative ability in predicting CC admission in febrile ED patients in a large independent validation cohort. In the ED, it is important to identify the “needle in the haystack”: those children requiring further investigation and urgent treatment. The use of a sepsis-specific score in a pediatric ED may be of limited value because of the low prevalence, whereas predicting which children might require CC admission may be more useful.

LqSOFA

For predicting CC admission, the LqSOFA demonstrated good prognostic value. With a cutoff of ≥2, the score revealed excellent specificity but was limited by a relatively low sensitivity, consistent with sensitivities reported for the qSOFA in both adults and children.^7–10 Given that the these scores are designed as tools to aid clinician decision-making, rather than simply predict outcome, it has been argued that this rate of false-negatives is unacceptably high.¹⁰ However, a cutoff of ≥1 for the LqSOFA produced a more favorable balance of sensitivity and specificity. Overall, the LqSOFA demonstrates superior discriminative ability than that reported with the qSOFA in the pediatric population, both in the current study and compared with previous research.^6,9,30 In the current study, the LqSOFA revealed the greatest positive predictive value of all 4 scoring systems, although this was only 34%. The low values for positive predictive value seen across all scoring systems likely reflect the low prevalence of sepsis within the ED population and highlight the difficulty in identifying such cases. There has rightly been a focus in recent years on preventing cases of sepsis being missed, leading to the prioritization of sensitivity over other performance measures. In the current study, NICE high-risk criteria achieved excellent sensitivity, greater than the LqSOFA and PEWS, but at the cost of limited specificity. The potential disadvantages of relying on such scoring systems, in overdiagnosis and overtreatment of sepsis, are significant.³¹ Therefore, scoring systems with high sensitivity but poor specificity should be avoided in favor of more balanced scoring systems, ideally those that are quick and simple to calculate. The use of CRT and HR as a proxy for cardiovascular dysfunction means the LqSOFA can be calculated rapidly by using routine clinical observations and without any equipment, such as appropriately sized BP cuffs, a significant advantage over other scoring systems. In addition, shock in children may be present before hypotension is measured³; therefore, these parameters may detect impending shock earlier. This facilitates its potential for standardized use across many other settings including primary care and resource-poor settings. The large number of age categories in the LqSOFA potentially provides greater accuracy but also increases the complexity of the score in comparison with other systems with fewer age categories (eg, NICE high-risk criteria). However, when used electronically, this greater complexity would not equate to a more time-consuming or complicated tool for the clinician; the number of components in any given scoring system has a far greater impact in this regard. Overall, the primary limitation of the use of the LqSOFA is the lower sensitivity achieved in comparison with PEWS. However, the excellent specificity suggests the LqSOFA may be useful in determining which children are at low risk of CC admission or in situations in which discriminant ability is prioritized over sensitivity.

qSOFA

For predicting both the primary and secondary outcome, the qSOFA performed less well than the LqSOFA and PEWS. Our results are consistent with previous research. Schlapbach et al⁶ reported an AUC of 0.64 for the qSOFA in predicting mortality in a PICU cohort. A limitation of their study is the use of a PICU population, given that “quick” organ dysfunction scores are designed to alert the clinician to those at risk for sepsis, and most PICU patients will already have some sort of organ dysfunction and/or organ support. In an ED study, Van Nassau et al⁹ assessed the ability of the qSOFA to predict PICU transfer and/or mortality, reporting an AUC of 0.72 and a sensitivity of 50%, although their study used a relatively small sample (n = 864). With our study, we provide further evidence that the qSOFA lacks the prognostic ability to justify its introduction into routine practice in the pediatric ED.

A further limitation of using the qSOFA in the pediatric population, highlighted by the current study, is the infrequency of BP measurements in the pediatric ED. In the validation cohort, 8354 (68.2%) patients did not have their BP recorded while in the ED; this is consistent with previous research.³² The lack of BP data may have affected the performance of the qSOFA; however, to exclude these patients from the study would mean excluding the vast majority of eligible patients. The results generated from such an analysis would provide little information on the potential “real-world” performance of the score. BP is included in many PEWS,³³ however, BP may not be measured at triage; NICE sepsis guidance recommends BP is measured only when tachycardia or prolonged CRT are present.²³ The infrequency of BP measurements may lead to missed opportunities for the early detection of sepsis. In the current study, almost one-quarter of those admitted to CC within 48 hours did not have their BP measured while in the ED, suggesting the absence of BP measurement is not confined to “well” children. A score that does not require BP measurement, such as LqSOFA, therefore presents a significant advantage.

PEWS

PEWS demonstrated the greatest discriminative ability for both the primary and secondary outcomes, consistent with previous research.³³ Most PEWS are designed to identify deterioration in hospitalized children and have therefore not been validated in an ED setting. A previous ED study revealed that PEWS was an independent predictor of ICU admission within 48 hours, or death within 30 days, in unselected pediatric ambulance patients presenting to the ED.³⁴ The large number of different heterogeneous PEWS in use is a disadvantage because this leads to inconsistency in detecting and responding to acute illness. qSOFA and especially LqSOFA therefore offer an advantage over PEWS as standardized tools to improve sepsis recognition. Importantly, PEWS was the only scoring system in routine clinical use during the study, and hence PEWS scores were known by treating physicians. This may have influenced decisions regarding transfer to CC, potentially altering the performance of the score in comparison with the other scoring systems, which had not been applied clinically. Furthermore, compared with only 3 and 4 items in the qSOFA and LqSOFA, respectively, PEWS often include several more variables and is therefore more complicated and time-consuming to calculate (Supplemental Fig 5).³³ Although a granular score may perform better than a parsimonious score such as ours, it may be less feasible in practice for rapid assessment in a busy ED.

NICE High-Risk Criteria

A previous study compared the NICE high-risk criteria in adults, with qSOFA, National Early Warning Score, and SIRS for their ability to predict mortality in an ED and ward setting.³⁵ NICE high-risk criteria achieved a sensitivity of only 58.9% and were not independently associated with an adverse outcome. In the current study, for the primary outcome we found good discriminative ability suggesting NICE high-risk criteria correlate well with risk of CC admission, and the criteria correctly identified all 5 sepsis-related deaths. However, 55% of validation cohort patients were positive for NICE high-8risk criteria, suggesting the criteria lack the specificity to be useful alone as a screening tool for possible sepsis in the ED. Given that the guidelines recommend senior review for children with NICE high-risk criteria features,²³ implementation is likely to have huge resource implications in an already stretched health service. Importantly, our analysis did not include many other NICE high-risk criteria features such as cyanosis, nonblanching rash, and “appears ill to health care worker,” suggesting the NICE high-risk criteria could be simplified.

Lactate and CRP

A strong association between lactate and mortality in pediatric sepsis has previously been reported.³⁶ In the current study, lactate revealed good discriminative ability for sepsis-related mortality, providing further support for including lactate measurement in risk stratification in pediatric sepsis.³⁷ CRP revealed poor versus good discrimination for the primary versus secondary outcomes, respectively, which is consistent with several studies in which researchers report CRP as a poorer predictor of sepsis than other available biomarkers.³⁸

Comparison of All Scores

All 4 scoring systems in the current study used different age-specific thresholds. Schlapbach et al⁶ noted the lack of age-specific cutoffs for the original qSOFA and therefore applied those used in the corrected 2005 Pediatric Sepsis definitions.^19,39 These age-specific thresholds are notably lower than those described by Bonafide et al²² and used in the LqSOFA. NICE high-risk criteria and the PEWS in the current study generally use thresholds in between these 2. This lack of consensus in age-specific thresholds may explain a large part of the difference in prognostic ability seen between the scoring systems and highlights the urgent need for standardization of reference ranges. The Bonafide et al²² reference ranges for hospitalized children are closer to the febrile child population than the O’Leary et al²¹ reference ranges for low acuity children presenting to the ED or the Fleming et al²⁰ reference ranges for healthy children used in our PEWS and are therefore the best ranges to identify children with severe enough illness to require CC admission.

For operationalization of the score in the ED, a highly sensitive score (NICE high-risk criteria ≥1) could be used as a screening test in triage to identify the group at risk for sepsis, followed by a highly specific score (LqSOFA≥2) to identify those children at highest risk of poor outcome. This can be implemented electronically (such as reported by Balamuth et al⁴⁰), with an LqSOFA score ≥2 prompting immediate senior review.

Strengths and Limitations

To our knowledge, this is the largest study in which the performance of quick assessment scores, including the qSOFA, was assessed in a pediatric ED population. Our population reflects a real-world high-income ED setting, with a low prevalence of sepsis. We compared different age-specific cutoffs and 2 other prediction scores for sepsis: NICE high-risk criteria and qSOFA. The LqSOFA can be calculated rapidly across a variety of different health care settings, because no equipment is required. Limitations include use of retrospective data, patients from a single center, criteria of CC admission which may depend on local practice, and the low mortality. In addition, hypothermic patients presenting with sepsis may have been missed.

Conclusions

Overall, with the findings of the current study, we demonstrate the superior performance of LqSOFA and question the use of the qSOFA to identify patients at risk for sepsis in the pediatric ED. The LqSOFA requires further evaluation in other settings before recommending more widespread use as entry criterion into randomized controlled trials in the pediatric ED.

Glossary

AUC

area under the receiver operating characteristic curve

AVPU

Alert, Voice, Pain, Unresponsive

BP

blood pressure

CC

critical care

CI

confidence interval

CRP

C-reactive protein

CRT

capillary refill time

ED

emergency department

GCS

Glasgow Coma Scale

HDU

high-dependency unit

HR

heart rate

LqSOFA

Liverpool quick Sequential Organ Failure Assessment

NICE

National Institute for Health and Care Excellence

NRI

net reclassification improvement

PEWS

Pediatric Early Warning Score

qSOFA

quick Sequential Organ Failure Assessment

RR

respiratory rate

SIRS

systemic inflammatory response syndrome