Skip to main content
PLOS One logoLink to PLOS One
. 2024 Jul 19;19(7):e0306172. doi: 10.1371/journal.pone.0306172

Validating the performance of organ dysfunction scores in children with infection: A cohort study

Shaojun Li 1,2,#, Tao Tan 3,#, Jing Li 2,4, Hongdong Li 1, Liang Zhou 1, Ke Bai 4, Li Xiao 5, Ximing Xu 5, Liping Tan 1,2,*
Editor: Dong Wook Jekarl6
PMCID: PMC11259267  PMID: 39028682

Abstract

Purpose

We aimed to validate the performance of six available scoring models for predicting hospital mortality in children with suspected or confirmed infections.

Methods

This single-center retrospective cohort study included pediatric patients admitted to the PICU for infection. The primary outcome was hospital mortality. The six scores included the age-adapted pSOFA score, SIRS score, PELOD2 score, Sepsis-2 score, qSOFA score, and PMODS.

Results

Of the 5,356 children admitted to the PICU, 9.1% (488) died, and 25.1% (1,342) had basic disease with a mortality rate of 12.7% (171); 65.3% (3,499) of the patients were younger than 2 years, and 59.4% (3,183) were male. The discrimination abilities of the pSOFA and PELOD2 scores were superior to those of the other models. The calibration curves of the pSOFA and PELOD2 scores were consistent between the predictions and observations. Elevated lactate levels were a risk factor for mortality.

Conclusion

The pSOFA and PELOD2 scores had superior predictive performance for mortality. Given the relative unavailability of items and clinical operability, the pSOFA score should be recommended as an optimal tool for acute organ dysfunction in pediatric sepsis patients. Elevated lactate levels are related to a greater risk of death from infection in children in the PICU.

Introduction

Sepsis is a predominant condition associated with high morbidity and mortality and is a major health burden for children worldwide [1]. It is estimated that 25.2 million children suffer from sepsis annually, and approximately 3.3 million children with sepsis die every year [2]. Sepsis is a potentially fatal syndrome caused by acute organ abnormalities resulting from an abnormal host response to infection [3]. Since the Sepsis-3 criteria were published in 2016 [3], criteria for applying Sepsis-3 in children are lacking [4]. Sepsis-3 included the Sequential Organ Failure Assessment (SOFA) as a scoring system for organ dysfunction in adult sepsis patients based on a large cohort study that revealed that the SOFA score had superior discrimination of hospital mortality compared to other scores in patients with suspected or definite infection [5, 6]. However, the SOFA score developed and validated with adult data was inapplicable to children. Given that current data are insufficient to propose a particular scoring system [711], the 2020 consensus of the Surviving Sepsis Campaign (SSC) did not recommend a specific screening or diagnostic tool for septic shock and sepsis-associated organ dysfunction (SAOD) in children [12].

Recently, several scores assessing pediatric organ dysfunction, including the pediatric SOFA (pSOFA) score, systemic inflammatory response syndrome (SIRS) score, quick SOFA (qSOFA) score, Pediatric Logistic Organ Dysfunction-2 (PELOD-2) score, and Pediatric Multiple Organ Dysfunction Score (PMODS), were adapted and validated in a series of cohort studies [8, 9, 1317]. Matics and colleagues, who utilized the age-adapted pSOFA score and validated the performance of the PELOD score, pSOFA score, PELOD2 score and PMODS, demonstrated that the pSOFA score had superior discrimination for hospital mortality in a large PICU cohort [8]. Schlapbach et al. revealed that the pSOFA score was a better predictor of hospital mortality than the qSOFA score or SIRS score in their cohort of children with definitive or suspected infection [9]. However, one of the cohorts included patients aged 21 years or younger, and the median age of the other cohort was 13 years. These cohorts were not representative of healthy children in the PICU and originated from developed countries [4, 18].

A retrospective cohort study was performed using a PICU electronic health record (EHR) database to validate and compare the performance of currently available age-adapted scores in children with suspected or confirmed infection, and we attempted to provide clinical evidence for selecting an optimal scoring system for pediatric guidelines for septic shock and SOAD. Given that the latest SSC guidelines do not recommend a specific organ dysfunction tool for children with sepsis, SIRS was included in this study. A total of six scores were applied to our cohort. The pSOFA or qSOFA score used mean arterial pressure (MAP) [9], the PELOD2 score was calculated excluding unavailable items [9, 10], the Sepsis-2 score was determined in accordance with the International Pediatric Sepsis Consensus Conference [7], and the SIRS score and PMODS were summarized on the basis of the original provenance [7, 11].

Methods

Study design, setting and participants

A single-center retrospective cohort study was performed, and patients aged four weeks to 18 years who were admitted to the PICU at the Children’s Hospital of Chongqing Medical University (CHCMU) in China with suspected or diagnosed infections from 2015 to 2021 were included. The CHUCMU is composed of two hospitals: Yuzhong Hospital, with a 32-bed PICU, and Liangjiang Hospital, with a 55-bed PICU. The data were obtained from the Sepsis Specialized Disease Database (SSDD) of EHRs in CHCMU, which included children in the PICU with definitive or suspected infection. The data captured from the SSDD were anonymous, and the authors could not identify individual participants during or after the data collection. The study was approved by the Ethics Committee of CHCMU. The protocol was registered in the Chinese Clinical Trial Registry (registration number: ChiCTR2100053198), an international clinical trial registry platform. Research data were accessed on December 31, 2021.

In the SSDD, suspected infection was defined when a patient was first given antibiotics combined with culture sampling within 24 hours or if culture sampling was obtained first and then antibiotics were given within 72 hours. Six scores, including the pSOFA score, age-specific SIRS score, PELOD2 score, Sepsis-2 score, age-adapted qSOFA score, and PMODS, were calculated based on the lowest values recorded during the first day in the PICU, after which we validated the performance of the six scores from the SSDD cohort (S1 and S2 Tables). Additionally, the alternate pSOFA (pSOFAal) and alternate qSOFA (qSOFAal) scores, which include systolic blood pressure (SBP) as a cardiovascular item, were compared with the pSOFA and qSOFA scores.

Data collection

We defined hospital mortality as the primary outcome and hospital stay and length of stay in the PICU as secondary outcomes. Baseline variables, inflammatory markers and candidate score variables were extracted, and the lowest and highest records during the first day of PICU admission were collected from the SSDD. The SIRS and Sepsis-2 scores followed the 2005 consensus for pediatric sepsis [7]. Age-adapted cardiovascular subscores of the pSOFA or pSOFAal scores were calculated by the PELOD2 threshold of MAP or systolic blood pressure (SBP), and renal subscores were defined by the PELOD2 threshold of the serum creatinine level [9, 10]. Age-adapted qSOFA and qSOFAal scores were determined by using age-adjusted respiratory rates and the MAP or SBP according to the 2005 Pediatric Sepsis Consensus [5, 7, 9]. Given that pupillary dilatation data were unavailable, the PELOD2 score was calculated using the remaining items [10].

Statistics

Continuous variables are presented as medians with interquartile ranges (IQRs) or means with standard deviations, and binary variables are expressed as numbers with percentages. A t test was used to analyze normally distributed data, the Wilcoxon rank sum test was applied to compare nonnormally distributed data, and Pearson’s chi-squared test was used for categorical variables. An adjusted risk model was developed for hospital mortality based on the patients’ baseline characteristics, which included age, sex and comorbidities, by a multivariable logistic regression model. Univariate or multivariate logistic regression models were applied to the six scores with or without the baseline model to estimate the associations between the models and the primary outcome. The discrimination ability of the models was evaluated by the area under the receiver operating characteristic curve (AUROC). Calibration plots were generated to determine the agreement between the predicted and observed outcomes, and Brier scores (BSs) were calculated to present the overall performance of all the models [19]. Decision curve analysis (DCA) and the clinical impact curve (CIC) were generated to assess the clinical net benefit of the models [20, 21]. Subgroup analyses were then performed for the primary outcome, and the patients were stratified by basic patient factors, including age group, sex, comorbidities and lactate level. Missing data were accounted for using multiple imputation (S1 Fig). Data analyses were performed using R version 4.0.5 (R Foundation). A two-sided p value less than 0.05 was considered to indicate statistical significance.

Results

Study cohort

From 2015 to 2021, 5,609 pediatric admission records were collected, and when duplicated records were removed and encounters with missing outcomes were excluded, 5,356 children in the PICU with suspected infection were eligible for the final cohort. Table 1 shows that children younger than 2 years accounted for the majority of the sample. The proportion of males was slightly greater than that of females. The overall mortality rate was 9.1%, and the mortality rate was greater among children with comorbidities. The three most common comorbidities were respiratory diseases, hematological diseases and traumatic diseases (S2 Fig). The median PICU length of stay was 6 days, and the median hospital length of stay was 21 days.

Table 1. Baseline characteristics of the children with infections in the cohort.

Characteristic Total (n = 5356) Survived (n = 4,868) Died (n = 488) p value
Age_group* <0.001
  <2 yr 3499 (65.3%) 3231 (66.4%) 268 (54.9%)
  >12_to_18_yr 299 (5.6%) 254 (5.2%) 45 (9.2%)
  >5_to_12_yr 787 (14.7%) 694 (14.3%) 93 (19.1%)
  2_to_5yr 771 (14.4%) 689 (14.2%) 82 (16.8%)
Sex* 0.226
Male 3183 (59.4%) 2906 (59.7%) 277 (56.8%)
Female 2173 (40.6%) 1940 (40.3%) 211 (43.2%)
WBC (109/L)# 11.96 (8.09,17.30) 11.92 (8.21, 17.11) 12.79 (5.83, 20.21) 0.84
CRP (mg/L)# 16.00 (5.00, 42.00) 15.00 (5.00, 40.00) 30.00 (5.00, 67.00) <0.001
PCT (ng/L)# 0.98 (0.17, 7.47) 0.85 (0.16, 5.94) 3.85 (0.56, 26.91) <0.001
Lac (mmol/L)# 1.80 (1.20, 3.00) 1.70 (1.10, 2.70) 5.30 (2.60, 9.20) <0.001
pSOFA* 4.00 (3.00, 6.00) 4.00 (3.00, 6.00) 7.00 (5.00, 10.00) <0.001
SIRS* 2.00 (1.00, 3.00) 2.00 (1.00, 3.00) 3.00 (2.00, 4.00) <0.001
Sepsis-2* 2.00 (1.00, 2.00) 2.00 (1.00, 2.00) 3.00 (2.00, 4.00) <0.001
qSOFA* 1.00 (1.00, 2.00) 1.00 (1.00, 2.00) 2.00 (1.00, 2.00) <0.001
PMODS* 5.00 (4.00, 7.00) 5.00 (4.00, 7.00) 8.00 (6.00, 10.00) <0.001
PELOD2* 6.00 (5.00, 7.00) 6.00 (5.00, 7.00) 8.00 (7.00, 9.00) <0.001
Comorbidities* 1342 (25.1%) 1171 (24.1%) 171 (35.0%) <0.001
MV* 4690 (87.5%) 4202 (86.3%) 488 (100%) <0.001
Hospital length of stay (day)# 21.00 (12.00, 32.00) 22.00 (13.00, 33.00) 9.00 (3.00, 19.00) <0.001
PICU length of stay (day)# 6.00 (3.00, 12.00) 6.00 (3.00, 12.00) 3.00 (1.00, 10.00) <0.001
ECMO* 22 (0.4) 13 (0.3) 9 (1.8) <0.001
VP* 958 (17.9) 656 (13.5) 302 (61.9) <0.001
RRT* 266 (5.0) 179 (3.7) 87 (17.8) <0.001

Note:

* n (%)

# Median (IQR); Abbreviations: WBC, white blood cell; CRP, C-reactive protein; PCT, procalcitonin; Lac, lactate; pSOFA, Pediatric Sequential Organ Failure Assessment; SIRS, systemic inflammatory response syndrome; qSOFA, quick SOFA; PMODS, Pediatric Multiple Organ Dysfunction Score; PELOD2, Pediatric Logistic Organ Dysfunction-2; MV, mechanical ventilation; ECMO, extracorporeal membrane oxygenation; VP, vasopressor; RRT, renal replacement therapy.

Models and primary outcomes

Most patients had a pSOFA score, PELOD2 score, PMODS, SIRS score or Sepsis-2 score of 2 or higher, whereas half of the children had a qSOFA score of 2 or lower (S3 Fig). The pSOFAal and qSOFAal scores were similar to the pSOFA and qSOFA scores (S3 Table). Compared with the surviving patients, the patients who died had significantly greater pSOFA scores, PELOD2 scores, SIRS scores, Sepsis-2 scores, qSOFA, scores and PMODSs (Table 1). For the pSOFA score, PELOD2 score, PMODS, qSOFA score and Sepsis-2 score, patients with scores of 2 or more points had more than a 4-fold increased risk of mortality than patients with a 2-fold change in the SIRS score (Figs 1 and S4).

Fig 1. Bar chart of the mortality distribution for each model score.

Fig 1

A pSOFA score, B SIRS score, C PELOD2 score, D Sepsis-2 score, E qSOFA score, F PMODS.

In the cohort, six scores with or without the adjusted baseline model were associated with a similar odds ratio (OR) and 95% confidence interval (95% CI) for in-hospital mortality [pSOFA score: 1.59 (1.53–1.65), 1.57 (1.51–1.63); SIRS: 1.46 (1.34–1.6), 1.39 (1.26–1.54); PELOD2 score: 2.1 (1.97–2.24), 2.07 (1.94–2.21); Sepsis-2 score: 2.71 (CI 2.46–2.98), 2.65 (2.41–2.92); qSOFA score: 4.28 (3.65–5.01), 4.25 (3.60–5.02); PMODS: 1.44 (1.39–1.5), 1.44 (1.38–1.49); un or adjusted model] (Fig 2).

Fig 2. Unadjusted and adjusted scores according to the baseline model regressed for hospital mortality among children with suspected or confirmed infection.

Fig 2

Note: OR, odds ratio; 95% CI, 95% confidence interval.

Performance of the models

Among the six scores, the pSOFA (AUROC 0.78, 95% CI 0.76–0.81) and PELOD2 (AUROC 0.79, 95% CI 0.77–0.81) scores had the highest discrimination, the Sepsis-2 score (AUROC 0.75, 95% CI 0.72–0.77) and PMODS (AUROC 0.74, 95% CI 0.72–0.77) had moderate discrimination, and the SIRS (AUROC 0.62, 95% CI 0.59–0.64) and qSOFA (AUROC 0.71, 95% CI, 0.68–0.73) scores had relatively poor discrimination. For the primary outcome, the AUC of each score adjusted for in the baseline model was similar to that of each score separately (Fig 3).

Fig 3. Comparison of the AUROCs of unadjusted and adjusted scores for hospital mortality.

Fig 3

AUROCs are reported for A. pSOFA, B. SIRS, C. PELOD2, D. Sepsis-2, E. qSOFA, and F. PMODS. Note: AUROC, area under the receiver operating characteristic curve; un, unadjusted; adj, adjusted.

Among the unadjusted scores, the calibration curves of the pSOFA and PELOD2 models in isolation presented superior consistency between the prediction and observation results, with superior overall performance, and the remaining four scores showed poor agreement with relatively poor overall performance (Fig 4). All the models adjusted by the baseline risk model for hospital mortality had nearly the same calibration and overall performance as the unadjusted models.

Fig 4. Calibration plots of scores unadjusted and adjusted for in the baseline model for hospital mortality among all children in the PICU cohort.

Fig 4

The DCA plots of the six scores indicated that the net benefits of the pSOFA and PELOD2 scores were greater than those of the other four scores, indicating that the former were optimal, while the latter were inferior (S7 Fig). As shown by the CIC, the pSOFA and PELOD2 scores were superior in terms of overall net benefit, with relatively wide threshold probabilities in comparison with the other scores, which indicated that the former possessed significant clinical applicability.

Regarding discrimination and calibration, there were no differences between the pSOFAal and pSOFA scores or between the qSOFAal and qSOFA scores either with or without the baseline model (S3 Table).

Subgroup analyses

Subgroup analyses were performed by age (groups aged ≥ 2 years or < 2 years), sex (male or female) and comorbidities (presence or absence of comorbidities) as the primary outcomes. The primary outcome exhibited similar changes between the patients in subgroup aged 2 years or older and those in subgroup aged younger than 2 years (S4 Fig). The data showed that the mortality of infected children increased with increasing lactate levels, and the mortality rate reached more than 30% when the lactate level was greater than 5 mmol/L (S5 Fig). In addition, lactate levels were positively correlated with pSOFA scores (S6 Fig). The levels of the inflammatory markers PCT and CRP were significantly greater in the nonsurviving group than in the surviving group (Table 1). Additionally, there was a relatively weak association between PCT or CRP levels and the six scoring systems (adjusted R2 ranging from 0.02 to 0.17) (Figs S9 and S10).

Discussion

In this cohort of PICU patients with infection in Southwest China, we validated six available scores to identify organ dysfunction in children with sepsis. The pSOFA score and PELOD2 score showed superior discrimination for hospital mortality compared with the other four models. These two scores also revealed good calibration between the prediction and observation and better clinical net benefit prediction than did the other scores for hospital mortality. In most PICUs, the MAP is calculated simultaneously by an electrocardiograph monitor when measuring SBP; therefore, for the pSOFA score, the MAP and SBP may be alternative cardiovascular indicators. Given the clinical applicability of these models, the pSOFAal score was calculated, and the SBP was used to replace the MAP-cardiovascular item of the pSOFA score, as in the qSOFAal score. Our data revealed that there was no difference in the use of MAP or SBP for either the pSOFA or qSOFA score. Compared with the PELOD2 score, the pSOFA score is more likely to be applied in clinical practice because it has fewer items. Therefore, the age-adapted pSOFA score should be considered when determining sepsis-associated organ dysfunction scores in children.

Since the sepsis-3 definition and criteria were established for adults in 2016, quite a few efforts have been made to apply these criteria in pediatric patients [8, 9, 15, 16, 18, 2224]. A range of small- to medium-sample cohort studies assessed the prognosis of children with different clinical conditions in the PICU using the SIRS, pSOFA, and qSOFA scores, and the data were not sufficient to provide an optimal tool for identifying organ abnormalities in patients with sepsis. For children outside the PICU, high-quality studies evaluating sepsis-associated organ dysfunction are lacking, although several cohort studies have developed predictive models using machine learning algorithms to predict the occurrence of sepsis [2325]. In recent years, an Australian and Zelanian PICU cohort included 2,715 children, and that study validated the prognostic accuracy of the pSOFA score, SIRS score, severe sepsis score, PELOD2 score and qSOFA score, revealing that the pSOFA score was better than the SIRS, Sepsis-2 and qSOFA scores [9]. Another American cohort in the PICU included 6,303 children aged younger than 21 years and used the pSOFA score, in which the MAP was separated into seven subgroups according to age; the pSOFA score exhibited superior performance in comparison with other organ dysfunction scoring systems, such as the PELOD score, PELOD2 score and PMODS [8]. These two cohort studies and our study recruited participants admitted to the PICU and presented similar accuracy among recently available organ dysfunction models. With our data, additional calibration and clinical applicability were performed, which further validated the precision and clinical benefits of the scores, and the results showed that the pSOFA score had superior predictive performance for poor prognosis in the PICU. In 2020, the updated international guidelines for SSC in children defined septic shock and SAOD and suggested systematic screening to identify SAOD or septic shock but did not provide special criteria for organ dysfunction [12]. Recently, the Pediatric Organ Dysfunction Information Update Mandate, which derived an evidence-based organ dysfunction list for ten organs from published literature reviews, provided the latest roadmap for pediatricians to conduct and validate single- or multiple-organ dysfunction in children with sepsis [26]. Therefore, this study is an important supplement to clinical evidence [12].

The blood lactate value is used to evaluate tissue perfusion in sepsis or septic shock patients, and in most settings, lactate can be measured rapidly [12, 27]. Lactate levels are recommended as a marker of tissue hypoxia in children according to sepsis guidelines, but unlike adult guidelines, the pediatric guidelines do not suggest a certain lactate level threshold as an indication of metabolic abnormalities in septic shock patients [1, 12, 28, 29]. Our study revealed that lactate levels are an important predictor of hospital mortality in children with infections in the PICU. A series of observational studies indicated that elevated lactate levels were correlated with poor outcomes in children with septic shock [16, 2830]. In a sepsis cohort from the PICU, lactate levels were independently associated with hospital mortality, and patients whose lactate levels were 2 mmol/L or greater had a twofold increased risk of in-hospital death [16]. Future studies should be performed to define the optimal threshold for hyperlactatemia in children.

Our cohort had relatively different demographics compared to those of other larger cohorts. In our Asian cohort, approximately two-thirds of the children were aged less than 2 years. In previous cohorts, white and black patients were included in the American cohort, and brown and white patients were included in the Australian cohort, accounting for nearly 30% of the patients younger than 2 years [8, 9]. Hence, these three large PICU cohorts possessed adequate population representativeness. In these studies, the varied distributions of hospital mortality might be attributed to the severity of the patients’ conditions [8, 9]. In our cohort, 9.1% of the children died, and there was a greater proportion (12.7%) of patients with comorbidities. The mortality rate was 2.6% in the American cohort and 5.8% in the Australian cohort. Additionally, sensitivity analyses were performed with a baseline risk model to increase the robustness of the statistical results, and available organ dysfunction model-adjusted or unadjusted baseline models presented with similar ORs and discrimination ability for the primary outcome.

This study has several limitations. First, given the retrospective cohort of children as young as seven years based on EHRs and in which follow-up for discharged children was very difficult, we evaluated only hospital mortality as the primary clinical endpoint. A proportion of participants may have died in the first weeks after discharge. Second, our cohort included only patients in the PICU and did not include hospitalized children outside the PICU because of the large amount of missing data for non-PICU patients; therefore, we could not validate the models’ performance for this population. Third, a small portion of patients in this PICU cohort had missing laboratory data, which may have affected the accuracy of the calculations. Fourth, the PELOD2 score was calculated without including pupillary dilation because it could not be extracted from the retrospective database. Fifth, we captured the lowest scores using the models on the first day of the PICU stay, so the results of this study were restricted to those after the first day of admission to the PICU. Researchers previously revealed that small-scale clinical features in the first hour allowed the derivation of robust severity assessments for sepsis-associated mortality in children.

In conclusion, we validated six recently available organ dysfunction models for in-hospital mortality in a PICU infection cohort. Compared to the Sepsis-2 score, qSOFA score, SIRS score and PMODS, the pSOFA and PELOD2 score had superior accuracy for predicting hospital death, good precision between predictions and observations and better clinical utility in hospital mortality prediction. Given the relative unavailability of items and the clinical operability of the PELOD2 score, it cannot be considered an optimal tool, although it has a similar predictive performance for mortality as the pSOFA score. Therefore, the pSOFA score should be recommended as an optimal tool for predicting acute organ dysfunction in pediatric patients with sepsis. Elevated lactate levels are related to a greater risk of death from infection in children in the PICU. A combination of both scores should be used to identify organ dysfunction earlier in sepsis patients.

Supporting information

S1 Fig. Distribution of missing data on candidate features for children with suspected or confirmed infections admitted to the PICU in the cohort (n = 5356).

(PDF)

pone.0306172.s001.pdf (398.1KB, pdf)
S2 Fig. Distribution of comorbidities among all children with suspected or confirmed infections admitted to the PICU in the cohort.

Note: The top three underlying diseases were respiratory diseases, hematological diseases and traumatic diseases. Abbreviations: resp, respiratory diseases; homo, hematological diseases; trau, traumatic diseases; heart, heart diseases; nutr, nutritional diseases; immu, autoimmune diseases; neur, neurological diseases; tumo, tumors; inhe, inherited metabolic disorders; CKD, chronic kidney disease.

(PDF)

pone.0306172.s002.pdf (357.9KB, pdf)
S3 Fig. Distribution of the candidate models for all children with suspected or confirmed infections admitted to the PICU in the cohort.

Note: The score distributions of the numbers of encounters are shown for A. pSOFA, B. pSOFAal, C. SIRS, D. PELOD2, E. Sepsis-2, F. qSOFA, G. qSOFAal, and H. PMODS.

(PDF)

pone.0306172.s003.pdf (464.6KB, pdf)
S4 Fig. Subgroup analyses by age group, sex and comorbidities as the primary outcome in children with 2 or more model points versus those with fewer than 2 model points.

Note: Mortality is shown for A. the group aged less than 2 years, B. the group aged 2 years or older, C. the male group, D. the female group, E. the group with comorbidities, F. the group without comorbidities, and G. the total cohort.

(PDF)

pone.0306172.s004.pdf (422.5KB, pdf)
S5 Fig. Bar chart of mortality changes associated with different lactate concentrations.

(PDF)

pone.0306172.s005.pdf (4.5KB, pdf)
S6 Fig. Scatter plot of the pSOFA score vs. lactate level for the survival group and death group according to the linear regression line.

(PDF)

pone.0306172.s006.pdf (2.2MB, pdf)
S7 Fig. DCAs for the candidate models.

A. pSOFA, B. SIRS, C. PELOD2, D. Sepsis-2, E. qSOFA, F. PMODS. Note: DCAs show that the pSOFA and PELOD2 scores have superior net benefits compared to the other five scores.

(PDF)

pone.0306172.s007.pdf (214.9KB, pdf)
S8 Fig. CICs for the candidate models.

A. pSOFA, B. SIRS, C. PELOD2, D. Sepsis-2, E. qSOFA, F. PMODS. Note: The pSOFA and PELOD2 scores are clinically applicable for in-hospital mortality prediction. Abbreviations: CIC, clinical impact curve.

(PDF)

pone.0306172.s008.pdf (230.6KB, pdf)
S9 Fig. Scatter plot of the six scores vs. PCT level.

A. pSOFA vs. PCT, B. SIRS vs. PCT, C. PELOD2 vs. PCT, D. Sepsis-2 vs. PCT, E. qSOFA vs. PCT, F. PMODS vs. PCT.

(PDF)

pone.0306172.s009.pdf (3.6MB, pdf)
S10 Fig. Scatter plot of the six scores vs. CRP level.

A. pSOFA vs. CRP, B. SIRS vs. CRP, C. PELOD2 vs. CRP, D. Sepsis-2 vs. CRP, E. qSOFA vs. CRP, F. PMODS vs. CRP.

(PDF)

pone.0306172.s010.pdf (3.4MB, pdf)
S1 Table. Scoring rules for the candidate models.

(DOCX)

pone.0306172.s011.docx (40.9KB, docx)
S2 Table. Variables of the scoring models and the number of missing values for each variable.

(DOCX)

pone.0306172.s012.docx (17.4KB, docx)
S3 Table. Performance of the pSOFAal vs. pSOFA scores and qSOFAal vs. qSOFA scores.

(DOCX)

pone.0306172.s013.docx (14KB, docx)
S4 Table. Subgroup analyses by age group, sex and comorbidities for endpoint events of the primary outcome in children with 2 or more model points versus those with fewer than 2 model points.

(DOCX)

pone.0306172.s014.docx (16.1KB, docx)
S1 Dataset. Original dataset.

(CSV)

pone.0306172.s015.csv (583.4KB, csv)

Acknowledgments

We thank the members of the Big Data Center for Children’s Medical Care, Children’s Hospital of Chongqing Medical University and Shanghai Synyi Medical Technology Co., Ltd., for providing the database platform.

Data Availability

"All relevant data are within the paper and its supporting information files(S1 Dataset.csv). "

Funding Statement

The study was supported by Chongqing Science and Technology Bureau and Health Commission Joint Medical Project (2021MSXM025), awarded to SL, Chongqing Medical University Graduate Smart Medical Project (YJSZHYX202007), awarded to SL, and Program for Youth Innovation in Future Medicine, Chongqing Medical University, Chongqing 400014, China, awarded to JL. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intens Care Med. 2021;47(11):1181–247. 10.1007/s00134-021-06506-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11. doi: 10.1016/S0140-6736(19)32989-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock(Sepsis-3). Jama-J Am Med Assoc. 2016;315(8):801–10. 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Kawasaki T, Shime N, Straney L, Bellomo R, MacLaren G, Pilcher D, et al. Paediatric sequential organ failure assessment score (pSOFA): a plea for the world-wide collaboration for consensus. Intens Care Med. 2018;44(6):995–7. doi: 10.1007/s00134-018-5188-7 [DOI] [PubMed] [Google Scholar]
  • 5.Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock(Sepsis-3). Jama-J Am Med Assoc. 2016;315(8):762–74. 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Shankar-Hari M, Phillips GS, Levy ML, Seymour CW, Liu VX, Deutschman CS, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama-J Am Med Assoc. 2016;315(8):775–87. doi: 10.1001/jama.2016.0289 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Goldstein B, Giroir B, Randolph A. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Me. 2005;6(1):2–8. doi: 10.1097/01.PCC.0000149131.72248.E6 [DOI] [PubMed] [Google Scholar]
  • 8.Matics TJ, Sanchez-Pinto LN. Adaptation and Validation of a Pediatric Sequential Organ Failure Assessment Score and Evaluation of the Sepsis-3 Definitions in Critically Ill Children. Jama Pediatr. 2017;171(10):e172352. doi: 10.1001/jamapediatrics.2017.2352 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Schlapbach LJ, Straney L, Bellomo R, MacLaren G, Pilcher D. Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit. Intens Care Med. 2018;44(2):179–88. doi: 10.1007/s00134-017-5021-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Leteurtre S, Duhamel A, Salleron J, Grandbastien B, Lacroix J, Leclerc F. PELOD-2: an update of the PEdiatric logistic organ dysfunction score. Crit Care Med. 2013;41(7):1761–73. doi: 10.1097/CCM.0b013e31828a2bbd [DOI] [PubMed] [Google Scholar]
  • 11.Proulx F, Gauthier M, Nadeau D, Lacroix J, Farrell CA. Timing and predictors of death in pediatric patients with multiple organ system failure. Crit Care Med. 1994;22(6):1025–31. doi: 10.1097/00003246-199406000-00023 [DOI] [PubMed] [Google Scholar]
  • 12.Weiss SL, Peters MJ, Alhazzani W, Agus M, Flori HR, Inwald DP, et al. Surviving Sepsis Campaign International Guidelines for the Management of Septic Shock and Sepsis-Associated Organ Dysfunction in Children. Pediatr Crit Care Me. 2020;21(2):e52–106. 10.1097/PCC.0000000000002198 [DOI] [PubMed] [Google Scholar]
  • 13.Parshuram CS, Dryden-Palmer K, Farrell C, Gottesman R, Gray M, Hutchison JS, et al. Effect of a Pediatric Early Warning System on All-Cause Mortality in Hospitalized Pediatric Patients: The EPOCH Randomized Clinical Trial. Jama-J Am Med Assoc. 2018;319(10):1002–12. 10.1001/jama.2018.0948 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Mann KD, Good NM, Fatehi F, Khanna S, Campbell V, Conway R, et al. Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting. J Med Internet Res. 2021;23(9):e28209. doi: 10.2196/28209 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Shime N, Kawasaki T, Nakagawa S. Proposal of a New Pediatric Sequential Organ Failure Assessment Score for Possible Validation. Pediatr Crit Care Me. 2017;18(1):98–9. doi: 10.1097/PCC.0000000000001009 [DOI] [PubMed] [Google Scholar]
  • 16.Schlapbach LJ, MacLaren G, Festa M, Alexander J, Erickson S, Beca J, et al. Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intens Care Med. 2017;43(8):1085–96. 10.1007/s00134-017-4701-8 [DOI] [PubMed] [Google Scholar]
  • 17.Graciano AL, Balko JA, Rahn DS, Ahmad N, Giroir BP. The Pediatric Multiple Organ Dysfunction Score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children. Crit Care Med. 2005;33(7):1484–91. doi: 10.1097/01.ccm.0000170943.23633.47 [DOI] [PubMed] [Google Scholar]
  • 18.Leclerc F, Duhamel A, Leteurtre S, Straney L, Bellomo R, MacLaren G, et al. Which organ dysfunction scores to use in children with infection? Intens Care Med. 2018;44(5):697–8. doi: 10.1007/s00134-018-5123-y [DOI] [PubMed] [Google Scholar]
  • 19.Yang W, Jiang J, Schnellinger EM, Kimmel SE, Guo W. Modified Brier score for evaluating prediction accuracy for binary outcomes. Stat Methods Med Res. 2022;31(12):2287–96. doi: 10.1177/09622802221122391 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Van Calster B, Wynants L, Verbeek J, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators. Eur Urol. 2018;74(6): 796–804. doi: 10.1016/j.eururo.2018.08.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol. 2016;34(21):2534–40. doi: 10.1200/JCO.2015.65.5654 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Macias CG, Remy KE, Barda AJ. Utilizing big data from electronic health records in pediatric clinical care. Pediatr Res. 2022:1–8. 10.1038/s41390-022-02343-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Balamuth F, Scott HF, Weiss SL, Webb M, Chamberlain JM, Bajaj L, et al. Validation of the Pediatric Sequential Organ Failure Assessment Score and Evaluation of Third International Consensus Definitions for Sepsis and Septic Shock Definitions in the Pediatric Emergency Department. Jama Pediatr. 2022;176(7):672–8. doi: 10.1001/jamapediatrics.2022.1301 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Husain AN, Eiden E, Vesoulis ZA. Use of an electronic medical record to optimize a neonatal sepsis score for mortality prediction. J Perinatol. 2022. doi: 10.1038/s41372-022-01573-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Wang X, Guo Z, Zhang X, Zhang G, Li Q, Tian X, et al. Inappropriate empirical antibiotic therapy was an independent risk factor of pediatric persistent S. aureus bloodstream infection. Eur J Pediatr. 2022. doi: 10.1007/s00431-022-04729-9 [DOI] [PubMed] [Google Scholar]
  • 26.Bembea MM, Agus M, Akcan-Arikan A, Alexander P, Basu R, Bennett TD, et al. Pediatric Organ Dysfunction Information Update Mandate (PODIUM) Contemporary Organ Dysfunction Criteria: Executive Summary. Pediatrics. 2022;149(1 Suppl 1):S1-12. doi: 10.1542/peds.2021-052888B [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021. Crit Care Med. 2021;49(11):e1063–143. doi: 10.1097/CCM.0000000000005337 [DOI] [PubMed] [Google Scholar]
  • 28.Slatnick LR, Miller K, Scott HF, Loi M, Esbenshade AJ, Franklin A, et al. Serum lactate is associated with increased illness severity in immunocompromised pediatric hematology oncology patients presenting to the emergency department with fever. Front Oncol. 2022;12:990279. doi: 10.3389/fonc.2022.990279 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Levy MM, Evans LE, Rhodes A. The Surviving Sepsis Campaign Bundle: 2018 update. Intens Care Med. 2018;44(6):925–8. 10.1007/s00134-018-5085-0 [DOI] [PubMed] [Google Scholar]
  • 30.Scott HF, Brou L, Deakyne SJ, Kempe A, Fairclough DL, Bajaj L. Association Between Early Lactate Levels and 30-Day Mortality in Clinically Suspected Sepsis in Children. Jama Pediatr. 2017;171(3):249–55. doi: 10.1001/jamapediatrics.2016.3681 [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Dong Wook Jekarl

4 Jan 2024

PONE-D-23-35521A cohort study validated the performance of organ dysfunction scores in children with infectionPLOS ONE

Dear Dr. Tan,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 18 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dong Wook Jekarl

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Note from Emily Chenette, Editor in Chief of PLOS ONE, and Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS: Did you know that depositing data in a repository is associated with up to a 25% citation advantage (https://doi.org/10.1371/journal.pone.0230416)? If you’ve not already done so, consider depositing your raw data in a repository to ensure your work is read, appreciated and cited by the largest possible audience. You’ll also earn an Accessible Data icon on your published paper if you deposit your data in any participating repository (https://plos.org/open-science/open-data/#accessible-data).

3. Thank you for stating the following financial disclosure:

“The study was supported by Chongqing Science and Technology Bureau and Health Commission Joint Medical Project (2021MSXM025), Chongqing Medical University Graduate Smart Medical Project (YJSZHYX202007), and Program for Youth Innovation in Future Medicine, Chongqing Medical University, Chongqing 400014, China.”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. Please provide a complete Data Availability Statement in the submission form, ensuring you include all necessary access information or a reason for why you are unable to make your data freely accessible. If your research concerns only data provided within your submission, please write "All data are in the manuscript and/or supporting information files" as your Data Availability Statement.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors present a retrospective cohort study on the validity of organ dysfunction scores in children with suspected infection admitted to the pediatric intensive care unit. The authors should be commended for their effort to analyse this large study population but I have several concerns.

Major concerns:

- Recently the PODIUM organ dysfunction scoring system has been proposed (PMID: 34970673). I highly recommend the authors assess the possibility to include this scoring system into their analysis to make their analysis more relevant. At the least this new organ dysfunction scoring system should be discussed in their manuscript to keep the discussion of the relevant literature accurate.

- The authors state they compare 8 organ dysfunction scores, but alternate SOFA, and alternate qSOFA only swap MAP for SBP. The results of this comparison could be presented as supplementary analysis and IMHO do not qualify for inclusion as separate scores. (the results of the analysis could probably be abbreviated that there was no difference in the use of MAP or SBP).

- The authors should also clarify why they included SIRS, which is not an organ dysfunction scoring system and has been removed from the definition of adult sepsis 7 years ago (obviously pediatric definitions are not yet published but will adopt this as well). This is relevant as the authors presumably report on a very sick population (>85% with mechanical ventilation) but only 70% had a SIRS score >=2, which begs the question how relevant this measure could be in their population?

- Discussion: your interpretation of the accuracy, calibration and net benefit of the scores seems a bit euphoric. They can hardly be called excellent and outstanding.

- Judgement of usefulness of PELOD-2 based on the availability of pupillary dilatation in your database is moot. This can be easily measured and is a failure of the design of the database for the analysis presented here

- please expand on the limitations section (e.g. children did not have proven infection, data validity for other settings, etc.)

Minor concerns:

- p3, l3. The estimates for sepsis incidence do not correspond with the numbers given in the cited reference

- p3, l5. Reference 3 cited by the authors only concerns adults with sepsis, it has no relevance whatsoever to the statement of the sentence

- p3, l11 and p4 l5. the use of the word invalids is inappropriate here (see also above)

- p4, l12. typo, this should probably say SAOD? please check all abbreviations for correctness

- p4, l20. I don't think the word multicentre applies here, please remove. The data are from two sites in the same city and affiliated with the same university.

- p4, l20. This is a retrospective study, patients cannot be recruited

- p6, l1. Supplementary Table 2 does not contain the information the authors refer to

- p6, l1-8. This is another example of poor language. The authors didn't develop a score

- p6, l1-8. I suggest the authors provide this information as a cross table (i.e. for each measure (e.g. SBP, row) they should indicate whether it was used in the respective score (columns pSOFA - SIRS) or not and add the number of missing values)

- p6, l21. Can the authors please add references to decision curve analysis and clinical impact curve

- p6, l22 - p7, l2. Can the authors please clarify what they exactly aimed to do with subgroup analysis, given they had done a multivariate regression model.

- p7, l3. Please explain how multiple imputation was done. Also show the rationale for doing so. Most organ dysfunction scores were developed by assuming that missing values indicate that the value was normal (this does obviously does not apply to systematically missing data as the authors had in their database for pupillary dilation).

- p7, l9. this should be suspected infection

- p7, l16. Why is this Suppl. Fig 3? What happened to Suppl. Fig 1 & 2, they are not mentionned.

- p8, l6-10. This information is given in supplementary figure 3. please add reference and shorten the text accordingly.

- p8, l11-15. It would be better to show a graph with mortality by each increase of 1 point of the different scoring systems as a supplementary figure. Then this section can probably be shortened as well.

- p8, l15-17. The authors presumably report on a very sick population, how is it possible that so many children have a SIRS score <2?

- p8, l20 - p9, l11. No need to formulate "odds ratio of 1.59 equalling an increase of 59% in the relative odds". This is what the odds ratio tells us already. please remove the unnecessary repetition of information.

- p9, l19-20. This is somewhat surprising. Can you specify what the added value of the organ dysfunction scores was in comparison of mortality prediction with the baseline model (AUC of the base line model).

- p9, l22. You mention Brier score here, but it's not discussed in the methods. Please add to methods with proper citation.

- p10, l3-4. The presentation of Brier scores here does not correspond to the way it's shown in Figure 3.

- p10, l5. Typo. this should say 5 and not 8.

- p10, l20-22. Lactate is part of PMODS and PELOD-2, how relevant can subgroup analyses be?

- p13, l1-5. How did you arrive at a lactate threshhold of 2?

p15, l4-7. if you want to suggest lactate should be used with pSOFA, why didn't you look at this in your data? too many missing values?

Tables:

Table 1:

- Please carefully check all number! The percentage for mortality by ethnicity is wrong.

- I suggest to give units for all lab values in the row names and clearly indicate which measure is shown (n (%) versus median (iqr) or (mean (sd) for normally distributed values, see comment above)

- please add the number (%) of children receiving inotropes, ecmo or dialysis to help better assess the disease severity in the population (also relevant regarding mortality)

Figures:

Figure 4: Consider showing score values <2 before >= 2, this would be more logical (0, 1, 2 ...)

Supplementary Figure 4: Please revise the range shown in the DCA plots. it likely dosen't make sense to show risks of mortality > 20-30%. Also consider grouping DCA and CIC in separate panels, or even separate figures to allow the figures to breath (this is in the supplementar material anyway). Please also don't include the graph interpretation in the figure legend.

General concerns with the manuscript presentation:

- The english language is quite poor throughout the mansucript. As PlosOne does not offer copyediting of manuscript I suggest the authors employ a professional service to correct their manuscript (e.g. the authors use the word invalids, probably to indicate children or sick patients on PICU, however I do not think this word is appropriate in the context of the manuscript). There are many other issues with the language, therefore the recommendation to seek professional support.

- please do not repeat all information given in tables or figures in the main text. The authors should reduce such unnecessary repetition (e.g. p7, l10-18; p8, l6-10; or p10, l3-l8)

Reviewer #2: The article is quite interesting, comparing various scores for the prognosis of mortality in the PICU. Strengths include the large sample, which led to a normal or approximately normal distribution of the evaluated scores, and the careful and very informative statistics. Another interesting aspect is that the population is Asian, which fills a gap, as these scores are developed and validated in Western populations.

The article needs to be rewritten for the most part, as it presents language problems throughout the text. For example, the use of expressions such as "invalids" to refer to patients often makes the text difficult to understand.

There are also many typographical errors, such as "morality" instead of "mortality" on pg 14, line 6

There are also many typographical errors, such as "morality" instead of "mortality" on pg 14, line 6

The excessive use of abbreviations also makes the text difficult to follow, and this needs to be reviewed

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Orlei Ribeiro de Araujo

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2024 Jul 19;19(7):e0306172. doi: 10.1371/journal.pone.0306172.r002

Author response to Decision Letter 0


10 Feb 2024

Reply Letter

Dear Editors and Reviewers,

On behalf of all the contributing authors, I would like to express our sincere appreciation of your letter and the reviewers’ constructive comments concerning our article titled “Validating the performance of organ dysfunction scores in children with infection: A cohort study” (Manuscript No: PONE-D-23-35521). These comments were all valuable and helpful for improving our article. According to the comments of the associate editor and reviewers, we have made extensive modifications to our manuscript and included additional data to make our results more convincing. In this revised version, changes to our manuscript are highlighted within the document in red text. Point-by-point responses to the comments of the academic editor and two reviewers are listed below.

We would like to thank you for allowing us to resubmit a revised copy of the manuscript, and we greatly appreciate your time and consideration.

Thank you, and best regards.

Yours sincerely,

Liping Tan

E-mail: tanlp0825@hotmail.com

Attachment

Submitted filename: Reply letter_plosone.docx

pone.0306172.s016.docx (73.8KB, docx)

Decision Letter 1

Dong Wook Jekarl

17 Apr 2024

PONE-D-23-35521R1Validating the performance of organ dysfunction scores in children with infection: A cohort studyPLOS ONE

Dear Dr. Tan,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 01 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Dong Wook Jekarl

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: This study is a paper evaluating the performance of the six scoring model using data from a large cohort of pediatric patients. Throughout the paper, there are typing errors and some duplicated sentences. Additionally, there are some points that the authors need to revise.

Despite similar performance between PLEOD2 and SOFA in the conclusion, the reasons why PLEOD2 is not optimal are presented. Please include this information in the abstract (in conclusion section) as well.

Throughout the abstract and the entirety of the manuscript, there is inconsistency in terminologies. Is "age-adapted SOFA" the same as "SOFA" mentioned in the results section? Is this consistent with "paediatric SOFA" as well? Please ensure uniformity and clarity by using one consistent term.

In the abstract, is "severe sepsis" correct, or should it be aligned with the Sepsis-2 criteria or SEVSEPSIS (Table 1) as mentioned in the manuscript? If they differ, please explain the distinction in the introduction. "Sepsis-2" (page 4, line 17) or "sepsis-2" (page 16, line 8)?

Additionally, supplementary Table 1 and 2 incorrectly label "PELOOD2." In the "Lactatemia" section of "PELOOD2," please include the units.(mmol/L?)

Page 3 Ln21 It seems that "age-adapted SOFA" is correct rather than "Adapted SOFA”

Page 5 Ln 19 Systolic Blood Pressure (SBP) should be spelled out before using the abbreviation.

Page 7 Lns12- 21 Please exclude any unnecessary parts from the content of the study cohort. It would be advisable to remove the mention of "Han nationality" from both the table and the main text.

Table 1: It was stated in the method that the worst value on the first day of PICU admission was used for scoring. Is this also the case for the CRP and PCT in Table 1? It would be beneficial to accurately describe this.

Page 9: Lines 11-13 and lines 15-17 contain redundant information. Please review and remove the duplicates.

Page 11 Ln15 It seems that the DCA plot corresponds to Supplementary Figure 7. Please verify and make the necessary correction.

Page 11 The content in lines 19-21 on page 11 overlaps with the content in lines 8-11 on page 10. Editing is required to address this duplication.

Page 12 In this paper, the association between lactate levels and mortality was analyzed. What the analysis results for inflammatory markers such as PCT or CRP are as presented in Table 1? Furthermore, is there any association between these PCT or CRP markers and six scoring models including SOFA? If there is an association, please describe it.

Page 14 Lns 2-5: Please add a reference for lines 2-5 on page 14.

Page 15 Lns 4-7: Please add a reference for lines 4-7 on page 15.

Page 15 Ln13: The sentence at line 13 on page 15 is unnecessary. Please delete it.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2024 Jul 19;19(7):e0306172. doi: 10.1371/journal.pone.0306172.r004

Author response to Decision Letter 1


16 May 2024

Dear Jekarl,

On behalf of all the contributing authors, I would like to express our sincere appreciation for your letter and the reviewers’ constructive comments concerning our article titled “Validating the performance of organ dysfunction scores in children with infection: A cohort study” (Manuscript No: PONE-D-23-35521). The comments were all valuable and helpful for improving our article. According to the comments of the associate editor and reviewers, we have made extensive modifications to our manuscript to make our results more convincing. In this revised version, changes to our manuscript are highlighted in red text. Our point-by-point responses to the comments of the academic editor and two reviewers are listed below. We employed a professional service, and this manuscript has been edited for proper English language, grammar, punctuation, spelling, and overall style by one or more of the highly qualified native English-speaking editors at AJE.

We would like to thank you for allowing us to resubmit a revised copy of the manuscript, and we greatly appreciate your time and consideration.

Thank you, and best regards.

Sincerely,

Liping Tan

E-mail: tanlp0825@hotmail.com

Attachment

Submitted filename: Response to Reviewers.docx

pone.0306172.s017.docx (53KB, docx)

Decision Letter 2

Dong Wook Jekarl

13 Jun 2024

Validating the performance of organ dysfunction scores in children with infection: A cohort study

PONE-D-23-35521R2

Dear Dr. Tan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dong Wook Jekarl

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Dong Wook Jekarl

9 Jul 2024

PONE-D-23-35521R2

PLOS ONE

Dear Dr. Tan,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Dong Wook Jekarl

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Fig. Distribution of missing data on candidate features for children with suspected or confirmed infections admitted to the PICU in the cohort (n = 5356).

    (PDF)

    pone.0306172.s001.pdf (398.1KB, pdf)
    S2 Fig. Distribution of comorbidities among all children with suspected or confirmed infections admitted to the PICU in the cohort.

    Note: The top three underlying diseases were respiratory diseases, hematological diseases and traumatic diseases. Abbreviations: resp, respiratory diseases; homo, hematological diseases; trau, traumatic diseases; heart, heart diseases; nutr, nutritional diseases; immu, autoimmune diseases; neur, neurological diseases; tumo, tumors; inhe, inherited metabolic disorders; CKD, chronic kidney disease.

    (PDF)

    pone.0306172.s002.pdf (357.9KB, pdf)
    S3 Fig. Distribution of the candidate models for all children with suspected or confirmed infections admitted to the PICU in the cohort.

    Note: The score distributions of the numbers of encounters are shown for A. pSOFA, B. pSOFAal, C. SIRS, D. PELOD2, E. Sepsis-2, F. qSOFA, G. qSOFAal, and H. PMODS.

    (PDF)

    pone.0306172.s003.pdf (464.6KB, pdf)
    S4 Fig. Subgroup analyses by age group, sex and comorbidities as the primary outcome in children with 2 or more model points versus those with fewer than 2 model points.

    Note: Mortality is shown for A. the group aged less than 2 years, B. the group aged 2 years or older, C. the male group, D. the female group, E. the group with comorbidities, F. the group without comorbidities, and G. the total cohort.

    (PDF)

    pone.0306172.s004.pdf (422.5KB, pdf)
    S5 Fig. Bar chart of mortality changes associated with different lactate concentrations.

    (PDF)

    pone.0306172.s005.pdf (4.5KB, pdf)
    S6 Fig. Scatter plot of the pSOFA score vs. lactate level for the survival group and death group according to the linear regression line.

    (PDF)

    pone.0306172.s006.pdf (2.2MB, pdf)
    S7 Fig. DCAs for the candidate models.

    A. pSOFA, B. SIRS, C. PELOD2, D. Sepsis-2, E. qSOFA, F. PMODS. Note: DCAs show that the pSOFA and PELOD2 scores have superior net benefits compared to the other five scores.

    (PDF)

    pone.0306172.s007.pdf (214.9KB, pdf)
    S8 Fig. CICs for the candidate models.

    A. pSOFA, B. SIRS, C. PELOD2, D. Sepsis-2, E. qSOFA, F. PMODS. Note: The pSOFA and PELOD2 scores are clinically applicable for in-hospital mortality prediction. Abbreviations: CIC, clinical impact curve.

    (PDF)

    pone.0306172.s008.pdf (230.6KB, pdf)
    S9 Fig. Scatter plot of the six scores vs. PCT level.

    A. pSOFA vs. PCT, B. SIRS vs. PCT, C. PELOD2 vs. PCT, D. Sepsis-2 vs. PCT, E. qSOFA vs. PCT, F. PMODS vs. PCT.

    (PDF)

    pone.0306172.s009.pdf (3.6MB, pdf)
    S10 Fig. Scatter plot of the six scores vs. CRP level.

    A. pSOFA vs. CRP, B. SIRS vs. CRP, C. PELOD2 vs. CRP, D. Sepsis-2 vs. CRP, E. qSOFA vs. CRP, F. PMODS vs. CRP.

    (PDF)

    pone.0306172.s010.pdf (3.4MB, pdf)
    S1 Table. Scoring rules for the candidate models.

    (DOCX)

    pone.0306172.s011.docx (40.9KB, docx)
    S2 Table. Variables of the scoring models and the number of missing values for each variable.

    (DOCX)

    pone.0306172.s012.docx (17.4KB, docx)
    S3 Table. Performance of the pSOFAal vs. pSOFA scores and qSOFAal vs. qSOFA scores.

    (DOCX)

    pone.0306172.s013.docx (14KB, docx)
    S4 Table. Subgroup analyses by age group, sex and comorbidities for endpoint events of the primary outcome in children with 2 or more model points versus those with fewer than 2 model points.

    (DOCX)

    pone.0306172.s014.docx (16.1KB, docx)
    S1 Dataset. Original dataset.

    (CSV)

    pone.0306172.s015.csv (583.4KB, csv)
    Attachment

    Submitted filename: Reply letter_plosone.docx

    pone.0306172.s016.docx (73.8KB, docx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    pone.0306172.s017.docx (53KB, docx)

    Data Availability Statement

    "All relevant data are within the paper and its supporting information files(S1 Dataset.csv). "


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES