Predictors of Septic Shock or Bacteremia in Children Experiencing Febrile Neutropenia Post-Chemotherapy

Abstract

Background

Febrile neutropenia (FN) is an early indicator of infection in oncology patients post-chemotherapy. We aimed to determine clinical predictors of septic shock and/or bacteremia in pediatric cancer patients experiencing FN and to create a model that classifies patients as low-risk for these outcomes.

Methods

This is a retrospective analysis with clinical data of a cohort of pediatric oncology patients admitted during July 2015 to September 2017 with FN. One FN episode per patient was randomly selected. Statistical analyses include distribution analysis, hypothesis testing, and multivariate logistic regression to determine clinical feature association with outcomes.

Results

A total of 865 episodes of FN occurred in 429 subjects. In the 404 sampled episodes that were analyzed, 20.8% experienced outcomes of septic shock and/or bacteremia. Gram-negative bacteria count for 70% of bacteremias. Features with statistically significant influence in predicting these outcomes were hematological malignancy (P < .001), cancer relapse (P = .011), platelet count (P = .004), and age (P = .023). The multivariate logistic regression model achieves AUROC = 0.66 (95% CI 0.56–0.76). The optimal classification threshold achieves sensitivity = 0.96, specificity = 0.33, PPV = 0.40, and NPV = 0.95.

Conclusions

This model, based on simple clinical variables, can be used to identify patients at low-risk of septic shock and/or bacteremia. The model’s NPV of 95% satisfies the priority to avoid discharging patients at high-risk for adverse infection outcomes. The model will require further validation on a prospective population.

This study performs univariate and multivariate analysis using simple clinical variables to identify pediatric oncology patients at low-risk for septic shock and/or bacteremia outcomes.

INTRODUCTION

Febrile neutropenia (FN) is the most frequent complication in pediatric oncology patients receiving chemotherapy. Empirical intravenous treatment with broad-spectrum antibiotics is the standard of care for these patients [1]. Nevertheless, the risk of severe complications such as bacteremia or septic shock is not the same for all the patients, especially in resource constrained environments. The identification of patients at low-risk of complications is relevant to determine which subjects may benefit from receiving a less intense treatment like switching to oral antibiotics for outpatient treatment [2, 3].

Models that predict risk should be as accurate as possible, and also interpretable [4]. Modeling research should be population-sensitive and models may require recalibration as population statistics evolve. Although rules for risk stratification have been proposed, there is a lack of a consensus on which risk prediction rule to apply in order to identify patients who can safely benefit from a less intensive care, like outpatient treatment and oral antibiotics [1]. Receiving treatment outside the hospital will alleviate financial burdens both on families and hospitals while preserving beds for those who require closer attention due to a more acute illness [5, 6]. Recently, researchers have developed instruments to evaluate patients’ risk, such that the patients with a low-risk of septic shock can receive courses of oral antibiotics at home without increasing the risk of complications [7].

The aim of our study was to develop a risk prediction model based on simple clinical variables, that can accurately identify patients at low-risk of severe complications such as septic shock and/or bacteremia in children with FN in Mexico.

METHODOLOGY

Study Design

We performed a retrospective analysis with clinical data of an observational prospective cohort study conducted from July 2015 to September 2017 at Hospital Infantil de México Federico Gómez, which is a public multidisciplinary referral center in Mexico City for children aged less than 18 years of age. The study was approved by the local research committees (protocol number: HIM-2014-026 SSA 1154). Due to no more than minimal risk of the research, IRB waived obtaining informed consent.

Patients

All patients with cancer from 1 to 18 years of age who developed FN in an outpatient setting and were hospitalized for further management were identified. Children with hematopoietic stem cell transplant (HSCT) were excluded. Subjects were followed up daily to document the course of the FN episode. Children were managed as per institutional guidelines with Cefepime and, due to a high rate of resistant pathogens, Amikacin was added for 3 days as a second Gram-negative agent. If subjects presented typhlitis, Piperacillin/Tazobactam were initiated. Meropenem plus Vancomycin were used in septic shock. If fever persisted after 96 h of broad-spectrum antibiotics, empirical antifungal therapy was initiated with Amphotericin B lipid complex. Children were monitored from their arrival to the emergency department until the resolution of FN episode and discontinuation of antibiotics.

Variables

Fever was defined as an oral temperature >38.3°C, or 2 consecutive measurements of >38.0°C. Neutropenia was defined as an absolute neutrophil count (ANC) <500 cells/mm³. Standardized data collection form was used to prospectively document the following variables: sex, age at admission, cancer type, chemotherapy phase, days since last chemotherapy, cancer status, prophylaxis, vital signs, maximum temperature, laboratory tests at admission [hemoglobin, platelets, absolute neutrophil count (ANC), absolute monocyte count (AMC)] and results of blood cultures collected at admission. Primary health outcomes of interest (HOIs) were: Bacteremia, a recognized pathogen (including organisms associated with mucosal barrier injury in the setting of mucositis or neutropenia) from >1 blood culture or common commensals from >2 blood culture drawn on separate occasions; and septic shock, a severe infection leading to cardiovascular dysfunction (including hypotension, need for treatment with a vasoactive medication, or impaired perfusion) [8]. HOIs were collected at the end of the FN episode.

Statistical Analyses

One ambulatory episode of FN per patient was randomly selected to avoid bias. We performed Mann–Whitney and Fisher tests during the univariate analysis stage. A two-sided Mann–Whitney test was conducted with respect to each continuous feature between the group experiencing the HOI vs. the group who did not experience the HOI to determine if there exists a statistically significant difference between the respective distributions. For each binary feature, we performed Fisher tests to measure the association between the risk feature vs. the occurrence of the HOI. We define statistical significance at the ɑ = 0.05 level throughout the analyses.

In the multivariate analysis stage, we fit a multivariate logistic regression model to predict the occurrence of the HOI in individual patients using the clinical risk features. The logistic regression model was trained on 80% of the sample using the limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) solver with a maximum of 5000 iterations, and the remaining 20% of the sample was reserved for testing. The performance of the model was evaluated using the area under the receiver operating characteristic curve (AUROC) measure. The optimal classification threshold from the ROC curve was calculated using Youden’s J Index, resulting in sensitivity, specificity, PPV, and NPV measures for the test set. All analyses were performed using statsmodels python package version 0.12.0.

RESULTS

A total of 865 episodes of FN in 429 patients were recorded over the course of 27 months. Of the 865 FN episodes, 792 (91.6%) began in the ambulatory setting. Of these subjects 225 (50.7%) had one FN episode, 96 (21.6%) had two episodes, 49 (11.1%) had three FN episodes and 59 (13.3%) had 4 or more FN episodes. Median number of FN episodes among individual subjects was 1 [interquartile range (IQR), 1–3]. One hundred thirty-three patients (31%) experienced at least one FN episode with an outcome of septic shock and/or bacteremia. One hundred nine (82%) patients experienced at least 1 FN episode with septic shock; 52 (39%) experienced at least 1 FN episode with bacteremia; and 17 (12.8%) experienced at least 1 FN episode with both outcomes in the same episode.

Patient characteristics associated with the random sample of 404 FN episodes that began in the ambulatory setting are shown in Table 1. The leading cancer type was acute lymphoblastic leukemia (ALL) in 45.3%, and every other cancer subtype occurred in less than 9% of subjects respectively. Each patient received his or her last chemotherapy treatment between 1 and 48 days (median 9, IQR, 6–12 days) prior. The patients experienced fever for 1–16 days (median 1 day, IQR, 1–2 days), and of the 350 patients (86.6%) who recovered from neutropenia during the hospital stay, the duration of neutropenia ranged from 2 to 33 days (median 7 days, IQR, 5–9 days). Length of hospital stay ranged from 2 to 81 days (median 9, IQR, 7–12 days); patients experiencing septic shock and/or bacteremia stayed at the hospital between 6 and 81 days (median 12 days, IQR, 9–17 days), whereas patients who experienced neither outcome stayed between 2 and 47 days (median 9 days, IQR, 7–11 days) (P = < .001). Of the 404 randomly sampled episodes that were analyzed, 84 (20.8%) experienced outcomes of septic shock and/or bacteremia. Of these 84 patients, 56 (66.7%) experienced septic shock without bacteremia; 19 (22.6%) experienced bacteremia without septic shock; and 9 (10.7%) experienced both septic shock and bacteremia within the same episode. Table 2 shows a summary of microorganisms detected in the subject’s blood cultures.

Table 1.

Summary of demographics of 404 febrile neutropenia episodes

Subject characteristic	n = 429
Sex = male	197 (48.8%)
Age in years (median, IQR)	7.7, 4.4-11.7
Days since recent chemotherapy treatment (Median, IQR)	9.0, 6.0-12.0
Cancer type
Hematological malignancies	241 (59.7%)
Acute lymphoblastic leukemia	183 (45.3%)
Acute myeloid leukemia	29 (7.2%)
Non-Hodgkin’s lymphoma	19 (4.7%)
Hodgkin’s lymphoma	10 (2.5%)
Solid tumors	163 (40.3%)
Rhabdomyosarcoma	35 (8.7%)
Osteosarcoma	17 (4.2%)
Ewing sarcoma	17 (4.2%)
High-grade cerebral tumor (III–IV)	17 (4.2%)
Neuroblastoma	15 (3.7%)
Retinoblastoma	15 (3.7%)
Hepatoblastoma	12 (3.0%)
Other	10 (2.5%)
Germ cell/Gonadal neoplasm	9 (2.2%)
Other sarcoma	7 (1.7%)
Low-grade cerebral tumor (I–II)	5 (1.2%)
Wilms tumor	4 (1.0%)
Remission	125 (30.9%)
Relapse	72 (17.8%)
Chemotherapy phase of acute leukemias
Not applicable*	182 (45.0%)
Maintainment	120 (29.7%)
Induction	62 (15.3%)
Consolidation	36 (8.9%)
Relapse	4 (1.0%)
Growth stimulating factor	135 (33.4%)

IQR, inter quartile range; FN, febrile neutropenia.

Not applicable as they were patients without leukemia diagnosis.

Table 2.

Microorganisms isolated from the subject’s blood cultures

Microorganism	n (%)
Gram-negative
Escherichia coli	7 (25)
Pseudomonas aeruginosa	5 (17.85)
Klebsiella pneumoniae	5 (17.85)
Pseudomonas putida	2 (7.14)
Salmonella spp.	1 (3.57)
Gram-positive
Viridans-group Streptococci	5 (17.85)
Staphylococcus aureus	2 (7.14)
Coagulase-Negative Staphylococci	1 (3.57)
Total	28 (100)

Bivariate Associations with Health Outcomes of Interest

Differences in distributions of continuous predictor variables between the population who experienced the HOI vs. the population who did not are shown in Table 3. The predictor variables with a statistically significant difference in distribution were platelet count, leukocyte count, AMC, and age (years). There was a statistically significant association between the HOI and the binary variables cancer relapse (P = .006) and hematological malignancy (P = .017); 24 (33.3%) out of 72 patients with cancer relapse developed the HOI, as compared to 60 (18.1%) out of 332 patients without cancer relapse; similarly, 60 (24.9%) out of 241 patients with a hematological malignancy developed the HOI, as compared to 24 (14.7%) out of 163 patients without a hematological malignancy. Meanwhile no significant association was found with patient sex (P = .902) and cancer remission status (P = .144).

Table 3.

Descriptive statistics and Mann–Whitney test results for continuous predictors vs. HOI (n = 404), with bolded statistically significant P-values (< .05)

Continuous feature	Median, IQR Total sample (n = 404)	Median, IQR Positive HOI (n = 81)	Median, IQR Negative HOI (n = 323)	Mann–Whitney P-value
Age (years)	7.52, 4.37–11.67	10.08, 5.39–13.19	7.34, 4.17–10.79	.002
Days since recent chemotherapy	9, 6–12	9, 5–12	9, 6–12	.352
Leukocytes	600, 300–1300	400, 200–900	700, 300–1400	< .001
Hemoglobin	9.8, 8.1–11.2	9.4, 7.8–11.0	9.8, 8.2–11.2	.177
ANC	52, 21–130	49.5, 18–112	55.0, 22.0–132.8	.171
AMC	55, 14–210	31, 6–103.5	60, 17.0–248.5	< .001
Platelets	53,500, 15,700–138,250	32,000, 8,750–62,000	70,000, 17,000–168,250	<0.001

HOI, health outcomes of interest; IQR, inter quartile range.

Logistic Regression to Predict Bacteremia and/or Septic Shock

Using the clinical features of each patient in the training set (n = 323 patient episodes), we fit a multivariate logistic regression model to predict the outcome of septic shock and/or bacteremia. The coefficient and odds ratio information associated with the model are shown in Table 4. The features with statistically significant (P < .05), non-zero coefficients in order of magnitude were hematological malignancy, platelet count, cancer relapse, and age. Hemoglobin level possessed a near-significant non-zero coefficient (P = .09).

Table 4.

Logistic regression model parameter results, trained on n = 323 patients, listed in decreasing order of coefficient value

Feature	Coefficient	P	Odds ratio	Odds ratio 95% CI<?Char=Equal?>
Hematological malignancy	1.423100	<.001	4.150	1.933, 8.917
Relapse	0.999200	.011	2.716	1.261, 5.847
Hemoglobin	0.119400	.090	1.127	0.981, 1.293
Age (years)	0.079300	.023	1.083	1.011, 1.160
Platelets	‐0.000006	.004	1.000	1.000, 1.000
ANC	‐0.000099	.951	1.000	0.997, 1.003
AMC	‐0.000100	.875	1.000	0.998, 1.002
Total leukocytes	‐0.000300	.371	1.000	0.999, 1.000
Days since last chemotherapy	‐0.026800	.456	0.974	0.908, 1.045

AMC, absolute monocyte count; ANC, absolute neutrophil count.

The final logistic regression model achieves an AUROC of 0.66 (95% CI 0.56–0.76), as compared to a random classifier baseline with AUROC of 0.50 (95% CI 0.44–0.56), shown in Figure 1. We computed the optimal probability classification cut-point threshold of 0.076 from the ROC curve using Youden’s J Index, which yields 25 true positives, 18 true negatives, 37 false positives, and only 1 false-negative out of the 81 subjects included in the testing set. This produces 96% sensitivity, 33% specificity, 40% PPV, and 95% NPV. The misclassified subject who developed the HOI (one false-negative), experienced neither cancer relapse nor a hematological malignancy; this patient was an infant with rhabdomyosarcoma who developed septic shock which resolved uneventfully with 7 days of antibiotics.

Figure 1.

Receiver operating characteristic (ROC) curve for occurrence of HOI on the testing set (n = 81 patient episodes). AUC = 0.66 (95% CI 0.56–0.76). Point A corresponds to the optimal classification threshold (model probability output < 0.076) per Youden’s J Index, where the model achieves sensitivity = 0.96, specificity = 0.33, PPV = 0.40, and NPV = 0.95.

DISCUSSION

This study identified simple, accessible features presented at child admission that can be used to identify patients at risk of experiencing an adverse outcome related to infectious complications in the setting of FN. Our measures of risk included two of the principal complications associated with invasive bacterial infection: septic shock and/or bacteremia, that can preclude the decision of offering the patient and outpatient treatment.

Several prediction rules had been designed to stratify patients into high-risk or low-risk of severe infections complications, but the majority of them had used different high-risk outcomes, as an international consensus for the most relevant outcome of bacterial infection is still missing. Some authors had developed prediction rules based on adverse outcomes [9–11], invasive bacterial infection [12, 13], microbiologically defined infection [14] or bacteremia [9, 15–17]. We focus our study on septic shock and/or bacteremia as our outcomes of interest, since both have standardized definitions used in the majority of clinical studies, and because they objectively represent the most serious infectious complications of patients with cancer and FN. Our finding of 33.3% prevalence of bacteremia (22.6% bacteremias plus 10.7% bacteremias with septic shock) with a noticeable predominance of Gram-negative bacteria, is greater than previously reported studies [15–18]; this may reflect differences in handling practices between high-income and middle-income countries. There is a low use rate of indwelling catheters in our institution which may reflect the small proportion of Gram-positive bacteremia.

Prior studies have explored risk factors for severe bacterial complications including disease-related factors (hematological malignancies, requirement of intensive chemotherapy), patient-related factors (time from last chemotherapy <7 days, previous history of FN), clinical characteristics (hypotension, tachycardia, tachypnea, fever <39°C, malnutrition) and laboratory exams (Neutropenia severity and duration, monocyte count <100/mm³, hemoglobin level <7 g/dL, platelet count <50,000/mm³, C reactive protein >90 mg/L) [9, 10, 13, 19]. Some of these factors are consistent with several risk predictors identified in our analyses. Our risk prediction model of septic shock and/or bacteremia is based on nine risk features, of which hematological malignancy, cancer relapse, age and platelet count were statistically significant. The final model had modest discriminatory performance (AUROC 0.66) with sensitivity of 96% and specificity of 33%. Nevertheless the sensitivity of our prediction model was satisfactory, compared with sensitivity between 85 and 100% reported in preceding prediction models [10, 15, 16, 18], our 33% specificity is comparable to the 16–58% specificity in the above-mentioned rules, which demonstrates an improvement in identifying truly low-risk patients. We considered it important to create a model that confidently allows us to identify low-risk patients, especially in middle-income countries where optimal selection of candidates for an outpatient treatment is crucial, as accessibility or proximity to an emergency service is not always the optimal. The model’s NPV performance of 95% demonstrates a low false-negative rate, which satisfies the priority to avoid discharging patients at high-risk for adverse infection outcomes.

The presence of hematological malignancy or cancer relapse were identified, in our study, as the most relevant independent variables associated with bacteremia and/or septic shock. These conditions may comprise the use of myeloablative chemotherapy, which should be considered when evaluating the risk of HOI in this group of patients. Some predictive rules have considered comorbidities requiring hospitalization, clinical assessment, and other reasons for inpatient observations risk factors for bacterial complications [13, 14, 17, 18]. Although there is evidence that medical clinical impression is important, this may vary depending on the treating physician‘s evaluation, which may limit the utility of this variables in a prediction model.

Consistent with previous reports, a low platelet count, which is used as a surrogate for the degree of marrow suppression and increased consumption in sepsis, is confirmed in our study as an independent predictive factor of an HOI. Thrombocytopenia has been identified by other authors as a variable of risk for bacteremia [16, 17] or severe invasive bacterial infection [9, 12, 18]. Additional studies have found that an AMC >100/mm³ or >155/mm³ has been associated with a low risk of bacteremia [15, 16]. In our study, the difference in AMC distribution between patients with septic shock and/or bacteremia and patients without either HOI was statistically significant, exclusively in the univariate analysis.

In this study, we propose an easy-to-apply prediction model, based exclusively on patient disease features and a complete blood count, mainly for Latin American countries. We did not include in our analyses C-reactive protein, as there could be institutions with limited hours to request this study, and because there have been authors that did not associated serum concentrations of C-reactive protein with a risk of bacteremia [17, 20]. Although there are two other risk stratification strategies carried out in Latin America [12, 19], the differences between populations, including limitations on availability of relevant data, did not allow us to empirically evaluate any of these published scales. As such, we analyzed the characteristics of FN episodes in our population and developed a risk prediction model within the context of our population. However, the results of our study should be prospectively evaluated in our population, and others, before adopting its routine application.

This study has several limitations. First, as with other retrospective studies, it can overestimate the predictive performance of the model. Second, other studies had explored the performance of their models at admission but also performing a reassessment after an initial period of inpatient observation, which have shown to increase the possibility of correctly categorized patients as high-risk or low-risk[6, 13, 17] In our study, we did not have the information to perform a belated evaluation of the model. Third, fever was recorded without specifying if it was measured at admission, due to this, we do not feel confident in analyzing this parameter as a risk variable presented upon admission of the patient, restricting the inclusion of this characteristic in the model.

Despite many attempts and considerable progress in developing a prediction model for bacteremia or bacterial complications, there is insufficient evidence to establish a universal predictor model. Furthermore, the scarce evidence in middle-income countries forces the need to continue performing research in the field. While other studies have accomplished similar results [9, 12, 13, 15, 16, 18, 19], to our knowledge, this study serves as the richest analysis of its kind within the Mexican context. Our study contributes, demonstrating that simple clinical variables can be used to identify patients at low-risk of septic shock and/or bacteremia. The results of this study should be prospectively validated before routine application.

Notes

Funding. This work was supported in part by the National Council of Science and Technology of Mexico (Consejo Nacional de Ciencia y Tecnología de México - CONACyT) [grant number 233555], and the Lucy Family Institute for Data and Society, University of Notre Dame, Notre Dame, IN, United States of America.

Conflicts of interest. The authors declare no conflicts of interest.