Prospective External Validation of the Esbenshade Vanderbilt Models Accurately Predicts Bloodstream Infection Risk in Febrile Non-Neutropenic Children With Cancer

Abstract

PURPOSE

The optimal management of fever without severe neutropenia (absolute neutrophil count [ANC] ≥500/µL) in pediatric patients with cancer is undefined. The previously proposed Esbenshade Vanderbilt (EsVan) models accurately predict bacterial bloodstream infections (BSIs) in this population and provide risk stratification to aid management, but have lacked prospective external validation.

MATERIALS AND METHODS

Episodes of fever with a central venous catheter and ANC ≥500/µL occurring in pediatric patients with cancer were prospectively collected from 18 academic medical centers. Variables included in the EsVan models and 7-day clinical outcomes were collected. Five versions of the EsVan models were applied to the data with calculation of C-statistics for both overall BSI rate and high-risk organism BSI (gram-negative and Staphylococcus aureus BSI), as well as model calibration.

RESULTS

In 2,565 evaluable episodes, the BSI rate was 4.7% (N = 120). Complications for the whole cohort were rare, with 1.1% (N = 27) needing intensive care unit (ICU) care by 7 days, and the all-cause mortality rate was 0.2% (N = 5), with only one potential infection-related death. C-statistics ranged from 0.775 to 0.789 for predicting overall BSI, with improved accuracy in predicting high-risk organism BSI (C-statistic 0.800-0.819). Initial empiric antibiotics were withheld in 14.9% of episodes, with no deaths or ICU admissions attributable to not receiving empiric antibiotics.

CONCLUSION

The EsVan models, especially EsVan2b, perform very well prospectively across multiple academic medical centers and accurately stratify risk of BSI in episodes of non-neutropenic fever in pediatric patients with cancer. Implementation of routine screening with risk-stratified management for non-neutropenic fever in pediatric patients with cancer could safely reduce unnecessary antibiotic use.

INTRODUCTION

Fever in children and adolescents receiving therapy for cancer is a common and important complication.¹ High-quality evidence-based clinical practice guidelines for management of fever with severe neutropenia (absolute neutrophil count [ANC] <500/µL) are well established and recommend use of a risk-stratification system to guide management.¹ By contrast, there is no consensus for optimal management of fever in patients without severe neutropenia (ANC ≥500/µL), and risk stratification is less delineated.¹ At many pediatric oncology sites, patients with non-neutropenic fever without signs of severe sepsis are routinely given empiric antibiotics regardless of risk assessment.^2-5 The Esbenshade Vanderbilt (EsVan) models use available clinical and laboratory data to accurately predict the risk of bacterial bloodstream infections (BSIs).² These models have been retrospectively externally validated.^5,6 Using risk stratification for these patient events may limit unnecessary antibiotic use, reduce antibiotic resistance, lessen the gut microbiome impact, and reduce health care costs.^7,8 The study sought to evaluate the generalizability and assess the safety of implementing EsVan models at 18 international sites (United States [n = 16], Australia [n = 1], and Lebanon [n = 1]). We aimed to (1) externally validate the prediction model across multiple independent and heterogeneous patient groups, and (2) report safety outcomes.

CONTEXT

• Key Objective

• To evaluate if the previously proposed Esbenshade Vanderbilt (EsVan) models accurately predict bacterial bloodstream infections (BSIs) in pediatric patients with cancer presenting with non-neutropenic fever and could they be used for risk stratified management.

• Knowledge Generated

• Among 2,565 prospectively collected episodes across 18 pediatric academic sites, the EsVan model accurately predicted BSI with C-statistics ranging from 0.775 to 0.789. Initial empiric antibiotics were withheld in 14.9% of episodes, with no deaths or intensive care unit admissions attributable to not receiving empiric antibiotics.

• Relevance (S. Bhatia)

• Implementation of a risk-based approach to management of non-neutropenic fevers could potentially mitigate the downstream consequences of unnecessary antibiotic administration.*

  *Relevance section written by JCO Associate Editor Smita Bhatia, MD, MPH, FASCO.

MATERIALS AND METHODS

EsVan Models

Since the publication of the initial EsVan model in 2015,² a series of modifications have been developed to implement the model in a variety of clinical settings. These models use clinical and laboratory variables that are available at or within 2 hours of presentations to a medical center. Details of these models have been previously described and are shown in Table 1,^6,9 and detailed formulas for these models are shown in the Data Supplement (Figs S1A-S1E, online only). For this study, the primary model we sought to validate was the EsVan2b model as it uses readily available patient data and has been prospectively shown to reliably predict BSI in clinical practice.⁹

TABLE 1.

Summary of EsVan Model Development, Validation, and Implementation

Model	EsVan	EsVan2aa	EsVan2bb	EsVan3ac	EsVan3bc
Year published, journal	2015, PBC	2017, Cancer	2017, Cancer	2020, JCO	2020, JCO
Development and internal validation cohort (setting)	RVC (retrospective)	RVC (retrospective)	RVC (retrospective)	RVC (retrospective)	PVC (prospective)
External validation cohort (setting)	FSC (retrospective) PVC (prospective)	FSC (retrospective) PVC (prospective)	FSC (retrospective) PVC (prospective)	FSC (retrospective) PVC (prospective)	FSC (retrospective) RVC (retrospective)
Clinical implementation period	April 2015-September 2017	October 2017-present	October 2017-present
Variables included in the models	Patient location at fever presentation Type of central venous catheter Hypotension Shaking chills ALL cancer diagnosis Stem-cell transplantation Upper respiratory infection symptoms Exposure to chemotherapy within 24 hours of fever presentation that causes fever Age, years Maximum fever ANC AMC	Type of central venous catheter Hypotension Shaking chills Stem-cell transplantation Upper respiratory infection symptoms Exposure to chemotherapy within 24 hours of fever presentation that causes fever Age, years Maximum fever ANC AMC	Type of central venous catheter Hypotension Shaking chills Stem-cell transplantation Upper respiratory infection symptoms Exposure to chemotherapy within 24 hours of fever presentation that causes fever Age, years Maximum fever ANC	Type of central venous catheter Hypotension Shaking chills Stem-cell transplantation Exposure to chemotherapy within 24 hours of fever presentation that causes fever Age, years Maximum fever	Type of central venous catheter Hypotension Shaking chills Stem-cell transplantation Exposure to chemotherapy within 24 hours of fever presentation that causes fever Age, years Maximum fever

Abbreviations: AMC, absolute monocyte count; ANC, absolute neutrophil count; BSI, bloodstream infection; EsVan, Esbenshade Vanderbilt; FSC, five-study cohort; PBC, Pediatric Blood & Cancer; PVC, prospective Vanderbilt cohort; RVC, retrospective Vanderbilt cohort.

^a Location and ALL cancer diagnosis: Less clinically reliable for BSI.

^b AMC- Not readily available at patient presentation.

^c ANC- With development in technology, ANC change may no longer be predictive for BSI; Upper respiratory symptoms- May be caused by other noninfectious causes, thus introduce more noise) Hypotension- Variable was classified in the opinion of the treating provider.

Study Cohort

Each site obtained local institutional review board approval and agreements were reached to collect data from the electronic medical record and to share deidentified data with the primary site (Vanderbilt). Each site prospectively monitored and identified new episodes of non-neutropenic (ANC ≥500/µL) fever occurring in pediatric patients (age 0-25 years) with cancer or histiocytosis with a central venous catheter in place during active therapy. Episodes were excluded if they occurred within 7 days of a previous fever episode, or if they occurred any time after an allogeneic stem-cell transplant or within 30 days after autologous transplant following the previously used eligibility criteria.^2,6,9 Episodes were prospectively captured and model-relevant variables were entered within a week of the episode into a web-based module (RiskPrediction¹⁰), which gives a real-time prediction of BSI risk. The data were saved using a one-way application programming interface directly into a site-specific REDcap database. Sites then collected outcome data for 7 days after febrile presentation. Management decisions including empiric antibiotic administration were according to standard of care at each institution. Four sites (D, G, J, and L), although not directly using the model results to guide decisions, routinely did not administer empiric antibiotics to well-appearing patients, while remaining sites routinely administer empiric antibiotics to all patients. Additional site-specific methods for each of the 18 participating sites are provided in the Data Supplement (Table S1).

Definitions

Fever was defined as temperature of 38.0°C sustained for ≥1 hour or 38.3°C once, as described previously.^2,6,9 Per IDSA guidelines, positive blood cultures were classified as true BSI if a common commensal organism (such as coagulase-negative Staphylococcus [CONS]) was isolated from two discrete cultures or another known pathogenic organism was isolated from a single culture.^11,12 As previously described, likely contaminants were classified as non-BSI.^2,6,9 High-risk organism BSI, as detailed in our previous works, was defined by the growth of any gram-negative or Staphylococcus aureus organism, as these organisms are associated with a high risk of complications.^2,6,9 Resulting EsVan predictions were then classified as per previous stratifications into low risk (predicted BSI risk <10%), intermediate risk (predicted BSI risk ≥10% to <40%), and high risk (predicted BSI risk ≥40%).⁹

Outcomes

The EsVan models used BSI as the primary outcome and were also applied separately for high-risk organism BSI. In addition, to evaluate the safety of withholding empiric antibiotics, we also captured mortality, need for intensive care unit (ICU) admission, and for those not initially admitted or who were discharged within 7 days, readmission by 7 days after presentation.

Statistical Analysis

Model validation followed the same approach described in our previous reports.^2,6,9 Detailed analytical approaches are reported in the Data Supplement (Methods S1). Briefly, patient demographics, clinical variables, and laboratory variables were summarized and compared between the current study cohort and the previous model development or validation cohorts. To evaluate the performance of the prediction model in this independent validation cohort, model discrimination and calibration were assessed by the area under the receiver operator characteristic curve (C-statistic) and calibration plot, respectively. The 95% CIs were estimated by using a 300-iteration bootstrap approach. Additionally, a benchmark C-statistic was calculated by refitting the variables for each version of the EsVan model to the new data set to evaluate the best possible C-statistic for the data set. Statistical significance was considered at a two-sided 5% level. All statistical analyses were performed using R software version 4.2 (R; Vienna, Austria).

RESULTS

Patient Characteristics

A total of 2,565 episodes across 18 study sites were included. The median number of evaluable episodes at each site was 97 (range, 11-720), and the median BSI incidence rate for a site was 4.4% (range, 0%-10.9%; Data Supplement, Table S2). Descriptions of the organisms isolated are provided in the Data Supplement (Table S3). Table 2 shows that the characteristics of the fever episodes were similar to the previously reported prospective Vanderbilt cohort, including the BSI rate (4.7% v 4.2%), high-risk organism BSI rate (3.2% v 2.8%), central line type (port-a-cath 82.9% v 82.0%), chills or shaking rigors (4.9% v 5%), exposure to chemotherapy agents that cause fever (10.5% v 12%), and history of autologous stem-cell transplant (5.5% v 7%). However, at presentation, participants in this study were older (median age, 6 years [IQR, 3.4-11] v 6 [3-10]; P < .01), had higher median temperature at presentation (38.6° v 38.5°; P < .01), more frequently had acute lymphoblastic leukemia (49.7% v 43%; P < .01), were more often hypotensive (3.4% v 1.2%; P < .01), were less likely to present outpatient (80.6 v 87%; P = .05) or have upper respiratory infection URI symptoms (30.6% v 40%; P < .01), and had lower median ANC (3,240 v 3,740; P < .01) and absolute monocyte count values (350 v 500; P < .01). Patients with high-risk organism BSI versus non–high-risk organism BSI were significantly more likely to have a higher presenting fever (median 39.0° v 38.6°; P = .005), chills (25.6% v 10.0%; P = .019), and numerically more likely to have hypotension (17.1% v 13.3%; P = .543).

TABLE 2.

Patient Characteristics by Study Cohort

Characteristic	International Validation Cohort (N = 2,565)	Retrospective Vanderbilt Cohort		Five-Study Cohort		Prospective Vanderbilt Cohort
		N = 932	P a	N = 1,196	P a	N = 937	P a
Positive bloodstream infection, No. (%)	120 (4.7)	91 (9.8)	<.01	61 (5.1)	.57	39 (4.2)	.52
Positive high-risk bloodstream infection, No. (%)	82 (3.2)	54 (5.8)	<.01	27 (2.3)	.11	26 (2.8)	.52
Age at the time of episode, yearsb	6.0 (3.4-11.0)	5.0 (3.0-10.0)	<.01	6.0 (3.0-12.0)	.13	6.0 (3.00-10.00)	<.01
Maximum temperature at presentationb	38.6 (38.3-39.0)	38.7 (38.3-39.1)	.46	38.6 (38.3-39.1)	.73	38.5 (38.2-38.9)	<.01
Absolute neutrophil countb	3,240 (1,700-5,800)	3,360 (1,858-5,735)	.20	3,240 (1,658-5,773)	.65	3,760 (2,042-6,150)	<.01
Absolute monocyte countb	350 (159-644)	460 (220-820)	<.01	400 (170-730)	<.01	500 (280-870)	<.01
Type of central venous line, No. (%)			<.01		<.01		.63
Port-a-cath	2,127 (82.9)	665 (71.4)		893 (74.7)		769 (82)
PICC line	85 (3.3)	33 (3.5)		93 (7.8)		37 (4)
External tunneled catheter (CVC)	353 (13.8)	234 (25.1)		211 (17.6)		132 (14)
Reports/observed chills or shaking rigors, No. (%)	125 (4.9)	110 (11.8)	<.01	94 (7.9)	<.01	49 (5)	.67
Hypotension, No. (%)	86 (3.4)	16 (1.7)	.01	52 (4.3)	.13	11 (1.2)	<.01
Chemotherapy exposure that causes feverc, No. (%)	269 (10.5)	94 (10.0)	.73	149 (12.0)	.07	112 (12.0)	.22
Upper respiratory infection symptomsd, No. (%)	786 (30.6)	419 (45.0)	<.01	409 (34.2)	.03	375 (40.0)	<.01
History of stem-cell transplant, No. (%)	141 (5.5)	66 (7.1)	.08	165 (13.8)	<.01	66 (7.0)	.09
Outpatient at presentation, No. (%)	2,068 (80.6)	750 (80.5)	.92	828 (69.3)	<.01	728 (87)	.05
Acute lymphoblastic leukemia diagnosis, No. (%)	1,274 (49.7)	446 (47.9)	.34	504 (42.1)	<.01	403 (43)	<.01

Abbreviations: CVC, central venous catheter; PICC, peripherally inserted central catheter.

^a Comparing the International Validation Cohort to the corresponding cohort use in previous studies.

^b For continuous variables, median (the 25th quartile-75th quartile) are presented. For categorical variables, percentages (frequencies) are presented.

^c Exposure to cytarabine, neuroblastoma antibody therapy (anti-GD2), or ATG within 24 hours of presentation.

^d Cough, rhinorrhea, and/or congestion.

Discrimination, Calibration, and Prediction

When we applied (with fixed model coefficients) the EsVan models to this international multisite cohort, the discrimination for BSI was robust. The models achieved C-statistics of 0.775, 0.781, 0.789, 0.773, and 0.781 for EsVan through EsVan3b, respectively (Table 3; Fig 1B; Data Supplement, Fig S2B), which are similar to the benchmark C-statistics estimated by refitting the model using the international multisite cohort (0.809, 0.809, 0.809, 0.798, and 0.798, respectively; Table 3).

TABLE 3.

Summary of Performance/Discrimination (C-statistic) on Study Cohorts

Modela	International Validation Cohort (N = 2,565)	RVC (N = 932)	Five-Study Cohort (N = 1,196)	Prospective Vanderbilt Cohort (N = 938)
EsVan	0.775 (0.809)b	0.898 (0.885)c	0.687	0.805
EsVan2	0.781 (0.809)b	0.895 (0.885)c	0.721	0.825
EsVan2b	0.789 (0.809)b	0.891 (0.881)c	0.723	0.820
EsVan3	0.773 (0.798)b	0.885 (0.878)c	0.724	0.819
EsVan3b	0.781 (0.798)b	0.876	0.720	0.840 (0.823)c

Abbreviations: EsVan, Esbenshade Vanderbilt; PVC, prospective Vanderbilt cohort; RVC, retrospective Vanderbilt cohort.

^a Models EsVan, EsVan2, EsVan2b, and EsVan3 were developed on RVC, and validated on other cohorts; model EsVan3b was developed on PVC and validated on other cohorts.

^b Benchmark C-statistic: Estimated by refitting the model using the International Validation Cohort.

^c Corrected C-statistic from a 300-iteration bootstrap method.

FIG 1.

(A) Calibration plot and (B) receiver operating curve of the EsVan2b model applied to the validation cohort. BSI, bloodstream infection; EsVan, Esbenshade Vanderbilt; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operator characteristic.

Among episodes at lower predicted risk of BSI (<10%, left tail of the curve), representing 87% of all episodes, the calibration curves are close to the 45° line, indicating no evidence of overfitting. However, the EsVan model tended to predict somewhat higher BSI risk than actually observed among those at intermediate/high risk (≥10%), indicated by the right tail of the calibration curves falling below the 45° line. The overall slopes for the calibration curves range from 0.77 to 0.78 for EsVan through EsVan3b, respectively (Fig 1A; Data Supplement, Fig S2A).

In this prospective cohort, the EsVan2b outperformed other model iterations (Table 3; Fig 1B). To further evaluate the utility of the predicted risk from model EsVan2b, we classified all episodes according to the predicted BSI risk into six risk groups (<10%, ≥10% to <20%, ≥20% to <30%, ≥30% to <40%, ≥40% to <50%, and ≥50% to 100%). The observed non–high-risk organism BSI rate (1%) and high-risk organism BSI rate (1.4%) were extremely low among those classified in the lowest risk group (<10%, N = 2,231) and this group made up 87% of the overall cohort. Among those classified as high-risk (≥40%, N = 83), the observed non–high-risk organism BSI rate was 4.8% and the high-risk organism BSI rate was 33.7% (Fig 2).

FIG 2.

Observed BSI by predicted risk groups for the EsVan2b model. BSI, bloodstream infection; EsVan, Esbenshade Vanderbilt.

Safety Outcomes and Adverse Events

Empiric antibiotics were given in 2,181 episodes (85.1%) and initially withheld in 381 (14.9%), with three cases unknown. The majority of the withheld cases occurred in sites not routinely administering empiric antibiotics to well-appearing non-neutropenic patients (percent of their total episodes: site D 63.6% [143/225], site G 80.4% [119/148], site J 62.0% [31/59], and site L 48.5% [47/97]). For the entire international multisite cohort, there were some differences in those who received antibiotics and those who did not (Data Supplement, Table S4). These included those receiving empiric antibiotics had a higher median ANC (3,360/µL v 2,640/µL), higher presenting temperature median 38.61° (IQR, 38.33° v 39.90°) versus 38.56° (IQR, 38.30°-38.90°), and were more likely to have an external tunneled catheter (14.8% v 7.9%), with all P < .01. The BSI rate was higher in participant episodes treated with empiric antibiotics versus those who did not (5.1 v 2.4%; P = .02). This was also true for high-risk organism BSI (3.6% v 1.0%; P = .01).

Need for ICU Care or Death

The need for ICU care within 7 days after presentation at least partially related to infection was 1.1% (n = 27). This was not different for those who received empiric antibiotics versus no empiric antibiotics (1.1% v 0.5%; P = .414). Both of the episodes where a patient did not receive empiric antibiotics but eventually went to the ICU were reassessed a second time and given antibiotics before they required ICU admission on the following day (one for respiratory support for a presumed bacterial pneumonia and one for hypotension caused by a CONS BSI and an Escherichia coli urinary tract infection). Among those who did not receive empiric antibiotics at presentation, there were no deaths. There was one episode of infection-related death due to sepsis in a patient with a progressive diffuse intrinsic pontine glioma who was end of life and had received empiric antibiotics at presentation. All-cause mortality within 7 days was 0.2% (n = 5); deaths were due to adrenal crisis/hypertensive emergency, tumor disease progression (n = 2), brain injury attributed to hyperleukocytosis at leukemia diagnosis, and the already mentioned patient with sepsis. All these patients had received initial empiric antibiotics.

Readmission to Hospital

Of the 2,565 episodes, 17.2% (n = 440) were admitted for at least 7 days (Fig 3). Of those who were discharged either directly from the emergency department or admitted for <7 days, 21.7% (n = 461) had to be reassessed for fever, infection concern, and/or because initial blood culture turned positive after discharge within 7 days, and of these, 51.6% (n = 238) were readmitted (11.1% of those initially discharged overall).

FIG 3.

Diagram of the study cohort patient outcome and management. BSI, bloodstream infection; ED, emergency department; ICU, intensive care unit.

Of the 238 readmitted patients, 13.4% (n = 32) had positive cultures initially but were afebrile and well-appearing on readmission, 7.6% (n = 18) were persistently febrile and had positive repeat blood cultures, 53.8% (n = 128) were persistently febrile and judged by the treating provider to require admission, and 21.4% (n = 51) were admitted for other reasons. Only 3.8% (9/238) of patients were ill-appearing and required ICU admission on repeat presentation, and 22.2% (n = 2) of these had BSI on readmission. Notably, only one of the nine patients who required ICU admission on repeat presentation had not received empiric antibiotics at initial febrile presentation but had received ceftriaxone before the ICU admission (previously mentioned CONS BSI).

EsVan2b Model Predictions and Outcomes Among Those Who Withheld Empiric Antibiotics

Among the 382 episodes in which initial empiric antibiotics were withheld, 92.1% (n = 352) were characterized as low risk (<10%) by the EsVan2b model, 6% (n = 23) intermediate risk (10% to <40%), and 1.8% (n = 7) high risk (≥40%). The actual overall BSI/high-risk organism BSI rate for these episodes was 2.3%/0.9% (predicted low risk), 4.3%/0.0% (predicted intermediate risk), and 14.3%/14.3% (predicted high risk). There were no deaths or severe decompensations in intermediate-/high-risk patients who did not receive initial empiric antibiotics.

DISCUSSION

The risk of infection-related mortality and complications in pediatric patients with non-neutropenic fever is very low.^3-5,13-20 The EsVan models have previously been retrospectively externally validated both in the United States and internationally, and EsVan stratified management has been implemented prospectively at one site showing safety and success of using the model to guide antibiotic management.^5,6,9 This study now adds prospective external validation of the models in over 2,500 episodes across 18 international institutions with C-statistics for all versions >0.77, suggesting that this approach may, in combination with clinical judgment, safely guide clinical practice.

The EsVan models all predict BSI in non-neutropenic fever with both good discrimination and calibration. Furthermore, the EsVan models are particularly accurate in predicting which patients are at low risk (<10%) of BSI, which across studies account for 74%-88% of the total number of episodes using a threshold of ANC ≥500/µL (87.0% in the current study).^5,6 Similar to the previous retrospective external validation, the models do slightly overpredict the observed risk of BSI. However, this is felt to be ideal as some types of non-BSIs that require antibiotic therapy such as meningitis or pneumonia can have severe symptoms that mimic BSI. The models can miss infections that present with minimal signs and symptoms; however, in both this study and the previous prospective validation, these missed patients are well-appearing and often afebrile when reassessed after their blood culture turns positive.⁹ In fact, in a prospective study that identified 834 low-risk episodes, empiric antibiotics were only given 21.1% of the time to those classified as low risk and were never required at all in 72.3% of low-risk cases by day ≥7 with no severe adverse events attributed to lack of or delayed antibiotic administration.⁹ This suggests that if reliable follow-up is available, low-risk episodes often do not require initial treatment with empiric antibiotics.

The EsVan models were designed to predict BSI, so other possible sources of infection should always be assessed independently of the BSI risk. Furthermore, if a patient is ill-appearing with strong clinical suspicion for BSI or other bacterial infection, empiric antibiotic therapy is indicated regardless of predicted BSI risk. In this study, as shown previously, the EsVan models predicted high-risk organism BSI (gram-negative and Staphylococcus aureus) more accurately than BSIs overall, with C-statistics >0.8, largely because these infections are more likely to be clinically symptomatic with higher fevers and chills. A possible approach could be omitting empiric antibiotics for all well-appearing patients with cancer who are low risk by the EsVan2b model presenting with non-neutropenic fever. As patients who are discharged home who clinically worsen or whose blood cultures flag as positive must be able to return quickly for reassessment, additional caution should be used in patients who live far away from a medical center or have concerns for unreliable follow-up.

This study used a web-based module, which is widely available (RiskPrediction¹⁰) and uses the EsVan2a and 2b models to provide real-time prediction. This study shows that using this module prospectively is feasible. All versions of the EsVan model perform and validate similarly, but the EsVan2a or 2b models (depending on monocyte count availability) are the best to use as they performed best across the study sites and remain our recommended models. However, the EsVan3b model, which does not require an ANC result, can be useful in cases where a patient is not expected to have severe neutropenia but blood counts are not readily available (nomogram to use this in previous publication).⁹

The study does have limitations. The number of episodes captured at each individual site was too low to provide reliable site-specific C-statistics; however, there is stability in the data across sites, suggesting overall estimates of the models are reliable. Although BSI was well predicted, we did not evaluate prediction of other clinically or microbiologically documented bacterial infections, which may require antibiotic therapy. Even a test that absolutely ruled out BSI would not guarantee that withholding antibiotics is safe in all cases as there may be other sources of bacterial infection. Another limitation is that most of the episodes captured were from high-income countries at pediatric centers, and so, additional study would be needed before expanding the use of this model into other populations. Finally, these models should not be used in patients with a history of allogeneic stem-cell transplant, as we previously showed that the existing EsVan models did not perform well in this population. Thus, a new model (EsVanAlloSCT) with further adjustment for the degree of immune suppression was developed and internally validated. External validation of this allogenic transplant–specific model has been planned.²¹ In addition, we plan to formally evaluate the reliability of the EsVan models after chimeric antigen receptor (CAR) T cells and other cellular therapies.

In conclusion, the EsVan models reliably predict the risk of BSI in episodes of non-neutropenic fever in pediatric patients with cancer across multiple sites and have now been prospectively validated. Since severe complications arising from delayed antibiotic administration are rare in those with a predicted BSI risk under 10%,⁹ we believe a randomized controlled trial would not be necessary and withholding initial antibiotics in these low-risk patients would be safe. Widespread adoption of this practice in the management of this common occurrence could have a significant impact on mitigating some of the detrimental effects of unnecessary antibiotic administration.^7,8

DATA SHARING STATEMENT

Data are available upon request from each participating site.

AUTHOR CONTRIBUTIONS

Conception and design: Zhiguo Zhao, Leonora Slatnick, Joshua Wolf, Adam J. Esbenshade

Provision of study materials or patients: Alexandria M. Walsh, Raya Saab, Meghan Mccormick, Catherine Aftandilian, Alison C. Tribble, Etan Orgel, Joshua Wolf, Jenna M. Demedis, Tamara P. Miller

Collection and assembly of data: Zhiguo Zhao, Pratik A. Patel, Leonora Slatnick, Anna Sitthi-Amorn, Kevin J. Bielamowicz, Farranaz A. Nunez, Alexandria M. Walsh, Jennifer Hess, Jenna Rossoff, Caitlin Elgarten, Regina Myers, Raya Saab, Maya Basbous, Meghan Mccormick, Catherine Aftandilian, Rebecca Richards, C. Nathan Nessle, Alison C. Tribble, Scott L. Coven, Daniel Runco, Jennifer Wilkes, Arun Gurunathan, Terri Guinipero, Jennifer A. Belsky, Karen Lee, Victor Wong, Megha Malhotra, Amy Armstrong, Lauren P. Jerkins, Lyndsay Fisher, Madison T. Stein, Natalie L. Wu, Troy Yi, Etan Orgel, Gabrielle M. Haeusler, Joshua Wolf, Jenna M. Demedis, Tamara P. Miller, Adam J. Esbenshade

Data analysis and interpretation: Zhiguo Zhao, Pratik A. Patel, Caitlin Elgarten, Raya Saab, Alison C. Tribble, Daniel Runco, Jennifer Wilkes, Karen Lee, Shane J. Cross, Etan Orgel, Gabrielle M. Haeusler, Joshua Wolf, Jenna M. Demedis, Tamara P. Miller, Adam J. Esbenshade

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Prospective External Validation of the Esbenshade Vanderbilt Models Accurately Predicts Bloodstream Infection Risk in Febrile Non-Neutropenic Children With Cancer

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).