Re-evaluating and recalibrating predictors of bacterial infection in children with cancer and febrile neutropenia

Abstract

Background

Numerous paediatric febrile neutropenia (FN) clinical decision rules (CDRs) have been derived. Validation studies show reduced performance in external settings. We evaluated the association between variables common across published FN CDRs and bacterial infection and recalibrated existing CDRs using these data.

Methods

Prospective data from the Australian-PICNICC study which enrolled 858 FN episodes in children with cancer were used. Variables shown to be significant predictors of infection or adverse outcome in >1 CDR were analysed using multivariable logistic regression. Recalibration included re-evaluation of beta-coefficients (logistic model) or recursive-partition analysis (tree-based models).

Findings

Twenty-five unique variables were identified across 17 FN CDRs. Fourteen were included in >1 CDR and 10 were analysed in our dataset. On univariate analysis, location, temperature, hypotension, rigors, severely unwell and decreasing platelets, white cell count, neutrophil count and monocyte count were significantly associated with bacterial infection. On multivariable analysis, decreasing platelets, increasing temperature and the appearance of being clinically unwell remained significantly associated. Five rules were recalibrated. Across all rules, recalibration increased the AUC-ROC and low-risk yield as compared to non-recalibrated data. For the SPOG-adverse event CDR, recalibration also increased sensitivity and specificity and external validation showed reproducibility.

Interpretation

Degree of marrow suppression (low platelets), features of inflammation (temperature) and clinical judgement (severely unwell) have been consistently shown to predict infection in children with FN. Recalibration of existing CDRs is a novel way to improve diagnostic performance of CDRs and maintain relevance over time.

Funding

National Health and Medical Research Council Grant (APP1104527).

Research in context.

Evidence before this study

Risk stratification in fever and neutropenia (FN), one of the most common complications of childhood cancer care, is recommended in international paediatric FN guidelines. Recalibration of existing risk prediction rules is advised as it avoids the loss of valuable scientific data by combing prior information captured in derivations studies, with contemporary information about a new, albeit similar, population at risk. We searched PubMed, with no restrictions on language or publication date, using the search terms: (febrile OR fever) AND (neutropenia OR neutropenic) AND (clinical decision rule OR risk prediction) AND (recalibration). Only one paediatric FN clinical decision rule (CDR) has undergone recalibration however, to date, it has not been implemented into practice.

Added value of this study

Using a large, prospectively collected database, designed to validate paediatric FN CDRs, we have evaluated the strength of association between individual clinical variables common across published paediatric FN rules and bacterial infection. Using these data we have recalibrated an additional five CDRs, reducing research waste and applying the findings to a contemporary population. We confirm a number of important and accessible variables remain consistently predictive of infection, in particular degree of marrow suppression (low platelets), features of inflammation (height of temperature) and clinical judgement (patient assessed as “severely unwell”). By applying previously published low-risk FN program eligibility criteria, we also highlight the importance of additional safe guards and a structured approach to implementing these pathways.

Implications of all the available evidence

In the context of a global pandemic, managing children with cancer and FN in 2020 using a standard inpatient approach may not be feasible for many centres. Safe and reliable pathways for identifying children at low risk of infection who can be treated at home with short-course oral or intravenous antibiotics are urgently required. Centres considering implementing any of the recalibrated rules should do so as part of a structured low-risk FN program and ensure appropriate measures are in place to monitor the clinical and, where possible, the economic and quality of life impact of this model of care.

Alt-text: Unlabelled box

1. Introduction

Advancements in the management of paediatric cancer, driven largely by risk stratification, have resulted in significant improvements in overall survival [1]. While cancer treatment becomes increasingly personalised, similar evolutions in the management of the complications of care are lacking. Regarding infectious complications, there are increasing global efforts to adopt similar risk-adapted approaches to prediction, prevention and treatment [2]. Specifically, risk stratification in fever and neutropenia (FN), one of the most common complications of care, is recommended although widespread uptake has not been realised [3], [4], [5], [6].

To date, there is no international consensus as to the most important outcome that should be predicted in children with FN [3]. Clinical decision rules (CDRs) have been designed to predict a spectrum of outcomes ranging from bacteraemia alone through to combinations of any bacterial, viral or fungal microbiologically defined infection and other clinical adverse outcomes such as sepsis and intensive care (ICU) admission [7]. More recently, the outcome of likely bacterial infection has been proposed, based on the understanding that this underscores the rationale for the early introduction of broad-spectrum antibiotics [8,9]. Prompt exclusion of bacterial infection has the potential to reduce unnecessary antibiotic exposure, thereby decreasing hospital length of stay and improving resource allocation.

The variation in the outcomes predicted in paediatric FN CDRs may, in part, explain the differences observed in the performance of these rules [7]. Many of these CDRs were also derived over two decades ago and therefore do not account for recent advancements in the treatment of childhood cancer. Rather than continually deriving new risk prediction rules, recalibration of existing rules is advised but rarely performed [10]. Such a strategy would enable clinical variables, consistently shown to predict infection, to be re-examined in contemporary datasets and the weight assigned to these variables readjusted as required. This approach avoids the loss of valuable scientific data by combing prior information captured in derivations studies, with contemporary information about a new, albeit similar, population at risk [11].

The objectives of this study are to (i) evaluate the strength of association between individual clinical variables common across published paediatric FN clinical decision rules and bacterial infection and (ii) recalibrate existing CDRs using these data. We hypothesise that we will identify clinical variables that are shown to be consistently predictive of infection across multiple different studies.

2. Methods

Prospectively collected data from the Australian Predicting Infectious ComplicatioNs in Children with Cancer (PICNICC) study was used for this analysis (Australian New Zealand Clinical Trials Registry 12616001440415) [9]. Eight tertiary paediatric cancer centres in Australia contributed data to this study which was open to recruitment from November 2016 to January 2018. A total of 858 FN episodes in 462 children with cancer were enrolled and available for analysis. Methodology and reporting of results followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis statement (TRIPOD) [12].

Detailed methodology is described elsewhere [9]. Data on consecutive episodes of FN in children (age <18yrs) with cancer or haematological malignancy were prospectively collected. Episodes were included if they had a documented fever and neutropenia. Episodes were excluded if FN treatment commenced at a non-participating site, the patient had undergone a hematopoietic stem cell transplant within the preceding three months or the episode occurred while they were receiving concurrent intravenous or oral antibiotics (excluding prophylaxis). The research assistant collecting the data was blinded to the CDR variables and outcome definitions and data accuracy was verified by the project manager and site investigators (oncology or infectious diseases physician).

The primary outcome for this study was ‘likely bacterial infection' (as defined below). Secondary outcomes included bacteraemia, intensive care unit (ICU) admission, 30-day mortality and eligibility for home based care. In the original dataset, clinical variables were collected at FN presentation and outcomes were collected at the end of FN episode and day 30. The date and time that an infection was first identified was also available in the dataset.

A formal low-risk FN program was not in use during the study period and FN episodes were managed according to state-based hospital FN guidelines that were in keeping with international recommendations [3]. Cessation of antibiotics and hospital discharge was typically considered in patients with marrow recovery, negative cultures and at least a 24h period of clinical stability and absence of fever. Antibacterial prophylaxis (excluding for Pneumocystis jirovecii pneumonia) was not routinely used.

2.1. Identification of clinical variables

Seventeen published paediatric FN CDRs were identified on systematic review of the literature [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29]. Previously published systematic reviews were also assessed to ensure all relevant studies were captured [7,30]. Studies that only reported individual variables for infection, rather than derived a CDR, were not considered.

Variables selected for analysis in the Australian-PICNICC study dataset were those that were included in more than one CDR. Disease-related factors including cancer diagnosis, relapse status, bone-marrow involvement were not considered, a priori, due to heterogeneity between studies in grouping of these diagnoses as well as the changes to definitions of relapse status over time with availability of molecular detection of minimal residual disease. Similarly, presence of a central venous catheter was not included as this present in 98% patients in this dataset, and was deemed a priori as non-discriminatory [9]. Finally, a documented focus of infection was excluded because if clinically apparent at presentation it does not need to be ‘predicted’ and, alternatively, if not apparent then this was included in the outcome rather than predictor.

Rules selected for recalibration were those where all variables were available in the existing dataset and did not include disease-related factors, presence of central venous catheter or focus of infection.

2.2. Definitions

Fever was defined as a single temperature ≥38°C and neutropenia as an absolute neutrophil count (ANC) <1000/mm3. ‘Severely unwell’ was defined as severe sepsis or septic shock (as per Goldstein et al) [31], altered conscious state (Glasgow Coma Score <15 or only responsive to voice or pain), documented as ‘severely unwell’ or equivalent in the patient record or either the blood pressure or respiratory rate in the mandatory emergency call range [32].

A likely bacterial infection was defined as any infection with a microbiologically documented bacterial cause or that was clinically documented in categories typically attributed to bacterial infection, including pneumonia, skin and soft-tissue infection, osteomyelitis or myositis, enterocolitis, otitis media or externa, sinusitis, epididymoorchitis, central venous catheter pocket or tunnel infection, pharyngitis, perianal abscess or cellulitis, peritonitis or lymphadenitis [8,9].

A microbiologically documented infection was defined as an infection that was clinically detectable and microbiologically proven. Bacteraemia was defined as a recognised pathogen (including organisms associated with mucosal barrier injury in the setting of mucositis or neutropenia) from ≥1 blood culture set or common commensals from ≥2 blood culture sets drawn on separate occasions [33]. Eligibility for home based care was defined as no severe sepsis at presentation, no relapsed/refractory disease, not in induction chemotherapy, no acute myeloid leukaemia or infant leukaemia, no hematopoietic stem cell transplant and no-other complication requiring inpatient care [34,35].

2.3. Statistical analysis

Clinical variables significantly associated with likely bacterial infection on univariate analysis were entered into a multivariable model. Logistic regression, with standard errors, was used to identify variables that remained significant. A p value <0.05 was consider significant for all analyses.

Models were updated by a process of recalibration using the outcome of likely bacterial infection. Recalibration included re-evaluation of beta-coefficients from those rules derived using logistic regression models, or recursive-partition analysis with selection of the most parsimonious split for tree-based models [11]. Analyses were performed using ‘R’ (version 3.6.0) packages ‘glm’ and ‘rpart.’ The 95% confidence intervals were calculated using an assumption of Normality using the package ‘confint’.

Sensitivity, specificity, positive predictive value and negative predictive value for each recalibrated CDR for the prediction of likely bacterial infection were calculated. To determine the overall discriminatory ability, the area under the receiver operating characteristic curve (AUC-ROC) were also calculated, with 95% confidence interval derived by 2000-iteration bootstrapping with replacement. To assess clinical utility, the following missed outcomes were reported where relevant: bacteraemia, ICU admission, and death.

The sensitivity and specificity of the Swiss Paediatric Oncology Group (SPOG) CDRs at day 2 was assessed using methodology described by this group [28]. Using variables collected at presentation, the sensitivity of the rule at day 2 (between 0900am and 1100am) was determined by combining the information on episodes with the outcome known at that time with the results of prediction on the remaining episodes.

Validation of the best performing recalibrated CDR was done in a separate dataset of 650 consecutive episodes of FN [36]. These data were collected retrospectively for the purpose of validating paediatric FN CDRs and included episode occurring between November 2011 and June 2015 at a single site [36,37]. The sensitivity, specificity, positive predictive value and negative predicative value of the recalibrated CDR for the prediction of likely bacterial infection was calculated in this separate dataset. The recalibrated rule was considered reproducible if there was overlapping sensitivity or specificity in the prospective recalibration (n=858) and retrospective validation (n=650) cohorts [36].

2.4. Ethics

The study had national and site specific Human Research Ethics Committee approval and informed patient consent was obtained.

2.5. Role of funding source

The funding source had no involvement in study design, data collection, analysis or manuscript preparation or approval.

3. Results

Outcomes predicted across the 17 FN CDRs included in the analysis were bacteraemia in seven [13], [14], [15], [16], [17], [18], [19], microbiologically defined infection in one [20] and the composite outcome of ‘adverse event’ in nine which included various combinations of infection and medical complications such as admission to the ICU [21], [22], [23], [24], [25], [26], [27], [28], [29]. Twenty five unique variables were identified which were grouped into factors relating to; underlying disease, chemotherapy, location before FN onset, patient factors, clinical presentation, marrow suppression, biomarkers and focus for infection (Table 1).

Table 1.

Comparison of individual variables common across paediatric febrile neutropenia clinical decision rules

	Dis	Chemo	Loc	Patient factors			Clinical presentation								Marrow status					Biomarker		Focus	Total
		Intensity	IP	Age	CVC	Nut.	CU	Temp	Rig.	Alt. MS	Cap.	Fluid	BP	Other*	Hb	Plat	WCC	ANC	AMC	IL-8	CRP
Bacteraemia
Ammann-2004[13]	•		•					•									•						4
Ammann 2003[14]	•#				•										•		•				•	•	7
Baorto [15]																			•				1
Lucas [16]	•								•			•	•										4
Rackoff [17]								•											•				2
Santolaya [18]	•^												•			•					•		5
SPOG-bact [19]			•						•						•	•							4
Microbiologically documented infection
PICNICC [20]	•						•	•							•		•		•				6
Adverse outcome
Alexander [21]	•**		•							•			•	•								•	10
Das [22]	•					•												•			•	•	5
Hakim [23]	•						•	•										•					4
Klaassen [24]																			•				1
Miedma [25]															•					•		•	3
Paganini [29]	•		•																				2
Rondinelli [26]				•	•			•							•							•	5
SPOG-AE [28]		•													•	•	•						4
West [27]									•		•												2

Dis is disease; chemo, chemotherapy; loc, location of FN onset; IP, inpatient; CVC, central venous catheter; nut, nutrition; CU, clinically unwell; temp, temperature; cap is capillary refill; BP, blood pressure; Hb, haemoglobin; Plat, platelet; WCC, white cell count; ANC, absolute neutrophil count; AMC, absolute monocyte count; IL-8, interleukin 8; CRP, C-reactive protein; focus, focus of infection; SPOG, Swiss Paediatric Oncology Group; bact, bacteraemia; AE, adverse event. *includes other vital signs, CXR changes, Mucositis, GI symptoms; #includes malignancy type and bone marrow involvement; ^includes malignancy type and time from chemotherapy; ** included type of cancer and relapse status

Fourteen unique variables were included in more than one published CDR of which 10 were analysed in our dataset (Table 1). Three were not assessed as they were excluded a priori (disease/chemotherapy-related factors, presence of central venous catheter and focus of infection). C-reactive protein was also excluded as this was not available in the primary dataset. The 10 variables included in the analysis were location (inpatient versus outpatient), presenting symptoms (clinically unwell, maximum temperature, rigors, hypotension), degree of marrow suppression (haemoglobin, platelets, white cell count (WCC), absolute neutrophil count (ANC), absolute monocyte count (AMC)).

In 198 (23%) FN episodes a likely bacterial infection was documented including 108 (13%) with a bacteraemia. Detailed demographic has been reported previously [9]. The median age of participants was 5.8 years (interquartile range, 3.5-10.7 years) and 449 (52%) had acute leukaemia, 66 (8%) had lymphoma and 343 (40%) had a solid tumour.

On univariate analysis, location, temperature, hypotension, rigors, severely unwell and decreasing platelets, WCC, ANC and AMC were significantly associated with likely bacterial infection (Table 2). On multivariable analysis, decreasing platelets, increasing temperature and the appearance of being clinically unwell remained significantly associated (Table 2). The model had an AUC-ROC of 0•67 (95% CI 0•63-0•71) for the prediction of bacterial infection and, at a threshold of 15%% the sensitivity was 93% and specificity was 23% (Table 3). The beta coefficients for the model are available in Table 1 of the online supplement.

Table 2.

Univariate and multivariable association with likely bacterial infection

	Univariate				Multivariable
	OR	Lower CI	Upper CI	p-value	OR	Lower CI	Upper CI	p-value
Requirement inpatient care	1·90	1·35	2·66	<0·001	0·39	0·09	1·59	0·189
Temperature*	1·46	1·12	1·91	0·005	1·67	1·15	2·42	0·009
Hypotension**	2·03	1·11	3·71	0·022	0·29	0·08	1·13	0·076
Rigors	1·88	1·14	3·10	0·013	0·91	0·43	1·92	0·818
Severely unwell***	2·36	1·42	3·92	0·001	9·3	1·76	49·21	0·009
Haemoglobin	0·97	0·89	1·05	0·449	0·96	0·92	1	-
Platelets^	0·93	0·91	0·99	<0·001	0·95	0·55	1·65	0·024
White cell count#	0·48	0·35	0·66	<0·001	0·84	0·58	1·22	0·850
Absolute neutrophil count#	0·64	0·49	0·84	0·001	0·83	0·55	1·25	0·371
Absolute monocyte count#	0·59	0·44	0·80	0·001	0·39	0·09	1·59	0·380

OR, odds ratio; CI, confidence interval; SE, standard error;

*Per degree above 37°C; **defined as systolic blood pressure below the mandatory emergency call threshold for age; ***defined as as severe sepsis or septic shock, altered conscious state, documented as ‘severely unwell’ or equivalent in the patient record or either the blood pressure or respiratory rate in the mandatory emergency call range; ^Per 10-platelet” increment (i.e. 30 × 10^9/L to 40 × 10^9/L); #natural log

Table 3.

Sensitivity and specificity of “new rule” at different thresholds

Low risk threshold	Low risk, n (%)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
10%	92 (10·7)	96·5 (92·9-98·3)	13·0 (10·7-15·8)	25·1 (22·1-28·3)	92·4 (85·1-96·3)
15%	164 (19·1)	92·9 (88·4-95·7)	23·0 (19·9-26·4)	26·7 (23·5-30·1)	91·5 (86·2-94·9)
20%	281 (32·8)	82·7 (76·9-87·4)	37·8 (34·2-41·6)	28·7 (25·1-32·5)	87·9 (83·6-91·2)

PPV, is positive predicitive value; NPV, negative predicitive value; CI, confidence interval

3.1. Rule recalibration

Of the seventeen CDRs identified, five were recalibrated. Five CDRs were not recalibrated as they included variables unavailable in the dataset [14,18,22,25,27]. A further six included disease-related variables [13,16,20,21,23,29] and one included presence of central venous catheter (Rondinelli) [26].

Of the five rules recalibrated, three used AMC and temperature (Klaassen, Baorto, Rackoff), one used haemoglobin, platelets, rigors and requirement for in-patient care (SPOG-bacteraemia) and one used chemotherapy intensity, haemoglobin, platelets and WCC (SPOG-adverse event) (Table 4). The optimal AMC cut-off was 0•015 cells/mm³ and temperature was 39•5°C, compared with their original values of 0•1 to 0•155 cells/mm³ and 39°C. Both the SPOG-bacteraemia and SPOG-adverse event rule recalibrations set each of the three factors at the same weighting (1), and haemoglobin was removed as uninformative.

Table 4.

Comparison of derivation and recalibrated (to predict bacterial infection) variables denoting ‘high risk’

	Derivation variables	Recalibrated variables
Baorto [15]	AMC < 155 cells/mm3	AMC<15 cells/mm3
Rackoff [17]	High risk = AMC < 100 cells/m3 and temperature ≥39°C; Low risk = AMC ≥ 100 cells/mm3; intermediate risk = AMC <100 cells/mm3 and temperature <39°C	High risk = AMC < 15 cells/m3 and temperature ≥39·5°C; Low risk = AMC ≥ 15 cells/mm3; intermediate risk = AMC <15 cells/mm3 and temperature <39·5°C
SPOG-bacteraemia [19]	Applied after 24 hours. Score for shaking or chills = 5; Hb ≥90g/L = 3; platelet <50 g/L = 3; Other need for inpatient care = 3	Score for shaking or chills = 1; platelet <50 g/L = 1; Other need for inpatient care = 1
Klaassen [24]	AMC < 100 cells/mm3	AMC<15 cells/mm3
SPOG-AE [28]	Applied after 24 hours. Total score ≥ 9 = high risk Score for preceding chemotherapy more intensive than ALL maintenance =4; Hb ≥ 90 g/L =5; WCC < 300 cells/mm3 = 3; platelet <50 g/L =3	Score for preceding chemotherapy more intensive than ALL maintenance =1; WCC < 300 cells/mm3 = 1; platelet <50 g/L =1

AMC is absolute monocyte count; Hb, haemoglobin

The sensitivity and specificity of the recalibrated CDRs is available in Table 5. Across all rules, recalibration increased the AUC-ROC and low-risk yield as compared to non-recalibrated data (Table 2, online supplement) [9]. In the SPOG-adverse event rule, recalibration also increased sensitivity and specificity, while for the remaining CDRs sensitivity reduced.

Table 5.

Sensitivity and specificity of recalibrated (RC) paediatric febrile neutropenia clinical decision rules for prediction of likely bacterial infection.

	Low risk, n (%)	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
RC-Baorto [15]	327 (38·1)	0·58 (0·54-0·63)	70·7 (64·0-76·6)	49·6 (45·7-53·6)	29·6 (25·7-33·9)	84·9 (81·0-88·2)
RC-Rackoff [17]*	385 (44.9)	0·60 (0·55-0·64)	70·7 (64·0-76·6)	49·6 (45·7-53·4)	29·6 (25·7033·9)	84·9 (81·0-88·2)
RC-SPOG bact [19] (LR≤0) (n=825)	312 (37·8)	0·64 (0·59- 0·67)	77·0 (70·5-82·4)	42·3 (38·5-46·2)	28·7 (24·9-32·7)	86·0 (81·6-89·3)
RC-Klaassen [24]	327 (38·1)	0·58 (0·54-0·63)	70·7 (64·0-76·6)	49·6 (45·7-53·6)	29·6 (25·7-33·9)	84·9 (81·0-88·2)
RC-SPOG AE [28] (LR≤0)	84 (9·8)	0·64 (0·60-0·68)	93·5 (91·6-97·6)	11·4 (9·1-14·0)	24·4 (21·5-27·6)	89·3 (80·9-94·3)

AUC-ROC is area under the receiver operating characteristic curve; CI, confidence interval; PPV, positive predicitive value; NPV, negative predicitive value; LR, low risk; RC, recalibrated

*intermediate and high-risk combined into a single high-risk group

At a threshold of zero, the recalibrated SPOG-adverse event CDR had the highest sensitivity for prediction of bacterial infection and the performance of this rule was further explored (Table 6). Validation of this recalibrated rule indicates reproducibility with overlapping sensitivity across all thresholds and both sensitivity and specificity at threshold of 2 (Table 3, online supplement). Details of clinically significant infections and adverse outcomes, stratified by the recalibrated SPOG-adverse event score is provided in Table 7. The proportion of episodes with a likely bacterial infection (including bacteraemia) was 10•7%, 16•4%, 23•2% and 38•1%, for risk-scores 0, 1, 2 and 3, respectively. Applying previously published eligibility criteria for home-based management of FN in children, the overall number of patients eligible for early transfer home would be 488 [34]. Notably, only one of 24 ICU admissions would have been missed in a patient that scored the maximum of three.

Table 6.

Clinical performance of the recalibrated SPOG-advere event [28] rule at each threshold at presentation and Day 2 for prediction of likely bacterial infection.

	Low risk, n (%)	Missed BSI, n (%)*	Missed LBI, n (%)*	Se (95% CI)	Sp (95% CI)	PPV (95% CI)	NPV (95% CI)
Low risk ≤ 0
Presention	84 (9·8)	3 (3·6)	9 (10·7)	93·5 (91·6-97·6)	11·4 (9·2-14·0)	24·4 (21·5-27·6)	89·3 (80·9-94·3)
Day 2	81 (9·4)	3 (3·7)	6 (7·4)	97·0 (93·6-98·6)	11·4 (9·2-14·0)	24·7 (21·8-27·9)	92·6 (84·8-96·6)
Low risk ≤ 1
Presention	382 (44·5)	25 (6·5)	58 (15·2)	70·1 (64·0-76·6)	49·1 (45·3-52·9)	29·4 (25·5-33·7)	84·8 (80·9-88·1)
Day 2	371 (43·2)	23 (6·2)	47 (12·7)	76·3 (69·9-81·7)	49·1 (45·3-52·9)	31·0 (27·1-35·3)	87·3 (83·6-90·3)
Low risk ≤ 2
Presention	666 (77·6)	61 (9·2)	124 (18·6)	37·4 (30·9-44·3)	82·1 (79·0-84·9)	38·5 (32·0-45·6)	81·4 (78·3-84·2)
Day 2	650 (75·8)	58 (8·9)	108 (16·6)	45·5 (38·7-52·4)	82·1 (79·0-84·9)	43·3 (36·7-50·1)	83·4 (80·2-86·1)

BSI, blood stream infections; LBI, likely bacterial infections; Se, sensitivity; Sp, specificity; PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval.

^⁎calculated as proportion of low-risk episodes

Table 7.

Clinically significant infections and adverse outcomes stratified by recalibrated SPOG-advere event [28] score threshold.

Threshold	0 (n=84)	1 (n=298)	2 (n=284)	3 (n=192)
Bacteraemia, n (%)	3 (3·6)	22 (7·4)	36 (12·7)	47 (24·5)
LBI, n (%)	9 (10·7)	49 (16·4)	66 (23·2)	74 (38·1)
ICU, n (%)	0	7 (2.3)	6 (2.1)	11 (5.7)
30-day mortality, n (%)	0	0	4 (1.4)	0
Eligible home care*, n (%)	73 (86·9)	182 (61·1)	129 (45·4)	104 (54·2)
-Missed bacteraemia [D2], n	2 [2]	7 [5]	7 [7]	21 [20]
-Missed LBI [D2], n	6 [4]	21 [15]	21 [19]	36 [34]
-Missed ICU [D2], n	0	0	0	1 [1]
-Missed 30-d mortality [D2], n	0	0	0	0

LBI, likely bacterial infection; ICU, intensive care unit admission; D2, day 2.

^⁎Eligibility for home-based care defined as no severe sepsis at presentation, no relapsed/refractory disease, not in induction chemotherapy, no acute myeloid leukaemia or infant leukaemia, no HSCT and no-other complication requiring inpatient care. Data presented in square brackets is the number of missed outcomes at day 2.

4. Discussion

Using a large, multisite, prospectively collected dataset we identified nine clinical variables, routinely available in the initial assessment of children with cancer and FN, that are associated with likely bacterial infection. These variables, either alone or in various combinations, have similarly been shown to be associated with infection in this population. A new model, incorporating markers of marrow suppression (decreasing platelets), degree of acute inflammation (temperature) and clinical presentation (well or unwell) was derived and, while very sensitive, the overall discriminatory ability was poor. Five existing FN CDRs were also recalibrated using the outcome of likely bacterial infection. Across all rules, recalibration improved the CDRs ability to predict bacterial infection, most notably for the SPOG-adverse event rule [28].

The recalibrated SPOG-adverse event rule had the best overall performance in our dataset [28]. Similar to previous derivation and validation studies, the sensitivity and negative predictive value of this recalibrated rule for prediction of likely bacterial infection improved after a period of in-hospital observation [28,36]. When applying additional safety-net criteria, as described in adult and paediatric low-risk FN programs, 57% of FN episodes would have been eligible for home-based care with very few missed infections and only one severe adverse outcome [34,35]. A suggested approach to implementation of this rule is available, and includes a minimum in-hospital observation period depending on the score [38]. To recalibrate the Swiss-derived SPOG-adverse event rule, prospective data from this Australian study was used in collaboration with the United Kingdom (UK) PICNICC research group [39]. We propose this recalibrated rule, developed from data collected in Australia, UK and Switzerland is called the AUS-rule to differentiate it for future use.

Our study did not show an association between haemoglobin and infection. Haemoglobin has previously been identified as a predictor of infection in at least six FN CDR derivation studies [14,19,20,25,26,28]. Curiously, an elevated haemoglobin was predictive of bacteraemia or adverse outcome in four studies, of which three were from the SPOG research network [14,19,28]. This association was further explored and the authors concluded that dehydration, rather than recent red blood cell transfusion, may in part explain the counterintuitive findings [40]. They also showed that haemoglobin was bimodally distributed, with severe anaemia also being associated with severe infection.

When selecting a paediatric FN CDR for implementation consideration must be given to timely access to the relevant clinical variables. Three of the recalibrated CDRs included monocyte count that requires manual assessment of the blood film and therefore may not be readily available outside standard ‘business hours.’[15,17,24] Reliability is also an important factor and may, in part, explain why many FN CDR developers have explored objective markers such as full blood examination parameters and temperature over more subjective measures such as severely unwell. However the latter has been shown to be consistently predictive of infection or adverse outcome on multivariate analyses suggesting, not surprisingly, that is an important component of the risk stratification process [20,39].

Although we were unable to re-evaluate all published FN CDRs, our study focused on rules that included variables shown to be predictive across multiple settings. Furthermore, while post hoc, this analysis was done using a large, prospectively collected database designed to capture variables and outcomes specific to paediatric FN risk prediction tools. This is also the first paediatric FN study to recalibrate the five FN CDRs, thereby preserving valuable discoveries made around the world and enhancing generalizability in a contemporary Australian dataset. It is important to acknowledge that the CDRs included in this analysis were derived in the era of traditional cytotoxic immunosuppressive cancer treatment and may not reflect novel approaches such as chimeric antigen receptor T-cell therapy and checkpoint inhibition. While recalibration may, in part, account for this, further research is required to understand risk factors and predictors of severe infection in these patients, many of whom may not present with the traditional FN syndrome.

Our study has re-evaluated and refined predictors of bacterial infection in children with cancer. Somewhat reassuringly, there are key components of risk prediction that remain consistent over time and location, namely the more unwell looking and the more marrow suppressed patients are, the more likely they are to have a bacterial infection. By exploring these variables consistently shown to be predictive, as well as recalibrating existing CDRs, we have avoided the loss of valuable scientific data captured in derivation studies around the world. The recalibrated SPOG-adverse event rule, or AUS-rule, showed the best performance in our dataset. The additional safety-net criteria further enhanced the potential safety of this rule emphasising that, while an important component of the risk stratification process, CDRs should not be used in isolation. Centres considering using this recalibrated rule should do so as part of a structured low-risk FN program, and ensure appropriate measures are in place to monitor the clinical and, where possible, the economic and quality of life impact of this model of care. A suggested approach to implementation of such a program is available through the Australian National Centre for Infections in Cancer and includes relevant organisational-, clinician- and patient-level resources [38]. Adaptations to this model should be made in consultation with local stakeholders (such as oncology, infectious diseases, emergency medicine, nursing and pharmacy) as well as patient and family representatives to enhance the uptake, safety and appropriate use of home-based FN care.