Abstract
Background:
The primary objective was to measure the proportion of episodes where care delivery was inconsistent with selected recommendations of a clinical practice guideline (CPG) on fever and neutropenia (FN) management. The influence of site size on CPG-inconsistent care delivery and association between patient outcomes and CPG-inconsistent care were described.
Methods:
This retrospective, multi-center study included patients < 21 years old with cancer who were at high risk of poor FN outcomes and were previously enrolled to a Children’s Oncology Group (COG) study at participating National Cancer Institute Community Oncology Research Program (NCORP) institutions from January 2014 through December 2015. Patients were randomly selected for chart review by participating sites from a COG-generated list. Care delivered in each episode was adjudicated (CPG-consistent or CPG-inconsistent) against each of five selected recommendations.
Results:
A total of 107 patients from 22 sites, representing 157 FN episodes, were included. The most common CPG-inconsistent care delivered was omission of pulmonary computerized tomography in patients with persistent FN (60.3%). Of 74 episodes where assessment of four (episodes without persistent FN) or five (episodes with persistent FN) recommendations was possible, CPG-inconsistent care was delivered with respect to at least one recommendation in 63 (85%) episodes. Site size was not associated with CPG-inconsistent care delivery. No statistically significant association between CPG-inconsistent care and fever recurrence was observed.
Conclusions:
In this cohort of pediatric patients at high risk of poor FN outcomes, CPG-inconsistent care was common. Opportunities to optimize resource stewardship by boosting supportive care CPG implementation are highlighted.
Keywords: oncology, fever and neutropenia, clinical practice guideline, cancer care delivery
Introduction
Fever and neutropenia (FN) is a severe and life-threatening complication of treatment in pediatric oncology patients. There is consensus that pediatric patients with FN require timely assessment and empiric treatment. Yet, FN management practices vary widely within and between pediatric centers.1–4
Clinical practice guidelines (CPGs) are statements that include recommendations intended to harmonize and optimize patient care for a specific clinical condition or presentation. CPGs are underpinned by a systematic review of evidence and incorporate an assessment of the benefits and harms of alternative care options.1 Since CPGs promote the incorporation of the best available published evidence into care, they are a critical tool for translating evidence to practice. Delivery of care that is consistent with rigorously developed CPGs has been shown to improve patient outcomes in many clinical contexts, including FN.6–9 Recognizing the value of CPGs and the need to promote their use, the Children’s Oncology Group (COG) created a task force to systematically identify published supportive care CPGs and endorse those that meet pre-defined criteria regarding quality and relevance. The COG is the world’s largest consortium that conducts trials in pediatric oncology patients. It first endorsed a CPG pertaining to the management of FN in pediatric oncology patients2 in February 2014.
While CPG endorsement is an important step to increase awareness, it does not ensure CPG-consistent care delivery. There is no published information documenting whether pediatric institutions are delivering CPG-consistent care to their patients presenting with FN. We hypothesized that supportive care CPG uptake would be low, specifically that CPG-inconsistent care would be delivered in more than half of episodes. Our primary objective was to measure the proportion of care episodes where the care delivered was inconsistent with selected recommendations of the COG-endorsed CPG on FN management. Secondary objectives were to describe the 1) influence of site size on the delivery of CPG-inconsistent care and 2) association between CPG-inconsistent care and patient outcomes.
Methods
This retrospective, multi-center study was developed by the COG and conducted in the National Cancer Institute (NCI) Community Oncology Research Program (NCORP), a network of Community-based and Minority/Underserved institutions across the United States. This study was one component of ACCL15N1CD, the COG’s first cancer care delivery research study. The overarching objective of ACCL15N1CD was to evaluate supportive care CPG implementation in pediatric oncology NCORP institutions. All 37 COG-member NCORP institutions were invited to open the study; 26 chose to participate and 22 contributed FN episodes. The study was approved by the NCI Pediatric Central Institutional Review Board (IRB) and IRBs at participating sites. The need for informed consent or assent was waived by the NCI Pediatric Central IRB.
Patients
Patients were considered for study inclusion if they had a cancer diagnosis associated with high risk of poor FN outcomes (i.e., newly diagnosed acute lymphoblastic leukemia (ALL) during induction, relapsed ALL, acute myelogenous leukemia (AML), advanced stage Burkitt (outside of maintenance phase) or mature B cell non-Hodgkin lymphoma (outside of maintenance phase), or myeloablative autologous hematopoietic cell transplant (HCT) or allogeneic HCT)11, were previously enrolled to a COG study (therapeutic or non-therapeutic) at a participating site from January 1, 2014 through December 31, 2015, and were less than 21 years of age at enrollment to that study. The COG created a list of patients meeting these criteria and then randomly selected patients for local chart review to identify episodes of FN. The number of randomly selected patients for chart review per participating site was proportional to the total number of COG enrollments at each participating site during the study period.
Study personnel at participating sites reviewed the list of randomly selected patients to confirm their eligibility for study inclusion. In addition to the eligibility criteria noted above, eligible patients were also required to have received inpatient or outpatient care at the participating institution from January 1, 2014 through December 31, 2015.
Once eligibility was confirmed, study personnel at participating sites reviewed the health records of each patient to identify FN episodes. Each patient could contribute one FN episode per chemotherapy cycle, and up to four FN episodes in total. When more than one FN episode occurred during a single chemotherapy cycle, the first was included. Similarly, when more than four FN episodes were identified for a patient, the first four that occurred during the study period were selected.
Definitions
Fever was defined as a single temperature of ≥ 38.3°C (100.9°F) or a temperature of ≥ 38.0°C (100.4°F) for at least 1 hour, regardless of the route used to measure temperature. Neutropenia was defined as an absolute neutrophil count less than 0.5 × 109/L or a count of less than 1 × 109/L with a decrease to less than 0.5 × 109/L within the following 48 hours.3 The initial empiric antibiotic regimen was defined as the systemic antibiotic therapy administered within the first 48 hours from onset of fever that was not given as prophylaxis or was not continued therapy directed at a previously identified source.
An episode of FN began with the first occurrence of fever during neutropenia and ended when IV broad-spectrum antibiotic therapy was discontinued, neutropenia had resolved or the patient died, whichever occurred first. If fever leading to an FN diagnosis was first documented at home, the episode started when the patient arrived at the participating site. Episodes of FN that did not trigger administration of antibiotics were assessed for the outcomes of interest and patients were followed until neutropenia resolved or the patient died.
Persistent FN was defined as persistent or recurrent fever in the presence of neutropenia in patients receiving empiric antibiotic therapy for at least 96 hours.4 FN episodes which occurred after a diagnosis of invasive fungal disease (IFD) were not eligible for evaluation of recommendations relating to the management of persistent fever.
Outcomes
The primary outcome was care delivery that was inconsistent with selected strong recommendations of the COG-endorsed CPG on FN management.2 The CPG used the Grading of Recommendations, Assessment, Development and Evaluation system where strong recommendations are those where the benefits clearly outweigh the consequences or vice versa and should usually be adopted as a matter of policy.5 This perspective formed the basis for selection of recommendations for adjudication.
Five CPG recommendations were chosen for assessment of CPG-inconsistent care: (1) performance of chest X-ray within the first 24 hours of FN onset in patients lacking lower respiratory signs or symptoms; (2) selection of CPG-inconsistent initial empiric antibiotic therapy; (3) discontinuation of empiric antibiotic therapy too early or too late; (4); obtaining a serum β-D glucan result; and (5) omission of pulmonary computerized tomography (CT) in patients with persistent FN. To determine if a patient was without symptoms consistent with a respiratory infection before an initial chest X-ray was obtained, the panel reviewed the history and physical examination notes written before the time of the chest X-ray. In terms of initial empiric antibiotic therapy, the following were considered to be CPG-inconsistent although they were not explicitly defined by the CPG recommendation: administration of initial empiric antibiotic agents four or more hours after patient presentation, vancomycin administration without clinical signs of a gram-positive infection, initial ceftriaxone administration and continuation of empiric antibiotic therapy that included two or more gram-negative antibiotics or vancomycin beyond 72 hours where there was no microbiological reason to continue these therapies. Antibiotic therapy duration was deemed to be CPG-inconsistent (too short) if it was discontinued before the blood cultures at 48 hours were negative, or before the patient had been afebrile for at least 24 hours and the white blood cell count was rising (minimum absolute neutrophil count of 0.1 × 109 /L and rising from baseline). Empiric antibiotic therapy duration was deemed to be too long if it was continued in patients who had received a minimum of 48 hours of empiric antibiotic therapy and had been afebrile for at least 24 hours with a minimum absolute neutrophil count of 0.2 × 109/L for 3 or more days. Duration of empiric antibiotic therapy was not adjudicated in patients with clinically or microbiologically documented infection, other than Clostridioides difficile.
Secondary outcomes were fever recurrence within seven days of discontinuation of broad-spectrum antibiotics, intensive care unit (ICU) admission, and infection-related death during the FN episode or within 7 days of discontinuation of empiric antibiotics.
FN Episode Adjudication
For each episode, site personnel uploaded de-identified sections of the patient’s health record into the study database. One study team member (AV, AS, or NS) reviewed these documents and created an episode summary using a standardized template. A second study team member (LLD) checked each episode summary against the uploaded documents. The adjudication panel (minimum three panelists (AG, AJS, BTF or LS) reviewed the episode summaries at virtual video conferences and adjudicated the care (CPG-consistent or CPG-inconsistent) against each of the five selected recommendations. Reasons that episodes were not applicable for adjudication varied by recommendation and included performance of chest X-ray at an external institution, infection at presentation, infection documentation during the episode and persistent fever despite bone marrow recovery (Figure 1). In addition, episodes where adjudication was applicable for all five recommendations (episodes involving persistent fever) or for all four recommendations (episodes not involving persistent fever) were comprehensively evaluated for CPG inconsistency.
Figure 1:
Diagram of episode identification and adjudication
To determine CPG-consistency, the adjudication panel used the information that would have been available when care delivery decisions were made. For example, decisions to obtain a chest X-ray at presentation were adjudicated using only the information regarding symptoms of possible chest infection that were documented prior to obtaining the chest X-ray. Information that was revealed later in the FN course was not used to adjudicate this aspect of care.
A proportion of all episode summaries (5%) were randomly selected for re-adjudication. If any discrepancy between adjudication decisions was identified, the adjudication panel discussed the episode once again to finalize the episode adjudication.
Statistical Analysis
Descriptive statistics were computed for all clinical and demographic characteristics and patient outcomes. The primary hypothesis was that the proportion of FN episodes involving any CPG-inconsistent care was greater than 0.50. The primary outcome was the delivery of CPG-inconsistent care during each episode. This was a composite outcome and differed for episodes that did and did not involve persistent fever. Episodes with persistent neutropenia were assessed based on all five CPG recommendations. Episodes without persistent neutropenia were assessed based on four CPG recommendations. The proportion of CPG-inconsistent episodes for each of the five CPG recommendations was also described separately.
The study was designed to test whether the true proportion of episodes with CPG-inconsistent care was greater than 0.50 assuming a one-sided, one-sample binomial z-test of proportion with an alpha level of 0.025. Assuming that the true proportion of CPG-inconsistent care was 0.65, that the correlation within an institution was 0.1, and the number of institutions was at least 15, enrollment of 150 FN episodes provided at least 80% power. Standard error for Wald tests was calculated using a hierarchical bootstrap approach to account for within-institution and within-patient correlation.6 Standard errors were estimated using 5,000 bootstrap samples.
Secondary analyses included assessment of the association between site size and receipt of CPG-inconsistent care and the association between CPG-inconsistent care and patient outcomes. The number of patients in each site enrolled on COG studies, excluding banking, biology and registry studies, was used as the proxy for site size. Sensitivity analysis using the number of all COG enrollments, including biology and registry studies, at each site as the proxy for site size was also performed. Association between site size and receipt of CPG-inconsistent care was examined using generalized linear mixed effects models to account for within-site and within-patient correlations through the inclusion of random intercepts for each site and patient. Potential confounders were identified a priori and were included as adjustment variables in the model. These included the Area Deprivation Index and site type (pediatric only vs. mixed population and minority/underserved NCORP vs. not minority/underserved NCORP). The Area Deprivation Index of Brokamp, et al. was used.7 It is based on the 2015 five-year American Community Survey and was developed for each census tract using principal components analysis and including the following measures: the fraction of households with income below poverty level, the median household income in 2015 inflation-adjusted dollars, the fraction of the population 25 years of age and older with at least a high school graduation or Graduate Education Development equivalency, the fraction of population with no health insurance coverage, the fraction of households receiving public assistance income, food stamps, or Supplemental Nutrition Assistance Program, and the fraction of vacant houses. The Area Deprivation Index for each zip code tabulation area was calculated using the mean of all intersecting census tracts.16 In post hoc analyses, the association between different site and patient characteristics and the odds of receiving CPG-inconsistent care were assessed using generalized linear mixed effects models.
The association between CPG-inconsistent care delivery and patient outcomes (i.e., fever recurrence, intensive care unit admission, and infection-related death) was investigated using generalized linear mixed effects models. These patient outcome variables were chosen to represent patient-level clinical “response” to a “treatment” such as CPG-inconsistent care. We therefore considered patient outcomes from different patients (even at the same institution) to be independent. Random intercepts were used to account for within-patient correlation. All analyses were conducted using R version 4.2.2.
Results
A total of 107 patients, contributing a total of 157 FN episodes, from 22 sites were included. Seventy-eight patients each contributed one episode of FN; 15 patients contributed two episodes; seven patients contributed three episodes, and seven patients contributed four episodes each. Figure 1 presents the flow of patients reviewed for eligible episodes and the episodes adjudicated for each aspect of FN care delivery evaluated. Patient, site, and episode characteristics are presented in Tables 1, 2, and 3 respectively. The median patient age at first episode onset was 6.0 years (Q1, Q3: 3.2, 13.2). Approximately half of all patients (48%) were female, with the most common diagnoses being ALL and AML. Empiric antibiotics were never initiated in two episodes (1.3%). Clinically and microbiologically documented infection was reported in 56 episodes (35.6%) and persistent fever developed during 58 episodes (36.9%). One patient died during an FN episode; this death was considered infection related. The number of episodes that were applicable for adjudication and reasons for inapplicability are presented in Figure 1.
TABLE 1.
Patient Characteristics
| Characteristic | Patients (N = 107) | Episodes (N = 157) |
|---|---|---|
| Median Age in years (interquartile range) | *6.0 (3.2, 13.3) | **6.1 (3.3, 13.1) |
| Female Sex; n (%) | 51 (47.7) | 74 (47.1) |
| Cancer Diagnosis; n (%) | ||
| Acute Lymphoblastic Leukemia | 54 (50.5) | 55 (35.0) |
| Acute Myelogenous Leukemia | 38 (35.5) | 79 (50.3) |
| Hodgkin Lymphoma | 1 (0.9) | 1 (0.6) |
| Leukemia - other | 5 (4.7) | 6 (3.8) |
| Lymphoma - other | 1 (0.9) | 1 (0.6) |
| Neuroblastoma | 1 (0.9) | 2 (1.3) |
| Non-Hodgkin Lymphoma | 7 (6.5) | 13 (8.3) |
| Stem Cell Transplantation; n (%) | 6 (5.6) | 8 (5.1) |
| Race/Ethnicity; n (%) | ||
| Hispanic/Latino | 29 (27.1) | 45 (28.7) |
| Native Hawaiian/Pacific Islander | 2 (1.9) | 3 (1.9) |
| Non-Hispanic Asian | 6 (5.6) | 7 (4.5) |
| Non-Hispanic Black | 8 (7.5) | 13 (8.3) |
| Non-Hispanic White | 56 (52.3) | 77 (49.0) |
| Unknown Race | 6 (5.6) | 12 (7.6) |
| Median Income by Zip Code; US$ | ||
| Minimum | 25,313.3 | 25,313.3 |
| 20th percentile | 41,977.9 | 41,435.0 |
| 40th percentile | 47,451.8 | 48,121.6 |
| 60th percentile | 55,157.8 | 57,524.4 |
| 80th percentile | 71,838.7 | 70,289.1 |
| Max | 209,785.4 | 209,785.4 |
| Median proportion with assisted income (interquartile range) | 0.14 (0.09, 0.20) | 0.14 (0.08, 0.20) |
| Median proportion with a high school education (interquartile range) | 0.88 (0.82, 0.92) | 0.87 (0.82, 0.92) |
| Median proportion with no health insurance (interquartile range) | 0.13 (0.08, 0.19) | 0.13 (0.09, 0.19) |
| Median proportion in poverty (interquartile range) | 0.15 (0.09, 0.20) | 0.15 (0.09, 0.21) |
| Median proportion in vacant housing (interquartile range) | 0.11 (0.07, 0.17) | 0.10 (0.07, 0.16) |
| Median deprivation index of location of residence (interquartile range) | 0.38 (0.30, 0.45) | 0.37 (0.30, 0.45) |
Age at the onset of the first FN episode
Age at episode onset
TABLE 2.
Characteristics of the 22 participating sites
| Site Characteristic | |
|---|---|
| Minority/Underserved NCORP Sites; n (%) | 9 (40.9) |
| Pediatric (vs. Mixed); n (%) | 16 (72.7) |
| Private (vs. Academic); n (%) | 16 (72.7) |
| Location; n (%) | |
| Midwestern United States | 4 (18.2) |
| Northeastern United States | 1 (4.5) |
| Southeastern United States | 10 (45.5) |
| Southwestern United States | 4 (18.2) |
| Western United States | 3 (13.6) |
| Size | |
| Median number of patients registered on all COG studies during study period* (interquartile range) | 79 (56, 95.5) |
| Median number of patients registered on COG treatment studies (excluding banking, biology, or registry) during study period* (interquartile range) | 39.5 (29, 52.5) |
January 1, 2014 to December 31, 2015
COG, Children’s Oncology Group; NCORP, National Cancer Institute Community Oncology Research Program
TABLE 3.
Proportion of applicable episodes with CPG-inconsistent care
| CPG-Inconsistent Care Delivered | Number of applicable episodes* | Number of applicable episodes with CPG-inconsistent care (%) | P-value** |
|---|---|---|---|
| Chest X-ray obtained in asymptomatic patient | 151 | 16 (10.6) | 0.999 |
| Initial empiric antibiotic selection | 140 | 66 (47.1) | 0.674 |
| Antibiotic duration too short or too long | 84 | 35 (41.7) | 0.864 |
| β-D glucan obtained | 157 | 22 (14.0) | 0.999 |
| Pulmonary CT not obtained for patients with persistent febrile neutropenia | 58 | 35 (60.3) | 0.148 |
| Any CPG-inconsistent care among five evaluated recommendations*** | 74 | 63 (85.1) | < 0.001 |
CPG, clinical practice guideline; CT, computerized tomography
Number of applicable episodes is presented in Figure 1. Not all episodes were applicable for every recommendation.
The p-value corresponds to the one-sided hypothesis test investigating the alternative hypothesis that the proportion of episodes with CPG-inconsistent care is greater than 0.5.
Episodes with and without persistent fever with neutropenia were assessed based on five and four CPG recommendations, respectively.
CPG-inconsistent care
Table 3 presents the number of episodes with CPG-inconsistent care for each CPG recommendation separately and for those episodes where it was possible to evaluate inconsistency with respect to four or five CPG recommendations as a composite endpoint. Of the individual CPG recommendations, omission of pulmonary CT in patients with persistent FN (60.3%), followed by initial antibiotic selection (47.1%) were the most commonly CPG-inconsistent. The first dose of initial empiric antibiotics was administered four or more hours after patient presentation in four episodes (2.9%). The initial empiric antibiotic regimens included vancomycin in 64 episodes (45.7%), and it continued for more than 72 hours in 35 episodes (25.0%). More than one gram-negative antibiotic was initially prescribed in 9 episodes (6.4%) and continued for more than 72 hours in 4 episodes (2.9%). The duration of empiric antibiotic administration was too long in 5 episodes (6.0%) and too short in 30 episodes (35.7%).
Of the 74 episodes where a composite evaluation was possible, 63 (85%) were CPG-inconsistent for at least one of the assessed recommendations. This was greater than the hypothesized proportion of 0.5 (p < 0.001).
CPG-inconsistent care and site size
The association between CPG-inconsistent care delivery and site size was investigated for each of the five CPG recommendations evaluated with adjustment for Area Deprivation Index, site type (private or academic), and community served (Minority/Underserved or not). Due to data sparsity, the model for the chest x-ray CPG recommendation was adjusted only for Area Deprivation Index.
The estimated odds ratio of CPG-inconsistent care delivery for each of the five CPG recommendations evaluated are presented in Supplementary Tables S1 and S2. No statistically significant associations between site size and CPG-inconsistent care delivery were observed in either the univariable or multivariable analyses. The sensitivity analyses to investigate how the results change when site size is defined by all COG enrollments, including banking, biology, and registry studies, showed similar results (results not presented).
CPG-inconsistent care and patient outcomes
The prevalence of fever recurrence, ICU admission, death, and infection-related death are presented in Table 4. Due to data sparsity, we were unable to estimate the odds of ICU admission, death and infection-related death in episodes with and without CPG-inconsistent care. Table S4 presents the estimated odds ratio and 95% CI for fever recurrence.
TABLE 4.
Prevalence of patient outcomes following episodes of fever with neutropenia
| Patient Outcome | Number of applicable episodes* | Number of applicable episodes with outcome (%) |
|---|---|---|
| Recurrence of fever within 7 days of antibiotic discontinuation | 140 | 18 (12.9) |
| ICU admission during episode or within 7 days | 153 | 19 (12.4) |
| Any death | 157 | 1 (0.6) |
| Infection-related death | 157 | 1 (0.6) |
Number of applicable episodes is presented in Figure 1
No statistically significant associations between CPG-inconsistent care (initial empiric antibiotic selection, empiric antibiotic therapy duration or pulmonary CT) and fever recurrence were observed.
CPG-inconsistent care and characteristics of sites and patients
Post hoc analysis of site characteristics other than size, revealed no association with CPG-inconsistent care delivery. (see Supplementary Table S3) In the post hoc analysis of associations between CPG-inconsistent care delivery and patient characteristics (Area Deprivation Index, Hispanic ethnicity, proportion below poverty level and proportion of vacant housing), we estimated that higher Area Deprivation Index was associated with higher odds of omission of pulmonary CT during episodes with persistent FN (OR 2.07, 95% CI 1.10, 3.87). Similarly, higher levels of poverty were associated with higher odds of omission of pulmonary CT (OR 2.40, 95% CI 1.04, 5.54) (see Supplementary Table S3).
Discussion
This retrospective study evaluated the care delivered in NCORP sites to pediatric cancer patients with FN. It establishes an important baseline for future work focusing on improving the quality of supportive care of pediatric patients receiving care at NCORP sites. In most FN episodes, care was CPG-inconsistent for at least one of five strong CPG recommendations. Delivery of CPG-inconsistent care was not associated with site size, when adjusted for Area Deprivation Index, site type (private or academic), and community served (Minority/Underserved or not). Notably, the number of episodes available for analyses associating CPG-inconsistent care with patient outcomes was low, limiting the interpretation of these findings. Nevertheless, CPG-inconsistent care was not associated with fever recurrence. Omission of pulmonary CT during episodes involving persistent FN was associated with two patient characteristics: Area Deprivation Index and proportion below poverty level. Since these results are based on post hoc analyses, these associations should be further explored in future work.
In our study cohort, sources of discordance between antibiotic administration and CPG recommendations varied. CPG-inconsistent care included initiation of broader therapy than recommended (e.g., inclusion of vancomycin or two gram-negative agents), administration of expedient but insufficient therapy (e.g. ceftriaxone use at presentation) or discontinuation of appropriately initiated therapy too late (e.g., after neutrophil count recovery) or too early (e.g., before neutrophil count recovery). We speculate that ceftriaxone, a third-generation cephalosporin, may have been given at presentation in an effort to meet expectations for time-to-antibiotic administration.17
While we did not observe statistically significant relationships between CPG-inconsistent care delivery and subsequent poor patient outcomes (fever recurrence) during the FN episode or immediately following, estimated odds ratios were substantially higher than one for two CPG recommendations: initial antibiotic selection and antibiotic duration. Wright et al10 reported that the use of CPG-consistent antibiotics in adults at low risk of poor FN outcomes reduced in-hospital mortality (OR 0.65, 95% CI, 0.42–0.95), but not among those at high risk of poor FN outcomes. Our results are not comparable to this adult cohort since we limited our cohort to patients at high risk of poor FN outcomes. Our inability to identify a statistically significant association between CPG-inconsistent care and the selected patient outcomes may be, in part, due to the limited number of events.
We assessed strong CPG recommendations, meaning that they could be expected to be implemented as policy with little need for institution-based or patient-based adaptation. They also were recommendations for which information could be reliably abstracted from the electronic health record. Supportive care CPGs and the COG’s Supportive Care Guideline Task Force were both relatively new to pediatric oncology at the start of our study period. The pediatric FN CPG was first published in September 2012 and endorsed by the COG in February 2014. Although COG-member institutions have been quick to adopt new evidence into cancer treatment protocols,11 the same may not be the case for supportive care CPGs. Low uptake of CPG recommendations into practice is common.12–14 For example, a study evaluating the quality of care in the United States identified that for acute care, in general (not restricted to cancer), 30% of patients do not receive CPG-recommended care.15,16 Surveys of management practices of pediatric cancer patients in other countries have shown inconsistencies with recommended practices that are similar to our findings and calls for CPG implementation are routine.17,18 CPG implementation is notoriously difficult and various strategies have been studied to improve utilization.19,20 For example, in the opinion of health care professionals who provide care at COG-member NCORP institutions, supportive care CPG implementation is facilitated by organizational prioritization and a culture of evidence-based care delivery and of valuing care standardization.21
COG-member institutions and individual clinicians may implement supportive care at their discretion unless specifically dictated by the COG trial in which a patient is enrolled. Implementation of COG-endorsed supportive care CPGs is not mandatory nor is it required that COG-member institutions delay CPG implementation for COG endorsement. Thus, there was a 15-month gap between publication of the 2012 FN CPG and the start of our study period during which COG-member sites and individual clinicians were free to incorporate the CPG recommendations into standards of care.
Notably, FN CPG updates were published in 2017 and 2023.8,9 The latest update continues to recommend against the initial empiric use of vancomycin. Further, it recommends against the addition of a second antibiotic with gram negative activity in stable patients outside of sites with a high prevalence of resistance and against the use of antibiotic agents for initial therapy other than an antipseudomonal penicillin, fourth generation cephalosporin or carbapenem.9 A significant change between the 2012 and 2023 FN CPGs is the allowance for initial antibiotic therapy discontinuation after a minimum of 48 hours in stable patients at low risk of poor FN outcomes without bone marrow recovery. All patients eligible for inclusion in this study were at high risk of poor FN outcomes and, thus, would not have been eligible for this shorter duration recommendation.
Our findings are an important alert for the need for antibiotic and resource stewardship within pediatric oncology. Initial empiric antibiotic therapy was CPG-inconsistent in almost half of the episodes with the primary source of inconsistency being empiric vancomycin use. Findings such as the CPG-inconsistent discontinuation of empiric antibiotic agents before marrow recovery, as defined in this study, and omission of pulmonary CT in patients with persistent FN indicate that equipoise may exist on these questions. This variation in practice, particularly with respect to pulmonary CT in patients with persistent fever, may offer an opportunity to address these evidence gaps with prospective studies.
The strengths of our study include its description of the care delivered in a large number of diverse NCORP institutions pertaining to five strong recommendations of a COG-endorsed CPG. In addition, the rigorous methods used for creation of care episode summaries and their adjudication increase the validity of our findings. However, our interpretation of the care delivered was limited by the existing health record documentation at participating sites and the accuracy of data abstraction. As well, we did not know the site-specific prevalence of resistant pathogens nor when a resistant infection was suspected by the treating team. Thus, our estimates of CPG-inconsistent care related to initial vancomycin use should be viewed cautiously. Our definition of neutrophil count recovery may have led to misclassification of the empiric antibiotic therapy duration being too short. Also, precision of estimates and analysis of the associations between CPG-inconsistent care and patient outcomes were limited by the low number of outcome events and the relatively small number of patients included. More precise estimates would have been possible had we included more patients. Further, since data collection was limited to seven days following the end of the FN episode, we were unable to describe late sequelae of CPG-inconsistent care such as antibiotic resistance and IFD. Since we conducted our study in NCORP institutions, our results may not be generalizable to non-NCORP institutions. Lastly, the currency of our findings relative to contemporary supportive care, although explained by the multi-layered process involved in conducting a multi-center, consortium study, may be questioned. It is possible that penetration of the recommendations is now better. However, given the very large proportion of CPG-inconsistent care (85%) reported in our cohort and the high rate of CPG-inconsistent management of FN reported recently in other jurisdictions,22 our message regarding the need to emphasize antibiotic stewardship and CPG-consistent care in the management of pediatric oncology patients remains pertinent.
Pediatric patients at high risk of poor FN outcomes cared for in participating NCORP sites in 2014 and 2015 often received CPG-inconsistent care. An association between receipt of CPG-inconsistent care and poor outcomes was not detected but this analysis was curtailed by the low number of outcome events. Findings regarding the association between patient characteristics and receipt of CPG-inconsistent care from the post hoc analyses need to be confirmed in future work. Our findings highlight opportunities to boost supportive care CPG implementation among the pediatric oncology community to optimize resource stewardship.
Supplementary Material
Acknowledgements
We appreciate the work of C. Bergheimer, E. Ha and D. Ross for organizing study activities and of the clinical research staff at participating sites for identifying care episodes and preparing the required documents. We are also thankful to A. Sivananthan and N. Stesco and A. Vennettilli for preparing the episode summaries for adjudication and for N. Kaur for administrative assistance. We thank Dr. H. Dang for assistance with statistical methods.
Funding
This work was supported by the National Cancer Institute of the National Institutes of Health by a grant (NCORP Grant UG1CA189955) to the Children’s Oncology Group. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Abbreviations:
- ALL
acute lymphoblastic leukemia
- AML
acute myelogenous leukemia
- COG
Children’s Oncology Group
- CPG
clinical practice guideline
- CT
computerized tomography
- FN
fever and neutropenia
- HCT
hematopoietic cell transplant
- ICU
intensive care unit
- IRB
institutional review board
- NCI
National Cancer Institute
- NCORP
National Cancer Institute Committee Oncology Research Program
Footnotes
Conflict of Interest Statement
Potential conflicts of interest: L. Lee Dupuis received salary support from the Children’s Oncology Group; funds were received by her institution.
Potential conflicts of interest: Brian Fisher has received grants from Pfizer, Merck, Allovir, NIH and has had a contract with the Food and Drug Administration, USA. All funds were received by his institution.
Potential conflicts of interest: A. Sugalski received salary support from the Children’s Oncology Group. Funds were received by his institution.
Potential conflicts of interest: A. Grimes received grant funding to their institution from the National Institutes of Health, Canadian Institute of Health Research, American Cancer Society and Cancer Prevention Research Institution of Texas. She also received payment from Servier Pharmaceuticals for services as a content expert, the Clark Hill Law Firm for expert testimony. A. Grimes also received salary support from the Children’s Oncology Group; funds were received by their institution.
Potential conflicts of interest: M. Nuño received salary support to their institution from the National Cancer Institute and grant funding to their institution from the National Cancer Institute, Department of Defense and St Baldrick’s Foundation.
Potential conflicts of interest: S. Parsons has received consulting fees from Seagen unrelated to the work described in this manuscript and is an external member of the Dana-Farber/Harvard Cancer Center Data
Safety Monitoring Board.
No conflicts: M. Beauchemin, P.D. Robinson, N. Santesso, L. Yu, A.R. Wrightson, A. Walsh.
References
- 1.Clinical Practice Guidelines: Directions for a New Program, Institute of Medicine. Washington, DC: National Academy Press; 1990. [Google Scholar]
- 2.Lehrnbecher T, Phillips R, Alexander S, et al. Guideline for the management of fever and neutropenia in children with cancer and/or undergoing hematopoietic stem-cell transplantation. Journal of clinical oncology. 2012;30(35):4427–4438. [DOI] [PubMed] [Google Scholar]
- 3.Hughes W, Armstrong D, Bodey G, et al. 2002 Guidelines for the use of antimicrobial agents in neutropenic patients with cancer. Clin Infect Dis. 2002;34(6):730–751. [DOI] [PubMed] [Google Scholar]
- 4.Lehrnbecher T, Aplenc R, Rivas Pereira F, et al. Variations in non-pharmacological anti-infective measures in childhood leukemia--results of an international survey. Haematologica. 2012;97(10):1548–1552. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Bmj. 2008;336(7650):924–926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ren S, Lai H, Tong W, Aminzadeh M, Hou X, Lai S. Nonparametric bootstrapping for hierarchical data. Journal of Applied Statistics. 2010;37(9):1487–1498. [Google Scholar]
- 7.Brokamp C, Beck AF, Goyal NK, Ryan P, Greenberg JM, Hall ES. Material community deprivation and hospital utilization during the first year of life: an urban population-based cohort study. Ann Epidemiol. 2019;30:37–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lehrnbecher T, Robinson P, Fisher B, et al. Guideline for the Management of Fever and Neutropenia in Children With Cancer and Hematopoietic Stem-Cell Transplantation Recipients: 2017 Update. J Clin Oncol. 2017;35(18):2082–2094. [DOI] [PubMed] [Google Scholar]
- 9.Lehrnbecher T, Robinson PD, Ammann RA, et al. Guideline for the Management of Fever and Neutropenia in Pediatric Patients With Cancer and Hematopoietic Cell Transplantation Recipients: 2023 Update. J Clin Oncol. 2023;41(9):1774–1785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wright JD, Neugut AI, Ananth CV, et al. Deviations from guideline-based therapy for febrile neutropenia in cancer patients and their effect on outcomes. JAMA internal medicine. 2013;173(7):559–568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Phillips CA, Barakat LP, Pollock BH, Bailey LC, Beidas RS. Implementation science in pediatric oncology: A narrative review and future directions. Pediatric blood & cancer. 2022;69(4):e29579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.FitzGerald JM, Boulet LP, McIvor RA, Zimmerman S, Chapman KR. Asthma control in Canada remains suboptimal: the Reality of Asthma Control (TRAC) study. Canadian respiratory journal : journal of the Canadian Thoracic Society. 2006;13(5):253–259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Grimshaw J, Eccles M, Thomas R, et al. Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966–1998. Journal of general internal medicine. 2006;21 Suppl 2:S14–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Latosinsky S, Fradette K, Lix L, Hildebrand K, Turner D. Canadian breast cancer guidelines: have they made a difference? CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne. 2007;176(6):771–776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Berwick DM, Hackbarth AD. Eliminating waste in US health care. Jama. 2012;307(14):1513–1516. [DOI] [PubMed] [Google Scholar]
- 16.Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? The Milbank quarterly. 1998;76(4):517–563, 509. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Haeusler GM, Slavin MA, Bryant PA, Babl FE, Mechinaud F, Thursky KA. Management of fever and neutropenia in children with cancer: A survey of Australian and New Zealand practice. Journal of Paediatrics and Child Health. 2018;54(7):761–769. [DOI] [PubMed] [Google Scholar]
- 18.Scheler M, Lehrnbecher T, Groll AH, et al. Management of children with fever and neutropenia: results of a survey in 51 pediatric cancer centers in Germany, Austria, and Switzerland. Infection. 2020;48:607–618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Chan WV, Pearson TA, Bennett GC, et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI Implementation Science Work Group: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Journal of the American College of Cardiology. 2017;69(8):1076–1092. [DOI] [PubMed] [Google Scholar]
- 20.Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implementation science : IS. 2011;6:26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sugalski AJ, Lo T, Beauchemin M, et al. Facilitators and barriers to clinical practice guideline-consistent supportive care at pediatric oncology institutions: a Children’s Oncology Group study. Implementation Science Communications. 2021;2:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Papan C, Reifenrath K, Last K, et al. Antimicrobial use in pediatric oncology and hematology in Germany and Austria, 2020/2021: a cross-sectional, multi-center point-prevalence study with a multi-step qualitative adjudication process. Lancet Reg Health Eur. 2023;28:100599. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.