Abstract
Background
The performance of serum biomarkers for the early detection of invasive aspergillosis expectedly depends on the timing of test results relative to the empirical administration of antifungal therapy during neutropenia, although a dynamic evaluation framework is lacking.
Methods
We developed a multi-state model describing simultaneously the likelihood of empirical antifungal therapy and the risk of invasive aspergillosis during neutropenia. We evaluated whether the first positive test result with a biomarker is an independent predictor of invasive aspergillosis when both diagnostic information used to treat and risk factors of developing invasive aspergillosis are taken into account over time. We applied the multi-state model to a homogeneous cohort of 185 high-risk patients with acute myeloid leukemia. Patients were prospectively screened for galactomannan antigenemia twice a week for immediate treatment decision; 2,214 serum samples were collected on the same days and blindly assessed for (1->3)- β-D-glucan antigenemia and a quantitative PCR assay targeting a mitochondrial locus.
Results
The usual evaluation framework of biomarker performance was unable to distinguish clinical benefits of β-glucan or PCR assays. The multi-state model evidenced that the risk of invasive aspergillosis is a complex time function of neutropenia duration and risk management. The quantitative PCR assay accelerated the early detection of invasive aspergillosis (P = .010), independently of other diagnostic information used to treat, while β-glucan assay did not (P = .53).
Conclusions
The performance of serum biomarkers for the early detection of invasive aspergillosis is better apprehended by the evaluation of time-varying predictors in a multi-state model. Our results provide strong rationale for prospective studies testing a preemptive antifungal therapy, guided by clinical, radiological, and bi-weekly blood screening with galactomannan antigenemia and a standardized quantitative PCR assay.
Introduction
The evaluation of biomarker performance to detect invasive aspergillosis is often limited by the absence of the gold standard for diagnosis and the empirical administration of antifungal therapy for a persistent or recurrent fever. As a case study, patients treated with chemotherapy for acute myeloid leukemia are at high risk of developing invasive aspergillosis that portends poor prognosis [1]. Invasive aspergillosis is rarely proven by direct microbiological detection or autopsy, and often remains a possible or a probable diagnosis depending on a combination of clinical, radiological, and microbiological criteria [2], [3]. Moreover, the risk of invasive aspergillosis evolves over time as it increases with neutropenia duration [4], [5], while it purposely decreases with the empirical administration of antifungal therapy [6]. Accordingly, the performance of biomarkers to detect invasive aspergillosis depends on the timing of test results during neutropenia [7]. The need to evaluate biomarker performance is particularly important in untreated patients because serial blood screening may accelerate the early detection of invasive aspergillosis, independently of other diagnostic information [8]–[11].
Aspergillus galactomannan antigen is the biomarker that is most often used in current practice to detect invasive aspergillosis in leukemic patients [12]. Galactomannan antigen detection was added by consensus to the indirect microbiological criteria of a probable invasive fungal disease in 2002 [2]. Serial blood screening with galactomannan antigen has been increasingly used to guide the early administration of antifungal therapy [13], [14] and to monitor treatment [15], [16]. The experience with (1->3)-β-D-glucan (β-glucan) antigenemia or PCR assays is more limited. β-glucan antigenemia was recently incorporated in the definition of a probable invasive fungal disease [3], although β-glucan is not specific of Aspergillus spp. [17], [18]. PCR assays targeting Aspergillus DNA are even less accepted in routine practice given the lack of common protocols [3].
Meta-analyses found a marked heterogeneity between diagnostic studies for all biomarkers [19]–[23]. Several time-invariant confounders of biomarker performance were consistently identified: distinct features of each biomarker such as the cutoff level used to define a positive test result; the criteria used to define invasive fungal diseases; and the enrollment of patients at different risks of developing invasive aspergillosis. As for time-varying confounders, the definition of a positive test result by two consecutive positive samples drastically reduced heterogeneity between diagnostic studies; however, a more stringent definition implies a lower sensitivity to detect invasive aspergillosis. Overall, the evaluation of biomarker performance according to the timing of test results was underreported in diagnostic studies. Marr et al. found that the sensitivity of galactomannan antigenemia was significantly decreased when anti-mold drugs are administered [24], and the performance of β-glucan antigenemia or PCR assays is thought to be similarly altered [21]–[23].
In the present study, we explored whether β-glucan antigenemia or PCR assays may independently detect invasive aspergillosis when risk management relies on serial blood screening with galactomannan antigenemia. We selected all febrile, neutropenic patients with acute myeloid leukemia from a large clinical trial that included the prospective collection of serum samples twice a week [25]. Because the trial design eventually involved delaying early antifungal therapy in the preemptive arm as compared to the empirical arm [25], we were able to evaluate the performance of β-glucan antigenemia and PCR assays in a homogeneous cohort with extended duration of neutropenia before treatment. We initially evaluated biomarker performance overall, and according to two restrictive approaches with selection of serum samples for evaluation: an early detection approach before treatment, and a confirmatory diagnosis approach in presence of a positive galactomannan antigenemia. Then, we performed a multi-state model describing simultaneously the likelihood of empirical antifungal therapy and the risk of invasive aspergillosis during neutropenia, and we evaluated whether β-glucan antigenemia and PCR assays may accelerate the early detection of invasive aspergillosis when both diagnostic information used to treat and risk factors of developing invasive aspergillosis are controlled for.
Methods
Patients
The adult patients evaluated in this study had been enrolled in a prospective, randomized, open-label, non-inferiority trial conducted from April 2003 to February 2006 in 13 French teaching hospitals (ClinicalTrials.gov Identifier: NCT001190463). The primary objective was to compare overall survival following either empirical or preemptive antifungal therapy in patients treated for hematological malignancies. In the empirical therapy arm, persistent or recurrent fever after day 4 of broad spectrum antibacterials led to the administration of antifungal therapy. In the preemptive therapy arm, the initiation of antifungal therapy was guided by clinical and radiological predefined criteria, and bi-weekly blood screening with galactomannan antigenemia. Patients received early antifungal therapy according to the trial protocol, either amphotericin B deoxycholate (1 mg/kg/d) or liposomal amphotericin B (3 mg/kg/d) depending on the level of creatinine clearance and concomitant nephrotoxic drugs. Results of the clinical trial have been reported [25]. The clinical trial and the present study had been approved by the ethics committee of Henri Mondor Teaching Hospital.
Of 293 patients enrolled in the trial, we selected all 185 (63%) febrile, neutropenic patients treated with chemotherapy for acute myeloid leukemia to constitute a homogeneous group at high risk of developing invasive aspergillosis. Exclusions involved: 6 patients without neutropenia; 8 neutropenic patients without any fever; 93 febrile, neutropenic patients treated for hematological malignancies other than acute myeloid leukemia; and 1 patient with acute myeloid leukemia who had no blood sample collected. As compared to excluded patients, the homogeneous cohort selected for the present study had prolonged neutropenia (median duration: 23 days vs. 11 days; P<.001) and a higher risk of developing invasive aspergillosis (11 (6.0%) vs. 1 (0.9%); P = .06), despite a higher rate of early antifungal therapy with intravenous amphotericin B (112 (60.5%) vs. 26 (24.1%); P<.001).
Antigen and DNA Detection Assays
Blood samples were prospectively collected twice a week during trial enrollment, and processed to serum samples. A portion of serum was immediately screened with ELISA Aspergillus galactomannan antigenemia (Platelia Aspergillus, Biorad, France), and GM index results were available to clinicians within 24 hours for treatment decision according to the trial protocol [25]. The remaining serum was stored at −70°C in two different aliquots.
For the present study, two blinded authors processed all 2,214 serum samples after trial completion. β-glucan was assayed on a specific aliquot using the Fungitell test (Associates of Cape Cod, Inc, Falmouth, MA) according to the manufacturer’s specifications; a positive test result was defined as a level of (1,3)-β-D glucan ≥80 pg/ml. After thawing, DNA extraction was performed from 1 ml serum with the MagNA Pure LC DNA as described previously [26]. A quantitative PCR (qPCR) assay targeting a mitochondrial DNA sequence of Aspergillus was evaluated [27]. This test included the uracil-N-glycosylase use for preventing amplicon contamination, and an internal control based on the amplification of mouse DNA to minimize the risk of false-negative test results [28]–[30].
Invasive Aspergillosis Case Definition
In the clinical trial, proven and probable invasive aspergilloses were defined by an independent blinded adjudication committee according to EORTC/MSG consensus criteria of 2002 [2]. Invasive aspergillosis was considered as ‘baseline’ for those documented by procedures before or within 24 h after the first dose of early antifungal therapy, and ‘breakthrough’ otherwise.
Performance of β-glucan Antigenemia and PCR Assays
We initially evaluated biomarker performance as usually done at the patient level. At the core of the evaluation is the definition of a patient tested positive for invasive aspergillosis. We considered that a patient with at least one positive test result is tested positive to maximize the sensitivity of β-glucan, qPCR, or both when used in combination. We evaluated biomarker performance overall with use of all samples collected during study enrollement, and according to two restrictive approaches with selection of serum samples to underline the importance of the timing of test results on biomarker performance: 1) an early detection approach with selection of samples collected after neutropenia or fever onset, and before treatment, invasive fungal disease, or neutropenia recovery; 2) a confirmatory diagnosis approach of a positive galactomannan antigenemia with selection of samples collected on the same sample or the consecutive sample of a GM index ≥0.5.
Then, we evaluated biomarker performance as a time-varying predictor at the patient level. At the core of the evaluation is whether the first positive test result with β-glucan or qPCR is an independent predictor of invasive aspergillosis when both diagnostic information used to treat and risk factors of developing invasive aspergillosis are taken into account over time. We developed a multi-state model describing the event history of the patients in continuous time, where events with inherent dependence are defined by the daily transitions between 3 distinct states: (1) ‘no antifungal therapy’ (2) ‘antifungal therapy’ and (3) ‘invasive aspergillosis’ ( Figure 1 ). All patients entered the model in the state of ‘no antifungal therapy’ on the first day of neutropenia or fever onset, whichever occurred first, as it may be considered as the starting point of invasive aspergillosis risk in febrile, neutropenic patients. Patients in the initial state free of antifungal therapy could: (a1) develop invasive aspergillosis (i.e., ‘baseline’ invasive aspergillosis); (b1) get treated; or (c1) remain untreated. Once antifungal therapy was started, patients could: (a2) develop invasive aspergillosis despite treatment (i.e., ‘breakthrough’ invasive aspergillosis); (b2) be removed from treatment; or (c2) remain treated. Invasive aspergillosis was the primary outcome of the multi-state model and considered as an absorbing state with no follow-up; otherwise, daily transitions were observed until neutropenia recovery, candidemia or death from causes other than invasive aspergillosis.
Figure 1. Multi-State Model of Invasive Aspergillosis and Antifungal Therapy in Febrile, Neutropenic Patients with Acute Myeloid Leukemia.
A semi-Markov process was used for the multi-state model, where each transition is specified as a separate hazard function and described by an accelerated failure time model. The choice of accelerated failure time models was guided by empirical findings suggesting that each hazard rate depends not only on observed characteristics, but also on the time elapsed in each state: (a1) the risk of invasive aspergillosis increases with neutropenia duration in patients treated with chemotherapy for acute myeloid leukemia [4], [5]; by contrast, (a2) the risk of invasive aspergillosis becomes flat or may even decrease during antifungal therapy as a result of its preventive efficacy [6]; (b1) the likelihood to start empirical antifungal therapy increases with neutropenia duration as clinicians may suspect the risk of invasive aspergillosis and other invasive fungal diseases; (b2) the likelihood to stop empirical antifungal therapy increases with treatment duration as treatment is stopped at neutropenia recovery.
The multi-state model allowed for examination of the independent effect of explanatory variables on each hazard function. We checked whether the first positive galactomannan antigenemia available to clinicians increased the likelihood to start antifungal therapy, and we evaluated whether the first positive test result with β-glucan or qPCR may accelerate the early detection of invasive aspergillosis, i.e., the biomarker was independently associated with an increased risk of invasive aspergillosis before antifungal therapy was started. The evaluation of biomarker performance was controlled for selected confounders on each hazard function. Time-invariant confounders included: age; primary antifungal prophylaxis; and preemptive therapy arm of the trial. Time-varying confounders included: daily assessment of fever above 38°C after day 4 of fever; time to first clinical or radiological pulmonary signs; and timing and length of stay in laminar air-flow rooms or other protective environment rooms with high-efficiency particulate air (HEPA) filters [31]–[33].
Statistical Analysis
For the evaluation of biomarker performance as usually done, we calculated sensitivity and specificity according to the proportion of patients with true and false positive and negative test results. Ninety-five percent confidence intervals [95% CIs] for sensitivity and specificity were estimated using exact binomial distributions. In addition, we assessed overall biomarker performance by calculating the Youden index (sensitivity+specificity-1) that varies from 0 (no diagnostic accuracy) to 1 (perfect diagnostic accuracy).
For the multi-state model, we estimated simultaneously all four hazard functions under the specifications of each accelerated failure time model and a Weibull parameterization of each baseline hazard [34]–[36]. The family of parametric models as the one used in this analysis not only allows the estimation of the effect of explanatory variables on the hazard rate, but also the estimation of the effect of the time elapsed in each state. The latter effect known as duration dependence is estimated by means of the shape parameter whose value describes how the hazard rate changes over time. A shape parameter equal to 1 indicates a hazard rate that does not vary over time. By contrast, a value above (below) 1 implies a low (high) variability, indicating that patients will transition from one state to another within a relatively short (long) time span. The regression coefficient of an explanatory variable directly measures the proportionate change in the hazard function for a unit change of the explanatory variable, all other things being equal. All analyses were performed using R version 2.11.0 (R Development Core Team, 2010).
Results
Table 1 presents the characteristics of the 185 febrile, neutropenic patients with acute myeloid leukemia selected for the present study as well as the risk management of invasive aspergillosis during neutropenia.
Table 1. Characteristics of 185 febrile, neutropenic patients treated for acute myeloid leukemia and by invasive aspergillosis.
| Characteristic of Patient and Risk Management of InvasiveAspergillosis | Overall,N = 185 | InvasiveAspergillosisa, N = 11 | No Invasive Aspergillosis,N = 174 | p-valueb |
| Age, mean (SD), yr | 54 (14) | 62 (8) | 53 (14) | .030 |
| Female | 88 (48) | 5 (46) | 83 (48) | .89 |
| Remission-induction chemotherapy for new diagnosis or first relapse | 134 (72) | 10 (91) | 124 (71) | .29 |
| Duration of neutrophil count <500/mm3, median (IQR), days | 23 (16–30) | 26 (20–32) | 23 (15–30) | .17 |
| Protective environment room: | ||||
| Laminar air-flow room | 63 (34) | 1 (9) | 62 (36) | |
| Other protective environment room with HEPA filters | 39 (21) | 1 (9) | 38 (22) | |
| Non-protective environment room | 83 (45) | 9 (82) | 74 (42) | .013 |
| Primary antifungal prophylaxis c | 77 (42) | 4 (36) | 73 (42) | .76 |
| Antifungal therapy with intravenous amphotericin B d | 112 (61) | 4 (36) | 108 (62) | .11 |
| Candidemia | 4 (2) | 0 | 4 (2) | 1.00 |
| Death at end of study e | 8 (4) | 2 (18) | 6 (3) | .07 |
Abbreviations: IA: invasive aspergillosis; IQR: interquartile range; HEPA: High-Efficiency Particulate Air.
Of 11 IA, the independent blinded adjudication committee of the trial defined 2 proven IA and 9 probable IA according to international consensus definitions of 2002.
By chi-square test or exact Fisher test for binary variables; by Wilcoxon sum-rank test for continuous variables.
Prophylaxis included oral amphotericin B (n = 54), fluconazole (n = 23), and/or itraconazole (n = 10).
Of 11 IA, 7 IA were documented by procedures before or within 24 h after the first dose of antifungal therapy, and 4 breakthrough IA occurred after antifungal therapy was started.
Causes of death included 2 IA, 4 bacterial sepsis, 1 cardiogenic shock, and 1 coma of unknown origin. Follow-up was censored at 14 days after neutropenia recovery or at 60 days of neutropenia.
Table 2 presents biomarker performance overall, and according to two restrictive approaches with selection of serum samples for evaluation. Overall (median (IQR) of 7 (5–9) samples per patient), all 11 patients with invasive aspergillosis had at least 1 positive test result with β-glucan or qPCR: sensitivity of each biomarker was 82% and 73%, respectively, although specificity was below 50%. In comparison, the selection of serum samples in an early detection approach (median (IQR) of 3 (2–5) samples per patient) was associated with a lower performance of each biomarker before treatment (Youden index below 0.15) as explained by a sharp loss in sensitivity (45% and 36%, respectively). The selection of 59 serum samples in a confirmatory diagnosis approach of a positive galactomannan antigenemia was associated with the best performance of each biomarker (Youden index above 0.45) as explained by an increased specificity above 90%.
Table 2. Performance of β-glucan and qPCR, overall and according to two restrictive approaches of invasive aspergillosis diagnosis with selection of serum samples.
| Definition of a Patient Tested Positivefor Invasive Aspergillosis | No. patients tested positive/No. patients with IA | Sensitivity, % (95% CI) | No. patients tested negative/No. patients without IA | Specificity, % (95% CI) | Youden indexa | |
| Overall performance: all serum samples collected during study enrollment (n = 1,534) | ||||||
| ≥1 positive test with ß-glucan | 9/11 | 82 (48–98) | 79/174 | 45 (38–53) | 0.27 | |
| ≥1 positive test with qPCR | 8/11 | 73 (39–94) | 79/174 | 45 (38–53) | 0.18 | |
| ≥1 positive test with ß-glucan or qPCR | 11/11 | 100 (72–100) | 39/174 | 22 (16–29) | 0.22 | |
| Early detection of IA: selection of serum samples before early antifungal therapy b (n = 747) | ||||||
| ≥1 positive test with ß-glucan | 5/11 | 45 (17–77) | 113/172 | 66 (58–73) | 0.11 | |
| ≥1 positive test with qPCR | 4/11 | 36 (11–69) | 113/172 | 66 (58–73) | 0.02 | |
| ≥1 positive test with ß-glucan or qPCR | 7/11 | 64 (31–89) | 76/172 | 44 (37–52) | 0.08 | |
| Confirmatory diagnosis of IA: selection of serum samples in presence of a positive GM test result c (n = 59) | ||||||
| ≥1 positive test with ß-glucan | 6/11 | 55 (23–83) | 159/174 | 91 (86–95) | 0.46 | |
| ≥1 positive test with qPCR | 6/11 | 55 (23–83) | 162/174 | 93 (88–96) | 0.48 | |
| ≥1 positive test with ß-glucan or qPCR | 7/11 | 64 (31–89) | 152/174 | 87 (81–92) | 0.51 | |
Abbreviations: IA: Invasive Aspergillosis; GM: Aspergillus galactomannan antigenemia; ß-glucan: (1,3)-b-D glucan antigenemia; qPCR: PCR assay targeting Aspergillus fumigatus mitochondrial DNA.
Youden index is calculated as follows: Sensitivity+Specificity-1; and varies from 0 (no diagnostic accuracy) to 1 (perfect diagnostic accuracy).
Selection of serum samples after neutropenia/fever onset, and before antifungal therapy, invasive fungal infection, or neutropenia recovery.
Selection of serum samples collected on the same sample or the consecutive sample of a GM positive result as defined by a GM index ≥0.5 according to the manufacturer’s specifications.
Table 3 presents the multi-state model describing the event history of the 185 febrile, neutropenic patients in continuous time. Several observations confirmed the relevance and validity of this approach. The baseline hazard of invasive aspergillosis increased with neutropenia duration before antifungal therapy was started (Weibull shape parameter = 1.89; P<.001), and significantly decreased during treatment (Weibull shape parameter = .97; P<.001). According to the trial protocol, the empirical administration of antifungal therapy was guided by the diagnostic information available to clinicians: the hazard rate for starting treatment increased with persistent or recurrent fever (acceleration factor = 1.21 [95% CI, 1.16–1.26] for each day of fever after day 4 of broad spectrum antibacterials), the first day of a radiological pulmonary sign (acceleration factor = 1.66 [95% CI, 1.37–2.02]), or the first day of a positive galactomannan antigenemia (acceleration factor = 1.25 [95% CI, 1.11–1.42]); the hazard rate for stopping treatment increased with the first day of fever resolution (acceleration factor = 1.07 [95% CI, 1.01–1.15]). In accordance with trial results, the preemptive therapy arm halved the hazard rate for starting treatment as compared to the empirical therapy arm of the trial (acceleration factor for starting treatment = 0.53 [95% CI, 0.50–0.55]), and treatment duration was consequently decreased before neutropenia recovery (acceleration factor for stopping treatment = 1.19 [95% CI, 1.11–1.28]). In addition, the multi-state model revealed that the risk of invasive aspergillosis before treatment was significantly decreased in laminar air-flow rooms (acceleration factor = 0.34 [95% CI, 0.21–0.55] for each day spent in a laminar air-flow room) as compared to non-protective environment rooms.
Table 3. Performance of β-glucan and qPCR to detect invasive aspergillosis in a multi-state model.
| Invasive Aspergillosis Before Therapya1 | Start Empirical Antifungal Therapyb1 | Invasive Aspergillosis During Therapya2 | Stop Empirical Antifungal Therapyb2 | |||||||||
| Baseline hazard | Estimate | P-value | Estimate | P-value | Estimate | P-value | Estimate | P-value | ||||
| Intercept | 6.85 | <.001 | 2.81 | <.001 | 11.07 | <.001 | 3.49 | <.001 | ||||
| Weibull shape parameter c | 1.89 | <.001 | 2.50 | <.001 | 0.97 | <.001 | 2.30 | <.001 | ||||
| Explanatory variables | Estimate | CI 95% | P-value | Estimate | CI 95% | P-value | Estimate | CI 95% | P-value | Estimate | CI 95% | P-value |
| Age, yr | 1.04 | (1.02–1.05) | <.001 | 0.99 | (0.99–0.99) | <.001 | 1.07 | (1.02–1.13) | 1.13 | 1.01 | (1.01–1.01) | <.001 |
| Diagnostic information: | N/E | |||||||||||
| Persistent or recurrent fever | 1.17 | (0.91–1.50) | .23 | 1.21 | (1.16–1.26) | <.001 | 0.93 | (0.87–0.99) | .023 | |||
| Clinical pulmonary sign | N/A | 1.23 | (0.69–2.20) | .49 | N/A | |||||||
| Radiological pulmonary sign | N/A | 1.66 | (1.37–2.02) | <.001 | N/A | |||||||
| Aspergillus GM antigenemia | N/A | 1.25 | (1.11–1.42) | <.001 | N/A | |||||||
| Preemptive therapy arm of the trial | N/A | 0.53 | (0.50–0.55) | <.001 | 1.19 | (1.11–1.28) | <.001 | |||||
| β-glucan antigenemia | 1.28 | (0.57–2.89) | .53 | N/A | N/A | |||||||
| Aspergillus qPCR | 2.10 | (1.20–3.66) | .010 | N/A | N/A | |||||||
| Preventive interventions other than antifungal therapy: | N/E | |||||||||||
| Laminar air-flow room | 0.34 | (0.21–0.55) | <.001 | 0.84 | (0.80–0.88) | <.001 | 0.80 | (0.75–0.86) | <.001 | |||
| Other protective environment room with HEPA filters | 0.89 | (0.61–1.31) | .56 | 1.03 | (0.98–1.09) | .25 | 1.28 | (1.17–1.39) | <.001 | |||
| Primary antifungal prophylaxis | 1.03 | (0.80–1.33) | .80 | 1.10 | (1.05–1.15) | <.001 | 0.73 | (0.68–0.78) | <.001 | |||
Abbreviations: GM: Aspergillus galactomannan antigenemia; ß-glucan: (1,3)-b-D glucan antigenemia; qPCR: PCR assay targeting Aspergillus fumigatus mitochondrial DNA; HEPA: High-Efficiency Particulate Air; N/A: not applicable; N/E: not estimable.
Explanatory variables excluded: 1) diagnostic information available to clinicians during the trial because they contribute eventually to the case definition of invasive aspergillosis; 2) the preemptive therapy arm of the trial because its effect is modeled by the decision to start and then stop antifungal therapy during neutropenia.
Explanatory variables excluded β-glucan and qPCR because they were not available to clinicians during the trial.
Explanatory variables included only age because of the limited number of breakthrough invasive aspergillosis (n = 4).
Explanatory variables excluded diagnostic information other than resolution of fever.
An (exponentiated) Weibull shape parameter above (below) 1 indicates that patients will transition from one state to another within a relatively short (long) time span.
As for the evaluation of biomarker performance as a time-varying predictor in the multi-state model, we found that qPCR assay may accelerate the early detection of invasive aspergillosis, independently of galactomannan antigenemia and other diagnostic information used to start treatment during neutropenia: the first positive test result with qPCR was an independent and significant predictor of invasive aspergillosis before antifungal therapy was started (P = .010), while β-glucan was not (P = .53). Similar results of biomarker performance were found whether a higher cutoff level (GM index ≥1.5) had been used to define a positive galactomannan antigenemia in serial blood screening (data not shown).
Discussion
Meta-analyses of biomarker performance to detect invasive aspergillosis found a marked heterogeneity between diagnostic studies [19]–[23]. Among potential sources of heterogeneity, Marr et al. found that the sensitivity of galactomannan antigenemia was significantly decreased by the administration of anti-mold drugs [24], and recommended evaluating biomarker performance in a dynamic approach accounting for the timing of test results [7]. To our knowledge, we present the first multi-state model allowing the evaluation of biomarker performance as a time-varying predictor of invasive aspergillosis before treatment, while multiple sources of heterogeneity may be controlled for. When the multi-state model was applied to a homogeneous cohort of 185 febrile, neutropenic patients screened for galactomannan antigenemia twice a week, we found that PCR assays may accelerate the early detection of invasive aspergillosis before antifungal therapy was started, while β-glucan would not. By contrast, the usual evaluation framework was unable to distinguish clinical benefits of β-glucan or PCR assays.
We confirm that the performance of biomarkers dramatically depends on the timing of test results relative to an evolving risk of invasive aspergillosis. When considering all serum samples collected during study enrollment, the sensitivity of β-glucan (82%) and qPCR (73%) to detect invasive aspergillosis was in the range of pooled estimates (77% [95% CI, 67–84] without difference in the detection of invasive aspergillosis and candidemia [22], and 88% [95% CI, 75–94] [21], respectively), although specificity of both biomarkers (45%) was significantly lower (85% [95% CI, 80–90] for β-glucan [22], and 75% [95% CI, 63–84] for PCR [21]). Such discrepancy may be attributed to the stringent case definition of invasive aspergillosis used in the trial, and false positive test results occurring outside the risk window of invasive aspergillosis, i.e., about half samples were collected before neutropenia/fever onset, during antifungal therapy, or after neutropenia recovery.
In the usual evaluation framework, two approaches allow decreasing false positive rates, while somewhat coping with the timing of test results. As advocated in recent meta-analyses, the definition of a positive test result may be restricted to two consecutive positive samples: specificity improved by about 10% for both β-glucan [23] and PCR assays [21]; however, biomarker sensitivity to detect invasive aspergillosis decreased by about the same extent, and two consecutive samples imply delaying antifungal therapy in patients who should benefit. Alternatively, evaluation may be restricted to serum samples collected in the risk window of invasive aspergillosis. We found that specificity of both biomarkers increased from 45% to 66%, although at the expense of a sharp loss in sensitivity. In addition, when risk management involves blood screening with galactomannan antigenemia twice a week, reserving β-glucan and/or PCR assays for confirming a positive galactomannan antigenemia may be a more cost-effective use of scare resources as the best overall performance was achieved from the drastic selection of these serum samples (4%, 59/1,534).
The usual evaluation framework is misleading on biomarker performance because neither the evolving risk of invasive aspergillosis nor the related kinetics of biomarkers are actually taken into account [7]. The use of serial blood screening was mainly supported by the general finding that the first positive galactomannan antigenenmia precedes clinical and radiological pulmonary signs of invasive aspergillosis [8], [13], [37]. Given the expected correlation of serum biomarkers among each other, serial blood screening would primarily benefit from additional biomarkers that accelerate the early detection of invasive aspergillosis, although the kinetics of biomarkers were barely compared [9]–[11]. Our multi-state model results provide strong rationale for prospective studies to test whether a refined preemptive therapy, guided by clinical, radiological, and bi-weekly blood screening with galactomannan antigenemia and a standardized PCR assay is not inferior to empirical therapy.
Our diagnostic study presents several strengths: a prospective, multicenter design; a homogeneous cohort of 185 febrile, neutropenic patients with acute myeloid leukemia; a central, blinded assessment of serum biomarkers; and the definition of invasive fungal diseases according to EORTC/MSG consensus criteria. As a limitation, the frequency of invasive fungal diseases was relatively low, and we could not model the competing risk of candidemia. However, invasive aspergillosis is the dominant invasive fungal disease in hematological wards, and candidemia is of less concern because of the broad use of primary prophylaxis with anti-yeast activity [38], [39]. Moreover, the evaluation of biomarker performance was controlled for multiple sources of heterogeneity in the risk management of invasive aspergillosis during neutropenia. Besides empirical antifungal therapy, the risk of invasive aspergillosis was significantly decreased in laminar air-flow rooms [31]–[33], while primary prophylaxis may also have been associated with a risk reduction of invasive aspergillosis whether drugs with anti-mold activity had been used [40].
Overall, our study results support the recent call to report the timing of test results in diagnostic studies [21]–[23]. However, the risk of invasive aspergillosis is a complex time function of neutropenia duration and risk management. Accordingly, the comparison of biomarker performance, and more broadly any preventative strategy of invasive fungal diseases, would benefit from the complete reporting of all known time-varying confounders related to the diagnosis and management of invasive fungal diseases.
Acknowledgments
We thank the investigators of the PREVERT trial: Cécile Pautas MD, Sébastien Maury MD PhD, Yosr Hicheri MD, Françoise Botterel MD PhD, Francoise Foulet MD (Henri Mondor Hospital, Créteil); Anne Vekhoff MD, Driss Chaoui MD, Muriel Cornet MD, Patrice Agnamey MD (Hôtel Dieu Hospital, Paris); Hassan Farhat MD, Sylvie Castaigne MD, Odile Eloy MD (Andre Mignot Hospital, Versailles); Felipe Suarez MD PhD, Agnès Buzyn MD PhD, Richard Delarue MD, Svetlana Challier MD (Necker Hospital, Paris); Nathalie Dhedin MD, Ahmad Aljijakli MD, Emmanuelle Delabesse PD, Annick Datry MD (Pitié-Salpétrière Hospital); Françoise Isnard MD, Loic Fouillard MD, Jean-Yves Poirot MD, Leila Meliani MD (Saint-Antoine Hospital, Paris); Lionel Adès MD, Claire Bouges-Michel MD, Michèle Deniau MD PhD (Avicennes Hospital, Bobigny); Frédérique Kuhnowski MD, François Dreyfus MD PhD, André Paugam MD, Marie-Thérèse Baixench MD (Cochin Hospital, Paris); Roland Leclercq MD PhD, Oumady Reman MD, Chantal Duhamel MD (University Hospital, Caen); Jean-Henri Bourrhis MD PhD, Sami Chehata MD, Isabelle Chachati MD (Gustave Roussy Institute, Villejuif); Vincent Foissaud MD, Christine Macnab MD (Hôpital des Armées, Clamart); Hervé Tilly MD, Stéphane Leprêtre MD, Christian Gray MD (Centre Henri Becquerel, Rouen); Emmanuel Raffoux MD PhD, Claire Lacroix PD (Saint-Louis Hospital, Paris). We also thank Jeremy D Goldhaber-Fiebert PhD and Eran Bendavid MD MS (Stanford Health Policy, Stanford University) for their critical revision of the article (no compensation) as well as Brandon J Farley PhD (University of Provence, Marseille) for his help in copy editing (financial compensation).
Funding Statement
Financial support French Ministry of Health (PHRC 2002 AOR02028). The study sponsor had no involvement in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Contributor Information
Cécile Pautas, Henri Mondor Hospital, Créteil.
Sébastien Maury, Henri Mondor Hospital, Créteil.
Yosr Hicheri, Henri Mondor Hospital, Créteil.
Françoise Botterel, Henri Mondor Hospital, Créteil.
Francoise Foulet, Henri Mondor Hospital, Créteil.
Anne Vekhoff, Hôtel Dieu Hospital, Paris.
Driss Chaoui, Hôtel Dieu Hospital, Paris.
Muriel Cornet, Hôtel Dieu Hospital, Paris.
Patrice Agnamey, Hôtel Dieu Hospital, Paris.
Hassan Farhat, Andre Mignot Hospital, Versailles.
Sylvie Castaigne, Andre Mignot Hospital, Versailles.
Odile Eloy, Andre Mignot Hospital, Versailles.
Felipe Suarez, Necker Hospital, Paris.
Richard Delarue, Necker Hospital, Paris.
Svetlana Challier, Necker Hospital, Paris.
Nathalie Dhedin, Pitié-Salpétrière Hospital.
Ahmad Aljijakli, Pitié-Salpétrière Hospital.
Emmanuelle Delabesse, Pitié-Salpétrière Hospital.
Annick Datry, Pitié-Salpétrière Hospital.
Françoise Isnard, Saint-Antoine Hospital, Paris.
Loic Fouillard, Saint-Antoine Hospital, Paris.
Jean-Yves Poirot, Saint-Antoine Hospital, Paris.
Leila Meliani, Saint-Antoine Hospital, Paris.
Lionel Adès, Avicennes Hospital, Bobigny.
Claire Bouges-Michel, Avicennes Hospital, Bobigny.
Michèle Deniau, Avicennes Hospital, Bobigny.
Frédérique Kuhnowski, Cochin Hospital, Paris.
François Dreyfus, Cochin Hospital, Paris.
André Paugam, Cochin Hospital, Paris.
Marie-Thérèse Baixench, Cochin Hospital, Paris.
Roland Leclercq, University Hospital, Caen.
Oumady Reman, University Hospital, Caen.
Chantal Duhamel, University Hospital, Caen.
Jean-Henri Bourrhis, Gustave Roussy Institute, Villejuif.
Sami Chehata, Gustave Roussy Institute, Villejuif.
Isabelle Chachati, Gustave Roussy Institute, Villejuif.
Vincent Foissaud, Hôpital des Armées, Clamart.
Christine Macnab, Hôpital des Armées, Clamart.
Hervé Tilly, Centre Henri Becquerel, Rouen.
Stéphane Leprêtre, Centre Henri Becquerel, Rouen.
Christian Gray, Centre Henri Becquerel, Rouen.
Emmanuel Raffoux, Saint-Louis Hospital, Paris.
Claire Lacroix, Saint-Louis Hospital, Paris.
Jeremy D Goldhaber-Fiebert, Stanford Health Policy, Stanford University.
Eran Bendavid, Stanford Health Policy, Stanford University.
Brandon J Farley, University of Provence, Marseille.
References
- 1. Segal BH (2009) Aspergillosis. N Engl J Med 360: 1870–1884. [DOI] [PubMed] [Google Scholar]
- 2. Ascioglu S, Rex JH, de Pauw B, Bennett JE, Bille J, et al. (2002) Defining opportunistic invasive fungal infections in immunocompromised patients with cancer and hematopoietic stem cell transplants: an international consensus. Clin Infect Dis 34: 7–14. [DOI] [PubMed] [Google Scholar]
- 3. de Pauw B, Walsh TJ, Donnelly JP, Stevens DA, Edwards JE, et al. (2008) Revised definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group (EORTC/MSG) Consensus Group. Clin Infect Dis 46: 1813–1821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Gerson SL, Talbot GH, Hurwitz S, Strom BL, Lusk EJ, et al. (1984) Prolonged granulocytopenia: the major risk factor for invasive pulmonary aspergillosis in patients with acute leukemia. Ann Intern Med 100: 345–351. [DOI] [PubMed] [Google Scholar]
- 5. Muhlemann K, Wenger C, Zenhausern R, Tauber MG (2005) Risk factors for invasive aspergillosis in neutropenic patients with hematologic malignancies. Leukemia 19: 545–550. [DOI] [PubMed] [Google Scholar]
- 6. Gotzsche PC, Johansen HK (1997) Meta-analysis of prophylactic or empirical antifungal treatment versus placebo or no treatment in patients with cancer complicated by neutropenia. BMJ 314: 1238–1244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Marr KA, Leisenring W (2005) Design issues in studies evaluating diagnostic tests for aspergillosis. Clin Infect Dis 41 Suppl 6S381–386. [DOI] [PubMed] [Google Scholar]
- 8. Maertens J, Verhaegen J, Lagrou K, Van Eldere J, Boogaerts M (2001) Screening for circulating galactomannan as a noninvasive diagnostic tool for invasive aspergillosis in prolonged neutropenic patients and stem cell transplantation recipients: a prospective validation. Blood 97: 1604–1610. [DOI] [PubMed] [Google Scholar]
- 9. Kami M, Fukui T, Ogawa S, Kazuyama Y, Machida U, et al. (2001) Use of real-time PCR on blood samples for diagnosis of invasive aspergillosis. Clin Infect Dis 33: 1504–1512. [DOI] [PubMed] [Google Scholar]
- 10. Kawazu M, Kanda Y, Nannya Y, Aoki K, Kurokawa M, et al. (2004) Prospective comparison of the diagnostic potential of real-time PCR, double-sandwich enzyme-linked immunosorbent assay for galactomannan, and a (1–>3)-beta-D-glucan test in weekly screening for invasive aspergillosis in patients with hematological disorders. J Clin Microbiol 42: 2733–2741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Barnes RA, White PL, Bygrave C, Evans N, Healy B, et al. (2009) Clinical impact of enhanced diagnosis of invasive fungal disease in high-risk haematology and stem cell transplant patients. J Clin Pathol 62: 64–69. [DOI] [PubMed] [Google Scholar]
- 12. Marchetti O, Lamoth F, Mikulska M, Viscoli C, Verweij P, et al. (2012) ECIL recommendations for the use of biological markers for the diagnosis of invasive fungal diseases in leukemic patients and hematopoietic SCT recipients. Bone Marrow Transplant 47: 846–854. [DOI] [PubMed] [Google Scholar]
- 13. Maertens J, Theunissen K, Verhoef G, Verschakelen J, Lagrou K, et al. (2005) Galactomannan and computed tomography-based preemptive antifungal therapy in neutropenic patients at high risk for invasive fungal infection: a prospective feasibility study. Clin Infect Dis 41: 1242–1250. [DOI] [PubMed] [Google Scholar]
- 14. Segal BH, Almyroudis NG, Battiwalla M, Herbrecht R, Perfect JR, et al. (2007) Prevention and early treatment of invasive fungal infection in patients with cancer and neutropenia and in stem cell transplant recipients in the era of newer broad-spectrum antifungal agents and diagnostic adjuncts. Clin Infect Dis 44: 402–409. [DOI] [PubMed] [Google Scholar]
- 15. Miceli MH, Grazziutti ML, Woods G, Zhao W, Kocoglu MH, et al. (2008) Strong correlation between serum aspergillus galactomannan index and outcome of aspergillosis in patients with hematological cancer: clinical and research implications. Clin Infect Dis 46: 1412–1422. [DOI] [PubMed] [Google Scholar]
- 16. Nouer SA, Nucci M, Kumar NS, Grazziutti M, Barlogie B, et al. (2011) Earlier response assessment in invasive aspergillosis based on the kinetics of serum Aspergillus galactomannan: proposal for a new definition. Clin Infect Dis 53: 671–676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Koo S, Bryar JM, Page JH, Baden LR, Marty FM (2009) Diagnostic performance of the (1–>3)-beta-D-glucan assay for invasive fungal disease. Clin Infect Dis 49: 1650–1659. [DOI] [PubMed] [Google Scholar]
- 18. Desmet S, Van Wijngaerden E, Maertens J, Verhaegen J, Verbeken E, et al. (2009) Serum (1–3)-beta-D-glucan as a tool for diagnosis of Pneumocystis jirovecii pneumonia in patients with human immunodeficiency virus infection or hematological malignancy. J Clin Microbiol 47: 3871–3874. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Pfeiffer CD, Fine JP, Safdar N (2006) Diagnosis of invasive aspergillosis using a galactomannan assay: a meta-analysis. Clin Infect Dis 42: 1417–1427. [DOI] [PubMed] [Google Scholar]
- 20.Leeflang MM, Debets-Ossenkopp YJ, Visser CE, Scholten RJ, Hooft L, et al.. (2008) Galactomannan detection for invasive aspergillosis in immunocompromized patients. Cochrane Database Syst Rev: CD007394. [DOI] [PubMed]
- 21. Mengoli C, Cruciani M, Barnes RA, Loeffler J, Donnelly JP (2009) Use of PCR for diagnosis of invasive aspergillosis: systematic review and meta-analysis. Lancet Infect Dis 9: 89–96. [DOI] [PubMed] [Google Scholar]
- 22. Karageorgopoulos DE, Vouloumanou EK, Ntziora F, Michalopoulos A, Rafailidis PI, et al. (2011) beta-D-glucan assay for the diagnosis of invasive fungal infections: a meta-analysis. Clin Infect Dis 52: 750–770. [DOI] [PubMed] [Google Scholar]
- 23. Lamoth F, Cruciani M, Mengoli C, Castagnola E, Lortholary O, et al. (2012) beta-Glucan antigenemia assay for the diagnosis of invasive fungal infections in patients with hematological malignancies: a systematic review and meta-analysis of cohort studies from the Third European Conference on Infections in Leukemia (ECIL-3). Clin Infect Dis 54: 633–643. [DOI] [PubMed] [Google Scholar]
- 24. Marr KA, Laverdiere M, Gugel A, Leisenring W (2005) Antifungal therapy decreases sensitivity of the Aspergillus galactomannan enzyme immunoassay. Clin Infect Dis 40: 1762–1769. [DOI] [PubMed] [Google Scholar]
- 25. Cordonnier C, Pautas C, Maury S, Vekhoff A, Farhat H, et al. (2009) Empirical versus preemptive antifungal therapy for high-risk, febrile, neutropenic patients: a randomized, controlled trial. Clin Infect Dis 48: 1042–1051. [DOI] [PubMed] [Google Scholar]
- 26. Botterel F, Farrugia C, Ichai P, Costa JM, Saliba F, et al. (2008) Real-time PCR on the first galactomannan-positive serum sample for diagnosing invasive aspergillosis in liver transplant recipients. Transpl Infect Dis 10: 333–338. [DOI] [PubMed] [Google Scholar]
- 27. Costa C, Costa JM, Desterke C, Botterel F, Cordonnier C, et al. (2002) Real-time PCR coupled with automated DNA extraction and detection of galactomannan antigen in serum by enzyme-linked immunosorbent assay for diagnosis of invasive aspergillosis. J Clin Microbiol 40: 2224–2227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Costa JM, Ernault P, Gautier E, Bretagne S (2001) Prenatal diagnosis of congenital toxoplasmosis by duplex real-time PCR using fluorescence resonance energy transfer hybridization probes. Prenat Diagn 21: 85–88. [DOI] [PubMed] [Google Scholar]
- 29. White PL, Bretagne S, Klingspor L, Melchers WJ, McCulloch E, et al. (2010) Aspergillus PCR: one step closer to standardization. J Clin Microbiol 48: 1231–1240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. White PL, Mengoli C, Bretagne S, Cuenca-Estrella M, Finnstrom N, et al. (2011) Evaluation of Aspergillus PCR protocols for testing serum specimens. J Clin Microbiol 49: 3842–3848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Cornet M, Levy V, Fleury L, Lortholary J, Barquins S, et al. (1999) Efficacy of prevention by high-efficiency particulate air filtration or laminar airflow against Aspergillus airborne contamination during hospital renovation. Infect Control Hosp Epidemiol 20: 508–513. [DOI] [PubMed] [Google Scholar]
- 32. Alberti C, Bouakline A, Ribaud P, Lacroix C, Rousselot P, et al. (2001) Relationship between environmental fungal contamination and the incidence of invasive aspergillosis in haematology patients. J Hosp Infect 48: 198–206. [DOI] [PubMed] [Google Scholar]
- 33. Yokoe D, Casper C, Dubberke E, Lee G, Munoz P, et al. (2009) Infection prevention and control in health-care facilities in which hematopoietic cell transplant recipients are treated. Bone Marrow Transplant 44: 495–507. [DOI] [PubMed] [Google Scholar]
- 34. Meyer BD (1990) Unemployment Insurance and Unemployment Spells. Econometrica 58: 757–782. [Google Scholar]
- 35. Heckman JJ, Taber CR (1994) Econometric mixture models and more general models for unobservables in duration analysis. Stat Methods Med Res 3: 279–299. [DOI] [PubMed] [Google Scholar]
- 36. Marino P, Sagaon-Teyssier L, Laetitia M, le Corroller-Soriano AG (2013) Sex Differences in the Return-to-Work Process of Cancer Survivors 2 Years After Diagnosis: Results From a Large French Population-Based Sample. J Clin Oncol 31: 1277–1284. [DOI] [PubMed] [Google Scholar]
- 37. Marr KA, Balajee SA, McLaughlin L, Tabouret M, Bentsen C, et al. (2004) Detection of galactomannan antigenemia by enzyme immunoassay for the diagnosis of invasive aspergillosis: variables that affect performance. J Infect Dis 190: 641–649. [DOI] [PubMed] [Google Scholar]
- 38. Freifeld AG, Bow EJ, Sepkowitz KA, Boeckh MJ, Ito JI, et al. (2011) Clinical practice guideline for the use of antimicrobial agents in neutropenic patients with cancer: 2010 Update by the Infectious Diseases Society of America. Clin Infect Dis 52: e56–93. [DOI] [PubMed] [Google Scholar]
- 39. Maertens J, Marchetti O, Herbrecht R, Cornely OA, Fluckiger U, et al. (2011) European guidelines for antifungal management in leukemia and hematopoietic stem cell transplant recipients: summary of the ECIL 3–2009 update. Bone Marrow Transplant 46: 709–718. [DOI] [PubMed] [Google Scholar]
- 40. Cornely OA, Maertens J, Winston DJ, Perfect J, Ullmann AJ, et al. (2007) Posaconazole vs. fluconazole or itraconazole prophylaxis in patients with neutropenia. N Engl J Med 356: 348–359. [DOI] [PubMed] [Google Scholar]