Invasive pulmonary aspergillosis (IPA) is a potentially lethal infection in patients with hematological diseases or following allogeneic stem cell transplantation. Early diagnosis is essential, as delayed treatment results in increased mortality.
KEYWORDS: diagnosis, galactomannan, invasive aspergillosis, lateral flow device
ABSTRACT
Invasive pulmonary aspergillosis (IPA) is a potentially lethal infection in patients with hematological diseases or following allogeneic stem cell transplantation. Early diagnosis is essential, as delayed treatment results in increased mortality. Recently, a lateral flow device (LFD) for the diagnosis of IPA was CE marked and made commercially available by OLM Diagnostics. We retrospectively analyzed bronchoalveolar lavage fluid (BALf) collected from adult hematology patients from 4 centers in The Netherlands and Belgium. Galactomannan was retested in all samples. All samples were applied to an LFD and read out visually by two independent researchers blinded to the diagnosis of the patient. All samples were also read out using a digital reader. We included 11 patients with proven IPA, 68 patients with probable IPA, 44 patients with possible IPA, and 124 patients with no signs of IPA (controls). In cases of proven IPA versus controls, sensitivity and specificity were 0.82 and 0.86 for visual readout and 0.82 and 0.96 for digital readout, respectively. When comparing patients with proven and probable IPA as cases versus controls, sensitivity and specificity were found to be 0.71 and 0.86, respectively. When excluding serum and BALf galactomannan as mycological criteria from the 2008 European Organization for Research and Treatment of Cancer Invasive Fungal Infections Cooperative Group (EORTC)/Mycoses Study Group of the National Institute of Allergy and Infectious Diseases (MSG) consensus definitions, the LFD was less specific than galactomannan when comparing subjects with proven and probable IPA to controls (0.86 versus 0.96; P = 0.005) but had similar sensitivity (0.76 versus 0.85; P = 0.18). In conclusions, in this large study of the CE-marked LFD in BALf from hematology patients, the LFD had a good performance for the diagnosis of IPA.
INTRODUCTION
Invasive pulmonary aspergillosis (IPA) remains a significant infectious complication in patients with hematological diseases or following allogeneic hematopoietic stem cell transplantation (HSCT) (1). Delayed initiation of Aspergillus-specific therapy increases overall mortality, making early diagnosis essential (2). The diagnostic tools currently used in clinical practice consist of fungal culture, direct microscopy (preferably using optical brighteners), and detection of galactomannan (GM), (1-3)-β-d-glucan (BDG), and/or Aspergillus DNA by PCR (3). However, these tools have several limitations in terms of sensitivity, turnaround time, and practicability. The sensitivities of direct tests, such as fungal culture or microscopy of samples taken from the site of infection (e.g., bronchoalveolar lavage fluid [BALf]), are as low as 20% to 50% (3). Indirect tests, which detect cell wall antigens produced by Aspergillus, such as GM on serum or BALf and BDG in serum, show better sensitivities than those of direct tests (4, 5) but require large numbers of samples to be cost-efficient, as these assays run on 96-well plates. Even when performed in-house, these tests are often run in batches, once or twice weekly, which increases the turnaround time and further delays diagnosis.
More recently, an Aspergillus-specific lateral flow device (LFD) has been developed, consisting of a self-contained immunochromatographic assay using a mouse monoclonal antibody (JF5) for the detection of an extracellular glycoprotein released by Aspergillus during active growth (6) (Fig. 1). Because of the single-test design and the minimal sample preparation required, this assay could provide a solution to some of the above-mentioned issues. In addition, preliminary evaluation has shown a sensitivity of 73% and specificity of 90% when applied to BALf (7).
FIG 1.
Lateral flow devices, showing from left to right a negative result, followed by increasing test line intensity. The control line is visible at the top, while the test line appears below the control line.
However, except for a study mentioned in one recent letter (8), all previously published studies with the LFD have used a prototype device. The current CE-marked LFD (AspLFD; OLM Diagnostics, Newcastle Upon Tyne, United Kingdom) differs in several aspects from this prototype, including the immunoglobulin G subclass of the antibody as well as the chromogen; this could impact the diagnostic characteristics of this test. A small retrospective comparative study (including 9 BALf samples) between the prototype device and the currently available assay showed only fair agreement between both assays (Cohen’s kappa, 0.43) (9). Although their sensitivities were fairly similar—though lowest in the hematology patients, at 68%—the novel assay proved to be more specific (9). Of note, samples were stored frozen between testing with the old and new devices, which could partly explain this difference.
The aim of this study was to assess the performance of the recently CE-approved LFD in a large multicenter cohort of hematology patients wo underwent diagnostic bronchoscopy with BALf sampling.
MATERIALS AND METHODS
This retrospective study comprised 247 BALf samples from 247 hemato-oncology patients from 2 academic centers in The Netherlands (Erasmus University Medical Center, Rotterdam, and Radboud University Medical Center, Nijmegen) and 2 centers in Belgium (University Hospitals Leuven, Leuven, and AZ St Jan Bruges, Bruges), collected between 2010 and 2018. Informed consent was waived due to the retrospective nature of this study on stored BALf samples previously collected as part of routine clinical care. The study had no impact on patient management. The LFD was provided by OLM Diagnostics. OLM Diagnostics had no role in the design of this study, its execution, analysis, interpretation of the data, or decision to publish.
Patient selection criteria included (i) age of ≥18 years, (ii) having an underlying hematological disease or following HSCT, (iii) availability of at least 500 μl of BALf sample for analysis stored at less than or equal to −20°C, (iv) a chest computed tomography (CT) scan performed within 7 days of BALf sampling, and (v) access to the full clinical data set. All four participating centers had an integrated care pathway for immunocompromised patients in place, using a standardized protocol: after 72 to 120 h of persistent fever unresponsive to broad-spectrum antibiotics, a CT scan of the chest was performed. Abnormal CT findings were followed by a bronchoscopy and collection of BALf for extensive microbiological (including GM detection) and microscopic analysis. Mold-active antifungal prophylaxis was given per institutional policy. We targeted a case/control ratio of 1:2 to reflect the estimated 30% incidence of IPA in hematology patients referred for bronchoscopy in our centers. The following clinical data were collected: demographic data, underlying disease, host factors, serum BDG (±3 days before or after BALf sampling, if available), GM in BALf and serum (±3 days before or after BALf sampling) as determined by the local laboratory, fungal culture results, other microbiological findings, microscopy (with the use of optical brighteners), histopathology (including autopsy) results, use of mold-active antifungals >24 h before bronchoscopy, use of mold-active prophylaxis, absolute neutrophil count, and chest CT scan and bronchoscopy findings. Survival through 12 weeks after initiation of Aspergillus-specific therapy was recorded, as well as time to last follow-up.
Case definitions.
Patients were classified independently by two physicians as having proven IPA, probable IPA, or possible invasive fungal disease in accordance with the revised European Organization for Research and Treatment of Cancer Invasive Fungal Infections Cooperative Group (EORTC)/Mycoses Study Group of the National Institute of Allergy and Infectious Diseases (MSG) consensus definitions (10). A primary analysis only considered patients with proven IPA as true cases. A secondary analysis also included patients with probable IPA as true cases. Patients considered to not have IPA (controls) were patients not fulfilling any of the EORTC/MSG clinical and mycological criteria, patients with features suggestive of IPA on pulmonary imaging but with a BALf GM optical density index (ODI) of <1.0 (see below), and a documented alternative diagnosis (e.g., bacterial) not receiving mold-active therapy, and patients not receiving any specific antimold therapy at all who survived for more than 6 months after bronchoscopy.
Study procedures.
Repeated GM and LFD testing were performed on all 247 BALf samples at the Belgian National Reference Centre for Mycosis in accordance with the manufacturer’s instructions.
(i) Galactomannan enzyme immunoassay.
BALf GM detection was performed using the Platelia Aspergillus enzyme immunoassay (Bio-Rad, Marnes-la-Coquette, France). All samples were retested in parallel with the LFD to correct for the long-term storage at less than or equal to −20°C. The GM ODI measured at the local lab was used for classification; the GM ODI tested at the central lab after storage was used for all other analyses. Sample pretreatment and addition of conjugate were performed manually, while incubation, washing, addition of chromogen and stopping solutions, and readout were performed automatically by a BEP-III analyzer (Siemens Healthcare, Erlangen, Germany). Although there is no universally agreed upon threshold for BALf GM positivity, we defined an ODI of ≥1.0 as positive, in line with a recent meta-analysis (11) and with the most recent EORTC/MSG consensus recommendations (12).
(ii) Aspergillus lateral flow device.
Briefly, BALf samples were defrosted at room temperature and vortexed. Seventy microliters of BALf was added to the release port on the LFD (AspLFD; OLM Diagnostics, Newcastle Upon Tyne, United Kingdom) and incubated at room temperature for 15 min. Hemorrhagic samples or samples that were viscous due to large amounts of mucus underwent pretreatment in accordance with the manufacturer’s instructions, consisting of heating at 100°C for 3 min after addition of 300 μl of EDTA-containing buffer to 150 μl of BALf, followed by centrifugation at 14,000 × g for 5 min. Seventy microliters of the supernatant was then applied to the LFD. The LFD was removed from its protective package immediately before application the sample. The appearance of the control line in the result window showed that the test had run correctly. The appearance of the Aspergillus-specific test line was determined after exactly 15 min, with results being recorded as positive if the test line was present. In the absence of a test line, the result was recorded as negative. Each LFD was independently assessed by two evaluators who were otherwise blinded to the final diagnosis (T.M. and E.G.). Immediately after reading was done, the readouts were compared between the 2 evaluators and discordant results were resolved by consensus. During the setup of this experiment, we noticed a delayed appearance of a test line after more than 15 min in some samples that were negative at the 15-min mark. We therefore performed a second visual readout of all 247 samples between 30 min and 1 h after applying the sample to the LFD. In addition, we investigated the added value of an objective readout method and quantification of results using a digital LFD reader (aLF reader; Qiagen Lake Constance, Stockach, Germany). Peak positions were determined using negative and positive controls included in the LFD kit.
Statistical analysis.
To calculate sensitivity and specificity with a maximum 95% confidence interval (CI) of 10% width at 80% power, we relied on data previously published by Heldt and Hoenigl (7), and calculated appropriate sample sizes using the method described by Buderer (13). Based on a pooled sensitivity of 73%, a pooled specificity of 90%, and an expected prevalence of 30% in hematology patients undergoing diagnostic bronchoscopy, we estimated a required total of at least 228 patients.
A 2-sided P value of ≤0.05 was considered statistically significant. The diagnostic characteristics of the LFD were compared to those of GM using McNemar’s chi-squared test, since GM and the LFD are paired observations. Cox regression was used to determine the relation between the LFD result and outcome, controlling for age, gender, HSCT, neutropenia, and prednisone-equivalent dose of ≥0.3 mg/kg of body weight/day for >21 days.
True positives were defined as patients with proven IPA according to the EORTC/MSG definitions and true negatives as patients without any evidence of IPA. These strict criteria were used because probable and (to an even greater extent) possible categories (as defined by consensus) are not definitive diagnoses but an assessment of the likelihood of having invasive fungal disease. However, for comparison with previously published findings, our secondary analysis also considered EORTC/MSG-defined probable cases as true positives. The EORTC/MSG definitions were used both with and without BALf and serum GM test results included. Indeed, as the GM assay itself is an accepted microbiological criterion in these definitions, a comparison of the diagnostic performance of the LFD and GM without the removal of GM from the definition leads to incorporation bias. The negative predictive value (NPV), positive predictive value (PPV), sensitivity, and specificity with likelihood ratios (LRs) and their 95% CIs were calculated.
Cohen’s kappa coefficient (with 95% CI) was calculated to measure the agreement between the LFD results (consensus of visual or digital readout) and GM ODI results, between the visual readings of the 2 evaluators, and between visual and digital readout. According to the classification by Landis and Koch, kappa values of >0.8 represent an almost perfect agreement. The effect of long-term storage on GM ODI values was evaluated using the paired Mann-Whitney U test.
Statistical analysis was performed using R v3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).
RESULTS
Key characteristics of the 247 patients included in this study are shown in Table 1. Eleven patients had proven IPA, 68 had probable IPA, 44 had possible invasive fungal disease, and 124 had no IPA, as defined by the EORTC/MSG definitions. Aspergillus species were cultured from 30 (12.1%) BALfs, and 75 (30.4%) had a GM ODI of ≥1.0. Empirical antifungal therapy was started in 21.5% of cases of proven and probable IPA. Retesting of BALf GM at the reference lab was not significantly different from the originally reported value (median GM ODI at time of sampling, 0.20 [interquartile range, 0.10 to 1.65] versus 0.20 after thawing [interquartile range, 0.10 to 1.45]; P = 0.37). Contingency tables for all subgroups are provided in the supplemental material.
TABLE 1.
Patient characteristicsa
| Parameter | Value for patients (n = 247) |
|---|---|
| Center (%) | |
| Belgium 1 | 40 (16.2) |
| Belgium 2 | 134 (54.3) |
| The Netherlands 1 | 33 (13.4) |
| The Netherlands 2 | 40 (16.2) |
| Age (yrs), median (IQR) | 63 (52, 71) |
| Male sex (%) | 148 (59.9) |
| Mold-active prophylaxis (%) | 17 (6.9) |
| Disease (%) | |
| Acute myeloid leukemia | 75 (30.4) |
| Allogeneic SCT | 68 (27.5) |
| Lymphoma | 58 (23.5) |
| Multiple myeloma | 14 (5.7) |
| Acute lymphoblastic leukemia | 10 (4.0) |
| Myelodysplastic syndrome | 8 (3.2) |
| Autologous SCT | 7 (2.8) |
| Other | 7 (2.8) |
| Neutropenia (%) | 118 (47.8) |
| Use of high-dose corticoids (%) | 85 (34.4) |
| T-cell suppression (%) | 125 (50.6) |
| Severe inborn immune deficit (%) | 1 (0.4) |
| Serum GM ODI, median (IQR) | 0.10 (0.07, 0.20) |
| Serum GM not performed, n (%) | 34 (13.8) |
| Aspergillus species (%) | |
| A. fumigatus | 25 (10.1) |
| A. flavus | 3 (1.2) |
| A. fumigatus + A. terreus | 1 (0.4) |
| A. versicolor | 1 (0.4) |
| Negative | 217 (87.9) |
| Serum β-d-glucan (pg/ml), median (IQR) | 0.00 (0.00, 123.61) |
| Absolute neutrophil count/mm³, median (IQR) | 140.00 (0.00, 3,200.00) |
SCT, stem cell transplantation; IQR, interquartile range; GM ODI, galactomannan optical density index.
Proven IPA versus controls.
The diagnostic performance of BALf GM and the LFD for 11 proven IPA cases versus 124 controls is shown in Table 2. Youden’s index was used to determine the optimal optical intensity (OI) cutoff to discriminate between cases of proven IPA and controls. The diagnostic performance of digital readout in this subgroup, using an OI cutoff of 33.15 mV (resulting in an area under the curve [AUC] of 0.921), is shown in Table 2. There was excellent agreement between the independent visual readouts of the 2 evaluators (disagreement on 6% of samples; Cohen’s kappa, 0.86; 95% CI, 0.79 to 0.93) and substantial agreement between visual and digital readout (Cohen’s kappa, 0.65; 95% CI, 0.48 to 0.83), resulting in a significantly improved positive predictive value due to a lower number of false positives using digital readout. The sensitivity of visual readout of the LFD was identical to GM (ODI cutoff ≥ 1.0) in this small subgroup of proven IPA (0.82 versus 0.82; P = 1.00), but specificity was lower (0.86 versus 0.96; P = 0.005). Diagnostic performance of digital readout was identical to that of GM (ODI cutoff ≥ 1.0) in this subgroup.
TABLE 2.
Diagnostic performance in cases of proven invasive pulmonary aspergillosis versus controlsa
| Test parameter | Value by indicated test (95% CI) |
|||
|---|---|---|---|---|
| LFD (visual readout) | LFD (digital readout) | GM ODI ≥ 1.0 | GM ODI ≥ 0.5 | |
| Sensitivity | 0.82 (0.48, 0.98) | 0.82 (0.48, 0.98) | 0.82 (0.48, 0.98) | 0.82 (0.48, 0.98) |
| Specificity | 0.86 (0.79, 0.92) | 0.96 (0.91, 0.99) | 0.96 (0.91, 0.99) | 0.93 (0.87, 0.97) |
| Positive predictive value | 0.35 (0.17, 0.56) | 0.64 (0.35, 0.87) | 0.64 (0.35, 0.87) | 0.50 (0.26, 0.74) |
| Negative predictive value | 0.98 (0.94, 1.00) | 0.98 (0.94, 1.00) | 0.98 (0.94, 1.00) | 0.98 (0.94, 1.00) |
| Positive likelihood ratio | 5.97 (3.54, 10.06) | 20.29 (8.23, 50.04) | 20.29 (8.23, 50.04) | 11.27 (5.66, 22.43) |
| Negative likelihood ratio | 0.21 (0.06, 0.74) | 0.19 (0.05, 0.66) | 0.19 (0.05, 0.66) | 0.20 (0.06, 0.69) |
CI, confidence interval; LFD, lateral flow device.
Proven or probable IPA versus controls.
To allow for a comparison with previous reports on the prototype version of the LFD (14–16) and with other diagnostic tests for IPA, we assessed the diagnostic performance in patients with EORTC/MSG-defined proven and probable IPA taken together as cases (n = 79) versus controls (n = 124), using different cutoffs (≥1.0 or ≥0.5) for BALf GM positivity (Table 3). Youden’s index was calculated again for each BALf GM cutoff. The sensitivity and specificity of visual readout of the LFD were significantly lower than those of BALf GM (≥1.0) in this subgroup (sensitivity, 0.71 versus 0.82; P = 0.020; specificity, 0.86 versus 0.96; P = 0.005). Serum GM had a significantly lower sensitivity (0.37 versus 0.73; P < 0.001) and higher specificity (1.00 versus 0.86; P < 0.001) than visual readout of the LFD. BDG was measured only for 9 patients and could therefore not be compared to the LFD. The receiver operating characteristic (ROC) curves for each subgroup are shown in Fig. 2. The agreement between BALf GM and the LFD was substantial, with a Cohen’s kappa of 0.61 (95% CI, 0.51 to 0.72) for visual readout, and a kappa of 0.63 (95% CI, 0.53 to 0.74) for digital readout (cutoff, 16.62 mV).
TABLE 3.
Diagnostic performance in cases of proven/probable invasive pulmonary aspergillosis versus controlsa
| Role of BALf GM as mycological criterion | No. of cases | Readout | Cutoff (mV) | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | PLR (95% CI) | NLR (95% CI) |
|---|---|---|---|---|---|---|---|---|---|
| BALf GM positive ≥ 1.0 | 79 | Visual | 0.71 (0.60, 0.81) | 0.86 (0.79, 0.92) | 0.77 (0.65, 0.86) | 0.82 (0.75, 0.88) | 5.17 (3.25, 8.22) | 0.34 (0.24, 0.48) | |
| Digital | 16.62 (AUC 0.826) | 0.72 (0.61, 0.82) | 0.86 (0.79, 0.92) | 0.77 (0.66, 0.86) | 0.83 (0.75, 0.89) | 5.26 (3.31, 8.36) | 0.32 (0.22, 0.46) | ||
| BALf GM positive ≥ 0.5 | 91 | Visual | 0.65 (0.54, 0.75) | 0.86 (0.79, 0.92) | 0.78 (0.67, 0.86) | 0.77 (0.69, 0.84) | 4.73 (2.97, 7.54) | 0.41 (0.31, 0.54) | |
| Digital | 16.62 (AUC 0.789) | 0.65 (0.54, 0.75) | 0.86 (0.79, 0.92) | 0.78 (0.67, 0.86) | 0.77 (0.69, 0.84) | 4.73 (2.97, 7.54) | 0.41 (0.31, 0.54) | ||
| BALf GM excluded as mycological criterion | 33 | Visual | 0.76 (0.58, 0.89) | 0.86 (0.79, 0.92) | 0.60 (0.43, 0.74) | 0.93 (0.87, 0.97) | 5.53 (3.41, 8.95) | 0.28 (0.15, 0.52) | |
| Digital | 14.21 (AUC 0.864) | 0.82 (0.65, 0.93) | 0.81 (0.73, 0.88) | 0.54 (0.39, 0.68) | 0.94 (0.88, 0.98) | 4.41 (2.95, 6.60) | 0.22 (0.11, 0.46) |
PPV, positive predictive value; NPV, negative predictive value; PLR, positive likelihood ratio; NLR, negative likelihood ratio; BALf GM, bronchoalveolar lavage fluid galactomannan; AUC, area under the curve.
FIG 2.
Receiver operating characteristic (ROC) curves of the lateral flow devices in different subgroups of invasive pulmonary aspergillosis (IPA). (a) Proven IPA versus controls. (b) Proven or probable IPA versus controls, galactomannan (GM) positive ≥ 1.0. (c) Proven or probable IPA versus controls, GM positive ≥ 0.5. (d) Proven or probable IPA versus controls, GM excluded as mycological criterion.
Of course, as GM is used as one of the mycological criteria in the EORTC/MSG criteria, this leads to a bias toward GM. Therefore, we omitted BALf and serum GM from the mycological criteria to allow for a direct comparison of the diagnostic characteristics of GM and the LFD. Specificity remained significantly higher for BALf GM (0.86 versus 0.96; P = 0.005), with a trend toward a higher sensitivity for BALf GM (0.76 versus 0.85; P = 0.18). However, 8 out of the 44 (18.1%) cases of possible invasive fungal disease had a positive LFD by visual readout, all with low OIs (median OI of the positive LFDs, 19.06 mV; interquartile range, 14.36 mV to 26.11 mV).
We found an exponential correlation between the GM ODI and the OI of the LFD as measured by the digital reader (Fig. 3). The correlation between the two was moderate, with an adjusted R2 of 0.52. Based on the results from this plot, we further identified 2 distinct subgroups, with a breakpoint around a GM ODI of 4.0. Indeed, LFD sensitivity was significantly lower in cases with BALf GM of <4.0 (0.47 versus 0.75; P = 0.014), while specificity was similar (0.86 versus 0.88; P = 0.909). Furthermore, in cases with a positive fungal culture, the GM ODI was significantly higher (median, 6.2 versus 2.75; P = 0.046) and there was a trend toward higher OIs of the LFD (median, 113.62 mV versus 33.87 mV; P = 0.054). The qualitative result of the LFD was not significantly different in culture-positive cases (77.8% positive versus 67.3%; P = 0.477). As only 17 patients (6.9%) were receiving mold-active prophylaxis, this subgroup was too small to assess the effect of prophylaxis on diagnostic performance. However, in the subgroup that received empirical antifungal therapy prior to BALf sampling, the sensitivity was significantly lower (0.47 versus 0.77; P = 0.032), while specificity was similar (1.00 versus 0.85; P = 0.610).
FIG 3.
Jitterplot of the galactomannan optical density index versus the intensity of the lateral flow device for all included patients. (a) Overview of all measurements. (b) Zoomed-in detail of measurements with galactomannan optical density index of ≤4.0.
When reading out the LFD between 30 min and 1 h after applying the sample, 7.9% of the initial negative results had become positive, increasing the sensitivity to 0.80 (95% CI, 0.69 to 0.88) and decreasing the specificity to 0.79 (95% CI, 0.71 to 0.86). In multivariate Cox regression, the LFD was not a significant predictor of mortality in cases of proven or probable IPA, either when used as a binary variable (P = 0.492) or when used as a continuous variable (P = 0.982).
DISCUSSION
We present the largest multicenter trial of a newly CE-approved LFD for the diagnosis of IPA in hematology patients to date, including a total of 247 patients from 4 hospitals in Belgium and The Netherlands, 79 of whom had proven or probable IPA according to consensus definitions. The primary analysis was restricted to the performance of the BALf LFD using only EORTC/MSG-proven cases as true positives and cases with no IPA as true negatives. Unfortunately, proven IPA is a rare condition; many patients are thrombocytopenic or in need of supplemental oxygen and are typically not eligible for invasive procedures. In addition, such an analysis introduces disease progression bias, especially when relying on autopsy data. Nevertheless, in this well-documented subgroup, the recently released LFD showed good diagnostic performance: sensitivity was identical to BALf GM (≥1.0), although specificity was significantly lower when the result was read visually. The excellent negative predictive value of 98% in proven IPA could allow clinicians to convincingly withhold mold-active antifungal therapy in at-risk patients with unexplained CT findings. However, generalizing this high NPV to all patient populations should be done cautiously, as it is greatly influenced by the prevalence of IPA (Fig. 4). Our results are well in line with previously reported studies on the LFD prototype assay (17). Importantly, a digital readout of the LFD greatly increased the performance of the assay in terms of specificity, positive predictive value, and positive likelihood ratio, making it identical in performance to GM (Table 2).
FIG 4.
Negative predictive value (NPV) and positive predictive value (PPV) as a function of prevalence of the tested population, in cases of proven IPA versus controls (a) and in cases of proven and probable IPA versus controls (b). The red line marks the estimated prevalence used in our study (30%).
Given the rarity of proven cases of IPA, the EORTC/MSG consensus definitions are often used as a diagnostic reference standard. However, these definitions were basically developed for clinical and epidemiological research and not for the accurate evaluation of diagnostic tests. Indeed, these criteria are subject to misclassification as well as to incorporation bias (e.g., BALf GM is one of the microbiological criteria for assigning probable disease). Nevertheless, this method of evaluation is still frequently used. We decided to compare the performance of the LFD to the definitions as published and to the definitions with exclusion of GM as a mycological criterion, and we found the diagnostic performance of the LFD to be similar to previously published results of the prototype device for hematology patients with proven or probable IPA (7) (sensitivity, 0.71 versus 0.67; P = 0.744; specificity, 0.86 versus 0.91; P = 0.36). This contrasts with the results of a comparative study of 14 cases of proven and probable IPA in which samples were tested using both the prototype and CE-marked LFD, which found an increased specificity for the CE-marked LFD (9). Furthermore, we noticed delayed positive reactions with appearance of a test line after 30 to 60 min in some cases, which resulted in an increased sensitivity but a decrease in specificity compared to the consensus reference. A similar effect was seen in a study of 9 hematology patients with proven or probable IPA (8). This could possibly be explained by nonspecific reactions, as the rates of conversion to a positive line are similar for cases and controls (8.9% and 7.3%, respectively). We therefore do not recommend delayed readout.
Interestingly, our study found a significantly lower sensitivity of the LFD for patients with a GM ODI of <4.0. We clearly demonstrated an exponential relation between the intensity of the LFD test line and the GM ODI. Visual readout of the LFD was reliable, with good interevaluator agreement, which was confirmed objectively by digital readout.
The quantitative and qualitative results of the LFD on BALf were not predictive of outcome in multivariate Cox regression. This is not unexpected, as similar results were seen with GM testing on BALf (18). This is likely the result of differences in BALf sampling techniques, which are not standardized between physicians and can even differ between procedures by the same physician. This can result in differences in sampling volume, leading to dilution. Furthermore, peripheral lesions and lesions in the upper lobes can be more difficult to reach.
The large sample size of our study allows for an estimation of the performance of the LFD with narrow confidence intervals. Furthermore, the use of independent and blinded observers and the use of a digital reader ensure a high methodological standard for our study. However, this study also has several limitations. The retrospective design implies an artificial prevalence of the disease, thereby influencing the predictive values. We tried to overcome this by selecting cases and controls in rates similar to what are seen in our centers. However, in settings where IPA is more (or less) frequent, these values will differ. Furthermore, the storage conditions of the samples at less than or equal to −20°C could theoretically influence the diagnostic performance of the test. We tried to remove this bias by retesting of GM in parallel with the LFD, which did not show any significant degradation over time. However, though similar in chemical structure, it is not guaranteed that the mannoprotein antigen detected by the LFD is equally stable as GM detected by the Platelia enzyme immunoassay.
In conclusion, the CE-marked BALf LFD appears to have good performance for diagnosing IPA in hematology patients, with even better performance for excluding IPA. The LFD can be used as a point-of-care test, unless the sample is hemorrhagic or heavily contaminated with mucus, in which case pretreatment in the lab is required. This test could be used as a first-line diagnostic tool in the bronchoscopy suite, given its short turnaround time and economic advantage over GM testing in low-volume settings. However, in view of its low positive predictive value, the LFD is no substitute for additional diagnostic testing (GM, BDG, or PCR) to definitively confirm or exclude the diagnosis of IPA.
Supplementary Material
ACKNOWLEDGMENTS
We thank OLM Diagnostics for providing the LFD tests used in this study. OLM Diagnostics had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
A.S. has received nonfinancial support from Abvie, Amgen, Roche, Gilead Sciences, and Pfizer. B.R. has received research grants from Gilead and MSD outside the context of this work and served as a speaker for Gilead, MSD, BMS, ViiV, and Pfizer. J.M. has received research grants from Merck/MSD, Gilead Sciences, and Pfizer, is a consultant to Astellas, Basilea, Bio-Rad, Merck/MSD, Pfizer, Schering-Plough, F2G, Gilead Sciences, Cidara, Scynexis, Amplyx, and Luminex, and has served on the speaker’s bureau of Astellas, Gilead Sciences, Bio-Rad, Merck/MSD, Pfizer, Schering-Plough, Basilea, and Viropharma/Shire. K.L. has received research grants from Gilead Sciences, MSD, and Pfizer, has received consultancy fees from Gilead Sciences, Pfizer, Abbott, MSD, and SMB Laboratoires Brussels, has received travel support from Pfizer, Gilead Sciences, and MSD, and has received speaker fees from Gilead Sciences, Roche, and Abbott. M.R. has received nonfinancial support from Gilead Sciences and Pfizer. P.E.V. has received research grants from Merck/MSD, Gilead Sciences, F2G, and Pfizer, is a consultant to Merck/MSD, Pfizer, F2G, Cidara, Scynexis, and Siemens, and has served on the speaker’s bureau of Gilead Sciences, Merck/MSD, and F2G. T.M. has received nonfinancial support from IMMY, Merck/MSD, and Gilead Sciences and consultancy fees from Gilead Sciences. A.S., E.D.K., and E.G. have no conflicts of interest to report.
T.M., J.M., K.L., and A.S. designed the experiment. T.M., A.D., A.S., and E.D.K. collected the clinical data. T.M., A.D., E.G., B.R., P.E.V., and M.R. collected the BALf samples. T.M. and E.G. performed the experiments. T.M. analyzed the data. T.M. and J.M. wrote the initial draft. All authors critically revised the initial draft and final manuscript.
Footnotes
Supplemental material for this article may be found at https://doi.org/10.1128/JCM.01913-18.
REFERENCES
- 1.Brown GD, Denning DW, Gow NAR, Levitz SM, Netea MG, White TC. 2012. Hidden killers: human fungal infections. Sci Transl Med 4:165rv13. doi: 10.1126/scitranslmed.3004404. [DOI] [PubMed] [Google Scholar]
- 2.von Eiff M, Roos N, Schulten R, Hesse M, Zühlsdorf M, van de Loo J. 1995. Pulmonary aspergillosis: early diagnosis improves survival. Respiration 62:341–347. doi: 10.1159/000196477. [DOI] [PubMed] [Google Scholar]
- 3.Lamoth F, Calandra T. 2017. Early diagnosis of invasive mould infections and disease. J Antimicrob Chemother 72:i19–i28. doi: 10.1093/jac/dkx030. [DOI] [PubMed] [Google Scholar]
- 4.Leeflang MMG, Debets-Ossenkopp YJ, Visser CE, Scholten RJPM, Hooft L, Bijlmer HA, Reitsma JB, Bossuyt PMM, Vandenbroucke-Grauls CM. 2008. Galactomannan detection for invasive aspergillosis in immunocompromized patients. Cochrane Database Syst Rev 2008:CD007394. [DOI] [PubMed] [Google Scholar]
- 5.He S, Hang J-P, Zhang L, Wang F, Zhang D-C, Gong F-H. 2015. A systematic review and meta-analysis of diagnostic accuracy of serum 1,3-β-d-glucan for invasive fungal infection: focus on cutoff levels. J Microbiol Immunol Infect 48:351–361. doi: 10.1016/j.jmii.2014.06.009. [DOI] [PubMed] [Google Scholar]
- 6.Thornton CR. 2008. Development of an Immunochromatographic lateral-flow device for rapid serodiagnosis of invasive aspergillosis. Clin Vaccine Immunol 15:1095–1105. doi: 10.1128/CVI.00068-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Heldt S, Hoenigl M. 2017. Lateral flow assays for the diagnosis of invasive aspergillosis: current status. Curr Fungal Infect Rep 11:45–51. doi: 10.1007/s12281-017-0275-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jenks JD, Mehta SR, Taplitz R, Law N, Reed SL, Hoenigl M. 2 November 2018. Bronchoalveolar lavage Aspergillus galactomannan lateral flow assay versus Aspergillus-specific lateral flow device test for diagnosis of invasive pulmonary aspergillosis in patients with hematological malignancies. J Infect doi: 10.1016/j.jinf.2018.10.014. [DOI] [PubMed] [Google Scholar]
- 9.Hoenigl M, Eigl S, Heldt S, Duettmann W, Thornton C, Prattes J. 2018. Clinical evaluation of the newly formatted lateral-flow device for invasive pulmonary aspergillosis. Mycoses 61:40–43. doi: 10.1111/myc.12704. [DOI] [PubMed] [Google Scholar]
- 10.De Pauw B, Walsh TJ, Donnelly JP, Stevens DA, Edwards JE, Calandra T, Pappas PG, Maertens J, Lortholary O, Kauffman CA, Denning DW, Patterson TF, Maschmeyer G, Bille J, Dismukes WE, Herbrecht R, Hope WW, Kibbler CC, Kullberg BJ, Marr KA, Muñoz P, Odds FC, Perfect JR, Restrepo A, Ruhnke M, Segal BH, Sobel JD, Sorrell TC, Viscoli C, Wingard JR, Zaoutis T, Bennett JE. 2008. Revised definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group (EORTC/MSG) Consensus Group. Clin Infect Dis 46:1813–1821. doi: 10.1086/588660. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Zou M, Tang L, Zhao S, Zhao Z, Chen L, Chen P, Huang Z, Li J, Chen L, Fan X. 2012. Systematic review and meta-analysis of detecting galactomannan in bronchoalveolar lavage fluid for diagnosing invasive aspergillosis. PLoS One 7:e43347. doi: 10.1371/journal.pone.0043347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.White L, Verweij P, Donnelly JP, Pappas PG, Maertens J. 2017. 2017 update on the EORTC/MSG invasive fungal disease definitions. 8th Trends in Medical Mycology, Belgrade, Serbia. [Google Scholar]
- 13.Buderer NMF. 1996. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med 3:895–900. doi: 10.1111/j.1553-2712.1996.tb03538.x. [DOI] [PubMed] [Google Scholar]
- 14.Hoenigl M, Prattes J, Spiess B, Wagner J, Prueller F, Raggam RB, Posch V, Duettmann W, Hoenigl K, Wölfler A, Koidl C, Buzina W, Reinwald M, Thornton CR, Krause R, Buchheidt D. 2014. Performance of galactomannan, beta-d-glucan, Aspergillus lateral-flow device, conventional culture, and PCR tests with bronchoalveolar lavage fluid for diagnosis of invasive pulmonary aspergillosis. J Clin Microbiol 52:2039–2045. doi: 10.1128/JCM.00467-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Prattes J, Lackner M, Eigl S, Reischies F, Raggam RB, Koidl C, Flick H, Wurm R, Palfner M, Wölfler A, Neumeister P, Thornton CR, Krause R, Lass-Flörl C, Hoenigl M. 2015. Diagnostic accuracy of the Aspergillus-specific bronchoalveolar lavage lateral-flow assay in haematological malignancy patients. Mycoses 58:461–469. doi: 10.1111/myc.12343. [DOI] [PubMed] [Google Scholar]
- 16.Heldt S, Prattes J, Eigl S, Spiess B, Flick H, Rabensteiner J, Johnson G, Prüller F, Wölfler A, Niedrist T, Boch T, Neumeister P, Strohmaier H, Krause R, Buchheidt D, Hoenigl M. 2018. Diagnosis of invasive aspergillosis in hematological malignancy patients: performance of cytokines, Asp LFD, and Aspergillus PCR in same day blood and bronchoalveolar lavage samples. J Infect 77:235–241. doi: 10.1016/j.jinf.2018.05.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Johnson GL, Sarker S-J, Nannini F, Ferrini A, Taylor E, Lass-Flörl C, Mutschlechner W, Bustin SA, Agrawal SG. 2015. Aspergillus-specific lateral-flow device and real-time PCR testing of bronchoalveolar lavage fluid: a combination biomarker approach for clinical diagnosis of invasive pulmonary aspergillosis. J Clin Microbiol 53:2103–2108. doi: 10.1128/JCM.00110-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mercier T, Guldentops E, Lagrou K, Maertens J. 2018. Galactomannan, a surrogate marker for outcome in invasive aspergillosis: finally coming of age. Front Microbiol 9:661. doi: 10.3389/fmicb.2018.00661. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.