Aspergillus-Specific Lateral-Flow Device and Real-Time PCR Testing of Bronchoalveolar Lavage Fluid: a Combination Biomarker Approach for Clinical Diagnosis of Invasive Pulmonary Aspergillosis

Abstract

Clinical experience with the impact of serum biomarkers for invasive fungal disease (IFD) varies markedly in hemato-oncology. Invasive pulmonary aspergillosis (IPA) is the most common manifestation, so we evaluated biomarkers in bronchoalveolar lavage (BAL) fluid. An Aspergillus-specific lateral-flow device (LFD), quantitative real-time PCR (qPCR), and the galactomannan (GM) test were used with 32 BAL fluid samples from 32 patients at risk of IPA. Eight patients had proven IPA, 3 had probable IPA, 6 had possible IPA, and 15 patients had no IPA by European Organization for Research and Treatment of Cancer Invasive Fungal Infections Cooperative Group/Mycoses Study Group of the National Institute of Allergy and Infectious Diseases (EORTC/MSG) criteria. The diagnostic accuracies of the tests were evaluated, and pairwise agreement between biomarkers was calculated. The diagnostic performance of the EORTC/MSG criteria was evaluated against the test(s) identified to be the most useful for IPA diagnosis. Using the EORTC/MSG criteria, the sensitivities of qPCR and LFD were 100% and the sensitivity of the GM test was 87.5% (GM test index cutoff, >0.8), with the tests having specificities of between 66.7 and 86.7%. The agreement between the results of qPCR and LFD was almost perfect (Cohen's kappa coefficient = 0.93, 95% confidence interval, 0.81 to 1.00). LFD and qPCR combined had a sensitivity of 100% and a specificity of 85.7%. Calcofluor staining and culture of all BAL fluid samples were negative for fungal infection. The median time from the start of mold-active antifungal therapy to the time of collection of BAL fluid was 6 days. Reversing roles and using dual testing by LFD and qPCR to classify cases, the EORTC/MSG criteria had a sensitivity of 83.3%. All three tests are useful for the diagnosis of IPA in BAL fluid samples. Despite the significant delays between the start of antifungal therapy and bronchoscopy, unlike microscopy and culture, the biomarkers remained informative. In particular, the combination of LFD and qPCR allows the sensitive and specific detection of IPA.

INTRODUCTION

Invasive fungal disease (IFD) is a major cause of infectious mortality in hemato-oncology patients due to their underlying disease and its treatment, which lead to periods of prolonged immunosuppression (1). Invasive pulmonary aspergillosis (IPA) is the most common cause of mortality due to mold disease (2, 3), and early diagnosis and treatment are vital for improving outcomes (4). However, early diagnosis is hampered by the limitations of current biomarker tests and a lack of consensus on the best samples to be tested (i.e., blood, bronchoalveolar [BAL] fluid, or tissue biopsy specimens) in terms of both test sensitivity and practicability. Direct examination or culture of pulmonary tissue remains the “gold standard” for the diagnosis of IPA (5, 6). However, lung biopsy in this acute care setting is rarely performed due to the associated risks. Consequently, in view of the diagnostic challenges and worse outcomes with late treatment, an empirical strategy has been—and remains—the standard of care in many hematology units (7). This approach leads to overtreatment with antifungal drugs, which have significant side effects and drug-drug interactions. Furthermore, health care systems are exposed to spiraling drug costs (8). Improving the means of diagnosis of IPA is an urgent clinical need.

Attempts have been made to optimize and standardize existing tests (5, 6, 9–11) and develop new technologies (12) for rapid and early diagnosis of invasive aspergillosis. The European Organization for Research and Treatment of Cancer Invasive Fungal Infections Cooperative Group (EORTC)/Mycoses Study Group of the National Institute of Allergy and Infectious Diseases (MSG) published definitions for IFD to facilitate clinical studies and research (5, 6); however, these definitions were not designed for clinical management. Due to the difficulty in establishing the gold standard diagnosis of proven IPA, the EORTC and MSG criteria have been used extensively to evaluate the performance of diagnostic methods (13–17). Publications have highlighted the poor diagnostic performance of the original EORTC/MSG definitions (18, 19), as well as the revised definitions (20, 21), compared to that of autopsy examination. This raises doubts as to the value of using the EORTC/MSG criteria, an imperfect reference standard, to assess new diagnostic approaches.

The aims of this study were to (i) compare the diagnostic performances of the Aspergillus lateral-flow device (LFD), an Aspergillus quantitative real-time PCR (qPCR), and the galactomannan (GM) test as single and combined diagnostic tools with BAL fluid samples from immunocompromised patients and (ii) evaluate the diagnostic performance of the EORTC/MSG criteria against the test(s) identified to be the most useful for IPA diagnosis.

MATERIALS AND METHODS

Ethics statement.

The work was ethically approved by the East London & the City Local Research Ethics Committee (REC reference number 05/Q0603/6 and, as amended on 28 October 2009, reference ReDA 003933 QM). All patients in the Barts study cohort gave informed written consent. BAL fluid samples from the Innsbruck University Hospital were obtained during routine diagnostic investigations and analyzed anonymously. This study was approved by the local ethics committee, Medical University Innsbruck (UN4529/308/4.1).

Study design and patient cohorts.

This retrospective study comprised 32 BAL fluid samples from 32 immunocompromised adults (patients with proven IPA, n = 8; patients with probable IPA, n = 3; patients with possible IPA, n = 6; patients with no IPA, n = 15), and the study had no impact on patient management. The samples from London, United Kingdom, were a subset of those recruited in an observational study of IPA diagnosis in the Division of Haemato-Oncology, St. Bartholomew's Hospital. The only selection criteria were the availability of BAL fluid for analysis and full clinical data. Twenty-four BAL fluid samples from 24 adults at high risk of IFD following intensive chemotherapy or allogeneic stem cell transplantation were obtained between August 2005 and February 2011. After 72 h of persistent fever unresponsive to antibiotics, a computed tomography (CT) scan of the chest was performed, and if the CT findings were abnormal, the CT was followed by a bronchoscopy and the collection of BAL fluid. Mold-active antifungal therapy was commenced per clinical policies. IPA was graded independently by two physicians in accordance with the revised criteria of the EORTC/MSG (6). Eight BAL fluid samples from 8 patients with histological and culture-proven IPA were obtained during routine diagnostic workups at the Innsbruck University Hospital between May 2008 and November 2012.

LFD, qPCR, and GM testing were performed on all 32 samples.

Study procedures and molecular tests.

The bronchoscopy and BAL procedures followed the local standard procedures of the Respiratory Department. All BAL fluid samples were frozen at −70°C prior to LFD, GM, and PCR testing.

Galactomannan enzyme immunoassay.

GM detection was performed by a Platelia Aspergillus enzyme immunoassay (PA-EIA; Bio-Rad, France). Techniques were carried out as recommended by the manufacturer. There is no universally agreed upon threshold for positivity for the GM test index in BAL fluid, although an index of >1.0 was suggested in a recent meta-analysis of GM test-BAL fluid studies (13). Because of the known impact of antifungals on GM test performance and the commencement of antifungal treatment before bronchoscopy in the study cohort, two values for the GM test index were evaluated: ≥0.8 and >1.0.

Aspergillus lateral-flow device.

Testing with the Aspergillus LFD was performed as previously described (22). Briefly, BAL fluid samples were defrosted at room temperature with gentle mixing. One hundred microliters of BAL fluid was added to the release port on the LFD device and incubated at room temperature for 15 min. Development of the control line in the result window shows that the test has run correctly. The development of the Aspergillus-specific test line was determined after exactly 15 min, with results being recorded as positive, if the test line was present. In the absence of a test line, the result was recorded as negative. Each LFD result was independently assessed by two users.

DNA extraction from BAL fluid.

Two hundred microliters of clinical sample was added to a 2-ml sample tube with 10 μl proteinase K (both from a Qiagen EZ1 DNA tissue kit) and gently vortexed. This tube was incubated at 56°C for 15 min, and then the tube was spun to remove condensation from the lid. Extraction was performed on an EZ1 robot (Qiagen, Germany), using the EZ1 DNA tissue card program and tissue kit, eluting into 50 μl. DNA extracts were stored at −20°C prior to PCR analysis. DNA-free water was run as a negative (fungus-free) extraction control in every batch of extractions to monitor for contamination during the extraction process. Aspergillus DNA spiked into saline was run as a positive extraction control in the first and last extractions run each day.

Pan-Aspergillus PCR.

qPCR testing was performed as described previously (10). The test targeted the 18S rRNA gene, and the limit of detection was 0.6 Aspergillus fumigatus genomes per reaction. Twenty-five percent of wells in every test were no-template controls (NTCs), and detection of amplification products in a single NTC invalidated the entire run, which was then repeated. A positive control containing six genome copies of A. fumigatus (a clinical isolate) was also included with each test to monitor intertest consistency. Amplification had to be reproducible, occurring in all 3 replicate wells, for a sample to be considered PCR positive. The SPUD qPCR test (23) was used as an exogenous inhibition control as previously described (10). PCR products were analyzed by direct sequencing, to confirm amplification of the target sequence.

Statistical analysis and case definitions.

True positives were defined only as cases proven according to EORTC/MSG criteria, and true negatives were defined as cases with no evidence of IFD according to EORTC/MSG criteria. These strict criteria that were used as probable and possible categories represent an assignment of the likelihood of IFD and are not definitive diagnoses. EORTC/MSG criteria were used both with and without GM test results included, as the GM test result is itself a criterion in the EORTC/MSG scoring and leads to incorporation bias. The negative predictive value (NPV), positive predictive value (PPV), sensitivity, and specificity with likelihood ratios (LRs) and diagnostic odds ratio (DORs) were calculated for each test, and 95% confidence intervals (CIs) were calculated for NPV, PPV, sensitivity, and specificity. For comparison with the results in the literature, these analyses were also performed using the cases classified as proven/probable according to EORTC/MSG criteria as true positives.

Cohen's kappa coefficient (including 95% confidence intervals) was calculated to measure the agreement between any two biomarkers. Kappa values were interpreted using the method of Landis and Koch (24), with values of >0.8 representing almost perfect agreement. This result was used to select a combination biomarker approach, which was then evaluated to determine the NPV, PPV, sensitivity, and specificity for the true-positive and true-negative populations.

Cases were reclassified using the optimal biomarker combination as the reference standard, and the diagnostic performance of the EORTC/MSG criteria (6) was evaluated against the biomarker assessment of a clinical diagnosis of IPA, with cases classified as proven/probable IPA according to EORTC/MSG criteria being interpreted as having a positive result and cases with no evidence of IFD according to EORTC/MSG criteria being interpreted as having a negative result. Statistical analysis was performed using SPSS, version 20 (SPSS Inc., Chicago, IL, USA).

RESULTS

The majority of the 32 immunocompromised patients in this study (25/32, 78%) had an underlying hematological malignancy, while 3/32 had undergone solid organ transplantation, 3/32 had severe underlying chronic lung diseases, and 1 patient had lung cancer. Eight patients had proven IPA, 3 had probable IPA, 6 had possible IPA, and 15 patients had no IPA, as defined by use of the EORTC/MSG criteria.

Individual test performance.

The sensitivity, specificity, PPV, NPV, positive and negative LRs, and DORs of all three tests (qPCR, LFD, the GM test with an index cutoff value of >1.0, and the GM test with an index cutoff value of >0.8) for proven versus no IPA are shown in Table 1. The sensitivity, specificity, PPV, and NPV for proven IPA versus no IPA were as follows: for qPCR, 100%, 87%, 80%, and 100%, respectively; for LFD, 100%, 80%, 73%, and 100%, respectively; for the GM test with an index cutoff value of >0.8, 88%, 67%, 58%, and 91%, respectively; and for the GM test with an index cutoff value of >1.0, 75%, 67%, 55%, and 83%, respectively. Performance figures for probable/proven IPA versus no IPA are shown for comparison in Table S1 in the supplemental material.

TABLE 1.

Diagnostic performance of BAL fluid GM test, LFD, and qPCR for proven IPA versus no IPA^a

Test	Sensitivity (%) (n = 8)	Specificity (%) (n = 15)	PPV (%)	NPV (%)	Positive LR	Negative LR	DOR
GM test (cutoff, 0.8)	87.50 (47.38–97.93)	66.67 (38.41–88.05)	58.33 (27.75–84.68)	90.91 (58.67–98.49)	2.62	0.19	14.00
GM test (cutoff, 1.0)	75 (35.05–96.07)	66.67 (38.41–88.05)	54.55 (23.50–83.08)	83.33 (51.58–97.42)	2.25	0.38	6.00
qPCR	100 (62.91–100)	86.67 (59.51–97.95)	80 (44.43–96.89)	100 (75.12–100)	7.50	0.00	NA
LFD test	100 (62.91–100)	80 (51.91–95.43)	72.73 (39.08–93.65)	100 (73.35–100)	5.00	0.00	NA

^aThe sensitivities, specificities, positive predictive values, negative predictive values, likelihood ratios, and diagnostic odds ratios are displayed, with 95% confidence intervals being given in parentheses. Abbreviations: GM, galactomannan, with cutoff values of 0.8 and 1.0 being used for the GM test; qPCR, quantitative real-time PCR; LFD, lateral-flow device; NPV, negative predictive value; PPV, positive predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio; NA, not applicable.

Test agreement analysis.

A pairwise comparison of the tests showed that the highest level of agreement was between qPCR and LFD, with a kappa coefficient value of 0.93 (almost perfect agreement; Table 2). All other pairs of biomarkers, regardless of the GM test threshold used for positivity (≥0.8 or >1.0), showed significantly less agreement. On the basis of this result, the LFD and qPCR combination biomarker approach was then evaluated to determine sensitivity, specificity, PPV, NPV, positive and negative LRs, and DORs for proven IPA versus no IPA. The results are shown in Table 3. Performance figures based on probable/proven IPA versus no IPA are shown for comparison in Table S2 in the supplemental material.

TABLE 2.

Pairwise agreement of tests^a

Test pair	Cohen's kappa coefficient (95% CI)
LFD test + qPCR	0.93 (0.81–1.00)
LFD test + GM test (cutoff, 0.8)	0.62 (0.35–0.89)
LFD test + GM test (cutoff, 1.0)	0.47 (0.16–0.78)
qPCR + GM test (cutoff, 0.8)	0.68 (0.43–0.93)
qPCR + GM test (cutoff, 1.0)	0.53 (0.22–0.83)

^aData are for 32 samples. Abbreviations: GM, galactomananan, with cutoff values of 0.8 and 1.0 being used for the GM test; qPCR, quantitative real-time PCR; LFD, lateral-flow device; CI, confidence interval.

TABLE 3.

Diagnostic performance of dual qPCR and LFD testing for proven IPA versus no IPA^a

Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Positive LR	Negative LR	DOR
100 (62.91–100)	85.71 (57.16–97.80)	80 (44.43–96.89)	100 (73.35–100)	7.00	0.00	NA

^aThe sensitivities, specificities, positive predictive values, negative predictive values, likelihood ratios, and diagnostic odds ratios are displayed. Abbreviations: qPCR, quantitative real-time PCR; LFD, lateral-flow device; NPV, negative predictive value; PPV, positive predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio; NA, not applicable.

Combined test performance.

As shown in Table 4, using dual LFD and qPCR positivity as a tool to direct antifungal therapy in the probable/possible/no IPA study patients, retrospective analysis indicated that 4/24 patients would have received/continued to receive antifungal therapy (samples 1 to 4), 19 would have had antifungal therapy withheld (if the patient was stable, with no other reason to suspect IFD; samples 6 to 24), and 1 would have required repeat testing due to inconclusive results (qPCR negative, LFD positive; sample 5). In reality, 23 of the 24 patients received systemic antifungal treatment, even though microscopy, including calcofluor staining, and mycological culture of all BAL fluid samples were negative for evidence of invasive fungal disease. The median delay between commencement of antifungal treatment and bronchoscopy was 6 days (range, 4 to 8 days).

TABLE 4.

Dual qPCR and LFD testing results and classification by 2008 EORTC/MSG criteria in unproven IPA cases^a

Sample no.	Dual test interpretationb	Interpretation by 2008 EORTC/MSG criteria (including GM test)
1	Positive	No IPA
2	Positive	No IPA
3	Positive	Probable IPA
4	Positive	Probable IPA
5	Indeterminate	No IPA
6	Negative	No IPA
7	Negative	No IPA
8	Negative	No IPA
9	Negative	Probable IPA
10	Negative	No IPA
11	Negative	No IPA
12	Negative	No IPA
13	Negative	No IPA
14	Negative	No IPA
15	Negative	No IPA
16	Negative	No IPA
17	Negative	No IPA
18	Negative	No IPA
19	Negative	Possible IPA
20	Negative	Possible IPA
21	Negative	Possible IPA
22	Negative	Possible IPA
23	Negative	Possible IPA
24	Negative	Probable IPA

^aData are for 24 samples. GM, galactomannan; qPCR, quantitative real-time PCR; LFD, lateral-flow device.

^bPositive, LFD and qPCR positive; negative, LFD and qPCR negative; indeterminate, any other combination of results.

The performance of the EORTC/MSG criteria was evaluated using the LFD-qPCR combination as the reference standard for a clinical diagnosis of IPA. In this reversal of roles, the EORTC/MSG criteria had an NPV of 86% (95% CI, 57 to 98%), with sensitivity, specificity, and PPV being 83%, 86%, and 83%, respectively (Table 5). In clinical practice, the question to be addressed is the management of a patient with suspected IPA in the absence of a proven diagnosis. Using the LFD-qPCR results to define the diagnosis of IPA in this setting, as shown in Table 4, the EORTC/MSG criteria had an NPV of 80% (95% CI, 52 to 95%), with the sensitivity, specificity, and PPV being 40%, 63%, and 22%, respectively.

TABLE 5.

Diagnostic performance of the EORTC/MSG criteria using the LFD and qPCR combination to assign a clinical diagnosis of IPA^a

Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Positive LR	Negative LR	DOR
83.33 (51.58–97.42)	85.71 (57.16–97.80)	83.33 (51.58–97.42)	85.71 (57.16–97.80)	5.83	0.19	30.00

^aCases with dual LFD and qPCR positivity were regarded as true positives, while cases with dual LFD and qPCR negativity were regarded as true negatives (the one indeterminate case was excluded from analysis). The sensitivities, specificities, positive predictive values, and negative predictive values for proven/probable IPA versus no IPA of the classification according to EORTC/MSG criteria are displayed, with 95% confidence intervals being given in parentheses. Abbreviations: qPCR, quantitative real-time PCR; LFD, lateral-flow device; NPV, negative predictive value; PPV, positive predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio.

DISCUSSION

Invasive pulmonary aspergillosis is the most common manifestation of IFD in hemato-oncology patients (25), and its diagnosis and treatment remain a challenge. We evaluated novel and established methods for the diagnosis of IPA. Aspergillus qPCR from BAL fluid was a promising single test method with a sensitivity of 100% in the proven IPA patients (82% in the proven/probable population) and a specificity of 87%. The novel, rapid LFD test also had a sensitivity of 100% in the proven IPA patients (82% in the proven/probable population) but a lower specificity of 80%. The GM test showed 75% sensitivity in the proven IPA cases when a cutoff index of 1.0, as deemed optimal by a recent meta-analysis of GM test-BAL fluid studies (13), was used but 88% sensitivity when a lower cutoff of 0.8 was used. The latter value may be more relevant to our clinical practices, where mold-active antifungal therapy was started in patients prior to bronchoscopy. Sensitivity was reduced further when the GM test results for the proven/probable population were analyzed (GM test with a cutoff of >1.0, 55%; GM test with a cutoff of >0.8, 82%). In all analyses, the GM test specificity was 67%. These findings are consistent with the disappointing GM test calcofluor staining performance reported in the largest prospective cohort of hematologic patients undergoing BAL to date (26).

The use of multiple biomarkers raises issues with regard to affordability, the ability of laboratories to deliver the tests in real time, as well as the interpretation of conflicting results. It is reassuring that all the Aspergillus qPCR-positive BAL fluid samples were positive by both the LFD and GM tests. Given the ubiquity of Aspergillus in the environment, combining qPCR with detection of a fungal protein would give clinicians greater confidence in using biomarker data for clinical management. The best combination was the Aspergillus qPCR with the LFD test, which had the highest diagnostic potential with BAL fluid samples, with a sensitivity of 100% in the proven IPA cases, a specificity of 86%, and almost perfect agreement between the two tests (Cohen's kappa coefficient, 0.93). These results support the findings of White et al. (14), in which the combination of LFD and qPCR testing was deemed optimal for the testing of serum.

In clinical practice, two factors may significantly impact biomarker detection: the use of mold-active drugs (as prophylaxis or treatment) and the availability of rapid bronchoscopy. There are conflicting reports about the effect of antifungal therapy on the performance of tests. Antifungal therapy has been reported to both decrease (27) and increase (28) the diagnostic performance of qPCR with BAL fluid. In a guinea pig model of IPA, the levels of antigen detected by LFD and GM test in the BAL fluid of animals treated with antifungals remained elevated, whereas serum sensitivity was reduced (29). In our study, the median interval between the start of mold-active antifungal treatment and bronchoscopy was 6 days for the dual LFD-qPCR-negative cases and 6 days for the dual LFD-qPCR-positive cases. The dual biomarker-positive BAL fluid samples were obtained after 4 to 8 days of mold-active antifungal therapy, which highlights the utility of biomarker testing in this setting, whereas microscopy and culture of BAL fluid were uninformative.

The EORTC/MSG criteria (5, 6) are often used as a diagnostic reference standard because of the rarity of proven IPA cases. However, misclassification by the use of these criteria can occur for many reasons; e.g., lung lesions may be missed due to delays in CT scanning (30), and there are multiple causes of lung lesions other than IPA. Furthermore, biomarker changes may be caused by a condition other than IPA (31) or by drug treatment (32). Furthermore, when evaluating the performance of the GM test, there is incorporation bias, as the test is part of the EORTC/MSG criteria for assigning a score of probable IPA, but this method of evaluation is still used (11). More recently, the limitations of using the EORTC/MSG criteria for assessing the value of a diagnostic test have been eloquently highlighted (33). In this study, we evaluated the performance of biomarker tests against the reference standard EORTC/MSG criteria using only proven cases as true positives and cases with no IFD by the EORTC/MSG criteria as true negatives. In a reversal of roles, the optimal biomarker approach was then used as the reference standard, with cases being classified according to the LFD-qPCR clinical diagnosis of IPA in order to evaluate the performance of the EORTC/MSG criteria. The sensitivity of the EORTC/MSG criteria was 83%. However, when the analysis was restricted to those cases without a proven diagnosis, which represents the situation in daily clinical practice, the sensitivity of the EORTC/MSG criteria was 40%, in keeping with evaluations of the performance of the EORTC/MSG criteria in studies of autopsy-proven cases (18–21). The EORTC/MSG criteria were developed for clinical research and were not designed for evaluating new approaches for the diagnosis of IFD. Vehreschild (33) proposes an alternative strategy, not based on the EORTC/MSG criteria: a comprehensive diagnostic approach, under the controlled conditions of an interventional clinical trial, in which patients who are negative for all biomarkers are classified as true negatives.

This study has a number of limitations. Selection of adult patients who are at high risk of developing IPA and who have had a bronchoscopy leads to a high pretest probability of IFD. The timing of the bronchoscopy in relation to the course of IFD may also introduce bias, as sample collection can occur at various stages of disease. However, the study patients followed a standardized protocol (see Materials and Methods), and in clinical practice such bias could be minimized by using integrated care pathways (34). BAL fluid sampling is also not standardized, raising issues about the interpretation of biomarker thresholds and quantification.

In conclusion, our study shows that the Aspergillus qPCR and the LFD test had the highest potential for the diagnosis of IPA in BAL fluid samples. A systematic approach to bronchoscopy combined with dual LFD and qPCR testing could make a significant difference to IFD management. We envisage LFD testing as a near-patient test in the bronchoscopy suite, allowing an immediate decision on antifungal drug usage, with qPCR results being available within 24 h for a definitive diagnosis. Studies with larger sample sizes are needed to further evaluate this diagnostic approach.