Measuring serum beta-d-glucan (BDG) is a useful tool for supporting a quantitative PCR (qPCR)-based diagnosis of suspected Pneumocystis pneumonia (PCP) with bronchoalveolar lavage (BAL) fluid. Since the 2000s, the Fungitell assay was the only BDG assay which was FDA cleared and Conformité Européenne (CE) marked.
KEYWORDS: beta-d-glucan, diagnosis, Pneumocystis jirovecii, pneumocystis pneumonia, prognosis
ABSTRACT
Measuring serum beta-d-glucan (BDG) is a useful tool for supporting a quantitative PCR (qPCR)-based diagnosis of suspected Pneumocystis pneumonia (PCP) with bronchoalveolar lavage (BAL) fluid. Since the 2000s, the Fungitell assay was the only BDG assay which was FDA cleared and Conformité Européenne (CE) marked. However, the Wako β-glucan test was also recently CE marked and commercialized. We analyzed archived sera from 116 PCP cases (who were considered to have PCP based on compatible clinical and radiological findings plus a BAL fluid qPCR threshold cycle value of ≤28) and 114 controls (those with a BAL fluid qPCR threshold cycle value of >45 and no invasive fungal infection) using the Fungitell and Wako assays in parallel and assessed their diagnostic performance using the manufacturer’s proposed cutoffs of 80 pg/ml and 11 pg/ml, respectively. We found the Wako assay to be more specific (0.98 versus 0.87, P < 0.001) and the Fungitell assay to be more sensitive (0.78 versus 0.85, P = 0.039) at the proposed cutoffs. Overall performance, as determined by the area under the receiver operating characteristic curve, was similar for both assays. We determined a new Wako assay cutoff (3.616 pg/ml) to match the sensitivity of the Fungitell assay (0.88 at a cutoff of ≥60 pg/ml). Using this newly proposed cutoff, the specificity of the Wako assay was significantly better than that of the Fungitell assay (0.89 versus 0.82, P = 0.011). In conclusion, the Wako assay performed excellently compared to the Fungitell assay for the diagnosis of presumed PCP based on qPCR. In addition, contrary to the Fungitell assay, the Wako assay allows for single-sample testing with lower inter- and intrarun variability. Finally, we propose an optimized cutoff for the Wako assay to reliably exclude PCP.
INTRODUCTION
Pneumocystis pneumonia (PCP) is an opportunistic pulmonary fungal infection caused by Pneumocystis jirovecii in patients with reduced cellular immunity, especially in those with lowered CD4+ T lymphocyte counts (1). The diagnosis classically relied on direct microscopy of respiratory samples (sputum, tracheal aspirate, bronchoalveolar lavage fluid [BALf], or biopsy samples) after tinctorial staining and with immunofluorescent staining since the 1990s (2). Later, Pneumocystis-specific PCRs were introduced, further increasing the diagnostic sensitivity (3, 4). However, given the high sensitivity of these PCR tests, some patients have positive test results while all other clinical and radiological data suggest the absence of active disease. In such patients and in patients from which no high-quality respiratory sample can be obtained (e.g., due to hypoxemia, which is present in up to half of all patients [5]), serum (1,3)-beta-d-glucan (BDG) can be helpful to exclude the possibility of PCP. Assays for BDG have been shown to have a good sensitivity of 91% to 96% in 3 meta-analyses (6–8). However, as assays for BDG also detect other fungal infections, they are less useful in making a definite diagnosis.
Several assays for the detection of BDG exist, including the Fungitec G assay (Seikagaku Corporation), the Fungitell assay (Associates of Cape Cod), the Dynamiker Fungus (1-3)-β-d-glucan assay (Dynamiker Biotechnology), the GKT-25M assay (Tianjin Era Biology Technology Co.), and the Wako β-glucan assay (Wako-BDG; Fujifilm Wako Chemicals). Until recently, only the Fungitell assay was Conformité Européenne (CE) marked and U.S. Food and Drug Administration (FDA) cleared. As such, Fungitell has been the predominant assay in Europe and North America. This assay uses Limulus amebocyte lysate (LAL), resulting in a change in color to yellow in the presence of BDG. However, due to this colorimetric reaction, interference from hemolytic, lipemic, or icteric samples is possible. Furthermore, as the assay is run on a 96-well plate including 4 calibration standards tested in duplicate, the assay requires a large sample number to be cost-efficient, leading to batching and an increased turnaround time. Finally, to obtain accurate results, the manufacturer recommends that all samples be run at least in duplicate. Despite this, Fungitell has an intra- and interassay coefficient of variation (CV) of up to 28.9% and 23.8%, respectively, according to the package insert. To minimize this variability, discordant duplicates are often retested, leading to a further increase in the turnaround time and cost.
Recently, the BDG assay from Fujifilm Wako was CE marked. Like Fungitell, Wako-BDG measures BDG concentrations using LAL. However, this assay results in a change in turbidity in the presence of BDG. This assay is therefore not influenced by hemolytic, lipemic, or icteric samples. Furthermore, gelation is assessed by a dedicated device which accepts single-sample vials, obviating the need for batching. Calibration to a standard curve is performed at the manufacturer on a per lot basis, obviating the need for calibration samples in every run. Finally, the repeatability of the assay, as reported by the manufacturer, appears to be very good, with an intra- and interrun CV of up to 4.7% and 6.6%, respectively.
We assessed the clinical and analytical performance of Wako-BDG in the diagnosis of PCP in both HIV-infected and non-HIV-infected patients and directly compared it to that of the Fungitell assay.
MATERIALS AND METHODS
We performed a retrospective case-control study on archived serum samples collected as part of routine clinical care between January 2010 and July 2018 at the University Hospitals Leuven, Leuven, Belgium, a tertiary care center. We included PCP cases and controls in a 1:1 ratio, stratified by HIV serostatus. Based on an expected sensitivity of 0.92 (8), a 95% confidence interval (CI) 5 percentage points wide, a power of 0.8, and an alpha value of 0.95, we required a sample size of 227 patients. Starting from all patients who had a Pneumocystis-specific quantitative PCR (qPCR) performed on BALf, we included patients that (i) had either strongly positive qPCR results (threshold cycle [CT] ≤ 28) or strictly negative qPCR results (threshold cycle > 45), (ii) had had a chest X-ray or computed tomography within 1 week of BALf sampling, and (iii) had available a serum sample collected from between 48 h before BALf sampling up to 7 days after BALf sampling that had been stored at less than or equal to −20°C. Patients with a positive qPCR result in the absence of clinical signs or symptoms of respiratory infection and patients with other fungal infections, as defined by the 2008 EORTC-MSG consensus definitions (9), were excluded. This study was approved by the Ethical Committee Research UZ/KU Leuven (reference number S61842). Due to the retrospective nature of this study, the need for informed consent was waived. This study was performed in accordance with the 2015 STARD guidelines for diagnostic accuracy studies (10).
Case definitions and data collection.
As qPCR alone is still not considered a gold standard for diagnosing PCP (as it can also detect colonization), we defined patients as having PCP (cases) if they had (i) a strongly positive qPCR result (threshold cycle ≤ 28); (ii) bilateral ground glass opacities on chest computed tomography or bilateral diffuse infiltrates on chest X ray; and (iii) any of the following clinical signs and symptoms of respiratory infection: dyspnea, cough, or hypoxemia. We defined controls as patients that (i) were immunocompromised due to HIV infection or due to any other immunocompromising factor, including but not limited to immunosuppressive drugs, leucopenia, and congenital immunodeficiencies; (ii) had any of the above-mentioned clinical signs and symptoms of respiratory infection; and (iii) had strictly negative qPCR results (threshold cycle > 45) on BALf. Data collection and disease classification were performed by investigators blind to the outcome of BDG testing. We collected from the electronic patient records a list of all pharmaceutical drugs administered to the patient within 24 h before serum sampling.
P. jirovecii PCR.
BALf samples from all patients were tested with an in-house-developed real-time PCR assay targeting mitochondrial rRNA from P. jirovecii, as described previously (11). Briefly, Pneumocystis DNA was extracted from BALf using an easyMAG instrument (bioMérieux, Marcy l'Etoile, France). The assay was run on an Applied Biosystems ABI 7900Fast real-time thermocycler (Thermo Fisher Scientific, Waltham, MA, USA; before December 2014) or on a Quantstudio Dx real-time PCR instrument (Thermo Fisher Scientific, Waltham, MA, USA; from December 2014 onwards). PCR analyses were considered negative if the threshold cycle (CT) values exceeded 45 cycles.
Beta-d-glucan analysis.
Archived samples were thawed at room temperature and briefly vortexed before testing. Both assays were performed in parallel by the same researcher in accordance with the manufacturer’s instructions. The researcher was blind to the disease classification of each patient at the time of testing.
(i) Fungitell assay.
Briefly, for the Fungitell assay (Associates of Cape Cod, East Falmouth, MA, USA), 5 µl of serum was added to 20 µl of pretreatment reagent in a glucan-free 96-well plate. After incubation (10 min) at 37°C, 100 µl of Fungitell reagent was added to the pretreated sample. Kinetic colorimetric results were measured for 40 min at 37°C using a Multiskan photometer (Thermo Fisher Scientific, Waltham, MA, USA) at 405 nm minus 492 nm. The mean change in the absorbance over time was calculated and compared to a standard curve to determine the BDG concentration. All samples were tested in duplicate. Samples were retested if the interpretation of the duplicates was discordant in combination with a CV of the duplicates above 20%.
(ii) Wako-BDG.
Briefly, for the Wako-BDG (β-glucan test; Fujifilm Wako Pure Chemical Corporation, Osaka, Japan), 100 µl of serum was added to 900 µl of pretreatment solution. After incubation (10 min) at 70°C, the sample was cooled on ice. Two hundred microliters of the pretreated sample was then added to the LAL reagent. Kinetic turbidimetric results were measured for 90 min at 37°C using an MT-6500 toxinometer with a single extension module (Fujifilm Wako Pure Chemical Corporation, Osaka, Japan). The BDG concentration was calculated by comparing the gelation time to that on a calibration curve that is provided with each lot by the manufacturer.
Stability testing.
We selected five samples with increasing BDG concentrations, ranging from below the cutoff for positivity up to 20 times the cutoff. We divided each sample into 12 separate aliquots, half of which were stored at room temperature (21°C) and half of which were stored at 4°C. Wako-BDG was performed daily on a fresh aliquot of each concentration stored under each storage condition for 5 days.
Reproducibility testing.
For each assay, we selected three samples with a BDG concentration below the cutoff for positivity, at about twice the cutoff, and at about5 times the cutoff. Each sample was divided into 10 separate aliquots and stored at −20°C until further testing. Using these aliquots, we then determined the within-run variability using 5 aliquots from each sample in one run. Similarly, one aliquot from each sample was tested during 5 different runs to determine the between-run variability for each assay.
Statistical analysis.
We calculated the sensitivity, specificity, negative likelihood ratio (NLR), and positive likelihood ratio (PLR), together with their respective 95% confidence intervals (CIs). For the negative predictive value (NPV) and the positive predictive value (PPV), we used a prevalence of 20% to more easily allow comparison with previous studies. We compared the sensitivity and specificity of both assays using McNemar’s chi-square test, PPV and NPV using the weighted generalized score statistic (12), PLR and NLR using a regression model approach (13), and the area under the receiver operating characteristic (ROC) curve using DeLong's test for two correlated ROC curves, as all these comparisons are pairwise observations. We defined a 2-sided P value of ≤0.05 to be statistically significant. We assessed both assays with the cutoff values for positivity provided by the manufacturer (80 pg/ml for Fungitell and 11 pg/ml for Wako-BDG). We recalculated the diagnostic performance of Wako-BDG using the cutoff that matched the sensitivity of Fungitell when applying the 60-pg/ml cutoff to rule out disease (based on the ROC curve) and to obtain the highest Youden index. We assessed the influence of the levels determined by Wako-BDG on survival both by using univariate Cox regression and after correcting for age, gender, underlying disease, HIV serostatus, and lactate dehydrogenase (LDH) levels. The rate of false positives and false negatives within groups of patients receiving a drug, grouped either by chemical substance (level 5 of the Anatomical Therapeutic Chemical [ATC] classification system; WHO Collaborating Centre for Drug Statistics Methodology, ATC classification index with defined daily doses, 2019, Oslo, Norway), by chemical subgroup (level 4 of the ATC), or by pharmacological subgroup (level 3 of the ATC), was compared, using Fisher’s exact test, to that for patients not receiving this drug, provided that the drug was used in ≥5 cases and ≥5 controls. We controlled the false discovery rate due to the multiple comparisons using the method described by Benjamini and Hochberg (14). To determine the correlation between the levels determined by the Fungitell assay and Wako-BDG, we performed linear regression on samples where the concentration of BDG was within the measurable range of each assay (i.e., 31 to 500 pg/ml for Fungitell and 2.359 to 600 pg/ml for Wako-BDG). Negative samples according to one assay and positive at more than 5 times the cutoff in the other were considered to be caused by non-disease-related reactions (such as contamination during testing) and were subsequently excluded from the linear regression analysis due to their high leverage effect on regression analysis. These samples were still used in all other analyses.
RESULTS
Two hundred thirty patients were included in the study (Fig. 1). The clinical characteristics of the patients are shown in Table 1. Of all samples, 69% were collected within 24 h of BALf sampling and 90% were collected within 48 h. Seven samples (3%) required retesting with the Fungitell assay because of a CV of >20% with discordant qualitative results between the two duplicates. Furthermore, one run of 21 samples needed retesting because the correlation coefficient of the standard curve was <0.980 due to a single outlier with an unexpectedly high value.
FIG 1.
STARD diagram of the flow of patients through this study.
TABLE 1.
Patient characteristicsa
| Characteristic | Value for the following patients: |
P value | |
|---|---|---|---|
| Cases (n = 116) | Controls (n = 114) | ||
| No. (%) of male patients | 64 (55.2) | 69 (60.5) | 0.491 |
| Median (IQR) age (yr) | 64 (55, 73) | 59 (43, 67) | 0.002 |
| No. (%) of patients infected with HIV | 13 (11.2) | 8 (7.0) | 0.382 |
| Median (IQR) CD4+ T cell count (no. of cells/µl) | 35.5 (7, 106) | 69 (0.5, 172.5) | 0.879 |
| Median (IQR) lymphocyte count (no. of cells/µl) | 400 (200, 800) | 500 (200, 950) | 0.459 |
| Median (IQR) LDH level (fraction of the upper limit of normal) | 1.52 (1.15, 2.13) | 1.01 (0.74, 1.35) | <0.001 |
| No. (%) of patients with the following underlying disease or condition: | |||
| Hematologic malignancy | 26 (22.4) | 39 (34.2) | |
| Solid tumor | 25 (21.6) | 12 (10.5) | |
| Solid organ transplant | 17 (14.7) | 11 (9.6) | |
| Allogeneic HSCT | 15 (12.9) | 25 (21.9) | |
| HIV infection | 13 (11.2) | 8 (7.0) | |
| Other | 20 (17.2) | 19 (16.7) | |
| No. (%) of patients alive at: | |||
| Wk 6 | 68 (58.6) | 88 (77.2) | 0.004 |
| Wk 12 | 62 (53.4) | 79 (69.3) | 0.020 |
IQR, interquartile range; HSCT, hematopoietic stem cell transplantation; HIV, human immunodeficiency virus; LDH, lactate dehydrogenase.
Diagnostic performance.
The diagnostic performance of Fungitell (cutoff for positivity, ≥80 pg/ml) and Wako-BDG (cutoff for positivity, ≥11 pg/ml) for the diagnosis of PCP is shown in Table 2. Predictive values at a prevalence of 20% are shown in Table 3. The median BDG levels of the cases were 781.825 pg/ml and 42.825 pg/ml for the Fungitell assay and Wako-BDG, respectively. The ROC curves of both assays had a similar area under the curve (0.930 versus 0.936, P = 0.72; Fig. 2). Nine samples (3/116 cases [2.6%] and 6/114 controls [5.3%]) had a Fungitell BDG level of between 60 and 80 pg/ml and were considered to have indeterminate results according to the manufacturer’s instructions. Since Fungitell is often used to exclude the possibility of disease (at a cutoff for negativity of 60 pg/ml), we determined a new cutoff for Wako-BDG (3.616 pg/ml) with the same sensitivity as that for a Fungitell cutoff of 60 pg/ml. Finally, we determined a new cutoff for Wako-BDG (6.222 pg/ml) with the highest Youden index, which aims for the maximal combination of sensitivity and specificity. The diagnostic performance of these new cutoffs is shown in Table 2, and these were compared to the respective performance of Fungitell. Contingency tables for all subanalyses are provided in Table 4. The correlation between the levels determined by Wako-BDG and Fungitell is shown in Fig. 3. We found a linear relationship between the concentrations reported by both assays, with Fungitell levels equal to 18.36 times those of Wako-BDG + 52.03 (adjusted R2 = 0.71, P < 0.001).
TABLE 2.
Diagnostic performance of both beta-d-glucan assays using different cutoffs
| Cutoff used and parameter | Value for the following assay: |
P value | |
|---|---|---|---|
| Wako | Fungitell | ||
| Manufacturer’s recommended cutoffs | |||
| Cutoff (pg/ml) | 11 | 80 | |
| % sensitivity (95% CIa ) | 0.78 (0.69, 0.85) | 0.85 (0.78, 0.91) | 0.039 |
| % specificity (95% CI) | 0.98 (0.94, 1) | 0.87 (0.79, 0.92) | <0.001 |
| Positive likelihood ratio (95% CI) | 44.22 (11.16, 175.29) | 6.49 (4.02, 10.46) | 0.004 |
| Negative likelihood ratio (95% CI) | 0.23 (0.16, 0.32) | 0.17 (0.11, 0.26) | 0.151 |
| Modified Wako cutoff to get a sensitivity equal to that achieved with a Fungitell cutoff of ≥60 pg/ml | |||
| Cutoff (pg/ml) | 3.616 | 60 | |
| % sensitivity (95% CI) | 0.87 (0.8, 0.93) | 0.88 (0.81, 0.93) | 0.763 |
| % specificity (95% CI) | 0.89 (0.81, 0.94) | 0.82 (0.73, 0.88) | 0.011 |
| Positive likelihood ratio (95% CI) | 7.64 (4.56, 12.8) | 4.77 (3.22, 7.07) | 0.016 |
| Negative likelihood ratio (95% CI) | 0.15 (0.09, 0.24) | 0.15 (0.09, 0.24) | 0.953 |
| Modified Wako cutoff with highest Youden index | |||
| Cutoff (pg/ml) | 6.222 | 80 | |
| % sensitivity (95% CI) | 0.84 (0.77, 0.91) | 0.85 (0.78, 0.91) | 0.763 |
| % specificity (95% CI) | 0.96 (0.9, 0.99) | 0.87 (0.79, 0.92) | 0.002 |
| Positive likelihood ratio (95% CI) | 19.26 (8.15, 45.55) | 6.49 (4.02, 10.46) | 0.003 |
| Negative likelihood ratio (95% CI) | 0.16 (0.11, 0.25) | 0.17 (0.11, 0.26) | 0.839 |
CI, confidence interval.
TABLE 3.
Predictive values of both assays using different cutoffs at a prevalence of 20%
| Cutoff used and parameter | Value for the following assay: |
P value | |
|---|---|---|---|
| Wako | Fungitell | ||
| Manufacturer’s recommended cutoffs | |||
| Cutoff (pg/ml) | 11 | 80 | |
| Positive predictive value (% [95% CIa]) | 0.92 (0.85–0.96) | 0.62 (0.54–0.70) | <0.001 |
| Negative predictive value (% [95% CI]) | 0.95 (0.92–0.96) | 0.96 (0.93–0.98) | 0.146 |
| Modified Wako cutoff to get a sensitivity equal to that achieved with a Fungitell cutoff of ≥60 pg/ml | |||
| Cutoff (pg/ml) | 3.616 | 60 | |
| Positive predictive value (% [95% CI]) | 0.66 (0.58–0.74) | 0.55 (0.47–0.62) | <0.001 |
| Negative predictive value (% [95% CI]) | 0.97 (0.94–0.98) | 0.96 (0.94–0.98) | 0.716 |
| Modified Wako cutoff with highest Youden index | |||
| Cutoff (pg/ml) | 6.222 | 80 | |
| Positive predictive value (% [95% CI]) | 0.83 (0.75–0.89) | 0.62 (0.54–0.70) | <0.001 |
| Negative predictive value (% [95% CI]) | 0.96 (0.94–0.98) | 0.96 (0.93–0.98) | 0.837 |
CI, confidence interval.
FIG 2.
Receiver operating characteristic (ROC) curves of the Fungitell and Wako assays. The official threshold for positivity is marked with a dot.
TABLE 4.
Contingency tables for both assays using the official cutoffs for positivity, a modified cutoff for Wako-BDG to achieve the same sensitivity and NPV as Fungitell, and a modified cutoff for Wako-BDG with the highest Youden indexa
| Cutoff used | Wako assay |
Fungitell assay |
||||
|---|---|---|---|---|---|---|
| Cutoff (pg/ml) | No. of patients |
Cutoff (pg/ml) | No. of patients |
|||
| PCP | No PCP | PCP | No PCP | |||
| Official cutoff for positivity | Positive, ≥11 pg/ml | 90 | 2 | Positive, ≥80 pg/ml | 99 | 15 |
| Negative, <11 pg/ml | 26 | 112 | Negative, <80 pg/ml | 17 | 99 | |
| Modified cutoff for Wako-BDG to achieve the same sensitivity and NPV as Fungitell | Positive, ≥3.616 pg/ml | 101 | 13 | Positive, ≥60 pg/ml | 102 | 21 |
| Negative, <3.616 pg/ml | 15 | 101 | Negative, <60 pg/ml | 14 | 93 | |
| Modified cutoff for Wako-BDG with the highest Youden index | Positive, ≥6.222 pg/ml | 98 | 5 | Positive, ≥80 pg/ml | 99 | 15 |
| Negative, <6.222 pg/ml | 18 | 109 | Negative, <80 pg/ml | 17 | 99 | |
PCP, pneumocystis pneumonia; Wako-BDG, Fujifilm Wako β-glucan test; Fungitell, Associates of Cape Cod Fungitell assay.
FIG 3.
Correlation between beta-d-glucan levels measured by the Wako and Fungitell assays. (a) Overview of all values. The area shaded in red marks values outside the reliable range, as defined by the manufacturers. (b) Regression plot of measurements within the reliable range.
We analyzed 260 unique molecules from a total of 2,158 drug administrations within 24 h before serum sampling. Of these, only 36 were used at least 5 times in cases and controls and were included for further analysis. Propofol, albumin, and paracetamol were significantly more likely to be used in samples that were falsely positive, according to Wako-BDG (P = 0.002, 0.006, and 0.020, respectively), and amlodipine was more likely to be used in samples with false-negative results (P = 0.038). However, after correcting for multiple comparisons, none of these remained significant. There was substantial—although not full—concordance between the two assays regarding the false-positive and -negative results (94.8% agreement; Cohen’s kappa statistic, 0.72).
The levels of BDG were not predictive of survival both in univariate analysis (Wako-BDG, P = 0.876; Fungitell, P = 0.278) and in multivariate Cox regression analysis (Wako-BDG, P = 0.978; Fungitell, P = 0.468).
Stability and reproducibility.
The average change in concentration over 5 days at room temperature was −1.14 ± 0.41 percentage points per day (P = 0.009). At 4°C, there was no significant change in concentration over 5 days (average change, −0.73 ± 0.79 percentage points; P = 0.365). The coefficient of variation ranged from 1.8 to 13.2% for Wako-BDG and from 4.2 to 31.6% for Fungitell for the within-run variability and from 3.3 to 7.6% for Wako-BDG and from 6.0 to 26.9% for the between-run variability.
DISCUSSION
The Fungitell assay has been used throughout Europe and North America since 2004, whereas the CE-marked Wako-BDG assay was introduced to the European market only in April 2018. To date, only two studies have compared the diagnostic performance of the two assays: one study with a small number (n = 12) of patients with proven or probable invasive fungal disease (15) and the other one in patients suffering predominantly from candidemia (16). In the present study, we compared the performance of these two BDG assays for diagnosing PCP in 116 PCP cases (both HIV-infected and HIV-non-infected patients) and 114 controls. The overall performance of the two assays, as assessed by the area under the ROC curves, was similar, which contrasts with a previous study performed (mainly) in patients with candidemia (16). This might be explained by the testing of all samples in parallel, which decreases the likelihood of degradation or contamination during storage. In the study by Friedrich et al. (16), Fungitell was performed at the time of sampling, but Wako-BDG was performed after the samples had been stored for 5 to 17 years.
Using the manufacturer’s recommended cutoffs for positivity, the Fungitell assay was significantly more sensitive, although Wako-BDG was significantly more specific with a higher positive predictive value and a higher positive likelihood ratio. These results are in line with those of a previous report from a smaller population (n = 63) of PCP patients (16). However, that study did not include any relevant controls, excluding the possibility that any conclusions with regard to specificity, likelihood ratios, and predictive values could be made.
As the cutoff for Wako-BDG was determined in the era before qPCR was an accepted diagnostic tool, less sensitive methods, such as microscopy, were used as a reference. However, qPCR has since been shown to have a greater sensitivity than microscopy (4). When using a reference standard with a lower sensitivity (such as microscopy), any comparator with a higher sensitivity (such as qPCR) results in false-positive results in cases that in reality may be true positive. We therefore tried to optimize the cutoff for the assay according to the Youden index using a case definition which included qPCR as a contemporary reference standard. This resulted in a lower cutoff of 6.222 pg/ml, which had a sensitivity and an NLR for diagnosing PCP similar to those of the Fungitell assay but an excellent and significantly higher specificity and PLR compared to those of the Fungitell assay.
Furthermore, serum BDG is often used to exclude PCP in patients who are unable to undergo BALf sampling. This requires a high sensitivity and NPV to reliably rule out disease in order to withhold treatment. We therefore determined the cutoff for Wako-BDG at which its sensitivity was the same as that of Fungitell (at a cutoff of 60 pg/ml for negativity). At a cutoff of 3.616 pg/ml, both assays had a similar sensitivity, but Wako-BDG had a significantly higher specificity, PPV, and PLR. This cutoff is very similar to the lower cutoff of 3.8 pg/ml suggested in a study of Wako-BDG in candidemia (16). However, the NPV and PPV are influenced by the prevalence of disease in the test population. As we used a case/control ratio of 1:1, the 50% prevalence of PCP in our study was artificially high. For most populations in which BDG is used, this prevalence would be significantly lower, resulting in a higher NPV and a lower PPV (Fig. 4). For example, at a prevalence of 20%, the NPV of Wako-BDG at a cutoff of 11 pg/ml was excellent at 0.95 and even better when using the modified cutoff of 3.616 pg/ml to exclude the presence of disease at 0.97.
FIG 4.
Positive and negative predictive values of the Wako assay depending on the prevalence of disease, using both the official cutoff (a) and a modified cutoff to achieve the same sensitivity as the Fungitell assay (b) (cutoff, 3.616 pg/ml).
In our stability testing, we saw only minor decreases in BDG levels by Wako-BDG over time at room temperature and no significant decrease at 4°C. This decrease was less pronounced than the decrease described in the package insert (−6% after 3 days at 4°C and −20% after 2 days at 25°C). This discrepancy could potentially be explained by the early degradation of BDG in vitro followed by relative stable concentrations. As our samples were archived, this could not be assessed in our experiment. Immediate analysis of freshly collected samples should resolve this issue.
We aimed to perform a high-quality comparative assessment of both assays by using a large cohort of cases and controls, by using sera collected close to the BALf sampling date (90% of samples were collected within 48 h), and by using a standard which incorporates clinical, radiological, and microbiological criteria. However, our study certainly has its limitations. First, our study included only a small number of samples from HIV-infected patients. Although previous studies with the Fungitell assay did not find a difference in performance between HIV-infected and non-HIV-infected patients, more data are needed for the Wako assay (6). Second, our study used an artificially high prevalence of 50%, which clearly influenced the predictive value of the test. Therefore, we provided additional data in Table 4 and Fig. 4 to allow clinicians to find the actual NPV and PPV for any given prevalence. We provided the predictive values for a prevalence of 20%, to allow comparison with previous studies. Lastly, we did not include any patients with a weakly positive PCR result due to a lack of a good diagnostic comparator in this setting, even though BDG testing is of particular interest in this population to provide additional diagnostic information. However, using Bayesian inference, we believe that the high NPV indicates that Wako-BDG can reliably be used in this setting.
In conclusion, our comparative analysis of testing for serum BDG for the diagnosis of PCP showed that Wako-BDG performed as well as or even better than the Fungitell assay, especially when optimizing the Wako-BDG cutoff value for positivity. Both assays had an excellent NPV, making them both useful to exclude PCP in patients that cannot undergo BALf sampling. Wako-BDG showed less inter- and intrarun variability and a good stability at 4°C. Collectively, the excellent performance as well as the technical flexibility of the single-sample Wako-BDG allows clinicians to make a timely diagnosis of presumed PCP also in centers with a low sample throughput.
ACKNOWLEDGMENTS
This work was supported by an unrestricted research grant from Fujifilm Wako Chemicals and the Research Foundation—Flanders (FWO; T004517N to T.M. and E.G.). Fujifilm Wako Chemicals provided the funding for the purchase of the Fungitell tests required for this study.
Neither Fujifilm Wako Chemicals nor the Research Foundation—Flanders had any role in the design of this study, its execution, analysis, interpretation of the data, or the decision to publish.
J.M., K.L., and T.M. designed the experiment. E.G. and T.M. collected the BALf samples. T.M. collected the clinical data, performed the experiments, analyzed the data, and wrote the initial draft. All authors critically revised the initial draft and approved the final manuscript.
J.M. has received research grants from Merck, Gilead Sciences, and Pfizer; is a consultant to Astellas, Basilea, Bio-Rad, Merck/MSD, Pfizer, Schering-Plough, F2G, Gilead Sciences, Cidara, Scynexis, Amplyx, and Luminex; and has served on the speaker’s bureaus of Astellas, Gilead Sciences, Bio-Rad, Merck, Pfizer, Schering-Plough, Basilea, and Viropharma/Shire. K.L. has received research grants from Gilead Sciences, Merck/MSD, and Pfizer; has received consultancy fees from Gilead Sciences, Pfizer, Abbott, MSD, and SMB Laboratoires Brussels; has received travel support from Pfizer, Gilead Sciences, and MSD; and has received speaker fees from Gilead Sciences, Roche, and Abbott. T.M. has received nonfinancial support from IMMY, OLM Diagnostics, Merck/MSD, and Gilead Sciences and consultancy fees from Gilead Sciences. E.G., K.B., and S.P. have no conflicts to report.
REFERENCES
- 1.Salzer HJF, Schäfer G, Hoenigl M, Günther G, Hoffmann C, Kalsdorf B, Alanio A, Lange C. 2018. Clinical, diagnostic, and treatment disparities between HIV-infected and non-HIV-infected immunocompromised patients with Pneumocystis jirovecii pneumonia. Respiration 96:52–65. doi: 10.1159/000487713. [DOI] [PubMed] [Google Scholar]
- 2.Alanio A, Hauser PM, Lagrou K, Melchers WJG, Helweg-Larsen J, Matos O, Cesaro S, Maschmeyer G, Einsele H, Donnelly JP, Cordonnier C, Maertens J, Bretagne S, Agrawal S, Akova M, Alanio A, Aljurf M, Averbuch D, Berg T, Blennow O, Bretagne S, Brüggemann R, Calandra T, Castagnola E, Cesaro S, Cordonnier C, Cornely O, De La Camara R, Donnelly P, Drgona L, Duarte R, Einsele H, Engelhard D, Girmenia C, Hargreaves R, Hauser P, Helweg-Larsen J, Herbrecht R, Hirsch H, Hubacek P, Kibbler C, Klyasova G, Kouba M, Kullberg B-J, Lagrou K, Ljungman P, Maertens J, Mallet V, Marchetti O, Maschmeyer G, et al. 2016. ECIL guidelines for the diagnosis of Pneumocystis jirovecii pneumonia in patients with haematological malignancies and stem cell transplant recipients. J Antimicrob Chemother 71:2386–2396. doi: 10.1093/jac/dkw156. [DOI] [PubMed] [Google Scholar]
- 3.Fan L-C, Lu H-W, Cheng K-B, Li H-P, Xu J-F. 2013. Evaluation of PCR in bronchoalveolar lavage fluid for diagnosis of Pneumocystis jirovecii pneumonia: a bivariate meta-analysis and systematic review. PLoS One 8:e73099. doi: 10.1371/journal.pone.0073099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lu Y, Ling G, Qiang C, Ming Q, Wu C, Wang K, Ying Z. 2011. PCR diagnosis of pneumocystis pneumonia: a bivariate meta-analysis. J Clin Microbiol 49:4361–4363. doi: 10.1128/JCM.06066-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.McKinnell JA, Cannella AP, Kunz DF, Hook EW, Moser SA, Miller LG, Baddley JW, Pappas PG. 2012. Pneumocystis pneumonia in hospitalized patients: a detailed examination of symptoms, management, and outcomes in HIV-infected and HIV-uninfected persons. Transpl Infect Dis 14:510–518. doi: 10.1111/j.1399-3062.2012.00739.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Onishi A, Sugiyama D, Kogata Y, Saegusa J, Sugimoto T, Kawano S, Morinobu A, Nishimura K, Kumagai S. 2012. Diagnostic accuracy of serum 1,3-β-d-glucan for Pneumocystis jiroveci pneumonia, invasive candidiasis, and invasive aspergillosis: systematic review and meta-analysis. J Clin Microbiol 50:7–15. doi: 10.1128/JCM.05267-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Karageorgopoulos DE, Qu J-M, Korbila IP, Zhu Y-G, Vasileiou VA, Falagas ME. 2013. Accuracy of β-d-glucan for the diagnosis of Pneumocystis jirovecii pneumonia: a meta-analysis. Clin Microbiol Infect 19:39–49. doi: 10.1111/j.1469-0691.2011.03760.x. [DOI] [PubMed] [Google Scholar]
- 8.Li W-J, Guo Y-L, Liu T-J, Wang K, Kong J-L. 2015. Diagnosis of pneumocystis pneumonia using serum (1-3)-β-d-glucan: a bivariate meta-analysis and systematic review. J Thorac Dis 7:2214–2225. doi: 10.3978/j.issn.2072-1439.2015.12.27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.de Pauw B, Walsh TJ, Donnelly JP, Stevens DA, Edwards JE, Calandra T, Pappas PG, Maertens J, Lortholary O, Kauffman CA, Denning DW, Patterson TF, Maschmeyer G, Bille J, Dismukes WE, Herbrecht R, Hope WW, Kibbler CC, Kullberg BJ, Marr KA, Muñoz P, Odds FC, Perfect JR, Restrepo A, Ruhnke M, Segal BH, Sobel JD, Sorrell TC, Viscoli C, Wingard JR, Zaoutis T, Bennett JE. 2008. Revised definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group (EORTC/MSG) Consensus Group. Clin Infect Dis 46:1813–1821. doi: 10.1086/588660. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HCW, Kressel HY, Rifai N, Golub RM, Altman DG, Hooft L, Korevaar DA, Cohen JF, STARD Group. 2015. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 351:h5527. doi: 10.1136/bmj.h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Van Eldere J, Padalko E, Vandeven J, Rijnders BJ. 2005. Comparison between quantitative PCR and immunofluorescence for detection of Pneumocystis jiroveci in respiratory samples, abstr M170. Abstr 45th Intersci Conf Antimicrob Agents Chemother. American Society for Microbiology, Washington, DC. [Google Scholar]
- 12.Kosinski AS. 2013. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Stat Med 32:964–977. doi: 10.1002/sim.5587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gu W, Pepe MS. 2009. Estimating the capacity for improvement in risk prediction with a marker. Biostatistics 10:172–186. doi: 10.1093/biostatistics/kxn025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 57:289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x. [DOI] [Google Scholar]
- 15.Yoshida K, Shoji H, Takuma T, Niki Y. 2011. Clinical viability of Fungitell, a new (1→3)-β-d-glucan measurement kit, for diagnosis of invasive fungal infection, and comparison with other kits available in Japan. J Infect Chemother 17:473–477. doi: 10.1007/s10156-010-0198-6. [DOI] [PubMed] [Google Scholar]
- 16.Friedrich R, Rappold E, Bogdan C, Held J. 2018. Comparative analysis of the Wako β-glucan test and the Fungitell assay for diagnosis of candidemia and Pneumocystis jirovecii pneumonia. J Clin Microbiol 56:e00464-18. doi: 10.1128/JCM.00464-18. [DOI] [PMC free article] [PubMed] [Google Scholar]