Validation of the MycAssay Pneumocystis Kit for Detection of Pneumocystis jirovecii in Bronchoalveolar Lavage Specimens by Comparison to a Laboratory Standard of Direct Immunofluorescence Microscopy, Real-Time PCR, or Conventional PCR

Abstract

Pneumocystis jirovecii pneumonia is a significant cause of morbidity and mortality in AIDS patients as well as those with non-HIV immunosuppressive diseases. To aid diagnosis, the commercial MycAssay Pneumocystis real-time PCR assay (Myconostica, Ltd., Manchester, United Kingdom) targeting the mitochondrial ribosomal large subunit (mtLSU) has been developed to detect P. jirovecii in bronchoalveolar lavage (BAL) specimens. Here, we validated this assay against a laboratory standard of direct immunofluorescence microscopy, a cdc2 real-time PCR assay, or conventional PCR and sequencing of mtLSU. While more sensitive than any of these three assays analyzed individually, the MycAssay Pneumocystis assay demonstrated 100% sensitivity, 100% specificity, a 100% negative predictive value, and a 100% positive predictive value for detecting the presence of P. jirovecii in BAL specimens compared to the laboratory standard. Of note, two samples with positive cycle threshold (C_T) values according to the MycAssay Pneumocystis assay lacked exponential amplification curves and thus were deemed negative. Also negative according to the laboratory standard, these samples highlight the importance of examining the amplification curves, in addition to noting the C_T values, when interpreting positive results. Comparison of the MycAssay Pneumocystis assay to a laboratory standard establishes this assay to be a highly sensitive and specific method for the detection of P. jirovecii in bronchoalveolar lavage specimens. The approach may also be useful for the clinical laboratory validation of other sensitive real-time PCR assays.

INTRODUCTION

Pneumocystis pneumonia (PCP) is a severe respiratory infection that affects immunocompromised individuals (6). It is the most prevalent opportunistic infection among AIDS patients, with a mortality rate ranging from 10% to 20% (12). PCP also afflicts patients with non-HIV immunosuppressive diseases, such as cancer, organ transplantation, and autoimmune or inflammatory diseases, especially if given long-term steroid therapy (4, 16). The mortality rate for non-HIV patients is as high as 35% to 55% (14). The preferred treatment regimen of high-dose trimethoprim-sulfamethoxazole plus corticosteroids can result in toxicity, severe rash, fever, or neutropenia (10), necessitating a switch to second-line treatment options such as clindamycin-primaquine, atovaquone, or pentamidine. However, these drugs also have significant side effects and are often associated with relapse and recurrence (6). Due to the severity of the disease and the potential adverse effects of treatment, accurate diagnosis is essential. However, the symptoms of PCP are nonspecific (fever, cough, and dyspnea) (5), such that accurate diagnosis relies heavily on laboratory testing.

Since the causative agent, Pneumocystis jirovecii, is a nonculturable organism, traditional laboratory detection includes direct microscopic examination of respiratory samples stained with calcofluor white, fluorescein-conjugated monoclonal antibodies, Grocott-Gomori methenamine silver, or methanol-Giemsa (5). Various real-time PCR assays that show increased sensitivity over microscopic examination have been developed. These include real-time PCR assays that target the dihydrofolate reductase gene (11), dihydropteroate synthase (9), heat shock protein 70 (8), the cdc2 gene (2, 19), and the mitochondrial ribosomal large-subunit gene (mtLSU) (1, 7, 15). Diagnosis of PCP is typically made on the basis of laboratory results, including microscopy and real-time PCR results, when available, as well as clinical information, including patient symptoms and underlying immune status (5, 6).

Recently, a commercial real-time PCR assay, the MycAssay Pneumocystis kit developed by Myconostica, Ltd. (Manchester, United Kingdom), was evaluated by comparison to microscopic examination of respiratory specimens and clinical diagnosis (7, 15). The results show a high negative predictive value (98% to 99%), indicating the value of the test in ruling out Pneumocystis infection. However, a substantial number of real-time PCR-positive results were unconfirmed by microscopy or clinical diagnosis, resulting in relatively low positive predictive values (59% to 70%) (7, 15). It is not known whether the MycAssay Pneumocystis kit returns false positives or whether these results represent true analytical positives reflecting a low fungal burden of P. jirovecii not detected by microscopic examination. In this study, we evaluated the MycAssay Pneumocystis kit compared to a laboratory standard of direct immunofluorescence microscopy (DFA), real-time PCR, or conventional PCR and sequencing in order to distinguish among these possibilities. This approach may be useful for clinical laboratories attempting to validate sensitive real-time PCR assays in the absence of clinical diagnostic data or an accepted “gold standard.”

MATERIALS AND METHODS

One hundred five bronchoalveolar lavage (BAL) specimens from patients with a clinical suspicion of P. jirovecii pneumonia received by Public Health Ontario in 2010 and 2011 were tested for P. jirovecii by direct immunofluorescence (IFA) using a Monofluo Pneumocystis jirovecii IFA test kit (Bio-Rad Laboratories, Montreal, Quebec, Canada) according to the manufacturer's instructions. The remaining specimen material was frozen at −80°C until further testing at a later date. After thawing at 4°C, DNA was isolated from remaining specimen material using a MycXtra fungal DNA extraction kit (Myconostica, Ltd., United Kingdom). Viscous specimens were liquefied with BD BBL MycoPrep (BD Diagnostics, MD) prior to DNA extraction according to the manufacturer's instructions. DNA was aliquoted into 3 aliquots of 15 μl each and stored at −80°C for further testing.

Real-time PCR for the detection of P. jirovecii was performed using the MycAssay Pneumocystis kit (Myconostica). Briefly, 10 μl of extracted DNA was added to 15 μl of proprietary PCR reagents containing primers and a molecular beacon targeting the mtLSU of P. jirovecii. Also included in the reaction mixture was an internal amplification control DNA fragment accompanied by primers and a molecular beacon for its detection. Amplification (95°C for 10 min, followed by 39 cycles of 95°C for 15 s, 57°C for 35 s, and 72°C for 20 s) and detection were performed using an Applied Biosystems 7500 real-time PCR system. Results were interpreted according to the manufacturer's instructions, which specify thresholds of 10,000 for the Pneumocystis assay and 4,000 for the internal amplification control assay. According to the manufacturer's instructions, samples were considered positive for P. jirovecii if the cycle threshold (C_T) value was <39.0 and negative if the C_T was 39.0 or undetermined (the signal failed to cross the threshold during 39 amplification cycles) with a valid internal amplification control value (C_T = 29.0 to 32.7). Amplification curves were also visually inspected for an exponential amplification pattern expressed on a logarithmic scale.

To confirm the presence of DNA in each sample, the human glyceraldehyde-3-phosphate dehydrogenase (GAPDH) gene was detected by real-time PCR using the TaqMan GAPDH control reagents (human) from Applied Biosystems (Foster City, CA). Briefly, 1 μl of forward primer, 1 μl of reverse primer, and 0.5 μl of JOE probe were combined with 12.5 μl of TaqMan universal PCR master mix (Applied Biosystems) and 5 μl of sample DNA in a total reaction volume of 25 μl. Amplification (50°C for 2 min, 95°C for 10 min, and 40 cycles of 95°C for 15 s and 60°C for 1 min) and detection were performed using the Applied Biosystems 7500 real-time PCR system. Samples were considered positive for DNA if the C_T value was <39.0 with an exponential amplification pattern.

In order to adjudicate discrepancies between IFA and the MycAssay Pneumocystis real-time PCR results, all samples were analyzed by another validated real-time PCR assay targeting the cdc2 gene of P. jirovecii (2, 19) and by conventional PCR and sequencing of the mtLSU. For the conventional PCR, the target was amplified using the pAZ102-E (5′-GATGGCTGTTTCCAAGCCCA-3′) and pAZ102-H (5′-GTGTACGTTGCAAAGTACTC-3′) primers previously described (17) and Phire polymerase (New England BioLabs, Ipswich, MA) according to the manufacturer's instructions. PCR cycling conditions were 98°C for 30 s, followed by 35 cycles of 98°C for 5 s, 53°C for 5 s, and 72°C for 20 s and by a final extension at 72°C for 1 min. While this was sufficient to produce a positive signal in samples with a high concentration of P. jirovecii DNA, an identical PCR performed using a 1-in-100 dilution of the original PCR product as the template was used in order to detect the target in samples with a low concentration of P. jirovecii DNA. PCR products were sequenced using a BigDye (version 1.1) cycle sequencing kit and a 3130xl genetic analyzer (Applied Biosystems).

A true-positive (analytical) MycAssay Pneumocystis test result was defined as a positive result by any single comparator assay, i.e., IFA, cdc2 gene real-time PCR, or mtLSU conventional PCR with sequencing.

A false-positive (analytical) MycAssay Pneumocystis test result was defined as IFA and comparator PCR results all negative.

RESULTS AND DISCUSSION

Of 105 BAL samples, 27 were positive and 78 were negative for P. jirovecii according to the MycAssay Pneumocystis real-time PCR assay. However, 10 of the negative samples also failed to produce a human GAPDH signal, which we included as a control to assess the validity of negative results. It is unknown whether sufficient cellular material was present in these specimens or whether DNA was successfully isolated. Since the internal amplification control C_T value was within the acceptable range (29.0 to 32.7) for these samples, the presence of PCR inhibitors can be ruled out. These samples were excluded from the analysis (Table 1). Incidentally, 4 samples that were positive for Pneumocystis according to the MycAssay Pneumocystis real-time PCR assay and at least one other method (cdc2 real-time PCR, mtLSU conventional PCR, or IFA) were also negative for GAPDH. These 4 results were considered valid and included in the analysis.

Table 1.

Comparison of detection of P. jirovecii in respiratory samples by the MycAssay Pneumocystis real-time PCR assay and a laboratory standard of IFA, cdc2 real-time PCR, or mtLSU conventional PCR and sequencing

MycAssay mtLSU real-time PCR result	No. of specimensb
	IFA			cdc2 real-time PCR			mtLSU conventional PCR			Laboratory gold standard
	Pos.	Neg.	Total	Pos.	Neg.	Total	Pos.	Neg.	Total	Pos.	Neg.	Total
Positive	9	18	27	16	11	27	26	1	27	27	0	27
Negative	0	68a	68	0	68a	68	0	68a	68	0	68a	68
Total	9	86	95	16	79	95	26	69	95	27	68	95

^aTwo samples had a MycAssay Pneumocystis real-time PCR assay C_T of <39.0 but did not have an exponential amplification pattern.

^bPos., positive; Neg., negative.

Of the remaining 68 negative samples, 2 had C_T values of <39.0, thus rendering them positive according to the MycAssay Pneumocysits kit instructions. However, the amplification plots of these 2 samples did not exhibit an exponential amplification pattern (Fig. 1). Therefore, they were deemed negative. The IFA, cdc2 real-time PCR, and mtLSU conventional PCR and sequencing results for these two samples were also negative, thus confirming them to be true negatives. These results highlight the importance of visually inspecting the amplification curves, in addition to evaluating the C_T values, rather than blindly following the manufacturer's recommendations for defining a positive result in the MycAssay Pneumocystis real-time PCR assay.

Fig 1.

Fluorescence emission versus cycle number from results of the MycAssay Pneumocystis real-time PCR assay demonstrating examples of samples with an exponential amplification pattern with C_T values of <39.0 interpreted as positive (a), nonexponential amplification with C_T values of <39.0 interpreted as negative (b), and a nonexponential amplification pattern with undetermined C_T values interpreted as negative (c). An arrowed line indicates the threshold of 10,000 specified by the MycAssay Pneumocystis kit. Rn is the ratio of the emission intensity of the reporter dye divided by the emission intensity of ROX at each cycle. Delta Rn is the Rn value baseline subtracted on a per-well basis.

Compared with IFA alone, 18 of the samples that were positive by MycAssay Pneumocystis real-time PCR were negative by IFA (Table 1), yielding a sensitivity of 100% and a specificity of 79%. Previous studies comparing real-time PCR assays to microscopy for the detection of P. jirovecii in respiratory specimens have also noted that substantial numbers of samples were positive by real-time PCR but negative by microscopy (2, 7, 9, 13, 15, 19). Together with these studies, our results suggest that real-time PCR is more sensitive than microscopy for detecting P. jirovecii in BAL specimens. To prove this, we compared the performance of our assay to that of a laboratory standard since clinical data were not available to us. According to our laboratory gold standard, a specimen was considered positive for Pneumocystis if it was positive by at least one of the three alternate assays: IFA, cdc2 real-time PCR, or mtLSU conventional PCR and sequencing. The laboratory standard confirmed that the 18 samples positive by MycAssay Pneumocystis real-time PCR but negative by IFA were indeed true positives (Table 1), conferring the MycAssay Pneumocystis kit with an analytical sensitivity of 100%, an analytical specificity of 100%, a positive predictive value of 100%, and a negative predictive value of 100% for the detection of P. jirovecii in BAL specimens.

Comparison with the laboratory standard also demonstrated that the MycAssay Pneumocystis real-time assay was more sensitive than either the cdc2 real-time PCR assay or the mtLSU conventional PCR and sequencing assay (Table 1). The sensitivity and specificity were 100% and 86%, respectively, for the cdc2 real-time PCR assay and 100% and 98.5%, respectively, for the mtLSU conventional PCR and sequencing assay. cdc2 is a single-copy gene, whereas mtLSU, the target of the MycAssay Pneumocystis real-time assay, exists as multiple copies per cell (9, 13), thus explaining the better correlation of the MycAssay Pneumocystis kit with the laboratory standard. Similar to other reports comparing conventional PCR and real-time PCR (3), the sensitivity of mtLSU conventional PCR and sequencing in this study was nearly as high as that of the MycAssay Pneumocystis real-time assay. The single discrepant result may be attributed to different polymerases and reaction mixtures.

Both positive and negative MycAssay Pneumocystis results should be interpreted in the context of clinical symptoms. Previous studies have suggested that in some individuals P. jirovecii may colonize lung tissue without causing infection. Consequently, positive results, especially those with high C_T values, must be interpreted in the context of patient symptoms in order to be able to distinguish colonization from infection (1, 5, 7, 8, 15, 19). Likewise, negative results could be the result of insufficient sampling, resulting in a less than detectable number of P. jirovecii cells in the BAL specimen. However, the GAPDH assay indirectly attempts to control for this. The GAPDH assay is most useful in interpreting a negative result using the MycAssay Pneumocystis kit, since we showed that true-positive results occurred even in the presence of a negative GAPDH result. The assay will require further validation using clinical criteria, in addition to a laboratory standard, to determine its clinical utility in predicting PCP in patients. Quantitative PCR may also shed light on the clinical significance of PCR testing (1, 8).

In conclusion, using a laboratory gold standard of IFA microscopy, cdc2 real-time PCR, or conventional PCR and sequencing of mtLSU, the MycAssay Pneumocystis kit was shown to have 100% sensitivity, specificity, and positive and negative predictive values. The very high negative predictive value is especially useful for clinicians, in that one can be reasonably certain that P. jirovecii is not present if the MycAssay PCR test is negative, thus eliminating the need for potentially toxic empirical treatment. This study also demonstrates that comparison to a laboratory gold standard represents a useful method for the laboratory validation of new, highly sensitive real-time PCR assays because it helps to distinguish between the enhanced sensitivity and reduced specificity of the new method compared to an existing laboratory method.