Evaluation of a Novel Multiplex PCR Panel Compared to Quantitative Bacterial Culture for Diagnosis of Lower Respiratory Tract Infections

Quantitative bacterial culture of bronchoalveolar lavage fluids (BALF) is labor-intensive, and the delay involved in performing culture, definitive identification, and susceptibility testing often results in prolonged use of broad-spectrum antibiotics. The Unyvero lower respiratory tract (LRT) panel (Curetis, Holzgerlingen, Germany) allows the multiplexed rapid detection and identification of 20 potential etiologic agents of pneumonia within 5 h of collection. In addition, the assay includes detection of gene sequences that confer antimicrobial resistance.

ABSTRACT

Quantitative bacterial culture of bronchoalveolar lavage fluids (BALF) is labor-intensive, and the delay involved in performing culture, definitive identification, and susceptibility testing often results in prolonged use of broad-spectrum antibiotics. The Unyvero lower respiratory tract (LRT) panel (Curetis, Holzgerlingen, Germany) allows the multiplexed rapid detection and identification of 20 potential etiologic agents of pneumonia within 5 h of collection. In addition, the assay includes detection of gene sequences that confer antimicrobial resistance. We retrospectively compared the performance of the molecular panel to routine quantitative bacterial culture methods on remnant BALF. Upon testing 175 BALF, we were able to analyze positive agreement of 181 targets from 129 samples, and 46 samples were negative. The positive percent agreement (PPA) among the microbial targets was 96.5%, and the negative percent agreement (NPA) was 99.6%. The targets with a PPA of <100% were Staphylococcus aureus (34/37 [91.9%]), Streptococcus pneumoniae (10/11 [90.9%]), and Enterobacter cloacae complex (2/4 [50%]). For the analyzable resistance targets, concordance with phenotypic susceptibility testing was 79% (14/18). This study found the Unyvero LRT panel largely concordant with culture results; however, no outcome or clinical impact studies were performed.

INTRODUCTION

Lower respiratory tract infections (LRTI) present with a wide variety of symptoms, severity, and causative agents. Therefore, a wide range of diagnostic methods is needed to detect and differentiate bacterial, viral, and fungal causative agents of LRTI, including quantitative bacterial culture, direct staining, and molecular methods. Cases of pneumonia can be partitioned into community-acquired pneumonia (CAP), hospital-acquired pneumonia (HAP), ventilator-associated pneumonia (VAP), and health care-associated pneumonia (HCAP) cases. Diagnostic and treatment guidelines for CAP have been well characterized, perhaps due to the comparative simplicity of disease presentation and the improved clinical outcomes observed (1). More complex, however, are recommendations for cases in which the patient is hospitalized or has long-term contact with the health care setting. It has been hypothesized that prolonged health care exposure can lead to colonization and microaspiration of multidrug-resistant organisms, presenting the opportunity for more severe infections with limited therapeutic options (2).

Multiplexed syndromic panels, such as the one evaluated in this study, can rapidly provide diagnostic details that help inform best management practices in complex cases of pneumonia. This study aimed to evaluate the performance of the Unyvero LRT panel (Curetis; Holzgerlingen, Germany), compared to quantitative bacterial culture performed as a part of routine diagnostics in the clinical laboratory.

MATERIALS AND METHODS

Study design.

Bronchoalveolar lavage fluid (BALF) specimens were submitted to the University of North Carolina Hospitals’ clinical microbiology laboratory for routine bacterial testing to include Gram staining and quantitative bacterial culture with antimicrobial susceptibility testing (AST) based on standard laboratory procedures (below). Specimens from cystic fibrosis patients and patients ≤16 years old were excluded. Specimens (n = 175) from 98 patients included 67 patients with one sample each and 31 patients with two to eight specimens each. Multiple aliquots of remnant BALFs were frozen at −70°C within 24 h of collection. Specimens were thawed prior to testing with the Unyvero instrument and were tested an average of 27 days after collection (median, 9 days; range, 1 to 188 days).

Following guideline recommendations, when pathogens present in culture at quantities less than 10,000 CFU/ml were not detected by PCR, results were considered true negative (3, 4). Medical records were reviewed to investigate potential sources of bacterial DNA in cases where a pathogen was detected by PCR and not isolated by culture. If a patient’s sample had grown the detected organism within 40 days of collection of the discordant sample, results were considered true positive. If culture results included mixed growth or overgrowth of an organism (defined as ≥400,000 CFU/ml), any detections that were not found in culture were not considered false positive, and instead targets were given the label “unable to analyze concordance.”

Culture methods.

BALF specimens were collected as part of routine clinical care and transported to the laboratory in sterile screw-cap containers. Gram stains were made using 100 μl of sample. All specimens were plated directly to buffered charcoal yeast extract agar and diluted 1:100, and 100 μl was plated to chocolate, colistin-nalidixic acid, and MacConkey agars (BD Diagnostics, Sparks, MD). Plates were examined at 24, 48, and 96 h of incubation. A mixture of two or more members of the oropharyngeal microbiota with no predominating pathogens was given the result “oropharyngeal flora present” (OPF). In a case where there was a predominating organism, it was identified via matrix-assisted laser desorption ionization-time of flight mass spectrometry (MALDI-TOF MS) (Vitek MS; bioMérieux, Durham, NC) and reported quantitatively, along with antimicrobial susceptibility results obtained by disk diffusion using Clinical and Laboratory Standards Institute breakpoints (5). If fewer than 10,000 organisms per ml were present, the report read “<10,000 CFU/ml.” In cases where three or more organisms were equally present, the laboratory reported “mixed organisms present” along with a summary of the Gram stain characteristics (e.g., “mixed Gram-positive organisms present”). The predominant pathogen from a culture with three or more organisms was identified by MS and underwent phenotypic AST; the remaining organisms were reported as mixed.

Unyvero testing.

The investigational-use-only (IUO) version of the Unyvero LRT panel was used according to the manufacturer’s instructions. Organism and resistance gene targets of the IUO LRT panel are shown in Table 1, along with the FDA-cleared targets. The in vitro diagnostic version of the LRT panel is FDA cleared for tracheal aspirates and bronchoalveolar lavage samples (BALF and mini-BALF). BALF specimens were kept at −70°C and thawed at room temperature immediately prior to testing. The provided sample tubes were inoculated with 180 μl of sample and loaded onto the instrument for a 30-minute lysis before being inserted into the provided cartridge for a 4.5-h PCR and detection step. The stated limit of detection for this assay is 10,000 CFU/ml.

TABLE 1.

Organisms and resistance markers included on the IUO LRT panel^a

Organism or resistance marker	Target(s)
Organism type
Gram-positive bacteria	Staphylococcus aureus*, Streptococcus pneumoniae*
Enterobacterales	Citrobacter freundii*, Escherichia coli*, Enterobacter cloacae complex*, Proteus spp.*, Klebsiella pneumoniae*, Klebsiella oxytoca*, Klebsiella variicola*, Serratia marcescens*, Morganella morganii*
Nonfermenters	Acinetobacter baumannii complex*, Stenotrophomonas maltophilia*, Legionella pneumophila*, Pseudomonas aeruginosa*
Others	Pneumocystis jirovecii*,b Haemophilus influenzae*, Mycoplasma pneumoniae*, Chlamydia pneumoniae*, Moraxella catarrhalis*
Resistance type
Macrolide/lincosamide	ermB
Oxacillin	mecA*, mecC (LGA251)
Penicillin	blaTEM*, blaSHV
Third-generation cephalosporin	blaCTX-M*
Carbapenem	blaKPC*, blaIMP, blaNDM*, blaOXA-23*, blaOXA-24/40*, blaOXA-48*, blaOXA-58*, blaVIM*
Sulfonamide	sul-1
Fluoroquinolone (E. coli and P. aeruginosa only)	gyrA83, gyrA87

^a*, target included on the FDA-cleared LRT panel.

^bBALF only.

Discordant analysis.

Discrepant microorganism results between quantitative culture and the LRT panel were analyzed by analyte-specific singleplex PCRs followed by bidirectional sequencing. DNA was extracted from a unique frozen BALF aliquot using the QIAamp DNA blood minikit (Qiagen, Germantown, MD). Analyte-specific PCRs with two different proprietary primer pairs (Unyvero primers and a second primer set) for each observed discrepant analyte were performed on extracted specimen DNA by Curetis. In general, the 23S rRNA gene was targeted for most pathogens; other targets included rpoB for Klebsiella pneumoniae and Enterobacter cloacae, psaA and lyt for Streptococcus pneumoniae and copB for Moraxella catarrhalis. Amplified products were bidirectionally sequenced at an external service provider (Microsynth, Balgach, Switzerland). Identity of aligned consensus sequences was confirmed by BLAST analysis. Any discrepant analyte that could not be amplified or could not be confirmed by sequencing was considered not present in the analyzed specimen.

This study was approved by the University of North Carolina at Chapel Hill Institutional Review Board.

RESULTS

Among the 98 patients tested, the mean and median ages were 55 and 58, respectively, with 45% being ≥60 years old. Samples were obtained from patients in a variety of clinical settings: 33% from the medical or surgical intensive care unit (ICU), 31% from the burn ICU, and 25% from other inpatient or ICU settings; only 11% were outpatients. Samples were partitioned into three groups based on the complexity of the results (low, medium, and high) for the purpose of discussion.

Among the 175 samples tested, culture results (n = 61) included either no growth, OPF not further identified, or growth of a pathogen that was not on the LRT panel (Candida spp., n = 7; Burkholderia spp., n = 5; three each of Corynebacterium spp. and Mycobacterium spp.; two each of Aspergillus spp., coagulase-negative Staphylococcus, Enterococcus spp., Klebsiella aerogenes, Neisseria meningitidis, Nocardia spp.; one each of Achromobacter sp., Providencia stuartii, Raoultella ornithinolytica, Streptococcus dysgalactiae, and Streptococcus mitis). For the 61 low-complexity-group specimens listed above, Unyvero was negative (n = 45), detected a pathogen that had been isolated from the patient in the previous 40 days (n = 11), or detected a pathogen considered to be a false positive (n = 5). Of the 11 samples where a pathogen had been detected in the previous 40 days, in which 12 distinct organisms were detected by Unyvero, all patients had been start on antibiotic therapy >12 h prior to collection. The remaining samples (n = 5) in this group had six false-positive detections. Discrepant analysis by sequencing revealed that all but one detection of S. aureus were true positives. Results for this low-complexity group of samples before and after discrepant analysis are shown in Table S1 in the supplemental material.

A group of medium-complexity samples (n = 73) were positive for one or more pathogens on the LRT panel. These samples (n = 73) had 102 true positive detections and two false-positive detections. Sequencing confirmed the presence of both organisms in the initial false-positive samples. In three samples, Unyvero missed detections. Sequencing confirmed the presence of the organisms for two of these false negatives; further investigation was not performed for one S. aureus false negative, since the culture was positive. Table S2 displays the results for this group of medium-complexity samples.

The final group of samples, the high-complexity group (n = 41), grew a pathogen that is present on the LRT panel in addition to OPF or a nonpanel pathogen. Unyvero detected LRT panel targets considered true positives (n = 53) and targets considered false positives (n = 6) in 38 samples. Of the six false positives, three detections were considered true positives based on sequencing results. The sample positive for P. jirovecii was negative by discrepant analysis, and for the two other false-positive detections (Haemophilus influenzae and Escherichia coli), discrepant analysis was not performed. Of the 41 samples in this group, the remaining 3 had three false-negative detections, and sequencing confirmed the presence of the organisms in all three samples. Results for the high-complexity group of samples are presented in Table S3.

Tables 2 and 3 present the cumulative pre- and post-discrepant analysis data for each target on the LRT panel, respectively. Positive percent agreements (PPA) were 96.5% prior to discrepant analysis and 97.0% after discrepant analysis. Negative percent agreements (NPA) pre- and post-discrepant analysis were 99.6% and 99.9%, respectively.

TABLE 2.

Cumulative agreement pre-discrepant analysis for all analyzed samples^a

Organism	No. of samples					% positive agreement	% negative agreement
	TP	FP	TN	FN	NA
S. aureus	34	2	135	3	1	91.9	98.5
S. pneumoniae	10	1	162	1	1	90.9	99.4
C. freundii	3	0	172	0	0	100	100
E. coli	2	3	165	0	5	100	98.2
E. cloacae complex	2	0	169	2	2	50.0	100
Proteus spp.	1	3	169	0	2	100	98.3
K. pneumoniae	10	0	164	0	1	100	100
K. oxytoca	0	0	175	0	0	NA	100
K. variicola	1	0	174	0	0	100	100
S. marcescens	10	0	165	0	0	100	100
M. morganii	0	0	175	0	0	NA	100
M. catarrhalis	6	1	164	0	4	100	99.4
P. aeruginosa	33	0	140	0	2	100	100
A. baumannii complex	13	1	161	0	0	100	99.4
S. maltophilia	26	0	147	0	2	100	100
L. pneumophila	2	0	173	0	0	100	100
P. jirovecii	1	1	173	0	0	100	99.4
H. influenzae	13	2	158	0	2	100	98.8
M. pneumoniae	0	0	175	0	0	NA	100
C. pneumoniae	0	0	175	0	0	NA	100
Total	167	14	3,291	6	22	96.5	99.6

^an = 175. TP, true positive; FP, false positive; TN, true negative; FN, false negative; NA, not analyzable.

TABLE 3.

Cumulative agreement post-discrepant analysis for all analyzed samples^a

Organism	No. of samples					% positive agreement	% negative agreement
	TP	FP	TN	FN	NA
S. aureus	36	1	135	3	0	92.3	99.3
S. pneumoniae	12	0	162	1	0	92.3	100
C. freundii	3	0	172	0	0	100	100
E. coli	6	1	165	0	3	100	99.4
E. cloacae complex	4	0	169	2	0	66.7	100
Proteus spp.	5	0	169	0	1	100	100
K. pneumoniae	11	0	164	0	0	100	100
K. oxytoca	0	0	175	0	0	NA	100
K. variicola	1	0	174	0	0	100	100
S. marcescens	10	0	165	0	0	100	100
M. morganii	0	0	175	0	0	NA	100
M. catarrhalis	10	0	164	0	1	100	100
P. aeruginosa	35	0	140	0	0	100	100
A. baumannii complex	14	0	161	0	0	100	100
S. maltophilia	26	0	147	0	2	100	100
L. pneumophila	2	0	173	0	0	100	100
P. jirovecii	1	1	173	0	0	100	99.4
H. influenzae	15	1	158	0	1	100	99.4
M. pneumoniae	0	0	175	0	0	NA	100
C. pneumoniae	0	0	175	0	0	NA	100
Total	191	4	3,291	6	8	97.0	99.9

^an = 175. TP, true positive; FP, false positive; TN, true negative; FN, false negative; NA, not analyzable.

Resistance genes (n = 89) were detected in 52 of the 175 samples tested. Of these, 54 genes in 42 samples were attributable to a particular organism. The system cannot differentiate which organism has the resistance gene if there is more than one pathogen present. Resistance genes were attributable to a particular organism when one or more resistance genes were detected in the presence of only one organism detection. Only 18 genes belonged to isolates that underwent phenotypic AST. Results are shown in Table 4. When samples were able to be analyzed, detections were largely concordant. The only discrepancies involved detections (n = 4) of a fluoroquinolone resistance marker (gyrA) for Pseudomonas aeruginosa where culture determined the organism to be ciprofloxacin susceptible. Detection of gyrA was not further evaluated, as this target is not FDA cleared. Concordance for resistance genes able to be analyzed was 77.8% (14/18). In samples (n = 63) where coinfections were detected either in culture or by Unyvero, multiple resistance genes were detected (n = 17), a single resistance gene was detected (n = 13), or no resistance genes were detected (n = 33). The most common resistance gene detected in the coinfection group was bla_TEM (n = 16).

TABLE 4.

Resistance genes attributable to an organism with phenotypic AST results^a

Organism	Unyvero resistance gene detected	Phenotypic AST result(s) (n)
S. aureus	mecA	Oxacillin resistant (6), ND (7)
E. coli	gyrAA83/gyrA87	ND (10)
	blaTEM	Ampicillin resistant (1), ND (4)
	sul-1	ND (2)
K. pneumoniae	blaTEM and blaSHV	Ampicillin resistant (2)
	blaCTX-M	ND (1)
	sul-1	ND (1)
K. variicola	blaSHV	Ampicillin resistant (1)
P. aeruginosa	gyrA83/gyrA87	Ciprofloxacin susceptible (4), ciprofloxacin intermediate (1), levofloxacin resistant (2), ND (4)
A. baumannii complex	blaTEM	ND (1)
Proteus sp.	blaTEM	ND (1)
	blaSHV	ND (1)
H. influenzae	blaTEM	ND (4), ampicillin resistant (1)

^aND, not done; AST, antimicrobial susceptibility testing.

DISCUSSION

Our study compared results from quantitative bacterial culture to rapid results from the Unyvero LRT molecular panel. Our evaluation considered historical microbiologic data and administration of antimicrobial therapy for samples for which PCR and culture results were discrepant. Culture and PCR results were analyzed for 175 samples from 98 patients; 31 patients had >1 sample tested, which may have introduced bias into the study cohort. Prior to sequencing analysis, the overall PPA was 96.5% and NPA was 99.6%. If a discrepancy was present, sequencing analysis allowed us to more accurately assess the performance of the panel in terms of detecting microbial DNA when the culture was negative. Initial comparison of culture and PCR results revealed 14 false-positive detections and 6 false negatives for bacterial targets. However, after sequencing, only four targets were considered to be false-positive detections, meaning that the presence of organism DNA could not be confirmed. A caveat in this analysis is that detection of microbial DNA, while confirming the analytic specificity of the LRT panel, does not equate to a positive culture result. This may reflect persistent detection of remnant nucleic acid after appropriate therapy.

While using convenience sampling for selecting our specimens bolstered our ability to analyze a wide variety of targets, the nature of the culture reporting methods limited our ability to analyze concordance for some targets in the samples analyzed. Although discrepant analysis is a flawed approach for evaluating diagnostic devices, its use allowed us to evaluate targets that were not able to be considered true or false positive for a variety of reasons. Nine culture results did not specify the organisms present, calling them mixed Gram-positive/Gram-negative organisms per routine laboratory protocol. However, if Unyvero detected three organisms, and the patient had a recent history of two of them, those two were considered true positives and the third was unable to be analyzed. If sequencing later confirmed the presence of the third, results in Table 3 reflect the assumption of a true positive. During testing, seven samples were completely or partially invalid and required retesting due to instrument error. Additionally, resistance data were challenging to interpret, as not all genes could be attributed to a particular organism and phenotypic AST was not part of routine workup for many isolates with resistance markers detected by LRT. Further investigation into resistance gene data could include a mediator resistance gene PCR. Finally, as shown in Tables 2 to 4, we were not able to find samples positive for all organisms and resistance markers on the IUO LRT panel.

The accuracy of pneumonia diagnostic testing, like all laboratory testing, relies on preanalytical factors. Despite the advancement of molecular technologies, the ability to obtain a sample that reflects the microbes present at the site of infection remains a barrier to accurate diagnostics (6). Further, interpretation of results has become more complex with the introduction of nucleic acid-based testing. A recent survey revealed that a majority of infectious disease physicians believe that novel diagnostics are becoming too complex for non-infectious disease physicians, with 79% agreeing that there should be stewardship in place for expensive or complicated testing (7). However, with appropriate stewardship and interpretation, results from molecular multiplex panels like LRT can provide valuable clinical information in a very critical time period (<5 h from specimen collection). A shorter time to result may reduce time on inappropriate microbial therapy and, therefore, improve patient outcomes. Further studies assessing outcomes and impact on clinical management are needed.

A challenge common to all nucleic acid amplification techniques is that detection does not imply causation. Useful to discerning true infection is a quantitative component that is common to many bacterial culture methods (6). It has been noted in other evaluations of this assay that the detection of a pathogen not isolated in culture could represent colonization and not infection (8). Further, viable organisms cannot be differentiated from nonviable ones, for example, those that have been successfully treated with antimicrobials (9). Due to the high analytical sensitivity, some pathogens can be found incidentally among control subjects. In our study, we observed very high correlation between the LRT panel and quantitative bacterial culture of BALF. For many molecular tests, improved sensitivity is countered by a decrease in clinical specificity, but this does not appear to be true for the LRT panel.

This is the first study to describe the performance characteristics of the Unyvero LRT assay. Previous studies described the performance of the precursor versions P50 and P55. Although the LRT and P50/P55 are based on the same biochemical principle, they cannot directly be compared to each other due to reporting differences, such as unmasked reporting of resistance genes independent of detection an associated host organism in P50/P55. Papan et al. reported an overall sensitivity of 73.1% and a specificity of 97.9% for P50 compared to culture of LRT samples from pediatric or neonate patients. Regarding detection of resistance genes, 75% concordance with an antibiogram was observed where a culture counterpart was available (10). Gadsby et al. found an overall sensitivity/specificity of 56.9/63.2% and a sensitivity/specificity for antibiotic resistance detection of 18.8/94.9% for P55 (11). Ozongwu et al. reported that P55 detected more antimicrobial resistance markers than routine culture and assumed that some instances of phenotypic resistance were missed (12). Microbiology is still challenged by a lack of full understanding of the genetic basis of antimicrobial resistance. While phenotypic AST is susceptible to inconsistencies and errors, it has been argued that in order to accurately develop and challenge new molecular resistance testing, we must maintain current methods while striving for consistency in newer technology (13).

The same principle can be applied to bacterial targets. Not every pathogen that can cause pneumonia can be covered by syndromic panels, which means that culture is still needed. Therefore, a careful cost-benefit analysis should be performed by institutions looking to implement a molecular pneumonia panel to determine how best to use the technology with their specific patient population. Jamal et al. evaluated the utility of P50 when real-time treatment changes are implemented (14). Patients were partitioned according to risk factors and onset severity. The most common pathogen in the cohort presenting a milder illness was Streptococcus pneumoniae (75% of patients). In the two cohorts presenting more severe illness, the most common pathogen was Acinetobacter baumannii (29%). Though this was a single institution with a limited sample size of 49 patients, they found that in 67.3% of cases, empirical treatment was changed as a consequence of the molecular diagnostic results. When treatment was adjusted, 62.2% of patients improved clinically while 16.3% showed no improvement (14). Successful implementation of rapid molecular diagnostics for detection of pneumonia-causing pathogens will likely involve an antimicrobial stewardship program that limits testing to severely ill patients for whom empirical treatment is not sufficient (9). Considerations may include implementing protocols to limit testing to patient populations at greatest risk for infection with antimicrobial-resistant organisms, patients on antibiotics, and patients for whom antimicrobial therapy can be narrowed based on test results (15–17).

While the analytical performance of the Unyvero LRT assay is comparable to that of traditional methods, the clinical utility of a qualitative molecular assay compared to quantitative culture deserves further investigation, as well as an evaluation of the impact to the patient outcomes when rapid results are acted upon. The LRT assay is able to rule out atypical or fastidious pathogens more quickly than culture. However, institutions implementing this assay must evaluate the negative predictive value of a result from this panel and how these results fit into current test availability. Equally, a positive result may not always indicate active infection, and the positive predictive value must be interpreted along with the clinical context. As observed in this study, positive PCR results from the LRT assay were largely concordant with culture results, suggesting that a rapid result could lead to appropriate or targeted antimicrobial therapy more quickly than quantitative culture. The most valuable utility of this assay may be for patients with HCAP/HAP/VAP or those who are critically ill or immunocompromised or have a disease that has been refractory to treatment.