Evaluation of commercial ResPlex II v2.0, MultiCode^®-PLx, and xTAG^® respiratory viral panels for the diagnosis of respiratory viral infections in adults

Abstract

Background

Commercial multiplex PCR panels for respiratory viruses (PRV) have been recently developed. ResPlex II Panel v2.0 (Qiagen), MultiCode^®-PLx (EraGen Biosciences), and xTAG^® (Luminex) PRV's were studied. All assays detect influenza A and B, adenovirus, parainfluenza 1–3, respiratory syncytial virus A and B, human metapneumovirus and human rhinovirus. The ResPlex II additionally detects coronavirus (229E, OC43, NL63, HKU1), coxsackie/echo virus, bocavirus and differentiates adenoviruses (B, E). The MultiCode-PLX assay detects 229E, OC43, and NL63, differentiates parainfluenza 4a, 4b and adenoviruses (B, C, E). The xTAG additionally subtypes influenza A as seasonal H1 and H3.

Study design

202 specimens collected from adult patients with signs of respiratory infection from November, 2008 to May, 2009 were used for evaluating the performance of the three commercial PRV assays. Viral culture and xTAG were used as the standards to assess sensitivity and specificity.

Results

The PRV assays detected more viruses than culture. When compared to culture, the xTAG PRV showed a sensitivity and specificity of 100% and 91%, compared to MultiCode-PLx with 89% and 87%, and ResPlex II with 89% and 94%, respectively. Co-infection was detected in a small subset of patient specimens. Each panel showed differences in sensitivities for individual viruses.

Conclusions

While the ResPlex II and MultiCode-PLx offer a broader virus detection range and greater ease of use, the xTAG PRV showed increased sensitivity to common viral targets represented in the assays, and also had the ability to differentiate human from non-human influenza A H1.

1. Background

Respiratory viral infections account for many hospitalizations and deaths in the US.¹ In addition to classical respiratory viruses, newly identified viruses, such as human metapneumovirus (HMPV), coronaviruses NL63 and HKU1, bocavirus and, recently, 2009 H1N1 influenza virus, can cause significant morbidity and mortality.2, 3 Rapid and accurate diagnosis of respiratory viral pathogens aids in antiviral therapy, and early diagnosis has the potential to reduce complications, antibiotic utilization and unnecessary laboratory testing.4, 5

The development of sensitive molecular assays has increased the detection rate of respiratory viral pathogens.6, 7, 8, 9 PCR and nucleic acid sequencing have shown greater sensitivity than Direct Fluorescent Antibody (DFA) and culture tests.¹⁰ Multiplex PCR assays can detect multiple respiratory viruses.6, 11, 12, 13, 14, 15 The xTAG^® is one of the first multiplex PCR assays to be cleared by the U.S. Food and Drug Administration (FDA) for the detection of 12 different respiratory viruses in nasopharyngeal specimens. Since then, additional commercial panels for respiratory viruses (PRV) assays have emerged. While comparative studies have shown that xTAG consistently performs better than other non-molecular assays,¹⁶ a comparison of xTAG has not been published relative to other multiplex PRV assays.

2. Objective

The goal of this study was to evaluate the performance of the FDA-cleared xTAG, the research use only ResPlex II and the Multicode-PLx for the detection of multiple viruses and subtypes in various clinical specimen types and to compare performance with viral culture and xTAG.

3. Study design

3.1. Preparation of specimens

202 specimens from adult patients who presented with signs and symptoms of respiratory infection from November, 2008 to May, 2009 were included. The samples consisted of 104 bronchoalveolar lavages (BAL), 48 nares, 15 throat and 14 nasopharyngeal swabs, 10 lung biopsies, 7 pleural fluids, and 4 sputa. Flocked swab specimens were submitted in viral transport medium (VTM; Becton-Dickinson, MD). Liquid specimens were filtered and diluted 1:1 with VTM prior to testing. Lung biopsies were homogenized, spun down and the supernatant diluted 1:1 with VTM.

3.2. Viral culture

A 0.2 ml of the specimen was inoculated into each of the R-mix Shell Vials and tissue culture cells (A549, Primary Monkey Kidney and Human Fibroblast; Diagnostic Hybrids, Athens, OH). The R-mix Shell Vials were centrifuged at 2000 rpm for 1 h. Inoculated cells were incubated at 35–37 °C. R-mix Shell Vials were stained with individual monoclonal antibodies for influenza (Flu) A and B; parainfluenza (Para) 1, 2, 3; Adenovirus and Respiratory Syncytial Virus (RSV) (Diagnostic Hybrids, Athens (Adeno), OH) at 48 h. Culture cells were observed for cytopathic changes up to 14 days.

3.3. Multiplex respiratory viral panel testing

Internal controls from each of the PRV assays were added to 400 μl of individual specimens prior to extraction. Extraction was performed using the Qiagen EZ1 Virus Minikit 2.0 and nucleic acid was eluted in 60 μl. Amplification/detection was performed as per manufacturer's instructions (Eragen,¹⁷ Qiagen¹⁸ and the Luminex¹⁹). The detection of the viruses for all PRV assays was based on xMAP technology²⁰ using a Luminex instrument. The PRV assays differed in the analysis steps and chemistry of detection17, 18, 19 (Supplementary Table 1). The MultiCode-PLx integrates multiplexed PCR with isoguanine/5-methyl-isocytidine and one step enzyme for site specific enzymatic labeling.¹⁷ The ResPlex II utilizes suspension arrays that use Qiagen's proprietary one step QIAplex PCR with superprimers on the Qiagen LiquiChip^® System.¹⁸ The xTAG employs multiple enzymes, including the Luminex suspension microarray platform.¹⁹ Strict adherence to good lab technique is required.

The following cut-off values were used: for xTAG, positive (mean fluorescence intensity or MFI > 300, based on manufacturer's package insert).¹⁹ For the ResPlex II and for the MultiCode-PLx, results were considered positive if an average signal of greater than 6 standard deviations above the negative control signals (typically MFI > 169 for ResPlex II and > 500 MFI for MultiCode-PLx⁷), respectively.

3.4. Data analysis

The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated in comparison to culture for the three PRV assays and to xTAG for the ResPlex II and MultiCode-PLx. The agreement between methods was assessed by the kappa value test. Kappa values from 0.41 to 0.8 and 0.81 to 1 indicate moderate and almost perfect agreement, respectively.²¹

4. Results

4.1. Comparison of culture with PRV assays

One hundred and twelve specimens (n  = 112/202; 55%) were negative by both culture and PRV's. The specimen with the highest positivity rate was nares (Supplementary Table 2). The xTAG and MultiCode-PLx were positive for 37% of the specimens and the ResPlex II for 33%. Single infections with FluA, FluB, Para3 and RSVB were the most prevalent. In general, the PRV assays detected a higher number of viruses compared to culture based on the larger repertoire of viral targets detected. The exception was Para3 where culture was more sensitive than ResPlex II and the MultiCode-PLx, although this was not the case for the xTAG where the detection of Para3 (n  = 11) was comparable to culture (Table 1 ). The performance of PRVs relative to culture was comparable for cultivatable viruses (Table 2 ).

Table 1.

Distribution of viruses detected by culture and respiratory virus panel assays.

	Specimens tested
Positive	C	R	M	X
Adeno	1	1	2	1
FluA	8	12	10	13
FluB	9	10	12	11
HMPV		2	2	4
Para3	11	8	9	11
Para4b		0	2	0e
HRV/Enterovirus		5	13b	16c
CVEV		7
RSVA	13a	3	3	7
RSVB		9	9	9
OC43		2	2	0e
229E		2	1	1e
Multipled		5	9	2
Total	42	54	74	75
% Positive	20.2	26.7	36.6	37.1
Negative	160	148	128	128

Culture (C), ResPlex II (R), MultiCode-PLx (M), xTAG (X); BAL (Bronchoalveolar lavage (BAL), other sites (lung, pleural fluid, sputum).

^aCulture does not differentiate RSVA from RSVB.

^bDetects HRV.

^cDoes not differentiate HRV/Enterovirus.

^dNot included as single infection.

^eNot available in the IVD version.

Table 2.

Correlation of ResPlex II, MultiCode-PLx and xTAG PRV assays with culture.

Culture/PRVa	ResPlex II	MultiCode®-PLx	xTAG
Positive/positive	37	37	42
Positive/negative	5	5	0
Negative/positive	9	18	13
Negative/negative	131	122	127
Sensitivity (%)	89	89	100
Specificity (%)	94	87	91
PPV (%)	80	67	76
NPV (%)	96	96	100
Kappa	0.79	0.67	0.82

Positive predictive value (PPV), negative predictive value (NPV).

^aOnly includes viruses that are culturable and viral targets detected by all methods.

The xTAG showed the highest sensitivity (100%) and NPV (100%; Table 2). The ResPlex II showed the highest specificity (94%) and PPV (80%). Kappa variable showed the highest value (0.82) for the xTAG, followed by the ResPlex II (0.79) and the MultiCode-PLx (0.67), confirming an almost perfect agreement of the xTAG compared to culture (Table 2).

4.2. Detection of mixed infections by the PRV assays

While single viruses were most prevalent, we detected a low rate of multiple viral infections (1–4.5% depending on the assay). The MultiCode-PLx detected 8 dual (Para2/3; Para2/HRV; FluA/HMPV; Para3/Adeno, RSVB/HRV, 3 Para3/Para4b) and 1 triple (RSVA/Para2/Para3) infection. The xTAG detected 2 dual (RSVB/HRV, FluA/HMPV), and the ResPlex II detected 6 dual (Para1/Para3, Para3/OC43, RSVB/HRV, FluA/HMPV, 2 CVEV/HRV) infections. Most of the multiple infections included a parainfluenza virus. Interestingly, Para1 and Para2 were detected as part of multiple infections only.

4.3. Comparison of the xTAG with ResPlex II and MultiCode-PLx

The three PRV assays performed similarly for the detection of Adeno, FluB, and RSVB. While the ResPlex II and the MultiCode-PLx detected similar number of HMPV (n  = 2), Para3 (n  = 8 and 9, respectively), and RSVA (n  = 3), the xTAG was the only assay that detected all Para3 (n  = 11), RSV (n  = 16), and FluA (n  = 13) comparable or better than culture (Table 1). Of the 13 specimens that were FluA positive, 11 were seasonal H1 and two were neither H1 nor H3, later confirmed to be the 2009 H1N1 (data not shown). The greatest variability was observed with the detection of HRV/Enterovirus. The xTAG detected greater number of HRV/Enterovirus (n  = 16), while the ResPlex II detected 12 and the MultiCode-PLx detected 13 (Table 1). The ResPlex II differentiated these as 5 HRV and 7 CVEV. In four specimens positive by the xTAG for HRV/Enterovirus, the ResPlex II detected CVEV. In two specimens positive for HRV by MultiCode-PLx, the ResPlex II detected CVEV.

While the FDA approved version of the xTAG does not result coronaviruses and parainfluenza 4, these targets are screened for in the assay and were included for comparison purposes. For the coronaviruses, two specimens were positive for OC43 by the ResPlex II and the MultiCode-PLx PRV and none by the xTAG. Two specimens were positive for 229E by the ResPlex II and one by the MultiCode-PLx and xTAG, respectively (Table 1). The MultiCode-PLx was the only panel that detected Para4. The ResPlex II detected 1 Adeno C and 1 Adeno E. The latter were also detected with the MultiCode-PLx and xTAG, although the xTAG does not differentiate Adeno subtypes.

The ResPlex II had the highest number (146/202 or 72%) of internal control (IC) failures. These were not attributable to sample inhibition since amplification of the IC's from the other 2 PRV assays was observed in the same sample. Of these, 51% (102/202) were on negative specimens based on the results obtained by the other two PRV's. MultiCode-PLx had one IC failure on a specimen that was strongly positive for Para3. The xTAG had 7 IC failures. Of these, six specimens were positive for HRV, Adeno, 3 FluB and RSVA. One specimen, negative for all viruses was positive for IC upon repeat testing.

In comparison to xTAG, the MultiCode-PLx showed the highest sensitivity (81%) and NPV (90%). However, the ResPlex II was more specific (99%) and had a higher PPV (98%). The kappa for ResPlex II and MultiCode-PLx was 0.75 and 0.78, respectively (Table 3 ).

Table 3.

ResPlex II and MultiCode^®-PLx: comparison with xTAG respiratory virus panel.

xTAG/other PRV assay	ResPlex II	MultiCode®-PLx
Positive/positive	54	55
Positive/negative	20	13
Negative/positive	1	6
Negative/negative	123	115
Sensitivity (%)	73	81
Specificity (%)	99	95
PPV (%)	98	90
NPV (%)	86	90
Kappa	0.75	0.78

Only includes viruses that are common in the three test respiratory virus panels.

4.4. Discrepant analysis

A specimen was considered discrepant if the culture or any of the PRV assay results did not correlate. Thirty-six discrepant samples were re-tested using previously isolated nucleic acid stored at −70 °C. Upon re-testing, 15/36 specimens correlated. Of the remaining 21 specimens, 2 FluB's and 1 RSVB that were positive by the MultiCode-PLx and xTAG were negative by the ResPlex II. Additionally, one specimen positive for 229E by ResPlex II was positive by the xTAG, but negative by the MultiCode-PLx (data not shown). The MultiCode-PLx had a higher number of false negatives (1 RSVA, 4 HRV/CVEV's, and 3 FluA's). Of the four positive HRV by MultiCode-PLx, two were positive by the xTAG for HRV/Entero and for CVEV by the ResPlex II. xTAG was the only panel positive for 2 HMPV, 1 RSVA, and 1 HRV (data not shown).

5. Discussion

Nucleic acid amplification tests have shown increased sensitivity and faster reporting compared to non-amplification tests.⁷ The higher detection rates is attributable both due to the inclusion of large number of viral targets, including newly identified viruses for which culture and/or DFA is not available, and also due to the increased sensitivity allowing detection of low levels of virus.2, 3, 6, 7, 8, 9, 15 The latter is especially important in older adults because of shedding of low titers of viruses as compared to children that typically shed larger titers.22, 23 Ultimately, accurate diagnosis will optimize antiviral treatment and implementation of infection control and public health measures.¹⁶ New commercial systems are now available ensuring standardization and quality control.7, 24 The xTAG has shown better performance compared to DFA,²⁴ however comparative data with other newer commercial multiplex PRV assays is lacking.

As predicted, the PRV assays were more sensitive than viral culture. The xTAG showed the best sensitivity to common viral targets when compared to the ResPlex II and MultiCode-PLx assays. Previous studies have shown MultiCode-PLx to be highly sensitive in detecting FluA and other viral targets.7, 8 In our study, the MultiCode-PLx detected some FluA's, FluB's and Adeno that were missed by culture. However, it missed 3 FluA's, 2 HRV's, and 2 RSVA's detected by the other PRV assays (Table 1). Additionally, we encountered specimens that were positive for Para3 and 4 that were not confirmed by the ResPlex II, xTAG or by culture (data not shown). This may reflect low primer specificity for these agents. While the ResPlex II has been shown to have good correlation with DFA cross reactivity between Para1, 3 and 4 have been reported.²⁵ Internal control failures were noted with the ResPlex II as previously described²⁵, but were minimal with xTAG or MultiCode-PLx assays. Compared to culture, it was as sensitive as the MultiCode-PLx, but showed higher specificity (Table 2). While parainfluenza cross reactivity was not observed, the ResPlex II did not detect some Para3's and RSVA's. When compared to the xTAG, the ResPlex II showed lower sensitivity than MultiCode-PLx. However, it depicted higher specificity (Table 3). Overall, the xTAG showed the highest reproducibility with sensitivities and specificities comparable to values reported in the product insert.¹⁹ In the specimens studied, we detected adenovirus and parainfluenza viruses. However, a more extensive evaluation of adenovirus serotypes is warranted due to lack of detection of all serotypes as per the product insert.¹⁹ We detected a low rate of co-infections compared to previous studies. However the relative contribution of each potential pathogen is not clear at this time.²⁶

The number of targets and subtype differentiation varied for the 3 assays. The xTAG detects lower number of viral targets compared to the ResPlex II and MultiCode-PLx and does not differentiate adenoviruses, or Enteroviruses from HRV. Interestingly there were two cases where the MultiCode-PLx reported HRV and the ResPlex II reported CVEV. Because of the proprietary nature of the primers and probes, it is difficult to know if differentiation of HRV and Enteroviruses can be accomplished reliably. Sequencing was not performed due to lack of any remaining sample after testing with culture, the three PRV assays including discordant testing. The xTAG was the only assay to distinguish influenza A seasonal H1 and H3 subtypes. During the 2008–2009 influenza outbreak, a non-subtypeable influenza A (non-H1, non-H3) result was indicative for novel H1N1 influenza.27, 28, 29 This proved to be very useful, saving valuable laboratory resources and time by not having to reflex to a second confirmatory assay.

Assay performance, turn around time, ease of use, cost and regulatory status are all important factors to consider in choosing a multiplex PCR assay.³⁰ The PRV assays differed in the analysis steps and chemistry of detection17, 18, 19 (Supplementary Table 1). The assay complexity impacts turnaround time, influencing patient management. The ResPlex II generates results in 5.5 h, it is easy to perform, and only has one reagent preparation step and amplification step. Bead hybridization, detection, and reporting of results were the second step and they were performed in quick succession. On the other hand, the xTAG was the most labor intensive (7–8 h) due to five distinctive reagent preparations steps, two different thermocycler programs, and the complexity of the enzymes and reagents. While the xTAG and ResPlex II required only one thermocycler, the MultiCode-PLx required two thermocyclers, one for the reverse-transcriptase and the other for the Target-Specific-Primer-Extension that are performed in separate areas. In the MultiCode-PLx all post-amplification steps are carried out at room temperature without transfer or washings, and are completed in about 4–4.5 h. A drawback of these PRV assays is that they are open platforms and potentially prone to contamination.

In summary, the performance of the PRV assays yielded better sensitivity compared to culture. Each assay showed differences in sensitivities for individual viruses. While the ResPlex II test offers the broadest range of virus detection, and the MultiCode-PLx the greatest ease of use, the xTAG showed increased sensitivity and had the ability to subtype Influenza A. New generation assays promise enhanced sensitivity, faster turn around time, and additional viral and bacterial targets.9, 31 These multiplex assays should assist in identifying respiratory virus infections, aiding in patient management and investigation of outbreaks.

Funding

None.

Ethical approval

Not applicable.

Conflict of interest

None.