A case-control study evaluating RT-PCR/ESI-MS technology compared to direct fluorescent antibody and xTAG RVP PCR

Abstract

Waste nasopharyngeal swabs (N = 244) were evaluated by the reverse-transcriptase polymerase chain reaction/electrospray ionization mass spectrometry PLEX-ID Broad Respiratory Virus Surveillance Kit version 2.5 compared to direct fluorescent antibody and xTAG Respiratory Virus Panel for percent agreement, sensitivity, and specificity. Sensitivity and specificity were 91% (111/122) and 95.1% (116/122), respectively. Sensitivity by virus, except parainfluenza, was 92.9–100%, and specificity was 99–100%.

Respiratory viruses, especially influenza A virus, are seasonal sources of morbidity and mortality both in the United States and globally (Mahony, 2010, Mahony et al., 2011, Swayne and Spackman, 2013). The 2009 H1N1 Influenza A pandemic illustrated the importance of diagnostics to detect and identify respiratory viruses, and in general, accurate diagnostics are important in infectious disease management, making treatment decisions, and for novel pathogen discovery (Mahony, 2010, Mahony et al., 2011, Swayne and Spackman, 2013).

Viral diagnostics have migrated from culture to rapid and molecular methodologies, and more laboratories are utilizing these methods (Mahony, 2010, Mahony et al., 2011). Respiratory virus diagnostics, such as culture, immunologic techniques, sequencing, and molecular diagnostics, have advantages and disadvantages. Culture has low complexity but also low sensitivity and requires viable organism. Immunologic techniques like direct fluorescent antibody (DFA) are rapid but can be laborious and have lower sensitivity than molecular diagnostics. Sequencing provides substantial information but is technically complex and impractical. Molecular diagnostics, although fairly complex, are becoming the preferred technology because of high sensitivity and shorter testing time.

Reverse-transcriptase polymerase chain reaction/electrospray ionization mass spectrometry (RT-PCR/ESI-MS) technology is a highly complex technology that provides accurate diagnostic information for respiratory viruses, particularly influenza A (Chen et al., 2011a, Chen et al., 2011b, Deyde et al., 2010, Deyde et al., 2011, Forman et al., 2012, Jeng et al., 2012, Sampath et al., 2007, Tang et al., 2013, Murillo et al., 2013).

Performance evaluations of RT-PCR/ESI-MS for respiratory virus detection have been performed, notably by Chen et al., 2011a, Chen et al., 2011b, Forman et al. (2012), Tang et al. (2013). We expanded on these studies utilizing retrospectively collected waste nasopharyngeal swabs (NPs) and compared PLEX-ID RT-PCR/ESI-MS RVS 2.5 kit (Abbott Molecular, Des Plaines, IL, USA) results to both combined DFA and xTAG Respiratory Virus Panel (xTAG RVP) (Luminex Corporation, Austin, TX, USA) results as well as DFA and xTAG RVP individually, to determine percent agreement, sensitivity, specificity, and kappa for RVS 2.5.

As part of an institutional review board–approved study, waste NPs and their test results, DFA or xTAG RVP, were collected from clinical virology from 10/2011 to 2/2013. Samples were selected with a positive:negative test result ratio of 1:1 (N = 244, positive n = 122, negative n = 122). All samples were initially tested by DFA (total N = 244, positive n = 44, negative n = 200); DFA negative samples were tested with xTAG RVP only if the patient was immunocompromised. xTAG RVP screening of immunocompromised patients yielded additional positive samples (n = 78) for comparison with RT-PCR/ESI-MS. The remaining samples (n = 122) were negative by either DFA only for non-immunocompromised individuals or negative by DFA and xTAG RVP for immunocompromised individuals.

Nucleic acid extraction was performed utilizing the Arrow (NorDiag, Oslo, Norway) per manufacturer’s instructions. The Broad Respiratory Virus Surveillance Kit version 2.5 (RVS 2.5) and PLEX-ID were utilized for amplification and analysis per manufacturer’s instructions. Positive reactions were defined as respiratory virus identification with a Q score ≥0.9, and results were compared to the reported diagnostic result, either DFA or xTAG RVP for determination of percent agreement, sensitivity, specificity, and kappa. All positive sample types were considered negative for specificity purposes for other viruses, i.e., samples respiratory syncytial virus (RSV) positive but influenza A/B negative were considered negative for calculating influenza specificity. Discordant testing was not performed due to sample and nucleic acid extract volume limitations.

Positive samples consisted of: 34% influenza (n = 42, influenza A = 37, influenza B = 5), 26% RSV (n = 32), 15% coronavirus (n = 18), 11% metapneumovirus (n = 13), 9% parainfluenza virus 1–3 (n = 11) and 5% adenovirus (n = 6).

By test, positive percent agreement was 93.2% (41/44) and 89.7% (70/78) for DFA and xTAG RVP, respectively. These results and positive percent agreement by specific virus are summarized in Table 1 .

Table 1.

Positive percent agreement and percent agreement by virus and by reference method, DFA, and xTAG RVP PCR.

Reference test	Reference test result	RVS 2.5 result	Percent agreement
DFA	44	41	93.2% (41/44)
xTAG RVP PCR	78	70	89.7% (70/78)
Total	122	111	91% (111/122)
Influenza DFA	17	16	94.1% (17/18)
Influenza xTAG RVP PCR	25	23	92% (23/25)
Total	42	39	92.9% (39/42)
RSV DFA	18	17	94.4% (17/18)
RSV xTAG RVP PCR	14	13	92.9% (13/14)
Total	32	30	93.8% (30/32)
Coronavirus DFA	0	0 (2a)	NA
Coronavirus xTAG RVP PCR	18	17	94.4% (17/18)
Total	18	17	94.4% (17/18)
Metapneumovirus DFA	4	4	100% (4/4)
Metapneumovirus xTAG RVP PCR	9	8	88.8% (8/9)
Total	13	12	92.3% (12/13)
Parainfluenza virus 1–3 DFA	4	3 (1a)	75% (3/4)
Parainfluenza virus 1–3 xTAG RVP PCR	7	4	57.1% (4/7)
Total	11	7	63.6% (7/11)
Adenovirus DFA	1	1 (2a)	100% (1/1)
Adenovirus xTAG RVP PCR	5	5	100% (5/5)
Total	6	6	100% (6/6)

^aIndicates a false-positive result defined as a result that was positive utilizing the PLEX-ID RVS 2.5 kit and negative by DFA.

DFA and xTAG RVP were also utilized as gold standards to calculate sensitivity and specificity, overall, and by virus for RVS 2.5 (Table 2 ).

Table 2.

Sensitivity, specificity, and kappa analysis for RVS 2.5 compared to combined DFA and xTAG RVP as gold standard assays.

Pathogen	Sensitivity (n)	Specificity (n)a	Kappa (95% CI)
All	91.0% (111/122)	95.1% (116/122)	0.86 (0.79–0.92)
Influenza	92.9% (39/42)	100% (202/202)	0.96 (0.91–1)
RSV	93.8% (30/32)	99.5% (211/212)	0.95 (0.88–1)
Coronavirus	94.4% (17/18)	99% (224/226)	0.91 (0.81–1)
Metapneumovirus	92.3% (12/13)	100% (231/231)	0.96 (0.88–1)
Parainfluenza 1–3	63.6% (7/11)	99.5% (231/233)	0.69 (0.44–0.93)
Adenovirus	100% (6/6)	99.0% (236/238)	0.85 (0.65–1)

^aCalculated using all non-virus positives as negative. For example, samples positive for RSV were considered negative for influenza virus.

RVS 2.5 results compared well to DFA and xTAG RVP. Overall agreement was 93% (227/244, kappa = 0.86, 95% confidence interval [CI]: 0.79–0.92); overall positive and negative agreement were both >90%. DFA and xTAG RVP were utilized as gold standard assays; sensitivity and specificity were both >90% for RT-PCR/ESI-MS compared to DFA and xTAG RVP. By virus, all percent agreements, sensitivities, and specificities were >90%. However, parainfluenza virus 1–3 had overall agreement and sensitivity of 63.4% (7/11). A limited number of parainfluenza 1–3 positives (n = 11) evaluated contributed to this; it does not appear that sample quality or volume was an issue because if this were the case, the results would be affected across all positive samples. The parainfluenza 1–3 false negatives as well as other false-negative results from the study were potentially because of the quantity of virus in the sample tested was below the limit of detection of 150 copies per well for all viruses the RVS primers amplify (Chen et al., 2011b).

The results here are similar to Chen et al., 2011a, Chen et al., 2011b and Forman et al., (2012). Chen et al. utilized RVS 2.0; Forman et al. and this evaluation employed RVS 2.5. Sensitivity and specificity improved from 89.1% and 80.3%, respectively, to 91% and 95.1%, respectively, compared to Chen et al. (2011a). Positive percent agreement improved from 77.9% to 91% compared to Chen et al., (2011b). This study reports higher agreement for metapneumovirus than Forman et al. (2012), at 92.3%, although this study evaluated fewer metapneumovirus positives (n = 13).

Nucleic acid extraction methodology utilized here differed between this study and Chen et al., 2011a, Chen et al., 2011b and Forman et al. (2012) and possibly contributed to sensitivity and specificity improvements.

Despite numerous limitations to the study, the results presented here add additional performance characteristic information to previously published studies that utilized RT-PCR/ESI-MS for the detection and identification of respiratory viruses (Deyde et al., 2010, Deyde et al., 2011, Sampath et al., 2007, Tang et al., 2013, Murillo et al., 2013).

Few samples positive for coronavirus, metapneumovirus and adenovirus were tested. No further analysis was performed on discrepant results due to low sample and nucleic acid extract volumes after RT-PCR/ESI-MS testing was completed. This was especially the case for the DFA-negative/RT-PCR/ESI-MS–positive samples (Table 1). Comparison of DFA with RT-PCR/ESI-MS was problematic because comparison of a non-amplified diagnostic, DFA, with an amplified diagnostic, RT-PCR/ESI-MS, may have skewed the RT-PCR/ESI-MS performance characteristics. Additionally, all of the xTAG RVP–positive results were initially DFA negative, and this discrepancy was also problematic. Additionally, DFA could have generated false-positive results, as 3 DFA-positive detections were not confirmed by RT-PCR/ESI-MS or xTAG RVP. A more accurate comparison would have utilized 2 amplified tests, such as xTAG RVP and type specific real-time PCR. Although DFA comparison was suboptimal, it was the best available comparison method based on clinical virology screening algorithms used to report respiratory virus diagnoses. Finally, xTAG RVP assay was only performed on DFA-negative samples collected among immunocompromised patient negatives as per clinical virology testing algorithm policy. Many DFA-negative samples collected among immunocompetent patients could have tested positive with xTAG RVP. Additionally, the DFA negative, RT-PCR/ESI-MS false positives (Table 1.) could have been more accurately confirmed if RT-PCR/ESI-MS and xTAG RVP had been directly compared for those samples.

RT-PCR/ESI-MS technology demonstrated the capability to detect and identify respiratory viruses when compared to DFA and xTAG RVP, and all have utility in screening NP samples for respiratory viruses. Based on the limitations associated with this study, further evaluations of RT-PCR/ESI-MS technology, including prospective analysis and comparison with multiple molecular methods, will be required before clinical implementation.