Comparative Evaluation of the Nanosphere Verigene RV+ Assay and the Simplexa Flu A/B & RSV Kit for Detection of Influenza and Respiratory Syncytial Viruses

Abstract

Using retrospective (n = 200) and prospective (n = 150) nasopharyngeal specimens, we evaluated the Nanosphere Verigene RV+ and the Focus Diagnostics Simplexa Flu A/B & RSV tests. Overall, RV+ demonstrated sensitivities and specificities of 96.6% and 100% for influenza A virus, 100% and 99.7% for influenza B virus, and 100% and 100% for respiratory syncytial virus (RSV), while the Simplexa test sensitivities and specificities were 82.8 and 99.7%, 76.2 and 100%, and 94.6 and 100%, respectively.

TEXT

Viral respiratory infections are a leading cause of morbidity and mortality, especially during the winter months, with the most severe infections being attributed to influenza virus and respiratory syncytial virus (RSV). The low sensitivity of rapid antigen tests (1, 2) and the delayed time to result of viral culture combined with the improved sensitivity afforded by molecular methods has led to an increase of FDA-cleared tests and systems designed to detect these viruses. The Nanosphere Verigene Respiratory Virus Plus (RV+) test (Northbrook, IL) is an FDA-cleared random access molecular test based on nanoparticle technology that does not require pre-extraction of specimens (3). The RV+ test detects influenza A virus, influenza B virus, RSV A, and RSV B while also providing influenza A virus-specific typing results for 2009 H1N1, H3, and H1. Also FDA cleared, the Focus Diagnostics Simplexa Flu A/B & RSV kit (Cypress, CA) is a multiplex real-time PCR assay that requires pre-extraction and batched testing. The Simplexa test provides results for influenza A virus, influenza B virus, and RSV but does not subtype influenza A virus. Both tests are FDA cleared for use on nasopharyngeal swabs.

We conducted a side-by-side, retrospective and prospective evaluation of the Nanosphere Verigene system running the RV+ test and the Focus Diagnostics Simplexa Flu A/B & RSV kit on an integrated cycler to determine the respective performance characteristics. Positive and negative specimens were originally identified by routine clinical testing by Xpert Flu A/B (n = 72; Cepheid, Sunnyvale, CA), Luminex RVP (n = 46; Austin, TX), or our laboratory-developed tests (LDTs) (n = 82). We have previously shown that Xpert and RVP are 100% specific for the viruses tested compared to our LDTs (reference 4 and our unpublished data). All discrepant results were simultaneously tested by our influenza A and B virus (4) and RSV (5) LDTs, in addition to repeating both test methods (RV+ and Simplexa). The RV+ and Simplexa tests were performed according to the manufacturers' instructions.

The retrospective evaluation included 200 nasopharyngeal (NP) specimens (192 NP swabs and 8 NP aspirates/washes), of which 55 were positive for influenza A virus (21 2009 H1N1, 23 H3, and 11 H1), 21 were positive for influenza B virus, 42 were positive for RSV (21 each of A and B), and 82 were negative for influenza and RSV. Specimens were collected between 2005 and 2012 (17 were collected prior to 2009), stored at −70°C, thawed once, and kept refrigerated until testing. For the prospective testing, 150 consecutive nasopharyngeal samples (144 NP swabs and 6 NP aspirates) were tested by the RV+ assay upon receipt at the lab; specimens were stored at 4°C and tested by the Simplexa assay and our LDTs once a batch had accumulated. All specimens were vortexed prior to testing. The prospective specimens were ordered for only influenza and/or RSV testing and collected between January and June 2012. Of the prospective samples, three were positive for influenza A virus (2 2009 H1N1 and 1 H3) and 51 were positive for RSV (44 A and 7 B), while 96 were negative. Due to the relative lack of influenza activity during the prospective study and the lack of statistically significant differences in RSV sensitivity in comparisons of retrospective and prospective studies, only cumulative data (n = 350) are presented. No inhibition was observed in any specimen type, even those not FDA cleared (e.g., NP aspirates and washes). Fisher's exact test for 2-by-2 contingency table analysis was performed and confidence intervals were determined using GraphPad (La Jolla, CA); a two-tailed Student's t test was performed using Excel (see Table 2). P values of <0.05 were considered statistically significant.

Table 2.

Statistical analysis of cycle threshold values of discrepant samples

Virus and test result	n	Mean CT (95% CI)	P value
Influenza A virus
RV+ positive	51	24.9 (23.2 26.7)}	0.0039
RV+ negative	2	38.2 (26.1, 50.3)
Simplexa positive	43	23.2 (21.7, 24.7)}	<0.0001
Simplexa negative	10	35.1 (33.2, 36.9)
Influenza B virus
Simplexa positive	13	28.0 (25.1, 30.9)}	0.0555
Simplexa negative	5	32.6 (31.1, 34.1)
RSV
Simplexa positive	82	22.0 (20.9, 23.1)}	<0.0001
Simplexa negative	5	34.9 (29.3, 40.5)

The combined results of the retrospective and prospective analyses are shown in Table 1. Although the difference did not reach statistical significance, the Verigene RV+ test was more sensitive for RSV detection than the Simplexa test (100% versus 94.6%, respectively; P = 0.059). However, for the detection of influenza A virus (96.6% versus 82.8%; P = 0.029) and influenza B virus (100% versus 76.2%; P = 0.048), the sensitivity of the Verigene RV+ assay was superior. Both tests were highly specific. The Nanosphere test had one false-positive influenza B virus result, and the Simplexa test had one false-positive influenza A virus result, resulting in overall specificities of 99.4% for both assays. Although not FDA cleared, NP aspirates (n = 14) performed comparably to NP swabs.

Table 1.

Sensitivity of the Verigene RV+ test and the Simplexa Flu A/B & RSV kit by virus (n = 350)

Test	% Sensitivity fora:
	Influenza A virus	Influenza B virus	RSV
Verigene RV+	96.6 (56/58)	100 (21/21)	100 (93/93)
Simplexa	82.8 (48/58)	76.2 (16/21)	94.6 (88/93)

^aValues in parentheses are number of samples positive by the test method/number positive by the reference method.

Based on the limits of detection (measured as 50% tissue culture infective doses [TCID₅₀]/ml) stated in the respective package inserts, we expected the two assays to perform similarly. However, direct comparison is difficult due to differences in strains used for limit-of-detection studies. Therefore, we evaluated the cycle threshold (C_T) values of our LDT to see if a correlation could be made between samples missed by the test assay and their LDT C_T value. The results of these analyses are shown in Table 2. We note that some positive specimens were originally identified using methods other than our LDTs and were not included in the C_T analysis (n = 14). There was a statistically significant difference between the mean LDT C_T values for detected positive specimens and the mean LDT C_T values for false-negative influenza A virus specimens by both RV+ (P = 0.0039) and Simplexa (P < 0.0001) and false-negative RSV specimens by Simplexa (P < 0.0001). Likely due to the relatively low numbers of influenza B virus-positive specimens, the difference in C_T values for Simplexa did not reach statistical significance (P = 0.0555). No influenza B virus or RSV samples were called falsely negative by the RV+ assay. These data suggest that the false-negative results obtained by RV+ and Simplexa are due to the lower analytical sensitivities of these tests than of our LDTs.

In terms of workflow, the two systems are significantly different. As an on-demand modular system using integrated extractions, the Verigene RV+ assay requires ∼5 min of hands-on time per specimen and reports results in 2.5 h. However, only one sample can be run per module at one time. In contrast, the Simplexa test requires 45 min of hands-on time (including pre-extraction of up to 24 samples) but still offers results in ∼2.5 h. However, since the Simplexa test is better performed as a batch test, there is the potential for a higher throughput. The throughput of the RV+ test depends on the number of Verigene processors. Both assays offer results faster than most LDT real-time RT-PCR assays, which require pre-extraction and batch testing. The average reagent cost per test in 2012 U.S. dollars is $70 for the RV+ and $49 for the Simplexa assay (including extraction). As noted above, the labor costs vary significantly for the two assays.

To our knowledge, this is the first study to fully assess the performance of these two assays. Jannetto et al. described per-analyte sensitivities and specificities of a predecessor Nanosphere assay as 100 and 99.8% for influenza A virus, 100 and 99.1% for influenza B virus, and 91.7 and 98.4% for RSV and demonstrated the current automated Nanosphere system to be functionally equivalent to its predecessor (3). Our RV+ data are consistent with those previously published and the package insert. Although there are no published data on the Simplexa assay, our results differ from those presented in the package insert. Of note, both package inserts compare performance to that of direct fluorescent-antibody testing and culture, while our study compares the tests to other molecular methods. Another possible explanation for differences in observed sensitivities could be regional strain differences. We have ruled out specimen storage effects by simultaneously testing all discrepant results by all methods. As such, only results obtained after discrepant analysis are included in our sensitivity and specificity calculations, which we acknowledge may bias the data in favor of the tests being evaluated.

In our hands, the Verigene RV+ assay showed sensitivities superior to those of the Simplexa Flu A/B & RSV kit for influenza A and influenza B viruses, while the sensitivities for RSV and all specificities were equivalent. In addition, the Verigene test has the added benefit of providing influenza A virus subtyping results that may prove helpful in epidemiologic investigations, the identification of new variant influenza strains and/or oseltamivir resistance prediction. These benefits are countered by the relatively low throughput of the Verigene system. Balancing the need for accurate, rapid results with the need for high throughput remains a challenge for many laboratories performing molecular detection of respiratory viruses. Depending on the needs of a given laboratory, one or both of these FDA-cleared tests may provide the necessary accuracy, time to result, and throughput to provide meaningful clinical data.