Abstract
This multicenter study evaluated the clinical performance of the 3M Rapid Detection RSV test (3MRSV) compared to a composite reference standard of R-Mix culture and direct specimen immunofluorescence for detection of respiratory syncytial virus (RSV). The performance of the BinaxNOW RSV test was also evaluated using this reference standard. In a secondary analysis, discordant results were arbitrated using the Gen-Probe/Prodesse ProFlu+ reverse transcription-PCR (RT-PCR) assay. Subjects were stratified into three groups as follows: group 1 (G1), all ages; G2, subjects <22 years old (FDA-cleared ages for 3MRSV testing); and G3, subjects <5 years old (FDA-cleared ages for BinaxNOW RSV testing). A total of 1,306 specimens (G1, n = 1,306; G2, n = 1,140; G3, n = 953) from subjects of all ages presenting with respiratory symptoms met study criteria for analysis. Sensitivities, specificities, positive predictive values, and negative predictive values of 3MRSV for G1 were 86.5%, 95.8%, 91.4%, and 93.2%, respectively, and those for G2 were 87.3%, 95.6%, 92.4%, and 92.5%, respectively. For those samples analyzed by both 3MRSV and BinaxNOW, the 3MRSV was more sensitive (G1, 86.3%; G2, 87.2%; and G3, 89.9%) than was BinaxNOW (G1, 70.84%; G2, 72.0%; and G3, 72.4%) (P < 0.05). Specificities for RSV detection from nasopharyngeal (NP) aspirates and NP swabs for all groups were comparable for 3MRSV and BinaxNOW, but 3MRSV was less specific than BinaxNOW when nasal washes/aspirates were tested (P < 0.05). The 3MRSV assay performed well for the detection of RSV, and the overall assay performance was superior to that of BinaxNOW. The 3MRSV reader eliminated user misinterpretation and provided test result and quality control documentation.
Respiratory syncytial virus (RSV) is the most common cause of bronchiolitis and pneumonia associated with hospital admissions in children younger than 1 year old (14). In addition, RSV is responsible for significant morbidity in older infants and young children, the elderly, and the immunocompromised (5, 8, 16). RSV is also associated with substantial morbidity in infants and children in an outpatient setting (9). Hall et al. estimated that RSV infection in children under the age of 5 years results in 1 of 334 hospitalizations, 1 of 38 visits to an emergency department, and 1 of 13 visits to a primary care provider each year in the United States (9). When population-based data were extrapolated to the entire U.S. population, these authors estimated that 2.1 million children under 5 years of age with RSV infection would have medical attention each year. Moreover, a hallmark of RSV infection is that a single infection does not confer lifelong immunity resulting in reinfection throughout life (8). Thus, RSV imposes a substantial burden of disease, particularly in the pediatric population.
Accurate diagnosis depends on detection of RSV in respiratory tract specimens, because differentiation of RSV infection from other viral respiratory infections based on clinical signs and symptoms alone is inaccurate (15). Antigen detection methods such as rapid membrane immunoassays offer the advantages of a rapid time to results (generally 15 to 30 min) and relative ease of use but are less sensitive than direct specimen immunofluorescent-antigen testing (DSFA) of respiratory epithelial cells, viral culture, and nucleic acid amplification tests (10, 11). Nevertheless, rapid antigen tests allow for RSV testing in clinical settings that lack the laboratory support for these more complex tests.
The performance of rapid immunoassays and thus the clinical utility of test results are influenced by a variety of factors, including specimen type, specimen collection method, timing of specimen collection relative to onset of symptoms, and patient age. Children shed higher titers of virus for longer time periods than adults; thus, rapid immunoassays are more sensitive in the pediatric population, particularly in children <5 years of age (2, 4, 7, 10). Additional factors affecting rapid test performance and utility include the subjective visual interpretation of results and low positive predictive values when testing is performed during periods of low disease prevalence (10).
This study was a prospective, multicenter, premarket clinical trial designed to establish the performance of the 3M Rapid Detection RSV test (3MRSV) (3M Health Care, Saint Paul, MN) for U.S. Food and Drug Administration (FDA) clearance as an in vitro diagnostic device. The 3MRSV is a qualitative immunochromatographic cartridge test for the detection of RSV in nasal aspirate/wash and nasopharyngeal aspirate/swab specimens. 3MRSV results were compared to results of DSFA, viral culture, and another rapid antigen test, the BinaxNOW RSV test (Binax, Inverness Medical, Waltham, MA).
After completion of this study, the 3M Rapid Detection RSV test received clearance from the FDA as an in vitro diagnostic assay for the detection of RSV F protein antigens in nasopharyngeal aspirate/swab specimens and nasal aspirate/wash specimens in symptomatic patients less than 22 years of age.
(This study was presented in part at the 25th Clinical Virology Symposium, Daytona Beach, FL, 2010.)
MATERIALS AND METHODS
Study subjects.
This study was conducted from December 2008 through January 2009, a period of high prevalence of RSV at each of the eight geographically diverse U.S. clinical trial sites that participated in this study (13). Inclusion criteria required the presence of at least two of the following symptoms: wheezing, coughing, nasal discharge, nasal congestion, fever (≥100.4°F), and/or rales on chest exam. A total of 1,501 subjects of all ages were enrolled in the study, 793 females and 707 males, with one gender not reported. Three subject groups were established: group 1 (G1) (n = 1,501) consisted of subjects of all ages (range from 1 week to >89 years), G2 (n = 1,279) consisted of subjects <22 years old (the age group cleared by the FDA for testing with 3MRSV), and G3 (n = 1,064) consisted of subjects <5 years old (the age group cleared by the FDA for testing with the BinaxNOW RSV test). The number of subjects (percentage of total) by various age subgroups was 1,087 (72.4%) for 0 to 5 years, 192 (12.8%) for 6 to 21 years, 131 (8.7) for 22 to 59 years, and 91 (6.1%) for ≥60 years. One site enrolled 28% of the subjects, and the other seven sites each enrolled between 6% and 13% of the subjects. This study was performed with the approval of each site's Institutional Review Board.
Study samples.
Samples used for this study consisted of respiratory secretions submitted to participating laboratories by physician order for routine diagnostic testing and for which there was residual material destined to be discarded; samples were deidentified before being made available to study personnel. Only one sample per subject was permitted. Respiratory samples were collected in accordance with each site's standard practice. Sample types included nasal washes/aspirates (NW/A) (n = 301 [G1], 285 [G2], and 263 [G3]), nasopharyngeal aspirates (NPA) (n = 593 [G1], 570 [G2], and 481 [G3]), nasopharyngeal swabs (NPS) (n = 605 [G1], 442 [G2], and 318 [G3]) in viral transport media (UTM, Diagnostic Hybrids [DHI], Athens, OH [two sites]; M4, Remel, Lenexa, KS [three sites]; M4RT, Remel [two sites]), and throat swabs (n = 2 [G3]) in viral transport media. Flocked swabs (Copan, Murrieta, CA) were used for sample collection at four of the five sites that collected NPS. NPA were collected using a catheter inserted into the nose directed posteriorly and toward the opening of the external ear. Suction was applied, and using a rotating movement, secretions were collected as the catheter was withdrawn. NW/A were collected by instilling saline, via a syringe and tubing, into the nostril and aspirating the recoverable nasopharyngeal specimen. The presence of mucus or blood (slight or excess) in the sample was noted. Samples were stored at 2 to 8°C if 3MRSV was not performed within 4 h of sample collection. Upon completion of all required testing, the remaining excess original sample was frozen in 300-μl aliquots at ≤−70°C in case discordance analysis was required.
3M Rapid Detection RSV test.
The 3MRSV was performed within 72 h of sample collection and according to the manufacturer's instructions. Refrigerated samples were stored at room temperature for 15 min prior to testing. Briefly, 150 μl of neat sample was added to a vial containing 150 μl of 3MRSV sample buffer. The neat sample and 3MRSV sample buffer were mixed 10 times using an assay pipette tip that contains fluorophore-tagged particles coated with anti-RSV antibodies directed against F protein antigens of RSV (Fig. 1). A 75-μl aliquot of the mixture was transferred into the well of a test cartridge and inserted into the 3M rapid detection reader. The sample migrates along the strip contained within the test cartridge, and anti-F protein antibodies bind to the corresponding RSV antigens if present. RSV-bound particles are captured at the RSV detection zone, and excess particles are captured at the internal standard (IS) zone. The rapid detection reader measures the fluorescence emitted by the complexes at the RSV and IS detection zones and calculates a ratio between the RSV and the IS zone readings. If the 3MRSV failed, it was repeated with neat sample up to two times. If the repeat testing failed, then the sample was diluted with an equal volume of the viral transport medium used at the trial site and the test repeated a final time. The result was scored as invalid if the final diluted sample did not yield an acceptable 3MRSV result. Positive and negative controls were processed on each day of sample testing. Each trial site performed 3MRSV using three different lot numbers.
FIG. 1.
Test procedure for the 3M Rapid Detection RSV test. 1, touch screen rapid detection reader. Sample (2) is added to buffer (3) and mixed with an assay tip (4) that contains fluorescence-dyed particles conjugated to specific RSV antibodies that bind to RSV F protein antigens, if present in the sample. The sample is then applied to the sample well of the test cartridge (5), which is inserted into the 3M rapid detection reader (6). As the sample migrates along the strip, RSV-bound particles are captured by anti-RSV antibodies at the detection zone. Excess fluorescence-dyed particles are captured at the internal standard (IS) zone. The reader measures the fluorescence emitted by the complexes at the detection zone and the IS zone and calculates a ratio (RAMP ratio) of the detection zone fluorescence reading to the IS zone reading. The reader then compares these ratios to predefined threshold limits to determine a positive or negative result for RSV in the tested sample. The instrument will flag the test as invalid if the sample fails to migrate through the cartridge or if the IS signal is low. The test time is approximately 17 to 18 min, including 2 to 3 min for sample preparation and a 15-min instrument time. The reader printout lists the target-specific result, kit lot number, expiration date, test date and time, sample and user ID, and test port serial number. Kit positive and negative controls, patient results, and internal instrument function checks are stored in the reader for easy reference. (Courtesy of 3M, reproduced with permission; originally published in reference 6.)
The total test time is approximately 17 to 18 min, including sample preparation, cartridge loading, and instrument time. A single test module contains two ports, and a maximum of three test modules can be connected to a single reader. Therefore, a maximum of six tests per reader can be performed at one time.
External controls.
For each day of testing, the external positive and negative kit controls were processed and tested according to the manufacturer's instructions. Controls were repeated if an invalid result (sample error 2) was obtained. Sample error 2 can be caused by failure of the sample to flow through the cartridge, but it may also be caused by not using the provided assay tips, a clogged sample pad, insufficient mixing, or interfering substances.
Primary comparator test methods.
The results of the 3MRSV were compared to those of cell culture (R-Mix [DHI], six sites; R-Mix Too [DHI], one site; R-Mix and fibroblast tube culture, one site) and direct specimen fluorescent-antibody testing (DSFA) using D3 Ultra reagents (DHI) (six sites) or Simulfluor (Millipore, Billerica, MA) (two sites). DSFA and cell culture inoculation were performed in accordance with the manufacturers' instructions and each trial site's validated procedures within 24 h of sample collection. A sample was considered positive for RSV if the DSFA and/or viral culture was positive and was considered negative for RSV if both DSFA and viral culture were negative.
Secondary comparator test method.
The results of the 3MRSV were also compared to those of the BinaxNOW rapid RSV test. The BinaxNOW RSV test was performed within 24 h of the performance of 3MRSV. Refrigerated samples were stored at room temperature for 15 min prior to testing. BinaxNOW testing was performed using 100 μl of neat sample and according to the manufacturer's instructions. The BinaxNOW assay requires approximately 1 min for addition of sample. The card is immediately sealed, and the test must be read in exactly 15 min. Samples were considered positive or negative using the same criteria as described above for 3MRSV.
Specimen inclusion criteria for data analysis.
Specimens required valid reference test data (DSFA and/or viral culture) and valid 3MRSV data to be included in the statistical analysis. Exclusion rules for analysis for samples with reference and 3MRSV results included the following: (i) DSFA was negative but no culture results were available, or vice versa, and therefore this sample could not be considered negative; (ii) DSFA or culture was performed more than 24 h after sampling and both results were negative, since lability of RSV could have resulted in a false-negative result; (iii) time deviations for 3MRSV results (3MRSV was run later than 4 h after sample collection and not refrigerated); (iv) 3MRSV results were obtained in a run containing a failed control; and (v) improper sample types (throat swabs) were tested.
Stratification of test results.
3MRSV results were compared to DSFA and viral culture results, stratified by subject groups G1 and G2 and by sample types. A separate comprehensive analysis of subjects >22 years old (n = 166), beyond overall sensitivity and specificity, was not statistically relevant due to the low number of RSV-positive samples in this group (n = 10), rendering detailed comparisons unreliable. 3MRSV results were compared to BinaxNOW RSV results, stratified by subject groups G1, G2, and G3 and by sample type.
Testing and analysis of samples with discordant results.
Further testing and analysis were performed for 42 samples for which there was discordance between DSFA and culture results (32 DSFA positive and culture negative, 1 DSFA positive and culture indeterminate, 4 DSFA indeterminate and culture positive, and 5 DSFA negative and culture positive) and for 36 samples that were deemed to be 3MRSV false positive (culture and DSFA negative). Discordance between the 3MRSV result and the culture, DSFA, or BinaxNOW result was resolved by testing a frozen (−70°C) aliquot of the original sample with the ProFlu+ reverse transcription-PCR (RT-PCR) assay (Gen-Probe/Prodesse, Waukesha, WI) after nucleic extraction using the NucliSENS easyMAG (bioMérieux, Durham, NC). Nucleic acid extraction and ProFlu+ were performed at two testing sites according to the manufacturer's instructions. A 3MRSV negative result was considered a true negative if the DSFA was positive and the ProFlu+ was negative. A 3MRSV positive result was considered a true positive if the reference tests (DSFA and culture) were negative and the ProFlu+ was positive. Results of discordance analysis were not included in the primary assessment of the 3MRSV performance since not all samples were tested by ProFlu+.
Statistical analysis.
The analysis of 3MRSV results compared to DSFA and viral culture results was performed separately for all samples that met the inclusion criteria for G1 and G2. Analysis of the performance of 3MRSV compared to BinaxNOW was performed for all samples meeting the inclusion criteria for G1, G2, and G3. Sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs) for 3MRSV and BinaxNOW were calculated using standard formulas and the significance between the values determined using McNemar's test. A P value of <0.05 was considered statistically significant. Values were recalculated after discordance analysis. Because only samples with discordant results for 3MRSV and the reference standard were tested with ProFlu+, the adjusted sensitivity and specificity values can be expected to be biased high (12) and were used to resolve only potential false-negative and false-positive 3MRSV results.
RESULTS
Sample inclusion.
A total of 195 samples were not included in the primary analysis of test performance, which required valid reference and 3MRSV results. The reasons for exclusion included improper sample collection (n = 2, throat swabs), culture and/or DSFA inoculation time beyond 24 h (n = 26), one reference method negative and one incomplete (n = 135), 3MRSV performed beyond 72 h of sample collection (n = 1), no 3MRSV result available due to an error code most often associated with the consistency (mucoid) of the sample (n = 27), or control failure (n = 4). After exclusion, 1,306 (87%) of the samples (G2, n = 1,140; G3, n = 953) gave valid results for the comparison of 3MRSV to the reference methods of DSFA and viral culture. (The final numbers of samples per type included in the analysis are shown in Table 2 for G1 and G2 and in Table 3 for G3).
TABLE 2.
Performance of 3MRSV by subject group and sample type
| Parameter | Value fora: |
|||||
|---|---|---|---|---|---|---|
| Nasal washes/aspirates |
Nasopharyngeal aspirates |
Nasopharyngeal swabs |
||||
| Group 1 | Group 2 | Group 1 | Group 2 | Group 1 | Group 2 | |
| No. tested | 262 | 246 | 519 | 503 | 525 | 391 |
| Sensitivity, % (95% CI) | 89.1 (0.819-0.936) | 89.9 (0.828-0.943) | 85.4 (0.799-0.897) | 85.3 (0.797-0.896) | 85.8 (0.789-0.907) | 88.2 (0.814-0.927) |
| Specificity, % (95% CI) | 94.1 (0.891-0.969) | 93.4 (0.880-0.965) | 95.3 (0.924-0.971) | 95.1 (0.921-0.970) | 96.9 (0.947-0.982) | 97.4 (0.946-0.987) |
| PPV, % | 91.6 | 91.6 | 91.9 | 91.8 | 90.6 | 94.1 |
| NPV, % | 92.3 | 92.1 | 91.3 | 90.9 | 95.2 | 94.5 |
Group 1, subjects of all ages; group 2, subjects <22 years old (age group FDA cleared for 3MRSV testing).
TABLE 3.
3MRSV performance by testing site for subjects <22 years old and for all sample types combined
| Parameter | Value for testing site: |
|||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
| No. tested | 229 | 90 | 135 | 178 | 182 | 77 | 166 | 83 |
| Sensitivity, % (95% CI) | 89.7 (0.810-0.947) | 83.3 (0.641-0.933) | 76.9 (0.579-0.890) | 89.8 (0.822-0.944) | 88.4 (0.799-0.936) | 88.0 (0.700-0.958) | 79.6 (0.671-0.882) | 92.9 (0.810-0.975) |
| Specificity, % (95% CI) | 98.7 (0.953-0.996) | 95.5 (0.875-0.984) | 95.4 (0.897-0.980) | 93.8 (0.862-0.973) | 97.9 (0.927-0.994) | 98.1 (0.899-0.997) | 93.8 (0.877-0.969) | 85.4 (0.716-0.931) |
| PPV, % | 97.2 | 87.0 | 80.0 | 94.6 | 97.4 | 95.7 | 86.0 | 96.7 |
| NPV, % | 94.9 | 94.0 | 94.5 | 88.2 | 90.4 | 94.4 | 90.5 | 92.1 |
Impact of sample characteristics on performance of 3MRSV and the BinaxNOW test.
Sample characteristics were assessed for all 1,501 samples. Overall, 3.9% of the samples (n = 58) were classified as bloody (50 slightly and 8 excessively). No relationship was found between a bloody sample and the need for retesting. Fourteen percent of the samples (n = 208) were classified as mucoid. The percentages of mucoid samples by sample types were 1.5% for NPS, 18.6% for NW/A, and 24.5% for NPA. A relationship between whether a sample was mucoid or nonmucoid and the need for retesting was identified. Primary testing using only 3MRSV yielded a significantly lower proportion of valid test results for mucoid samples (67.3%) than for nonmucoid samples (95.7%) (P < 0.0001). Repeat 3MRSV testing was necessary for 121 (8.1%) of the samples (66 mucoid and 55 nonmucoid). Due to repeated invalid results with neat sample, dilution was required for 78 samples (5.2%), of which 51 samples (24.5%) were mucoid and 27 samples (2.1%) were nonmucoid (P < 0.0001). Seventy-five of the 78 samples were from G2 subjects. One sample was not diluted and retested with 3MRSV due to insufficient remaining sample. There were 43 samples that tested 3MRSV positive after dilution, of which only 1 was considered a false positive (DSFA and culture negative). There were 13 samples that tested 3MRSV negative after dilution, of which 1 was DSFA positive and therefore the 3MRSV result was considered a false negative. Of the remaining 23 samples (16 NPA, 6 NW/A, and 1 NPS) with invalid 3MRSV results after dilution, 15 were positive by DSFA and/or culture. The final 3MRSV invalid rate for mucoid samples (9.1%) was significantly higher (P < 0.0001) than that for nonmucoid samples (0.3%). The overall 3MRSV invalid test rate for all samples was 1.5% (23/1,501 samples). There were initially 24 samples with invalid BinaxNOW results. Two samples did not have sufficient volume for retesting. After retesting with BinaxNOW, 7 of the 22 remaining samples again gave invalid results, for a final BinaxNOW invalid rate of 0.5%.
Overall performance of the 3MRSV.
The performance measures of the 3MRSV for detection of RSV in all sample types (n = 1,306) compared to a composite standard of DSFA and viral culture for G1 (all ages) and G2 (<22 years old) are shown in Table 1. Measures for G1 and G2, respectively, were as follows: sensitivities, 86.5% and 87.3%; specificities, 95.8% and 95.6%; PPVs, 91.4% and 92.4%; and NPVs, 93.2% and 92.5%. The sensitivity of 3MRSV for detection of RSV in subjects ≥22 years old was 50%, which was significantly lower than the sensitivity in subjects <22 years old (P = 0.033) (data not shown). The specificity of 3MRSV for the detection of RSV in subjects <22 years old was 95.6%, which was comparable to the specificity in subjects ≥22 years old (96.8%). The mean performance measures were weighted by the NPA sample type (45%) for G2 subjects. For all subjects (G1), the ratio was 20:40:40 for NW/A to NPA to NPS. Overall, there were 443 RSV-positive samples from G1 (110 NW/A, 199 NPA, and 134 NPS) and 433 RSV-positive samples from G2 (109 NPW/A, 197 NPA, and 127 NPS). Initially, there were 58 samples considered false negative when tested with 3MRSV, including 16 DSFA-positive/culture-positive, 32 DSFA-positive/culture-negative, 1 DSFA-positive/culture-indeterminate, 4 DSFA-indeterminate/culture-positive, and 5 DSFA-negative/culture-positive samples. Initially, there were 36 DSFA-negative/culture-negative samples that were considered false positive when tested with 3MRSV.
TABLE 1.
3MRSV assay sensitivity, specificity, positive predictive value, and negative predictive value by subject group for all sample types combined
| Parametera | Value for subject groupb: |
|
|---|---|---|
| 1 | 2 | |
| No. tested | 1,306 | 1,140 |
| Sensitivity, % (95% CI) | 86.5 (83.0-89.3) | 87.3 (83.8-90.1) |
| Specificity, % (95% CI) | 95.8 (94.3-97.0) | 95.6 (93.8-96.9) |
| PPV, % | 91.4 | 92.4 |
| NPV, % | 93.2 | 92.5 |
| RSV prevalence, % | 34 | 38 |
3MRSV sensitivity, specificity, positive PPV, and NPV were determined by comparison to direct specimen antibody testing and/or viral culture.
Group 1, subjects of all ages; group 2, subjects <22 years old (age group FDA cleared for 3MRSV testing).
Performance of the 3MRSV by specimen type.
The performance measures of the 3MRSV by sample type for G1 (n = 1,306) and G2 (n = 1,140) are shown in Table 2. Sensitivities (range, 85.3% to 89.9%), specificities (range, 93.4% to 97.4%), PPVs (range, 90.6% to 94.1%), and NPVs (range, 90.9% to 95.2%) were consistent for all specimen types and across both age groups. The type of transport medium used did not affect the overall performance of the 3MRSV (data not shown).
Performance of the 3MRSV by testing site.
The performance measures of the 3MRSV by testing site for all sample types combined for subjects <22 years old (G2, n = 1,140) are shown in Table 3. The PPV and NPV calculations for each site were determined using the respective prevalence at each site. Overall, the sensitivities ranged from 76.9% to 92.9%, specificities from 85.4% to 98.7%, PPVs from 80.0% to 97.4%, and NPVs from 88.2% to 94.9%. The differences in sensitivities and specificities by site were not statistically different (P > 0.3).
Overall performance of the 3MRSV after discordant-result analysis.
There were 12 samples (2 NW/A, 8 NPA, and 2 NPS) with a positive reference test (DSFA) and a negative 3MRSV (originally considered false negatives) that were ProFlu+ negative and therefore true negatives (DSFA false positive). There were five samples (one NW/A, three NPA, and one NPS) with negative reference tests (DSFA and culture) and a positive 3MRSV (initially considered false positives) that were ProFlu+ positive and therefore true positives. The sensitivities and specificities of the 3MRSV marginally increased after discordance analysis for G1 (from 86.5% to 89.0% and from 95.8% to 96.4%, respectively) and for G2 (from 87.3% to 89.3% and from 95.6% to 96.3%, respectively).
Comparison of the 3MRSV to the BinaxNOW test.
There were 1,306 samples tested by both 3MRSV and BinaxNOW, of which 7 samples had invalid BinaxNOW results (0.54%) after repeat testing. These samples were excluded from the comparison between 3MRSV and BinaxNOW. Table 4 shows the performance of 3MRSV compared to BinaxNOW for subjects in G1, G2, and G3 (<5 years, the approved age group for testing with BinaxNOW; n = 953). For all sample types combined in the G3 age group, 3MRSV was more sensitive than BinaxNOW (P < 0.05); the specificities of 3MRSV and BinaxNOW were comparable (P = 0.26). When the G3 data were sorted by individual sample types, the sensitivities of 3MRSV were all significantly higher (P < 0.05) than the sensitivities of BinaxNOW. The specificities of G3 NW/A results for 3MRSV were significantly less (P < 0.05) than those of the BinaxNOW results. However, the specificity of 3MRSV testing of G3 NPA and NPS was comparable to that of BinaxNOW testing of NPA and NPS.
TABLE 4.
Comparisons of the sensitivities and specificities of the 3MRSV and BinaxNOW RSV by subject group and by sample type
| Groupa | Sample typeb | Sensitivity (%)c |
Specificity (%)d |
||
|---|---|---|---|---|---|
| 3MRSV | BinaxNOW | 3MRSV | BinaxNOW | ||
| 3 | NW/A | 89.7* | 75.7 | 93.2** | 100 |
| NPA | 85.6* | 72.2 | 96.2 | 96.2 | |
| NPS | 89.9* | 69.8 | 98.9 | 97.8 | |
| All | 87.9* | 72.4 | 96.5 | 97.6 | |
| 2 | NW/A | 89.9* | 76.2 | 93.3** | 100 |
| NPA | 85.0* | 73.1 | 95.0 | 96.0 | |
| NPS | 88.2* | 66.9 | 97.3 | 98.1 | |
| All | 87.2 | 72.0 | 95.6 | 97.6 | |
| 1 | NW/A | 89.1* | 75.5 | 93.9** | 100 |
| NPA | 85.1* | 72.8 | 95.2 | 95.9 | |
| NPS | 85.8* | 64.2 | 96.9 | 98.5 | |
| All | 86.3* | 70.8 | 95.8 | 97.8 | |
Group 1, subjects of all ages; group 2, subjects <22 years old (age group FDA cleared for 3MRSV testing); group 3, all subjects <5 years old (age group approved for use with the BinaxNOW RSV test).
NW/A, nasal wash/aspirate; NPA, nasopharyngeal aspirate; NPS nasopharyngeal swab.
*, 3MRSV was significantly more sensitive than BinaxNOW (P < 0.05).
**, 3MRSV was significantly less specific than BinaxNOW (P < 0.05).
It should be noted that BinaxNOW is not approved for testing of NPA or in subjects 5 years old or older; thus, the following G1 and G2 comparisons should be considered results generated with off-label use of the BinaxNOW test. As shown in Table 4, for all sample types combined and for each individual sample type, 3MRSV was significantly more sensitive (P < 0.05) for the detection of RSV in G1 and G2 samples than was BinaxNOW. The overall G1 and G2 3MRSV specificities were less than those for BinaxNOW, but the differences were not statistically significant (G1, P = 0.15; G2, P = 0.31). The specificities of NW/A results for 3MRSV were significantly less (P < 0.05) than those for BinaxNOW. However, the specificities of 3MRSV testing of NPA and NPS were comparable to those of BinaxNOW testing of NPA and NPS.
External control performance.
Overall, results were available for 249 positive and 249 negative controls. All negative controls were 3MRSV negative, and 247/249 (99.2%) of the positive controls were 3MRSV positive. Seventeen controls (3.6%) resulted in sample error 2, which related to a failure to detect sample in the cartridge. Sixteen of the 17 controls were processed at two sites using the same lot of M4 medium.
DISCUSSION
The 3MRSV demonstrated overall good performance for the detection of RSV in all three samples types (NPW/A, NPA, and NPS) collected from symptomatic subjects of all ages combined (86.5% sensitivity and 95.8% specificity). Both the sensitivity and specificity of 3MRSV increased when samples with discordant reference test results or 3MRSV-positive and reference test-negative results were tested with the ProFlu+ RT-PCR assay; however, because not all samples in the study were tested with ProFlu+, the results of the discordance analysis were not used to calculate the sensitivity and specificity of the assay as noted in the package insert. The performance of the 3MRSV in this study was consistent with numerous studies of other rapid RSV antigen tests that demonstrated that the sensitivities of the tests ranged from 59% to 89% and the specificities ranged from 93% to 100% (11). It should be noted that the trial was conducted during the months of high RSV prevalence (13), when RSV rapid antigen detection tests demonstrate optimal performance. During periods of lower RSV prevalence, the PPV of RSV rapid antigen tests would generally decline.
NPS and NPA sample types met the FDA performance requirements of lower confidence intervals (CIs) for sensitivity and specificity (>60% and >90%, respectively). These requirements were also met in the age stratification analysis. The NW/A sample type met the FDA guidance for performance requirements of lower confidence intervals for sensitivity but was 1 to 3% (depending on what age groups were included) below FDA guidance for performance requirements of lower confidence intervals for specificity. In addition, there were too few RSV-positive samples (n = 10) identified from subjects ≥22 years old; therefore, the test has been FDA cleared for use in symptomatic patients <22 years of age.
Although all sample types performed well in this study, caution should be used when testing highly mucoid samples. These samples required a dilution step more often than nonmucoid samples, and they also had the highest rate of invalid results. Of the three sample types, NPA had the highest rates of mucoid samples and invalid results. For the samples with valid results after dilution, the dilution step did not significantly affect the performance of the test, as the concordance with the reference results was 96.4%. However, an invalid 3MRSV could result in a missed RSV diagnosis. In addition, invalid results that require repeat testing can also increase the overall cost per test. The costs per 3M test (instrumentation, reagents, and technical time) and BinaxNOW test (reagents and technical time) range from approximately $13.00 to $16.00 and $10.50 to $13.50, respectively, depending on the testing volume. The repeat test rates, due to invalid results, for 3MRSV and BinaxNOW would add averages of approximately $1.16 and $0.25 per test, respectively. The need to perform repeat testing for 3MRSV and BinaxNOW would result in a comparable delay in result reporting (17 to 18 min versus 16 min).
Blood present in the samples did not adversely affect the performance of the 3MRSV. The specificity of the test was not affected by the transport medium type, whereas it has been noted that M4 medium may affect the specificity of the 3M Influenza A+B test. Although there were 16 sample error 2 control failures when M4 medium was used to process the control swabs, further control testing with the same lot of M4 medium yielded all valid test results. Patient samples collected in M4 medium did not give any more invalid results than observed with other transport media.
The sensitivity of the 3MRSV was significantly better (P < 0.0001) than that of the BinaxNOW RSV test for all age groups, including the age group (<5 years) approved by the FDA for testing with BinaxNOW. Similar results were found in a study by Borek et al. which compared BinaxNOW RSV results to those of DSFA and/or tissue culture in nasopharyngeal aspirate and wash samples from children and adults (1). The sensitivity and specificity of BinaxNOW were 74% and 100%, respectively. A study by Cruz et al. that compared BinaxNOW to viral culture for 14,756 pediatric respiratory specimens demonstrated a sensitivity of 81% and a specificity of 93.2% for BinaxNOW (3).
The observation of enhanced sensitivity of the 3MRSV over the BinaxNOW test was consistent with a previous study that demonstrated that the 3M Rapid Detection Influenza A+B test (3MA+B) was significantly (P < 0.001) more sensitive than the BinaxNOW influenza A&B test for the detection of both influenza A and B viruses (6). The significantly better sensitivities of the 3MRSV and 3MA+B compared to the BinaxNOW tests may be due to the enhanced detection format and more consistent interpretation of results. False-positive results were found with both the BinaxNOW and 3MRSV, as noted with the 3MA+B. The specificities of 3MRSV for NPA and NPS were comparable to those of BinaxNOW, but for NW/A the specificity of 3MRSV was significantly less than that of the BinaxNOW RSV test.
One of the major benefits of the 3MRSV was the use of instrumentation to read and interpret test results. Generally, rapid antigen tests require little technical expertise; however, the result relies on user interpretation that can be subjective and can lead to errors. Automation of these processes should reduce the risk of user misinterpretation. Finally, the rapid detection reader provides a variety of printouts, such as instrument function checks, control results, lot numbers, reagent expiration dates, and patient results, that can be used to document quality control and help laboratories meet their various quality assurance obligations.
Acknowledgments
This study was funded by 3M Company (St. Paul, MN), and the data were used for U.S. FDA submission of the 3MRSV as an in vitro diagnostic device.
We sincerely thank Dan J. Morse, Clinical Research 3M Center, for statistical analysis. We also thank the trial site technologists and coordinators who participated in this study.
Footnotes
Published ahead of print on 12 May 2010.
REFERENCES
- 1.Borek, A. P., S. H. Clemens, V. K. Gaskins, D. Z. Aird, and A. Valsamakis. 2006. Respiratory syncytial virus detection by Remel Xpect, Binax Now RSV, direct immunofluorescent staining, and tissue culture. J. Clin. Microbiol. 44:1105-1107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Casiano-Colon, A. E., B. B. Hulbert, T. K. Mayer, E. E. Walsh, and A. R. Falsey. 2003. Lack of sensitivity of rapid antigen tests for the diagnosis of respiratory syncytial virus infection in adults. J. Clin. Virol. 28:169-174. [DOI] [PubMed] [Google Scholar]
- 3.Cruz, A. T., A. C. Cazacu, J. M. Greer, and G. J. Demmler. 2007. Performance of a rapid assay (BinaxNOW) for detection of respiratory syncytial virus at a children's hospital over a 3-year period. J. Clin. Microbiol. 45:1993-1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Englund, J. A., P. A. Piedra, A. Jewell, K. Patel, B. B. Baxter, and E. Whimbey. 1996. Rapid diagnosis of respiratory syncytial virus infections in immunocompromised adults. J. Clin. Microbiol. 34:1649-1653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Falsey, A. R., C. K. Cunningham, W. H. Barker, R. W. Kouides, J. B. Yuen, M. Menegus, L. B. Weiner, C. A. Bonville, and R. F. Betts. 1995. Respiratory syncytial virus and influenza A infections in the hospitalized elderly. J. Infect. Dis. 172:389-394. [DOI] [PubMed] [Google Scholar]
- 6.Ginocchio, C. C., M. Lotlikar, L. Falk, S. Arora, M. Kowerska, M. Bornfreund, and R. Manji. 2009. Clinical performance of the 3M Rapid Detection Flu A+B Test compared to R-Mix culture, DFA and BinaxNOW Influenza A&B Test. J. Clin. Virol. 45:146-149. [DOI] [PubMed] [Google Scholar]
- 7.Hall, C. B., R. G. Douglas, Jr., and J. M. Geiman. 1975. Quantitative shedding patterns of respiratory syncytial virus in infants. J. Infect. Dis. 132:151-156. [DOI] [PubMed] [Google Scholar]
- 8.Hall, C. B. 2001. Respiratory syncytial virus and parainfluenza virus. N. Engl. J. Med. 344:1917-1928. [DOI] [PubMed] [Google Scholar]
- 9.Hall, C. B., G. A. Weinberg, M. K. Iwane, A. K. Blumkin, K. M. Edwards, M. A. Staat, P. Auinger, M. R. Griffin, K. A. Poehling, D. Erdman, C. G. Grijalva, Y. Zhu, and P. Szilagyi. 2009. The burden of respiratory syncytial virus infection in young children. N. Engl. J. Med. 360:588-598. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Henrickson, K. J. 2007. Diagnostic assays for respiratory syncytial virus. Pediatr. Infect. Dis. J. 26:S36-S40. [DOI] [PubMed] [Google Scholar]
- 11.Leland, D., and C. C. Ginocchio. 2007. Role of cell culture for virus detection in the age of technology. Clin. Microbiol. Rev. 20:49-78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.McAdam, A. J. 2000. Discrepant analysis: how can we test a test? J. Clin. Microbiol. 38:2027-2029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Panozzo, C. A., A. L. Fowlkes, and L. J. Anderson. 2007. Variation in timing of respiratory syncytial virus outbreaks: lessons from national surveillance. Pediatr. Infect. Dis. J. 26:S41-S45. [DOI] [PubMed] [Google Scholar]
- 14.Shay, D. K., R. C. Holman, R. D. Newman, L. L. Liu, J. W. Stout, and L. J. Anderson. 1999. Bronchiolitis-associated hospitalizations among US children, 1980-1996. JAMA 282:1449-1446. [DOI] [PubMed] [Google Scholar]
- 15.Stempel, H. E., E. T. Martin, J. Kuypers, J. A. Englund, and D. M. Zerr. 2009. Multiple viral respiratory pathogens in children with bronchiolitis. Acta Paediatr. 98:123-126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Thompson, W. W., D. K. Shay, E. Weintraub, L. Brammer, N. Cox, L. J. Anderson, and K. Fukuda. 2003. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 289:179-186. [DOI] [PubMed] [Google Scholar]