Evaluation of Three Immunoassay Kits for Rapid Detection of Influenza Virus A and B

Abstract

Influenza causes high morbidity and mortality in very young and elderly individuals, which can be controlled with antivirals and/or vaccines. The success of therapeutic measures is predicated on the rapid and precise diagnosis of the infection. We compared three rapid influenza immunoassay (RIIA) kits for the diagnosis of influenza virus A and B using 178 respiratory specimens submitted for routine testing. BD Directigen Flu A+B (Directigen), Directigen EZ Flu A+B (EZ), and NOW Flu A NOW Flu B (NOW; Binax) tests had comparable combined influenza virus A and B specificities, varying from 94 to 98%. In contrast, the sensitivity of EZ was significantly lower (39%) than that of NOW (76%) and marginally lower than that of Directigen (56%). The differences in sensitivity were most evident in patients who were >9 years old (Directigen, 53%; EZ, 32%; and NOW, 69%). Among specimens, bronchoalveolar lavage fluids yielded the most discrepant results, with sensitivities varying from 0 (EZ) to 100% (NOW), followed by nasopharyngeal swabs (sensitivities of 27 to 100%) and nasal washes (50 to 81%). The Directigen kit format allowed for faster completion but more cumbersome performance and more difficult interpretation compared with the other two kits. Overall, NOW provided the most accurate diagnoses and had user-friendly technical characteristics. However, the low overall sensitivity of the RIIAs indicates that these can be used as screening tools only.

Influenza viruses cause seasonal epidemics associated with high morbidity and mortality, particularly in the extreme age groups (3, 9, 11, 12). In recent years, emphasis has been placed on developing strategies to combat influenza. Algorithms have been devised to limit the spread of the virus to high-risk individuals by combining rapid diagnosis with antiviral- or vaccine-based prophylactic treatment. In addition, early therapeutic interventions are recommended for high-risk patients in order to decrease the morbidity and mortality caused by influenza (5, 9). The effectiveness of these strategies, however, is predicated on rapid, sensitive, and specific diagnosis of the infection (4).

Present methods of diagnosing viral respiratory infections include tissue culture isolation (TC); rapid shell vial detection (SV); direct antigen detection using enzyme, fluorescence, optical, or chromatographic immunoassays; and PCR (10, 14). The rapid influenza immunoassays (RIIA) provide a result in 15 to 30 min. They are frequently used at the point of care or as a screening method in full-menu virology laboratories because they allow the institution of therapeutic or prophylactic measures with minimal delays. The goal of this study was to compare three RIIA kits marketed for combined detection of influenza virus A and B.

MATERIALS AND METHODS

Study design.

This was a prospective study conducted with Institutional Review Board approval from January through March 2003, during peak influenza activity. Respiratory specimens were collected at clinical sites according to specific instructions (http://clinlab.uch.edu) and mixed with 3 ml of M4 transport medium (Microtest, Lenexa, Kans.) when indicated. Specimens were transported and stored at refrigerator temperature until tested. The presence of influenza antigens was investigated using the BD Directigen Flu A+B (Directigen; Becton-Dickinson, Sparks, Md.), Directigen EZ Flu A+B (EZ; Becton-Dickinson), and NOW Flu A NOW Flu B (NOW; Binax, Portland, Maine) kits. However, due to gaps in reagent availability, not all specimens were tested by all three methods. In addition, all specimens were tested by rapid respiratory viral cultures, which included shell vials and conventional tube tissue culture. PCR was performed as a tiebreaker on specimens that had a positive RIIA result by a single method. A true-positive specimen was defined by a positive result obtained by rapid respiratory culture, by two or more antigen detection methods, or by a single antigen detection method confirmed by PCR.

Directigen uses a ColorPAC device with a flow controller shared by the influenza A and B wells. The specimen is mixed with extraction reagent E (proprietary) and filtered before being added to the A and B test wells. After the flow controller is removed, the wells are allowed to completely absorb four other proprietary reagents added sequentially, with 2- and 5-min incubations before the third and last reagents, respectively. A positive influenza A or B specimen generates a purple triangle against a light yellow to light purple background. A valid negative result is indicated by a purple control dot against the light yellow to light purple background.

The EZ test has separate influenza A and B devices. Specimens are mixed with an extraction reagent and filtered before being dispensed onto the A and B wells. After a 15-min incubation, influenza A- or B-containing specimens will result in a visible reddish-purple line against a white to light pink background in addition to the positive control line in the appropriate well. A valid negative result is indicated by the absence of the test line with a visible control line.

NOW has separate influenza A and B devices. Specimen is added directly to each apparatus. After a 15-min incubation, a positive influenza A or B specimen generates a pink to purple sample line in addition to the control line. A valid negative influenza A or B result is represented by the presence of the control line only.

Respiratory TC was performed by inoculating 0.3 ml of specimen into each of six tissue culture tubes: two rhesus monkey kidney tubes, each from a different vendor (Diagnostic Hybrids Inc., Athens, Ohio, and Viromed, Minnetonka, Minn.), and one Hep-2 tube (American Type Culture Collection), one A549 tube (American Type Culture Collection), and two primary human embryonic lung fibroblast tubes prepared in our laboratory. Tubes were incubated at 37°C for up to 28 days in maintenance medium consisting of Eagle's medium (Diagnostic Hybrids Inc.) with penicillin, streptomycin, amphotericin B, and 2% fetal bovine serum (Sigma, St. Louis, Mo.), except for rhesus monkey kidney tubes, which were maintained in fetal bovine serum-free medium. One human embryonic lung fibroblast tube was incubated at 33°C on a roller drum to enhance rhinovirus isolation. Medium was changed at 24 to 48 h after inoculation, after each of the weekly guinea pig red blood cell hemadsorptions, and as dictated by the appearance of the monolayer. Tubes were examined by light microscopy daily during the first week after inoculation and thrice weekly thereafter for up to 28 days. Monolayers that showed cytopathic effect or positive hemadsorption were scraped and stained with specific monoclonal antibodies. The procedure entailed spotting cells onto slides, followed by acetone fixation and staining with type-specific monoclonal antibodies (Dako, Carpenteria, Calif.). Slides were read with a fluorescence microscope. A positive result was defined as the presence of bright green fluorescence in the cytoplasm of two or more cells.

SV cultures.

R-Mix SVs (Diagnostic Hybrids Inc.) were activated by incubation at 37°C for 2 to 4 h prior to inoculation. Growth medium was replaced by Refeed medium (Diagnostic Hybrids Inc.) immediately prior to infection, after which cultures were inoculated with 0.2 ml of the clinical specimen. Thereafter, R-Mix SVs were centrifuged at 700 × g for 60 min and incubated at 37°C. Coverslips harvested at 48 h were stained as described above.

PCR.

RNA, extracted from clinical specimens with a QIAGEN RNA extraction kit (QIAGEN, Valencia, Calif.), was reverse transcribed and amplified during 40 cycles using the one-step reverse transcription-PCR kit (QIAGEN) and the following biotinylated primer pairs: for influenza A, 5′-AAGGGCTTTCACCGAAGAGG and 5′-CCCATTCTCATACTGCTTC, and for influenza B, 5′-ATGGCCATCGGATCCTCAAC and 5′-TGTCAGCTATTATGGAGCTG (7). The amplicon was allowed to bind to capture probes 5′-GTCCTCGGAGGACTTGAATGGAATGAT and 5′-GTCAAGAGCACCGATTATCAC for influenza A and influenza B, respectively (7) (0.5 μg/ml in 1 M ammonium acetate), previously immobilized onto 96-well plates. Bound amplicon was detected with streptavidin-horseradish peroxidase (R&D Systems, Minneapolis, Minn.) and tetramethylbenzidine colorimetric substrate (R&D Systems). The optical density was measured at 450 nm using an enzyme immunoassay (EIA) reader. A sample with an optical density of ≥0.6 was considered positive for the presence of the nucleic acid of interest. This test has an analytic sensitivity of ≥1 50% tissue culture infective dose U/ml. Positive and negative controls, consisting of viral isolates and water, respectively, were included in each run. The runs were considered valid if the absorbance of the negative control was <0.3 and that of the positive control was >1.0.

Statistical analysis.

The sensitivity and specificity of each kit were calculated and compared by the chi-square test and the Fisher exact test using Instat3 software (GraphPad Software, San Diego, Calif.).

RESULTS

Diagnostic accuracy of influenza A and B rapid tests.

The study was conducted on 178 specimens, including nasal wash (NW) fluids, sputa, bronchoalveolar lavage (BAL) fluids, tracheal aspirates, and nasopharyngeal swabs (NPS). The ages of patients contributing specimens varied from <1 to 92 years, including 28 individuals who were ≤8 years old and 148 individuals who were ≥12 years old (and 2 individuals of unknown ages). The gender distribution showed 83 females, 92 males, and 3 cases in which the gender was not recorded.

There were 30 influenza A and 7 influenza B specimens that met true-positive criteria (Table 1). Specificity varied from 94% for EZ (95% confidence interval [CI] of 89 to 98%) to 98% for Directigen (95% CI of 91 to 100%) and was not significantly different across tests. There were seven specimens that were tested by PCR, since they were positive by only one antigen detection method. All seven specimens had negative PCR results, confirming the TC results. Combined influenza A and B sensitivities of Directigen, EZ, and NOW were 56% (95% CI of 35 to 76%), 39% (95% CI of 20 to 61%), and 76% (95% CI of 50 to 93%), respectively. There was a marginal difference in sensitivity when all three tests were analyzed together (P = 0.06). Individual comparisons showed that NOW had significantly higher sensitivity than EZ (P = 0.03). The corresponding positive predictive values (PPV) and negative predictive values (NPV) were as follows: for Directigen, 93% (95% CI, 68 to 100%) and 85% (95% CI, 74 to 92%), respectively; for EZ, 56% (95% CI, 30 to 80%) and 89% (95% CI, 83 to 94%), respectively; and for NOW, 93% (95% CI, 66 to 100%) and 81% (95% CI, 58 to 95%), respectively. NPV were not significantly different among kits, but the PPV was lower for EZ than for either of the other two tests (P = 0.01).

TABLE 1.

Diagnostic characteristics of influenza A and B rapid immunoassays

Assay	Influenza Aa		Influenza B		Influenza A or B
	Sensitivity (%)	Specificity (%)	Sensitivity (%)	Specificity (%)	% Sensitivity (95% CI)	% Specificity (95% CI)
Directigen	12/18 (66)	67/68 (99)	2/7 (29)	79/79 (100)	56 (35-76)	98 (91-100)
EZ	7/17 (41)	125/129 (97)	2/6 (33)	137/140 (98)	39 (20-61)	94 (89-98)
NOW	11/15 (73)	19/20 (95)	2/2 (100)	33/33 (100)	76 (50-93)	94 (73-100)

^aSensitivity values indicate the number of positive results out of the number of true-positive specimens tested, and specificity values indicate the number of negative results out of the number of true-negative specimens tested.

Specimen type and patient age as determinants of influenza rapid test diagnostic characteristics.

To determine the role of respiratory specimen type in the rapid test results, we analyzed the sensitivity and specificity of each kit for the three most common specimens received by the Virology Laboratory (Table 2). There was wide variability in the ability to detect influenza viruses in BAL fluids (0 to 100% sensitivity). EZ had significantly lower sensitivity than the other two kits (P = 0.02). For NPS and NW fluids, sensitivities varied from 27 to 100%, but differences were not statistically significant (P values of 0.20 and 0.34, respectively).

TABLE 2.

Specimen-specific sensitivity of influenza A and B rapid immunoassays

Assay	No. of positive results/no. of true-positive specimens testeda
	BAL (n = 64)	NPS (n = 31)	NW (n = 75)
Directigen	2/3 (67)	3/11 (27)	9/11 (81)
EZ	0/5 (0)	3/8 (38)	6/9 (67)
NOW	2/2 (100)	7/7 (100)	4/8 (50)

^aParentheses contain percentages.

To assess differences in test characteristics by age, patients were grouped according to their ages as younger or older than 9 years. This threshold was selected because it is commonly assumed that by the age of 9 most individuals have been infected by influenza (9). Hence, our analysis was targeted to differentially evaluate the performances of the three kits for a group that had many primary influenza episodes (individuals who were <9 years old) and for another group of subjects undergoing reinfection. There were 28 specimens from patients who were <9 years old, including 6 influenza true-positive samples, and 148 specimens from patients who were >9 years old, including 30 specimens that were positive for influenza (1 of the influenza-positive samples was from an individual whose age had not been recorded). The age distribution was balanced across the three kits. Sensitivities of Directigen, EZ, and NOW for the younger group (<9 years old) were 71, 75, and 100%, respectively. For the older group, the corresponding numbers were 53, 32, and 69%, respectively. Although there was a trend toward lower sensitivity in the older age group for all assays, these differences were only marginally significant (P = 0.1). The sensitivities of all three tests were not statistically different for the younger group, but the comparison for the older group showed a marginal difference across all tests (P = 0.1) and a significantly higher sensitivity of the NOW test when directly compared with the EZ test (P = 0.04).

Specificities of Directigen, EZ, and NOW in the younger age group were 100, 94, and 100%, respectively, whereas in the older age group they were 98, 94, and 94%, respectively. These data indicate that age did not significantly impact the specificities of the assays.

Technical characteristics of influenza A and B rapid tests.

To determine differences in technical challenge and labor demands of each test, we compared them with respect to total duration of the test, technical complexity, hands-on time, ease of interpretation of results, specimen requirements, and cost (Table 3). Directigen was the most rapid (8 min) but had the most challenging readout and the highest cost of the three kits. EZ and NOW required similar lengths of time and technical skills. EZ was the most economical test when both labor and reagents were factored into the analysis.

TABLE 3.

Comparison of technical and labor characteristics of influenza A and B EIA

Characteristic	Directigen	EZ	NOW
Duration of test	8 min	16 min	15.5 min
Hands-on time	1 min	1 min	0.5 min
Discrimination of results	Intermediate	Easy	Easy
Specimen restrictions	None	None	None
Cost	10 unitsa	6.5 units	8.5 units

^aUnits were arbitrarily chosen to provide a common denominator.

DISCUSSION

We found that three influenza virus A and B rapid diagnosis immunoassay kits, Directigen, EZ, and NOW, differed with respect to sensitivity but not to specificity. NOW had the highest sensitivity in this study (76%), whereas Directigen and EZ had sensitivities of 56 and 39%, respectively. A minority of specimens only was tested by all three kits, which could have potentially introduced a bias. However, the positive specimens were evenly distributed across the three kits, which allowed us to draw meaningful conclusions with respect to the sensitivity of the kits. Previous reports in the medical literature (2, 6, 8, 13) have typically shown higher sensitivities of EIA-based methods of influenza detection than the ones reported here, but these other studies used samples from pediatric patients only. Our results are in accordance with previous studies performed in adults (13).

The type of specimen influenced the sensitivity of each assay. We were particularly interested in the performance of these rapid tests on BAL fluids because they are frequently obtained from the sickest patients and immunocompromised hosts. Previous studies that included BAL fluids among specimens used to investigate the diagnostic accuracy of influenza rapid detection kits (6) did not provide a breakdown of sensitivity by specimen type. In this study, BAL fluids were associated with decreased sensitivity when tested by EZ. The number of influenza-containing BAL specimens was too small to draw definitive conclusions with respect to the other kits. The sensitivity of rapid antigen detection tests with BAL fluids deserves further investigation.

Age is a known determinant of influenza viral shedding and, therefore, the amount of antigen available for rapid detection. Children typically have less immunity to influenza than adults and tend to excrete large amounts of virus. A test with marginal sensitivity for influenza would be expected to show significant differences between children and adults. In this study, we stratified the subjects as younger or older than 9 years. This is the age when the recommendations for influenza vaccination change from two doses to a single dose based on the assumption that by this age most individuals would have already been infected by the wild-type virus (9). The data showed a trend toward higher sensitivity in the younger group across all tests. However, the number of specimens from children who were <9 years old was much smaller than the number from the older group (28 versus 148), which limited our ability to reach statistical significance. There were no significant differences among the three assays with respect to sensitivity in the younger age group, but NOW was superior to EZ in the older group. Due to the small sample size of this study, we could not study the interaction between age and specimen type on the sensitivity of each kit.

The rapid detection kits have streamlined execution formats conducive to point-of-care use. The Directigen assay, which was most cumbersome to perform and difficult to interpret, has the potential to provide results twice as fast as the other kits, although the 7.5- and 8-min differences for EZ and NOW assays, respectively, do not seem clinically relevant. Cost probably varies with discounts obtained by specific laboratories. At our institution, EZ was the most economical test.

A limitation of this study was the unequal number of specimens tested by the different methods. NOW, in particular, was tested on only 35 specimens. However, the number of positive samples tested by NOW, 17, was not significantly different from the number of positive specimens used in the evaluation of the Directigen and EZ tests (25 and 23, respectively).

Rapid tests constitute an important screening tool for management of influenza on individual and community levels. The high PPV of these tests allows early interventions that may limit the spread of the virus in high-risk populations (1). Conversely, the NPV of these tests is not always optimal, and negative results need to be backed up by culture or PCR whenever they play a critical role for clinical management.