Comparison of the MChip to Viral Culture, Reverse Transcription-PCR, and the QuickVue Influenza A+B Test for Rapid Diagnosis of Influenza

Abstract

The performance of a diagnostic microarray (the MChip assay) for influenza was compared in a blind study to that of viral culture, reverse transcription (RT)-PCR, and the QuickVue Influenza A+B test. The patient sample data set was composed of 102 respiratory secretion specimens collected between 29 December 2005 and 2 February 2006 at Scott & White Hospital and Clinic in Temple, Texas. Samples were collected from a wide range of age groups by using direct collection, nasal/nasopharyngeal swabs, or nasopharyngeal aspiration. Viral culture and the QuickVue assay were performed at the Texas site at the time of collection. Aliquots for each sample, identified only by study numbers, were provided to the University of Colorado and Vanderbilt University teams for blinded analysis. When referenced to viral culture, the MChip exhibited a clinical sensitivity of 98% and a clinical specificity of 98%. When referenced to RT-PCR, the MChip assay exhibited a clinical sensitivity of 92% and a clinical specificity of 98%. While the MChip assay currently requires 7 to 8 h to complete the analysis, a significant advantage of the test for influenza virus-positive samples is simultaneous detection and full subtype identification for the two subtypes currently circulating in humans (A/H3N2 and A/H1N1) and avian (A/H5N1) viruses.

Infections caused by the influenza virus have a significant influence on modern society. The loss of productivity due to illness has an enormous economic impact, and there are about 36,000 deaths in the United States caused by influenza each year (4). While there are antiviral therapies available to treat influenza, zanamivir and oseltamivir have shown clinical benefits only when administered within 36 to 48 h of the appearance of symptoms (24, 34). Another class of drugs, adamantanes, is no longer considered effective, due to widespread viral resistance (31). In order to ensure the appropriate use of antivirals and antibiotics, rapid diagnosis of an influenza virus infection is required. While cell culture isolation is the “gold standard” for the identification of influenza viruses, it is not routinely used in the outpatient setting, due to a 2- to 14-day turnaround time for final results (7, 13). Although shell vial culture techniques are more rapid, they still require 2 to 3 days (3, 22). Recently, reverse transcription PCR (RT-PCR) has emerged as a useful technique for molecular diagnosis of influenza in a clinical lab setting because it has excellent sensitivity and can provide results within a few hours (1, 32, 35). In addition, the following rapid (<30 min) immunoassay tests are available for point-of-care diagnosis of influenza: Directigen Flu A and Directigen Flu A+B (Becton Dickinson Diagnostic Systems, Sparks, MD), Flu OIA (Bio Star, Inc., Boulder, CO), ZstatFlu (ZymeTx, Inc., Oklahoma City, OK), the BinaxNOW influenza A and B test, and the QuickVue Influenza A+B test (Quidel Corporation, San Diego, CA). Most of these rapid point-of-care tests have been compared against viral culture or RT-PCR (8, 12, 18, 20, 27-29). For these rapid tests, the reported sensitivities range from 75% to 94% and the reported specificities range from 86% to 100%. The positive and negative predictive values (PPV and NPV) range from 74% to 100% and 87% to 99%, respectively.

Recently, a novel DNA microarray (the MChip assay) and an associated assay for rapid (7 h) detection and subtype identification of influenza viruses were described (5, 6). Several studies have investigated the utility of diagnostic microarrays for influenza virus detection and subtyping (17, 19, 23, 30, 33), but all have relied upon a multiple-gene approach including targets against the matrix (M), hemagglutinin, and neuraminidase genes. Due to coevolution of the matrix proteins and the antigenic surface proteins hemagglutinin and neuraminidase (11, 26), MChip was designed with capture and label sequences that target only the M gene segment and are able to provide virus type and subtype information for a wide range of influenza viruses. The single-gene approach offers significant analytical advantages over multiplex PCR, including improved sensitivity, simplicity, and reliability (21). In a blinded study of 39 human patient samples and 14 negative control samples, the MChip assay exhibited a clinical sensitivity of 95% and a clinical specificity of 92% compared to results provided by either the CDC or the Colorado Department of Public Health (6). In a separate study, MChip was evaluated for its ability to subtype the rapidly evolving and highly pathogenic A/H5N1 viruses (5). In that study, 24 A/H5N1 viruses were correctly identified, with no false positives.

The blinded study summarized here was designed to evaluate the reliability of the MChip assay as a simple diagnostic test to identify influenza A virus in patient samples. The performance of the MChip assay was compared to viral culture with R-Mix shell vials, RT-PCR, and the QuickVue Influenza A+B test.

MATERIALS AND METHODS

Clinical specimens.

A total of 102 respiratory specimens were collected during a prospective clinical trial conducted from 29 December 2005 through 2 February 2006 at Scott & White Hospital and Clinic in Temple, Texas. The study was approved by the Scott & White Hospital Institutional Review Board. Patients presenting to the pediatric clinic, family medicine clinic, and the emergency department were eligible if they had an acute respiratory illness with one or more of the following symptoms: cough, congestion, coryza, and wheezing. The presence or absence of fever was also recorded, although it did not affect eligibility. Research personnel screened patients for eligibility and obtained informed consent prior to their enrollment and participation in the study. Involvement in the study was limited to the acute-care visit and consisted of a brief interview to obtain subject demographic characteristics and illness data and collection of a respiratory secretion specimen.

Samples were collected from a wide range of age groups by using direct collection, nasal/nasopharyngeal swabs, or nasopharyngeal aspiration. Nasopharyngeal aspiration was used primarily for children under two years of age, whereas swab collection methods were routinely used for patients two years and older. A subset of patients who were able to blow their nose also provided a sample of nasal mucus through a direct-collection method. Individual sample specimens were split into three aliquots and were processed immediately after collection and grouped according to the following testing, medium, and storage conditions. One aliquot was used for rapid diagnostic testing for influenza performed by onsite research personnel within 15 min of specimen collection using the QuickVue Influenza A+B test (Quidel Corporation, San Diego, CA) according to directions in the manufacturer's package insert. A second aliquot was used for viral culture testing and was placed in 2 ml of Universal Transport Medium (Copan Diagnostics, Inc., Corona, CA), which is a proprietary medium that contains salts, pH indicator, protein stabilizers, and antibacterial and antifungal agents and is intended for the efficient storage and transport of respiratory viral specimens. Culture specimens were maintained at 2 to 8°C for no longer than 8 h before the initiation of culture processing by the Scott & White Hospital virology laboratory. The specimens were batch processed once or twice daily using R-Mix shell vials (Diagnostic Hybrids, Inc., Columbus, OH) as previously described by Fader (10). The remaining third aliquot was placed in 1 ml of Universal Transport Medium and stored at −80°C until shipment to the two research laboratories (University of Colorado and Vanderbilt University) for the MChip microarray testing and PCR experiments. In preparation for shipment, specimens were rapidly thawed and 0.3 ml of sample was transferred into each of two separate transport tubes and quickly refrozen until subsequent shipment overnight on dry ice to the respective investigators. The remaining volume of specimen was refrozen at −80°C for archival.

In order to ensure a suitable distribution of positive and negative samples for challenging the MChip performance, the 102-specimen panel for the MChip and RT-PCR comparison was selected by an investigator who was not involved in any of the testing but who was informed of the QuickVue and culture results prior to sample selection and shipment. This allowed each study site's group of investigators and personnel to be blinded to the other site's test results.

RT-PCR testing.

All RT-PCR experiments were performed at the Vanderbilt University site. Original specimens were received on dry ice and stored at −80°C until processing. Samples were thawed, and 200 μl of specimen was added to 300 μl of lysis buffer (MagNApure Total nucleic acid extraction kit; Roche Applied Science, Indianapolis, IN). Samples were extracted in batches of 32 according to the manufacturer's external lysis protocol. Extracted nucleic acid was eluted in 100 μl of elution buffer and immediately processed for quantitative RT-PCR. The remaining nucleic acid was stored at −80°C. Real-time RT-PCR for the detection and separate subtyping of influenza A virus was performed with the Smart Cycler II (Cepheid, Sunnyvale, CA). The primer and probe sequences were developed and kindly provided by Steve Lindstrom of the Centers for Disease Control and Prevention, Atlanta, GA (15). The probes were labeled at the 5′ end with 6-carboxyfluorescein and at the 3′ end with the nonfluorescent quencher Blackhole Quencher 1 (Operon Biotechnologies, Inc., Huntsville, AL). Reactions were performed using the QuantiTect Probe RT-PCR kit (QIAGEN, Valencia, CA) according to the manufacturer's instructions. Extensive optimization was carried out to determine the optimal annealing temperatures, cycle times, and primer/probe concentrations (data not shown). The influenza A virus assay targeted the M gene, and the H1/H3 subtyping assays targeted the hemagglutinin gene. Final concentrations of primers and probe in the reaction mix were 1 μm and 0.3 μm, respectively. Cycling conditions were reverse transcription for 30 min at 50°C, HotStarTaq polymerase activation for 15 min at 95°C, and 45 cycles of denaturation for 8 s at 94°C and annealing/extension for 60 s at 60°C. Fluorescent data were collected during the annealing/extension step.

In order to determine the influenza A virus assay sensitivity, the amplified matrix target gene from a clinical isolate collected at Vanderbilt University in January 2005 was TA cloned into a plasmid vector (pGEM-T Easy; Promega, Madison, WI) and both strands of the plasmid sequenced to confirm the insert. Plasmid DNA was digested with HincII, and in vitro transcription by T7 RNA polymerase was used to generate RNA runoff transcripts. RNA transcripts were purified with on-column DNase treatment (RNeasy; QIAGEN, Valencia, CA) and quantitated by spectrophotometry. Serial 10-fold dilutions of the in vitro RNA transcripts were used to determine the linear performance and sensitivity of the real-time RT-PCR assay. The assay performance was linear and reproducible over a wide range of dilutions and was capable of detecting a minimum of 88 RNA copies/reaction (data not shown).

Samples were assayed in batches of 32. Each run included no-template controls and influenza A virus-positive RNA controls. The cycle threshold was defined by the Smart Cycler software as the cycle at which the fluorescent signal crossed 10 times the background. Specimens were considered positive if the cycle threshold was less than 40 cycles, and all positive samples were confirmed by repeat assay. All specimens that tested positive by the influenza A virus real-time RT-PCR assay were subsequently tested by the H1 and H3 real-time RT-PCR subtyping assays.

MChip testing.

All MChip experiments were performed at the University of Colorado as described previously (5, 6). Briefly, RNA was extracted from the specimens by using the QIAamp viral RNA mini kit according to the manufacturer's instructions (QIAGEN, Valencia, CA). Extracted RNA was amplified by RT-PCR followed by runoff transcription using a T7 polymerase. Amplified RNA was fragmented by base-catalyzed hydrolysis according to the method of Mehlmann et al. (23) and hybridized to a microarray slide containing 15 capture and label sequences (6) specific to the M gene segment of influenza A virus. Relative fluorescence intensities were used as input variables for a neural net analysis as described in detail by Dawson et al. (6). The training set for the neural net consisted of 158 virus samples (57 H3N2 virus samples, 23 H1N1 virus samples, and 78 H5N1 virus samples) and 39 negative control samples. The trained neural network was used to identify influenza viruses, including the subtype, for the 102 unknown patient samples. A neural network output score of greater than 0.8 was considered the cutoff for positive assignment of the sample.

Statistics.

Sensitivity, specificity, PPV, and NPV were calculated using either viral culture or RT-PCR as the reference standard. Confidence intervals (95%) were calculated using the Wilson score method (36) (no continuity correction) as described by Newcombe (25).

RESULTS AND DISCUSSION

The goal of the present study was a quantitative comparison of the diagnostic performance of the recently developed MChip to well-established methods for influenza diagnosis, including viral culture, RT-PCR, and QuickVue. A total of 102 respiratory specimens that had been collected from patients with influenza-like illness and tested by QuickVue Influenza A+B and viral culture were selected for subsequent analysis by RT-PCR and MChip. The median age of the patients from whom the specimens were collected was 7.5 years (range, 3 months to 86 years). Of the 102 samples analyzed by viral culture, 57 (56%) were positive for influenza A virus, 9 (9%) were positive for respiratory syncytial virus, and 36 (35%) were negative for viruses. RT-PCR detected influenza A virus in 61 (60%) samples and obtained negatives for 41 (40%). QuickVue detected influenza A virus in 53 (52%) samples and gave negative results for 49 (48%) samples. The MChip detected influenza A virus in 57 (56%) samples and gave negative results for 45 (44%) samples.

Table 1 contains a quantitative comparison of method performances. Using viral culture as the reference, the MChip exhibited a clinical sensitivity of 98% and a clinical specificity of 98%. Within the calculated confidence intervals, the MChip sensitivity was similar to those of the QuickVue test (93%) and RT-PCR (98%). The PPV and NPV were also similar within the calculated confidence intervals. The MChip exhibited a clinical specificity of 98%, which was similar to that of the QuickVue test (100%) but substantially greater than the specificity for RT-PCR (89%). The relatively low specificity for RT-PCR originated from a number of samples that were positive by RT-PCR but negative by all other tests. One potential explanation for this result is that RT-PCR is more sensitive than viral culture. Such an explanation is consistent with previous comparative studies for influenza virus detection (2, 9, 12, 16), as well as arguments that RT-PCR would be a more reliable reference standard than viral culture (28). However, it is important to note that the lower specificity for RT-PCR when referenced to viral culture is also consistent with RT-PCR yielding a higher rate of false positives. We intend to quantitatively investigate both postulates in future work.

TABLE 1.

Assay performance summary and comparison

Assay	% Indicated value (95% confidence interval)
	Sensitivity	Specificity	PPV	NPV
Referenced to viral culture
MChip	98 (91-100)	98 (89-100)	98 (91-100)	98 (88-100)
QuickVue	93 (83-97)	100 (92-100)	100 (93-100)	92 (81-97)
RT-PCR	98 (90-100)	89 (77-95)	92 (82-96)	98 (87-100)
Referenced to RT-PCR
MChip	92 (82-96)	98 (87-100)	98 (91-100)	89 (77-95)
QuickVue	85 (74-92)	97 (87-100)	98 (90-100)	82 (69-90)
Viral culture	92 (82-96)	98 (87-100)	98 (91-100)	89 (77-95)

In order to account for the possibility that RT-PCR is a viable standard, the MChip results were also referenced to RT-PCR. As summarized in Table 1, the resulting clinical sensitivity was 92% and the clinical specificity was 98%. The PPV and NPV were 98% and 89%, respectively. Thus, the MChip performance is sufficient for diagnostic purposes when referenced to either viral culture or RT-PCR. In fact, the clinical sensitivity and specificity determined for the MChip in this blinded study are better than the average values reported for most rapid tests (8, 12, 14, 18, 20, 27, 29).

It is instructive to consider the samples for which there was disagreement between the various analytical methods. As summarized in Table 2, a comparison of the performances of MChip and RT-PCR shows six cases in which there was disagreement. In four of those cases (samples 8 to 11 in Table 2), RT-PCR yielded a positive result and all three other techniques yielded a negative result. As discussed above, it is not clear whether these results indicate a higher sensitivity for RT-PCR or a higher degree of susceptibility to false positives. Unexpectedly, there was one case (sample 3) for which RT-PCR yielded a negative result and all three other techniques yielded a positive result. There was one sample (sample 1) for which the MChip provided a negative result while all three other methods yielded a positive result. Excluding the six samples discussed above, in all other cases, MChip results were the same as those noted for RT-PCR. In addition, no statistically significant trends were observed when the data were analyzed by patient age. It is to be noted that no influenza B virus-positive specimens were included in the study.

TABLE 2.

Summary of method discrepancy^a

Sample	Sample type	Patient age	Result with indicated method
			QuickVue	Viral culture	RT-PCR	MChip
1	NP aspirate	18 mo	Flu A	Flu A	Flu A	Neg
2	NP swab	23 mo	Neg	Neg	Flu A	Flu A
3	NP aspirate	3 yr	Flu A	Flu A	Neg	Flu A
4	Direct collect	3 yr	Neg	Flu A	Flu A	Flu A
5	Direct collect	6 yr	Neg	Flu A	Flu A	Flu A
6	Direct collect	22 yr	Neg	Flu A	Flu A	Flu A
7	Direct collect	40 yr	Neg	Flu A	Flu A	Flu A
8	Nasal swab	9 yr	Neg	Neg	Flu A	Neg
9	Direct collect	27 yr	Neg	Neg	Flu A	Neg
10	Direct collect	34 yr	Neg	Neg	Flu A	Neg
11	Nasal swab	86 yr	Neg	Neg	Flu A	Neg

^aNP, nasopharyngeal; Flu A, influenza; Neg, negative.

One significant advantage of the MChip is the simultaneous detection and subtype identification for influenza virus-positive samples in a single experiment. None of the other methods evaluated in this study provided subtype information without additional testing. For viral culture, an additional test involving reference antibodies must be conducted to determine influenza virus subtype. For RT-PCR, several additional RT-PCR experiments targeting the H1, H3, and H5 genes are necessary to determine the subtype. Subtype information is especially important in locations like Southeast Asia, where H3N2, H1N1, and H5N1 can concurrently circulate.

In this study, a separate subtype analysis by RT-PCR (data not shown) was performed on the influenza A virus-positive samples and the results compared to those obtained directly with the MChip method. Both analytical methods indicated that all influenza A virus-positive samples were the A/H3N2 subtype (data not shown).

In summary, the MChip exhibited performance characteristics similar to those of other well-established diagnostic tests for influenza viruses, with MChip demonstrating a clinical sensitivity of 98% and a specificity of 98% compared to viral culture. When MChip was compared to RT-PCR, the sensitivity and specificity were 92% and 98%, respectively. Although the MChip assay currently requires 7 to 8 h to complete the analysis, efforts to reduce the analysis time to approximately one hour are under way. Even with the current analysis time of 7 to 8 h, the MChip demonstrated a significant advantage over the other test methods by providing simultaneous detection and full subtype identification for the two subtypes currently circulating in humans (A/H3N2 and A/H1N1) and avian (A/H5N1) viruses. This capability promises to provide a significant advantage to public health officials, vaccine manufacturers, and physicians involved in direct patient care, for all of whom early detection and rapid subtype information can be critically important.