Are Rapid Influenza Antigen Tests Still Clinically Useful in Today's Molecular Diagnostics World?

Abstract

Influenza virus infection and disease historically contribute to widespread cases of seasonal morbidity and in some cases mortality. Prompt and accurate diagnosis is crucial for optimal patient management. Rapid influenza direct antigen testing (RIDT) offers a faster turn-around-time for results but test performance (ie, sensitivity and specificity) varies widely. Nucleic acid amplification testing (NAAT) can offer a viable alternative. The objective of this retrospective study was to compare the test performance of RIDT with NAAT. RIDT testing included the Directigen EZ Flu A+B or the Veritor System for Rapid Detection of Flu A+B. NAAT employed the SimplexaTM Flu A/B™ RSV assay. A total of 5,795 specimens collected from October to March for the 2012/2013 (n=953), 2013/2014 (n=2060) and 2014/2015 (n=2783) seasons were co-tested by RIDT and NAAT. Using NAAT as the gold standard, RIDT tests had a sensitivity range of 0 to 15.7% and a specificity of 98.2 to 100% for influenza type A. For influenza type B, RIDT tests had a sensitivity of 0 to 33.3% and a specificity of 98.9 to 100%. These findings suggest that RIDT has unacceptably low sensitivity for both influenza A and influenza B, despite high specificity. The key advantage of RIDT in previous years (faster turnaround time) has been challenged by newer NAAT technology that provides results in a turn-around-time comparable to RIDT, but with superior test performance.

Introduction

It is estimated that 5 to 20% of the population will contract influenza during the fall through winter respiratory season each year in the United States.¹ The exception is Hawai‘i, where influenza is present all year long with peaks also occurring during the respiratory season. Immunocompetent individuals recover from the flu without complications within about a week. However, more than 200,000 patients in the United States are also admitted to the hospital every year for flu-related complications, which are potentially life-threatening and sometimes fatal.²

The signs and symptoms of influenza disease are not specific to the influenza virus, because other respiratory viruses can present with a similar clinical syndrome. Therefore, rapid and accurate influenza detection is critical for the differential diagnosis, infection control, appropriate antiviral treatment (if indicated), and control over unwarranted antibiotic usage.³

Influenza virus type A and type B are the common etiological agents of influenza. Influenza viruses are RNA viruses and more prone to acquiring mutation than DNA viruses. As such, they are capable of antigenic shift and drift over time and the shift is defined by strain subtyping.⁴ This strain variability can impact vaccine efficacy, pathogenicity, variation in viral shedding, antiviral resistance, and could also have an influence on diagnostic test performance depending upon the methodology.⁴

Clinical diagnostic laboratory testing for influenza virus detection consists of conventional culture, rapid culture (shell vial), RIDT, and NAAT.⁵ Conventional and rapid cultures have been considered the “gold standard,” but have been successfully challenged with the “platinum standard” of NAAT as a result of a superior test performance. This is evidenced by a high test positivity in a population with influenza (sensitivity) and a high test negativity in a population without influenza (specificity).⁶ Historically, RIDT has offered more rapid turn-around-times but has been shown to vary widely in test performance depending upon the methodology or even the influenza virus strain(s) in circulation.⁶ Newer NAAT platforms maintain the test performance of predecessor NAAT tests, but now have turn- around-times comparable to RIDT. This allows for “point of care testing” and shorter turn-around-time by providing more clinically actionable results.^7–13

Studies in the literature have compared various RIDT formats and found a wide range of test performance, as summarized in Table 1.^{11, 14–26} However, none of these studies have focused on patient populations in Hawai‘i. The current study addresses RIDT and NAAT test performance in this population by comparing test results over three consecutive respiratory seasons in Hawai‘i.

Table 1.

Literature search showing the median, mean and range sensitivity and specificity for influenza virus type A (n = 28) and type B (n = 7) test performance using Rapid Influenza Diagnostic Tests.

Statistical Value	Influenza A		Influenza B
	Sensitivity	Specificity	Sensitivity	Specificity
Median	63.5	99.0	78.6	98.7
Mean	59.1	98.3	73.3	97.3
Range	9.7 – 95.8	91.1 – 100	40 – 92.9	89.7 – 100

Methods

Patients and Specimens

The patient population (ages less than 1 year to greater than 65 years) in this study included patients residing in or visiting the State of Hawai‘i from whom specimens were collected and submitted for influenza testing during the 2012/13, 2013/14 and 2014/15 influenza seasons. Testing was performed at Diagnostic Laboratory Services (DLS, Aiea, HI, the reference lab for The Queen's Medical Center). Data was collected from the healthcare professional (HCP) that ordered both RIDT and NAAT (by reverse transcriptase polymerase chain reaction, or RT-PCR) tests, or from the HCP that ordered RIDT testing with a negative result that was reflexed to RT-PCR. It should be noted that the request for reflex testing varied among the community-based healthcare providers. However, in compliance with the Queen's Medical Center's Infection Control Committee policy for hospitalized patients, all specimens with negative RIDT results were reflexed to NAAT testing for confirmation. Therefore, hospital-based testing resulted in a substantial number of specimens with both tests.

Specimens consisted mostly of nasopharyngeal swabs, but nasal aspirates or throat swab specimens were also included in some cases. All testing for the 2012 to the 2015 influenza seasons was performed at Diagnostic Laboratory Services, Inc. (Aiea, Hawai‘i). Patient identifiers and other protected health information (PHI) with the exception of age and collection date were de-linked.

Laboratory Testing

Prior to February 8, 2013, the RIDT used by DLS was the Becton Dickinson Directigen EZ Flu A+B (Directigen; BD Diagnostics, Sparks, MD). This RIDT was replaced by the BD Veritor System for Rapid Detection of Flu A+B (Veritor; BD Diagnostics, Sparks, MD) beginning on February 8, 2013. Both RIDTs were used exactly according to the package insert instructions.

NAAT was performed using the Simplexa™ Flu A/B & RSV assay (Focus Diagnostics) with the 3M Integrated Cycler instrument. The Simplexa™ Flu A/B & RSV assay uses real-time RT-PCR amplification for the detection and differentiation of influenza type A, influenza type B, and respiratory syncytial virus (data on RSV was excluded from this study). RT-PCR testing consisted of two automated steps, RNA extraction and real-time nucleic acid amplification and detection.²⁷

Analysis

RIDT testing was compared NAAT, which served as the gold standard. RIDT results were classified as true positive (TP), if the specimen was positive for influenza A or influenza B by both RIDT and NAAT; true negative (TN), if the specimen was negative for influenza A and B by both RIDT and NAAT; false positive (FP), if the specimen was positive for influenza A or influenza B by RIDT, but negative for influenza viruses by NAAT; or false negative (FN), if the specimen was negative for influenza A or B by RIDT, but positive for either influenza virus by NAAT. Sensitivity was calculated using the formula TP/(TP+FN), and specificity was calculated as TN/(TN+FP).

Results

RIDT performance was benchmarked using the NAAT as the gold standard for the 2012/13, 2013/14 and 2014/15 respiratory virus seasons. A total of 5,796 specimens were co-tested with 953 tested in the October to March 2012/13, 2060 specimens in the 2013/14, and 2783 specimens in the 2014/2015 respiratory seasons. Circulating influenza type A virus strains present during this time period consisted almost exclusively of the seasonal influenza type A H3N2 with a rare appearance of the influenza type A 2009 H1N1, which was mostly reported in late 2012 and early 2013.²⁸ The percentage of specimens positive for influenza virus type A virus peaked at 32.0% in February 2013, 22.2% in January 2014, and 31.1% in January 2015. Likewise, the percentage of specimens positive for influenza virus type B peaked in the month of May at 21.1%, 8.3%, and 11.0% in 2013, 2014 and 2015, respectively.

Based on the NAAT result serving as the gold standard, the RIDT revealed a monthly sensitivity and specificity range of 0–15.7% and 98.2–100% respectively for influenza virus type A, and 0–33.3% and 98.9–100% for influenza virus type B respectively, as shown in Table 2. The negative and positive predictive values ranged from 80.9–99.5% (NPV) and 84.2–100.0% (PPV) for influenza virus type A virus detection, and from 95.5–100% (NPV) and 0–100% (PPV) for influenza type B virus detection respectively (data not shown). The highest discordance between RIDT and NAAT results for influenza virus type A was seen in January 2015. It should be noted that this observation coincidentally occurred during the month with the highest number of influenza positive tests seen over the entire study period.

Table 2.

Rapid Influenza Diagnostic Test (RIDT) influenza type A and type B test performance range compared to Nucleic Acid Amplification Tests (NAAT) as the reference (gold standard) for the influenza seasons 2012–2013, 2013–2014, and 2014–2015.

Year	Influenza A		Influenza B
	Sensitivity	Specificity	Sensitivity	Specificity
2012	0	100	0	100
2013	0 – 6.7	98.2 – 100	0 – 33.3	98.9 – 100
2014	0 – 14.3	100	0	100
2015	9.8 – 15.7	99.5 – 100	0 – 21.4	100

Discussion

From 1976 to 2006, influenza accounted for up to 49,000 deaths annually in the United States.^29,30 Influenza virus infection may be more severe when accompanied by related coinfections and complications. It is also associated with an increase in hospitalizations of the very young, elderly, and those with other risk factors; increased hospitalizations are especially prominent during epidemics.³⁰ A timely and accurate diagnosis is critical for optimal patient care for several reasons: first, it enables the proper and timely use of appropriate antivirals. Second, it avoids the use of unwarranted antibiotics, which in turn may mitigate the global rise in antimicrobial resistance.³¹

Conventional cell culture, rapid culture (shell vial), RIDT, and NAAT have all been used for diagnostic support of influenza virus infection. While conventional cell culture is typically considered the “gold standard,” its clinical utility is limited because the test requires up to 5 to 7 days; shell vial culture is another technique with excellent performance, but despite offering a vast improvement in timeliness compared to conventional cell culture, requires 24 to 48 hours for completion.⁵ These turnaround times may compromise the HCP's ability to provide antiviral treatment within the optimal window of 48 hours upon illness onset.³² NAAT offers the best test performance with a shorter turnaround time (18 minutes to 3 hours), but at a higher cost. Given the high sensitivity and specificity of NAAT, it was used as the gold standard in the current study. The current study demonstrates that while RIDT has a faster turnaround time, it demonstrates unacceptably low sensitivity for both Influenza A and influenza B viruses compared to NAAT (as shown in Tables 1 and 2).²⁶

RIDT tests in the present study were substantially less sensitive, with a higher number of false negative results. This finding was even observed during the peak of respiratory season when influenza incidence was the highest. The first RIDT used was the BD Directigen EZ Flu A+B nasopharyngeal assay, which has a fifteen minute turnaround time to results. According to the package insert, this test should exhibit a sensitivity of 91%, a specificity of 93%, a PPV of 88%, and a NPV of 94.8% for Influenza type A virus. Likewise, the second and more extensively used RIDT assay was the BD Veritor System, which has a turnaround time of ten minutes, and according to the package insert should exhibit a sensitivity of 78.8%, a specificity of 97.8%, a PPV of 93.8%, and a NPV of 91.4% for influenza type A virus. However, the medical literature indicates that RIDT often exhibits a wide range of test performance (i.e. especially for test sensitivity) and is consistently unpredictable across different assays (Table 1). Based on the literature, sensitivity ranges from 9.7–95.8% for influenza type A and 40–92.9% for influenza type B (Table 1). One of the RIDT tests used in the present study (BD Veritor System) was challenged in a Centers for Disease Control and Prevention study against six other RIDT for the detection of influenza type A H3N2 virus.³³ The BD Veritor and the Sofia revealed the lowest limit of detection among the seven RIDT evaluated.³³

There are many variables to consider in accounting for differences in test performance, such as the type of assay, the circulating viral subtype and even the course of disease and viral shedding pattern for the particular circulating influenza virus subtype.⁶ Another factor to consider is the high variability in performance of RIDTs when evaluating emerging strains of influenza viruses.³³ In a study by Yang, et.al., the test performance of RIDTs in detecting emerging influenza A subtypes demonstrated a lower sensitivity for a newly emerging strain, the pandemic Influenza A (H1N1 subtype at 55.8%) compared to a seasonal strain that had been in circulation (Influenza A, H3N2 subtype) at 71.0%.³⁴ These differences were further modified by the underlying demographics. The performance of the BD Veritor was found not only to be affected by the circulating viruses, but also by patient age, which was inversely related to the test's sensitivity. The BD Veritor system's sensitivity for children less than 2 years of age was 85.7%, compared to 60.3% for children and adults between 2 and 39 years, and 33.3% for adults aged ≥ 40 years.²⁰ This may be due to the fact that there is an inversely proportional relationship between the amount of virus shedding and the age of the patient.²⁰

The variations in sensitivity is dependent upon a variety of factors that cannot be controlled and suggests that in order for the test findings of RIDTs to reliably guide treatment, there is a need for more RIDT quality assurance. In other words, each RIDTshould be accessed for the ability to detect multiple virus subtypes on a continual basis. This would be a challenging task. However, such a testing recommendation has been proposed by the FDA. Typically, if an assay is cleared by the Food and Drug Administration (FDA), manufacturers are not required to conduct regular quality assurance and reevaluation on the assay. However, due to the poor test performance of RIDT flu testing, the FDA proposed regulation §866.3328 of the Code of Federal Regulations (CFR), which specifies the need for a mandatory annual analytical reactivity testing of contemporary influenza strains for all RIDT tests.³⁵ This CFR also includes testing for emerging subtypes that pose a danger to public health.³⁶

By contrast, NAAT has the distinct advantage of consistently exhibiting the highest sensitivity and specificity. It is noteworthy that NAAT also exhibits a high test performance even when emerging influenza virus strains are encountered; NAAT has been shown to be more consistent and to excel in test performance compared to RIDT.⁶ In fact, RIDT test performance has been so variable over the past seasons that the Queen's Medical Center Infection Control Committee has incorporated an algorithm that required a reflex for all negative RIDT to the more sensitive and specific RT-PCR. Hence, RIDT testing may even be clinically and financially wasteful, as it may often be followed by NAAT support as follow-up testing, which is more expensive and extends the turn-around-time on test reporting.

In the current study, RIDT test performance revealed the highest number of false negative results near the influenza peak of the respiratory seasons. This was unexpected, because the highest test performance is expected to occur during the time of peak influenza incidence.³⁷ Furthermore, this observation has clinical relevance, since it is most critical to optimize patient care during the peak of flu season.This peak period is when most influenza-related hospitalizations may occur and antiviral therapy may be most needed to reduce transmission. The potential risks of using a less accurate test such as RIDT are delayed diagnosis, increased transmission, and potential hospitalizationwith the chance of serious complications, including death.^29,30 Ironically, RIDT sensitivity is lowest among the elderly, which is the population at greatest risk for serious negative outcomes.

Patient age was not formally evaluated in the present study, but the expected relationship between age and sensitivity was anecdotally confirmed in the present study. The lowest percentages of false negative results was observed in patients less than one year of age, while the older patient age groups had an incremental proportion of false negative results (data not shown).

Finally, another noteworthy anecdotal observation was a change in HCP behavior reflected in the type of influenza testing ordered. Over the course of the three seasons under study, the influenza tests ordered by HCP's revealed a steady increasing trend in requests for NAAT testing and a corresponding decrease in orders for RIDT testing. This observation may very well reflect the lack of HCP confidence in the RIDT test performance compared to NAAT, although this hypothesis was not formally evaluated. In summary, a current challenge for the field of molecular diagnostics is to simplify the testing and reduce turnaround times while maintaining optimal test performance. For influenza testing, a viable solution has finally been realized with the recent introduction of the FDA cleared and in some cases CLIA waived latest generation of NAAT tests. This includes the cobas^â Liat System (Roche Diagnostics), the Alere i Influenza System (Alere), and the Xpert® Xpress Flu/RSV (Cepheid).^10,38,39 The Alere i Influenza System employs isothermal NAAT testing to provide ultra-rapid nucleic acid amplification to detect and distinguish between influenza virus type A and B in approximately 10 minutes.¹⁰ The cobas^â Liat technology consists of a simplified, automated RT-PCR assay exhibiting high test performance, which also has the advantage of a greatly reduced testing time period compared to conventional NAAT.³⁸ The cobas^â Liat System is a rapid in vitro qualitative NAAT that discriminates between influenza virus type A and B and generates a result in about 18 minutes.³⁸ The specimen (i.e. nasopharyngeal swab) undergoes nucleic acid extraction and amplification/detection using RT-PCR. This assay requires no additional reagent preparation aside from the addition of the specimen to the Liat tube. In addition, there is virtually no cross-contamination between samples because each Liat Tube is self-contained.³⁸ The cobas^â Liat package insert claims a 100% sensitivity, 97.1% specificity, 96.1% PPV, and 100% NPV.³⁸ Any one of the newly introduced, “near patient diagnostic testing” NAAT is comparable to RIDT in turnaround time. However, they all have the added advantage of high test performance similiar to the “classic” RT-PCR molecular test formats.

The current study has some limitations. First, because of the use of de-identified data, other variables that may have influenced the outcome of the study may not have been considered in the analysis. Second, the study group was a patient population residing in the state of Hawai‘i, and may not be equivalent to other patient populations, which may exhibit a difference in test performance. Likewise, these results may also reflect circulating influenza subtypes in Hawai‘i that may be different from those circulating in other geographic locations. Lastly, the data gathered in Hawai‘i came from a single testing laboratory, Diagnostic Laboratory Services. Therefore, the findings may not be representative of the entire patient resident population in the State of Hawai‘i.

Despite these limitations, other investigators have reported similar results showing that many RIDTs are not reliable. The current study confirms the findings from other investigators in support of the unreliability of RIDTs for the patient population in the state of Hawai‘i. This report allows our medical community to better understand the usefulness of ordering a diagnostic test with the highest test performance and the tradeoff advantage between increased test cost and patient outcome. It is important to educate HCPs on the RIDT test performance variability and the advantage of using a more reliable, but more expensive NAAT test in order to consistently achieve more favorable patient outcomes. Based upon the findings in the present study and our literature review confirming these findings, the recommendation to HCP's is to order NAAT in place of RIDT. In consideration of the emergence of new generation NAAT technologies with higher test performance and test turn-around-time comparable to RIDT, the higher cost of the NAAT may be outweighed by the benefits of reliability, accuracy, and comparable timeliness.

Conclusion

The current report describes the advantages and disadvantages of two of the most common influenza tests, RIDT and NAAT. In this study, NAAT was chosen as the gold standard. RIDT revealed an overall very weak test performance, even during the peak of the influenza seasons where the test performance should have been the highest. These findings are confirmed by multiple studies reported elsewhere, and demonstrate the low RIDT reliability to be a significant concern for influenza detection and management in Hawai‘i. The newly introduced NAAT methodology has improved turn-around-time while maintaining high test performance. Examples include such tests such as the cobas® LIAT system, Alere i Influenza System and the Xpert® Xpress Flu/RSV. Anyone of these tests have satisfied the need for faster turnaround times that are comparable to RIDT, while consistently maintaining optimal test performance. These newly introduced, “near patient diagnostic testing” NAAT show great promise in now offering a viable replacement to RIDT. Similar to other investigators, the results presented in this study have provided further evidence that RIDT is not reliable in the Hawai‘i patient population and it may be time to consider replacing RIDT with the newly introduced rapid “near patient diagnostic testing” NAAT. HCP's are strongly urged to request NAAT for influenza testing in place of RIDT to avoid delayed or missed diagnoses of influenza.

Abbreviations and Acronyms

RIDT

Rapid Influenza Direct antigen Testing

RT-PCR

Reverse Transcriptase Polymerase Chain Reaction

PPV

Positive Predicted Value

NPV

Negative Predicted Value

NAAT

Nucleic Acid Amplification Test

LOD

Lowest Limit of Detection

PHI

Personal Health Information

HCP

Health Care Providers

Conflict of Interest

None of the authors identify a conflict of interest.

Author Disclosures

Matthew J. Bankowski PhD, is a compensated member of the Roche Molecular Diagnostics Advisory Board.