Factors influencing the diagnostic accuracy of the rapid influenza antigen detection test (RIADT): a cross-sectional study

Abstract

Objective

To evaluate the diagnostic accuracy of the rapid influenza antigen detection test (RIADT) and determine which symptoms are relevant to results.

Design

Single-centre, cross-sectional study.

Setting

Primary care centre, Tokyo, Japan.

Participants

82 consecutive outpatients presenting with upper respiratory symptoms and fever ≥37°C at any time from symptom onset, between December 2010 and April 2011.

Main outcome measures

Results of history and physical examination including age, sex, temperature, time of test from symptom onset, vaccination record and current symptoms (sore throat, arthralgia and/or myalgia, headache, chills, cough and/or throat phlegm, nasal discharge) were recorded. The RIADT and a fully automated respiratory virus nucleic acid test (Verigene Respiratory Virus Plus; VRV), the latter being the gold standard, were performed. Patients were divided into four groups: false negative (FN), RIADT− and VRV+; true positive (TP), RIADT+ and VRV+; true negative (TN), RIADT− and VRV−; and false positive, RIADT+ and VRV−. Groups were compared regarding age, sex, temperature, time of test from symptom onset, vaccination record and symptoms.

Results

RIADT sensitivity, specificity, positive predictive value and negative predictive value were 72.9% (95% CI 61.5% to 84.2%), 91.3% (79.7% to 102.8%), 95.6% (89.5% to 101.6%) and 56.8% (40.8% to 72.7%), respectively. Time from symptom onset to test was shorter for the FN group than the TP group (p=0.009). No significant differences were detected for the other factors assessed. Results revealed higher temperatures for FN than TN patients (p=0.043), and more FN than TN patients had chills (p=0.058).

Conclusions

The RIADT sensitivity was low, due to early administration of the test. In the epidemic season, the RIADT should not be used for suspected influenza until 12 h after symptom onset. A positive RIADT firmly supports the influenza diagnosis; a negative result does not confirm its absence. High fever and chills might indicate influenza, but additional tests are sometimes necessary.

Strengths and limitations of this study.

• Influenza is one of the most common diseases that general physicians see clinically. The rapid influenza antigen detection test (RIADT) is helpful for diagnosis, but its low sensitivity is sometimes misleading, resulting in underdiagnosis. We aimed to evaluate the diagnostic accuracy and characteristics of the RIADT and determine which symptoms are relevant to the results. Our findings may help general physicians when using the RIADT.

• Clinically, the sensitivity of RIADT is relatively low and the specificity is high; current results confirm this. Testing too early could be a factor increasing false negative results. For patients presenting with high fever and upper respiratory symptoms soon after onset, RIADT should not be used.

• The presence of high fever and chills may be helpful indicators of influenza, even if the RIADT result is negative, but additional examination is necessary for patients with symptoms inconsistent with influenza virus infection.

• The high specificity of the RIADT means that a positive result provides firm support for the diagnosis of influenza.

• The following limitations need to be acknowledged. First, the 82 participants enrolled in the current study were relatively young and healthy. Second, the reference or ‘gold’ standard in this study was not viral culture or reverse transcriptase-PCR, but the Verigene System Verigene Respiratory Virus Plus, which detects influenza virus nucleic acid.

Introduction

Influenza is a rapid-onset systemic illness caused by the influenza virus; patients present with high fever, chills, cough, myalgia, sore throat and headache.^{1 2} Previously, the diagnosis was made from these symptoms.^3–6 However, since their introduction to Japan in 1999, rapid influenza antigen detection tests (RIADTs) using immunochromatography have dramatically changed the influenza diagnostic procedure.^{7 8} Before the introduction of RIADTs and anti-influenza drugs, physicians told patients to stay home if they had no suspected complications. Now physicians use RIADTs to diagnose influenza and therefore can prescribe antiviral drugs soon after symptom onset.^8–13 Making the distinction between the flu and other respiratory diseases serves to improve individual care management.^14–16 Detection of influenza virus can reduce inappropriate antibiotic use, guide antiviral therapy and decrease use of other laboratory studies and healthcare costs.¹⁴ Currently, many RIADTs are available; their sensitivities and specificities have improved and their usefulness has been widely recognised.¹⁷ RIADTs help physicians to diagnose influenza during epidemics, but the RIADT results make it difficult to diagnose flu during periods of transition from epidemic to non-epidemic times, or when patients present with atypical symptoms.^{17 18}

For example, when patients’ flu-like symptoms are typical of influenza but the RIADT results are negative, whether or not the patient actually has influenza is unclear. Physicians may question whether samples were obtained correctly or whether the result is a false negative (FN), and may hesitate to prescribe anti-influenza drugs.

According to a meta-analysis reported in 2012, the pooled RIADT sensitivity was 62.3% (95% CI 57.9% to 66.6%), and specificity was 98.2% (95% CI 97.5% to 98.7%).¹⁹ Thus, the specificity is very high but the sensitivity is relatively low. Several factors affecting the results of RIADTs have been reported, but some remain controversial.^{19 20}

This study was designed to evaluate the diagnostic accuracy and characteristics of one RIADT, the RapidTesta FLUII (Sekisui Medical, Tokyo, Japan), and to determine which symptoms are associated with the results obtained. In addition, we sought to identify predictors of influenza for patients with FN RIADT results.

Methods

From December 2010 to April 2011, during the influenza epidemic season in Japan,²¹ participants were enrolled in the Departments of General Medicine of Juntendo University Hospital and Juntendo University Nerima Hospital, both in Tokyo, Japan. Consecutive cases were enrolled that met the following inclusion criteria: adult patients presenting with any upper respiratory symptoms, and fever ≥37°C at any time after symptom onset. All of the participants were physically examined and historical data including age, sex, vaccination status, temperature and symptoms (sore throat, arthralgia and/or myalgia, headache, chills, cough and/or throat phlegm and nasal discharge) were recorded, as was the time to test from symptom onset. Vaccination status indicated whether or not an influenza vaccine had been administered during that season before symptom onset. The temperature was taken on presentation by an outpatient physician. Symptoms recorded were those participants reported on presentation. Only a few patients had taken antipyretics and their temperature was around 36°C, but we could not analyse the data with regard to antipyretics use. All were tested by the RIADT and a fully automated respiratory virus nucleic acid test, Verigene Respiratory Virus Plus (VRV; NanoSphere, Chicago, Illinois, USA), the latter being the gold standard for this study. Based on the test results, patients were divided into four groups, as follows: FN, RIADT− and VRV+; true positive (TP), RIADT+ and VRV+; true negative (TN), RIADT− and VRV−; and false positive (FP), RIADT+ and VRV−. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the RIADT were calculated. In order to ascertain the cause of false-negative results and to find a predictor of influenza, comparisons were made between pairs of groups (FN vs TP and then FN vs TN) with respect to age, sex, vaccination status, time from symptom onset to test, temperature and symptoms.

Laboratory confirmation

Two nasopharyngeal specimens were collected from each patient by a physician, using sterile cotton swabs, following the procedure detailed in the RIADT manufacturer's package insert.

Rapid influenza antigen detection test

RIADTs are immunoassays using the antigen-antibody reaction, based on colloidal gold immunochromatography. The test results are checked visually. The RIADT used for this study, the RapidTesta FLU II, requires a 1×10⁵ tissue-culture infective dose (TCID)₅₀ per mL for type A influenza, and 1.2×10⁵ TCID₅₀/mL for type B influenza, to produce a positive result.²² RIADT was performed using one of the nasopharyngeal specimens, according to the manufacturer's instructions. In brief, samples were diluted in the medium immediately, and then dripped into the test device. Results were determined 15 min after the test start. When influenza A or B is present, an additional red line appears next to the control red line on the letter ‘A’ or ‘B’ indicated on the test device. The procedures were performed by outpatient physicians and residents who had been well trained in the technique.

Fully automated respiratory virus nucleic acid test

The second nasopharyngeal swab was immediately placed into Universal Viral Transport medium (UTM; Becton Dickinson, Tokyo, Japan) and tested by the Verigene System within 24 h. The system uses a multiplexed microarray-based technology. A nucleic acid detection cartridge named VRV Nucleic Acid Test was selected, which could detect the following influenza viruses: A (H1N1), A (H3N2), pandemic 2009 influenza A (H1N1), influenza B virus and respiratory syncytial virus (A, B). This system was approved by the US Food and Drug Administration (FDA) in May 2009.²³

Specimens were tested by Verigene test according to the manufacturer's instructions by the physician attending the study patients. Briefly, the test cartridge was preloaded with wash solutions, oligonucleotide probe solution and signal amplification solution. The extraction tray, amplification tray and test cartridge were then loaded onto the Verigene System. Following the addition of 200 µL of UTM containing materials expressed from the nasopharyngeal swabs, the analysis began; this consisted of a programmed, totally automated extraction, reverse transcriptase-PCR (RT-PCR) and hybridisation sequence. The final readout of the microarray was made by the insertion of the slide array into a reader.²⁴ The result for each virus type, ‘Detected’ or ‘Not detected,’ was displayed on a monitor. Approximately 2.5 h was required from sample procurement to final readout.

Specimen collection was performed by one of five physicians, who also read the results. These physicians were trained by an instructor from the manufacturer before the study started.

Statistical analysis

The sensitivity, specificity, PPV and NPV of the RIADT used were determined using standard methods. Statistical analyses were performed using statistical software package, STATA SE V.12 (StataCorp LP, College Station, Texas, USA). Continuous variables (age, the time from symptom onset and temperature) were analysed by the Wilcoxon rank sum test, and the Fisher's exact test was used for comparing patient sex, vaccination status and symptoms. Significance was assigned to results having p values <0.05, and borderline significance was assigned to p values >0.05 and <0.10.

Results

A total of 82 consecutive patients meeting eligibility criteria were enrolled from December 2010 to April 2011 (Juntendo University Hospital: 37 patients; Juntendo University Nerima Hospital: 45 patients). There was no selection discretion on the part of the attending physicians. Table 1 shows characteristics for all patients. The median age was 30.5 (range 20–63) years, and 42.7% (35/82) were men. During the 2010/2011 flu season, 48.8% (40/82) of the patients were vaccinated for influenza. The average time from symptom onset to diagnostic test was 18.9±17.2 h; 13.4% (11/82) came to the hospital within 6 h from symptom onset and 72% (59/82) within 24 h.

Table 1.

Baseline patient characteristics

Patient characteristics	N (%)
Age (years)
Median 30.5, range 20–63
<30	37 (45.1)
30–49	36 (43.9)
≥50	9 (11.0)
Male sex (%)	35 (42.7)
Vaccination status (%)	40 (48.8)
Time to test from symptom onset (hours)
Mean 18.9±17.3
<6	11 (13.4)
6–12	14 (17.1)
12–24	34 (41.5)
24–48	13 (15.8)
≥48	9 (11.0)
Unknown	1 (1.2)

Table 2 shows the RIADT and VRV results as well as the RIADT accuracy. By RIADT results, 54.9% (45/82) of patients were positive and the other 45.1% (37/82) were negative. By VRV results, 72.0% (59/82) were positive and the other 28.0% (23/82) were negative. Dividing them into four groups, the number of patients in each group was: FN, 16; TP, 43; TN, 21 and FP, 2. The prevalence of influenza A or B virus infection was 72%. When the VRV test was used as the gold standard, the RIADT sensitivity, specificity, PPV and NPV were 72.9% (95% CI 61.5% to 84.2%), 91.3% (79.7% to 102.8%), 95.6% (89.5% to 101.6%) and 56.8% (40.8% to 72.7%), respectively.

Table 2.

RIADT (RapidTesta FLU II) accuracy

	RIADT
	A(+) and/or B(+)	(−)	Total
Verigene test (VRV)
Positive	43	16	59
Negative	2	21	23
Total	45	37	82

Prevalence: 72%.

Sensitivity: 72.9% (95% CI 61.5% to 84.2%).

Specificity: 91.3% (95% CI 79.7% to 102.8%).

PPV: 95.6% (95% CI 89.5% to 101.6%).

NPV: 56.8% (95% CI 40.8% to 72.7%).

NPV, negative predictive value; PPV, Positive predictive value; RIADT, rapid influenza antigen detection test; VRV, Verigene Respiratory Virus Plus.

Tables 3 and 4 show the patients’ clinical characteristics and presenting symptoms by group. One patient was excluded from the FN group due to development of bacterial pneumonia 72 h after presenting with a high fever. No significant differences were found in age, sex and vaccination status among the FN, TP and TN groups. The time from symptom onset to test was significantly earlier in the FN group compared with the TP group (11.4±10.9 vs 22.0±17.3 h, p=0.009). However, no significant differences between these groups were found for the other factors and symptoms assessed. The FN and TN groups were compared in order to reveal any differences in diagnostic factors or symptoms. The temperature of the FN negative group was higher than the TN group (38.2±0.8 vs 37.6±0.8, p=0.043). More patients presented with chills in the FN group (7/15 vs 3/21, p=0.058: borderline significance). No other significant differences were found in symptoms.

Table 3.

Patient characteristics by group

				p Value
Patient characteristics	FN group (n=15†)	TP group (n=43)	TN group (n=21)	FN vs TP	FN vs TN
Age in years
Median	31	30	29	0.880	0.653
Range	20–57	21–62	23–63
Age distribution (years)	n (%)	n (%)	n (%)
<30	6 (40.0)	19 (44.2)	11 (52.4)
30–49	6 (40.0)	20 (46.5)	8 (38.1)
≥50	3 (20.0)	4 (9.3)	2 (9.5)
Male sex	7 (46.7)	17 (39.5)	9 (42.9)	0.763	1.000
Vaccination status	5 (33.3)	22 (51.2)	13 (61.9)	0.368	0.176
Time to test from symptom onset (hours)
Mean	11.4±10.9	22.0±17.3	15.7±16.8	0.009*	0.449
Time to test from symptom onset (hours)	n (%)	n (%)	n (%)
<6	3 (20.0)	4 (9.3)	2 (14.3)
6–12	6 (40.0)	6 (14.0)	2 (9.5)
12–24	5 (33.3)	16 (37.2)	11 (52.4)
24–48	0 (0.0)	11 (25.6)	2 (9.5)
≥48	1 (6.7)	6 (14.0)	2 (9.5)
Unknown	0 (0.0)	0 (0.0)	1 (4.8)

*p<0.05.

†Excluded one patient with bacterial pneumonia that developed after influenza virus infection.

FN, false negative; TN, true negative; TP, true positive.

Table 4.

Symptoms and clinical characteristics at presentation

				p Value
Patient characteristics	FN group (n=15†)	TP group (n=43)	TN group (n=21)	FN vs TP	FN vs TN
Temperature (°C)
Mean	38.2±0.8	38.0±0.7	37.6±0.8	0.593	0.043*
Temperature (°C)	n (%)	n (%)	n (%)
≤37.4	2 (13.3)	8 (18.6)	9 (42.9)
37.5 to 38.0	5 (33.3)	15 (54.9)	7 (33.3)
≥38.1	8 (53.3)	20 (46.5)	5 (23.8)
Symptoms
Sore throat	11 (73.3)	32 (74.4)	14 (66.7)	1.000	0.729
Arthralgia/myalgia	6 (40.0)	22 (51.2)	10 (47.6)	0.554	0.741
Headache	7 (46.7)	13 (30.2)	6 (28.6)	0.345	0.310
Chills	7 (46.7)	16 (37.2)	3 (14.3)	0.553	0.058**
Cough/throat phlegm	9 (60.0)	25 (58.1)	10 (47.6)	1.000	0.516
Nasal discharge	2 (13.3)	15 (34.9)	6 (28.6)	0.118	0.424

*p<0.05.

**0.05<p<0.1.

†Excluded one patient with bacterial pneumonia who developed after influenza viral infection.

FN, false negative; TN, true negative; TP, true positive.

Combining the RIADT result and presence of temperature ≥37.8°C or chills increased the sensitivity and the NPV from 72.9% to 96.6% (95% CI 92.0% to 101.2%) and from 56.8% to 90.5% (77.9% to 103%), respectively. The specificity and the PPV were 82.6% (67.1% to 98.1%) and 93.4% (87.2% to 99.7%), respectively.

Discussion

Rapid influenza antigen detection tests

Influenza virus infection is confirmed by virus isolation, viral nucleic acid detection (eg, by RT-PCR) or detecting a rising serum antibody titre in the acute and convalescent period.^{19 23 25 26} However, these tests are time consuming and costly, so they are rarely used clinically in Japan.²⁴ Before introduction of RIADTs to Japan in 1999, physicians diagnosed influenza by assessing clinical symptoms and epidemiological information.

RIADTs are widely used nowadays, because they are simple, inexpensive and require no special facilities, equipment or technology.²⁷ Use of RIADTs can decrease unnecessary blood tests, imaging studies and antibiotic use.

According to our RIADT correlative examination results for type A influenza, the sensitivity is 94.3% and specificity is 97.8%, and the values for type B influenza are 87% and 100%, respectively, referencing results of virus culture and PCR.²² Clinically, the specificity is high but the sensitivity is low. Reported factors which lower the sensitivity are the following: type B influenza virus, pandemic 2009 influenza A (H1N1) virus, the timing of the test, use in adult patients, low fever, small amounts of sample, prior vaccination and poor sampling technique.^{19 20}

Time from symptom onset to the test

In the current study, the mean time from symptom onset to test in the FN group was 11.4±10.9 h, and 22.0±17.3 h in the TP group. The FN group was tested significantly earlier than the TP group (p=0.009). This suggests that testing too early is a factor increasing FN results, a finding that is consistent with a previous report.¹⁹

The RIADT used for this study, the RapidTesta FLU II, requires a 1×10⁵ TCID₅₀/mL for type A influenza, and 1.2×10⁵ TCID₅₀/mL for type B influenza, to produce a positive result. The influenza virus proliferates in respiratory tract epithelial cells and appears in respiratory secretions 24 h before symptom onset. The peak of viral shedding is 24 h after symptom onset, and then the virus load decreases rapidly.²⁸ In the current study, we assumed that the amount of virus in the FN group was less than that in the TP group.

Symptoms

The report of a systematic review from 2004 regarding the clinical diagnosis of influenza states that the medical history and physical examination findings of rigour, fever and sweating are best for positive influenza diagnosis (likelihood ratios +7.2, +4 and +3, respectively).²⁹ Fever was defined as a temperature ≥37.8°C or higher.² In the current study, the temperature of the FN group was significantly higher than that of the TN group (38.2±0.8 vs 37.6±0.8, p=0.043), but no significant difference of temperature was detected between the FN and TP groups (38.2±0.8 vs 38.0±0.7, p=0.593). This indicates that the temperature of patients with influenza is relatively high. It should be noted that, even if RIADT is negative, it is possible that patients presenting with high fever have influenza.

A previously reported prospective cohort study revealed the relationship between chills and bacteraemia: chills were divided into four categories of none, mild, moderate and severe, and a greater degree of chills suggested a higher risk of bacteraemia.³⁰ We did not categorise chills, but found that the FN group had more chills than did the TN group. The p value of 0.058, as ‘borderline significant’, was caused by having an insufficient number of participants. The presence of chills is also thought to be an indicator of influenza if the RIADT result is negative.

The positive RIADT result gives physicians firm support for the diagnosis of influenza because of the high specificity of this test, but a negative result does not confirm its absence. Presence of high fever and chills is indicative of influenza, but if patients’ presenting symptoms are inconsistent with influenza or acute upper respiratory infection, additional studies are necessary to make an accurate diagnosis. High fever with moderate to severe chills sometimes indicates bacteraemia or other bacterial infection.³⁰

Limitations

The 82 participants enrolled in the current study were relatively young and healthy. This is because our hospitals are located in central Tokyo and most of the patients are young or middle-aged healthy workers. We did not exclude elderly patients or patients with comorbid diseases. Elderly people tend to present with atypical symptoms and often have underlying primary illnesses. Also, because almost all of our patients are adults, paediatric patients were not included in this study. This group has distinctive symptoms, and the RIADT has the highest sensitivity in these patients. For this reason, applying similar research methods to different age groups may produce different results.

The US Center for Disease Control and Prevention suggested that the 2009 pandemic influenza A (H1N1) virus would continue to spread for years to come as a seasonal influenza virus.³¹ All the viruses detected in the current samples were 2009 pandemic influenza A (H1N1) or B. The sensitivity of RIADTs for 2009 pandemic influenza A (H1N1) virus was previously reported to be a little lower than that for influenza A (H3N2) virus,²⁰ but in the current study, the sensitivity was equal for the two strains.

The reference, or ‘gold,’ standard in this study was not viral culture or RT-PCR but analysis using the Verigene test VRV, which detects influenza viral nucleic acid. When direct fluorescent antibody identification and viral culture were used as the gold standards, the sensitivity of the VRV system used in this study for influenza A was 98.7% (95% CI 96.8% to 99.5%) and the specificity was 93.2% (95% CI 91.1% to 99.9%). For influenza B, the sensitivity was 100% (95% CI 91.8% to 100%) and the specificity was 99.7% (95% CI 99.1% to 99.9%).³² A reference standard was applied.

Conclusions

Consistent with previous reports, the sensitivity of the RIADT used in this study was low, due to early administration of the test. Administration of an RIADT too early after symptom onset causes FN results. In the influenza epidemic season, practitioners should not use RIADT for patients with upper respiratory symptoms and high fever for at least 12 h after onset. A positive RIADT result after this gives the physician firm support for a diagnosis of influenza. A negative RIADT result does not mean ‘no influenza’. Presence of high fever and chills might predict influenza, but additional tests are necessary for patients with specific symptoms inconsistent with a diagnosis of influenza virus infection.