Performance Evaluation of Five Rapid At-Home COVID-19 Antigen Tests against the Omicron Variant

ABSTRACT

Rapid coronavirus disease 2019 (COVID-19) antigen tests can be used to aid in quickly identifying positive cases, which can help mitigate the spread of COVID-19 infection. Using previously characterized Omicron-positive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), non-Omicron-positive SARS-CoV-2, and negative samples, we evaluated five brands of at-home rapid COVID-19 antigen tests (On/Go at-home COVID-19 rapid antigen self-test, iHealth COVID-19 antigen rapid test, QuickVue SARS antigen test, Abbott BinaxNOW COVID-19 card home test, and InBios SCoV-2 Ag detect rapid self-test). Our results showed that these rapid tests had similar levels of sensitivity to Omicron and non-Omicron variants (On/Go, 76.4% and 71.0%; iHealth, 73.0% and 71.0%; QuickVue, 84.3% and 74.3%; BinaxNOW, 69.7% and 71.0%; and InBios, 66.3% and 64.5%, respectively). In conclusion, rapid COVID-19 antigen tests can continue to be used as part of public health measures to combat the spread of the Omicron variant, as their sensitivity was not significantly affected.

IMPORTANCE The emergence of the Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is due to mutations as part of the virus evolution process. These mutations might affect the sensitivity of diagnostic tests that are currently being used to detect the virus. Because rapid coronavirus disease 2019 (COVID-19) antigen tests are commonly used in the general population, it is important to assess their performance in detecting the Omicron variant. Here, we compared the performance of five brands of rapid tests against Omicron and non-Omicron variants using nasopharyngeal swab samples in viral transport media. Our result found no difference in their performance, suggesting no reduction in sensitivity when used to detect the Omicron variant.

INTRODUCTION

Coronavirus disease 2019 (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The disease was first reported in Wuhan, China, in December 2019 and quickly spread around the world, creating a pandemic (1). Continuous selection of several mutations has led to the emergence of SARS-CoV-2 variants (2). The Omicron variant (B.1.1.529) of the virus was first reported in South Africa in November 2021 and quickly replaced the previously dominant Delta variant (B.1.617.2) in the United States. The Omicron variant was noted to have multiple mutations in its genome that were shown to increase its transmissibility, reduce the effectiveness of vaccines, reduce the effectiveness of monoclonal antibody treatment, and affect the sensitivity of several diagnostic tests.

Early detection of SARS-CoV-2 infection is important for preventing further spread of the disease (3). Real-time reverse transcriptase PCR (RT-PCR) remains the gold standard for the diagnosis of COVID-19. However, the assay is categorized as a high-complexity test, which requires trained personnel and specialized equipment.

Rapid COVID-19 antigen tests (RCATs) provide an alternative method of detecting SARS-CoV-2 (4). These tests are faster, relatively less expensive, and easier to perform compared to RT-PCR, albeit with lower sensitivity and specificity. The U.S. Food and Drug Administration (FDA) granted emergency use authorization (EUA) to multiple different brands of RCATs that are available over the counter for at-home use. In addition, federal and local governments have also been handing out RCATs to their residents to help curb the spread of the virus. However, these products were developed prior to the emergence of the Omicron variant; thus, their performance has not been evaluated against it.

In this study, we evaluated five RCATs that received FDA EUA for at-home usage against Omicron and non-Omicron samples. These respiratory samples had previously been tested for SARS-CoV-2. Our objective was to assess the performance, including the sensitivity and specificity, of the On/Go at-home COVID-19 rapid antigen self-test, iHealth COVID-19 antigen rapid test, QuickVue SARS antigen test, Abbott BinaxNOW COVID-19 card home test, and InBios SCoV-2 Ag detect rapid self-test.

RESULTS

In this study, we assessed the performance of five RCATs against the Omicron variant of SARS-CoV-2 using the TaqPath COVID-19 combo kit for RT-PCR as the gold standard. We found that the sensitivities of all the RCATs evaluated were much lower than those indicated in their instructions for use. Overall, the sensitivities published by the manufacturer versus those discovered in this investigation were 94.3% versus 72.5% for iHealth, 87.0% versus 75.0% for On/Go, 84.6% versus 70.0% for BinaxNOW, 96.8% versus 81.7% for QuickVue, and 86.7% versus 65.8% for InBios, respectively. Overall, all RCATs evaluated in this study had 100% specificity, except for QuickVue (87.5%) (Table 1). The overall sensitivity of the RCATs varied, with QuickVue having the highest sensitivity (81.7%), followed by On/Go (75.0%), iHealth (72.5%), BinaxNOW (70.0%), and InBios (65.8%).

TABLE 1.

Device performance using respiratory samples compared to real-time reverse transcriptase PCR results

Test type	No. of samplesa				Sensitivity (95% CI) (%)b	Specificity (95% CI) (%)
	TP	FN	FP	TN
iHealth	87	33	0	32	72.5 (63.6–80.3)	100.0 (89.1–100.0)
On/Go	90	30	0	32	75.0 (66.3–82.5)	100.0 (89.1–100.0)
BinaxNOW	84	36	0	32	70.0 (61.0–78.0)	100.0 (89.1–100.0)
QuickVue	98	22	4	28	81.7 (73.6–88.1)	87.5 (71.0–96.5)
InBios	79	41	0	32	65.8 (56.6–74.2)	100.0 (89.1–100.0)

^aTP, true positives; FN, false negatives; FP, false positives; TN, true negatives.

^bSensitivity (TP/TP+FN) and specificity (TN/TN+FP) were calculated. The upper and lower bounds for 95% confidence intervals (CIs) are shown.

When the positive samples were divided into Omicron and non-Omicron groups, the sensitivity of these RCATs fluctuated (Table 2). The sensitivity of the iHealth (71.0% to 73.0%), On/Go (71.0% to 76.4%), QuickVue (74.2% to 84.3%), and InBios (64.5% to 66.3%) tests increased when used to test Omicron samples, with QuickVue having the biggest jump (10.1%) in sensitivity. Only the sensitivity of BinaxNOW numerically decreased when used to test Omicron-positive samples (71.0% to 69.7%). The sensitivity of all RCATs was at or above 90% when used to test positive samples with threshold cycle (C_T) values of 25 or lower for both Omicron and non-Omicron variants, but they dropped significantly for samples with C_T values higher than 25. For Omicron samples with a C_T value greater than 25, QuickVue had the highest sensitivity (46.2%), followed by On/Go (19.2%), iHealth (15.4%), BinaxNOW (7.7%), and InBios (3.9%). On the other hand, for non-Omicron samples, BinaxNOW and QuickVue had the highest sensitivity (22.2%), followed by iHealth and On/Go (11.1%); InBios could not detect any positive non-Omicron samples with C_T values above 25 (0.0%).

TABLE 2.

Device sensitivity using Omicron and non-Omicron variant respiratory samples for three C_T value groups

Test type	CT value	Data for:
		Omicron samples			Non-Omicron samples
		No. of samples:		Sensitivity (95% CI) (%)	No. of samples:		Sensitivity (95% CI) (%)
		Tested	RCAT positive		Tested	RCAT positive
iHealth	Total	89	65	73.0 (62.6–81.9)	31	22	71.0 (52.0–85.8)
	<20	27	27	100.0 (87.2–100.0)	10	9	90.0 (55.5–99.80)
	20–25	36	34	94.4 (81.3–99.3)	12	12	100.0 (73.5–100.0)
	>25	26	4	15.4 (4.4–34.9)	9	1	11.1 (0.3–48.3)
On/Go	Total	89	68	76.4 (66.2–84.8)	31	22	71.0 (52.0–85.8)
	<20	27	27	100.0 (87.2–100.0)	10	9	90.0 (55.5–99.80
	20–25	36	36	100.0 (90.3–100.0)	12	12	100.0 (73.5–100.0)
	>25	26	5	19.2 (6.6–39.4)	9	1	11.1 (0.3–48.3)
BinaxNOW	Total	89	62	69.7 (59.0–79.0)	31	22	71.0 (52.0–85.8)
	<20	27	27	100.0 (87.2–100.0)	10	9	90.0 (55.5–99.80
	20–25	36	33	91.7 (77.5–98.3)	12	11	91.7 (61.5–99.8)
	>25	26	2	7.7 (1.0–25.1)	9	2	22.2 (2.8–60.0)
QuickVue	Total	89	75	84.3 (75.0–91.1)	31	23	74.2 (55.4–88.1)
	<20	27	27	100.0 (87.2–100.0)	10	9	90.0 (55.5–99.8)
	20–25	36	36	100.0 (90.3–100.0)	12	12	100.0 (73.5–100.0)
	>25	26	12	46.2 (26.6–66.6)	9	2	22.2 (2.8–60.0
InBios	Total	89	59	66.3 (55.5–76.0)	31	20	64.5 (45.4–80.8)
	<20	27	27	100.0 (87.2–100.0)	10	9	90.0 (55.5–99.8)
	20–25	36	31	86.1 (70.5–95.3)	12	11	91.7 (61.5–99.8)
	>25	26	1	3.9 (0.1–19.6)	9	0	0.0 (0.0–33.6)

DISCUSSION

Overall, the RCATs had similar levels of sensitivity against Omicron (range, 66.3% to 84.3%) and non-Omicron variants (range, 64.5% to 74.3%). All five RCATs showed lower sensitivity than the manufacturers claimed. These lower-than-expected sensitivities could be attributed to the choice of samples that the manufacturers used to assess the initial positive agreement. They might be skewed more toward samples with lower C_T values because the manufacturers’ intended target for the RCATs are symptomatic patients, who tend to have higher viral loads than asymptomatic cases (1). It could also be the different types of samples that we were using to evaluate the rapid tests. Instead of inserting the tip of the swab into a nostril to collect a sample as suggested by the manufacturers, we dipped a swab into viral transport medium (VTM) collected previously containing residual nasopharyngeal swab. Additionally, even though the VTM helped preserve the samples well for molecular assay detection, some VTM brands might decrease the sensitivity of the RCATs (5). On the other hand, we did observe the specificity of these RCATs to be very close to the numbers posted on the instructions (all 100%), with the exception of QuickVue (87.5%). It is worth noting that this might be the tradeoff of having the highest sensitivity of all the RCATs evaluated here.

We found a C_T value of 25 to be the limit of detection (LOD) at which the RCATs could provide reliable positive results, which is consistent with a similar study performed before Omicron became the dominant variant (6–8). For samples with a C_T value higher than 25, the sensitivity declined significantly. Some studies also have postulated that C_T values of 25 may be the threshold for patient infectivity (7, 9). Nevertheless, C_T values are semiquantitative and can be affected by different factors, such as the patient immune status, collection time, and collection method. Therefore, this LOD based on a C_T value of 25 cannot be reliably generalized.

As these RCATs were intended to be used by the general public, most of whom are without prior laboratory experience, we also assessed the ease of use and also ease of interpretation of these kits. Both iHealth and On/Go had a similar system, with a swab that was added to the provided buffer, followed by dropping the mixture into the cassette. We found these two assays easy to perform, and their results were also easily interpreted, as evidenced by only two disagreements between the two independent operators. The BinaxNOW test required the buffer to be added to a well in a card, into which the swab was inserted and twirled before the card flap was closed. It was slightly more cumbersome, as there was difficulty with inserting the swab and potential leakage from the card when trying to read the result. In addition, the card tended to slant to the side when the flap was closed, causing inconsistent bands to form at the end of the incubation period. QuickVue had the highest sensitivity but also the lowest specificity of all the RCATs tested. The assay was fairly simple, too, requiring the insertion of the swab into the provided buffer and then of the strip into the buffer-swab mixture. The biggest drawback of this test was how difficult the interpretation of this assay was, as shown by the highest number of discrepant interpretations between the two independent operators. The InBios test used a cassette similar to that of On/Go and iHealth, but the swab was inserted into the cassette, and the buffer was then dropped into it. Unfortunately, when we tested the VTM, the cassette had many leaks, causing it to be very difficult to handle; however, this may not be an issue when testing a nasal swab directly.

There are several limitations in this study. First, even though we used the same VTM that was tested using RT-PCR, all the RCAT instructions for use were for nasal swabs. Second, these samples had undergone one freeze-thaw cycle; although we have demonstrated that this did not affect the C_T values, the same might not be applicable to the viral antigens (see Fig. S1 in the supplemental material). Third, we used C_T values as an estimate of the viral titer in the sample. Fourth, we did not have data on the onset of symptoms of these samples. Fifth, our operators are more experienced than laypeople, causing bias in our observation of the RCATs’ ease of use and interpretation. Finally, the Omicron variant that was used in this study was subvariant BA.1, as identified by S-gene target failure using the TaqPath COVID-19 combo kit. These results might differ for the other subvariants of Omicron.

The RCATs are used to help inform patients if isolation is necessary due to SARS-CoV-2 infection. The demand for RCATs in the United States is highest during the holidays, when people use the assays to test themselves prior to seeing their friends and family members (10). Thus, it was reassuring to see that most of the tests we investigated still had high specificity and were sensitive enough to detect positive cases with C_T values lower than 25, which is associated with viral shedding. In conclusion, we observed no statistically significant differences in the sensitivity and specificity among the RCATs tested, even when used against the Omicron variant. This indicates that RCATs continue to be a useful part of public health measures to combat COVID-19.

MATERIALS AND METHODS

Respiratory specimens.

The respiratory specimens were COVID-19 clinical residual nasopharyngeal swabs in viral transport medium (VTM) samples from the Naval Infectious Diseases Diagnostic Laboratory. These samples had previously been tested using the TaqPath COVID-19 combo kit (Thermo Fisher), which has EUA from the FDA. A total of 152 samples, consisting of 32 negative samples, 31 SARS-CoV-2-positive non-Omicron samples, and 89 SARS-CoV-2 positive Omicron samples, were used to evaluate the RCATs. The non-Omicron samples included the Alpha, Delta, and Iota variants. The positive samples were categorized based on their N-gene C_T values as high titer (C_T, <20), medium titer (C_T, 20 to 25), and low titer (C_T, >25); the C_T values were obtained using the TaqPath COVID-19 combo kit. These samples were deidentified prior to use in this study. The extracted positive samples were previously sequenced to confirm their lineage. The proportion of samples with different C_T values or viral titer that were used in this study was similar between the Omicron and non-Omicron groups. Among the positive Omicron samples, there were 27, 36, and 26 samples with C_T values less than 20, between 20 and 25, and higher than 25, respectively. For the non-Omicron samples, there were 10, 12, and 9 samples with C_T values of less than 20, between 20 and 25, and higher than 25, respectively.

Rapid COVID-19 antigen tests.

Five different RCATs that had received FDA EUA were selected for this study. They were the On/Go at-home COVID-19 rapid antigen self-test (Intrivo, CA), iHealth COVID-19 antigen rapid test (iHealth Labs Inc., CA), QuickVue SARS antigen test (Quidel, San Diego, CA), Abbott BinaxNOW COVID-19 card home test (Abbott, IL), and InBios SCoV-2 Ag detect rapid self-test (InBios, Seattle, WA). The enclosed manufacturer’s instructions were followed for each RCAT with a slight modification: the swab was dipped into the VTM instead of swabbing the nasal cavity. The evaluation was performed by two independent operators. After each run, the presence of the control band and the test band was observed by two operators independently. Any discrepancy between the first and second operators in reading the RCAT results was solved by a third observer, who acted as a tiebreaker to make a final determination on that test. All results were observed within the allowable time, as noted on the manufacturer’s instructions.

On/Go at-home COVID-19 rapid antigen self-test.

Briefly, the swab was dipped into the sample for 30 s with 10 swirling motions to mimic swabbing the inside of the nasal cavity. The swab was then dipped into the provided buffer and vigorously shaken for five times. The swab was pinched to make sure that all liquid had been squeezed out and then discarded. Afterwards, the buffer tube was capped with the provided lid and mixed using a flicking motion three times. Three drops of this mixture were then added to the provided well in the RCAT cassette, and the test was incubated for 10 min. At the end of 10 min, the result was rated by the observers as noted above.

iHealth COVID-19 antigen rapid test.

Briefly, the swab was dipped into the sample for 30 s with 10 swirling motions to mimic swabbing the inside of the nasal cavity. The swab was then dipped into the provided buffer and vigorously shaken 15 times. The swab was pinched to make sure that all liquid had been squeezed out and then discarded. Afterward, the buffer tube was capped with the provided lid. Three drops of this mixture were added to the provided well in the RCAT cassette, and the test was incubated for 15 min. At the end of 15 min, the result was rated by the observers, as noted above.

QuickVue SARS antigen test.

Briefly, the swab was dipped into the sample for 30 s with 10 swirling motions to mimic swabbing the inside of the nasal cavity. The swab was then dipped into the provided buffer, stirred four times, and left in the tube for 1 min. After incubation and removal of the swab, the provided strip was inserted into the mixture and incubated for 10 min. At the end of 10 min, the result was rated by the observers, as noted above.

Abbott BinaxNOW COVID-19 card home test.

Briefly, the swab was dipped into the sample for 30 s with 10 swirling motions to mimic swabbing the inside of the nasal cavity. Six drops of the provided buffer were added to the appropriate location on the test card. The sample swab was inserted in the RCAT card, rotated three times, and incubated for 15 min. The result was rated by the observers, as noted above.

InBios SCoV-2 Ag detect rapid self-test.

Briefly, the swab was dipped into the sample for 30 s with 8 swirling motions to mimic swabbing the inside of the nasal cavity. The swab was then clipped into the groove located on the RCAT cassette. Afterward, the provided buffer was added drop by drop onto the clipped swab, and the test was incubated for 20 min. At the end of 20 min, the result was rated by the observers as noted above.

Analysis of results.

The confidence intervals for proportions were analyzed using GraphPad Prism 8.