Clinical Evaluation of Severe Acute Respiratory Syndrome Coronavirus 2 Rapid Antigen Tests During the Omicron Wave in South Africa

Abstract

We evaluated the performance of nasal and nasopharyngeal Standard Q COVID-19 [coronavirus disease 2019] Ag tests (SD Biosensor) and the Panbio COVID-19 Ag Rapid Test Device (nasal; Abbott) against the Abbott RealTime severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) assay during the Omicron (clades 21M, 21K, and 21L) wave in South Africa. Overall, all evaluated tests performed well, with high sensitivity (range, 77.78%–81.42%) and excellent specificity values (>99%). The sensitivity of rapid antigen tests increased above 90% in samples with cycle threshold <20, and all 3 tests performed best within the first week after symptom onset.

The SD Biosensor and Panbio rapid antigen tests performed well in samples with Omicron 21M, 21K, and 21L infections. The Panbio test performed best in infections with Omicron 21L, which had significantly higher viral load than 21M and 21K.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variant B1.1.529 was first reported on 24 November 2021 by the Network for Genomic Surveillance in South Africa and later designated by the World Health Organization as the Omicron variant of concern [1]. Owing to its enhanced transmissibility, Omicron has spread quickly around the world and currently represents the dominant variant globally. The Omicron SARS-CoV-2 variant has >30 mutations in the spike glycoprotein, with 15 located in the receptor-binding domain which is key for viral entry into the cells. The evolution and fast expansion of the Omicron SARS-CoV-2 variant was first noted in South Africa through the increase of S-gene target failures, using the Thermo Fischer TaqPath COVID-19 [coronavirus disease 2019] assay, resulting from the deletion of codons 69 and 70 in the spike (S) gene. The performance of reverse-transcription polymerase chain reaction (RT-PCR) tests that are not targeting the S-gene, was not affected by the Omicron variant.

Although their sensitivity is lower compared with SARS-CoV-2 RT-PCR, rapid antigen tests offer quick and affordable results at the point of care, enabling reliable detection of high viral load samples associated with the presence of infectious virus [2]. These tests have become a crucial tool to detect cases in a timely manner, even in resource-limited settings, and they therefore represent an important tool in controlling the pandemic.

Most widely used rapid antigen tests target the nucleocapsid protein and therefore should not be affected by the high degree of mutations in the S-gene. However, in addition to >30 mutations in the S-gene, Omicron has several mutations in the nucleocapsid, including P13L, Del31–33, R203K, and G204R, with R203K and G204R associated with enhanced infectivity in human lung cells [3]. Furthermore, Omicron sublineages have additional nucleocapsid mutations including S413R found in BA.2 and BA.3. In the current study, we evaluated the performance of 3 commonly used rapid antigen tests in comparison with the Abbott RT-PCR assay during the Omicron wave in KwaZulu-Natal, South Africa.

METHODS

Clinical Specimens

The evaluation was performed in the province of KwaZulu-Natal in South Africa at drive-through testing centers from December 2021 until February 2022 (spanning the fourth wave of SARS-CoV-2 infections). Residents of the selected communities were offered SARS-CoV-2 testing if they met any of the following criteria: testing positive for COVID-19 in the previous 7 days; presence of COVID-19 symptoms in the previous 7 days; exposure to COVID-19 5–10 days earlier; healthcare worker status; or physician referral for testing. Study participants provided demographics, symptom type and onset date, vaccination status, and informed consent.

Two separate evaluation studies were performed; in the first, both nasal and nasopharyngeal (NP) Standard Q COVID-19 Ag tests from SD Biosensor were evaluated, and in the second the Panbio COVID-19 Ag Rapid Test Device (nasal) was evaluated. Three swab specimens (1 nasal and 2 NP) were collected for evaluation of the SD Biosensor kits, and 2 (1 nasal and 1 NP) for the Panbio test evaluation. In both evaluations, the nasal swab specimen was collected first to avoid cross-contamination between sites. This was followed by the NP swab specimen for the second rapid antigen test (in the first evaluation), and then the NP swab specimen for the SARS-CoV-2 RT-PCR reference test.

Rapid antigen tests were performed immediately after sample collection on site by trained medical staff. Swab specimens for RT-PCR were shipped without additives at room temperature to the central laboratory for processing within 3 hours of collection. Results from the South African Health Products Regulatory Authority– approved rapid antigen test were reported immediately to the participants, and a confirmatory RT-PCR result was provided within 24 hours after sample collection. The study was approved by the KwaZulu-Natal Biomedical Research Ethics Committee (approval no. BREC/00001195/2020).

SARS-CoV-2 RT-PCR

On arrival at the laboratory, NP swab specimens were resuspended in 2 mL of viral transport medium. The Abbott RealTime SARS-CoV-2 assay (target sequences in the SARS-CoV-2 RdRp and N genes of the SARS-CoV-2 genome) was used to test for the presence of SARS-CoV-2. All samples that tested positive (irrespective of cycle threshold [Ct] values) were sequenced at KwaZulu-Natal Research Innovation and Sequencing Platform (KRISP) [1]. Briefly, RNA was extracted on an automated Chemagic 360 instrument (Perkin Elmer). Libraries for whole-genome sequencing were prepared using the Oxford Nanopore Midnight protocol with rapid barcoding per the manufacturer’s protocol and sequenced on the GridION. Sequences with >80% coverage were deposited on the GISAID sequence database. The GISAID accession numbers of sequences are provided in Supplementary Table 1.

SARS-CoV-2 Rapid Antigen Tests

The following 3 kits were evaluated: Standard Q COVID-19 Ag test (SD Biosensor; nasal), Standard Q COVID-19 Ag test (SD Biosensor; NP), and Panbio COVID-19 Ag Rapid Test Device (nasal). According to the manufacturers, the tests detect SARS-CoV-2 nucleocapsid protein with no cross-reaction with other common respiratory pathogens except SARS-coronavirus [4, 5]. All samples were collected, and assays were performed by trained medical staff and per manufacturer protocols. All 3 tests are World Health Organization emergency use listing procedure approved and are the most procured rapid antigen tests in low- and low middle-income countries.

Statistical Analysis

GraphPad Prism software (version 8.3.1; GraphPad Software) and SPSS software (version 24) were used to perform the statistical analysis. Test performance characteristics were calculated in reference to Abbott RealTime SARS-CoV-2 assay results. The 95% confidence intervals (CIs) were calculated to assess the level of uncertainty induced by sample size, using the Wilson score method. The D’Agostino-Pearson omnibus normality test was used to assess data distribution; t test, to assess differences in Ct values between true- and false-positive results and vaccination status groups; and Kruskal-Wallis with Dunn multiple comparisons test, done to assess differences in Ct values between symptom categories and Omicron clades. Fully vaccinated participants were classified as any participants who received either 1 dose of Johnson & Johnson’s Janssen COVID-19 vaccine or 2 doses of Comirnaty/Pfizer-BioNTech COVID-19 vaccine ≥2 weeks before testing.

RESULTS

Study Sample Characteristics

The evaluation of Standard Q Ag tests was performed on 297 samples (Table 1). The median age of participants was 33 years (interquartile range [IQR], 25–49 years). The overall SARS-CoV-2 positivity in the study group was 41.75%, with a median Ct value (IQR) of 13.90 (0.40–18.09). Most study participants presented for testing within the first week after symptom onset (67.00%). Fully vaccinated participants made up 59.26% of the study cohort with 32.95% (58 of 176) having received 1 dose of Johnson & Johnson’s Janssen COVID-19 vaccine and 67.05% (118 of 176) having received 2 doses of Comirnaty/Pfizer-BioNTech COVID-19 vaccine. The majority (98.39%) of the SARS-CoV-2–positive samples were classified as Omicron sublineage BA.2 (Nextstrain clade 21K). Two SARS-CoV-2–positive samples lacked sequencing data and were excluded from the SD Biosensor evaluation.

Table 1.

Participant and Sample Characteristics for Evaluations of Rapid Antigen Tests

	Participants or Samples, % (No.)a
Characteristic	Evaluation 1: SD Biosensor Tests (n = 297)	Evaluation 2: Panbio Test (n = 462)
Age, median (IQR), y	33 (25–49)	41 (26–55)
Female sex	56.23 (167)	50.87 (235)
PCR positivity	41.75 (124)	39.83 (184)
Presence of symptoms
Asymptomatic or presymptomatic	27.61 (82)	31.17 (144)
<7 d after symptom onset	67.00 (199)	62.99 (291)
≥7 d after symptom onset	5.39 (16)	5.84 (27)
Vaccination status
Fully vaccinated	59.26 (176)	64.29 (297)
Unvaccinated	32.99 (98)	26.84 (124)
Partially vaccinated	7.74 (23)	8.87 (41)
HIV positive	0.34 (1)	0.43 (2)
Oxygen saturation, median (IQR)	97 (97–99)b	98 (96–99)b
Ct, median (IQR)	13.90 (10.40–18.09)	14.06 (9.79–21.07)
Omicron lineage among SARS-CoV-2 positive
21M	1.61 (2/124)	5.98 (11/184)
21K	98.39 (122/124)	73.37 (135/184)
21L	1.61 (2/124)	20.65 (38/184)

Abbreviations: Ct, cycle threshold; HIV, human immunodeficiency virus; IQR, interquartile range; PCR, polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

^aData represent % (no.) of participants or samples unless otherwise specified.

^bOxygen saturation data were missing for 24 patients in evaluation 1 and 6 in evaluation 2.

The evaluation of the Panbio Ag test device was performed on 462 samples (Table 1). The median age (IQR) of study participants was 41 (26–55) years). The overall SARS-CoV-2 test positivity was 39.83%, with a median Ct value (IQR) of 14.06 (9.79–21.07). Most study participants presented within the first week after symptom onset (62.99%), with 64.29% being fully vaccinated. Of the fully vaccinated study participants, 19.87% (59 of 297) received 1 dose of Johnson & Johnson’s Janssen COVID-19 vaccine, and 80.13% (238 of 297) received 2 doses of Comirnaty/Pfizer-BioNTech COVID-19 vaccine. The majority (73.37%) of the SARS-CoV-2–positive cases were identified as Omicron sublineage BA.2 (Nextstrain clade 21K). For the Panbio Ag Test, 23 positive samples with SARS-CoV-2 clades other than Omicron were excluded from the evaluation. The 2 evaluations were comparable with respect to study participant and sample characteristics, with the exception of study participant age, which was significantly higher in the Panbio Ag test evaluation (P = .001).

Test Performance Evaluation

The overall test performance for nasal and NP Standard Q Ag tests is summarized in Figure 1 and Supplementary Table 2A. The overall sensitivity and specificity of the nasal Standard Q COVID-19 Ag test were 79.84% (95% CI, 71.93%–85.95%) and 100.00% (97.83%–100.00%), respectively. The sensitivity of the test increased in samples with lower Ct values: for samples with a Ct <25 the sensitivity was 85.71% (95% CI, 78.05%–91.01%), and for those with a Ct <20 it was 92.93% (86.12%–96.53%).

Figure 1.

Sensitivity (A) and specificity (B) of the Panbio Ag test (nasal), Standard Q Ag test (nasopharyngeal [NP]), and Standard Q Ag test (nasal) across different categories. Error bars represent 95% confidence intervals.

Similar results were obtained for the NP Standard Q Ag test (Figure 1 and Supplementary Table 2A), with an overall sensitivity of 79.03% (95% CI, 71.05%–85.27%) and specificity of 99.42% (96.80%–99.90%). As with the nasal kit, the sensitivity increased in samples with lower Ct values, to 84.81% (95% CI, 77.03%–90.30%) in samples with a Ct <25 and 91.92% (84.86%–95.85%) in those with a Ct <20. With respect to symptom onset time, both tests performed best in individuals presenting within the first week after symptom onset, with sensitivities of 83.51% (95% CI, 74.87%–89.58%) for the nasal and 82.47% (73.71%–88.76%) for the NP kit (Figure 1 and Supplementary Table 2B). Exclusion of 4 samples with Omicron clade 21M (parental lineage B1.1.528) or 21L (sublineage BA.2) did not affect the sensitivity of the tests (NP, 78.69% [95% CI, 70.60%–85.02%]; nasal, 79.52% [71.50%–85.72%]) (Figure 1 and Supplementary Table 2C).

For the nasal Panbio Ag test device, the overall sensitivity was 81.42% (95% CI, 75.16%–86.39%), and the overall specificity was 99.64% (97.99%–99.94%) (Figure 1 and Supplementary Table 3A). The sensitivity increased in samples with lower Ct values, to 88.55% (95% CI, 82.82%–92.55%) in samples with a Ct <25 and to 93.20% (87.93%–96.26%) in those with a Ct <20. As with the other 2 kits, the sensitivity was highest in patients presenting within the first week after symptom onset, 86.62% (95% CI, 80.05%–91.26%) (Figure 1 and Supplementary Table 3B). With respect to Omicron lineage, highest sensitivity was observed for Omicron 21L/BA.2 (100.00% [95% CI, 90.82%–100.00%]), followed by Omicron 21K/BA.1 (77.61% [69.84%–83.84%]), with the lowest sensitivity observed for Omicron 21M infections (63.64% [35.38%–84.83%]) (Figure 1 and Supplementary Table 3C).

As expected, the majority of false-negative results for all 3 tests were observed in samples with higher Ct values (Supplementary Figure 1). We observed similar sensitivity values across the first week after symptom onset for all tests (Supplementary Table 4). Overall, samples from study participants presenting within the first week after symptom onset had significantly lower Ct values than samples from participants with no symptoms (P < .001) (Supplementary Figure 2A). There was a significant difference in Ct values between the 3 Omicron lineages’ samples with 21M had significantly higher Ct values than samples with 21L/BA.2 (P < .001) or 21K/BA.1 (P = .001), and samples with 21L/BA.2 had significantly lower Ct values than samples with 21K/BA.1 (P < .001) (Supplementary Figure 2B).

For both evaluations, all false-negative values with high SARS-CoV-2 viral load (Ct <20) occurred in infections with Omicron 21K/BA.1. Further analysis of the viral sequences with >80% coverage from false-negative samples with high SARS-CoV-2 viral loads (Ct <20) did not reveal additional amino acid changes in the nucleocapsid protein (Supplementary Table 5 [6]). In addition to P13L, Del31–33, R203K, and G204R, 1 sample had a P142S substitution that was previously found in BA.1.21 at 0.3% (19 of 6581) [7]. While the sensitivity for all 3 tests was slightly higher in unvaccinated individuals (range, 84.00%–86.05%) versus fully vaccinated individuals (78.57%–80.65%) (Supplementary Table 6), we observed no significant differences in SARS-CoV-2 Ct values or day of presentation after symptom onset between participants depending on their vaccination status (Supplementary Figure 2C and 2D).

DISCUSSION

The emergence of each novel SARS-CoV-2 variant of concern prompts the need to evaluate its potential impact on the performance of diagnostic tests currently in use. In the current study, we evaluated the performance of 3 commonly used rapid antigen kits during the Omicron wave in South Africa.

We found an high sensitivity overall (range, 79.03%–81.42%) for all 3 tests, with excellent specificity values as well. The sensitivity of rapid antigen tests increased in samples with lower Ct values (indicative of higher viral load [8]), increasing to >90% in samples with Ct values <20. As expected, all 3 tests performed best in participants presenting within the first week after symptom onset, when the SARS-CoV-2 viral load is highest [9–12]. Our results are consistent with previously published data in on the circulation of other SARS-CoV-2 variants [13–15]. As previously reported (before emergence of the Omicron variant), we observed similar performance for the nasal and NP Standard Q Ag tests performed on equivalent samples [16].

We also examined the impact of vaccination status on test performance because vaccination and preexisting immunity could potentially affect symptom presentation and timing with respect to infectiousness and viral load. We observed slightly higher sensitivity in unvaccinated individuals for all 3 tests, consistent with the previous observations showing that previous immunity is associated with a lower SARS-CoV-2 viral load on infection [17]. We did not observe significant differences in Ct values depending on the vaccination status of the study participants, but it is important to note that these results could be confounded by natural SARS-CoV-2 infections, on which we did not have data.

With respect to Omicron lineage, in the analysis of the Panbio Ag test, the highest sensitivity was observed for 21L/BA.2, followed by 21K/BA.1 and finally 21M/parental lineage B1.1.528. The observed differences in rapid Ag test performance are likely due to observed differences in Ct values between infections with the 3 lineages, with 21L/BA.2 having the lowest Ct values. Similar observations with regard to viral load differences and infectiousness between Omicron sublineages have been reported elsewhere [18, 19]. While we do not have data on BA.4 and BA.5 sublineages, based on their nucleocapsid profile and increase in infectiousness, test performance with these sublineages is likely to resemble that with BA.2 [20]. All false-negative samples with high SARS-CoV-2 viral load (defined as Ct <20) belonged to the 21K/BA.1 Omicron sublineage, with no unique additional nucleocapsid amino acid changes.

One limitation of our study is the lack of the field performance data on the evaluated tests in the same community in previous waves; however, our results are consistent with those of previously published studies performed in similar settings and in a similar manner. Overall, our data indicate that the performance of the SD Biosensor and Panbio rapid SARS-CoV-2/COVID-19 antigen test was not negatively affected by the emergence of Omicron subtypes BA.1 and BA.2, showing that rapid antigen tests remain an important tool for managing the pandemic.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.