SARS-CoV-2 rapid antigen test: High sensitivity to detect infectious virus

Abstract

Background

The COVID-19 pandemic has highlighted the need for rapid, cost effective and easy-to-use diagnostic tools for SARS-CoV-2 infections that can be used in point of care settings to limit disease transmission.

Objective

We evaluated two rapid antigen immunochromatographic tests, Abbott Panbio™ COVID-19 Ag Rapid Test (Panbio) and Zhejiang Orient Gene/Healgen Biotech Coronavirus Ag rapid test cassette (Orient gene) for detection of infectious SARS-CoV-2.

Results

The tests were evaluated on nasopharyngeal samples taken from individuals having respiratory and/or COVID-19 related symptoms, which had been analyzed for SARS-CoV-2 RNA using real-time PCR. In total 156 PCR-positive, and 130 (Panbio) and 176 (Orient Gene) PCR-negative samples were analyzed. Overall sensitivity and specificity were 71.8% and 100% for Panbio and 79.5% and 74.4% for the Orient Gene test respectively. The false positives by the Orient Gene test were verified as SARS-CoV-2 negative by in-house real-time PCR assay and were negative for the four seasonal coronaviruses. Subgroup analysis revealed that the antigen tests had high sensitivity for samples with Ct-values <25 (>88%) and for samples containing infectious viruses as determined by cultivation on Vero cells, 94.1% and 97.1% for the Panbio and Orient gene tests, respectively. Furthermore, both tests had a sensitivity of <50 picogram for nucleocapsid protein. No sample with a Ct-value >27 was shown to contain infectious virus.

Conclusion

The results indicate that the rapid antigen tests, especially the Panbio tests may be a valuable tool to detect contagious persons during the ongoing pandemic.

1. Introduction

The current COVID-19 pandemic has highlighted the need for rapid diagnostics of ongoing SARS-CoV-2 infections to limit disease transmission. This need has resulted in a significant burden on diagnostic laboratories which frequently have been overwhelmed leading to large delays in analysis and reporting [1,2]. The reverse transcription real-time PCR (RT-qPCR) diagnostic assays used have high sensitivity and specificity, making them gold standard for COVID-19 diagnosis [2,3]. However, several disadvantages exist with these assays, such as cost and time needed for perform the laboratory analysis, further delayed due to the relatively few and often centralized facilities with capacity to perform the tests [1]. Due to the massive diagnostic need and importance of fast results, rapid antigen diagnostic tests (RAD) are a promising tool to help controlling disease transmission. The RADs are fast (20–30 min), easy to use and without requirement for specialized laboratories or equipment and can thus be1 used in point of care settings [4]. There are now a variety of RADs available on the market. However, it is of importance to properly validate these antigen tests before introducing as a supportive diagnostic method. Few studies evaluating RADs exists, yielding significant differences in sensitivity and specificity depending on manufacturer and/or type of samples used [4], [5], [6]. A further aspect of the current qPCR diagnostics is interpretation of the cycle threshold (Ct) value. Due to the high sensitivity of the qPCR assays, a very low number of virus genes can be detected, and SARS-CoV-2 RNA can be present in different body fluids for prolonged periods of time [7]. However, presence of virus genes does not translate into infectivity of the person, and studies have shown RNA positive samples to be negative for infectivity, particularly after longer time of shedding [8,9]. A previous study further observed that no respiratory samples with a SARS-CoV-2 qPCR Ct-value >24 showed infectivity in cell-culture [10]. To curtail spread of infection it would be important to have an estimate as to which Ct-value (SARS-CoV-2 genomic levels) indicate infectivity, and whether the RADs, while having lower overall sensitivity compared to qPCR, may have high sensitivity with regards to infectivity. In this study, we have evaluated RADs by two suppliers: the Abbott Panbio™ COVID-19 Ag Rapid Test and the Zhejiang Orient Gene/Healgen Biotech Coronavirus Ag rapid test cassette. Both RADs are immunochromatographic assays detecting the SARS-CoV-2 nucleocapsid protein without any specialized instruments. The tests were evaluated with nasopharyngeal samples from SARS-CoV-2 RT-qPCR positive and SARS-CoV-2 RT-qPCR negative samples from symptomatic individuals. We further correlated the sensitivity of the RADs to Ct values and infectivity.

2. Material and methods

2.1. Clinical samples

All samples were obtained from symptomatic individuals tested due to suspected SARS-CoV-2 infection in Region Östergötland, Sweden, between October and November 2020. Samples (nasopharyngeal swabs) were taken and dissolved in phosphate-buffered saline (PBS) or universal transport medium (Copan UTM™) and stored at +4 °C until analysis with RT-qPCR using Abbott Real Time SARS-CoV-2 or Alinity m SARS-CoV-2 AMP assays (Abbott, Solna, Sweden). In total 156 RT-qPCR positive and 176 RT-qPCR negative samples were used in this study. Samples were decoded without any possibility of back-track to the individual person. After RT-qPCR analysis, the samples were stored at 4 °C and tested with the rapid antigen tests within 1 week (median 2 days, range 1 to 7 days). Samples with Ct-value >40 (n = 2) were excluded from analysis.

2.2. Rapid antigen testing

Two rapid antigen tests were used, Abbott Panbio™ COVID-19 Ag Rapid Test and Zhejiang Orient Gene/Healgen Biotech Coronavirus Ag rapid test cassette. Ten samples were tested simultaneously with both antigen tests to enable a direct comparison. One drop of buffer was added to each test stick, followed by 50uL of nasopharynx sample and additional 2 (Orient Gene) and 3 (Panbio) drops of buffer in room temperature, according to the manufacturer's instructions. After 20 min of incubation, two persons, independently of each other, read all 20 test sticks to determine positivity or negativity. Positivity was further categorized into three strengths: “+++” where the intensity of the test band was stronger than the control band, “++” where the test band intensity was similar to the control band and “+” where the test band intensity was weaker than the control band. There was no difference in observations made by the two persons. Photos were taken for documentation.

2.3. Sensitivity testing with recombinant SARS-CoV-2 nucleocapsid protein

To test the sensitivity and detection limit of the nucleocapsid protein (NCp), purified NCp (Nordic Biosite, Sweden, Code: OOEF01087) at a concentration ranging between 500 ng to 5pg, with 10-fold dilutions was tested with both kits. Each kit buffer was used for making the dilution series. One drop of buffer was added, followed by 50uL of sample containing the protein. Both kits were tested at the same time to make it possible to directly compare the readings. After 20 min of incubation, two persons, independently of each other, made the readings. Photos were taken for documentation.

2.4. Infection of Vero E6 cells

All SARS-CoV-2 PCR positive samples were cultured on Vero E6 cells. Vero E6 cells were cultured in Dulbecco's Modified Eagle Medium (DMEM) supplemented with 10% fetal calf serum (FCS) and gentamycin. Cells were seeded as monolayer in 48-well plates and at time of confluency they were infected with the nasopharyngeal sample in a biosafety level 3 (BSL3) laboratory. Before infection the cells were washed two times with DMEM and 50uL of nasopharyngeal samples in total volume of 350uL DMEM supplemented with 2% FCS and gentamycin was added to each 48-well with cells. Samples were blind passaged twice on Vero E6 cells for 3 days each. Cells exhibiting CPE were further tested for presence of SARS-CoV-2 using the Panbio antigen test and considered as SARS-CoV-2 associated CPE if positive with the RAD. All Panbio positive CPE were also positive with the Orient gene test.

2.5. In-house SARS-CoV-2 qPCR for verification of PCR-negativity

A subset of specimens (n = 19) yielding false positive results as determined by Orient Gene Biotech test were analyzed again using a qPCR assay for the RdRp gene as described with slight modifications [11]. Briefly, RNA extraction was done using QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) according to manufacturer's instructions, and reverse transcription was carried out using iScript™ cDNA Synthesis Kit (Biorad, Solna, Sweden). The real time PCR was performed on CFX96 (Biorad) using iTaq Universal Probes Supermix (Biorad) with following cycling conditions: initial denaturation at 95 °C for 3 min, followed by 45 cycles of 95 °C for 5 s and 60 °C for 30 s. The primers (RdRp_SARSr-F and RdRp_SARSr-R) and probe (RdRp_SARSr-P2) targeting the RdRp gene of SARS-CoV-2 were used [11].

2.6. Multiplex RT-qPCR for detection of seasonal coronavirus

To test the specificity of both kits, a cell adapted strain of a human Alphacoronavirus 229E was tested with both RADs and yielded a positive signal with Orient Gene Biotech test. This led us to investigate whether the false positive results with Orient Gene Biotech using the SARS-CoV-2 PCR negative nasopharyngeal samples was due to cross-reactions with seasonal coronavirus. A subset of such samples (n = 19) was analyzed for the 4 seasonal coronaviruses, NL63, 229E, OC43 and HKU1, using a multiplex real-time PCR FTD HCoV (Fast Track diagnostics Luxembourg) on a CFX96 Touch Real-Time PCR Detection System (Biorad), according to the manufacturer's instructions.

2.7. Statistics

Sensitivity, specificity and Cohen´s Kappa with 95% confidence interval (Clopper-Pearson exact test) was calculated with SPSS using RT-qPCR results as a reference standard. Sub-analysis was made by stratifying by Ct-values and samples containing infectious virus as determined by CPE in Vero E6 cells.

2.8. Ethical statement

Samples investigated were originally collected routinely from individuals seeking testing due to COVID-19 disease related symptoms. Decoded clinical samples without person-related data and traceability, not originally taken for research purposes, does not require ethical or legal clearance according to the Swedish ethics review authority.

3. Results and discussion

In this study we have evaluated two RADs using clinical nasopharyngeal samples previously screened for SARS-CoV-2 RNA with RT-qPCR. The samples were all collected from individuals seeking testing due to COVID-19 related symptoms following protocols established by the regional health authorities in region Östergötland, Sweden. Hence, it is likely that many of the SARS-CoV-2 negative samples contained other respiratory infection agents, which are common during the time of sample collection in October and November. This makes this sample set suitable for analysis of specificity of the RADs, in contrast to using SARS-CoV-2 negative samples from asymptomatic individuals. The overall analytical results are shown in Table 1 . The Panbio RAD detected 112 of the 156 RT-qPCR positive samples, yielding a sensitivity of 71.8%. None of the tested 130 RT-qPCR negative samples were positive, yielding a specificity of 100% (Table 1). These results are similar to other studies that have evaluated the Panbio RAD with RT-qPCR as a reference standard [5,12]. The RAD from Orient Gene Biotech detected 124 of the 156 RT-qPCR positive samples, yielding a sensitivity of 79.5%. However, 45 of the 176 SARS-CoV-2 RT-qPCR negative samples analyzed were false positive, yielding of specificity of 74.4%. Most of these positives exhibited a test band intensity that was weaker than the control band intensity (+). We proceeded to re-test a subset of these (n = 35) with the Orient Gene test, yielding 19 positives. This discrepancy between tests is likely due to the observation that most false positives showed a weak test band intensity, which could lead to different results between occasions due to stochastic processes, or differences in sample storage between the two tests. This large number of false positives using the Orient Gene RAD led us to investigate whether the samples had been accurately diagnosed initially. We thus extracted RNA and performed an in-house qPCR for the RdRp gene of SARS-CoV-2 on all samples that were false positive (n = 45). All of the samples tested were negative for SARS-CoV-2, thus confirming the initial false-positive results. The specificity for this brand is markedly lower compared to previously reported for other RAD tests [4]. Furthermore, in a recent publication, the Orient Gene/Healgen test exhibited the lowest specificity of 7 commercial RADs tested (88%), and it was found to cross-react with several non-coronaviruses thus corroborating our findings [4]. The low specificity could partly be related to that all the samples examined in this study were from individuals showing signs of respiratory infections, thus increasing the likelihood of detecting cross- reactivity to other respiratory agents, as was also reported previously [4]. As the Orient Gene RAD was positive for a laboratory adapted human alphacoronavirus 229E, we investigated whether the high false positive rate could be due to cross reactivity to seasonal coronavirus infections. We performed a multiplex qPCR for seasonal coronavirus on a subset (n = 19) of the false positive samples. None of the samples analyzed were positive for any of the four seasonal coronaviruses. Thus, cross reactivity to seasonal coronavirus cannot have been the sole reason for the false positivity rate.

Table 1.

Sensitivity and specificity comparison between SARS-CoV-2 rapid antigen tests and RT-qPCR.

	Tested (n)	SARS-CoV-2 qPCR	Sensitivity (95% CI)	Specificity (95% CI)	Cohen´s kappa (95% CI)
		Positive	Negative
Panbiob	286	156	130	71.8% (64.0–78.7)	100% (97.2–100)	0.70 (0.62–0.78)
Orient Gene Biotechc	332	156	176a	79.5% (72.3–85.6)	74.4% (67.3–80.1)	0.54 (0.45–0.63)

a) 46 additional RT-qPCR negative samples were analyzed with Orient Gene Biotech rapid antigen test.

b) Abbott Panbio™ COVID-19 Ag Rapid Test.

c) Zhejiang Orient Gene/Healgen Biotech Coronavirus Ag rapid test cassette.

We subsequently performed a subgroup analysis regarding viral load as determined by the Ct-values from the RT-qPCR diagnostics. The median Ct-value of the positive samples was 21.9 (range, 10.6–39.8). This analysis showed a clear association between Ct-values and RAD sensitivity (Table 2 ), with high sensitivities for both Panbio and Orient Gene with samples having Ct-values <25, which then dropped notably for Ct-values >25. The relatively low sensitivity in the Ct-value range 25–30, 45% and 55% for the Panbio and Orient Gene test respectively, may be of concern, as this indicates a relatively high viral load that might have importance for disease transmission.

Table 2.

Assay sensitivity of the rapid antigen tests in comparison to Ct-values and sample infectivity.

	Nr positive	Sensitivity% (n)	Sample infectivity% (n)	Sensitivity/infectivity% (n)
Panbio	156	71.8 (112/156)	41.7 (65/156)	95.4 (62/65)
Ct <20	47	93.6 (44/47)	68.1 (32/47)	93.8 (30/32)
Ct 20–25	66	84.8 (56/66)	45.5 (30/66)	100 (30/30)
Ct 25–30	20	45.0 (9/20)	15.0 (3/20)	66.6 (2/3)
Ct >30	23	13.0 (3/23)	0.0
Orient Gene Biotech	156	79.5 (124/156)	41.7 (65/156)	96.9 (63/65)
Ct <20	47	97.9 (46/47)	68.1 (32/47)	100 (32/32)
Ct 20–25	66	90.9 (60/66)	45.5 (30/66)	96.7(29/30)
Ct 25–30	20	55.0 (11/20)	15.0 (3/20)	66.6 (2/3)
Ct >30	23	30.4 (7/23)	0.0

We then proceeded to assess the correlation between infectivity and RT-qPCR results. In total 65 of the 156 PCR positive (41.7%) nasopharyngeal samples developed SARS-CoV-2 associated CPE in Vero E6 cells (Table 2). As expected, the infectivity rates were strongly associated with Ct-values, with no sample with Ct >27 developing CPE in cell culture (Table 2). Previous studies have reported different results with regards to which Ct-values that corresponds to infectivity, ranging from results that are similar to what we observed (Ct-value <24), to higher (Ct-value<34) [8,10,13]. An important limitation to Ct-values, is that these cannot be readily compared between studies as these depend on numerous factors from sample collection to the type of molecular diagnostics used. To better account for estimating sensitivity of the RADs, we thus compared directly to infectivity as measured by cell culturing of the same samples used for RAD testing. Most interestingly, RAD tests correlated better with infectivity than with PCR results, with overall sensitivities of 95.4% and 96.9% for the Panbio and Orient Gene tests, respectively. These findings are of interest and indicate that RADs may have high sensitivity for detecting contagious individuals. It is however important to note that sample infectivity as determined by cell culturing cannot be directly translated whether a person is infectious, more studies are needed to assess this. However, several studies have proposed that detection of infectious virus on cell culture are more reliable than PCR to determine if a person is contagious [13] ([14]. When detection limit for the NCp protein was investigated, as little as 50pg was detected, with both RADs tested. This detection limit is in the range of what conventional enzyme linked immunosorbent assay (ELISA) can detect. ELISA is the standard method of protein detection [15] but which requires experienced laboratory personnel, advanced lab equipment and hours of analysis time.

This study has some limitations. Sample storage at +4 °C after the initial real-time PCR analysis may have contributed to lower sensitivity of the RADs. Most samples (~75%) were analyzed within two days of the PCR test, and all samples within one week. Likewise, sample storage might have influenced the infectivity results. Both tests were however run on the same samples simultaneously, enabling a direct comparison. Moreover, the observed Ct cut-off of 27 for sample infectivity is lower than that described for some other studies [8]. Different assay sensitivities could impact infectivity results, and a lower sensitivity of the infectivity assay would yield an artificially better test performance with respect to detecting infectious viruses.

Due to sample and assay availability, the Panbio was tested on 130 negative samples whereas the Orient Gene Biotech test was tested on an additional 46 negative samples (total 176). Analyzing the Orient Gene specificity on only the first 130 negative samples yielded a similar specificity as for all 176 samples, thus this likely had no major effect on the specificity results. Other limitations include the absence of clinical information from the tested persons due to anonymization of samples. Similarly, no samples from asymptomatic persons were available, and the capacity of the RADs to detect asymptomatic SARS-CoV-2 could thus not be evaluated.

To conclude, the Panbio test showed a good sensitivity for specimens with high viral load (Ct-value < 25) and for samples containing infectious viruses as determined by CPE in cell-culture. The test further had 100% specificity even though all PCR-negative samples were taken from individuals with respiratory and/or COVID-19 related symptoms, suggesting infection with other pathogens. The Orient Gene/Healgen Biotech test, although exhibiting a good sensitivity, exhibited a low specificity with these samples.

Our results thus indicate that the rapid antigen tests have a high sensitivity with regards to sample infectivity and that the Panbio´s RAD may be a valuable tool for diagnosing contagious individuals and controlling disease transmission in the ongoing pandemic.

Declaration of Competing Interest

Henrik Olsson and Mikael Jämtberg are Co-chief executive officers at Noviral Sweden AB, which is a distributor the Orient Gene Biotech and Abbott Panbio rapid antigen tests.