Comparative evaluation of four rapid SARS-CoV-2 antigen detection tests using universal transport medium

Dear Editor,

The containment of the ongoing COVID-19 pandemic requires reliable detection of COVID-19 cases, currently done by real-time reverse-transcription polymerase chain reaction (RT-PCR) [1]. The gap between the number of samples and laboratory capacities to perform RT-PCR in a timely manner, however, is a major limitation of the public health response to COVID-19 [1]. Therefore, there is a critical demand for alternative detection methods, especially rapid diagnostic tests (RDTs), which due to their ease of use might serve as point-of-care tests in community-based settings [2]. Antibody detection tests for SARS-CoV-2 are limited by the delay in humoral immune response, whereas newly developed assays targeting viral antigens have the potential for early diagnosis [2]. However, the accuracy and real-world performance of such assays is unknown and their validation is therefore of high priority [2]. Here we present a head-to-head comparison of four novel antigen-based RDTs for the detection of SARS-CoV-2 from respiratory specimens.

Two of the evaluated assays were based on classical immunochromatography and two used immunofluorescence (Table 1 ). Samples were obtained from patients presenting respiratory symptoms and/or fever between March 16 and April 26, 2020, and consisted of naso-oropharyngeal swabs placed in universal transport medium (UTM-RT® System, Copan Diagnostics, Murrieta, USA). UTM specimens were initially examined for SARS-CoV-2 by COVID-19 Genesig® Real-Time PCR (Primerdesign Ltd., Chander's Ford, UK). Exponential amplification curves with cycle threshold (Ct) values ≤ 40 were considered positive. Samples were kept at −80 °C before testing by 1) Biocredit COVID-19 Antigen Test (RapiGen Inc.), 2) StrongStep® COVID-19 Antigen Test (Liming Bio-Products Co.), 3) Huaketai New Coronavirus (Savant Biotechnology Co.), and 4) Diagnostic Kit for 2019-nCoV Ag Test (Bioeasy Biotechnology Co.). Noteworthy, the test protocol deviated from manufacturer's instructions by using an equivalent volume of UTM (instead of the recommended test buffer), as previously described [3,4]. Samples were selected by convenience among the 5276 respiratory specimens processed for SARS-CoV-2 during the study period. Due to the shortage of test kits, a 2:1 distribution of positive to negative samples was chosen. Seventeen positive specimens had been used in a previous evaluation [3].

Table 1.

Characteristics and performance of four rapid SARS-CoV-2 antigen-detection tests.

Characteristics	Test N°1	Test N°2	Test N°3	Test N°4
Commercial name	Biocredit One Step SARS-CoV-2 Antigen Test	StrongStep® COVID-19 Antigen Test	Huaketai New Coronavirus (SARS-CoV-2) N Protein Detection Kit (FIA)	Diagnostic Kit for 2019-Novel Coronavirus (2019-nCoV) Ag Test (FIA)
Manufacturer	RapiGen Inc., Anyang-si, Gyeonggi-do, Rep. of Korea	Liming Bio-Products Co., Jiangsu, China	Savant Biotechnology Co., Beijing, China	Bioeasy Biotechnology Co., Shenzhen, China
Catalogue N° (lot N°)	G61RHA20 (H073001SD)	500200 (2003014)	BCT-HKT-050 (20031501)	YRLF04401025 (2002N408)
Certificationa	CE-IVD	CE-IVD	CE-IVD	CE-IVD
Primary specimenb	NP/OP swab	NP/OP swab	Throat swab	NP/OP swab, sputum
Incubation (ambient)b	5–8 minutes	15–20 minutes	15 minutes ± 1 minute	10 minutes ± 0 minutes
Readoutb	Visual: coloured bands	Visual: coloured bands	Visual: fluorescent bandsc	Automated: fluorescence reader
Performanced
Sample size (n)	109e	19f	109e	111
Sensitivity	62% (49/79) CI95% 51–71.9	0% (0/9) CI95% 0–29.9	16.7 (13/78) CI95% 10–16.5	85% (68/80) 75.6–91.2
Specificity	100% (30/30) CI95% 88.7–100	90% (9/10) CI95% 59.6–98.2	100% (31/31) CI95% 89-100	100% (31/31) CI95% 89-100
Accuracy	72.5%	47.4%	40.4%	89.2%
Kappa coefficient	0.5	−0.1	0.1	0.8
Sensitivity, high VLg	84.9% (45/53) CI95% 72.9–92.1	NA	21.2% (11/52) CI95% 12.2–34	100% (54/54) CI95% 93.4–100
Sensitivity, low VLh	15.4% (4/26) CI95% 17.5–37.7	NA	7.7% (2/26) CI95% 2.1–24.1	53.8% (14/26) CI95% 25.5–37.4
Mean Ct of false negatives (range)	29.6 (17.5–37.7)	NA	21.9 (10.7–37.7)	34.4 (25.5–37.4)

FIA, fluorescence immune assay; NP, nasopharyngeal; OP, oropharyngeal; UTM, universal transport medium.

^aAccording to https://www.finddx.org/covid-19/pipeline.

^bAccording to manufacturer's recommendation.

^cUsing UV flashlight recommended and provided by manufacturer.

^dStudy protocol included deviation from manufacturer's instructions (see text).

^eTwo invalid results were excluded.

^fTesting was suspended after 19 samples due to poor test performance.

^gSamples with high viral loads (Ct ≤ 25).

^hSamples with low viral loads (Ct > 25).

Assays were tested in parallel from the same sample, performed under BSL2 conditions by the same trained technician, who was blinded to RT-PCR results. Assays with visual output were read by two independent observers, conferring with a third in case of disagreement. RT-PCR served as reference method; for samples with discordant result, tests were repeated. Demographic and clinical data were obtained from mandatory notification forms and analysed anonymously. Samples with high viral loads (Ct value ≤ 25) were compared to those with low viral load (Ct values > 25), as previously described [4]. Statistical analysis considered sensitivity, specificity, accuracy, and Kappa coefficient using standard formulas, and Wilson score Confidence Interval at 95% (OpenEpi version 3.01, GraphPad Prism version 8.4.2). Study materials were purchased with laboratory funds, except for Savant RDT, which was provided free-of-charge through a local provider. The study was approved by the institutional review board (Comité Ético Científico, Facultad de Medicina Clínica Alemana, Universidad del Desarrollo, Santiago, Chile) and need for informed consent was waived.

The study included a total of 111 samples from symptomatic patients; 55% were female, with a median age of 40 years. Eighty specimens were RT-PCR positive, representing 22% of all positives during the study period; 31 samples were RT-PCR negative. The median duration from symptom onset to sampling was 2 days (IQR 1–5 days); 88% of specimens (96/109; missing data, n = 2) were taken during the first week of symptoms. Ct values ranged from 10.7 to 37.7 (mean, 22.5). Test performances showed significant differences (Table 1). The evaluation of the Liming Bio kit was stopped after 19 samples, due to its poor results. The other three assays had sensitivities ranging from 16.7% (Savant) to 85% (Bioeasy) and a specificity of 100%. Sensitivities were significantly higher in specimens with high viral loads (Ct values ≤ 25) for RapiGen (84.9%) and Bioeasy (100%) (Table 1). Concordance between these two tests was 82%, while their agreement with Savant was 67% and 50%, respectively. The visual readout of RapiGen was clear, regardless of the intensity of bands. The interpretation of Savant, requiring a UV flashlight provided by the manufacturer, was difficult; its sensitivity might have been higher using an automated reader. Bioeasy cassettes were interpreted by a desktop instrument with options for QR coding, printing, connectivity to laboratory information systems. Overall, the three systems were easy to use and gave a qualitative result in 10–20 minutes.

Although our study directly compared the assays from the same sample material, the off-label use of UTM might have influenced test results. However, some of the assays showed favourable overall sensitivities, suggesting the potential use of antigen-based RDTs as alternative (or adjunct) tools to RT-PCR. As in other studies [3,5], the performance was significantly higher in specimens with high viral loads (Ct ≤ 25). Since culture studies have shown a significant reduction of infectivity with low viral counts (Ct > 24) [6], antigen testing might play a crucial role within strategies aiming to determine the contagiousness of infected individuals.

In conclusion, the study demonstrated a significant heterogeneity of test performance, which might have been influenced by the use of UTM as a non-validated sample material. The results emphasize that rapid antigen detection has the potential to serve as an alternative diagnostic method, especially as a screening tool for patients with high viral loads during early and infective stages of infection.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CRediT authorship contribution statement

T. Weitzel: Conceptualization, Formal analysis, Methodology, Project administration, Validation, Writing - original draft, Writing - review & editing. P. Legarraga: Data curation, Formal analysis, Supervision, Validation, Writing - review & editing. M. Iruretagoyena: Data curation, Formal analysis, Validation, Writing - review & editing. G. Pizarro: Data curation, Investigation, Writing - review & editing. V. Vollrath: Supervision, Validation, Writing - review & editing. R. Araos: Validation, Writing - review & editing. J.M. Munita: Validation, Writing - review & editing. L. Porte: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing - review & editing.

Declaration of competing interest

There is no conflict of interest.