Rapid COVID‐19 antigenic tests: Usefulness of a modified method for diagnosis

Abstract

The current reliable recommended test for coronavirus disease 2019 (COVID‐19) diagnosis is quantitative reverse‐transcription polymerase chain reaction (RT‐qPCR). Rapid antigen test devices could be useful as they are less expensive, faster without the need of specialized laboratories to perform the test. We report the performances of two rapid immunochromatographic antigen testing devices compared with RT‐qPCR for severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) detection in nasopharyngeal samples. We carried out a lateral‐flow tests study on 401 nasopharyngeal swab samples from nonduplicated suspected COVID‐19 subjects. An equal volume of universal transport medium tubes‐containing samples (dilution ratio = 1:15) were added to the manufacturer's extraction buffer solution (dilution ratio = 1:2) and analyzed on BioSpeedia COVID19Speed‐Antigen Test and on Abbott Panbio™ COVID‐19 Ag Rapid Test, devices. Qualitative results were compared to those obtained by the RT‐qPCR (Allplex™ SARS‐CoV‐2 Assay Seegene). Based on our data, the overall sensitivity for BioSpeedia and Panbio devices was estimated at 65.5% and 75.0%, respectively. The sensitivity was greater for cycle threshold values less than 25 achieving 90.4 and 96.8 for BioSpeedia and Panbio devices, respectively. A perfect specificity of 100.0% was observed for both devices.

Highlights

We evaluated the performances of two rapid COVID‐19 antigen tests, namely BioSpeedia COVID19Speed‐Antigen Test and Abbott Panbio™ COVID‐19 Ag Rapid Test, during the second wave of COVID‐19 pandemic in Belgium. For this purpose, 401 nasopharyngeal swab samples from suspected COVID‐19 subjects were analyzed on both devices with a modified method and the results were compared with those obtained on the routine platform RT‐qPCR (Seegene, Allplex™). Our data showed a high sensitivity (>90.0%) for samples with high viral load (Ct < 30 and particularly Ct < 25). Based on our data, both devices could be reliable and efficient alternative tests during COVID‐19 outbreaks, particularly in a high prevalence population, when there is limited availability of molecular tests to quickly rule in positive patients.

1. INTRODUCTION AND OBJECTIVES

The diagnosis of coronavirus disease 2019 (COVID‐19) appeared to be challenging as the quantitative reverse‐transcription polymerase chain reaction (RT‐qPCR) is the only reliable method ¹ to detect an ongoing infection which is expensive, time‐consuming, and laborious to implement needing specialized laboratories to perform the assay. ² In recent studies, although less sensitive, rapid antigen tests proved to be a more convenient and less expensive method to provide faster diagnosis, particularly in the first days after onset of symptoms.3, 4 The aim of our study was to compare the analytical performance of two rapid antigen devices namely BioSpeedia COVID19Speed‐Antigen Test (BS) and Abbott Panbio™ COVID‐19 Ag Rapid Test (PB) with RT‐qPCR on clinical samples for COVID‐19 diagnosis.

2. MATERIAL AND METHODS

2.1. Studied subjects and samples

A lateral‐flow tests study was conducted on 401 nonduplicate nasopharyngeal swab samples collected in universal transport medium tubes (UTM, COPAN®, 3.0 ml tubes, Copan Italia S.p.a., Brescia, Italy) obtained from a mixture of symptomatic, presymptomatic, pauci‐ and a‐symptomatic individuals, (index) cases and contacts between November 13 and December 9, 2020. ⁵ The incidence (14 days) per 100,000 inhabitants in Belgium and in the province of Namur were 1232 and 1949 cases, respectively. The number of samples required for the accuracy assessment was calculated based on the Belgian population (11.5 million) COVID‐19 prevalence in November 2020, (about 4.5%), the probability of a positive test/run (more than 40%), an α = 0.05 and a β = 0.1 (confidence interval [CI] of 95%). ⁶ Our study fulfilled the ethical principles provided by the Declaration of Helsinki.

2.2. Rapid COVID‐19 antigen devices

Selected samples were tested on BS and PB devices, two CE‐IVD devices before the RT‐qPCR routine assay. All tests were immediately performed after sampling. The principle of both tests was similar. Briefly, the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) antigen would react with gold‐conjugated antibody and form an immune complex that will migrate and be captured by anticonjugate antibodies pre‐coated in the test region. If SARS‐CoV‐2 antigen is present in a sample, a purple‐colored line would appear in the test line region, indicating a positive result (Figure S1A). Even a weak line should be considered as a positive result. To validate a test result, a colored line always must appear in the control line region (Figure S1B). We developed a modified method to use the primary sample collected for RT‐qPCR avoiding any additional sampling. To that end, 200 μl of samples in UTM tubes (3.0 ml; 1:15 dilution ratio) were added to 200 μl (8 drops) of the related extraction buffer into the extraction tube. (1:2 dilution ratio). The final mixture had an estimated dilution ratio of 1:30. After closing the tube caps and a brief vortex mix, two drops of the mixed solution for BS and five drops for PB were dispensed to the drop sample region of each device. Reading was performed at 15, 20, and 30 min after dispensing mixed solutions by two technologists.

2.3. RT‐qPCR assay

The COVID‐19 diagnosis was confirmed by using the coronavirus Allplex™ SARS‐CoV‐2 Assay (Seegene Inc.) on the nasopharyngeal swab samples in UTM targeting the RNA‐dependent RNA polymerase (RdRp)/Spike (S), Envelope (E), and Nucleocapsid (N) genes. The extraction of the viral RNA was carried out on the Seegene STAR^LET and the amplification was performed on the CFX 96 thermal cycler (BioRad). The amplification parameters were as follow: 20 min at 50°C for reverse transcription, 15 min at 95°C for activation, followed by 44 cycles of 15 s at 94°C, and 30 s at 58°C. A cycle threshold value less than 39 for the four genes, was defined as a positive test.2, 7 We calculated the means of Ct value as the main quantitative reference results and to classify positive samples into high (Ct < 25), moderate (Ct ≥ 25 and < 30) and low (Ct ≥ 30) viral load groups. A result with Ct value ≥ 39 was considered as a negative.

2.4. Statistical analysis

Means and 95% CIs, median and interquartile rates were determined in obtained results. The agreement between the methods was determined by the kappa index. A p < 0.05 was considered as statistically significant. Data analysis was performed using GraphPad Prism software (Version 8.0).

3. RESULTS

Of 401 samples, 232 (58%) were detected positives by RT‐qPCR including 156 samples showing high viral load with means of detected E, RdRP/S, and N genes Ct values of 17.6, 19.0, and 18.0, respectively. Of 232 samples, 40 displayed moderate viral loads with means of detected E, RdRP/S, and N genes Ct values of 27.7, 28.1, and 28.5, respectively. Thirty‐six samples had a low viral load (≥ 30) with means of detected E, RdRP/S, and N genes Ct values of 34.9, 34.8, and 34.5, respectively. All samples were tested positive for cited targets with a difference less than two Ct for high and moderate viral loads.

BS device detected 141 true‐positive for Ct values < 25 (n = 156 true positive), while PB device detected 151 true positive (sensitivity = 90.4 vs. 96.8). If considering samples having Ct values < 30, BS device detected 152 (n = 196) and PB detected 174 (77.6% vs. 88.8%). For the group of samples with Ct values ≥ 30 (n = 36), none of the two devices detected true positives. Both devices showed a high specificity without any false‐positive result. Based on RT‐qPCR results for Ct < 25, the degree of agreement between methods was almost perfect for BioSpeedia and Panbio devices with kappa value estimated at 0.91 and 0.97, respectively. For Ct values < 30, BS had a substantial agreement with an estimated kappa = 0.76 (0.70–0.83), while an excellent agreement was retained by PB showing a calculated kappa of 0.88 (0.83–0.93).

Regarding the time to positivity after dispensing samples onto the test device, BS gained sensitivity by increasing the reading time from 15 to 20 min for Ct values < 25 (137/156 to 141/156 true positives) and for Ct values between 25 and 30 (8/40 vs. 11/40). For PB, while no difference in detection by increasing reading times for Ct values < 25 was observed, a gain of sensitivity was noticed (18/40 vs. 23/40) for Ct values between 25 and 30. For both tests, we did not observe an additional gain of sensitivity nor any false positivity by increasing reading times up to 30 min. The accuracies for BS device were estimated at 95.4% and 88.0% for Ct values < 25 and < 30, respectively. The accuracies for PB device were estimated at 98.5% and 94.0% for Ct values < 25 and < 30, respectively. All analytical findings are summarized in Table 1.

Table 1.

Associations and discrepancies according to COVID‐19 gold standard diagnosis assay (232/401 positive RT‐qPCR)

Sample groups n = 401	Analytical parameters	BioSpeedia COVID19Speed‐antigen test	Panbio™ COVID‐19 Ag rapid test
Ct values < 25 n = 156 true positives	Sensitivity % (95% CI)	90.4 (84.6–94.5)	96.8 (92.7–99.0)
	Accuracy % (95% CI)	95.4 (92.5–97.4)	98.5 (96.5–99.5)
	NPV % (95% CI)	91.9 (87.4–94.8)	97.1 (93.5–98.8)
	Kappa (95% CI)	0.91 (0.86–0.95)	0.97 (0.94–1.00)
Ct values < 30 n = 196 true positives	Sensitivity % (95% CI)	77.6 (71.1–83.2)	88.8 (83.5–92.8)
	Accuracy % (95% CI)	88.0 (84.2–91.1)	94.0 (91.0–96.2)
	NPV % (95% CI)	79.3 (74.8–83.3)	88.5 (83.8–91.9)
	Kappa (95% CI)	0.76 (0.70–0.83)	0.88 (0.83–0.93)
Overall Ct values n = 232 true positives	Sensitivity % (95% CI)	65.5 (59.0–71.6)	75.0 (68.9–80.4)
	Ct values median (IQR)	22.12 (18.30–27.74)
	Overall specificity % (95% CI)	100.0 (97.8–100.0)
	Accuracy % (95% CI)	80.1 (75.8–83.9)	85.5 (81.7–88.8)
	Overall PPV % (95% CI)	100.0 (97.8–100.0)
	NPV % (95% CI)	67.9 (63.9–71.6)	74.5 (70.0–78.5)
	Kappa (95% CI)	0.62 (0.55–0.69)	0.72 (0.65–0.78)

Note: Overall, 152 and 174 samples were detected as positive by BioSpeedia and Panbio devices, respectively. Of 232, 156 samples showed Ct values < 25. Of 156, 141 samples were correctly detected by BioSpeedia cassette and 151 others were correctly classified by Panbio device. 196 (n = 232) showed Ct values < 30, BioSpeedia cassette detected 152 samples and 174 were correctly classified using Panbio device. The sensitivity and specificity were defined as the percentage of true positive and true negative results correctly identified by devices, respectively. The PPV and NPV were defined as the probability of positive and negative results among true patients and healthy volunteers, respectively. Accuracy was defined as the closeness of the obtained results to an expected value.

Abbreviations: 95% CI, 95% confidence interval; Ct, cycle threshold; COVID‐19, coronavirus disease 2019; IQR, interquartile range; NPV, negative predictive value; PPV, positive predictive value; RT‐qPCR, quantitative reverse‐transcription polymerase chain reaction.

4. DISCUSSION

Here, we showed that rapid antigen devices could be useful during COVID‐19 outbreak. BS and PB appeared to be reliable for Ct values < 25, less sensitive for Ct values between 25 and 30 and showed a poor sensitivity for Ct values > 30. (Table 1) However, we showed both devices were more reliable than the first commercialized rapid antigen devices such as Coris COVID‐19 Ag Respi‐Strip Test, Standard Q COVID‐19 Antigen Test, QuickNavi COVID‐19 Antigen, Espline SARS‐CoV‐2, and ImmunoAce SARS‐CoV‐2. Indeed, previous studies reported a lack of sensitivity (up to 30.0%) for several devices.4, 8 In a recent study using PB device, the sensitivity on 1575 nasopharyngeal swab samples was estimated between 95.0% and 98.0% for the Ct values < 32. ⁹ In another study on 255 nasopharyngeal swab samples, PB showed sensitivities of 100.0% and 87.5% for Ct values < 25 and < 30, respectively. ¹⁰ Considering the dilution factor and that we used UTM for testing, we concluded that our results were in line with previous studies. If considering BS results, the device appeared to be highly performant to detect positive samples with Ct values < 25. Both devices showed a perfect specificity without any false positive reported. This result should be interpreted cautiously as the incidence of COVID‐19 cases was high at the time of the study. The agreements between Panbio device and RT‐qPCR were excellent for all positive samples with Ct values < 30, while those between BP device and RT‐qPCR were excellent for Ct values < 25 and substantial for Ct values < 30 confirming the reliability of both tests. We recommend to read the device between 20 and 30 min after dispensing samples to prevent early false‐negative results. Moreover, later device reading (between 20 and 30 min) did not impact the results. Our results showed that UTM tubes could be used on such devices for COVID‐19 diagnosis. Our study has limitations. First, we carried out this study during the second surge of the COVID‐19 outbreak in Belgium. Hence, we could not gather clinical data because of the mass screening strategy and the large number of samples that were needed to be handled every day. We also could not determine the exact sampling time after the onset of symptoms due to the heterogeneity of clinical data. Nevertheless, based on Ct values and obtained results, rapid antigen devices appeared more useful in the first 5 days after the symptom onset when the viral load is high. Such a hypothesis has been confirmed in recent studies.3, 10, 11 Then, we assumed that the lack in sensitivity, compared with other studies, for Ct values ≥ 30 could be explained partially by the dilution factor (1:30). Indeed, we could not study the impact of sample's dilution as we did not collect swabs directly into the extraction buffer tube. Finally, our study was conducted in a high prevalence population, during the second surge, to evaluate the antigenic test device's accuracy for faster diagnosis/isolation of positive subjects. Hence, further studies should be carried out in a low prevalence population to evaluate if such devices could be considered for use in mass detection strategy. ⁵

5. CONCLUSION

Rapid antigenic tests are attractive user‐friendly alternatives to a more sensitive but laborious RT‐qPCR, especially in times of pandemic where molecular testing capacity may be saturated. Our modified method of using these tests (using UTM tubes) could be easily integrated into the laboratory workflow without impacting the preanalytical process.

Given the high specificity of devices, a positive result would not require confirmation by molecular test thus allowing faster management of COVID‐19 positive patients. However, negative results must be retested by a molecular method. Biospeedia and Panbio devices appeared to be reliable and could be used in these situations.

Further studies should evaluate the performance of these devices used in off‐laboratory on‐site settings by qualified personnel in collectivities (such as nursing homes or schools) or as part of a mass screening strategy. Such a setting could achieve maximal sensitivity by avoiding the dilution process.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

All the authors have approved the entire content of the submitted manuscript and any subsequent revised version and have accepted responsibility for the entire work. Reza Soleimani (Clinical Pathology resident, Medical Microbiology Service): Conceived the project (methodology, validation protocol, and samples collections), collected the data, and wrote the manuscript in collaboration with Corentin Deckers, reviewed the literature, and responded to the reviewers. Corentin Deckers (Clinical Pathology Resident, Medical Microbiology Service): Reviewed and gathered all results and wrote the abstract. Te‐Din Huang and Olivier Denis (Clinical Pathologists, Head of Medical Microbiology Service): Supervised the whole project, methodology, medical validation, manuscript preparation, responses to reviewers, and covered costs of the work/publication. Pierre Bogaerts (PhD, Head of Molecular Biology Service), implicated in technologists training, validation of RT‐qPCR, and revised the manuscript. Stéphanie Evrard, GN, Isaline Wallemme, Boutaina Habib, and Pauline Rouzé (laboratory technologistes) carried out and read all tests (RT‐qPCR and antigenics).

Supporting information

Supporting information.

Soleimani R, Deckers C, Huang T‐D, et al. Rapid COVID‐19 antigenic tests: Usefulness of a modified method for diagnosis. J Med Virol. 2021;93:5655‐5659. 10.1002/jmv.27094