Evaluation of the Boson rapid Ag test vs RT–PCR for use as a self–testing platform

Abstract

The gold standard test available for detecting COVID–19 patients is Real Time RT–PCR. However, this method is expensive, needing special equipment and skilled laboratory staff. Recently, less expensive antigen tests have become available, that could easily and rapidly identify new COVID–19 cases. Our objective was to evaluate the Boson Rapid Antigen Test Card versus the RT–rtPCR, using samples taken both by laymen (self–testing) and professionals. The sensitivity, specificity and accuracy rates were, 98.18%, 100.00%, and 99.28%, respectively. The positive and negative predictive values were 100.00% and 98.82%, respectively. The detection rate for asymptomatic patients was 90.48%, and detection rate for Ct values ≥30 was 91.67%. Our results indicate a high coincidence rate between the Boson and the referencing RT–rtPCR method, meeting the performance standards recommended by the WHO. Therefore, this test could facilitate a fast self–testing screening method, for the detection of infected individuals.

1. Introduction

A new coronavirus, the SARS–CoV–2 was detected in Wuhan, China on December 2019 [1]. On January 2020, the World Health Organization (WHO) announced that “COVID–19″ was a public health emergency of international concern (PHEIC), followed by its declaration, shortly after (March 2020), as a world pandemic [2], [3], [4].

SARS–CoV–2 virus belongs to the beta (β) coronaviruses (Nidovirales order, Coronaviridae family), which are polymorphic, crown–like, enveloped viruses, approximately 60 to140 nm in diameter, with a positive sense (+) single–stranded RNA (ssRNA) [5,6]. The virus shows more than 85% nucleotide sequence homology with the bat SARS–like coronaviruses [6], [7], [8], [9]. The virus encodes 4 main structural proteins. The nucleocapsid (N) is involved in viral RNA synthesis and packaging, the membrane (M) protein forms the viral core, by binding to internal nucleoproteins and interacts with the envelope (E) and spike (S) proteins to form the viral envelope [8,[10], [11], [12], [13], [14]]. Moreover, viral entry is facilitated by the heavily glycosylated S protein, binding to the human angiotensin–converting enzyme 2 (ACE–2) [15], and Heparan sulfate as an assisting cofactor, thus making it a major target for detection and therapy [16].

SARS–CoV–2 symptoms range from asymptomatic to symptomatic, with an incubation period between 1 to 14 days (usually 3 to 7 days). Major mild symptoms include fever, cough, sore throat, myalgia and nasal congestion. More severe symptoms include dyspnea, hypoxemia and pneumonia, with critical cases progressing rapidly into acute respiratory and multiple organ failures [[5], [6], [7],14,18]. One in three individuals with SARS–CoV–2 is asymptomatic representing a large proportion of unidentified cases. Rapid tests can effectively be used to target asymptomatic individuals to allow early identification and proactive separation from susceptible hosts [19].

Nucleic acid amplification tests (NAATs), such as reverse transcription real–time PCR (RT–rtPCR) are the gold standard for the detection of SARS–CoV–2 (high sensitivity and specificity), they are complex, time consuming and expensive, and require high–end laboratory infrastructure with highly skilled staff [18,20]. Recently, specific Rapid Antigen Detection (RAD) tests, based on sandwich immunoassay and lateral flow antigen assays (LFAs) have become available for the detection of SARS–CoV–2, which are simple, cheap and fast to execute [17, 18, 21, 22] and can be used at point–of–care (poc) settings [23]. Usually, RAD tests identify the nucleocapsid (N) or the spike proteins (S1 and S2) of SARS–CoV–2 virus [24]. The Boson Rapid SARS–CoV–2 Antigen Test Card (Xiamen Boson Biotech Co. Ltd, Fujian, P.R. China) is designed for the rapid qualitative determination of SARS–CoV–2 virus antigen in nasal swabs (NS) at 15 to 20 minutes, from individuals suspected of COVID–19. The purpose of the present study was to evaluate the clinical performance of the Boson RAD test in nasal swabs, comparing self–tested participants (laymen), in comparison to the gold standard RT–rtPCR and nasopharyngeal (NP) samples taken by healthcare workers (professionals).

2. Experimental design

2.1. Subject selection criteria and sample size

In the study population, 833 individuals were included, from all age groups and educational backgrounds. The criteria for including participants were: a) Symptomatic individuals (within 7 days of onset) who were suspected of Covid–19. b) Asymptomatic individuals without a known SARS–CoV–2 exposure. c) Parents or legal guardians consent form for children's participation (as required by law). The main criteria for exclusion from the study were: (1) Previous experience of self–collection or self–testing. (2) Prone to nosebleeds. (3) Unable to provide informed consent form due to various disabilities. (4) Duplicate enrolment. Furthermore, rejection criteria from the study were: (1) The normal test was not completed due to equipment or operation factor problems (i.e., the sample was contaminated during operation), for both the RAD test and RT–rtPCR procedures. (2) PCR extracts were stored in a liquid format for more than 24 hours before freezing. (3) PCR extracts were repeatedly used (frozen and thawed) for more than 3 times. Each subject was informed about the clinical trial and, for participating, they were asked to sign an informed consent.

2.1.1. Laymen patients

Laymen patients included those without a medical or related background and/ or knowledge of how to use rapid antigen tests. The experimental design was according to the requirements of “Template for Manufacturers of Molecular and Antigen Diagnostic COVID–19 Tests for Non–Laboratory Use” by the US FDA [25]. Asymptomatic patients were identified as those with no relevant clinical symptoms (such as fever, cough, sore, throat and other symptoms or signs that are self–perceived or clinically recognizable) 24 hours prior to and after sample collection.

2.1.2. Professionals

Professionals included the clinical laboratory staff. They were responsible for collecting NP samples and for carrying out testing with the RT–rtPCR test kits, “blind” to the results obtained by the Boson RAD test.

2.2. Locations

Specimen collection took place in 2 sampling sites of the Locus Medicus S.A. diagnostic center, Athens, Greece, from May 2021 to July 2021. Prevalence of SARS–CoV–2 in Greece on 31st of July 2021 was 4.76%, with a cumulative incidence ratio of 1.48% during the study period. These values were estimated by consolidating data from various databases [26], [27], [28], [29]. In the first, a specific isobox was used, with 4 different points for self–sampling, in a way that privacy was ensured. In the second, an indoor place within the laboratory was used to accommodate participants. Eight different areas were set, each one divided with curtains to ensure visual and acoustic privacy.

2.3. Methods

In this clinical trial, the BOSON Rapid SARS–CoV–2 Antigen Test Card (Xiamen Boson Biotech Co. Ltd., Fujian, P.R. China) was used for the rapid qualitative detection of SARS–CoV–2 virus antigen in anterior NS. The results were compared in parallel (blind) with a gold standard RT–rtPCR, the SACACE SARS–CoV–2 Test Kit (Sacace Biotechnologies Srl, Italy). The specific experimental process followed is indicated in Fig. B.1 .

Fig. B.1.

Flowchart of clinical trial.

2.3.1. Viral RNA extraction and detection using RT–rtPCR

The NP samples collected by professional staff were immediately forwarded to the Biosafety level II premises of the laboratory for viral RNA extraction using the QIAamp Viral RNA Mini kit (Qiagen, Germany) (Supplementary Material, S1). The RT–rtPCR followed, targeting the E and N genes specific for SARS–CoV–2 virus and a region of E gene common for all SARS–like coronaviruses (SARS–CoV, SARS–CoV–2), using a Real Time thermal cycler (SaCycle–96) according to the manufacturer's instructions (SARS–CoV–2 Real–TM Handbook, Sacace Biotechnologies, Italy) (Supplementary Material, S2). Reverse transcription and amplification were performed in a single, one step reaction of 50 cycles. An internal control (synthetic RNA) was added prior to extraction, that fluorescence's in different channel, in order to confirm the validity of negative results (absence of PCR inhibitors and proper RNA extraction). Additionally, the method used was subjected to reproducibility, repeatability and Limit of Detection (LOD) studies (Accuplex SARS–CoV–2, reference material 0505–0126, 5000 copies/mL) and the LOD was set to 300 copies/mL. The method was accredited according to ELOT IEC 15189:2012 standards.

2.3.2. Boson Rapid SARS–CoV–2 antigen test card

The self–testing procedure on site was observed by a professional or a principal physician, without any influence by them. Children <14 years old were assisted by an adult. The participants were asked to follow the manufacturer's IFU on how to collect the NS without any time restrictions, perform the test and interpret the results. Briefly, after opening the lid of the extraction buffer bottle, the participants squeezed the buffer into the extraction tube. Then, the swab was inserted, no less than 2.5 cm, into both nostrils and rolled 3–4 times along the mucosa. Then, it was placed into the extraction tube, rolled 5 times and left in the buffer for 1 minute. The extraction tube was pinched with both fingers, while holding the swab, in order to clear any remaining solution, and then the swab was removed. Lastly, the nozzle cap was installed onto the extraction tube, card was removed from the protective cover and 3 drops of the test specimen were added into the specimen well (S). The result was read at 15 to 20 minutes and was considered positive, when both control (C) and SARS–CoV–2 antigen (T) lines appeared (Supplementary Material, S3). The methodology of the test is an immunochromatic based, one step in vitro test and the principle is described in detail in Supplementary Materials (S4).

2.4. Statistical analysis

Data were analyzed using SPSS (IBM Corp.) by comparing the performance characteristics of the Boson Rapid SARS–CoV–2 Antigen Test Card versus the SACACE RT–rtPCR, in NS and NP samples from symptomatic and asymptomatic individuals. Furthermore, Sensitivity (PPA), Specificity (NPA), Accuracy (OPA), Positive (PPV), and Negative (NPV) predictive values were calculated according to the formulas presented in Supplemental Material (S5). The 95% confidence intervals (CIs) were calculated using the normal approximation formula: $\widehat{p}\pm z\times \sqrt{\frac{\widehat{p}(1-\widehat{p})}{n}}$, where p is PPA, NPA, OPA, PPV or NPV and z = 1.96. Continuous variables are presented by mean ± standard deviation (SD) and analyzed using t tests or their non–parametric equivalent (Mann–Whitney), where applicable. The Pvalue was set at <0.05.

3. Results

A total of 833 samples were taken for the clinical trials. Distribution analysis showed that the participants’ age ranged between 9 to 85 years, with 444 being males (53.30%) and 389 (46.70%) females. The mean ± SD of age of the 330 positive cases for the SARS–CoV–2 virus was 32.94 (SD = 1.11). 455 (54.60%) individuals had a college or above educational background and approximately half of the samples were taken at either sampling site location (Supplementary Material, S6).

The performance characteristics of the Boson Rapid SARS–CoV–2 Antigen Test Card were evaluated. RT–rtPCR identified 39,62% positive (330/833) and 60.38% (503/833) negative cases for SARS–CoV–2 virus, respectively. The Boson Rapid Test, identified the presence of the virus in 324 cases, with a sensitivity and specificity rate of 98.18% (324/330; 95% CI: 96.74%–98.62%) and 100.00% (503/503; 95% CI: 99.90%–100.00%), respectively. Total accuracy was calculated at 99.28% (827/833; 95% CI: 98.71%–99.85%). The overall PPV and NPV of the evaluated reagent against the RT–rtPCR were 100.00% (324/324; 95% CI: 99.88%–100.00%) and 98.82% (324/330; 95% CI: 97.88%–99.76%), respectively (Table A.1 ).

Table A.1.

Performance characteristics of the Boson Rapid SARS–CoV–2 Antigen Test Card versus the SACACE RT–rtPCR.

Evaluated Reagent Results	RT–rtPCR
	Positive	Negative	TOTAL
Positive	324	0	324
Negative	6	503	509
Total	330	503	833
Sensitivity	98.18%(96.74%–99.62%)a
Specificity	100.00%(99.90%–100.00%)
Total coincidence rate	98.28%(98.71%–99.85%)
Positive predictive value	100.00%(99.88%–100.00%)
Negative predictive value	98.82%(97.88%–99.76%)

^a95% confidence intervals shown in parentheses.

An independent samples t test was conducted to compare the Ct values of the RT–rtPCR, between the true positive (TP) and false negative (FN) cases of the Boson RAD test. There was a significant difference in the scores for the TP (M = 22.83, SD = 7.29) when compared to the FN (M = 36.39, SD = 3.51) cases [t (328) = 5.831, P< .001, Fig. B2 ]. Moreover, the detection rate of the RAD test for cases that had Ct values ≥30 was identified at 91.67% (66/72).

Fig. B.2.

Distribution of Cycle threshold (Ct) values corresponding of the Boson Rapid SARS–CoV–2 test. The false negative cases had a significantly higher Ct values (36.39), when compared with the true positive cases (22.83). Horizontal bars and X mark represent the median and mean values, respectively. Note: Ct values on the y axis represent the mean value taken from all three Ct values of the targets (E and N genes specific for SARS–CoV–2 virus and a region of E gene common for all SARS–like coronaviruses) of each case.

In this clinical trial, for the 330 confirmed positive cases by RT–rtPCR (asymptomatic = 42; symptomatic = 288), the onset of symptoms was 0 to 7 days. A 0 day of symptom onset referred to a patient that did not exhibit any COVID–19 related symptoms prior to and during sample collection. Forty two individuals did not develop any symptoms 24 hours prior to, during and after the sampling collection. Therefore, these cases were classified as asymptomatic and were included at Day 0. The Boson Rapid SARS–CoV–2 Antigen Test Card was able to detect 38 asymptomatic cases with a detection rate of 90.48% (38/42; 95% CI: 81.60%–99.35%). For the symptomatic patients on Day 0 (122), the detection rate of the rapid test was 99.18% (121/122; 95% CI: 97.58%–100.00%). Overall detection rate on Day 0 was 96.95% (159/164; 95% CI: 94.32%–99.58%). At 1 day of symptom onset the detection rate was 98.77% (80/81; 95% CI: 96.36%–100.00%). The detection rates for 2 to 7 days of symptom onset were 100.00% (all symptomatic) (Table A.2 ).

Table A.2.

Performance characteristics of the Boson Rapid SARS–CoV–2 Antigen Test Card in relation to the SACACE RT–rtPCR for different days of symptom onset.

Day (s) of symptom onset	Positive cases by real time RT–PCR	Positive cases by rapid SARS–CoV–2 antigen test card	Detection rate
0	Asymptomatic: 42	38	90.48%(95% CI: 81.60%–99.35%)
	Symptomatic: 122	121	99.18%(95% CI: 97.58%–100.00%)
	164	159	96.95%(95% CI: 94.32%–99.58%)
1	81	80	98.77%(95% CI: 96.36%–100.00%)
2	34	34	100%(95% CI: 99.65%–100%)
3	18	18	100%(95% CI: 99.53%–100%)
4	15	15	100%(95% CI: 99.48%–100%)
5	9	9	100%(95% CI: 99.34%–100%)
6	5	5	100%(95% CI: 99.11%–100%)
7	4	4	100%(95% CI: 99.01%–100%)
TOTAL	330	324	98.18%(95% CI: 96.74%–99.62%)

4. Discussion

This clinical study evaluated the diagnostic accuracy and characteristics of the BOSON Rapid SARS–CoV–2 Antigen Test Card (self–testing) against the referencing reagent SACACE RT–rtPCR. The study was conducted on 833 participants, both symptomatic and asymptomatic. To our knowledge, we are the first to demonstrate the clinical performance characteristics of this RAD test. The relative sensitivity, specificity and overall accuracy of 98.18%, 100%, and 99.28%, respectively, met the recommended performance profile cutoffs established by the WHO (≥80% for sensitivity and ≥97% for specificity) [30,31]. Our results were even better than the manufacturer's published recommendations (96.77%, 99.20%, and 98.72%, respectively). Although the sensitivity of a RAD test will correlate with the level of the viral load of the sample, our group of patients included a mixture of high, medium and low viral load samples, providing useful information about the performance characteristic of the RAD test (despite the fact that, we expected to have more positive, high viral load SAR–CoV–2 samples compared to the general population, since our laboratory is a diagnostic centre for Covid–19 testing). Such RAD tests, with high sensitivity on low viral load samples (Ct≥27) could potentially be used as an epidemiological surveillance tool [43]. Although some studies have questioned the performance characteristics of the RAD tests, reporting sensitivities down to 56%–85% [9,23,[32], [33], [34], [35], [36], [37], [38], [39], [40]], they appear to have many limitations, such as modified sample processing methods, refrigerated samples that could potentially affect viral antigens, different means of transportation, time between sample collection, initiation of RT–rtPCR, and variations on the background prevalence of the SARS–CoV–2, that could have an impact on the results. On the other hand, other research groups have reported results in parallel to our findings [19,22,[41], [42], [43], [44], [45], [46]]. It appears that the performance characteristics (low–high) of different RAD tests, including the BOSON, would depend largely on the infectious status of the individuals (low–high viral loads) and the disease prevalence at the time of testing (low–high), and not between self–collected and professional–collected samples [36]. During the study period, the prevalence of SARS–CoV–2 in Greece was extrapolated as medium [26], [27], [28], [29], indicating that the RAD test has the potential to be used in medium–prevalence settings, as well. However, regardless of the disease prevalence, the success of a self–testing RAD test approach will largely depend on the education and the participation of the population [48].

Furthermore, our results indicated that the detection limit of the test, showing sensitivity of 91.67% for Ct values of ≥30, was superior to that of other RAD tests from different manufacturers [19,41,43,44,46]. We believe that any RAD test will generate a positive line regardless of the PCR KIT used, especially when they are ≤27 cycles. However, in this study we have used the most sensitive method, after comparing results from many PCR kits, and it was found that the Sacace PCR was the most sensitive for detecting SARS–CoV–2 at higher Ct values. Also, it is of particular interest that all the FN cases were between 0 and 1 days of symptoms onset, a period in which the viral load is expected to be limited. Studies have indicated that PCR positive samples with low viral load (Ct >25–35) have lower possibility to transmit the virus effectively [43], and also show decreasing sensitivity to RAD tests [36]. Others have even suggested that individuals positive for SARS–CoV–2 by RT–rtPCR, but negative with a RAD test are unlikely to be infectious [42,43]. Even days after infection, RT–rtPCR tests can detect low levels of viral RNA, but with no evidence that these could be contagious [45]. As a result, the Boson RAD test could faster (sensitivities at 0 and 1 days of onset, 96.95% and 98.77%, respectively), and more reliably identify people with high, medium, even low viral loads, especially between 2 and 7 days of symptoms onset [47]. Moreover, the Boson RAD test exhibited a 90.48% sensitivity ratio for asymptomatic patients. To our knowledge this is one of the highest sensitivity rates derived from asymptomatic cases previously published [24,35]. Improved turn–around times is critically important, especially in remote locations or long–term care centers [24]. Although RAD tests cannot be used as the sole basis to diagnose or exclude SARS–CoV–2 infection, however, this test provides a useful tool for screening, identifying and quickly isolating cases, and could facilitate a fast–screening test at point of care (poc) settings, for the detection of Covid–19 individuals even before becoming infectious [48].

Our study has several strengths including the large sample population and the large number of positive cases. In addition, the immediate transportation and analysis of samples limited the possibility of FP or FN due to factors, such as time, sample, and testing conditions. After all, sensitivity of detection to any method depends largely on the time of testing and conditions of the specimens [19]. Future considerations for the Boson RAD test should be to evaluate the performance characteristics against different types of samples, such as saliva, and under low prevalence, poc settings. Also, the exact LOD and ability of detecting emerging variants should be established [49].

5. Conclusions

In conclusion, our clinical study evaluated the performance characteristics of the Boson Rapid SARS–CoV–2 Antigen Test Card in untrained users, in a real–world diagnostic setting. The high sensitivity makes it an excellent alternative to RT–rtPCR, as a COVID–19 diagnostic tool that could allow self–sampling to become self–testing. Faster diagnosis of cases would allow to identify more individuals with transmittable virus, to initiate isolation and therapy, to assess the population's immunity status and therefore, to effectively manage the pandemic.

CRediT Authorship Contribution Statement

ML: Data Curation, Formal Analysis, Writing – original draft preparation, Review and Editing, Visualization, Manuscript Finalization; VM: Supervision RT–rtPCR, Methodology RT–rtPCR, Data Curation, Writing – Review and Editing; MP: Investigation RT–rtPCR, Writing – Review; EV: Investigation RT–rtPCR, Writing – Review; NGM: Conceptualization, Methodology, Data Curation, Formal Analysis, Writing – Review and Editing; HPK: Conceptualization, Methodology, Formal Analysis, Writing – Review and Editing; DN: Investigation BOSON RAD, Writing – Review; VT: Clinical Investigation, Writing – Review and Editing; GG: Clinical Supervision, Methodology BOSON RAD, Investigation BOSON RAD, Data Curation, Writing – Review and Editing.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not–for–profit sectors. The whole clinical and experimental funding was carried out independently by Locus Medicus S.A.

Declaration of competing interests

The authors report no conflicts of interest relevant to this article.

Acknowledgments

We would like to thank Xiamen Boson Biotech Co. Ltd., Fujian, P.R. China, for kindly providing the Rapid SARS–CoV–2 Antigen Test Cards.

APPENDIX A.

Table A.1 and Table A.2

APPENDIX B.

Figure B.1 and Figure B.2..