Implementation of rapid SARS-CoV-2 antigenic testing in a laboratory without access to molecular methods: Experiences of a general hospital

Highlights

• •The COVID-19 Ag Respi-Strip assay is a new immunochromatographic diagnostic tool for antigenic diagnosis of SARS-CoV-2.
• •This test did not reduce significantly the number of samples outsourced for COVID-19 confirmation.
• •The sensitivity of this rapid test is poor and improvements are needed to enhance its performances.

Abstract

Background

The COVID-19 Ag (Antigen) Respi-Strip assay is a new immunochromatographic diagnostic tool recently available for antigenic diagnosis of SARS-CoV-2. The proposed sensitivity is not higher than 60 %, but its high specificity allows both quick decisions for the management of patients and confirmation by molecular diagnosis for only negative tests. However, from the first tests performed, we suspected that the sensitivity observed with routine use was much lower than that announced by the manufacturer.

Materials and methods

Over a period of one month, we compared the negative results obtained with the COVID-19 Ag Respi-Strip kit with those obtained from qRT-PCR performed in a laboratory qualified for the molecular diagnosis of SARS-CoV-2. All samples tested were naso-pharyngeal smears from UTM-RT medium.

Results

Of 774 patients tested, 714 negative samples were sent for confirmation, and 159 were found to be positive by qRT-PCR. The median positive percentage agreement was 23.9 % (95 % CI: 14.2 %–38.2 %). The Cohen’s kappa score was 0.35.

Conclusion

Using this immunochromatographic assay as a triage test did not significantly reduce the number of samples outsourced for COVID-19 confirmation by qRT-PCR. In addition, even if the turn-around time is short, the assay is completely manual, which is not suitable for large volumes of routine samples. The sensitivity of this rapid test is poor, and improvements are needed to enhance its performance.

1. Introduction

Since the launch of the COVID-19 Ag Respi-Strip assay (Coris Bioconcept, Gembloux, Belgium) upon completion of a validation study under the National Competent Authority supervision, we enthusiastically implemented the company’s proposed algorithm allowing the integration of this rapid test in the management of patients suspected of COVID-19. This decision was based on the significant specificity reported (99.5 %) that allows quick decisions regarding the management of patients. Negative results require additional examinations by medical imaging and molecular detection by qRT-PCR.

We read with great interest the early April WHO advice on the use of point-of-care immunodiagnostic tests for COVID-19 [1] as well as the article recently published on the test validation [2] and wanted to evaluate our current way of working.

2. Materials and methods

This prospective study was conducted over a 1-month period between April 5, 2020, and May 4, 2020, at a single 550-bed hospital site. The beginning of this period corresponded to the epidemic peak of COVID-19 in Belgium. Nasopharyngeal samples for the diagnosis of COVID-19 were taken from UTM-RT swabs (Copan spa, Brescia, IT) and sent to the laboratory. The antigenic assessment was performed using the COVID-19 Ag Respi-Strip kit according to the manufacturer’s instructions. After antigenic testing was performed, the molecular assessment of SARS-CoV-2 was outsourced to a university centre where it was carried out by qRT-PCR using E-gene SARS-CoV-2 primers/probes.

3. Results

A rapid on-site verification of the performance of the COVID-19 Ag Respi-Strip kit was carried out on 56 samples; it showed a sensitivity of 30 % (95 % CI: 16.7 %–47.9 %) a specificity of 100 %, and a positive predictive value of 100 %, validating the decision not to confirm a positive result. During the investigation period, 912 tests were performed. Some patients were tested more than once for follow-up according to the handling clinician’s decision. After removing duplicates, 776 patients remained for evaluation. Two tests were removed from the statistical analysis (one non-conform and one invalid). Sixty (60) out of 774 antigenic strips were positive. Fig. 1 shows the evolution of positive and negative molecular confirmations over the weeks as well as the percentage of positive molecular and antigenic tests. The total number of positive PCR samples was 159. The positive percentage agreement during the 4 weeks ranged from 14.3 % to 34.7 % with a median of 23.9 % (95 % CI: 14.2 %–38.2 %). The Cohen’s kappa score was 0.35.

Fig. 1.

Evolution of the number of positive and negative PCRs among samples sent for confirmation of negative antigenic testing and the percentage of positive PCRs (solid line) and antigenic testing (dashed line) during the 4 weeks of observation.

4. Discussion

Under routine conditions, the sensitivity of the antigen detection of SARS-CoV-2 with the immunochromatographic COVID-19 Ag Respi-Strip kit was significantly lower than that announced by the manufacturer or reported by Vandenberg [2], although we limited ourselves to using qRT-PCR as the comparison method. In our series, we observed a median sensitivity of 23.9 %. Moreover, compared with the expected performance, the poor observed sensitivity gave rise to 80 % more false negative samples and 2.2 times fewer positive samples answered on site.

Some authors reported a sensitivity of similar molecular methods close to 70 % [3]. To obtain a better understanding of the actual sensitivity, we used the patient database constructed for a serological evaluation for which approval was obtained from our Ethics Committee [4]. When combining molecular diagnosis, chest CT scans and suggestive clinical patterns of COVID-19, we observed that PCR was positive in only 199 patients among the 236 patients (76.2 %) with suggestive symptoms of COVID-19 or a positive chest CT. These results are in line with the previously published false negative rate of approximately 20 % for qRT-PCR [[5], [6], [7]].

As mentioned in the WHO advice [1], the performances of antigenic tests depend on several factors, such as the time from onset of illness, the specimen viral content, and other preanalytical and analytical considerations, as has been previously reported for other respiratory viruses. The WHO estimates that at least half of COVID-19-infected patients might be missed by such tests and therefore does not currently recommend their use. Vandenberg calculated their performance based on a threshold cycle (Ct) below 22 [2]. In the meantime, a notice from the manufacturer signalled that some hospitals observed negative results with a Ct of 13.45 or positive results with a Ct higher than 33, underlining the unclear relationship between protein and RNA detection. The main limitation of our investigation was our inability to match our results against the Ct of the molecular analyses since these were outsourced. We also focused on the sensitivity of the test. However, specificity did not appear to be a priority, given that the high prevalence of disease amplifies positive predictive value of the test when it is are prescribed for patients in a COVID-19-compatible clinic.

5. Conclusion

Our routine results demonstrate that the concordance between antigenic and molecular testing is fair regarding the kappa score [8] and that this rapid assay does not reduce costs per patient. A thorough validation in appropriate populations and settings should have been performed by the manufacturer before a large-scale implementation.

Funding

There were no funding resources to declare for this study.

CRediT authorship contribution statement

Laurent Blairon: Conceptualization, Methodology, Data curation, Formal analysis, Validation, Writing - original draft. Alain Wilmet: Conceptualization, Methodology, Data curation, Formal analysis, Validation, Writing - review & editing. Ingrid Beukinga: Conceptualization, Methodology, Data curation, Formal analysis, Validation, Writing - review & editing. Marie Tré-Hardy: Conceptualization, Methodology, Data curation, Formal analysis, Validation, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.