Validation and implementation of the Panbio COVID-19 Ag rapid test for the diagnosis of SARS-CoV-2 infection in symptomatic hospital healthcare workers

Summary

Background

RT-PCR is the current recommended laboratory method to diagnose SARS-CoV-2 in healthcare workers (HCW). As RT-PCR is not widely available and is time-consuming, it limits decision making on removal from and return to work of possibly contagious HCW.

Aim

In this study we evaluated the Panbio™ COVID-19 Ag rapid test (PanbioCAgRT) in 825 hospital HCW.

Methods and finding

This study consisted of two phases. In the validation phase, we tested hospital HCW with mild symptoms (three days or less) in parallel using the PanbioCAgRT and the RT-qPCR test. The PanbioCAgRT demonstrated 86.7% sensitivity, 100% specificity, 100% PPV and 98.5% NPV with regard to RT-qPCR. For HCW with PanbioCAgRT-/RT-qPCR+, the median Ct value was 30.9, whereas for the HCW with PanbioCAgRT+/RT-qPCR+ the median Ct value was 19.3 (P<0.001). In the second phase, we implemented an on-site antigen test-based strategy for symptomatic hospital HCW: HCW that tested positive with the PanbioCAgRT on-site were considered SARS-CoV-2 positive and were sent home. HCW that tested negative with the PanbioCAgRT on-site were allowed to work with PPE pending RT-qPCR test results from the laboratory. Sensitivity of the antigen test-based strategy was 72.5% and NPV was 97%. For HCW with PanbioCAgRT-/RT-qPCR+ median Ct values were 27.8.

Conclusion

The PanbioCAgRTt validated in this study showed a high sensitivity and specificity in samples obtained from HCW with high viral loads. The antigen-based testing strategy proposed in this study seems to be effective, safe and easy to implement in a wide range of occupational healthcare settings.

Introduction

Access to rapid, reliable testing for SARS-CoV-2 infection in and possible contagiousness of healthcare workers (HCW) is key to maintaining an adequate work force. Currently, reverse transcriptase real time polymerase chain reaction (RT-PCR) is the recommended test for detection of SARS-CoV-2 infection in symptomatic HCW. [1,2] In the Netherlands, high-throughput testing centres for symptomatic healthcare employees using RT-PCR are widely available in hospitals, with test results offered within six to ten hours. Most HCW in long-term care facilities and primary care centres can make use of these hospital-based testing centres as well, but logistic difficulties may prolong time to results.

In HCW crucial to the healthcare system, rapid test results can enable employees who tested negative for SARS-CoV-2 to return to work quickly (wearing personal protective equipment (PPE) until their symptoms have resolved). HCW crucial to the healthcare system that test positive might be removed sooner from their work environment, therewith lowering the risk of further spreading the virus to colleagues.

Antigen tests are generally less sensitive than RT-PCR for SARS-CoV-2 detection, but are easy to perform, relatively inexpensive, have short turnaround times, and may facilitate decentralized testing. In addition, early data suggest that antigen tests may play a key role in rapidly identifying those at highest risk for transmitting disease. [[3], [4], [5], [6], [7], [8]] Abbott Panbio™ COVID-19 Ag rapid test (PanbioCAgRT) is a lateral flow immunoassay that is used to detect the nucleocapsid protein of SARS-CoV-2 in approximately 15 minutes. Antigenic tests have been shown to be most valid in the days around the onset of symptoms, when the viral load in the nasopharynx is highest. [3,[9], [10], [11], [12], [13]].

To our knowledge, this is the first study that was initiated to evaluate the use of rapid antigen tests in hospital HCW. The primary objective of the study was to determine the clinical specificity and sensitivity of the PanbioCAgRT for detection of SARS-CoV-2 compared to the RT-qPCR in hospital HCW being mildly symptomatic for three days or less (validation phase). Secondary objective was to determine the clinical sensitivity of the PanbioCAgRT for different cycle threshold (Ct) value groups. After the validation phase, an antigen test-based strategy for HCW was implemented in our hospital and described in this study.

Methods

Ethics

The medical research ethics committee (MREC) Brabant decided it was not necessary to subject the study to the Medical Research Involving Human Subjects Act (WMO) and did not require full review by an accredited MREC (NWO2021-10). All participants have provided informed consent.

Setting

The study was performed at a single peripheral teaching hospital in the Netherlands, with 700 beds offering general and complex medical care. Currently, the hospital has over 4000 employees.

The validation phase of the study was conducted between October 5th and October 30^th 2020. The implementation phase was conducted between November 5^th and November 29^th 2020. The overall incidence of SARS-CoV-2 infection (confirmed by RT-PCR) among all routinely tested hospital HCW (also HCW that were not included in the study) varied during the course of the study and was 9% during the first two weeks of the study (validation phase), 16% during week three to four (validation phase), 11% during week five to six (implementation phase), and 10% in week seven to eight (implementation phase).

Study design and participants

Hospital HCW employed at the Jeroen Bosch hospital with mild symptoms suggestive of COVID-19 (according to the national COVID-19 protocol of the National Institute for Public Health and the Environment (RIVM)), presenting to the hospital testing centre were informed about the study and enrolled if they consented. Only participants that were 16 years or older and were symptomatic for three days or less were included. Asymptomatic HCW with recent exposure to a SARS-CoV-2 infected person were excluded.

In the validation phase, HCW were tested by both PanbioCAgRT and RT-qPCR. During this phase, the participants and occupational health personnel were unaware of the PanbioCAgRT results.

In the implementation phase, a rapid antigen-based test strategy for symptomatic HCW (aged ≥16 years) in our hospital was realised. The excellent specificity of the PanbioCAgRT (see results) led us to decide to extend the test period by one day (from three days or less from symptom onset to four days or less from symptom onset) to create more flexibility for our HCW to schedule. With the rapid antigen-based strategy, hospital HCW that tested positive with the PanbioCAgRT were considered SARS-CoV-2 positive, (due to excellent specificity during the validation phase (see results)), did not undergo further testing and were directly sent home after test results became available. HCW that tested negative with the PanbioCAgRT received a confirmation RT-qPCR test and could, if considered crucial to healthcare, return to work pending RT-qPCR results, using PPE at all times.

Study procedures

From each symptomatic HCW, two nasopharyngeal samples and one oropharyngeal sample were collected. One nasopharyngeal and one oropharyngeal swab were taken for routine viral genome detection by RT-qPCR and placed in a single 3 mL viral transport medium (VTM). A second nasopharyngeal swab, provided in the PanbioCAgRT test kit, was taken for antigen testing as recommended by the manufacturer. The second nasopharyngeal swab was obtained through the contralateral nostril. All swabs from individual participants were taken by the same trained health care professional.

In the validation phase, both the VTM tube and PanbioCAgRT tube were, as recommended by the manufacturer, transported within two hours to the hospital's medical microbiological laboratory. The PanbioCAgRT was immediately performed by an experienced laboratory technician and the VTM tube was tested by RT-qPCR on the same day.

In the implementation phase, the PanbioCAgRT was performed immediately at the sample collection site by a trained nurse. When the PanbioCAgRT was positive, the VTM tube of the HCW was discarded. When the PanbioCAgRT was negative, the VTM tube was transported to the hospital's medical microbiological laboratory and was tested using RT-qPCR on the same day.

Adequate PPE was used while collecting the swabs and performing the antigen tests.

Data collection

Clinical data were collected for each HCW upon presentation with a questionnaire including the number of days post symptom onset, known contact to a previous SARS-CoV-2 infected person, and type of symptoms.

Panbio rapid antigen test

The Panbio™ COVID-19 rapid antigen test device (Abbott Rapid Diagnostics Jena GmbH, Jena, Germany) was used as recommended by the manufacturer, using only materials provided by the manufacturer in the kits. This test is a qualitative membrane-based immunoassay (immunochromatography) for the detection of nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. The assay was manually read, with two individuals reading the results separately after the indicated time of 15 minutes. In case of discordant results consensus was sought. Only coloured visible bands were considered a positive result. All antigen test results were photographically documented.

RT-qPCR testing

Nucleic acid extraction was performed with an automated sample preparation system MP96 (Roche Diagnostics, Switzerland); the input volume was 500 μl, the elution volume 100 μl. The real-time (RT)-qPCR assay targeted the SARS-CoV-2 RdRp gene with forward primer TGA AAT GGT CAT GTG TGG CG, reverse primer CAA ATG TTA AAA ACA CTA TTA GCA TAA GCA G and probe FAM–CCA GGT GGA ACC TCA TCA GGA GAT GC– BHQ1. The assay included Phocine Distemper Virus as an RNA internal extraction and amplification control. Samples showing an exponential fluorescence curve and a Cycle threshold (Ct) value <40 were considered as positive. A Ct value >40 was not observed.

Statistics

Primary outcome was the PanbioCAgRT clinical sensitivity and specificity with 95% confidence interval (95% CI) compared to qRT-PCR, which was considered gold standard. The secondary outcome was clinical sensitivity with 95% CI compared to qRT-PCR stratified by Ct value category.

Continuous variables were shown as median and interquartile range (IQR). Mann-Whitney U-test was used to compare differences between Ct values of PanbioCAgRT+/PCR+ and PanbioCAgRT-/PCR+ subjects. A P value of ≤0.05 was considered statistically significant. All data was analyzed using Python 3.7.7 with the Pandas (version 1.1.3), Numpy (version 1.19.2), Matplotlib (version 3.3.2), Seaborn (version 0.11.0) and SciPy (version 1.5.2) packages.

Results

Validation phase

Between October 5^th and October 30^th 2020, 433 participants were included in the analysis. Of the 433 participants, 45 tested positive (10.4%) by RT-qPCR and 39 (9%) tested positive by PanbioCAgRT. None of the samples that tested positive by PanbioCAgRT tested negative by RT-qPCR. Therefore, the overall sensitivity of the PanbioCAgRT was 86.7% (95% CI:72.5–94.5), specificity was 100% (95% CI: 98.8–100), positive predictive value was 100% (95% CI: 88.8–100) and negative predictive value was 98.5% (95% CI: 96.6–99.4) (Table I).

Table I.

Agreement between PanbioCAgRT and RT-qPCR in HCW in validation phase

	RT-qPCR positive	RT-qPCR negative	Total
PanbioCAgRT positive	39	0	39
PanbioCAgRT negative	6	388	394
Total	45	388	433

The overall range of Ct values was 16.3–36.4 (median 19.5 IQR 18.1–24.0). For samples that tested positive by both PanbioCAgRT and RT-qPCR, the range of Ct values was 16.3–28.3 (median 19.3 IQR 17.8–21.1). For samples that tested negative by PanbioCAgRT and positive by RT-qPCR Ct values ranged between 27.4 – 36.4 (median 30.9 IQR 29.5–32.0). There was a statistically significant difference between Ct-values of subjects with PanbioCAgRT+/PCR+ and subjects with PanbioCAgRT-/PCR+ (P<0.001) (Figure 1). Table II shows sensitivity of the PanbioCAgRT stratified by Ct values.

Figure 1.

Distribution of Ct values of symptomatic HCW in validation phase with positive RT-qPCR (n = 45).

Table II.

Sensitivity of the PanbioCAgRT in HCW in validation phase according to Ct values

Ct value category	RT-qPCR + (n)	PanbioCAgRT + (n)	PanbioCAgRT – (n)	Sensitivity (%)
Ct <20	25	25	0	100%
Ct <25	34	34	0	100%
Ct <30	41	39	2	95.1%
Overall	45	39	6	86.7
Ct ≤28	40	39	1	97.5%

Bold denotes cut-off Ct value that still provides excellent sensitivity for the antigen test. When samples contain viral RNA with lower loads (Ct values of >28), sensitivity decreases.

Implementation phase

Between November 5^th and November 29^th 2020, 392 symptomatic HCW were included to participate in the implementation of the antigen-based test strategy. Of these 392 HCW, 29 tested positive by PanbioCAgRT and did not receive a confirmation PCR based on the 100% specificity of the test in the validation phase. Of the 363 HCW that tested negative by PanbioCAgRT, 11 tested positive by RT-qPCR and 352 tested negative by RT-qPCR. Sensitivity of the PanbioCAgRT was 72.5% (95% CI: 55.9–84.9) and negative predictive value was 97.0% (95% CI:94.5–98.4). Ct-values of the 11 PanbioCAgRT-/RT-qPCR+ samples ranged between 21.8 and 35.6 (median 27.8 IQR: 26.5–30.6).

Discussion

In the validation phase, we found an overall clinical specificity of 100% (95% CI: 98.8%–100%) and sensitivity of 86.7% (95% CI: 72.5%–94.5%) of the PanbioCAgRT compared to RT-PCR. The sensitivity in our validation study was slightly higher than found in most previous reports, [3,[9], [10], [11],14] and comparable to other studies. [12,15] In line with other studies, SARS-CoV-2 RNA load was significantly higher in antigen test+/RT-PCR+ samples than in antigen test-/RT-PCR+ samples. [3,[9], [10], [11], [12], [13], [14], [15]] The sub-analysis in our validation study indicated that sensitivity of the PanbioCAgRT was 97.5% in samples with Ct values ≤28.

Previous studies reported that lower Ct values are associated with higher viral culture positivity. [[16], [17], [18], [19]] Even though there currently is no direct evidence whether cell culture positivity or higher viral load correlates with contagiousness, both are commonly recognized as surrogates of infectivity. [20] In two recent studies, SARS-CoV-2 could not be cultured from antigen test-/RT-qPCR+ samples. [3,7] Albert et al. stated that the SARS-CoV-2 RNA load threshold associated with culture positivity in their study was close to previous published results on virus culture (around 10⁶ copies/ml or Ct 25), suggesting that patients with RT-PCR proven COVID-19 but negative PanbioCAgRT were unlikely to be contagious. [3] A recent Dutch study showed that 97.3% of mildly symptomatic infectious individuals (patients with positive SARS-CoV-2 cultures) were detected with PanbioCAgRT. [6] In three other recent reports the sensitivity of the antigen test for detection of SARS-CoV-2 infectivity based on virus culture were 96.4%, 84%, and 86.4% respectively. [4,5,8] Although more studies on this topic are needed, antigen tests may play a key role in rapidly identifying HCW at highest risk for transmitting disease.

In a healthcare setting, the best test is not necessarily one that determines whether a person has any evidence of SARS-CoV-2, but may be the one that quickly and accurately identifies individuals capable of transmitting the infection to others. Laboratory-independent antigen tests could be key to detecting contagious HCW to prevent further transmission, but also keeping non-contagious HCW at work during this challenging pandemic. [3] With our antigen-based strategy, highly contagious HCW (with a positive antigen test), could be rapidly removed from the work environment and stopped being a risk for colleagues and patients. Less or non-contagious HCW (with negative antigen test), could return to work wearing PPE pending RT-PCR results. Moreover, if staff shortages continue despite mitigation strategies, it might be considered that (SARS-CoV-2 infected but) antigen test negative HCW, might return to work wearing PPE until symptoms have resolved without performing PCR as secondary confirmation. [1] However, in this case one should consider to repeat antigen testing the following day(s) to ensure the viral load remains low.

The rapid antigen-based rapid strategy that we implemented in our hospital was easy to perform and had a short turnaround time of approximately 20 minutes as the PanbioCAgRT was performed on-site. Remarkably, the sensitivity of the PanbioCAgRT in the implementation phase (72.5%) was lower than the sensitivity in the validation phase (86.7%). In addition, in the implementation phase, one sample with high viral load (Ct 21.8) was not detected by the antigen test. A possible explanation may be the difference in type of personnel that performed the antigen test: in the validation phase, the antigen test was performed by experienced laboratory technicians and in the implementation phase, the antigen test was performed by trained nurses. Though all antigen tests that were performed on-site were photographed and reassessed at a later time point by our laboratory technicians without any inconsistencies. Another difference between the two study phases was the prolonged inclusion time in the implementation phase of four days or less after symptom onset, compared to three days or less after symptom onset in the validation phase. However, the sensitivity in the implementation phase did not change when we excluded HCW with symptoms for four days from our analysis (data not shown). Another difference between the two study phases was the time from collection of the nasopharyngeal swab and preparation of the extraction tube with the swab and buffer (at the testing site), to the dispense of the fluid into the specimen well on the test device. In the validation phase, this time was at least 30 minutes (maximum two hours), as it required transport of the extraction tubes to the laboratory. In the implementation phase, this time was only a few minutes, as the antigen test was performed at the sampling site. In the validation phase, the prolonged incubation of the swab with the extraction buffer could have had a positive effect on the sensitivity, but this hypothesis needs to be investigated in a technical validation study. This somewhat longer incubation time certainly did not influence the specificity of the test, as all positive PanbioCAgRT were confirmed by RT-PCR as mentioned. Other factors such as the prevalence of SARS-CoV-2 in our population, the protocols for sample collection and the personnel that collected the nasopharyngeal swabs were similar in the two phases.

Conclusion

The PanbioCAgRTt validated in this study showed high sensitivity and specificity in samples obtained from HCW during the first four days of symptoms and with high viral loads. Previous studies reported that high viral loads are associated with higher culture positivity. [[16], [17], [18], [19]] Based on the results of recent studies that compared antigen test to culture, [[3], [4], [5], [6], [7], [8]] SARS-CoV-2 antigen tests have the potential to better identify individuals that are likely to be shedding and transmitting infectious virus than RT-PCR and might therefore be implemented in a wide range of occupational healthcare settings.

Credit author statement

Eva Kolwijck: Conceptualization, Methodology, Resources, Formal Analysis, Visualization, Writing- Original draft preparation, Supervision Miranda Brouwers-Boers: Conceptualization, Methodology, Validation, Investigation, Data curation, Project Administration Jorrit Broertjes: Validation, Investigation, Writing- Original draft preparation. Kelly van Heeswijk: Validation, Investigation Natasja Runderkamp: Validation, Investigation Angelique Meijer: Resources, Writing- Reviewing and Editing, Mirjam Hermans: Conceptualization, Methodology, Resources, Writing- Original draft preparation, Writing - Review & Editing, Project Administration Sander Leenders: Conceptualization, Methodology, Resources, Writing - Review & Editing, Supervision.

Conflict of interest statement

The authors declare no conflicts of interest.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.