Screening for SARS-CoV-2 infection in asymptomatic individuals using the Panbio COVID-19 antigen rapid test (Abbott) compared with RT-PCR: a prospective cohort study

Abstract

Background

Antigen-based point-of-care tests for identification of SARS-CoV-2 may markedly enhance effectiveness of population-based controlling strategies. Previous studies have demonstrated >70% sensitivity and high specificity compared with reverse transcriptase real-time PCR (RT-PCR) in symptomatic individuals, but test performance for asymptomatic individuals is unknown.

Methods

Test performance of the Panbio COVID-19 Ag Rapid Test (Abbott) was compared with RT-PCR in a longitudinal cohort study of asymptomatic football players and staff members of professional football clubs. Based on timing of symptoms and prior and subsequent test results, positive RT-PCR tests were categorised as presymptomatic, early or late infection, or persistent RNA shedding.

Findings

2425 tests were performed in 824 individuals, of which 52 (6.3%) were SARS-CoV-2 positive based on RT-PCR. There were 2406 paired sets from asymptomatic subjects for analysis. Sixteen Panbio tests were inconclusive, for which sensitivity analyses were performed (considering results as either positive or negative or being excluded). Sensitivity of Panbio for screening of asymptomatic individuals ranged from 80.0% (61.4–92.3) to 86.67% (69.2–96.2) and specificity from 99.53% (95% CI 99.2 to 99.8) to 100% (95% CI 99.8 to 100). Sensitivity of Panbio to detect subjects with presymptomatic/early infection (n=42) ranged from 81.82% (95% CI 67.3 to 91.8) to 90.91% (95% CI 78.3 to 97.5) with specificity always above 99%.

Interpretation

The Panbio COVID-19 Ag rapid test identifies 81%–90% of presymptomatic and early asymptomatic SARS-CoV-2 infections with high specificity. This test may therefore be adopted in testing strategies such as targeted screening of specific populations where prevalence is low.

Strengths and limitations of this study.

• The unique longitudinal prospective cohort allows analysis of Panbio COVID-19 Ag rapid test characteristics in different phases of SARS-CoV-2 infection, demonstrating its applicability in population screening.

• The study demonstrates the feasibility of performing the Panbio COVID-19 Ag rapid test as point-of-care test, outside of a laboratory or hospital setting.

• The studied cohort is comprised predominantly of male professional athletes and as such is not a representative sample from the general population.

• Reverse transcriptase real-time PCR reference tests were performed by three different commercial laboratories, limiting the possibility for overall analyses using Ct values as a cut-off.

Introduction

Rapid diagnosis of SARS-CoV-2 infection and subsequent contact tracing are essential in the containment of the current pandemic.1 In most countries testing is targeted towards individuals with symptoms of a potential SARS-CoV-2 infection. However, infectiousness most likely occurs before symptom onset and some infections remain asymptomatic.2 Screening of asymptomatic individuals may therefore also identify infectious individuals.

Currently, reverse transcriptase real-time PCR (RT-PCR) is the standard for detection of SARS-CoV-2 infection. In a validation study of the Abbott Panbio COVID-19 Ag rapid test in community-dwelling subjects with symptoms of respiratory tract infection, this assay had specificity of 100% and sensitivity above 95% to detect SARS-CoV-2-infected subjects with low Ct values in RT-PCR (ie, infections with a high viral load in nasopharyngeal samples).3 Considering the short turnaround time, user friendliness, low cost and opportunity for decentralised testing, lateral flow assays (LFA) would also be a suitable test for screening of asymptomatic individuals.

We, therefore, validated the Panbio COVID-19 Ag rapid test in a cohort of asymptomatic football players and staff of football clubs from the Dutch national leagues who are longitudinally tested for SARS-CoV-2 infection with RT-PCR at approximately weekly intervals.

Materials and methods

Populations and study period

The study population consisted of football players, staff and referees from 13 different professional football clubs and the national teams in the Netherlands. Starting August 2020, the Royal Netherlands Football Association (KNVB) required all individuals to be tested for SARS-CoV-2 infection by RT-PCR independent of presence of symptoms, 2 days prior to each match. The study period of the LFA validation ran from 1 October 2020 to 9 November 2020. All individuals were sampled for routine RT-PCR testing, using a combined throat/nasopharyngeal swab, after which they underwent an additional nasopharyngeal swab for LFA testing. At two test visits for two different teams (n=104 tests), only LFA tests were performed, followed by sampling for RT-PCR on the following day. Data regarding symptoms of potential SARS-CoV-2 infection before and during the study period and (prior) test results were collected by the physicians of each of the football clubs.

The SARS-CoV-2 prevalence in this asymptomatic cohort was expected to be low. Therefore, it was not feasible to continue the study until the recommended 100 SARS-CoV-2-positive individuals were included.3 We aimed to include at least 50 individuals who tested positive for SARS-CoV-2 infection.

Diagnostic tests

Reverse transcriptase real-time PCR

The samples were tested with RT-PCR by either Eurofins (Brugge, Belgium; commercially available platform Viasure, CerTest Biotech, Spain, according to manufacturer’s instructions, as well as laboratory-developed platform), Synlab Laboratories (Luik, Belgium; commercially available platform TaqPath COVID-19 Multiplex, Thermo Fisher, Waltham, Massachusetts, USA, according to manufacturer’s instructions) or U-diagnostics (Utrecht, The Netherlands; laboratory-developed platform). RT-PCR results were obtained from the physicians of the participating football clubs and positive or discrepant results were verified. Residual material of these samples was not available for further analyses or repeated testing on a single platform.

Lateral flow assays

The Panbio COVID-19 Ag rapid test device by Abbott (Lake Country, Illinois, USA) is a membrane-based immunochromatography assay which detects the nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. Collected swabs were transferred into dedicated sample collection tubes containing a lysis buffer provided with the test kit. Samples were processed on site, directly after collection, by trained personnel wearing personal protective equipment and in accordance with the manufacturer’s protocol. Test results were recorded after 15 min of assay initiation and documented by photograph.

Analysis

Specificity and sensitivity with 95% CIs of the LFA were calculated using the RT-PCR results as reference test. Testing was categorised into either screening (testing of individuals not known to be infected with SARS-CoV-2) or follow-up of individuals who have tested positive for SARS-CoV-2 by RT-PCR before. The RT-PCR results were then categorised into different phases of infection. Presymptomatic infection was defined as a first positive test before onset of symptoms of SARS-CoV-2 infection. Early asymptomatic infection includes the first positive test for individuals who remain asymptomatic during the course of their infection (and therefore not identified as presymptomatic) and all positive test results within 7 days of the first positive test. Late infection was defined as a positive test at least 7 days after the first positive RT-PCR result or at least 7 days after onset of symptoms with relief of symptoms for more than 24 hours. Persistent RNA shedding was defined as a positive test more than 4 weeks after the first positive RT-PCR result and not suspected for reinfection based on absence of symptoms and high Ct values (>32). The LFA results were analysed for these different phases. Specificity and sensitivity were determined based on presymptomatic/early asymptomatic infection versus late infection/no infection. Since RT-PCR was performed in different laboratories and using different assays, Ct values cannot directly be compared. Therefore, no sensitivity analyses based on Ct values were performed.

Patient and public involvement

Patients or the public were not directly involved in the design, conduct, or the reporting or dissemination plans of this study.

Results

Population characteristics

Our cohort included 824 individuals (94% male) with a median age of 27 years (range 16–80 years, IQR 21–40). Overall, 2425 samples were tested by RT-PCR and LFA. Of these, 2321 tests were obtained at the same time, and 104 tests were taken with a 1-day interval between LFA and PCR. The median number of LFA tests per subject was 3 (range 1–6, IQR 1–4).

Based on RT-PCR, 52 of 824 subjects tested positive for SARS-CoV-2 at least once during the study period (prevalence of SARS-CoV-2 infection: 6.3%). Of these 52, 29 developed (mostly mild) symptoms of SARS-CoV-2 infection; runny nose or nasal congestion (n=12), sore throat (n=12), fever (n=12), headache (n=6), muscle ache (n=5), cough (n=5), altered smell or taste (n=5). A completely asymptomatic infection occurred in 23 individuals (44%; prevalence of asymptomatic SARS-CoV-2 infection: 2.8%).

Of the 2425 tests, 18 were performed while the individual reported symptoms at time of testing. These 18 tests were therefore not included in the analysis. One LFA test was invalid due to absence of a control band and was excluded for analysis. The comparison between RT-PCR and LFA, therefore, includes 2406 paired tests (figure 1).

Figure 1.

Inclusion diagram. Diagram showing the flow of participants in the study.

LFA results

Of 2406 tests, 68 (2.8%) were positive by RT-PCR and 42 (1.7%) by LFA (table 1). Of these 68 tests, 30 were first positive PCR tests and 38 were follow-up tests of individuals who tested positive previously. Sensitivity of the LFA was determined both for the purpose of screening and for its use in patient follow-up, by including only results at the time of first positive PCR or results of positive follow-up PCR, respectively. Sixteen LFA tests (0.67%) were considered inconclusive because of weak unclear bands, of which five were positive with RT-PCR. We therefore determined sensitivity by either excluding them from analysis, or by considering them as positive or negative to determine best case and worst case scenarios (table 1). Sensitivity of LFA ranged from 51.43% (95% CI 34.0% to 68.6%) in patient follow-up to 85.71% (95% CI 67.3% to 96.0%) in screening of all asymptomatic individuals when inconclusive results were excluded. Specificity of LFA was 100% (95% CI 99.8% to 100%) with both follow-up and screening.

Table 1.

Test characteristics of the LFA compared with the RT-PCR

	PCR result			Total
	First positive	Follow-up positive	Negative
LFA result
Positive	24	18	0	42
Negative	4	17	2327	2348
Inconclusive	2	3	11	16
Total	30	38	2338	2406

	Sensitivity	Specificity
Screening (first positive PCR)
Defining inconclusive as positive	86.67% (69.2% to 96.2%)	99.53% (99.2% to 99.8%)
Defining inconclusive as negative	80.0% (61.4% to 92.3%)	100.0% (99.8% to 100%)
Excluding inconclusive results	85.71% (67.3% to 96.0%)	100.0% (99.8% to 100%)
Follow up
Defining inconclusive as positive	55.26% (38.3 % to 71.4%)	99.53% (99.2% to 99.8%)
Defining inconclusive as negative	47.37% (31.0% to 64.2%)	100.0% (99.8% to 100%)
Excluding inconclusive results	51.43% (34.0% to 68.6%)	100.0% (99.8% to 100%)

Sensitivity and specificity are calculated for screening and follow-up and reported with 95% CI.

LFA, lateral flow assay; RT-PCR, reverse transcriptase real-time PCR.

The positive RT-PCR results were classified as presymptomatic (n=12), early infection (n=32), late infection (n=21) and persistent RNA shedding (n=3), and test characteristics were determined for presymptomatic/early asymptomatic infection, late infection/persistent RNA shedding and no infection (table 2). Sensitivity of LFA to detect subjects with presymptomatic/early asymptomatic infection ranged from 81.82% (95% CI 67.3% to 91.8%) with inconclusive test results considered negative, to 90.91% (95% CI 78.3% to 97.5%) with inconclusive test results considered positive, with specificity always above 99%. Sensitivity of LFA to detect subjects with late infection/persistent RNA shedding ranged from 25.0% (95% CI 9.8% to 46.7%) to 29.17% (95% CI 12.6% to 51.1%), with specificity always above 99%.

Table 2.

Test characteristics of the LFA for detection of early and late SARS-CoV-2 infection

	SARS-CoV-2 infection				No infection	Total
	Presymptomatic	Early asymptomatic infection	Late infection	Persistent RNA shedding
LFA result
Positive	9	27	6	0	0	42
Negative	1	3	14	3	2327	2348
Inconclusive	2	2	1	0	11	16
Total	12	32	21	3	2338	2406

	Sensitivity	Specificity
Detection of early infection
Defining inconclusive as positive	90.91% (78.3% to 97.5%)	99.53% (99.2% to 99.8%)
Defining inconclusive as negative	81.82% (67.3% to 91.8%)	100% (99.8% to 100%)
Excluding inconclusive results	90.0% (76.3% to 97.2%)	100% (99.8% to 100%)
Detection of late infection
Defining inconclusive as positive	29.17% (12.6% to 51.1)	99.53% (99.2% to 99.8%)
Defining inconclusive as negative	25.0% (9.8% to 46.7%)	100% (99.8% to 100%)
Excluding inconclusive results	26.09% (10.2% to 48.4%)	100% (99.8% to 100%)

Sensitivity and specificity are calculated for detection of presymptomatic/ early asymptomatic infection (early infection) vs no infection, and for detection of late infection/ persistent RNA shedding (late infection) vs no infection, and reported with 95% CI.

LFA, lateral flow assay.

Discrepancy analysis

False-positive LFA results were not observed. Eleven LFA tests, obtained from 11 individuals at distinct test visits, were considered inconclusive and were tested negative with RT-PCR. Follow-up tests confirmed the negative RT-PCR test result.

False-negative LFA results were observed for four individuals with presymptomatic or early asymptomatic infection. In two cases, SARS-CoV-2 infection was confirmed in a subsequent test. In one of these subjects, there was a sampling delay of 1 day between LFA and subsequent RT-PCR. The other two false-negative tests occurred in asymptomatic individuals with a single positive RT-PCR test and negative RT-PCR tests in the preceding and following week.

Most false-negative LFA tests (17/21) occurred in subjects with late infection or persistent RNA shedding. All samples were obtained a median of 15 days (IQR 10–18) after the first positive test. Compared with the false-negative LFA tests during late infection, samples with a positive LFA result (6/21) were obtained a median of 10 days (IQR 7–11) after the first positive test showing higher test sensitivity at earlier time points.

Discussion

The Panbio COVID-19 Ag rapid test identified SARS-CoV-2-infected subjects in a cohort of asymptomatic individuals, with a specificity above 99% and sensitivity of 80.0%. False-negative LFA results were mostly observed in a late phase of infection (on average 2 weeks after the first positive test) with low viral loads in nasopharyngeal swabs. The estimated sensitivity of LFA in presymptomatic and early asymptomatic infections was between 81.82% and 90.91%.

Although our cohort is not a representative sample from the general population, as it constitutes predominantly males who are often young professional athletes, our results are in line with previous reports of this LFA. In our previous validation in an ambulant symptomatic population, overall sensitivity was 72.6% but increased to 95.2% when using a Ct value of 32 as cut-off.3 In a Spanish cohort of 1369 individuals, an overall sensitivity of 71.4% was reported, which increased for symptomatic patients who presented within 5 days since symptom onset (83.1%), and for those with high viral load (87.7%).4 Another Spanish study retrospectively analysing frozen swabs reported a sensitivity of 79.5% among asymptomatic individuals and also observed an association with Ct value (Ct value <25, 100% sensitivity; Ct value <30, 98.6% sensitivity).5

For our analysis, three different laboratories performed the RT-PCR analyses, limiting the possibility for overall analyses using Ct values as a cut-off. Although repeated testing of the same individuals is a limitation of our study design as it results in lower overall sensitivity of the LFA due to the inclusion of a relatively large number of late infections during follow-up tests, the unique longitudinal prospective cohort enables us to distinguish different phases of infection and analyse test characteristics for each of these phases individually. The Panbio COVID-19 Ag rapid test identified 11 of 12 presymptomatic infections and 29 of 32 early asymptomatic infections. In a longitudinal cohort of basketball players and staff, asymptomatic individuals rapidly progressed from a negative test to a peak Ct value, as demonstrated by repeated testing,6 which could explain the good performance of this LFA during early infection. These results, combined with easy applicability and rapid results, demonstrate the potential for its use in targeted screening of asymptomatic individuals especially in populations where prevalence is low, such as screening attendees before events, passengers at transportation hubs or as part of outbreak management in non-healthcare settings. However, this test should not be used in instances where high test sensitivity is critical such as screening of healthcare workers, those who work with elderly and other at-risk groups or screening of asymptomatic patients prior to invasive procedures.

Sensitivity of the Panbio COVID-19 Ag rapid test decreased substantially during late infection (ie, more than 1 week after the first positive test). The LFA did not detect 14 of 21 late infections. Although samples were tested on different platforms limiting comparisons, all these samples had high Ct values. The risk of transmission is considered to be low or absent during late infection and will only further decrease in following days. Therefore, isolation of these individuals at that time most likely has limited effects from a public health perspective. This is supported by an analysis of LFA, RT-PCR and virus culture, in which the Panbio COVID-19 Ag rapid test detected 97.3% of samples with positive cultures, reflecting infectious individuals.7 Corroborating this, samples from symptomatic individuals visiting primary healthcare centres which tested positive by RT-PCR (Ct >25) and negative by LFA could not be cultured in vitro, indicating these individuals are indeed less infectious.8

Furthermore, modelling studies have demonstrated that surveillance effectiveness depends largely on frequency of testing and speed of reporting, rather than test sensitivity.9 10 The current study also demonstrates the feasibility of performing the Panbio COVID-19 Ag rapid test as point-of-care test, outside of a laboratory setting, by trained personnel with appropriate infection prevention control measures and use of personal protective equipment. Results were immediately reported to the physician of the football club, allowing direct intervention in case of positive LFA test results.

We observed inconclusive results in 0.66% of LFA tests. Yet, in 5 of these 16 inconclusive results, SARS-CoV-2 appeared detectable with RT-PCR. We, therefore, recommend additional sampling for RT-PCR in case of inconclusive LFA results.

In conclusion, the Panbio COVID-19 Ag rapid test is able to identify early SARS-CoV-2 infections in asymptomatic individuals and can be used in targeted screening strategies for detection of SARS-CoV-2 infection in asymptomatic individuals. Caution is advised in situations where high test sensitivity is critical.

Data availability statement

Data are available upon reasonable request. Deidentified participant data can be made available on request per email to the corresponding author following publication.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The medical research ethics committee (MREC) of Utrecht decided the study is not subject to the Medical Research Involving Human Subjects Act (WMO) and did not require full review by an accredited MREC. All participants have provided written informed consent.