Abstract
Background
Performance of point-of-care tests in different clinical scenarios and on different samples remains undetermined. We comprehensively evaluated the performance of the nasopharyngeal Panbio COVID-19 Ag Rapid Test Device.
Methods
This is a prospective study that includes consecutive patients attending 3 primary care centers (PCCs) and an emergency department. The antigen test was performed at point-of-care in nasopharyngeal and nasal swabs and in saliva. Positive percent agreement (PPA) and negative percent agreement (NPA) were calculated with the reverse-transcription polymerase chain reaction (RT-PCR) assay as reference standard.
Results
Of 913 patients included, 296 (32.3%) were asymptomatic and 690 (75.6%) came from the PCC. Nasopharyngeal swabs were collected from 913 patients, nasal swabs were collected from 659 patients, and saliva was collected from 611 patients. The RT-PCR was positive in 196 (21.5%) nasopharyngeal samples (NPS). Overall, PPA (95% CI) in NPS was 60.5% (53.3–67.4), and it was lower in nasal swabs (44.7%) and saliva (23.1%). Test performance in NPS was largely dependent on the cycle threshold (Ct) in RT-PCR, with PPA of 94% for Ct ≤25 and 80% for Ct <30. In symptomatic patients, the PPA was 95% for Ct ≤25, 85% for Ct <30, and 89% for the symptom triad of fever, cough, and malaise. Performance was also dependent on age, with a PPA of 100% in symptomatic patients >50 years with Ct <25. In asymptomatic patients, the PPA was 86% for Ct <25. In all cases, NPA was 100%.
Conclusions
The nasopharyngeal Panbio COVID-19 Ag test performed at point-of-care has a good sensitivity in symptomatic patients with Ct <30 and older age. The test was useful to identify asymptomatic patients with lower Ct values.
Keywords: antigen, COVID-19, Panbio, point-of-care, SARS-CoV-2
The nasopharyngeal Panbio-COVID-19 Ag test performed in real-life conditions at point-of-care is highly sensitive in symptomatic patients, particularly with Ct <30 and older age. The test is useful to identify asymptomatic patients with lower Ct values.
The rapid spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic worldwide requires the urgent adoption of effective preventive measures. Early diagnosis and rapid isolation of infected people are central to contain disease transmission. Although real-time reverse-transcription polymerase chain reaction (RT-PCR) is currently the reference assay for the diagnosis of SARS-CoV-2 infection, novel rapid antigen tests have emerged with several potential key advantages over molecular methods [1]. In contrast to the RT-PCR, the antigen test (1) is relatively inexpensive, simple to perform, and easy to interpret, (2) does not require infrastructure, and (3) enables obtaining point-of-care results within a few minutes. As a result, it allows immediate decisions about isolation and therapeutic interventions on infected individuals. Moreover, antigen tests are capable of identifying infected people early after infection, when viral loads are high and the likelihood of transmission is highest [2]. Despite the lower sensitivity when compared with the molecular assays, the possibility of repetitive testing with a low-cost procedure and the real-time detection of the most infective patients make the antigen a potentially high valuable test in terms of surveillance, to track and prevent the spread of the infection [3].
Information on the performance of the point-of-care SARS-CoV-2 antigen tests is limited. The sensitivity of the first-generation antigens is overall low [4]. In addition, most studies have been conducted in laboratory specimens, involved a relatively low number of samples, and the minority with available clinical data primarily included symptomatic patients [5–8]. To assess the real performance of a point-of-care test, it should be used in real-life conditions, including consecutive patients, and obtain results on site. The uncertainties about the antigen are the accuracy of the test in asymptomatic patients and how it performs in additional clinical settings, such as childhood, or old age, among others. Another relevant question is whether a more convenient sample would be a suitable alternative for diagnosis. Antigen tests are currently authorized to be performed on nasopharyngeal (NP) or nasal swabs, which need to be collected by healthcare professionals. Because saliva can be self-collected, antigen assessment in this sample would facilitate large-scale testing.
The Panbio COVID-19 antigen Rapid Test Device (RTD) (Abbott Rapid Diagnostic Jena GmbH, Jena, Germany) has been recently marketed for the qualitative detection of SARS-CoV-2 antigen in human NP swab specimens, with high sensitivity and specificity. We evaluated the performance of this point-of-care test in real-life conditions, in 3 primary care centers (PCCs) and an emergency department (ED). We assessed the accuracy of the test in symptomatic and asymptomatic patients, in different clinical scenarios, and in NP, nasal, and saliva samples.
METHODS
Study Design, Setting, and Data Collection
A prospective study was conducted from September 15 to October 29, 2020 in 3 PCC and an ED. Consecutive patients, either with COVID-19 signs/symptoms or asymptomatic contacts attending the PCC, and a majority of symptomatic patients presenting to the ED were included in the study, and only patients who refused to participate were excluded. Demographic and clinical data from primary care patients were collected using a structured questionnaire. The questionnaire included information about 6 specific symptoms and their temporality and the number of days since the initiation of symptoms. Clinical data from patients who attended the ED were obtained from the electronic health records.
Patient Consent Statement
The patient’s written consent was obtained. The design of the work was approved by the COVID-19 Institutional Committee of Hospital General Universitario de Elche (Spain).
Specimen Collection
At the PCC, patients were asked to fill out the questionnaire about symptoms and to repeatedly spit up to a minimum of 1 mL of saliva into a 100-mL sterile empty container. Then, a nasal swab from 1 nostril and 2 consecutive NP swabs (1 swab for each nostril) were obtained by a qualified nurse according to the recommended standard procedure. At the ED, 2 consecutive NP swabs, also with a different swab for each nostril, were obtained by a clinician.
Microbiological Procedures
Antigen Detection
Nasal swabs, 1 of the 2 NP swabs, and the saliva samples obtained at the PCC were tested onsite within minutes after collection for antigen detection. One of the NP swabs obtained at the ED was also analyzed onsite for antigen detection immediately after collection. The antigenic assessment in all the samples was performed using the Panbio COVID-19 Ag RTD, an immunochromatographic test with a membrane strip precoated with antibodies to the SARS CoV-2 nucleocapsid. The kit was used according to the manufacturer’s instructions. In brief, nasal and NP swabs were immersed in 2 extraction tubes containing 300 µL of buffer from the kit. A third swab was soaked in the saliva sample and then immersed in a third 300-µL tube. The 3 tubes were ready to be applied to the corresponding antigen device.
Severe Acute Respiratory Syndrome Coronavirus 2 Ribonucleic Acid Detection
The second NP swab was preserved in a 3-mL transport tube containing guanidine salt solution (Mole Bioscience, SUNGO Europe B.V., Amsterdam, Netherlands). After collection of all samples, NP specimens were transported daily by the same healthcare workers who collected the samples at the PCC to the clinical microbiology laboratory for immediate molecular analysis by RT-PCR. Nasopharyngeal samples (NPS) from the ED were also sent to the same microbiology laboratory. Nucleic acid extraction was performed using 300 µL NP specimen on Chemagic 360 Nucleic Acid Purification Instrument (PerkinElmer España SL, Madrid, Spain). Then, 10 µL eluate was used for real-time RT-PCR assay targeting the E-gene [LightMix Modular SARS-CoV (COVID19) E gene; TIB MOLBIOL, Berlin, Germany, distributed by Roche]. Testing was performed according to the manufacturer’s guidelines on Cobas z 480 Analyzer (Roche, Basilea, Suiza).
Statistical Analyses
Continuous variables are expressed as median ± 25th and 75th percentiles (Q1, Q3), and categorical variables are expressed as percentages. Wilcoxon or Student’s t test was used to compare continuous variables, and the χ 2 or Fisher’s exact test was used for categorical variables comparison. The percentage agreement (positive percent agreement [PPA], negative percent agreement [NPA], and overall predictive agreement) for Panbio antigen test in the NP, nasal, and saliva samples compared with the reference standard RT-RCR test in NP swab was calculated. Performance agreement was evaluated using Cohen’s kappa coefficient. Performance was also evaluated in NPS stratifying by age, sex, the number of cycles of amplification in RT-PCR (cycle threshold [Ct]), and duration of symptoms. Multivariate logistic regression was performed to assess predictors of the sensitivity of the antigen test in symptomatic patients. The estimated sample size for a sensitivity of at least 91.4% (according to the manufacturer), a precision of 2.5%, and a statistical power of 80% was 762 patients. For a specificity of at least 97%, sample size required was 377 patients.
RESULTS
During the study period, 913 patients were included; all of them had a NP swab for RT-PCR, 904 (99%) had a second NP swab for antigen test, 659 (72%) had a nasal swab, and 611 (67%) had a saliva sample collected. A total of 690 (75.6%) NP samples were collected from the PCC, and 223 (24.4%) were collected from the ED.
Clinical characteristics of the patients are shown in Table 1. Median (Q1–Q3) age was 40.6 (23.0–55.6) years, and 423 (46.3%) were men. The most common comorbidities were dyslipidemia in 80 (22.2%) patients, hypertension in 124 (17.0%), and diabetes in 60 (8.2%). There were 617 (67.6%) symptomatic patients and 296 (32.4%) were asymptomatic. Median (Q1–Q3) number of days from symptom onset was 3 (2–5) days, and the most frequent symptoms were cough (50.1%), followed by fever (46.8%), sore throat (31.9%), and nasal congestion (31.3%). Median (Q1–Q3) Ct was 24 (16–30); 22 (16–29) in symptomatic and 28 (21–32) in asymptomatic patients (P = .012); and 21 (15–27) in patients ≥50 years and 26 (18–31) in <50 years (P = .02).
Table 1.
Clinical Characteristics of the Patients
| Variable | All Patients N = 913 | Symptomatic N = 617 | Asymptomatic N = 296 | P |
|---|---|---|---|---|
| Age (years) median (Q1–Q3) | 40.6 (23–55.6) | 41 (24–56.3) | 39.9 (20.4–52.5) | .086 |
| Sex (men) | 423 (46.3) | 289 (46.8) | 134 (45.3) | .671 |
| Dyslipidemia | 80 (22.2) | 63 (24.3) | 17 (16.7) | .124 |
| Hypertension | 124 (17.0) | 96 (19.0) | 28 (12.4) | .033 |
| Diabetes | 60 (8.2) | 49 (9.7) | 11 (4.9) | .029 |
| Cardiomyopathy | 55 (7.6) | 43 (8.5) | 12 (5.3) | .171 |
| Obesity | 42 (5.8) | 25 (5.1) | 17 (7.6) | .228 |
| COPD | 25 (3.4) | 20 (4.0) | 5 (2.2) | .277 |
| Active cancer | 17 (2.3) | 14 (2.8) | 3 (1.3) | .295 |
| Primary care center | 690 (75.6) | 416 (67.4) | 274 (92.6) | <.001 |
| Emergency department | 223 (24.4) | 201 (32.6) | 22 (7.4) | <.001 |
| Positive SARS-CoV-2 RNAa | 196 (21.5) | 156 (25.3) | 40 (13.5) | <.001 |
| Ct, median (Q1–Q3) | 24 (16–30) | 22 (16–29) | 28 (21–32) | .012 |
| Ct 0–20 | 78 (40.0) | 68 (43.9) | 10 (25.0) | .10 |
| Ct 21–25 | 29 (14.9) | 24 (15.5) | 5 (12.5) | |
| Ct 26–30 | 40 (20.5) | 28 (18.1) | 12 (30.0) | |
| Ct 31–35 | 40 (20.5) | 30 (19.4) | 10 (25.0) | |
| Ct >35 | 8 (4.1) | 5 (3.2) | 3 (7.5) | |
| Positive antigen (any NP-nasal-saliva) result | 120 (13.1) | 106 (17.2) | 14 (4.7) | <.001 |
| Positive NP antigen result | 118 (12.9) | 105 (17.2) | 13 (4.5) | <.001 |
| No. of days with symptoms | 3 (2–5) | |||
| Cough | 309 (50.1) | |||
| Fever | 289 (46.8) | |||
| Sore throat | 196 (31.9) | |||
| Nasal congestion | 193 (31.3) | |||
| Dyspnea | 115 (18.6) | |||
| Anosmia/ageusia | 53 (8.8)/44 (7.1) | |||
| Others: malaise/headache | 104 (17)/140 (23) |
Abbreviations: COPD, chronic obstructive pulmonary disease; Ct, cycle threshold of reverse-transcription polymerase chain reaction; NP, nasopharyngeal; RNA, ribonucleic acid; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
NOTE: Categorical variables are represented by number and (%).
aPerformed on nasopharyngeal samples.
Performance of the Antigen Test by Type of Sample and Site of Care
There were 196 (21.5%) samples positive for SARS-CoV-2 by RT-PCR and 120 (13.1%) positive antigen results. Performance of the test by type of sample is shown in Figure 1 and Table 2. In NPS, the overall PPA and NPA of the antigen test were 60.5% (95% confidence interval [CI], 53.3%–67.4%) and 100% (95% CI, 99.3%–100%), respectively. In the saliva, the PPA was 23.1% (95% CI, 16.2–31.9), and in nasal samples the PPA was 44.7% (95% CI, 36.1%–53.6%).
Figure 1.
Performance of Panbio COVID-19 Ag Rapid Test Device by type of sample.
Table 2.
Performance of the Panbio COVID-19 Antigen Rapid Test Device by Type of Sample and Site of Care
| Variable | TP | FP | TN | FN | PPA (95% CI) | NPA (95% CI) | OPA (95% CI) | Kappa (95% CI) |
|---|---|---|---|---|---|---|---|---|
| Type of Sample | ||||||||
| NP sample | 118 | 0 | 709 | 77 | 60.5% (53.3–67.4) | 100% (99.3–100) | 91.5% (89.4–93.2) | 0.71 (0.65–0.77) |
| Nasal sample | 59 | 0 | 527 | 73 | 44.7% (36.1–53.6) | 100% (99.1–100) | 88.9% (86.2–91.2) | 0.56 (0.48–0.65) |
| Saliva sample | 28 | 0 | 490 | 93 | 23.1% (16.2–31.9) | 100% (99–100) | 84.8% (81.6–87.6) | 0.33 (0.23–0.42) |
| Site of Care | ||||||||
| Primary care | 78 | 0 | 544 | 59 | 56.9% (48.2–65.3) | 100% (99.1–100) | 91.3% (88.9–93.3) | 0.68 (0.60–0.75) |
| Emergency department | 40 | 0 | 165 | 18 | 69% (55.3–80.1) | 100% (97.2–100) | 91.9% (87.3–95) | 0.77 (0.67–0.87) |
Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; FN, false negative; FP, false positive; NP, nasopharyngeal; NPA, negative percent agreement; OPA, overall predictive agreement; PPA, positive percent agreement; TN, true negative; TP, true positive; y, year.
NOTE: Unless specified, all analyses have been performed in NP samples.
By site of care, the PPA in NPS at the ED was 69.0% (95% CI, 55.3%–80.1%). At the PCC, the PPA was 56.9% (95% CI, 48.2%–65.3%) (Table 2).
Performance of the Antigen Test in Nasopharyngeal Samples by Polymerase Chain Reaction Cycle Threshold, Age, and Sex
The performance of the test by Ct in NPS is shown in Figure 2A. In the analysis including all patients, a gradual decline in sensitivity was observed with increasing Ct values, with a more prominent decrease from Ct >28. The PPA was 94% (95% CI, 85%–98%) for Ct ≤25 and 80% (95% CI, 67%–85%) for Ct <30.
Figure 2.
Performance of nasopharyngeal Panbio COVID-19 Ag Rapid Test Device in different scenarios. (A) Performance according to cycle threshold (Ct) values. (B) Performance according to age.
Table 3 and Figure 2B show the performance of the antigen test according to age group. The PPA increased with increasing age, with lower sensitivity among children and young adults (PPA 38.1% [95% CI, 24.0%–54.3%] for 15–30 years) and higher sensitivity in older patients (PPA 72.4% [95% CI, 52.5%–86.6%] for ≥65 years).
Table 3.
Performance of the Panbio COVID-19 Antigen Rapid Test Device in Nasopharyngeal Samples According to Different Factors
| Variable | TP | FP | TN | FN | PPA (95% CI) | NPA (95% CI) | OPA (95% CI) | Kappa (95% CI) |
|---|---|---|---|---|---|---|---|---|
| Sex | ||||||||
| Men | 64 | 0 | 317 | 37 | 63.4% (53.1–72.6) | 100% (98.5–100) | 91.1% (87.9–93.6) | 0.72 (0.64–0.81) |
| Women | 54 | 0 | 392 | 40 | 57.4% (46.8–67.5) | 100% (98.8–100) | 91.8% (88.9–94) | 0.69 (0.60–0.77) |
| Age | ||||||||
| ≤14 y | 10 | 0 | 107 | 8 | 55.6% (31.3–77.6) | 100% (95.7–100) | 93.6% (87.4–97) | 0.68 (0.48–0.88) |
| 15–30 y | 16 | 0 | 145 | 26 | 38.1% (24–54.3) | 100% (96.8–100) | 86.1% (80.1–90.6) | 0.49 (0.33–0.65) |
| 31–50 y | 42 | 0 | 244 | 27 | 60.9% (48.4–72.2) | 100% (98.1–100) | 91.4% (87.6–94.1) | 0.71 (0.61–0.81) |
| 51–65 y | 29 | 0 | 111 | 8 | 78.4% (61.3–89.6) | 100% (95.8–100) | 94.6% (89.3–97.5) | 0.85 (0.74–0.95) |
| >65 y | 21 | 0 | 102 | 8 | 72.4% (52.5–86.6) | 100% (95.5–100) | 93.9% (87.9–97.1) | 0.80 (0.67–0.93) |
| Symptomatic | ||||||||
| Overall | 105 | 0 | 456 | 51 | 67.3% (59.3–74.5) | 100% (99–100) | 91.7% (89.1–93.7) | 0.75 (0.69–0.82) |
| ≤3 DSO | 49 | 0 | 273 | 13 | 79% (66.5–87.9) | 100% (98.3–100) | 96.1% (93.3–97.8) | 0.86 (0.79–0.93) |
| ≤4 DSO | 65 | 0 | 334 | 18 | 78.3% (67.6–86.3) | 100% (98.6–100) | 95.7% (93.1–97.3) | 0.85 (0.79–0.92) |
| ≤5 DSO | 76 | 0 | 364 | 22 | 77.6% (67.8–85.1) | 100% (98.7–100) | 95.2% (92.8–96.9) | 0.85 (0.78–0.91) |
| ≤6 DSO | 81 | 0 | 382 | 23 | 77.9% (68.5–85.2) | 100% (98.8–100) | 95.3% (92.9–96.9) | 0.85 (0.79–0.91) |
| ≥7 DSO | 24 | 0 | 74 | 28 | 46.2% (32.5–60.4) | 100% (93.9–100) | 77.8% (69.3–84.5) | 0.50 (0.36–0.64) |
| Ct ≤25 | 87 | 0 | 0 | 5 | 95% (87–98) | |||
| Ct ≤30 | 102 | 0 | 0 | 18 | 85% (77–91) | |||
| Ct ≤35 | 105 | 0 | 0 | 45 | 70% (62–77) | |||
| Asymptomatic | ||||||||
| Overall | 13 | 0 | 253 | 26 | 33.3% (19.6–50.3) | 100% (98.1–100) | 91.1% (87.1–94) | 0.46 (0.30–0.63) |
| Ct ≤25 | 12 | 0 | 0 | 2 | 86% (56–97) | |||
| Ct ≤30 | 13 | 0 | 0 | 13 | 50% (32–68) | |||
| Ct ≤35 | 13 | 0 | 0 | 23 | 36% (21–54) |
Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; Ct, cycle threshold of reverse-transcription polymerase chain reaction; DSO, days from symptom onset; FN, false negative; FP, false positive; PPA, positive percent agreement; NPA, negative percent agreement; OPA, overall predictive agreement; TN, true negative; TP, true positive; y, year.
NOTE: Unless specified, all analyses have been performed in nasopharyngeal samples.
No remarkable differences were found in the antigen test performance by sex. The PPA in men was 63.4% (95% CI, 53.1–72.6), and the PPA in women was 57.4% (95% CI, 46.8–67.5) (Table 3).
Performance of the Antigen Test in Nasopharyngeal Samples in Symptomatic Patients
A total of 617 (67.6%) patients presented with clinical symptoms. Median (Q1–Q3) age was 41.0 (24.0–56.3) years and 289 (46.8%) were men (Table 1). In 156 (25.3%) patients, the RT-PCR was positive in the NPS, and in 105 (17.2%) the antigen test was positive. The PPA (95% CI) in the NPS samples was 67.3% (59.3–74.5). The PPA by type of sample in symptomatic patients is shown in Figure 1.
The performance of the antigen test in NPS by Ct is shown in Figure 2A and Table 3. The sensitivity of the test decreased more slowly with increasing Ct in symptomatic patients than in the overall sample, but a faster decrease was again observed from Ct values >28. The PPA was 95% (95% CI, 87–98) for Ct ≤25 and 85% (77–91) for Ct <30. By age, the antigen test performance increased with increasing age, with a PPA of 79.3% (95% CI, 66.3–88.4) in symptomatic patients >50 years.
Table 2b shows the performance of the antigen test in NPS by number of days with symptoms. The PPA was approximately 80% for a period of less than 7 days from symptom onset, and it fell during the following days.
The performance of the antigen test varied in relation to the characteristics of the symptoms. The highest sensitivity was observed for malaise and ageusia, with a PPA of 75% each, followed by sore throat with PPA of 73%, and cough, nasal congestion, and dyspnea, the 3 with PPA of 69%. The PPA for the triad of cough, fever, and malaise was 89%.
Figure 3 shows the performance of the antigen test according to the presence of symptoms, age, Ct, and days after symptoms onset. The highest PPA of the test was observed for Ct <25, for which the PPA was >90% for age >15 years; 100% for >50 years; and 95% for duration of symptoms <7 days.
Figure 3.
Performance of nasopharyngeal Panbio COVID-19 Ag Rapid Test Device after stratification for different factors. (A) Performance stratified by the presence of symptoms, age, and cycle threshold (Ct) values. (B) Performance stratified by symptom duration and age.
A multivariate logistic regression was run to explore the independent factors associated with antigen test performance among symptomatic patients, including age, sex, Ct values, and duration of symptoms categorized into <7 or ≥7 days. The model showed that the PPA was independently associated with age, with an odds ratio (OR) of 1.20 (95% CI, 1.03–1.40) for each 5-years-older period, duration of symptoms with an OR of 3.99 (1.22–13.06) for <7 days, and inversely associated with the Ct, with an OR of 0.66 (95% CI, 0.57–0.76).
Performance of the Antigen Test in Nasopharyngeal Samples in Asymptomatic Patients
A total of 296 patients were asymptomatic, with median (Q1–Q3) age of 39.9 (20.4–52.5) years, and 134 (45.3%) were men. The PPA by type of sample in asymptomatic patients is shown in Figure 1. A total of 39 (13.2%) patients had a positive RT-PCR in the NPS, and 13 (4.4%) had a positive NP antigen test. The PPA in the NPS was 33.3% (95% CI, 19.6–50.3), and Cohen’s kappa coefficient was 0.46 (95% CI, 0.30–0.63).
Figures 2A and 3A and Table 3 show the performance of the antigen test by Ct and age in asymptomatic patients. Again, a decrease in the sensitivity was observed with increasing Ct, but it was much more pronounced than in symptomatic patients, mainly from Ct >20 (Figure 2A). However, for low Ct, the sensitivity was high for all age groups, with an overall PPA of 86% (95% CI, 56–97) for Ct ≤25.
DISCUSSION
We evaluated a recent generation point-of-care antigen test for SARS-CoV-2 in real-life conditions in a large population of consecutive patients and onsite, where the test was conceived to be performed. Our data show that the sensitivity of the antigen test is largely dependent on the Ct values, age, and the presence and duration of symptoms. The sensitivity of the antigen test was highest in symptomatic patients older than 50 years and with Ct values associated with an increased risk of infectivity, reaching 100% in this scenario. Although the performance of the test was overall lower in asymptomatic patients, again the antigen identified with a sensitivity higher than 85% those with lower Ct, and therefore with higher contagious risk. In all cases, the specificity of the antigen test was approximately 100%. Finally, although the saliva would facilitate mass testing for surveillance, the low sensitivity of the antigen in this specimen does not support its use as an alternative sample.
In contrast to SARS-CoV, SARS-CoV-2 infection is associated with high levels of viral shedding at the initial stages of the infection in the upper respiratory tract, which facilitates detecting the virus during the most infectious period. The availability of a rapid point-of-care test for the diagnosis allows testers to adopt immediate and real-time decisions, which is a clear advantage over the RT-PCR in controlling the spread of the infection. Although the antigen test showed an overall lower sensitivity than the RT-PCR in our study, and that reported by the manufacturer, the test was highly accurate in symptomatic patients exhibiting lower Ct values, with a sensitivity greater than 95% for Ct of 25 or lower and at least 85% for Ct of less than 30. These results are in agreement with the sensitivity described with BinaxNOW COVID-19 Ag CARD, with 100% of samples with viral loads equivalent to Ct of 29–30 being detected by the rapid antigen test [9]. High SARS-CoV-2 viral load has been associated with severity of disease and mortality [10, 11], and several studies support a correlation of Ct values with infectivity, as defined by growth in cell culture. Although breakpoints fluctuate among different studies [12–15], a diagnostic Ct value of RT-PCR equal to or greater than 33 was associated with no isolation of SARS-CoV-2 using cell-based cultures, nor with active viral replication [12, 13]. Other studies report no SARS-CoV-2 recovery in cell culture with Ct values higher than 29.5 [14], or even higher than 24, with a decrease by 32% of positive cultures for each unit of increase in Ct [15]. Although the correlation between viral growth and infectivity needs to be confirmed, our data suggest that the point-of-care antigen test is useful to detect most SARS-CoV-2-infected symptomatic patients and to identify those with significant transmission risk. Another factor influencing the performance of the antigen test was the duration of symptoms. As specified by the manufacturer and also previously reported [6, 16], we found a higher sensitivity of the test within a period of less than 7 days from the initiation of symptoms.
Because most SARS-CoV-2 infections are asymptomatic, the performance of the antigen test in this scenario needs to be established, and this information is key for strategies aimed at preventing the spread of the infection at the community level. Our study shows that the sensitivity was poorer when compared with that of patients with symptoms. Identical to symptomatic participants, the sensitivity was highly dependent on the Ct, and we found a PPA higher than 85% for Ct ≤25. Asymptomatic SARS-CoV-2 transmission has been previously reported [17, 18], but the secondary attack rate from either asymptomatic or presymptomatic patients was found to be lower (in a meta-analysis) compared with that of patients with symptoms [19] and to be as low as 0.3%–0.6% [20]. In presymptomatic patients, SARS-CoV-2 growth in viral culture was rarely observed with a Ct above 25 [17]. Although the performance of the test was inferior in asymptomatic individuals, the increase in sensitivity observed with lower Ct coupled with the lower transmission risk described within this group could make the antigen a potentially helpful tool to identify those with infective risk among asymptomatic patients. Similar to other studies [21], because we did not follow up with patients, we cannot distinguish the proportion of asymptomatic SARS-CoV-2-infected individuals who remained asymptomatic throughout from those who were presymptomatic and developed symptoms later in the course of the infection. The latter patients may have the chance to be detected by the test in ulterior examinations, thereby increasing the sensitivity of an assay that allows repeated testing because of its inexpensiveness and simplicity.
In addition to the viral load, the sensitivity of the antigen test was highly dependent on age. We found that older patients showed the best antigen test performance, a finding that was not previously described. Younger children showed the poorest antigen test performance, and there was a gradual increase in the sensitivity of the test with age, with the highest values in older patients. Several factors might explain this finding. Children showed less cooperation or even resistance during the collection of the NPS. Other factors such as temporality of symptoms or the higher Ct values among younger patients might also have played a role. However, our study showed that age was associated with the antigen test performance independently of the Ct and duration of symptoms, a finding that merits further investigation.
We explored the performance of the antigen test in alternative locations to the nasopharynx (recommended by the manufacturer), which could be more useful for surveillance, such as the nose or saliva. Because saliva can be self-collected, this sample would be the most advantageous if mass testing was considered with the point-of-care antigen test. Saliva has additionally shown to be a suitable alternative sample to NP swab for SARS-CoV-2 detection by RT-PCR [22]. Unfortunately, our study shows that the sensitivity of the point-of-care antigen test is low in the saliva. Different factors could be implicated in the lower sensitivity, including the dilution effect of the extraction buffer, lower viral shedding in the saliva, inadequate quality of the sample collected, etc. The same result occurred with the nasal swabs, where the test performance was not satisfactory.
Limitations of the study include (1) the lack of statistical power for the analysis of the test performance in specific subgroups, especially in asymptomatic patients, in whom sensitivity according to Ct ranges might not be accurate due to small sample sizes, and (2) the incomplete information about the number of days since the risk contact in asymptomatic patients. The strengths of our study are (1) the real-life conditions in which the antigen test has been used to assess its true performance, (2) the inclusion of consecutive unselected patients, which allowed us to analyze how it performs in diverse clinical scenarios, and (3) the onsite execution of the test.
CONCLUSIONS
In conclusion, the sensitivity of the NP Panbio COVID-19 antigen RDT is closely related to the Ct values, age, and the presence and duration of symptoms. The test performance is optimal in symptomatic patients older than 50 years with viral loads linked with infectivity, and in asymptomatic patients the test was useful to identify those with lower Ct values. The saliva was not shown to be a suitable alternative sample for antigen detection.
Acknowledgments
Financial support. This work was supported by the RD16/0025/0038 project as a part of the Plan Nacional Research + Development + Innovation (R+D+I) and cofinanced by Instituto de Salud Carlos III - Subdirección General de Evaluación y Fondo Europeo de Desarrollo Regional; Instituto de Salud Carlos III (Fondo de Investigaciones Sanitarias (Grant Numbers PI16/01740, PI18/01861; CM 19/00160, COV20-00005).
Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.
Members of the COVID19-Elx-Rapid Diagnostic Tests Study Group Félix Gutiérrez, Mar Masiá, Sergio Padilla, Guillermo Telenti, Lucia Guillén, Javier García-Abellán, Cristina Bas, María Andreo, Fernando Lidón, Vladimir Ospino, José López, Marta Fernández, Vanesa Agulló, Gabriel Estañ, Javier García, Cristina Martínez, Leticia Alonso, Joan Sanchís, Ángela Botella, Paula Mascarell, María Selene Falcón, Sandra Ruiz, José Carlos Asenjo, Carolina Ding, Mar Carvajal, Alba de la Rica, Inmaculada Candela, Jorge Guijarro, Cristina la Moneda, Cristina Jara, Raquel Mora, Juan Manuel Quinto, Sergio Ros, Daniel Canal, Pascual Pérez, Francisco Carrasco, Carolina Garrido, Carlos Gosálbez, Jaime Sastre, Manuel Sánchez, Carlos de Gregorio, Juan Navarro, Andrés Navarro, Nieves Gonzalo, Clara Pérez, Adoración Alcalá, José Pastor, José Luis Rincón, Montserrat Ruiz, and Juan Antonio Gutiérrez.
References
- 1. La Marca A, Capuzzo M, Paglia T, et al. Testing for SARS-CoV-2 (COVID-19): a systematic review and clinical guide to molecular and serological in-vitro diagnostic assays. Reprod Biomed Online 2020; 41:483–99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. He X, Lau EHY, Wu P, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med 2020; 26:672–5. [DOI] [PubMed] [Google Scholar]
- 3. Mina MJ, Parker R, Larremore DB. Rethinking covid-19 test sensitivity—a strategy for containment. N Engl J Med 2020; 383:e120. [DOI] [PubMed] [Google Scholar]
- 4. Dinnes J, Deeks JJ, Adriano A, et al. ; Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2020; 8:CD013705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Young S, Taylor SN, Cammarata CL, et al. Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS antigen point-of-care test. J Clin Microbiol 2020; 59:e02338–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Linares M, Pérez-Tanoira R, Carrero A, et al. Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms. J Clin Virol 2020; 133:104659. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Blairon L, Wilmet A, Beukinga I, Tré-Hardy M. Implementation of rapid SARS-CoV-2 antigenic testing in a laboratory without access to molecular methods: experiences of a general hospital. J Clin Virol 2020; 129:104472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Lambert-Niclot S, Cuffel A, Le Pape S, et al. Evaluation of a rapid diagnostic assay for detection of SARS-CoV-2 antigen in nasopharyngeal swabs. J Clin Microbiol 2020; 58:e00977-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Perchetti GA, Huang ML, Mills MG, et al. Analytical sensitivity of the Abbott BinaxNOW COVID-19 Ag CARD. J Clin Microbiol 2020; doi: 10.1128/JCM.02880-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Zheng S, Fan J, Yu F, et al. Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China, January-March 2020: retrospective cohort study. BMJ 2020; 369:m1443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Pujadas E, Chaudhry F, McBride R, et al. SARS-CoV-2 viral load predicts COVID-19 mortality. Lancet Respir Med 2020; 8:e70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. La Scola B, Le Bideau M, Andreani J, et al. Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards. Eur J Clin Microbiol Infect Dis 2020; 39:1059–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Basile K, McPhie K, Carter I, et al. Cell-based culture of SARS-CoV-2 informs infectivity and safe de-isolation assessments during COVID-19. Clin Infect Dis 2020; doi: 10.1093/cid/ciaa1579 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Gniazdowski V, Morris CP, Wohl S, et al. Repeat COVID-19 molecular testing: correlation of SARS-CoV-2 culture with molecular assays and cycle thresholds. Clin Infect Dis 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Bullard J, Dust K, Funk D, et al. Predicting infectious SARS-CoV-2 from diagnostic samples. Clin Infect Dis 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Corman VM, Haage VC, Bleicker T, et al. Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests [preprint]. medRxiv doi: 10.1101/2020.11.12.20230292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Arons MM, Hatfield KM, Reddy SC, et al. ; Public Health–Seattle and King County and CDC COVID-19 Investigation Team. Presymptomatic SARS-CoV-2 infections and transmission in a skilled nursing facility. N Engl J Med 2020; 382:2081–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Chau NVV, Thanh Lam V, Thanh Dung N, et al. The natural history and transmission potential of asymptomatic SARS-CoV-2 infection. Clin Infect Dis 2020; 71:2679–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Buitrago-Garcia D, Egli-Gany D, Counotte MJ, et al. Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: a living systematic review and meta-analysis. PLoS Med 2020; 17:e1003346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Luo L, Liu D, Liao X, et al. Contact settings and risk for transmission in 3410 close contacts of patients with COVID-19 in Guangzhou, China: a prospective cohort study. Ann Intern Med 2020; 173:879–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Oran DP, Topol EJ. Prevalence of asymptomatic SARS-CoV-2 infection: a narrative review. Ann Intern Med 2020; 173:362–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Azzi L, Carcano G, Gianfagna F, et al. Saliva is a reliable tool to detect SARS-CoV-2. J Infect 2020; 81:e45–50. [DOI] [PMC free article] [PubMed] [Google Scholar]