Diagnostic accuracy of SARS-CoV-2 saliva antigen testing in a real-life clinical setting

Abstract

Background

SARS-CoV-2 antigen tests with saliva facilitate examination in settings that lack trained personnel. However, little is known about the diagnostic accuracy in real-life clinical settings. Therefore, we studied the diagnostic accuracy of a saliva antigen test in diagnosing SARS-CoV-2 infection in a primary/secondary care testing facility.

Methods

Individuals who presented at a COVID-19 testing facility affiliated with a Swiss university hospital were prospectively recruited (n=377). Saliva specimen was obtained, and the PCL Inc. COVID19 Gold antigen test was conducted in parallel with 2 real-time polymerase chain reaction (RT-PCR) assays from a nasopharyngeal swab.

Results

RT-PCR results were positive in 53 individuals, corresponding to a prevalence of 14.1% (missing material in 1 individual). The PCL saliva antigen test was positive in 22 individuals (5.8%) and negative in 354 (93.9%). The sensitivity of the saliva antigen test was 30.2% (95% confidence interval 18.3, 44.3), both overall and in symptomatic individuals. The specificity was 98.1% (96.0, 99.3).

Conclusions

The diagnostic accuracy of a SARS-CoV-2 saliva antigen test in a primary/secondary care testing facility was remarkably lower than that reported in the manufacturer's specifications.

Background

Testing for SARS-CoV-2 is an essential component of the pandemic response. Rapid antigen tests using saliva were suggested as a quick, simple, comfortable, and non-invasive testing method. Only minimal training is required to conduct these tests, facilitating application in various primary care and even self-testing settings. Several studies suggested that saliva antigen tests might have an adequate performance in diagnosing SARS-CoV-2 infection (Mattiuzzi et al., 2020). However, little is known about the diagnostic accuracy in real-life clinical settings, which might be significantly different from that reported in manufacturer data (Jegerlehner et al., 2021, Mattiuzzi et al., 2020). Manufacturers often claim a sensitivity of approximately 95% (94.3% in case of the assay mentioned below).

In this prospective cross-sectional study, we assessed the diagnostic accuracy of a SARS-CoV-2 saliva antigen test in a real-life primary/secondary care setting.

Patients and Methods

This study was conducted in line with an established prospective cross-sectional study; all methodological details were described previously (Jegerlehner et al., 2021). Consecutive individuals presenting at a COVID-19 testing facility affiliated with a Swiss university hospital between September and December 2021 were included, in a period when the Delta variant was predominant, at over 90%. The following inclusion criteria were applied: (a) suspected SARS-CoV-2 infection (including asymptomatic individuals following exposure), (b) age ≥18 years, and (c) signed informed consent. The flow of the individuals is given in Figure S1 of the supplementary material. The study protocol was approved by the appropriate ethical committee (Kantonale Ethikkommission Bern #2020-02729). All participants signed informed consent.

Clinical data were obtained using a detailed questionnaire (CDC, 2020, Health, 2021, Jegerlehner et al., 2021). The time point of last oral intake (food or drink) was recorded. A specially trained nurse collected the saliva specimen in parallel with the nasopharyngeal swab collected for real-time polymerase chain reaction (RT-PCR). Sample material was processed within 15 minutes (saliva antigen test) or 12 hours (RT-PCR, stored at 4°C), respectively. Details of the RT-PCR determination have been reported previously (Brigger et al., 2021, Horn et al., 2022, Jegerlehner et al., 2021).

An immunochromatographic lateral-flow immunoassay was used for the detection of SARS-CoV-2 (COVID19 Gold; PCL Inc., Seoul, Rep. of Korea; www.pclchip.com). SARS-CoV-2 antibodies are labeled with small gold particles and attached on a nitrocellulose membrane. The saliva antigen test was performed in parallel by a trained nurse. The instructions of the manufacturer were strictly followed (package leaflet); internal and external controls were applied. Participants were asked not to eat, drink, or smoke 30 minutes before sampling. After collecting saliva in the mouth, the participants spitted approximately 500 µL in the test tube filled with 500 µL of extraction buffer. The tubes were mixed and 2 drops were applied to the sample hole of the test card. The results were recorded after 10 minutes.

Statistical analyses were done using the Stata 14.2 statistical software (College Station, Tx: StataCorp LP). As measures of diagnostic accuracy, sensitivities and specificities were calculated with the help of a 4 × 4 table, considering the saliva antigen test as the index test and the RT-PCR as the reference standard (Mallett et al., 2012).

Results

Overall, 377 participants were eventually included (Figure S1). Most individuals presented with symptoms consistent with SARS-CoV-2 infection (n=327; 86.7%). Fifty asymptomatic individuals were referred for workup upon exposure (13.3%). Detailed patient characteristics are given in Table 1 . Fifty-three individuals tested positive with the RT-PCR done with the nasopharyngeal swab (prevalence 14.1%). Overall, the sensitivity of the saliva antigen test was 30.2% (95% confidence interval [CI] 18.3, 44.3) and specificity was 98.1% (95% CI 96.0, 99.3). Among symptomatic patients, the sensitivity was 30.2 % (95% CI 18.3, 44.3) and specificity 97.8% (95.3, 99.2).

Table 1.

Characteristics of 377 study participants who presented at a COVID-19 testing facility affiliated with an emergency department of a university hospital.

Characteristic	All individuals	RT-PCR negative individuals	RT-PCR positive individuals	Missing values
Numbers of patients(%)	377 (100)	323 (85.7)	53 (14.1)	1 (0.3)
Age,mean (SD)	31.4 (10.4)	31.5 (10.5)	30.3 (10.2)	0
Female(numbers, %)	221 (58.6)	191 (59.1)	29 (54.7)	0
Reason for testing(numbers, %)
Symptomsa	327 (86.7)	273 (84.5)	53 (100)	0
Exposureb	50 (13.3)	50 (15.5)	0	0
Presence of symptoms(numbers, %)
Any symptom	327 (86.7)	273 (84.5)	53 (100)	0
Acute respiratory symptoms	172 (45.7)	144 (44.6)	28 (52.8)	0
Fever	81 (21.5)	62 (19.2)	19 (35.9)	0
Loss of smell and taste	27 (7.2)	19 (5.9)	8 (15.1)	0

All numbers and percentages refer to the subset of patients indicated in the respective column (all individuals, RT-PCR positives, or RT-PCR negatives).

Abbreviations: RT-PCR, real-time polymerase chain reaction; SD, standard deviation.

^aIndividuals presenting at a COVID-19 testing facility because of symptoms consistent with COVID-19.

^bIndividuals presenting because of exposure to individuals with SARS-CoV-2 infection.

The number of false-negative test results was 37, and the number of false-positive test results was 6 (n=16 true positives; n=317 true negatives). The sensitivity of the saliva antigen test according to adapted cycle thresholds (CTs) of RT-PCR is given in Figure 1, panel B. The sensitivity ranged from 30.2% (CT 40) to 33.3% (CT 26). Among 37 individuals with false-negative test results, the time point of last food or drink intake was shorter than 30 minutes in 2 individuals (25 minutes, 20 minutes) (Fig. 1) .

Figure 1.

Diagnostic accuracy of a SARS-CoV-2 saliva antigen test in a real-life clinical setting. 377 individuals who presented at a COVID-19 testing facility affiliated with a university hospital were included. (A) Sensitivities and specificities are given according to RT-PCR, including 95% confidence intervals. (B) Sensitivities in relation to adapted CTs of RT-PCR. The manufacturer uses 40 cycles, and if a signal is detected within these 40 cycles, the sample is considered positive.

CT, cycle threshold; RT-PCR, real-time polymerase chain reaction.

Discussion

In a prospective cross-sectional study conducted in the real-life clinical setting of a primary/secondary care testing facility, the overall sensitivity of a saliva antigen test was 30.2%. Lower CT thresholds of the RT-PCR did not significantly change the sensitivity. This result is substantially lower than that reported in the manufacturer's specifications (sensitivity 94.3%).

Our results are consistent with previous studies that have shown low sensitivities of antigen tests in real-life clinical settings (De Marinis et al., 2021, Igloi et al., 2021, Jegerlehner et al., 2021, Kritikos et al., 2021). However, our results contrast with other studies and manufacturers’ data investigating antigen tests with more restricted study designs (Graham et al., 2021). A Cochrane Review pointed to the limitations of these studies, major methodological concerns, and a high risk of bias (Dinnes et al., 2020). As a limitation, our results were obtained using 1 particular antigen test in 1 particular setting. However, strikingly low sensitivities were observed in several studies assessing antigen tests in realistic settings (De Marinis et al., 2021, Igloi et al., 2021, Jegerlehner et al., 2021, Kritikos et al., 2021).

In conclusion, the diagnostic accuracy of the PCL saliva antigen test in diagnosing SARS-CoV-2 infection in a primary/secondary care testing facility was considerably lower compared with that reported in manufacturer's data. This should be taken into consideration when setting up testing strategies.

Competing interests

MN received research support from Roche diagnostics outside of the present work. The other authors declare no conflict of interest.