A novel strategy for SARS-CoV-2 mass screening with quantitative antigen testing of saliva: a diagnostic accuracy study

Abstract

Background

Quantitative RT-PCR (RT-qPCR) of nasopharyngeal swab (NPS) samples for SARS-CoV-2 detection requires medical personnel and is time consuming, and thus is poorly suited to mass screening. In June, 2020, a chemiluminescent enzyme immunoassay (CLEIA; Lumipulse G SARS-CoV-2 Ag kit, Fujirebio, Tokyo, Japan) was developed that can detect SARS-CoV-2 nucleoproteins in NPS or saliva samples within 35 min. In this study, we assessed the utility of CLEIA in mass SARS-CoV-2 screening.

Methods

We did a diagnostic accuracy study to develop a mass-screening strategy for salivary detection of SARS-CoV-2 by CLEIA, enrolling hospitalised patients with clinically confirmed COVID-19, close contacts identified at community health centres, and asymptomatic international arrivals at two airports, all based in Japan. All test participants were enrolled consecutively. We assessed the diagnostic accuracy of CLEIA compared with RT-qPCR, estimated according to concordance (Kendall's coefficient of concordance, W), and sensitivity (probability of CLEIA positivity given RT-qPCR positivity) and specificity (probability of CLEIA negativity given RT-qPCR negativity) for different antigen concentration cutoffs (0·19 pg/mL, 0·67 pg/mL, and 4·00 pg/mL; with samples considered positive if the antigen concentration was equal to or more than the cutoff and negative if it was less than the cutoff). We also assessed a two-step testing strategy post hoc with CLEIA as an initial test, using separate antigen cutoff values for test negativity and positivity from the predefined cutoff values. The proportion of intermediate results requiring secondary RT-qPCR was then quantified assuming prevalence values of RT-qPCR positivity in the overall tested population of 10%, 30%, and 50%.

Findings

Self-collected saliva was obtained from 2056 participants between June 12 and Aug 6, 2020. Results of CLEIA and RT-qPCR were concordant in 2020 (98·2%) samples (Kendall's W=0·99). Test sensitivity was 85·4% (76 of 89 positive samples; 90% credible interval [CrI] 78·0–90·3) at the cutoff of 0·19 pg/mL; 76·4% (68 of 89; 68·2–82·8) at the cutoff of 0·67 pg/mL; and 52·8% (47 of 89; 44·1–61·3) at the cutoff of 4·0 pg/mL. Test specificity was 91·3% (1796 of 1967 negative samples; 90% CrI 90·2–92·3) at the cutoff of 0·19 pg/mL, 99·2% (1952 of 1967; 98·8–99·5) at the cutoff of 0·67 pg/mL, and 100·0% (1967 of 1967; 99·8–100·0) at the cutoff of 4·00 pg/mL. Using a two-step testing strategy with a CLEIA negativity cutoff of 0·19 pg/mL (to maximise sensitivity) and a CLEIA positivity cutoff of 4·00 pg/mL (to maximise specificity), the proportions of indeterminate results (ie, samples requiring secondary RT-qPCR) would be approximately 11% assuming a prevalence of RT-qPCR positivity of 10%, 16% assuming a prevalence of RT-qPCR positivity of 30%, and 21% assuming a prevalence of RT-qPCR positivity of 50%.

Interpretation

CLEIA testing of self-collected saliva is simple and provides results quickly, and is thus suitable for mass testing. To improve accuracy, we propose a two-step screening strategy with an initial CLEIA test followed by confirmatory RT-qPCR for intermediate concentrations, varying positive and negative thresholds depending on local prevalence. Implementation of this strategy has expedited sample processing at Japanese airports since July, 2020, and might apply to other large-scale mass screening initiatives.

Funding

Ministry of Health, Labour and Welfare, Japan.

Introduction

Rapid detection of SARS-CoV-2 is crucial for the prevention and containment of COVID-19 outbreaks in communities and hospitals. Screening of asymptomatic people is a particularly urgent priority, given that substantial viral shedding occurs before symptom onset.¹ Studies in the past year have shown that infectiousness peaks at or before symptom onset,² and that live virus can be isolated from asymptomatic individuals.³ Approximately half of infections are asymptomatic but transmissible for at least 10 days after initial infection.3, 4 Thus, a comprehensive strategy is needed to increase diagnostic testing capabilities for mass screening of SARS-CoV-2.⁵

Research in context.

Evidence before this study

A PubMed search from database inception up to Feb 17, 2021, for articles published in English, with the search terms “antigen test” AND (“NAAT” OR “PCR”) AND (“COVID” OR “SARS-CoV-2”), provided 41 articles. 31 of these pertained to the evaluation of various tests, mostly rapid non-quantitative antigen tests, compared with PCR, and four were case reports or case series. One randomised controlled trial summary, a simulation of pretravel testing, and a meta-analysis were also identified. Three articles evaluated the performance of the Lumipulse G SARS-CoV-2 Ag assay (Fujirebio, Tokyo, Japan), a quantitative antigen test based on chemiluminescent enzyme immunoassay (CLEIA), but all previous articles used nasopharyngeal fluid and not saliva.

Added value of this study

Our study compared the utility of the Lumipulse CLEIA test with that of quantitative RT-PCR (RT-qPCR) when applied on self-collected saliva, including performance in asymptomatic people who might transmit SARS-CoV-2 within communities. To our knowledge, our data are the first to show high concordance of results between CLEIA and RT-qPCR in the testing of saliva, in the largest prospective cohort to date. In addition, by using two different thresholds (derived from the manufacturer's package insert) to define test positivity and negativity, we were able to maximise both sensitivity and specificity.

Implications of all the available evidence

This study showed similar accuracy between CLEIA and RT-qPCR, and suggested the potential value of the use of these tests in a stepwise manner. Considering the rapid turnaround time and ability to vary positive and negative thresholds, CLEIA is more suited as the initial test, with RT-qPCR reserved for individuals with indeterminate results. Implementation of this two-step strategy has expedited sample processing at Japanese airports since July, 2020, and might be applicable in various large-scale mass screening initiatives.

The current gold standard for viral detection is nucleic acid amplification tests (NAAT), such as quantitative RT-PCR (RT-qPCR) with nasopharyngeal swab (NPS) samples.6, 7 Traditionally, preferred screening tests are simple, inexpensive, and acceptable to those being tested, in addition to having high sensitivity and specificity. However, NPS sampling in the context of RT-qPCR requires specialised medical personnel with protective equipment, posing the risk of viral transmission to health-care workers, and false-negative results might occur due to errors in sampling technique.7, 8 Self-collected saliva testing has substantial logistic advantages over non-self NPS sampling by eliminating these issues. We and others have shown that the accuracy of self-collected saliva and non-self NPS samples in the detection of SARS-CoV-2 by RT-qPCR is equivalent in large-scale direct comparative studies.9, 10, 11, 12

Although RT-qPCR is accurate and reliable, it is time consuming as a screening test. A rapid alternative is antibody-based serological assays, but these cannot be used for early diagnosis of infection.⁶ Viral antigen detection in saliva might be a candidate strategy to achieve earlier diagnosis than RT-qPCR.¹³ However, one study showed that the sensitivity of an immunochromatographic assay to detect viral antigen was only 11·7% when testing the saliva of patients with COVID-19.¹⁴ In June, 2020, a quantitative antigen test that uses a chemiluminescent enzyme immunoassay (CLEIA) was developed, which can detect SARS-CoV-2 proteins in NPS or saliva samples within 35 min, called Lumipulse G SARS-CoV-2 Ag (Fujirebio, Tokyo, Japan).15, 16, 17 In the present study, we prospectively compared the utility of this CLEIA test against RT-qPCR when applied on self-collected saliva, including performance in asymptomatic people, and we propose a two-step strategy for mass screening of SARS-CoV-2.

Methods

Study design and participants

We did a diagnostic accuracy study to develop a mass-screening strategy for salivary detection of SARS-CoV-2 by CLEIA in hospitalised patients with clinically confirmed COVID-19, close contacts identified at community health centres, and international arrivals at two airports. Three separate cohorts were included to form a test dataset in this analysis: an inpatient cohort of consecutive, clinically confirmed patients with COVID-19 admitted to Hokkaido University Hospital (Sapporo, Japan), Sapporo City General Hospital (Sapporo), National Hospital Organization Hokkaido Medical Center (Sapporo), and Otaru Kyokai Hospital (Otaru, Japan); a contact tracing cohort enrolling consecutive people who had been in close contact with clinically confirmed patients with COVID-19, with close contact defined as being within approximately 2 m of an infected person for at least 15 min without a mask, from three community health centres in Hokkaido and Tokyo, Japan; and an airport quarantine cohort enrolling consecutive asymptomatic arrivals tested at quarantine stations in Tokyo International Airport (Tokyo, Japan) and Kansai International Airport (Osaka, Japan). In the inpatient cohort, COVID-19 was confirmed by a positive RT-qPCR test of an NPS sample. Participants in the contact tracing cohort were identified by the health centres and were independent of the inpatient cohort. Results comparing the utility of NPS with saliva samples for NAAT in asymptomatic people in the contact tracing and airport quarantine cohorts have been published in recent months.11, 18 In all cohorts, the participants were requested to provide background information (sex and age, and, in the airport quarantine cohort, last point of departure) and saliva in addition to mandatory NPS sampling (a national requirement for SARS-CoV-2 testing at the time of study) by medical personnel. People who did not provide sufficient saliva volume were excluded from the analysis. Saliva samples were self-collected either immediately before or immediately after NPS sampling, in a sterilised 15 mL polystyrene sputum collection tube (Toyo Kizai, Warabi, Japan) at partitioned booths (with sampling instructions in each booth), and transported at 4°C without transport media. All specimens were analysed within 48 h at a central laboratory (SRL, Tokyo, Japan) to ensure consistency in sample processing. This study was approved by the institutional ethics board of Hokkaido University Hospital (Division of Clinical Research administration number 020-0116) and informed consent was obtained from all individuals orally due to the requirement for rapid processing at airports and health centres.

Procedures

Saliva was diluted 4-fold with phosphate-buffered saline and centrifuged at 2000 g for 5 min to remove cells and debris. RNA was extracted from 200 μL of the supernatant with the QIAsymphony DSP Virus/Pathogen Kit and QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) following the manufacturer's instructions. RT-qPCR tests were done as described previously,¹¹ according to the manual by the Japan National Institute of Infectious Diseases (NIID).19, 20 Briefly, 5 μL of extracted RNA was used as a template. One-step RT-qPCR was done with the THUNDERBIRD Probe One-step qRT-PCR Kit (Toyobo, Osaka, Japan) and 7500 Real-Time RT-PCR System (Thermo Fisher Scientific, Waltham, MA, USA) to obtain the cycle threshold (Ct) values. SARS-CoV-2 nucleocapsid protein (SARS-CoV-2-N) primers for the N2 region (NIID_2019-nCOV_N_F2 and NIID_2019-nCOV_N_R2) and an N2 probe (NIID_2019-nCOV_N_P2) designed by the NIID were used for RT-qPCR (appendix p 2).

The Lumipulse G SARS-CoV-2 Ag assay is a sandwich CLEIA that uses monoclonal antibodies against SARS-CoV-2-N on a Lumipulse G1200 automated machine (Fujirebio), which can assay 120 samples per h. 100 μL of saliva diluted 4-fold with PBS was analysed to measure SARS-CoV-2-N concentration according to the manufacturer's instructions. In this assay, the treatment solution and the specimen were consecutively aspirated with a single-use tip and dispensed into a suspension of magnetic beads coated with the monoclonal antibody. After a 10-min incubation followed by a wash step, alkaline phosphatase-conjugated anti-SARS-CoV-2-N monoclonal antibody was added and the suspension incubated for another 10 min. After a second wash step, 3-(2ʹ-spiroadamantane)-4-methoxy-4-(3ʹʹ-phosphoryloxy)phenyl-1,2-dioxetane disodium salt (also known as AMPPD) substrate solution was added and developed for 5 min. The amounts of SARS-CoV-2-N were determined from the developed chemiluminescence signals on the Lumipulse G1200 automated machine. Calibration of Lumipulse G SARS-CoV-2 Ag was done with recombinant SARS-CoV-2-N expressed from Escherichia coli (0, 50, 1000, and 5000 pg/mL) provided as part of the assay according to manufacturer's instructions. The standardisation of Lumipulse G SARS-CoV-2 Ag is based on highly purified SARS-CoV-2-N established by the research group on development of diagnostic tests for COVID-19 (SRL, Tokyo, Japan) under the support of the Japan Agency for Medical Research and Development.

Statistical analysis

We calculated the proportions of RT-qPCR positive and negative samples at defined CLEIA outputs (≥4·00 pg/mL, 0·67 to <4·00 pg/mL, 0·19 to <0·67 pg/mL, and <0·19 pg/mL). The proportions of RT-qPCR positivity and negativity were calculated in the overall cohort and in each of the three cohorts. Subsequently, we estimated the sensitivity and specificity of CLEIA, using antigen concentrations of 0·19, 0·67, and 4·00 pg/mL as cutoffs (ie, CLEIA results were considered positive if equal or greater than the cutoff and negative if smaller than the cutoff). These cutoff values were taken from the package insert of the reagents and based on in-company validation. We defined the sensitivity of CLEIA as the proportion of samples with CLEIA positivity given RT-qPCR positivity (CLEIA[+]|RT-qPCR[+]) and the specificity of CLEIA as the proportion of samples with CLEIA negativity given RT-qPCR negativity (CLEIA[–]|RT-qPCR[–]). For specimens testing positive by RT-qPCR, the correlation between the Ct value and the CLEIA antigen concentration was shown in a scatter plot with Kendall's coefficient of concordance (W). Sensitivity and specificity were also calculated with several other cutoff values, ranging from 0·02 pg/mL to 100·00 pg/mL. An additional sensitivity analysis was done with use of Ct values ranging from 30 to 37 (which includes Ct=35, the threshold for viral transmission²¹) as conditional RT-qPCR positivity.

As a post-hoc analysis, we also evaluated CLEIA as an initial test in a two-step testing strategy, by selecting separate antigen cutoff values for specimen negativity and positivity that would not require further validation by RT-qPCR. Results smaller than the lower cutoff were considered negative and results equal to or greater than the upper cutoff were considered positive. All results between the two cutoffs were considered indeterminate and in need of RT-qPCR for confirmatory testing; the upper and lower cutoffs were varied and the proportion of participants within the indeterminate category was estimated. The proportion of secondary RT-qPCR tests as a function of the lower cutoff value, with the upper cutoff value fixed at 4·00 pg/mL, was estimated assuming an RT-qPCR positivity of 10%, 30%, and 50%, as follows:

$$[\text{probability}\text{of}\text{RT}-\text{PCR}-\text{positivity}\times (\text{sensitivity}\text{at}\text{the}\text{lower}\text{cutoff}-\text{sensitivity}\text{at}\ge 4.00\text{pg}/\text{mL})]+[\text{probability}\text{of}\text{RT}-\text{PCR}\text{negativity}\times (\text{specificity}\text{at}\ge 4.00\text{pg}/\text{mL}-\text{specificity}\text{at}\text{the}\text{lower}\text{cutoff})]$$

Sample size for the three cohorts was determined separately, reflecting different assumptions around a credible interval (CrI) set to 90%. Sample size in the airport quarantine cohort was calculated as 1818 on the basis of an 80% probability that the 90% CrI for specificity would be greater than 99·0%, with an expected specificity of 99·5%. In the inpatient cohort, with an expected sensitivity of 60·0%, 40 participants were needed to accomplish an 80% probability of the 90% CrI exceeding 40·0%. Sample size in the contact tracing cohort was calculated as 250 on the basis of a probability of RT-qPCR-positivity of 10%, and 25 positive samples being needed to keep the 90% CrI of sensitivity within 30·0% under an expected sensitivity of 70·0%.

All statistical analyses were done with R version 4.0.2. The 90% CrI for the proportion x/n was the range between the 5th and 95th percentile of the beta distribution, beta(x + 1, n – x + 1). According to the study design, this construction of CrIs is based on Bayesian estimation with beta(1,1) as the uninformative prior distribution reflecting the unknown proportion before the study.

Role of the funding source

The funder of the study was involved in selecting the study sites, but had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Results

Of 2600 people screened for study enrolment, 2077 (79·9%) gave consent, and 2056 (79·1%) were included for analysis after exclusion of 21 people with insufficient saliva volume (figure 1 ). Of the final study population, the airport quarantine cohort contributed 1763 (85·7%) participants (enrolled between June 12 and June 23, 2020), the contact tracing cohort 251 (12·2%) participants (enrolled between June 12 and July 7, 2020), and the inpatient cohort 42 (2·0%) participants (enrolled between June 12 and Aug 6, 2020), with a total of 132 (6·4%) symptomatic people and 1924 (93·6%) asymptomatic people. The total population consisted of 1048 (51·0%) male participants, 913 (44·4%) female participants, and 95 (4·6%) people who did not specify their gender (appendix p 3). Participants in the airport quarantine cohort had a median age of 33·5 years (IQR 22·6–47·4), whereas the inpatient cohort was highly represented by older patients (median age 69·8 years [51·6–83·4]). In the contact tracing cohort, median age was similar between symptomatic participants (n=90; 42·2 years [34·8–59·6]) and asymptomatic participants (n=161; 44·9 years [29·8–66·4]). The last point of departure in the airport quarantine cohort was reasonably distributed between North America (713 [40·4%] of 1763 participants), Asia and Oceania (583 [33·1%]), and Europe (467 [26·5%]).

Figure 1.

Participant enrolment in the three study cohorts

Overall, 89 (4·3%) of 2056 participants tested positive for SARS-CoV-2 on RT-qPCR during the study; 38 (90·5%) of 42 participants tested positive in the inpatient cohort, 47 (18·7%) of 251 tested positive in the contact tracing cohort, and four (0·2%) of 1763 tested positive in the airport quarantine cohort. The virus was assumed to have cleared in the four patients who tested negative in the inpatient cohort after they initially tested positive on hospital admission. In the total study population, results of CLEIA and RT-qPCR were concordant in 2020 (98·2%) of 2056 samples. The median antigen concentration measured by CLEIA was 4·29 pg/mL (IQR 0·72–359·02) in RT-qPCR-positive specimens, compared with 0·06 pg/mL (0·01–0·11) in RT-qPCR-negative specimens (figure 2A ). The maximum antigen concentration in RT-qPCR-negative specimens was 2·42 pg/mL. The scatter plot of antigen concentrations against Ct values (figure 2B) indicated high correlation between CLEIA and RT-qPCR, with a Kendall's W of 0·99.

Figure 2.

Comparison of viral load between RT-qPCR and CLEIA in saliva samples

(A) Histogram of CLEIA antigen concentration according to diagnostic outcome of RT-qPCR. Numbers of participants with each antigen concentration range are shown above each column. (B) Antigen concentration measured by CLEIA and Ct values from RT-qPCR were plotted according to symptomatic and asymptomatic status. W indicates Kendall's coefficient of concordance. Data were plotted for RT-qPCR-positive samples (n=89). A histogram of Ct values is also shown. RT-qPCR=quantitative RT-PCR. CLEIA=chemiluminescent enzyme immunoassay. Ct=cycle threshold. *Minimum antigen concentration was displayed as 0·01 pg/mL by the Lumipulse G1200 machine.

Increasing the antigen cutoff value (ie, the value at and above which a CLEIA result was considered positive, and below which a CLEIA result was considered negative) led to reduced test sensitivity but increased test specificity (table ). For the overall population, a cutoff value of 0·19 pg/mL yielded a sensitivity (CLEIA[+]|RT-qPCR[+]) of 85·4% (76 of 89 samples; 90% CrI 78·0–90·3), a cutoff value of 0·67 pg/mL yielded a sensitivity of 76·4% (68 of 89; 68·2–82·8), and a cutoff value of 4·00 pg/mL yielded a sensitivity of 52·8% (47 of 89; 44·1–61·3). Specificity (CLEIA[–]|RT-qPCR[–]) was 91·3% (1796 of 1967 samples; 90% CrI 90·2–92·3) at a cutoff value of 0·19 pg/mL, 99·2% (1952 of 1967; 98·8–99·5) at a cutoff value of 0·67 pg/mL, and 100·0% (1967 of 1967; 99·8–100·0) at a cutoff value of 4·00 pg/mL (table). Sensitivity and specificity at other cutoff values are shown in figure 3 and the appendix (pp 4–5). We generally observed no marked difference in CLEIA sensitivity or specificity between symptomatic and asymptomatic people (table). Although some variation in sensitivity was evident for cutoff values smaller than 0·67 pg/mL, the small numbers of participants in these groups might not accurately reflect meaningful differences. The conditional sensitivity with use of a Ct value of 35 or less for RT-qPCR positivity (CLEIA[+]|Ct≤35), reflecting the threshold for viral transmission,²¹ was 97·3% (73 of 75 samples; 90% CrI 91·9–98·9) at a cutoff value of 0·19 pg/mL, 89·3% (67 of 75; 81·8–93·7) at a cutoff value of 0·67 pg/mL, and 62·7% (47 of 75; 53·1–71·2) at a cutoff value of 4·00 pg/mL. Conditional sensitivity and specificity with use of Ct values between 30 and 37 are shown in the appendix (p 6).

Table.

Proportion of positive and negative RT-qPCR results by chemiluminescent enzyme immunoassay antigen concentrations

	Positive RT-qPCR result	Negative RT-qPCR result
Overall population (n=2056)
≥4·00 pg/mL	47 (52·8%)	0
0·67 to <4·00 pg/mL	21 (23·6%)	15 (0·8%)
0·19 to <0·67 pg/mL	8 (9·0%)	156 (7·9%)
<0·19 pg/mL	13 (14·6%)	1796 (91·3%)
Symptomatic population (n=132)
≥4·00 pg/mL	23 (56·1%)	0
0·67 to <4·00 pg/mL	10 (24·4%)	2 (2·2%)
0·19 to <0·67 pg/mL	5 (12·2%)	10 (11·0%)
<0·19 pg/mL	3 (7·3%)	79 (86·8%)
Asymptomatic population (n=1924)
≥4·00 pg/mL	24 (50·0%)	0
0·67 to <4·00 pg/mL	11 (22·9%)	13 (0·7%)
0·19 to <0·67 pg/mL	3 (6·3%)	146 (7·8%)
<0·19 pg/mL	10 (20·8%)	1717 (91·5%)

Data are n (%). RT-qPCR=quantitative RT-PCR.

Figure 3.

Diagnostic performance against antigen cutoff value

Graphs were plotted by the cutoff values for antigen concentration. The solid line indicates point estimates and the dashed lines indicate 90% credible intervals. The cutoff value indicates the antigen concentration equal to and above which a sample is considered positive and below which a sample is considered negative. CLEIA=chemiluminescent enzyme immunoassay. RT-qPCR=quantitative RT-PCR. (CLEIA[+]|RT-qPCR[+])=CLEIA positivity given RT-qPCR positivity. (CLEIA[–]|RT-qPCR[–])=CLEIA negativity given RT-qPCR negativity.

Based on the high concordance of the two tests, and given the logistic advantages of CLEIA, we assessed a two-step testing strategy post hoc. Antigen concentrations equal to and greater than the upper cutoff value were considered positive, and concentrations less than the lower cutoff value were considered negative, with concentrations between the two thresholds considered indeterminate and in need of secondary testing by RT-qPCR. When the cutoff values were set to 0·19 pg/mL and 4·00 pg/mL, test sensitivity was 85·4% and test specificity was 100·0%. Reducing the range between the cutoff values decreased both sensitivity and specificity with reduced diagnostic performance, albeit with a decrease in the number of secondary RT-qPCR tests needed. The proportion of secondary RT-qPCR tests needed as a function of the lower cutoff value was plotted assuming three different probabilities of RT-qPCR positivity (figure 4 ). Unsurprisingly, by increasing the probability of RT-qPCR positivity, the proportion of secondary RT-qPCR tests needed after CLEIA testing increased across the range of lower cutoff values tested. Applying a lower cutoff of 0·19 pg/mL and an upper cutoff of 4·0 pg/mL, the proportions of results requiring secondary RT-qPCR would be approximately 11% assuming a prevalence of RT-qPCR positivity of 10%, 16% assuming a prevalence of RT-qPCR positivity of 30%, and 21% assuming a prevalence of RT-qPCR positivity of 50%.

Figure 4.

Proportion of secondary RT-qPCR tests needed after initial CLEIA in a two-step strategy

The proportion of secondary RT-qPCR tests needed (representing indeterminate results on CLEIA) against the lower cutoff value (ie, cutoff for CLEIA test negativity) was plotted for different probabilities of RT-qPCR positivity. The upper cutoff value (ie, for CLEIA test positivity) was set at 4·00 pg/mL. RT-qPCR=quantitative RT-PCR. CLEIA=chemiluminescent enzyme immunoassay.

Discussion

Our results showed high correlation between antigen concentrations measured by CLEIA and RNA load measured by RT-qPCR, indicating CLEIA to be a reliable and accurate test. Improved diagnostic accuracy can be attained if the two tests are used in combination. Accordingly, CLEIA has been in use across Japanese airports as part of a two-step strategy with NAAT since July, 2020. Quantitative CLEIA could allow for positive and negative thresholds to be adjusted according to estimated prevalence in a local population, and would provide a pretest estimate on the number of RT-qPCR tests that might be necessary for final diagnosis. We found that a cutoff value of 4·00 pg/mL would give 100·0% specificity (CLEIA[–]|RT-qPCR[–]), but with a lower sensitivity of 85·4% (CLEIA[+]|RT-qPCR[+]). If the diagnosis must be made by CLEIA alone with one cutoff, a low value, such as 0·67 pg/mL, should be used to minimise the compromise in sensitivity while maintaining a specificity greater than 99%, to avoid the isolation of non-infected individuals due to false-positive results. Conversely, if capacity is sufficient to implement RT-qPCR as a confirmatory test after initial testing by CLEIA, a lower threshold, such as 0·19 pg/mL, should be set to increase the sensitivity of CLEIA and its suitability for screening purposes. For example, assuming a situation in which 100 000 people might be tested with an estimated prevalence of 10%, 10 000 people will be positive by RT-qPCR. In this scenario, applying the most accurate cutoffs for positivity (4·00 pg/mL) and negativity (0·19 pg/mL), 5281 people (10 000 × 47/89; table) will test positive in an initial CLEIA test, with another 3258 patients having an indeterminate CLEIA result and testing positive by NAAT. Of the 90 000 people with RT-qPCR negativity, 7824 people will test negative by NAAT after an indeterminate result from CLEIA. By employing a two-step strategy, NAAT would only be needed in 11 082 of 100 000 people (ie, about 11% of all individuals). At an estimated prevalence of 50%, a similar calculation shows that the number of people who would require confirmation by NAAT to be 20 639 (∼21%). Therefore, this two-step testing strategy exploits the advantage of rapid and accurate quantitative antigen testing to save the resources for NAAT to approximately 10–20% of an entire test population.

Previous reports on the Lumipulse G SARS-CoV-2 Ag assay in NPS samples showed a concordance with RT-qPCR of 91·4% in 313 samples¹⁵ and 98·2% in 548 samples.¹⁶ By contrast, Kobayashi and colleagues¹⁷ reported a lower concordance of 66·0% in 100 NPS samples, although samples with discrepant results were collected considerably later in the course of infection than those with concordant results. To our knowledge, our study is the first to evaluate saliva samples with CLEIA. In addition, we obtained specimens from more than 2000 symptomatic and asymptomatic people, and found 98·2% (2020 of 2056 samples) of CLEIA results to be in agreement with results by RT-qPCR. All samples were collected prospectively and consecutively, providing credibility to our results, with the implication that CLEIA might be a viable alternative to RT-qPCR. Furthermore, all tests were done with self-collected saliva, enabling simultaneous parallel sample collection and rapid processing, which are in need at sites of mass screening. Despite use of a central laboratory to ensure quality control in this study, the size of the automated machine (Lumipulse G1200; 1·20 m × 0·80 m × 1·45 m) is small enough to be installed at points of care.

A positive RT-qPCR result does not necessarily indicate presence of live virus,²² and studies in the past year have shown COVID-19 patients with Ct values greater than 33 or 34 on RT-qPCR not to be contagious.23, 24 Conversely, in our study, 15 (0·7%) of 2056 samples had antigen concentrations between 0·67 pg/mL and 4·00 pg/mL but were negative on RT-qPCR. This discrepancy could reflect a false-positive CLEIA, but the possibility of a false-negative RT-qPCR result cannot be ruled out, and the clinical implication of this discrepancy remain to be elucidated.

Large-scale comparative studies in recent months have shown that saliva and NPS samples have equivalent efficacy in the detection of SARS-CoV-2 by RT-qPCR.9, 10, 11, 12 Since saliva testing has substantial logistic advantages over NPS testing, we are confident that a combination of CLEIA and NAAT with self-collected saliva is the best available testing method for mass screening of SARS-CoV-2. Reverse transcriptase loop-mediated isothermal amplification (LAMP)²⁵ has become the second most common NAAT after RT-qPCR with several advantages over RT-qPCR, including rapid turnaround time within 30 min, ease of implementation, and potential utility at the point of care with a simple device.11, 14, 26, 27, 28, 29, 30, 31 In 2020 we reported that LAMP had equivalent efficacy to RT-qPCR in detecting SAR-CoV-2 when testing saliva from asymptomatic people in our contact tracing and airport quarantine cohorts.¹¹ For these reasons, LAMP is currently being used at the international airport quarantine stations in Japan as the confirmatory NAAT test after indeterminate results from CLEIA.

Limitations of this study include the absence of longitudinal clinical follow-up and the low number of positive cases in the airport quarantine cohort. We were unable to confirm whether the participants who tested negative did not subsequently develop COVID-19. However, simple follow-up might be inadequate as a large number of false-negative people might never develop symptoms and yet be infectious (although a low viral load might attenuate transmission²³). Therefore, negative test results might warrant repeated testing in individuals with strongly suspected infection, such as people travelling from countries with high prevalence of COVID-19 or people consistently in close contact with known infected patients. In the present study, we showed that test specificity with CLEIA was 99·8% or greater if the higher cutoff was set to 2·43 pg/mL, as the highest concentration in the 1967 specimens that tested negative on RT-qPCR. Considering the differences between the specificities of CLEIA and RT-qPCR, a higher CLEIA positivity cutoff would be necessary if the greatest priority is minimising false-positives.

In summary, we showed CLEIA to be a reliable alternative to RT-qPCR with high concordance between the results of the two tests in a large population. Furthermore, use of these tests in a stepwise strategy with self-collected saliva is more efficient than use of NPS and RT-qPCR alone in real-life mass screening settings. A two-step testing methodology with CLEIA and LAMP has already been implemented at Japanese airport quarantine stations, facilitating expeditious processing of samples from international travellers, with all tests done at the point of care. We believe that the two-step test strategy with self-collected saliva is presently the most effective method for screening large numbers of people in a short period. Further studies on prospectively validating the two-step approach and data on longitudinal follow-up after point-of-care testing might be warranted.

Data sharing

All distributable data are provided in the appendix (pp 7–9).

Declaration of interests

IY reports a policy research grant from the Ministry of Health, Labour and Welfare, Japan, during the conduct of the study; and personal fees from Chugai Pharmaceutical, AstraZeneca, Japan Tobacco Pharmaceutical Division, and Nippon Shinyaku, outside the submitted work. PYS reports personal fees from AYUMI Pharmaceutical, Japan Pharmaceutical Manufacturers Association, Alexion Pharmaceuticals, and Kyowa Kirin, outside the submitted work. YU reports a policy research grant from the Ministry of Health, Labour and Welfare, Japan, during the conduct of the study. YY reports a policy research grant from the Ministry of Health, Labour and Welfare, Japan, during the conduct of the study. TT reports policy research grant from the Ministry of Health, Labour and Welfare, Japan, during the conduct of the study; personal fees from Merck Sharp & Dohme, Takeda Pharmaceutical, Pfizer Japan, and Bristol Myers Squibb, grants and personal fees from Kyowa Hakko Kirin, grants, personal fees, and non-financial support from Novartis Pharma, grants from Chugai Pharmaceutical, Sanofi, Astellas Pharma, Teijin Pharma, Fuji Pharma, Nippon Shinyaku, the Japan Society for the Promotion of Science (Grants-in-Aid for Scientific Research), and the Center of Innovation Program of the Japan Science and Technology Agency, and non-financial support from Janssen Pharmaceutical, outside the submitted work. All other authors declare no competing interests.