Clinical validation of quantitative SARS-CoV-2 antigen assays to estimate SARS-CoV-2 viral loads in nasopharyngeal swabs

Abstract

Background

Expansion of the testing capacity for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is an important issue to mitigate the pandemic of coronavirus disease-2019 (COVID-19) caused by this virus. Recently, a sensitive quantitative antigen test (SQT), Lumipulse® SARS-CoV-2 Ag, was developed. It is a fully automated chemiluminescent enzyme immunoassay system for SARS-CoV-2.

Methods

In this study, the analytical performance of SQT was examined using clinical specimens from nasopharyngeal swabs using reverse transcription polymerase chain reaction (RT-PCR) as a control.

Results

Receiver operating characteristic analysis of 24 SARS-CoV-2-positive and 524 -negative patients showed an area under the curve of 0.957 ± 0.063. Using a cut-off value of 1.34 pg/ml, the sensitivity was 91.7%, the specificity was 98.5%, and the overall rate of agreement was 98.2%. In the distribution of negative cases, the 99.5 percentile value was 1.03 pg/ml. There was a high correlation between the viral load calculated using the cycle threshold value of RT-PCR and the concentration of antigen. The tendency for the antigen concentration to decrease with time after disease onset correlated with that of the viral load.

Conclusions

Presented results indicate that SQT is highly concordant with RT-PCR and should be useful for the diagnosis of COVID-19 in any clinical setting. Therefore, this fully automated kit will contribute to the expansion of the testing capability for SARS-CoV-2.

1. Introduction

Since December 2019, diseases caused by the novel coronavirus 2019, severe acute respiratory coronavirus 2 (SARS-CoV-2), emerged in China and rapidly spread throughout the world. The World Health Organization stated that the coronavirus disease-2019 (COVID-19) pandemic is not close to ending and is still a serious public health threat [1]. An analysis of viral load kinetics indicated that access to SARS-CoV-2 testing as well as the frequency and rapidity of tests should be prioritized to control the spread of SARS-CoV-2 [2].

Currently, nucleic acid testing (NAT) using reverse transcription polymerase chain reaction (RT-PCR) has been used for the diagnosis of COVID-19. However, its complicated operation, relatively long time to obtain results (several hours to several days), and cost are issues that have prevented the expansion of NAT for the general testing of COVID-19 in several countries. Rapid antigen tests (RAT) are expected to address these issues; however, recent studies have reported their high specificity and poor sensitivity compared with RT-PCR [3,4]. In addition, RAT is a qualitative test and should not be used to estimate viral loads when monitoring the status of virus replication.

Recently, a sensitive quantitative antigen test (SQT), “Lumipulse® SARS-CoV-2 Ag”, was developed based on a fully automated chemiluminescent enzyme immunoassay system that enables the rapid and multiple testing of SARS-CoV-2 N antigens in nasopharyngeal swabs or saliva.

In this study, we investigated the performance of the SQT and compared it with that of RT-PCR using clinical samples.

2. Materials and methods

2.1. Patient samples

Five hundred and forty-eight nasopharyngeal swab specimens were collected from COVID-19 patients or suspected patients with COVID-19 hospitalized in Omori Hospital, who had been diagnosed by RT-PCR. Some samples were collected from the same patients at multiple time points. Informed consent was obtained from all participants in the study. Specimens were stored at −80 °C and did not melt during storage (within 2 months). The study protocol was approved by the Ethics Committee of the Faculty of Medicine, Toho University (No. A20028_A20020_A20014_A19099).

2.2. SARS-CoV-2 RNA detection

Total RNA was extracted from nasopharyngeal swabs using a QIAamp Viral RNA mini kit (Qiagen, Hilden, Germany) or BD MAX™ ExK TNA-3 (Swab) (Becton Dickinson, NJ, USA). Then, one-step RT-qPCR was performed using the QuantStudio® 5 (Applied Biosystems, USA) or BD MAX™ open system. TaqMan Fast Virus 1-Step Master Mix (Thermo Fisher Scientific, MA, USA) and BD MAX™ TNA MMK (SPC) were used as one-step RT-qPCR master mixes. The primer set (NIID_2019-nCOV_N_F2 and NIID_2019-nCOV_N_R2) and FAM labeled probe (NIID_2019-nCOV_N_P2) were used as they were previously reported to target gene N [5]. The detection limit value of the measurement system was set at 5 copies/reaction. The RNA copy number was calculated using the cycle threshold (Ct) value of the AccuPlex SARS-CoV-2 reference (SeraCare, MA, USA) using known RNA copies.

2.3. Measurement of SARS-CoV-2 antigen

SARS-CoV-2 antigen was measured using the Lumipulse® SARS-CoV-2 Ag kit and the Lumipulse® G1200 automated immunoassay analyzer according to the manufacturer's instructions (Fujirebio Inc., Tokyo, Japan) using nasopharyngeal swab specimens frozen after RT-PCR. To remove viscous materials from the samples, specimens were centrifuged at 1300 ×g for 10 min, and the supernatants were used for the analysis. First, 100 μL of the supernatant was added to the treatment solution and consecutively aspirated using a single tip. Then, the mixture was dispensed into an anti-SARS-CoV-2 Ag monoclonal antibody-coated magnetic particle solution and incubated for 10 min at 37 °C. After the first washing step, alkaline phosphatase-conjugated anti-SARS-CoV-2 Ag monoclonal antibody was added and incubated for 10 min at 37 °C. After another washing step, the substrate solution was added and incubated for 5 min at 37 °C. The resulting reaction signals were proportional to the amount of SARS-CoV-2 Ag in the sample, which allowed the quantitative determination of SARS-CoV-2 Ag in the nasopharyngeal swabs. When the antigen levels were greater than the upper range of the measurement (5000 pg/mL), we tested samples diluted with Lumipulse® sample diluent (Fujirebio Inc.).

2.4. LAMP method

A loop-mediated isothermal amplification (LAMP) test, Loopamp 2019-nCoV detection reagent (Eiken Chemical Co., Ltd., Tokyo, Japan), was performed according to the manufacturer's protocol.

2.5. Statistical analysis

Statistical analysis was performed with R (The R Foundation for Statistical Computing, Vienna, Austria) and EZR (Saitama Medical Center, Jichi Medical University, Saitama, Japan), which is a graphical user interface for R and Excel (Microsoft Corporation, Redmond, WA, USA). Receiver operating characteristic (ROC) curve analysis was conducted using R to evaluate the assay performance and to visualize the curves, and areas under the ROC curves (AUC), sensitivity, specificity and the overall percent agreement were calculated. Because the negative group distribution did not follow a normal distribution or have a lognormal distribution, a nonparametric method (99.5 percentile) was used to establish a negative reference value.

3. Results

The ROC analysis of SQT by RT-PCR showed an AUC of 0.957 ± 0.063 and a cut-off value of 1.34 pg/mL using Youden's Index (Fig. 1 A). The distribution of antigen levels in 510 samples collected from persons without a history of COVID-19, whom had been diagnosed by RT-PCR, is shown in Fig. 1B. The 99.5 percentile determined by a nonparametric method was 1.03 pg/mL, and was sufficiently below the cut-off value. Using this cut-off value, the SQT had a sensitivity of 91.7%, a specificity of 98.5%, and overall rates of agreement of 98.2%.

Fig. 1.

ROC curve and negative distribution of measuremented SARS-CoV-2 Ag value by Lumipulse®. (A) Receiver operating characteristic (ROC) curve for sensitive quantitative antigen test (SQT). ROC was performed by SQT (n = 548; n = 30 SQT-positive patients and n = 518 SQT-negative patients). The area under the curve (AUC) was 0.957 ± 0.063. (B) Distribution of SARS-CoV-2 antigen concentration in Lumipulse®-negative cases. This figure shows SQT-negative (−) cases without a history of COVID-19 (+) (n = 510). The 99.5 percentile was 1.03 pg/mL.

There were eight RT-PCR-negative and SQT-positive cases (1.5%) (Table 1 , Fig. 2 B). In all these cases, specimens were collected several times and were from patients with COVID-19 history. We examined the dilution-linearity of five available samples to eliminate the possibility of non-specific interference and variation in the measurements [6]. In the 5-fold dilution measurement, the antigen concentration was decreased approximately in proportion to the degree of dilution in each case (Table 2 ). The corresponding values were obtained when the samples were diluted 5-fold for cases of discrepancy, suggesting that the SQT measures the antigen level accurately.

Table 1.

2 by 2 table: Specificity/sensitivity of SQT compared with RT-PCR. A cut-off value for SQT of 1.34 pg/mL was used.

Fig. 2.

Analysis of viral load and antigen concentration. (A) Correlation analysis between viral load and SARS-CoV-2 antigen concentration. This figure shows cases with a positive test for either RT-PCR or SQT. Open circles indicate RT-PCR-positive and SQT-positive cases (n = 22). Cross marks indicate RT-PCR-negative and SQT-positive cases (n = 8). Triangles indicate RT-PCR-positive and SQT-negative cases (n = 2). (B) Relationship between days after onset and viral load or SARS-CoV-2 antigen concentration. The viral load in RT-PCR-positive cases and the antigen concentrations in SQT-positive cases are shown. Black circles indicate viral load (n = 24) and gray squares indicate SARS-CoV-2 antigen concentrations (n = 30). Black arrowheads indicate PCR-negative and SQT-positive 8 cases, and gray indicates PCR-positive and SQT-negative 2 cases. The dotted line represents the approximate line of viral load, and the dashed-and-dotted line represents the approximate line of SARS-CoV-2 antigen.

Table 2.

Antigen concentration of RT-PCR-negative and SQT-positive cases. Each case from A to E shows a sample from five RT-PCR-negative (−) patients who were SQT-positive (+). A value of 1 indicates no dilution and a value of 1/5 indicates a 5-fold dilution.

RT-PCR (-) SQT (+)	Antigen concentration (pg/mL)
Case	Undiluted	Diluted (1/5)
A	8.85	1.57
B	53.74	13.10
C	100.22	16.44
D	15.25	3.11
E	17.85	5.47

However, there were two cases with PCR-positive and SQT-negative findings (Table 1, Fig. 2B). These two specimens were collected at 2 and 10 days after the onset of symptoms (initial sample collection); the LAMP method for RNA detection gave a negative result for one of these specimens (data not shown) [7].

The log of antigen concentrations correlated well with the log of RNA copies calculated from the Ct value of the RT-PCR analysis (Fig. 2A). The regression line obtained by the Passing-Bablok method indicated that the cut-off value of the SQT test corresponded to RNA titers under 10 copies/reaction.

Changes in the viral load and antigen concentration from COVID-19 onset were analyzed in RT-PCR-positive and SQT-positive cases, respectively (Fig. 2B). Both viral load and antigen concentration tended to decrease with disease duration from symptom onset. The measurement of the antigen concentration should be used to monitor the viral loads of COVID-19 patients. It was reported that almost 70% of samples with a Ct value of 25 or less contained infectious viruses, whereas no samples with a Ct value of 35 or more contained infectious viruses in COVID-19 patients [8]. An Indian epidemiologic study showed that 71% of COVID-19 patients identified by contact tracking had no secondary infection [9]. Thus, the early identification of patients with high viral shedding seems to be important for infection control of SARS-CoV-2. However, viral loads expressed as Ct values is an issue because these values differ between testing kits and between facilities. Monitoring antigen levels can provide a more rapid and stable quantitative basis for infection control, treatment progress, and discharge decisions. It might also be possible to quantitatively determine the likelihood of relapse.

4. Discussion

The results of this study showed that SQT had a high concordance rate with the RT-PCR method, and even samples with a low viral load showed positive results with SQT. In addition, the antigen concentration correlated well with the RNA copy number. These data suggest that SQT might replace RT-PCR for the diagnosis of COVID-19.

Using SQT, the antigen concentration can be obtained in approximately 30 min using a fully-automated analyzer with a throughput rate of 60 or 120 samples per hour, which will contribute to the expansion of the testing capability for SARS-CoV-2. The cost of SQT is lower than that of NAT, and the insurance points of SQT (600) is less than half of that of NAT (1350 or 1800) in Japan. Because the SQT can provide test results in a short period of time (30 min), it is particularly useful in facilities where it is difficult for many people to stay for a long time, such as airports.

Because false-positive results are likely to occur when sputum is used for antigen tests [10], especially if the viscosity of the sample is high or impurities are present, centrifugation might be an effective method to avoid false-positive results. Several discrepancies were found in this study. Although limitations in the volume and/or concentration of samples hindered further analysis by Western blotting or different NAT tests, the dilution-linearity test suggested a specific immunoreaction for measurements. Further analysis of discrepant specimens will help the manufacturer improve the SQT.

Correlation analysis between the antigen test and RT-PCR showed low variability in the correlation between antigen concentration and RNA titer in samples with high viral loads, and scattered correlation patterns were observed in samples with low viral loads, which were collected from patients at later periods after symptom onset. Virological factors that might affect these observations are not clear; however, we think that the accuracy and precision of the measurements might affect the correlation patterns because the SQT showed high linearity of dilution with a coefficient of variation less than 10% (Table 2) [11]. In addition, differences in the analyte stability in samples might affect the scattered correlation pattern.

Generally, proteins are more resistant to degradation than RNAs [[12], [13], [14], [15]]. In some cases, intracellular viruses do not replicate as RNA but only produce antigenic proteins [16,17]. Differences in stability or viral production of the antigen protein might account for the scattered correlation in samples with low viral loads and the discrepancies observed in this study.

Funding

This research was supported by the Japan Agency for Medical Research and Development under Grant Number JP19fk0108113.

ICMJE Statement

Contributors YI was responsible for the organization and coordination of the study. KA, TN, SO and SYa were responsible for the data analysis. YI, KA, SYo, KK and KT prepared this study design. TMa, TMi, and SYo contributed to take informed consent and collection of clinical specimens. All authors contributed to the writing of the manuscript.

Declaration of competing interest

SYa and SO are employees of Fujirebio, Inc. The other authors have no conflict of interest to declare.