Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients

Abstract

In routine clinical practice, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is determined by reverse-transcription PCR (RT-PCR). In the current pandemic, a more rapid and high-throughput method is in growing demand. Here, we validated the performance of a new antigen test (LUMIPULSE) based on chemiluminescence enzyme immunoassay. A total of 313 nasopharyngeal swabs (82 serial samples from 7 infected patients and 231 individual samples from 4 infected patients and 215 uninfected individuals) were analyzed for SARS-CoV-2 with quantitative RT-PCR (RT-qPCR) and then subjected to LUMIPULSE. We determined the cutoff value for antigen detection using receiver operating characteristic curve analysis and compared the performance of the antigen test with that of RT-qPCR. We also compared the viral loads and antigen levels in serial samples from seven infected patients. Using RT-qPCR as the reference, the antigen test exhibited 55.2% sensitivity and 99.6% specificity, with a 91.4% overall agreement rate (286/313). In specimens with > 100 viral copies and between 10 and 100 copies, the antigen test showed 100% and 85% concordance with RT-qPCR, respectively. This concordance declined with lower viral loads. In the serially followed patients, the antigen levels showed a steady decline, along with viral clearance. This gradual decline was in contrast with the abrupt positive-to-negative and negative-to-positive status changes observed with RT-qPCR, particularly in the late phase of infection. In summary, the LUMIPULSE antigen test can rapidly identify SARS-CoV-2-infected individuals with moderate to high viral loads and may be helpful for monitoring viral clearance in hospitalized patients.

Introduction

A novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), emerged in December 2019 in Wuhan, China (Zhu et al., 2020). Within a few months, SARS-CoV-2 had spread around the world, threatening human life (Organization WH, 2020). To date, 11 million have been infected with SARS-CoV-2, and 0.52 million have died from coronavirus disease 2019 (COVID-19) (Organization WH, 2020). As Japan continues to battle a second wave of COVID-19 epidemic, more than 1,000 newly infected patients are being confirmed daily. As of August 3, 2020, 38,687 individuals have been infected with SARS-CoV-2 in Japan, of whom 1,012 have passed away.

The World Health Organization (WHO) has raised a global warning and announced the need for a test system for COVID-19-suspected patients (World Health Organization, 2020). SARS-CoV-2 is known to be spread even by infected people who experience only mild symptoms or are asymptomatic carriers (Li et al., 2020, Bai et al., 2020, Rothe et al., 2020). Therefore, there is a need to expand testing to asymptomatic individuals in certain regions. Furthermore, there have been concerns that environmental contamination can result in the further spread of the virus, particularly in hospitals (Ong et al., 2020, Hirotsu et al., 2020a).

In routine clinical practice, SARS-CoV-2 infection is determined by reverse-transcription PCR (RT-PCR) analysis (Corman et al., 2020). The RT-PCR test is conducted using different types of specimens, including sputum, nasopharyngeal swabs, pharyngeal swabs, saliva, stool, bronchoalveolar lavage fluid, and endotracheal aspirate fluid (KK-W et al., 2020, To et al., 2020, Wang et al., 2020). As we and another group have previously reported, using a pooling strategy with RT-PCR is one of the most effective methods for screening individuals, saving time, reagents, and cost (Hogan et al., 2020, Hirotsu et al., 2020b).

However, the RT-PCR test is not rapid (it typically takes 3–4 h for results), and it requires specialized laboratory equipment and skilled technicians, while antigen tests are simple and can be performed routinely in clinical laboratories (Lai et al., 2020, Clerc and Greub, 2010). Antigen tests have been widely used to detect infection with viruses other than SARS-CoV-2 (Clerc and Greub, 2010). The development of a more cost-effective and high-throughput test system will be important for preventing viral spread and monitoring infection levels in COVID-19 patients.

Here, we present a newly developed SARS-CoV-2 antigen test system, named LUMIPULSE SARS-CoV-2 Ag kit, based on chemiluminescence enzyme immunoassay (CLEIA). We compared quantitative RT-PCR (RT-qPCR) results for viral load with the LUMIPULSE results for antigen level following the testing of 313 nasopharyngeal swabs. We also examined antigen levels in a series of samples collected from hospitalized patients with COVID-19 infections.

Materials and methods

Patients and samples

We collected 313 nasopharyngeal swabs from individuals at Yamanashi Central Hospital. All samples were obtained using cotton swabs and viral transport media in UTM® (Copan Diagnostics, Murrieta, CA, USA). The viral transport media were stored at 4 °C until nucleic acid extraction. Total nucleic acids were extracted within 2 h after swab collecting.

We deemed individuals to be non-infected for negative results obtained by RT-qPCR. Our cohort provided two groups of samples: the first group was 82 serial samples from 7 infected patients, and the second as 231 individual samples from 4 infected patients and 215 non-infected individuals, all of which were analyzed with RT-qPCR. Of the 313 nasopharyngeal swabs, 58 tested positive, and 255 tested negative for SARS-CoV-2 based on RT-qPCR.

The Institutional Review Board of the Clinical Research and Genome Research Committee at Yamanashi Central Hospital approved this study and the use of an opt-out consent method (Approval No. C2019-30 and C2020-9). The requirement for written informed consent was waived. Patient participation in the study was optional. All methods were performed in accordance with the relevant guidelines and regulations.

Viral nucleic acid extraction

Total nucleic acid was automatically isolated from nasopharyngeal swabs using the MagMAX Viral/Pathogen Nucleic Acid Isolation Kit (Thermo Fisher Scientific, Waltham, MA, USA) on the KingFisher Duo Prime System (Thermo Fisher Scientific), as previously described (Hirotsu et al., 2020b, Hirotsu et al., 2020c). Briefly, we added 200 μL viral transport media, 5 μL proteinase K, 265 μL binding solution, 10 μL total nucleic acid-binding beads, 0.5 mL wash buffer, and 0.5–1 mL 80% ethanol to each well of a deep-well 96-well plate. Nucleic acids were eluted with 70 μL elution solution. Total nucleic acids were immediately subjected to an RT-qPCR test, and residual samples were stored at −80 °C.

One-step RT-qPCR

According to the protocol developed by the National Institute of Infectious Diseases (NIID) in Japan (Shirato et al., 2020), we performed one-step RT-qPCR to detect SARS-CoV-2 (Hirotsu et al., 2020d). The primer/probe set (N2) amplified the nucleocapsid (N) gene of SARS-CoV-2 (NC_045512.2). The reaction mixture was 5 μL 4× TaqMan Fast Virus 1-Step Master Mix (Thermo Fisher Scientific), 1.0 μL 10 μM forward primer (5′-AAATTTTGGGGACCAGGAAC-3′), 1.4 μL 10 μM reverse primer (5′-TGGCAGCTGTGTAGGTCAAC-3′), 0.8 μL 5 μM probe (5′-FAM-ATGTCGCGCATTGGCATGGA-TAMRA-3′), 6.8 μL nuclease-free water, and 5 μL nucleic acid sample in a 20 μL total volume. The expected amplicon size was 158 bp. For the internal positive control, the human ribonuclease P 30 subunit (RPP30) gene was used (Integrated DNA Technologies, Coralville, IA, USA) (Hirotsu et al., 2020a, Hirotsu et al., 2020d).

The RT-qPCR assays were conducted in a StepOnePlus Real-Time PCR System (Thermo Fisher Scientific) with the following cycling conditions: 50 °C for 5 min for reverse transcription, 95 °C for 20 s, and 45 cycles of 95 °C for 3 s and 60 °C for 30 s. The threshold was set to 0.2.

A threshold cycle (Ct) value was assigned to each PCR reaction, and the amplification curve was visually assessed. Following the national protocol (version 2.9.1), we deemed a sample to be positive when a visible amplification plot was observed, but negative when no amplification was observed. The absolute copy number of the viral load was determined using the Ct value of the AccuPlex SARS-CoV-2 reference (SeraCare, Milford, MA, USA).

SARS-CoV-2 antigen (Ag) test

The remaining viral transport media from each nasopharyngeal swab was frozen after RT-qPCR. These samples were sent to an outside laboratory (Fujirebio, Inc., Tokyo, Japan). Once thawed, the viral transport medium was viscous; hence, samples were centrifuged at 1,300 ×g for 10 min, and the supernatants were used for subsequent analysis.

We used 100 μL supernatant per sample of thawed viral transport media from each nasopharyngeal swab to measure the antigen level with the LUMIPULSE SARS-CoV-2 Ag kit (Fujirebio) on the LUMIPULSE G600II automated immunoassay analyzer (Fujirebio), following the CLEIA method.

In this assay, the treatment solution and the sample were consecutively aspirated using a single tip. The mixture was dispensed into the anti-SARS-CoV-2 Ag monoclonal antibody-coated magnetic particle solution and then incubated for 10 min at 37 °C. After the first wash step, alkaline phosphatase-conjugated anti-SARS-CoV-2 Ag monoclonal antibody was added and incubated for 10 min at 37 °C. Following another wash step, the substrate solution was added and incubated for 5 min at 37 °C. The resulting reaction signals were proportional to the amount of SARS-CoV-2 Ag in the sample, allowing a quantitative determination for SARS-CoV-2 Ag in the nasopharyngeal swabs.

Where the antigen level could not be measured because it exceeded the detection limit, we tested diluted samples and calculated the antigen levels of the original sample, based on the dilution factor.

Statistical analyses

Statistical analyses were performed in R (https://www.r-project.org/) and Excel (Microsoft Corp., Redmond, WA, USA). Receiver operating characteristic (ROC) curve analyses were conducted using Analyse-it (Analyse-it Software, Ltd., Leeds, UK) to evaluate the assay performance and visualize the curves. The areas under the ROC curves (AUCs), sensitivity, and specificity were calculated.

Results

Comparison between RT-qPCR and the antigen test

We performed the antigen test on 313 nasopharyngeal swabs, including 58 positive samples from 11 infected patients and 255 negative samples from 215 non-infected individuals, as determined by RT-qPCR. These samples were subjected to the antigen test in a blinded manner.

The median antigen level of the PCR-positive samples was 1.57 pg/mL (range 0.12–194,795 pg/mL) and that of the PCR-negative samples was 0.27 pg/mL (range 0–2.46 pg/mL) (Fig. 1 A). The mean antigen level of the PCR-positive samples was significantly higher than that of the PCR-negative samples (p = 0.02, Student’s t-test, Fig. 1A).

Fig. 1.

Comparison of the results between the SARS-CoV-2 antigen (Ag) test and RT-qPCR.

(A) A total of 313 nasopharyngeal samples were subjected to RT-qPCR (58 positive and 255 negative). The same samples were also subjected to the Ag test. The beeswarm plot shows the SARS-CoV-2 Ag levels in the RT-qPCR-positive (light blue) and -negative samples (red).

(B) Receiver operating characteristic (ROC) curve analysis. The Ag test achieved an area under the ROC curve (AUC) value of 0.848 ± 0.044.

(C) Comparison of data obtained with the Ag test and RT-qPCR. An overall agreement of 91.4% was achieved between the two tests, with 55.2% sensitivity and 99.6% specificity obtained with the Ag test.

To determine the cutoff antigen level for distinguishing the SARS-CoV-2 infection status, we conducted ROC curve analyses. When the cutoff for the antigen level was set to 1.31 pg/mL, the accuracy reached its highest level. ROC analyses yielded an AUC value of 0.848 ± 0.044, suggesting that the antigen test accurately detected SARS-CoV-2 (Fig. 1B).

There were 32, 1, 254, and 26 true-positive, false-positive, true-negative, and false-negative results, respectively (Fig. 1C). When the RT-qPCR results were used as a reference, the antigen test diagnosed SARS-CoV-2 infection status with a sensitivity of 55.2% and a specificity of 99.6%. The overall concordance between RT-qPCR and the antigen test was 91.4% (286/313).

Correlation of viral loads with antigen levels in nasopharyngeal swabs

The primer/probe set used in RT-qPCR amplified the nucleocapsid gene of SARS-Co-V-2 (Hirotsu et al., 2020d). The antigen test also detected a portion of the nucleocapsid protein. We next examined the relationship between the SARS-CoV-2 viral loads (as determined by RT-qPCR) and the antigen levels (Fig. 2 ). The SARS-CoV-2 viral load was positively correlated with the antigen level (R² = 0.768).

Fig. 2.

Correlation between antigen levels and viral loads.

A positive correlation (R² = 0.768) was observed between the SARS-CoV-2 antigen (Ag) level (log₁₀ pg/mL) and the viral titer (log₁₀ copies/test).

Relationship between viral load and antigen test performance

To determine the limit of detection of the antigen test, we investigated the relationship between the number of viral copies in the samples and the positive results of the antigen test.

The antigen test determined samples to be positive with 100% concordance with RT-qPCR when the viral load in the samples was > 100 copies (17/17 samples) and with 85% concordance when the viral load was > 10 copies but < 100 copies (23/27 samples) (Table 1 ). The concordance rate gradually declined with decreasing viral load (60% concordance for samples with 10–100 copies, 33% for samples with 1–10 copies, and 26% for samples with less than 1 copy; Table 1).

Table 1.

Relationship between viral roads and positive results in an antigen test

SARS-CoV-2 viral load (log10 copies/test)	Total number of samples	Positive with antigen test	Concordant rate	Cumulative rate
>2	17	17	100%	100%
1–2	10	6	60%	85%
0–1	12	4	33%	69%
<0	19	5	26%	55%

Therefore, the antigen test was highly accurate when the viral load was > 100 copies, whereas lower viral loads (< 100 copies) resulted in some samples being missed (i.e., false-negative results).

Kinetics of viral loads based on antigen levels in serial samples from seven patients

We performed the antigen test and RT-qPCR on a series of nasopharyngeal swabs from seven infected patients. A total of 82 samples were collected from these patients (range 5–27 samples per patient). Overall, there was a strong correlation between the RT-qPCR and antigen test results (Fig. 3 ). During the clinical course of these seven patients, the antigen levels showed a similar declining trend along with viral load, as quantitated by RT-qPCR (Fig. 3).

Fig. 3.

Dynamic changes in antigen levels and viral loads in hospitalized patients.

A series of 82 nasopharyngeal swabs was collected from seven hospitalized patients. The data shown are from three representative hospitalized patients. Longitudinal antigen levels (pink) and viral loads (light blue) were measured in these samples after the onset of symptoms. The labels on the right of the diagram indicate the SARS-CoV-2 viral loads (log₁₀ copies/test) and the labels on the left show the SARS-CoV-2 Ag levels (log₁₀ pg/mL). The dashed light-blue line indicates the cutoff level below which SARS-CoV-2 was not detectable (ND) by RT-qPCR. The dashed pink line indicates the cutoff level for the antigen test.

Of particular interest, there were abrupt positive-to-negative turns and negative-to-positive turns observed with RT-qPCR, especially as the viral load decreased in the latter phase of the infection (cases #2 and #3, Fig. 3), whereas the antigen test rarely showed these abrupt turns.

Discussion

In this study, we validated the assay performance of an antigen test based on CLEIA (LUMIPULSE) and compared the results with those of RT-qPCR. To the best of our knowledge, this is the first report on the clinical validation of the LUMIPULSE SARS-CoV-2 Ag kit. This antigen test is commercially supplied by Fujirebio, Inc. (Tokyo, Japan) and it was recently approved as an in vitro diagnostic test for COVID-19 in Japan on June 19, 2020. Compared to the RT-qPCR test, this antigen test can process 60–120 samples in 30 min per run on an automated machine, which greatly shortens the turnaround time. This test could, therefore, be used as a routine high-throughput test in a hospital setting, especially during a pandemic.

There are some limitations of the antigen test. It has low sensitivity compared with RT-qPCR, which can detect a lower SARS-CoV-2 titer by means of the PCR amplification process. However, the antigen test accurately detected SARS-CoV-2 in all samples with > 100 copies/test. Where samples had a viral load of < 100 copies as quantitated by RT-qPCR, the sensitivity of the antigen test decreased. Second, the presence of the SARS-CoV-2 antigen does not necessarily mean the presence of viable virus. We should carefully consider whether SARS-CoV-2 antigen-positive patients are infectious to other persons.

The viral load tended to be higher at the onset of infection, which is when human-to-human transmission is at its highest (Wolfel et al., 2020, He et al., 2020). Epidemiologically, a key issue is finding asymptomatic and presymptomatic super spreaders to prevent community and nosocomial infection (Savvides and Siegel, 2020). Super spreaders are more likely to be high viral load carriers. Along with their higher viral loads, super spreaders have contact with others, be at close distance with them, and talk loudly without wearing a mask. The clusters of COVID-19 are reported in closed environments, which could contribute to secondary transmission and promote super-spreading events (Nishiura et al., 2020). In this context, our results showed that the antigen test could be used to identify COVID-19-infected individuals who pose a high risk of transmission.

According to the guidelines of the Japanese government, saliva and nasopharyngeal swabs can be used for testing with the LUMIPULSE SARS-CoV-2 Ag kit. Notably, the self-collection of saliva may decrease the infection risk of healthcare workers (To et al., 2020). The Japanese government recommends that the antigen test and nucleic acid amplification test of saliva be applied to symptomatic patients within 9 days of onset when viral loads are high.

In the USA, the Sofia SARS Antigen Fluorescent Immunoassay (FIA) (Quidel, San Diego, CA, USA), a point-of-care assay, has obtained emergency use authorization from the Food and Drug Administration. This FIA uses immunofluorescence-based lateral flow technology for qualitative detection. By comparison, LUMIPULSE SARS-CoV-2 Ag measures the antigen level quantitatively and has the capacity for high-throughput analysis on an automated machine.

In contrast to the rapid immunochromatographic test, the LUMIPULSE SARS-CoV-2 Ag kit quantitatively measures the antigen levels present in samples. Notably, our results for seven patients who were followed from time of admission at hospital to discharge suggested that the SARS-CoV-2 antigen levels declined in these consecutively collected samples. This implied that antigen levels could be used to distinguish between the early and late phases of the COVID-19 clinical course. The stable trend in the serial antigen test results contrasted with the abrupt changes observed when using RT-qPCR, which often showed a mixture of negative and positive results for the same sample. This may confuse clinicians wishing to investigate treatment effects or the timing of discharge, for example. Thus, the LUMIPULSE antigen test may offer a wide range of clinical applications, including judgments of infectivity and determinations of the current infection phase.

In summary, both RT-qPCR and LUMIPULSE antigen test quantitatively measure virus RNA and antigen level, respectively. Therefore, we could investigate and monitor the clinical condition of COVID-19 patients using these tests. Furthermore, both tests are expected to identify asymptomatic or presymptomatic SARS-CoV-2 infected persons who are likely to have high viral loads. Combination assay will help estimate the infection phase of COVID-19 patients in routine clinical practice.

Declaration of Competing interests

YH reports receiving grant support from Fujirebio; SY and SK, being employed by Fujirebio. No other potential conflict of interest relevant to this article was reported.

Author contributions

Yosuke Hirotsu contributed to the study design, data collection, data analysis and writing, review, and editing. Makoto Maejima, Masahiro Shibusawa, Yuki Nagakubo, Kazuhiro Hosaka, Kenji Amemiya, and Hitomi Sueki contributed to the sample preparation and data collection and analysis. Miyoko Hayakawa, Hitoshi Mochizuki contributed to the supervision. Toshiharu Tsutsui, Yumiko Kakizaki, Yoshihiro Miyashita contributed to the resource provision. Shintaro Yagi, Satoshi Kojima contributed to the data collection, data analysis, and writing, review, and editing. Masao Omata contributed to the supervision and writing, review, and editing