Ultra-sensitive Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Antigen Detection for the Diagnosis of Coronavirus Disease 2019 (COVID-19) in Upper Respiratory Samples

Abstract

An ultra-sensitive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleocapsid antigen assay (S-PLEX, MesoScale Diagnostics) was evaluated in 250 retrospective and 200 prospective upper respiratory specimens. In samples with cycle threshold <35, there was 95%–98% positive and 93%–96% negative percent agreement with reverse transcription-polymerase chain reaction. S-PLEX may provide a high-throughput alternative to nucleic acid-based testing for coronavirus disease 2019 (COVID-19) diagnosis.

The coronavirus disease 2019 (COVID-19) pandemic has placed extreme pressure on laboratory testing infrastructure and the supply chain for critical reagents and consumables necessary for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acid amplification testing [1, 2]. Alternatives to viral RNA detection, such as antigen testing have been introduced, but have been limited by suboptimal sensitivity and specificity [3–6]. In this study, we evaluated the analytical and clinical performance of an ultra-sensitive electrochemiluminescence immunoassay to detect SARS-CoV-2 nucleocapsid antigen (SARS-CoV-2 S-PLEX, MesoScale Diagnostics [MSD], Rockville, MD) in 450 upper respiratory samples.

METHODS

SARS-CoV-2 nucleocapsid antigen was quantified using an ultra-sensitive antigen capture immunoassay platform, S-PLEX Direct Detection Assay, S-PLEX SARS-CoV-2 N Kit (catalog #K150ADHS, MSD), performed according to manufacturer instructions (see Supplemental Methods). Clinical test performance was assessed in upper respiratory specimens received as part of routine clinical care at Stanford Clinical Virology Laboratory, which serves tertiary-care academic hospitals and affiliated outpatient facilities in the San Francisco Bay Area of California.

To assess preliminary S-PLEX assay performance characteristics, a retrospective cohort of 250 upper respiratory swabs (70 positives, 180 negatives) previously assayed via Panther Fusion real-time reverse transcription polymerase chain reaction (RT-PCR) for clinical care from March 1 to November 10, 2020, was selected [7]. These samples included 226 nasopharyngeal swabs, 23 mid-turbinate swabs, and 1 oropharyngeal swab. They were collected in Viral Transport Medium or phosphate-buffered saline, and were stored frozen at –80°C until selection for S-PLEX antigen testing. The positivity threshold was based on the assay limit of detection for the retrospective cohort, and on the Mesoscale 99th percentile of RT-PCR–negative retrospective samples for the prospective cohort.

To assess real-world performance, a prospective cohort of 100 upper respiratory swabs consecutively positive for SARS-CoV-2 by Panther Fusion RT-PCR from November 13 to November 15, 2020, were tested by S-PLEX, alongside 100 upper respiratory swabs without detectable RNA randomly selected from the same period. The samples included 151 nasopharyngeal swabs and 49 mid-turbinate swabs that were stored fresh at 4°C for <7 days before S-PLEX testing. The 200 samples were randomly distributed over 3 plates, with the operator blinded to RT-PCR results.

Positive percent agreement (PPA) and negative percent agreement (NPA) were calculated using Panther Fusion RT-PCR as comparator. These values are analogous to sensitivity and specificity assuming that presence of disease is defined by detection of SARS-CoV-2 RNA. Additional details regarding clinical data collection and statistical analysis are included in the Supplemental Methods.

RESULTS

Analytical validation established limit of blank at 2.37 log₁₀ fg/mL (234 fg/mL), limit of detection at 2.45 log₁₀ fg/mL (282 fg/mL), and linear range from 2.65 to 6.24 log₁₀ fg/mL (450 to 1.7 × 10⁶ fg/mL) in our laboratory (Supplemental Tables 1–2, Supplemental Figure 1). Analytical specificity was assessed using 56 respiratory specimens negative for SARS-CoV-2 by RT-PCR and positive for the following respiratory viruses: 42 seasonal coronavirus, 2 adenovirus, 2 influenza A, 2 influenza B, 2 human metapneumovirus, 2 parainfluenza, 2 rhinovirus, and 2 respiratory syncytial virus. One of 56, a seasonal coronavirus sample, was antigen-positive by S-PLEX with a concentration of 2.93 log₁₀ fg/mL (852 fg/mL). Specimens used in analytical validation were not used for subsequent evaluation of clinical performance.

Of the 450 clinical specimens evaluated for S-PLEX clinical performance, 88.4% belonged to adults and 50.0% belonged to females. For the retrospective cohort, median cycle threshold (C_t) across all RT-PCR–positive specimens was 33.2 (interquartile range [IQR], 26.0–36.9). PPA was 72.9% (95% confidence interval [95% CI]: 60.9–82.8). NPA was 96.1% (95% CI: 92.2–98.4). Median C_t of the 19 false-negative samples was 37.9 (IQR 36.9–38.2), with minimum of 32.3. Given that onward transmission potential of individuals with late RT-PCR C_t values remains unclear [8–10], a separate analysis was performed after exclusion of the 27 RT-PCR–positive specimens with C_t >35. PPA was 95.4% (95% CI: 84.2–99.4), and NPA was 96.1% (95% CI: 92.2–98.4). Median C_t among the remaining 43 positive specimens was 27.0 (IQR 23.4–32.3). Area under the curve was 0.940 in all specimens, and 0.997 when excluding samples with C_t >35 on receiver operating characteristic analysis (Supplemental Figure 2, Supplemental Table 3). Focused chart review of the 7 individuals with antigen-positive RT-PCR–negative samples revealed that none met COVID-19 clinical or epidemiologic linkage criteria [11]. These samples had a median antigen concentration of 2.57 (IQR 2.55–2.68) log10 fg/mL or 374 (IQR 358–483) fg/mL.

For the prospective cohort, positivity threshold was set at 2.53 log₁₀ fg/mL (339 fg/mL). Median C_t of the 100 prospectively collected positive samples was 23.3 (IQR 19.2–32.8). PPA was 88.0% (95% CI: 80.0–93.6) and NPA was 93.0% (95% CI: 86.1–97.1) for all samples. When excluding 18 samples with C_t >35, PPA was 97.6% (95% CI: 91.5–99.7) and NPA was 93.0% (95% CI: 86.1–97.1). Median C_t among the 82 remaining positive samples was 21.9 (IQR 18.4–28.3).

Focused chart review was performed to investigate discrepant results in 7 individuals with positive S-PLEX and negative RT-PCR in this prospective cohort, who had median antigen concentrations of 2.68 (IQR 2.63–2.70) log₁₀ fg/mL or 482 (IQR 428–504) fg/mL. One asymptomatic individual underwent initial RT-PCR testing because of household exposure, but developed COVID-19 9 days later. Another presented with sore throat and congestion without known exposure, but did not undergo subsequent RT-PCR. Of the remaining 5, 1 was asymptomatic and tested for pretravel screening and 4 had no available clinical data.

Reviewing results from retrospective and prospective cohorts combined, C_t and antigen concentration were linearly correlated by Passing-Bablok regression (R² = 0.63), though with considerable inter-individual variation (Figure 1A). Samples were collected 0–41 days after initial presentation in RT-PCR–positive individuals, with median of 0 days (IQR 0–0.75, maximum 37) in antigen-positive individuals, and median of 0 days (IQR 0–18, maximum 41) in antigen-negative individuals (Figure 1B). However, days since initial presentation was not significantly associated with antigen concentration after multivariable adjustment (Supplemental Table 4). Antigen was detectable 37 days after initial presentation in a nonimmunocompromised outpatient with mild disease who had recovered. Another nonimmunocompromised individual with severe disease remained hospitalized at the time of antigen detection, 30 days after initial presentation. Similarly, individuals with false-negative antigen results (n = 31) spanned a range of illness trajectories at the time of sample collection, from initial diagnostic (n = 19), to persistently symptomatic and/or hospitalized (n = 5), to fully recovered (n = 7).

Figure 1.

Relationship between log₁₀ antigen concentration (fg/mL) versus cycle threshold (C_t) value in (A) RNA-positive, antigen-positive samples with antigen levels in the linear range (n = 79), and (B) all RNA-positive samples with available clinical data (n = 167). (A) Passing-Bablok regression (dashed line) of antigen versus RNA levels, with shaded area representing the bootstrap 95% confidence interval. (B) Subset by cohort (retrospective on left and prospective on right), and colored by number of days between initial presentation and specimen collection. The horizontal dashed line represents the limit of detection threshold, above which samples were called positive for antigen. The vertical dashed line represents a C_t of 35.

DISCUSSION

In this study, we performed a comprehensive assessment of an ultrasensitive electrochemiluminescence immunoassay for SARS-CoV-2 nucleocapsid antigen detection in 450 individuals, and demonstrated similar PPA to RT-PCR for upper respiratory specimens with C_t <35. A prior retrospective study assessing S-PLEX demonstrated similar performance in frozen adult and pediatric samples [12]. Our study generally corroborates these findings and extends them to a larger and clinically characterized prospective cohort using fresh clinical specimens. The 93% NPA in our study, however, was noted to be lower than the specificity in this prior study and several commercially available antigen assays (NPA 97%–100%) [3–6]. This may be partially explained by differences in patient populations and study design (prospective with consecutive positive samples versus retrospective with selected well-characterized specimens). Further work is required to determine whether antigen-positive, RNA-negative specimens are falsely positive, or represent biologically relevant findings. Repeat/confirmatory testing may be required for specimens with low-positive antigen levels by S-PLEX.

This assay offers a high-throughput, relatively compact SARS-CoV-2 nucleocapsid antigen testing approach well suited for implementation in high-complexity laboratories, but would not supplant point-of-care rapid testing. We estimate that 2 clinical laboratory scientists can perform ~960 tests in an 8-hour shift, although future advances may increase throughput or reduce turnaround time (currently ≥4 hours). Given the observed performance, use of this assay may be envisioned in high-volume and/or surge SARS-CoV-2 testing settings, especially to identify the high-priority group of individuals with C_t <35 who are most likely to contribute to onward transmission [8–10]. Nonetheless, its lower overall PPA compared with RT-PCR is an assay limitation. Indeed, despite observed late C_t values, 61% (19/31) of false-negative samples were collected on the day of initial diagnosis. Although it remains unclear whether these individuals were infectious, a missed diagnosis could hinder appropriate clinical management. Ultimately, this assay is expected to offer sensitivity between that of RT-PCR and rapid antigen tests, and may offer a complementary diagnostic option to help mitigate ongoing molecular supply shortages.

Limitations of this study include limited patient follow-up time precluding a full assessment of the relationship between antigen detection and clinical outcomes, limited data about different specimen types and collection media, and lack of a third laboratory method, such as serology, to adjudicate discrepant RT-PCR and antigen results.

In summary, the S-PLEX ultrasensitive SARS-CoV-2 nucleocapsid antigen assay demonstrated comparable PPA to RT-PCR in upper respiratory samples with C_t <35, suggesting potential use for acute SARS-CoV-2 diagnosis. The observed NPA of 93% suggests confirmatory testing may be required for low-positive samples. Further work will be needed to assess the benefit of expanding testing to additional sample types including plasma, perform cost-effectiveness and feasibility analyses of different testing strategies, and assess the potential incremental utility as a prognostic tool for COVID-19.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Financial support. This work was supported by MesoScale Diagnostics (MSD), which provided the S-PLEX SARS-CoV-2 N Kits, BioTek 405 Select automated 96-well plate washer, and MESO SECTOR S 600 Reader used in this study. MSD had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Potential conflicts of interest. No authors have potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.