Utility of Roche Elecsys anti-SARS-CoV-2 S in ascertaining post-vaccine neutralizing antibodies

Abstract

With widespread global COVID-19 vaccine coverage, a scalable, cost-effective, and standardized tool to ascertain post-vaccine immunity is a dire need. Neither clinical evaluations of vaccine efficacy, nor live virus antibody neutralization assays fulfill these criteria. Commercially available anti-S binding immunological assays have the potential to fill this gap, but need to be systematically evaluated for their utility to serve as surrogates for the aforementioned, widely accepted tools of determining vaccine efficacy. In this study, we evaluated an anti-S binding immunological assay (Roche Elecsys Anti-SARS-CoV-2 S) by utilizing two hundred and fifty-five archived serum specimens, either pre-pandemic, or those exposed to natural infections or vaccines with their neutralizing titers pre-determined through a live virus, pseudotyped antibody neutralization assay. Roche Elecsys Anti-SARS-CoV-2 S demonstrated good sensitivity (98%) and specificity (99%), just as has been reported in some other previously conducted studies using this assay. Only a mild correlation, however, with the live virus pseudotyped lentivirus antibody neutralization assay (Spearman's r = 0.26) was observed. We conclude that, as such, Elecsys Anti-SARS-CoV-2 S has a high sensitivity and specificity for detecting anti-SARS-CoV-2 S proteins, though the assay does not always correlate well with live virus assays for quantitative outcomes.

1. Background

Clinical evaluations of vaccine efficacy are an important epidemiological tool to predict the possibility of future COVID-19 infection waves, as well as that of critical illness and death due to the disease. Notwithstanding the cost and time-intensive nature of these evaluations, however, these are also being hampered by high COVID-19 vaccination coverage and viral prevalence [1]. Although there is no ideal immunological assay to ascertain efficacy of vaccines, neutralizing antibody titers are considered a relevant measure of immunogenicity [2]. Neutralizing antibody levels have shown to correlate with vaccine efficacy for several viruses, including SARS-CoV-2 [3], and have been used in the past as vaccine efficacy endpoints for other viral vaccines.

Live virus assays (PRNT and pseudotyped virus neutralization assays) are considered to be the most reliable methods of determining neutralizing antibody titers [4]. Notwithstanding, these assays are laborious, technically demanding and resource intensive, rendering them non-feasible for population-level assessment of immunity to vaccines. This brings forth the need for alternate immunological endpoints [5]. Studies suggest that RBD of the SARS-CoV-2 spike (S) protein is a target for 90% of the neutralizing SARS-CoV-2 immune sera [6,7]. It is possible thus that RBD S-protein-based, binding immunologic assays serve as useful tools for ascertaining the neutralizing antibody titers, and thus evaluation of protection against the virus [8], [9], [10]. Ascertaining the strength of titer correlation of live virus and binding immunological assays could be one possible way of determining the utility of RBD S-protein-based, binding immunologic assays to serve as surrogates to live virus assays.

In September 2020, Roche launched an S-protein-based, binding immunologic assay (Elecsys Anti-SARS-CoV-2 S) to quantitatively measure the level of antibodies against the RBD domain located in the spike protein. The assay is automated and high throughput, rendering it a potentially ideal substitute to live virus assays for evaluations of post-vaccine immunity.

2. Objectives

This study evaluated the specificity and sensitivity of Elecsys Anti-SARS-CoV-2 S in both vaccinated and COVID-19 recovered individuals, and its correlation to live virus, pseudotyped, antibody neutralization assay.

3. Study design

The study was conducted at the Indus Hospital & Health Network, Karachi. The protocol was approved by the Institutional Review Board (IRD-IRB) of Indus Hospital, Karachi (IRD_IRB_2020_07_018). All experiments were carried out in compliance with relevant laws and guidelines, and with the ethical standards of the Declaration of Helsinki.

De-identified, archived serum specimens of (1) BBIBP-CorV vaccine recipients (n=60), (2) BBIBP-CorV vaccine recipients with prior SARS-CoV-2 positivity (n=60), (3) non-vaccinated, COVID-19 recovered individuals (n=60), and (4) and pre-pandemic specimens (n=75) were used to quantify anti-RBD S1 using the Roche Elecsys Anti-SARS-CoV-2 S assay, following manufacturer's instructions. The vaccine or virus exposed specimens had been collected between April to August, 2021, 4–8 weeks following the last documented exposure to the virus or the vaccine in each case. These specimens had previously been evaluated by a pseudotyped lentivirus antibody neutralization assay described by Crawford et al, with minor modifications [11]. The lentivirus spike (S) protein (plasmid: BEI Resources, catalogue # NR-53742) was of the Wuhan-Hu-1 variant (GenBank: NC_045512), albeit with a C-terminal deletion of amino acids 1257-1278, intended to enhance viral titers.

To ascertain sensitivity, specificity, PPV, NPV, and diagnostic accuracy, standard calculation metrics were used (Table 1 ). Correlation of the log-transformed titers obtained using Roche Elecsys Anti-SARS-CoV-2 S (log U / ml) to the log-transformed IC50 values (log IC50) of the pseduotyped lentivirus neutralization assay, was determined through Spearman rank-order correlation coefficient (rs).

Table 1.

Calculated diagnostic indices of Roche Elecsys Anti-SARS-CoV-2 S.

Statistic	Value	95% CI
Clinical Sensitivity	98.33%	95.21% to 99.65%
Clinical Specificity	98.67%	92.79% to 99.97%
Positive Predictive Value	99.44%	96.19% to 99.92%
Negative Predictive Value	96.10%	88.92% to 98.70%
Diagnostic Accuracy	98.43%	96.03% to 99.57%

4. Results and discussion

Of the 180 serum specimens obtained from individuals exposed to either the vaccine, SARS-CoV-2, or both, only 3 gave negative results by Elecsys Anti-SARS-CoV-2 S. All three of these had been exposed to the virus, but not the vaccine. Of the 75 pre-pandemic specimens, only one turned a positive result with the assay. Table 1 gives the overall calculated diagnostic indices.

Several other studies [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32] have also ascertained clinical sensitivity of the assay after exposure to the SARS-CoV-2 virus or vaccine or both, and specificity through evaluation of pre-pandemic specimens. Sensitivities have been reported at time intervals varying from 14 days to 240 days post-exposure. Almost all studies have reported high specificity (99-100%), while the reported sensitivities range from 93 to 100%. Our study thus confirms high sensitivity and specificity of the assay in the detection of exposure to the virus or the vaccine.

The correlations between Roche Elecsys Anti-SARS-CoV-2 S and the pseudotyped lentivirus antibody neutralization assay was, however, mild though statistically significant (Fig. 1 ).

Figure 1.

Distribution of Elecsys Anti SARS-CoV-2 S titers in Log U / ml by the corresponding, ascending Log IC50 titers, as determined by the pseudotyped lentivirus antibody neutralization assay. (Spearman's rs = 0.26, p-value = 0.00054).

Organizing the live virus antibody neutralizing titers on an interval scale, we observed an overall increase of Elecsys Anti SARS-CoV-2 S mean antibody titer with progressively increasing ranges of log IC50 values. One notable exception though was the last interval (log IC50 4-4.99), which only included a few data points. Of note, however, the mean titer obtained through Roche Elecsys Anti-SARS-CoV-2 S rose significantly only from the log IC50 ranges of 0-0.99 to 1-1.99 (Fig. 2 ). Other differences between the mean titers obtained through Elecsys Anti-SARS-CoV-2 S across the various log IC 50 intervals were statistically non-significant.

Figure 2.

Distribution of Elecsys Anti-SARS-CoV-2 S titers in Log U / ml by Log IC50 titer ranges, as determined by the pseudotyped lentivirus antibody neutralization assay *p = 0.0004, ns = not significant

A few other studies have also determined correlation of Roche Elecsys Anti-SARS-CoV-2 S with one of the live virus assays. There is more diversity, however, of reported outcomes with regards to the strength of this correlation than there is with sensitivity and specificity. Wilkins et al. [12], Jochum et al. [23], Auerswald et al. [25], and Takei et al. [31] report relatively high correlation coefficients (Spearman's / Pearson’ r in the range of 0.7 – 0.85), while Einhauser et al. [14] have reported moderate correlation (0.53) between the two assays. Rus et al [24] reported a Cohen's Kappa of 0.56. Rubio-Acero et al [13] report a binary outcome of a significant increase in mean titers obtained through Elecsys Anti-SARS-CoV-2 S when compared against the live virus neutralization assay dilution categories of < 1:5 or > 1:80. These findings may suggest the assay's sensitivity for a dichotomous outcome, but relatively non-consistent quantitative correlation with neutralizing antibody titers, as determined by live virus assays.

Notwithstanding, our assessment of the correlation between the two assays is lower that reported in the literature thus far, bringing forth a possible conclusion that the Elecsys Anti-SARS-CoV-2 S assay may not be the ideal surrogate for neutralizing virus assays. A non-linear relationship between the two assays may be a contributing factor, as observed by Tolan et al. [32].

On the other hand, absence of a strong observed correlation between the two assays for a quantitative outcome may signify a need to standardize the laboratory-developed live virus antibody neutralization assays [1]. Currently, a host of methods are in use for these assays across laboratories. These include the microneutralization assays, pseudotyped lentivrus neutralization assays, and the PRNT live virus neutralization assays, with additional variability in each of these categories inducted by diverse protocols and interpretation criteria used by each laboratory. Employing standard control materials for assay calibration, and inter-laboratory comparison of the neutralizing antibody titers obtained through these assays may facilitate standardization [36]. Ultimately however, unless extensive development and validation of these laboratory-developed assays for accuracy, precision, analytical sensitivity, analytical specificity, linearity, limit of detection, and other parameters has been undertaken, their performance and reproducibility may be compromised when compared to commercial assays [33,34].

Another factor meriting consideration is the documented variability in the correlation of the two assays with the type of prior virus antigen variant exposure, vaccination status, and vaccine type [35,36]. Additional, undocumented prior virus exposures in participants of this study are highly likely given the prevalence of the virus at the time of study participant specimen collection. Furthermore, data on the predominant variants in Pakistan during the specimen collection interval remains sparse, though globally the Delta variant dominated [37]. As regards assay reagent antigens, the spike protein used for the pseudotyped lentivirus assay was from the Wuhan-Hu-1 variant, while the sequence of reagent antigen used in the commercial Roche Elecsys Anti-SARS-CoV-2 S remains proprietary information. These factors likely induced unaccounted for variability in the antibody profile of the study participants. Taken together, the diversity of magnitude, frequencies, and antigenic types in exposures of the study participants to the SARS-CoV-2 antigens, as well as different sensitivities of the two assays to various antibody types, are other plausible factors accounting for the low correlation we observed between the two assays.

We conclude that, as such, Elecsys Anti-SARS-CoV-2 S has a high sensitivity and specificity for detecting anti-SARS-CoV-2 S proteins, though the assay does not always correlate well with live virus assays for quantitative outcomes. We surmise that the laboratory developed nature, lack of standardization criteria, and availability of external proficiency testing for live virus assays are some factors contributing to this imprecision. Therefore, in our conclusion, we do not necessarily cast doubt on the utility of Roche Anti-SARS-CoV-2 S as a quantitative assay, but rather also focus on the current limitations of the live virus assays. In addition, in the face a wide variety of magnitudes and types of antigen exposures to SARS-CoV-2 vaccines or virus variants, stratification of results by these variables may engender more informative results. Studies focusing on the correlation of anti-S binding immunological assay titers with clinical evaluations of vaccine efficacy might also provide some reliable answers to the utility of these assays as surrogates for vaccine efficacy.

Declaration of Competing Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.