Weak correlation between antibody titers and neutralizing activity in sera from SARS‐CoV‐2 infected subjects

Abstract

Plenty of serologic tests for severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) have been developed so far, thus documenting the importance of evaluating the relevant features of the immune response to this viral agent. The performance of these assays is currently under investigation. Amongst them, LIAISON® SARS‐CoV‐2 S1/S2 IgG by DiaSorin and Elecsys Anti‐SARS‐CoV‐2 cobas® by Roche are currently used by laboratory medicine hospital departments in Italy and many other countries. In the present study, we firstly compared two serologic tests on serum samples collected at two different time points from 46 laboratory‐confirmed coronavirus disease‐2019 (COVID‐19) subjects. Secondly, 85 negative serum samples collected before the SARS‐CoV‐2 pandemic were analyzed. Thirdly, possible correlations between antibody levels and the resulting neutralizing activity against a clinical isolate of SARS‐CoV‐2 were evaluated. Results revealed that both tests are endowed with low sensitivity on the day of hospital admission, which increased to 97.8% and 100% for samples collected after 15 days for DiaSorin and Roche tests, respectively. The specificity evaluated for the two tests ranges from 96.5% to 100%, respectively. Importantly, a poor direct correlation between antibody titers and neutralizing activity levels was evidenced in the present study. These data further shed light on both potentials and possible limitations related to SARS‐CoV‐2 serology. In this context, great efforts are still necessary for investigating antibody kinetics to develop novel diagnostic algorithms. Moreover, further investigations on the role of neutralizing antibodies and their correlate of protection will be of paramount importance for the development of effective vaccines.

Highlights

• A comparative analysis between two serologic assays for the detection of antibodies directed against SARS‐CoV‐2 was carried out on sera from subjects testing positive for SARS‐CoV‐2 nasopharyngeal swabs.

• Results revealed that both tests are endowed with low sensitivity on the day of hospital admission, which increased for samples collected after 15 days.

• A poor direct correlation between antibody titers and neutralizing activity levels was evidenced in the present study.

Abbreviations

AUC

area under the curve

CI

confidence interval

COI

cutoff index

CPE

cytopathic effect

DMEM

Dulbecco's modified Eagle's medium

ECLIA

electrochemiluminescence immunoassay

EDA

Endpoint Dilutions Assay

FBS

fetal bovine serum

ICU

intensive care unit

MOI

multiplicity of infection

N

nucleocapsid

NEAA

non‐essential amino acids

P/S

penicillin/streptomycin

PFU

plaque forming unit

PRA

plaque reduction assay

S

spike

SE

standard error

TCID₅₀

median tissue culture infectious dose

1. BACKGROUND

Tracking the seroprevalence of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) positive subjects certainly represents an urgent epidemiological need for facing coronavirus disease‐2019 (COVID‐19) pandemic waves worldwide. Additionally, commercially available serological assays are of pivotal importance from a diagnostic point of view, especially in those subjects with a clinical picture suggestive for COVID‐19 but lacking a molecular‐based confirmation of SARS‐CoV‐2 infection. ¹ To date, several studies have characterized the kinetics of SARS‐CoV‐2‐elicited antibodies evidencing immunoglobulin M (IgM) within 5 days from symptoms onset for and IgG within approximatively 7 days, ² , ³ , ⁴ , ⁵ but the persistence of the humoral immunity towards SARS‐CoV‐2 seems to be poor. ⁶ , ⁷ A plethora of serologic tests are invading the market, and some of them have been evaluated. ⁸ However, other assays deserve further analyses as there is no consensus so far on antigens used for the antibody testing nor for the antibody isotype to be detected. These last aspects can be of pivotal importance for evaluating antibody response to candidate vaccines, for selecting plasmas for clinical trials, and for dissecting unknown immunological aspects related to seroconversion. Indeed, according to WHO, as of 24th September 2020, no correlations between seroconversion, neutralizing activity, and immunity have been made. ⁹ In the present study, we evaluated the performances of two commercial serology tests, the LIAISON® SARS‐CoV‐2 S1/S2 IgG by DiaSorin and Elecsys Anti‐SARS‐CoV‐2 cobas® by Roche, on 46 COVID‐19 patients and 85 sera collected before the current pandemic. Data obtained from both commercial assays were then compared to the neutralizing activity of each serum against a clinical isolate of SARS‐CoV‐2, a clinical isolate from a patient admitted to San Raffaele Hospital, Milan (Italy) during the early COVID‐19 pandemic in Lombardy.

2. MATERIALS AND METHODS

2.1. Clinical samples

The study was reviewed and approved by San Raffaele Hospital IRB in the COVID‐19 Biobanking project “COVID‐BioB” N° CE: 34/int/2020 19/March/2020 ClinicalTrials.gov Identifier: NCT04318366. Forty‐six serum samples were randomly collected from laboratory‐confirmed symptomatic COVID‐19 patients on their admission to the hospital (T0) and 15 days later (T15). Eighty‐five “pre‐pandemic” serum samples, spanning from 2012 to 2018, were also tested for the presence of anti‐SARS‐CoV‐2 antibodies.

2.2. Immunoassays

Elecsys Anti‐SARS‐CoV‐2 cobas® by Roche and LIAISON® SARS‐CoV‐2 S1/S2 IgG assay by DiaSorin were used for detecting anti‐SARS‐CoV‐2 antibodies in all serum samples. Analyses were performed according to the manufacturer's instructions by using cobas® and LIAISON® XL Analyzer platforms. In brief, Elecsys by Roche uses a recombinant SARS‐CoV‐2 nucleocapsid (N) antigen. The electrochemiluminescence immunoassay (ECLIA) can detect the presence of IgG, IgM, and IgA antibodies recognizing the N protein. ¹⁰ According to the producer, samples positive for anti‐SARS‐CoV‐2 antibodies show a cutoff index (COI) equal to or greater than 1. All samples with a COI < 1.0 are considered negative for the presence of SARS‐CoV‐2 antibodies. The SARS‐CoV‐2 S1/S2 IgG assay by DiaSorin can detect IgG antibodies directed against two recombinant SARS‐CoV‐2 proteins: the S1 and S2 which are involved in both docking and fusion processes of the virus. ¹¹ According to manufacturer instructions, the test by DiaSorin can detect the presence of neutralizing antibodies directed against the spike protein of SARS‐CoV‐2. Samples featuring < 12.0 AU/ml are considered negative according to manufacturer instructions, those ranging between 12.0 and 15.0 AU/ml are undetermined and those above 15 AU/ml are positive. According to the manufacturer instructions, the tests were performed once except for undetermined values that were tested twice.

2.3. Virus and cells

Vero E6 (Vero C1008, clone E6—CRL‐1586; ATCC) cells were cultured in Dulbecco's modified Eagle's medium (DMEM) supplemented with non‐essential amino acids (NEAA), penicillin/streptomycin (P/S), Hepes buffer, and 10% (v/v) fetal bovine serum (FBS). A clinical isolate of SARS‐CoV‐2 (hCoV‐19/Italy/UniSR1/2020; GISAID Accession ID: EPI_ISL_413489) was obtained and propagated in Vero E6 cells.

2.4. Virus titration

Virus stocks were titrated using both Plaque Reduction Assay (PRA, PFU/ml) and Endpoint Dilutions Assay (EDA, TCID₅₀/ml). In PRA, confluent monolayers of Vero E6 cells were infected with eight 10‐fold dilutions of virus stock. After 1 h of adsorption at 37°C, the cell‐free virus was removed. Cells were then incubated for 48 h in DMEM containing 2% FBS and 0.5% agarose. Cells were fixed and stained, and viral plaques were counted. In EDA, Vero E6 cells were seeded into 96 wells plates and infected at 95% of confluency with base 10 dilutions of virus stock. After 1 h of adsorption at 37°C, the cell‐free virus was removed, cells were washed with PBS 1×, and complete medium was added to cells. After 48 h, cells were observed to evaluate the presence of a cytopathic effect (CPE). TCID₅₀/ml of viral stocks were then determined by applying the Reed–Muench formula.

2.5. Microneutralization experiments

Vero E6 cells were seeded into 96 wells plates 24 h before the experiment performed at 95% cell confluency for each well. Serum samples were decomplemented by incubation at 56°C for 30 min, and dilutions (two‐fold serial dilutions, starting from 1:10) incubated with SARS‐CoV‐2 at 0.001 multiplicity of infection (MOI) for 1 h at 37°C. Virus–serum mixtures and positive infection control were applied to Vero E6 monolayers after washing cells with PBS 1×, and virus adsorption was carried out at 37°C for 1 h. Then, the cells were washed with PBS 1× to remove cell‐free virus particles and virus‐containing mixtures and controls were replaced with complete DMEM supplemented with 2% FBS. The plates were incubated at 37°C in the presence of CO₂ for 72 h. The experiments were performed in triplicate. Neutralization activity was evaluated by comparing CPE presence detected in the presence of virus–serum mixtures to positive infection control.

2.6. Data analysis and statistics

Test performances were evaluated by sensitivity and specificity with the associated standard error (SE) and 95% confidence interval (CI). Differences between sensitivities and specificities, respectively, were assessed by exact binomial test for paired study design. ¹² The overall performances were also measured, by means of the ROC curves and the associated area under the curve (AUC). For graphical purposes, we considered smooth ROC curves based on kernel estimators ¹³ with unbiased cross‐validation bandwidth selection. We verified differences between AUCs by means of stratified bootstrap resampling for paired data. ¹⁴ Relationships between values obtained with the two diagnostic methods and neutralizing activity of sera were investigated by Spearman's correlation coefficient and fitting spline functions. p < .05 were considered significant. Al computation and analysis were performed in the R environment (R ver. 4.0.0).

3. RESULTS

3.1. Sensitivity and specificity of DiaSorin and Roche diagnostic tests

Pre‐pandemic sera were tested with DiaSorin and Roche diagnostic assays (Figure 1A). Three out of 85 samples tested positive with DiaSorin, and one tested as undetermined (12.0–15.0 AU/ml) with the same test, while none out of the 85 samples tested positive with the Roche test. Thus, the diagnostic specificity observed on the tested samples for the DiaSorin assay was 96.5% (SE, 2%; 95% CI, 92.5%–100%) and 100% for the Roche test (SE, 0%; 95% CI, 100%–100%). Then, 46 serum samples were randomly collected from laboratory‐confirmed symptomatic COVID‐19 patients. Admission to the hospital (T0) and 15 days later (T15) time points were evaluated for each patient, showing an overall increase in serum IgG titers 15 days after hospital admission (Figure 1B). The diagnostic sensitivity at T0 was 19.6% (SE, 5.8%; 95% CI, 8.1%–31%), and at T15 was 100% (SE, 0%; 95% CI, 100%–100%). On the other hand, the diagnostic sensitivity of the Roche test obtained on the same samples at T0 was 45.7% (SE, 7.3%; 95% CI, 31.2%–60%), and at T15 was 100% (SE, 0%; 95% CI, 100%–100%). Roc curves, calculated for the two tests at the two different time points, show comparable performance at T15 (p = .2961). Thus, the Roche diagnostic test showed a statistically better performance than DiaSorin at T0 on the tested samples (p < .001) (Figure 1C).

Figure 1.

Sensitivity and specificity of the two commercial assays. A, “Pre‐pandemic” samples tested with both DiaSorin and Roche assays (●, black line). B, Antibody levels detected by DiaSorin (, T0 solid blue line, T15 dotted blue line) and Roche (, T0 solid red line, T15 dotted red line) tests of sera from subjects with positive nasopharyngeal swabs. C, Roc curves for DiaSorin (T0 solid blue line, T15 dotted blue line) and Roche (T0 solid red line, T15 dotted red line) tests are reported. *p < .001

3.2. Neutralizing activity evaluation of a limited cohort of serum samples

Five patients were randomly selected for the characterization of the neutralizing activity of their serum samples against SARS‐CoV‐2, at both T0 and T15 (Figure 2A). Results showed that samples collected at T0 were not able to neutralize virus infection, consistent with the low antibody titers detected by both diagnostic tests, apart from infection inhibition above 50% observed only at a very high concentration (1:10 for ID #7 and 1:20 for sample ID #3). At T15, only one out of five samples (ID #4) showed detectable neutralizing activity also when a high dilution (1:800) was used, while two samples (ID #2 and #3) strongly neutralized the infection only when used up to 1:160. Notably, the T15 of sample ID #4 potently inhibited viral replication despite its low antibody titer compared to other T15‐samples detected by DiaSorin and Roche (48.1 AU/ml and 13 COI, respectively) (Figure 2B).

Figure 2.

Evaluation of serum neutralizing activity at different serum concentrations. A, Neutralizing activity of five patients’ serum, at T0 and T15. Sera were two‐fold serial diluted (starting from 1:10) and tested with 0.001 MOI of SARS‐CoV‐2. B, Antibody titers detected for each serum at both timepoints are also reported for DiaSorin and Roche, respectively. DiaSorin readings indicated as <3.8 AU/ml refers to values falling outside the instrument's linearity range. One out of five patients admitted to ICU is highlighted in the table

3.3. Neutralizing activity evaluation of all tested sera

As the previous analysis showed that no specific neutralizing activity was detected at T0, all remaining sera were tested at T15. All T15 samples were tested at 1:100 (Figures 3) and 1:200 dilution (Figure 4), based on results observed in the five sera tested at different dilutions in which the neutralizing activity was appreciable only at the lowest dilutions suggesting possible nonspecific activity. As a result, the neutralizing activity of 1:100 diluted sera does not directly correlate with antibody titers detected with both DiaSorin and Roche test, as highlighted by the graph (Figure 3A). Moreover, no apparent relationship between ICU admission and both antibody titer and neutralizing activity was appreciated from our data and this aspect is not directly addressed in our evaluation. None of the five “pre‐pandemic” sera checked for their neutralizing activity, including those testing positive with the DiaSorin kit, neutralized the virus. Spearman correlation analysis confirms the lack of correlation between anti‐SARS‐CoV‐2 antibody presence detected with both diagnostic serologic assays and the neutralizing activity (Figure 3B). Results obtained with 1:200 dilution of sera confirmed what was observed, emphasizing the lack of correlation between neutralizing activity and antibody titers revealed by both diagnostic tests (Figure 4A). Once again, a Spearman correlation analysis confirms how the two variables are unrelated (Figure 4B).

Figure 3.

Characterization of anti‐SARS‐CoV‐2 serum antibodies and evaluation of neutralizing activity. A, Sera reported in the graphs ordered by their neutralizing capability at a dilution of 1:100 (, grey line). Levels of SARS‐CoV‐2 specific antibodies detected with the tests by Roche and DiaSorin are also indicated (●, green and blue lines). Red dots indicate patients admitted to ICU. B, Spearman correlation analyses between values obtained with the two diagnostic methods and neutralizing activity of sera at 1:100 dilution

Figure 4.

Characterization of anti‐SARS‐CoV‐2 serum antibodies and evaluation of neutralizing activity at a lower dilution. A, Graphs rank the sera basing on their neutralizing capability. All sera were tested at a dilution of 1:200 (, grey line). Levels of SARS‐CoV‐2 specific antibodies detected with the tests by Roche and DiaSorin are also indicated (●, green and blue lines). Red dots indicate patients admitted to ICU. B, Spearman correlation analyses between values obtained with the two diagnostic methods and neutralizing activity of sera at 1:200 dilution

4. DISCUSSION

A comparative analysis between two serologic assays for the detection of antibodies directed against SARS‐CoV‐2 was carried out on sera collected from subjects testing positive for SARS‐CoV‐2 RNA on nasopharyngeal swabs. The same analyses were also performed on sera collected before the COVID‐19 current pandemic. Overall, both commercial assays are characterized by a good sensitivity when analyzing serum samples collected from subjects 15 days after their presentation to the physician (T15), with the test by DiaSorin behaving slightly better compared to Roche at this timepoint. Even so, LIAISON SARS‐CoV‐2 S1/S2 by DiaSorin showed less specificity than the Roche test as it detected anti‐SARS‐CoV‐2 antibodies also on control sera collected well before the COVID‐19 pandemic and confirmed by recently published results. ¹⁵ What was observed for positive results obtained with DiaSorin assays on “pre‐pandemic” sera was possibly due to cross‐reactions in accord to what was described by the producer which reported 3  of 168 positive detections in sera collected before October 2019. ¹¹ The sensitivity of both DiaSorin and Roche assays was dramatically lower (19.6% and 45.7%, respectively; p < .001), for sera collected at T0. Differences in terms of sensitivity between the two methods at T0 could be mainly attributed to the capability of Roche assay to detect also for IgM antibodies. ¹⁰ This observation can be of help when testing serology in subjects with a clinical picture suggestive for COVID‐19 but negative to the direct detection of SARS‐CoV‐2 on nasopharyngeal swabs or bronchoalveolar lavages. Other important differences for the two commercial assays possibly impacting their performances include the SARS‐CoV‐2 recombinant antigens included in the commercial kits. The Elecsys by Roche detects antibodies able to selectively bind a recombinant form of nucleocapsid protein N of SARS‐CoV‐2. However, DiaSorin assay detects IgGs able to bind the S1 or the S2 recombinant portions of the virus spike glycoprotein. The S protein mediates at least two crucial steps in the early phases of the viral productive infection: the docking to host cell receptor or putative co‐receptors and the fusion process. This observation and the need to infer possible correlations between the presence of antibodies against SARS‐CoV‐2 S protein triggered the second round of laboratory investigations we performed. For this purpose, all samples from COVID‐19 subjects already tested with both serological commercial tests were also tested for their neutralizing activity against a clinical isolate of SARS‐CoV‐2. A first pilot microneutralization experiment was performed by using 0.001 MOI of the virus against several dilutions of a small cohort of five sera. From this experiment, it was evident that the detected antibody levels were unrelated to the neutralizing capability of sera tested at dilutions spanning from 1:10 to 1:800. On that basis, all sera were then checked for their neutralizing activity against a clinical isolate of SARS‐CoV‐2 at a dilution of 1:100 and 1:200. The results underlined the lack of relationship between antibody titer detected with the two commercial tests and neutralizing activity at the tested concentrations. This observation does not surprise for antibodies directed against N proteins revealed by Roche test but should be considered cautiously when referring to the indication reported by the DiaSorin brochure ¹¹ of the commercial kit which states that the test can give a clue on the presence of neutralizing antibodies directed against SARS‐CoV‐2. Although the results by Wu et al. ¹⁶ seem to refute our conclusions, it is crucial to underline that their data derived from samples of patients with common or mild symptoms, none of them admitted to the ICU. Also, pseudovirus instead of the true whole virus was used for the neutralization assays, a safer methodology but endowed with intrinsic limitations. Thus, their results are not comparable to the present study.

5. CONCLUSIONS

Our observation can impact directly the methods to be used for detecting possible correlates of protection from infections for subjects enrolled in vaccine clinical trials based on the S protein. ¹⁷ , ¹⁸ However, it is of pivotal importance to highlight that even the serum neutralizing activity was not related to protection so far, as the contribution of humoral immunity in the effective resolution of the SARS‐CoV‐2 infection is still under question. ⁷ , ¹⁵ , ¹⁹ , ²⁰ , ²¹ Gathering our observations and literature data, we believe that great efforts are still necessary for implementing observations on antibody kinetics to develop novel diagnostic algorithms useful both for epidemiological and clinical purposes. Moreover, further investigations elucidating the clinical role of neutralizing antibodies and the possibility of detecting them with binding assays will be of paramount importance for addressing the development of effective vaccines.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Nicola Clementi, Massimo Clementi, Nicasio Mancini conceived and directed the study, Elena Criscuolo, Roberta A. Diotti, Marta Strollo, and Serena Rolla performed all the experiments, Massimo Locatelli provided advice on the design of the study, Elena Criscuolo, Roberta A. Diotti, and Nicola Clementi analyzed all the data, Alessandro Ambrosi performed the statistical analysis, Elena Criscuolo and Nicola Clementi assembled the figures and wrote the manuscript. Elena Criscuolo, Roberta A. Diotti, Massimo Locatelli, Roberto Burioni, Nicasio Mancini, Massimo Clementi, and Nicola Clementi revised the manuscript. All authors read and approved the final manuscript.

Criscuolo E, Diotti RA, Strollo M, et al. Weak correlation between antibody titers and neutralizing activity in sera from SARS‐CoV‐2 infected subjects. J Med Virol. 2021;93:2160–2167. 10.1002/jmv.26605

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.