Abstract
The information provided by SARS‐CoV‐2 spike (S)‐targeting immunoassays can be instrumental in clinical‐decision making. We compared the performance of the Elecsys® Anti‐SARS‐CoV‐2 S assay (Roche Diagnostics) and the LIAISON® SARS‐CoV‐2 TrimericS IgG assay (DiaSorin) using a total of 1176 sera from 797 individuals, of which 286 were from vaccinated‐SARS‐CoV‐2/experienced (Vac‐Ex), 581 from vaccinated/naïve (Vac‐N), 147 from unvaccinated/experienced (Unvac‐Ex), and 162 from unvaccinated/naïve (Unvac‐N) individuals. The Roche assay returned a higher number of positive results (907 vs. 790; p = 0.45; overall sensitivity: 89.3% vs. 77.6%). The concordance between results provided by the two immunoassays was higher for sera from Vac‐N (ϰ: 0.58; interquartile ranges [IQR]: 0.50−0.65) than for sera from Vac‐Ex (ϰ: 0.19; IQR: −0.14 to 0.52) or Unvac‐Ex (ϰ: 0.18; IQR: 0.06−0.30). Discordant results occurred more frequently among sera from Unvac‐Ex (34.7%) followed by Vac‐N (14.6%) and Vac‐Ex (2.7%). Antibody levels quantified by both immunoassays were not significantly different when <250 (p = 0.87) or <1000 BAU/ml (p = 0.13); in contrast, for sera ≥1000 BAU/ml, the Roche assay returned significantly higher values than the DiaSorin assay (p < 0.008). Neutralizing antibody titers (NtAb) were measured in 127 sera from Vac‐Ex or Vac‐N using a S‐pseudotyped virus neutralization assay of Wuhan‐Hu‐1, Omicron BA.1, and Omicron BA.2. The correlation between antibody levels and NtAb titers was higher for sera from Vac‐N than those from Vac‐Ex, irrespective of the (sub)variant considered. In conclusion, neither qualitative nor quantitative results returned by both immunoassays are interchangeable. The performance of both assays was found to be greatly influenced by the vaccination and SARS‐CoV‐2 infection status of individuals.
Keywords: antibodies, DiaSorin immunoassay, neutralizing antibodies, receptor binding domain antibodies, Roche immunoassay, SARS‐CoV‐2, S‐trimeric antibodies
1. INTRODUCTION
Extensive evidence supports a major role of antibodies binding the spike (S) protein of SARS‐CoV‐2 in providing protection against acquisition of infection and development of COVID‐19, in particular those displaying virus‐neutralizing activity elicited following vaccination or after natural infection 1 , 2 , 3 , 4 , 5 , 6 ; nevertheless, elucidation of antibody thresholds conferring protection remains elusive. Detection and quantitative estimation of S‐binding antibodies are pivotal for assessment of immune status against SARS‐CoV‐2, due to their potential use to gauge the need to administer booster vaccine dose or S‐specific monoclonal antibodies on a preexposure basis at the individual level in highly vulnerable populations such as immunosuppressed patients. 7 , 8 Numerous immunoassays permitting quantitative assessment of SARS‐CoV‐2‐S binding antibodies have been marketed, with potential differences in the analytical design, performance characteristics, immunoglobulin class measured, and the nature of SARS‐CoV‐2 antigen to which target antibodies are directed (i.e., receptor binding domain [RBD]—the S protein in its trimeric conformation, or the S1 or S2 subunits). 9 As a result, immunoassays may return discordant qualitative or quantitative results, 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 the latter despite calibration to the first WHO SARS‐CoV‐2 antibody international standard. 19 Moreover, the correlation between antibody levels quantitated by S‐targeting antibody assays and virus neutralizing antibody titers (NtAb) may differ notably across immunoassays. 20 , 21 , 22 Against this background, using a large panel of sera we carried out a head‐to‐head comparison study assessing the performance of two widely used (electro)chemiluminescence immunoassays: the Elecsys® Anti‐SARS‐CoV‐2 S assay (Roche Diagnostics), which quantifies total antibodies against the RBD domain on the S protein, and the LIAISON® SARS‐CoV‐2 TrimericS IgG assay (DiaSorin S.p.A), which measures IgG against the S protein on its trimeric conformation according to the vaccination and SARS‐CoV‐2 infection status of participants.
2. MATERIALS AND METHODS
2.1. Sera and participants
This retrospective study included 1176 consecutive sera from 797 individuals (median age: 58 years; range: 1–100; 428 female/369 male), collected between January 2021 and April 2022. Serological determinations were performed upon physician or public health authority request. History of past SARS‐CoV‐2 infection was established after examination of the Valencian Community Microbiology registry (RedMiVa). Vaccination status of participants was obtained from the Valencian Community Vaccination Registry. According to their vaccination and SARS‐CoV‐2 infection status at the time of testing, participants were grouped into the following categories: (i) vaccinated/SARS‐CoV‐2‐experienced (Vac‐Ex); that is, patients who had completed a full vaccination schedule and had a record of a positive active infection diagnostic test (AIDT) result and/or serological evidence of past infection (presence of SARS‐CoV‐2‐Nucleocapsid [N] IgG); (ii) vaccinated/SARS‐CoV‐2‐naïve (Vac‐N), in whom no record or serological evidence of previous SARS‐CoV‐2 infection was available; (iii) non‐vaccinated/SARS‐CoV‐2‐experienced (Unvac‐Ex); (iv) non‐vaccinated/SARS‐CoV‐2‐naïve (Unvac‐N). The study was approved by the INCLIVA Research Ethics Committee (December, 2020), and informed consent was waived due to its retrospective nature.
2.2. SARS‐CoV‐2‐S‐targeting (electro)chemiluminescence immunoassays
The Roche Elecsys® anti‐SARS‐CoV‐2 S is an electrochemiluminescence sandwich immunoassay (ECLIA) that detects total antibodies (IgG/IgM/IgA) directed against the RBD of the S protein. According to the manufacturer's the quantification range is between 0.4 and 25 000 BAU/ml (1/100 dilution) and 0.8 BAU/ml is used as a cut‐off for positivity. The assay is linear up to 50 000 BAU/ml (1/200 dilution). The LIAISON® SARS‐CoV‐2 TrimericS IgG assay (DiaSorin S.p.A) is a chemiluminescence assay that detects IgG against the S protein on its trimeric conformation. The quantification range is between 4.81 and 2080 BAU/ml, and the cut‐off for positivity is 33.8 BAU/ml. According to the manufacturer's, the assay keeps linearity up to 41 600 BAU/ml (1/20 dilution). Specimens were assayed at the Microbiology Service of the Hospital Clínico Universitario of Valencia in singlets within 24 h of collection. Sera were kept at 4°C until testing.
2.3. Virus neutralization assay
NtAb targeting the S protein were measured using a Green Fluorescent protein‐expressing vesicular stomatitis virus pseudotyped with the Wuhan‐Hu‐1 variant, and Omicron BA.1 and BA.2 sublineages, following a previously published protocol, 23 , 24 with several modifications (Supporting Information material). Sera resulting in less than 50% neutralization at the lowest dilution tested were arbitrarily ascribed a reciprocal antibody titer of 10 (the limit of detection of the assay). Cryopreserved (−70°C) sera were retrieved for virus neutralization assays, which were carried out at the Institute for Integrative Systems Biology (I2SysBio), Universitat de Valencia‐CSIC.
2.4. Statistical analysis
Continuous variables are reported in medians and interquartile ranges (IQR). Medians were compared by the Mann−Whitney U test or Kruskal−Wallis test, for two or more unpaired samples and the Wilcoxon rank test for paired data, as appropriate. Frequency comparison across groups was performed using Fisher's exact test. Cohen's kappa and Spearman's rank correlations were applied to evaluate between‐assay agreement; correlation coefficient interpretation was as previously suggested. 25 Fisher‐Z‐transformation of ρvalues was conducted to statistical comparison. 26 Being a the number of specimens returning positive results by both comparison tests, b and c the number of specimens retrieving discordant (positive/negative) results by either method and d, the number of specimens testing negative by both tests, the positive percent agreement (PPA) was defined by [a/a + b] x 100 and the negative percent agreement by [d/c + d] x 100. When deemed appropriate, the term sensitivity was used referring to the rate of detection of antibodies in Vac‐Ex, Vac‐N, and Unvac‐Ex. The specificity of the respective assay was calculated taking into account sera from Unvac‐N individuals. The Bland−Altman test was used to evaluate mean differences of values measured by the two immunoassays. Receiver operating characteristic (ROC) curves were built to assess the sensitivity and specificity for every possible cut‐off value of a given immunoassay predicting NtAb detectability (≥10 reciprocal IC50). Two‐sided exact p values were reported. A p < 0.05 was considered statistically significant. The analyses and graphs were performed using SPSS version 20.0 (SPSS) and GraphPad Software Inc. v6.0, respectively.
3. RESULTS
A total of 1176 sera from 797 individuals were included in the study, of which 286 sera were collected from 204 Vac‐Ex, 581 from 316 Vac‐N, 147 from 134 Unvac‐Ex, and 162 from 143 Unvac‐N. Of relevance, a record of a positive active infection diagnostic test (Valencian Community RedMiva Registry) was available from 232 individuals. According to the dominance of SARS‐CoV‐2 variants in the Valencian Community at the time of diagnosis, the ancestral, Alpha, Delta, and Omicron variant were allegedly responsible of 13, 114, 100, and 6 cases, respectively. Median time since diagnosis to sample collection was 120 days (IQR: 30−320 days). Note that in vaccinated participants, time elapsed since receipt of last vaccine dose to collection of sera was a median of 51 days (IQR: 19.3−103.8) for SARS‐CoV‐2‐naïve, and a median of 98 days (IQR: 30.3−180.3) for SARS‐CoV‐2‐experienced. For those with a record of a positive AIDT, the time from diagnosis to serum testing was a median of 172.5 (IQR: 18.3−281.8) and 29 days (IQR: 6.3−110.5) in vaccinated and non‐vaccinated individuals, respectively.
3.1. Qualitative agreement between immunoassays
Overall, a larger number of sera returned positive results by the Roche assay compared to the DiaSorin assay (907 vs. 790; p = 0.45; ϰ: 0.69; 95% CI: 0.71−0.84), this translating into a higher overall sensitivity for detection of previous exposure or vaccination was higher for the Roche assay (89.3% vs. 77.6%). This was consistent across all study groups, except for the Unvac‐N, but only reached statistical significance for sera from Unvac‐Ex individuals (p ≤ 0.001; Table 1). Nonetheless, concordance between results provided by the two immunoassays differed across groups, being higher for sera from Vac‐N (ϰ: 0.58; IQR: 0.50−0.65) than for sera from Vac‐Ex (ϰ: 0.19; IQR: −0.14 to 0.52) or Unvac‐Ex (ϰ: 0.18; IQR: 0.06−0.30). The rate of discordant results, in most cases anti‐RBD positive/anti‐S‐trimeric negative, was higher among sera from Unvac‐Ex (34.7%) followed by Vac‐N (14.6%) and Vac‐Ex (2.7%) (Table 1 and Figure 1). The specificity of both assays was calculated only considering sera from Unvac‐N individuals, and was found to be 98.8% for the Roche assay and 98.1% for the DiaSorin assay. Interestingly, the overall sensitivity of the DiaSorin assay may increase by 11% (88.9%) by lowering the antibody threshold for positivity from 33.8 (as recommended by the manufacturer) to 4.9 BAU/ml without a major decrease in specificity (from 98.1% to 95.1%).
Table 1.
Qualitative results returned by Roche and DiaSorin immunoassays according to vaccination and SARS‐CoV‐2 infection status of participants
| Parameter | Study group | |||
|---|---|---|---|---|
| Vaccinated/SARS‐CoV‐2 experienced (n = 286) | Vaccinated/SARS‐CoV‐2 naïve (n = 581) | Unvaccinated/SARS‐CoV‐2 experienced (n = 147) | Unvaccinated/SARS‐CoV‐2 naïve (n = 162) | |
| Detectable anti‐RBD total antibodies (%) | 284 (99.3) | 487 (83.8) | 134 (91.2) | 2 (1.2) |
| Detectable anti‐S‐trimeric IgG (%) | 278 (97.2) | 422 (72.6) | 87 (59.2) | 3 (1.9) |
| p Valuea | 0.62 | 0.058 | <0.001 | 1.00 |
| Kappa index (95% CI) | 0.19 (−0.14 to 0.52) | 0.58 (0.50−0.65) | 0.18 (0.06−0.30) | ‐ |
| Positive percent agreement (PPA) | 97.0 | 83.0 | 63.0 | ‐ |
| Negative percent agreement (NPA) | 11.0 | 50.0 | 18.0 | 97.0 |
| Discordant results | ||||
| Anti‐RBD total antibodies positive/anti‐S‐trimeric IgG negative (%) | 7 (2.4) | 75 (12.9) | 49 (33.3) | 2 (1.2) |
| Anti‐RBD total antibodies negative/anti‐s‐trimeric IgG positive (%) | 1 (0.3) | 10 (1.7) | 2 (1.4) | 3 (1.9) |
Abbreviations: RBD, receptor binding domain; S, SARS‐CoV‐2 spike protein.
Frequency comparison across groups (detectable anti‐RBD total antibodies vs. detectable anti‐S‐trimeric IgG) was performed using Fisher's exact test or χ 2 test, as appropriate.
Figure 1.
Antibody levels (in BAU/ml) measured by the Roche Elecsys® anti‐SARS‐CoV‐2 S and LIAISON® SARS‐CoV‐2 TrimericS IgG assays in sera from individuals displaying different vaccination and SARS‐CoV‐2 infection status. The threshold for positivity of both assays are shown.
3.2. Quantitative agreement between immunoassays
Of the 774 sera testing positive by both serological tests, 469 returned antibody levels falling between the quantitative range of both immunoassays and were included in the analysis detailed below. The remaining 305 sera yielded antibody levels that could not be fully quantified by the DiaSorin immunoassay (n = 262) or the Roche immunoassay (n = 208), and were excluded from the analyses. Overall, antibody levels measured by the two immunoassays did not differ significantly (Table 2); nevertheless, a trend toward higher BAU/ml values measured by Roche assay for sera from Vac‐Ex was noticed; on the contrary, a trend toward higher BAU/ml measured by the DiaSorin assay was observed for sera from Vac‐N and Unvac‐Ex. The overall correlation between antibody levels quantified by both immunoassays was strong (ρ = 0.82; 95% CI: 0.78−0.84; p < 0.001), and remained strong for sera from Vac‐Ex (ρ: 0.84; 95% CI: 0.82−0.92; p < 0.001) and Vac‐N (ρ: 0.80; 95% CI: 0.78−0.84; p < 0.001), but was only moderate for sera from Unvac‐Ex (ρ: 0.66; 95% CI: 0.49−0.78; p < 0.001), this difference reaching statistical (p = 0.0048). To compare quantitative results yielded by the two immunoassays across different antibody concentration ranges, we arbitrarily grouped sera into four categories based on the DiaSorin immunoassay results: <250, 250−1000, 1001−2080, and >2080 BAU/ml. As shown in Table 3, antibody levels quantified by both immunoassays were not significantly different when <1000 BAU/ml; in contrast, for sera ≥1000 BAU/ml, the Roche assay returned significantly higher values; this was irrespective of the vaccination and SARS‐CoV‐2 infection status of participants (not shown). The Bland−Altman plot depicted in Figure 2 shows the impact of antibody concentration on the level of discordance between quantitative values measured by the two immunoassays (A, all sera; B, sera with <1000 BAU/ml; C, sera with ≥1000 BAU/ml, as quantitated by the DiaSorin assay).
Table 2.
Quantitative results returned by Roche and DiaSorin immunoassays according to vaccination and SARS‐CoV‐2 infection status of participants
| Parameter | Vaccinated/SARS‐CoV‐2 experienced (n = 94) | Vaccinated/SARS‐CoV‐2 naïve (n = 305) | Unvaccinated/SARS‐CoV‐2 experienced (n = 70) |
|---|---|---|---|
| Anti‐RBD total antibodies levels (BAU/ml) | 1868 (213.8−13 313) | 429.6 (122.6−1441) | 135.4 (53.7−530.2) |
| Anti‐S‐trimeric IgG (BAU/ml) | 1170 (366.4−2690) | 495 (180−1241) | 226.5 (80.63–586.3) |
| p Value for comparison of mediansa | 0.09 | 0.55 | 0.09 |
| Correlation between anti‐RBD total antibodies levels and anti‐S‐trimeric IgG (Rho value/p value)b | 0.88/<0.001 | 0.80/<0.001 | 0.66/<0.001 |
Abbreviations: RBD, receptor binding domain; S, SARS‐CoV‐2 spike protein.
Mann−Whitney U test.
Spearman's correlation test.
Table 3.
Quantitative results returned by the Roche and DiaSorin immunoassays according to antibody concentration
| Parameter | Anti‐S‐trimeric IgG quantitative results | |||
|---|---|---|---|---|
| <250 BAU/ml (n = 153) | 250−1000 BAU/ml (n = 159) | 1001−2080 BAU/ml (n = 94) | >2080 BAU/ml (n = 63) | |
| Anti‐RBD total antibodies in BAU/ml. Median (IQR) | 88.45 (25.09–172.7) | 467.9 (206.4–1090) | 1756 (719–3775) | 23 476 (13 940–39 904) |
| Anti‐S‐trimeric IgG in BAU/ml. Median (IQR) | 97.10 (51.25–170.5) | 534 (368–686) | 1405 (1208–1733) | 5490 (3520–9790) |
| p Value for comparison of mediansa | 0.87 | 0.13 | 0.007 | <0.001 |
| Correlation between anti‐RBD total antibodies levels and anti‐S‐trimeric IgG (Rho value/p value)b | 0.42/<0.001 | 0.34/<0.001 | 0.18/0.07 | 0.78/<0.001 |
Abbreviations: BAU, binding antibody units; IQR, interquartile range; RBD, receptor binding domain; S, SARS‐CoV‐2 spike protein.
Wilcoxon signed rank test.
Spearman's correlation test.
Figure 2.
Bland–Altman plot for the Roche Elecsys® anti‐SARS‐CoV‐2 S and LIAISON® SARS‐CoV‐2 TrimericS IgG assays for all sera included in the panel (A), or sera returning <1000 BAU/ml (B) or ≥1000 BAU/ml, as measured by the LIAISON® SARS‐CoV‐2 TrimericS IgG assay. Plots show bias in log10 BAU/ml.
3.3. Correlation between antibody levels measured by the two immunoassays and NtAb
A total of 127 sera randomly chosen among those returning positive results by both the Roche and DiaSorin assays were evaluated for their ability to neutralize vesicular stomatitis virus (VSV) pseudotyped with different S genotypes (Wuhan‐Hu‐1/Omicron BA.1 and BA.2). These sera had been collected from fully vaccinated individuals (median age: 62 years; IQR: 44−82; 62 male/65 female) who were either SARS‐CoV‐2 naïve (n = 88) or experienced (n = 39) at the time of testing. The rate of sera displaying detectable NtAb was higher against Wuhan‐Hu‐1 (89%) than against Omicron BA.1 (56%) or BA.2 (70%; Table 4). Overall, NtAb titers were highest for Wuhan‐Hu‐1 followed for Omicron BA.1 and Omicron BA.2. Striking differences were noted across Vac‐Ex and Vac‐N; a higher rate of sera with detectable NtAb and higher NtAb titers against Wuhan‐Hu‐1 and Omicron BA.1 was observed in Vac‐Ex compared to Vac‐N (p < 0.001 for both comparisons); on the contrary, the rate of detectable NtAb against Omicron BA.2 were higher in Vac‐N as compared to Vac‐Ex (p = 0.015) although antibody levels were comparable in both groups (p = 0.92) (Table 4). Overall, correlation between antibody levels measured by the Roche and DiaSorin immunoassays with NtAb titers was moderate for those against Wuhan‐Hu‐1 and Omicron BA.1, but weak for those targeting BA.2 (Table 5). Strikingly, no correlation at all was observed between antibody levels measured by either immunoassay and NtAb against Omicron BA.2 (Table 5). Correlation between NtAb against all (sub)variants was always higher for sera from Vac‐N compared to those from Vac‐Ex, irrespective of the immunoassay used; yet, the differences reached statistical significance for NtAb against Wuhan‐Hu‐1 and Omicron BA.2, but not for those against BA.1 (Supporting Information: Table 1). By building ROC curves, we next determined the antibody level thresholds as measured by the immunoassays that would optimally (maximum combined sensitivity and specificity) predict detection of NtAb against the different (sub)variants. As shown in Table 6, irrespective of whether sera were run on the Roche or DiaSorin platform, overall antibody cut‐offs associated with NtAb detectability were lowest for Wuhan‐Hu‐1 and highest (and less accurate) for Omicron BA.2. Nevertheless, lower antibody levels as measured by the Roche immunoassay compared to those quantified by the DiaSorin immunoassay were associated with detection of NtAb against Wuhan‐Hu‐1 (1.9‐fold); On the contrary, higher antibody levels as measured by the Roche immunoassay compared to those quantified by the DiaSorin immunoassay were associated with detection of NtAb against Omicron BA.1 and BA.2 (2.8‐fold). A similar analysis using only sera from Vac‐N yielded similar results (Table 7).
Table 4.
Neutralizing antibody titers against SARS.COV‐2 (sub)variants in sera testing positive by both the Roche and Diasorin immunoassays collected from vaccinated individuals
| SARS‐CoV‐2 (sub)variant | No of sera testing positive (%)a/median NtAb titer (IQR) | |
|---|---|---|
| SARS‐CoV‐2 naïve | SARS‐CoV‐2 experienced | |
| Wuhan‐Hu‐1 | 74 (84)/445 (124–1724) | 39 (100)/2122 (585–7578) |
| Omicron BA.1 | 39 (44)/20 (20–92) | 32 (82)/276 (20–798) |
| Omicron BA.2 | 38 (76)/107 (25–566) | 20 (60)/60 (25–505) |
Abbreviation: NtAb, neutralizing antibodies.
A total of 127 were assayed for quantitation of NtAb against Wuhan‐Hu‐1 and Omicron BA.1. A total of 83 sera were assayed for NtAb against Omicron BA.2.
Table 5.
Correlation between quantitative values returned by the Roche and the DiaSorin (electro)chemiluminescent immunoassays and neutralizing antibody titers against several SARS‐CoV‐2 (sub)variants in vaccinated individuals according to their SARS‐CoV‐2 infection status
| SARS‐CoV‐2 (sub)varianta | Rho value [95% CI] (p value) for the correlation between the Roche/DiaSorin immunoassays and NtAb titers | ||
|---|---|---|---|
| All participants | SARS‐CoV‐2 ‐naïve | SARS‐CoV‐2‐experienced | |
| Wuhan‐Hu‐1 | 0.69 [0.58−0.77] (<0.001)/0.71 [0.61−0.79] (<0.001) | 0.71 [0.60−0.81] (<0.001)/0.75 [0.64−0.83](<0.001) | 0.40 [0.09−0.64] (<0.001)/0.37 [0.06−0.62] (<0.001) |
| Omicron BA.1 | 0.64 [0.52−0.73] (<0.001)/0.67 [0.55−0.75](<0.001) | 0.61 [0.46−0.73] (<0.001)/0.60 [0.45−0.73] (<0.001) | 0.52 [0.24−0.72] (<0.001)/0.59 [0.33−0.77] (<0.001) |
| Omicron BA.2 | 0.30 [0.08−0.49] (<0.001)/0.24 [0.02−0.44] (<0.001) | 0.52 [0.27−0.70] (<0.001)/0.49 [0.02−0.68] (<0.001) | −0.05 [(−0.34 to 0.31] (0.78)/−0.15 [−0.48 to 0.21] (0.4) |
Abbreviation: NtAb, neutralizing antibodies.
A total of 127 were assayed for quantitation of NtAb against Wuhan‐Hu‐1 and Omicron BA.1. A total of 83 sera were assayed for NtAb against Omicron BA.2.
Table 6.
Antibody level thresholds as measured by the Roche and DiaSorin immunoassays predicting detectability of spike‐binding neutralizing antibodies according to SARS‐CoV‐2 (sub)variant
| Immunoassay | SARS‐CoV‐2 (sub)varianta | Antibody threshold (BAU/ml) | Sensitivity (%) | Specificity (%) | AUC (p value) |
|---|---|---|---|---|---|
| Roche | Wuhan‐Hu‐1 | >300 | 92.04 | 92.86 | 0.96 (<0.001) |
| Omicron BA.1 | >5257 | 88.73 | 92.86 | 0.95 (<0.001) | |
| Omicron BA.2 | >8816 | 74.32 | 77.78 | 0.74 (0.016) | |
| DiaSorin | Wuhan‐Hu‐1 | >586 | 83.19 | 92.86 | 0.96 (<0.001) |
| Omicron BA.1 | >1810 | 81.69 | 92.86 | 0.94 (<0.001) | |
| Omicron BA.2 | >3055 | 63.51 | 77.78 | 0.71 (0.039) |
Abbreviations: AUC, area under the receiver operating characteristic curve; BAU, binding antibody units.
A total of 127 were assayed for quantitation of neutralizing antibodies against Wuhan‐Hu‐1 and Omicron BA.1. A total of 83 sera were assayed for neutralizing antibodies against Omicron BA.2.
Table 7.
Antibody level thresholds as measured by the Roche and DiaSorin immunoassays predicting detectability of spike‐binding neutralizing antibodies in vaccinated, SARS‐CoV‐2 naïve participants according to SARS‐CoV‐2 (sub)variant
| Immunoassay | SARS‐CoV‐2 (sub)varianta | Antibody threshold (BAU/ml) | Sensitivity (%) | Specificity (%) | AUC (p value) |
|---|---|---|---|---|---|
| Roche | Wuhan‐Hu‐1 | >300 | 90.54 | 92.86 | 0.96 (<0.001) |
| Omicron BA.1 | >4123 | 92.31 | 91.84 | 0.95 (<0.001) | |
| Omicron BA.2 | >8851 | 69.05 | 87.50 | 0.77 (0.012) | |
| DiaSorin | Wuhan‐Hu‐1 | >586 | 77.03 | 92.86 | 0.95 (<0.001) |
| Omicron BA.1 | >1505 | 79.49 | 91.84 | 0.94 (<0.001) | |
| Omicron BA.2 | >3055 | 61.90 | 87.50 | 0.72 (0.05) |
Abbreviations: AUC, area under the receiver operating characteristic curve; BAU, binding antibody units.
A total of 88 were assayed for quantitation of neutralizing antibodies against Wuhan‐Hu‐1 and Omicron BA.1. A total of 50 sera were assayed for neutralizing antibodies against Omicron BA.2.
4. DISCUSSION
To date, international agency (i.e., FDA or ECDC) approval of quantitative SARS‐CoV‐2‐S‐targeting immunoassays extends to use for research purposes, evaluation of vaccine immunogenicity in clinical trials, and seroprevalence studies. Beyond their use for these purposes, immunoassays may be instrumental for clinical‐decision making at the individual level in the near future. For example, the European Conference on Infections in Leukemia recommends (ungraded) that oncohematological patients who have been vaccinated before or during hematological treatment should be assessed 6 months after the end of treatment and revaccinated if they have low antibody titers, although no specific definition of low antibody titers is provided. In this regard, a threshold of around 250 BAU/ml has been suggested as discriminating between those exhibiting (above) or not (below) reasonable protection against symptomatic disease (at least disease caused by the ancestral variant), in both patients with hematological disorders 27 and the general population. 6 , 28 Likewise, the Spanish Agency of Medicines and Medical Devices currently recommends Evusheld infusion for highly vulnerable individuals, preferentially when SARS‐CoV‐2 seronegativity or low anti‐S antibody levels (<260 BAU/ml) are documented. 7 In this context, head‐to‐head analytical and clinical performance comparison of commercially available S‐specific quantitative immunoassays is of key importance, even when these tests detect different antibody isotypes or incorporate different antigenic targets. Here, two widely used assays, the Roche anti‐RBD total antibody and the DiaSorin S‐trimeric IgG assay, were compared by parallel testing a large number of sera from individuals with different vaccination and SARS‐CoV‐2 infection status. We also investigated the correlation of antibody levels quantitated by the two immunoassays with NtAb titers; these were measured using a S‐pseudotyped virus neutralization assay of three different S genotypes: the ancestral variant present in approved vaccines, and the Omicron BA.1 and BA.2 S belonging to subvariants either dominant or rapidly expanding, respectively, in our setting at the time of study design.
From a qualitative standpoint, both assays correlated reasonably well (ϰ index: 0.68); nevertheless, the Roche assay returned a higher number of positive results than the DiaSorin assay, thus exhibiting a higher overall sensitivity for detection of previous virus exposure, vaccination or both. Interestingly, PPA was higher for sera from Vac‐Ex (0.97) or Vac‐N (0.83) than for sera from Unvac‐Ex (0.63). Differences in the array of epitopes within the S protein eliciting stronger antibody responses following vaccination or natural infection may account for our observation. 29 , 30 Moreover, the fact that Unvac‐Ex and Vac‐Ex differed critically in the timeframe of virus exposure in subjects (median of 29 days for Unvac‐Ex vs. 172.5 days for VacEx) may also have impacted on the results, due to the potentially varying kinetics of antibody decay over time across different antibody specificities. Notably, qualitative agreement between immunoassays was highest in sera from Vac‐N (ϰ index: 0.58). By only considering sera from Unvac‐N individuals, we found the specificity of both assays to be comparable (98.8% for the Roche assay and 98.1% for the DiaSorin assay). Importantly, the difference in the overall sensitivity between the two immunoassays may be substantially narrowed by lowering the DiaSorin assay cut‐off for positive results from 33.8 to 4.9 BAU/ml, without significantly decreasing its specificity, as in around 80% of sera testing Roche positive/DiaSorin negative, values in the latter fell within 4.9–33.8 BAU/ml. It is important to note that while the overall sensitivity of both assays calculated herein was substantially lower than that indicated by the respective manufacturer (98.8% and 98.7% for the Roche assay and the DiaSorin assay, respectively), the specificity was only slightly decreased (99.8% and 99.5% for the Roche assay and the DiaSorin assay, respectively, according to the respective manufacturer). To gauge the relevance of this apparent discrepancy it needs to be taken into account that both assays were originally evaluated by the manufacturers using sera drawn from unvaccinated individuals and shortly after SARS‐CoV‐2 infection due to the Wuhan‐Hu‐1 ancestral variant.
From a quantitative perspective, no significant differences were observed in antibody levels measured by the two immunoassays for sera belonging to any of the study groups. Nevertheless, these correlated to a lesser extent for sera from Unvac‐Ex (ρ = 0.66) than those from Vac‐Ex (ρ = 0.88) and Vac‐N (ρ = 0.80), likely due to differences in the antigenic specificity of antibodies generated following vaccination or natural infection. In agreement with our findings, previous studies reported correlation levels between the Roche and DiaSorin immunoassays ranging from 0.80 to 0.93 in panels mostly comprising sera from vaccinated/naïve individuals. 13 , 14 , 15 , 16 , 17 , 18 The sizable difference between correlation levels across Vac‐Ex and Unvac‐Ex observed in our study are also congruent with findings by Lukaszuk et al. 17 A relatively novel observation of this study relates to the impact of antibody concentration on the magnitude of quantitative discordances across the two immunoassays; in effect, for sera with <1000 BAU/ml, the DiaSorin assay returned higher values than the Roche assay, whereas the reverse was documented for sera ≥1000 BAU/ml. Although not explicitly stated, the occurrence of such differences in quantitative estimations of antibody levels provided by the Roche and DiaSorin immunoassays can be inferred from data from previous studies. 13 , 16 , 17 , 18
We next evaluated the correlation between antibody levels returned by the Roche and DiaSorin immunoassays and NtAb titers against Wuhan‐Hu‐1, Omicron BA.1, and BA.2 measured using an S‐pseudotyped virus neutralization assay. To this end, we randomly selected 127 sera collected from Vac‐Ex and Vac‐N. Given the timing of sera collection, both the rate of sera exhibiting neutralizing activity (any titer) and NtAb titers were higher against the ancestral variant than against Omicron BA.1 and BA.2. Several major observations were made regarding the degree of correlation between antibody levels measured by both immunoassays and NtAb titers. First, the correlation level was substantially lower for sera in the Vac‐Ex group compared to those in the Vac‐N, irrespective of the SARS‐CoV‐2 variant considered; this difference was strikingly evident for NtAb against Omicron BA.2 (ρ: 0.52 in Vac‐N vs. ρ: −0.05 in Vac‐Ex). Second, antibody level cut‐offs predicting NtAb detection (at any level) varied widely depending upon the SARS‐CoV‐2 (sub)variant considered and the immunoassay employed. This was also the case when sera from Vac‐N individuals were analyzed separately. Of note, we chose a binary outcome for categorization of NtAb results (detectable vs. undetectable), as no NtAb threshold has clearly been defined for protection against SARS‐CoV‐2 infection (due to any variant). In effect, Wuhan‐Hu‐1 NtAb detectability was associated with lower antibody levels as measured by both binding immunoassays than was the case for Omicron (sub)variants; this was not unexpected as both immunoassays contain the S or RBD protein as target antigens derived from the ancestral variant, and the Wuhan‐Hu‐1 variant substantially diverges antigenically from Omicron sublineages. 31 Third, RBD and S‐trimeric binding antibody levels predicting NtAb against Omicron BA.2 were substantially higher than those against Omicron BA.1 irrespective of the immunoassay used, likely reflecting the higher degree of variation of sequence within critical antibody binding sites in BA.2 S protein 31 ; moreover, prediction of Omicron BA.2 NtAb detectability was somewhat inaccurate (AUC: 0.71). Fourth, antibody level thresholds predicting detection of NtAb against all SARS‐CoV‐2 variants screened were substantially dissimilar for the Roche and DiaSorin assay. The current study has several limitations. First, as stated above, sera yielding contradictory qualitative results across immunoassays were not further analyzed to account for discrepancies, impeding an optimal specificity analysis. Second, no sera from Unvac‐Ex individuals were run on the NtAb assay. Our study has also some strengths: the large number of sera tested, the heterogeneity of participants in terms of vaccination and SARS‐CoV‐2 infection status, and the use of S‐pseudotyped virus neutralization assays instead of a surrogate NtAb method.
In summary, our data revealed that neither qualitative nor quantitative results returned by the Roche and DiaSorin immunoassays (the latter despite calibration to the first WHO SARS‐CoV‐2 antibody standard) are necessarily interchangeable. In both assays, moreover, analytical performance was found to be greatly influenced by individuals' vaccination, and SARS‐CoV‐2 infection status. Importantly, an impact of antibody concentration on the magnitude of quantitative discordances across the two immunoassays was noticed. Our data also highlight the need for continuous evaluation of S‐based immunoassays for their performance in epidemiological settings dominated by emerging (sub)variants, particularly the correlation between BAUs provided and NtAb titers. Widespread future use of indicators such as absence of S‐directed antibodies or a given antibody level threshold to determine eligibility for an additional vaccine dose (either Wuhan‐Hu‐1‐based or adapted) or S‐targeted monoclonal antibody therapies, may increase the clinical impact of between‐assay differences in sensitivity and quantitative performance. In this context, our data may contribute toward optimal immunoassay selection.
AUTHOR CONTRIBUTIONS
David Navarro and Ron Geller conceived and designed the analysis. Jorge Camacho, Joao Zulaica, Beatriz Álvarez‐Rodríguez, Luciana Rusu, and María Jesús Alcaraz performed the experiments. Jorge Camacho and Estela Giménez collected the data. Jorge Camacho, Eliseo Albert, and Beatriz Olea performed the analysis. David Navarro and Jorge Camacho wrote the final version of the manuscript.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
Supporting information
Supplementary information.
Supplementary information.
ACKNOWLEDGMENTS
Estela Giménez (Juan Rodés Contract, JR18/00053) and Eliseo Albert (Juan Rodés Contract; JR20/00011) hold contracts funded by the Carlos III Health Institute (co‐financed by the European Regional Development Fund, ERDF/FEDER). Beatriz Álvarez‐Rodríguez holds the GVA postdoctoral fellowship (APOSTD/2021/017). This study work was supported by Instituto de Salud Carlos III, Madrid, Spain (FIS, PI21/00563) to David Navarro, and by the European Commission NextGenerationEU fund (EU 2020/2094), through CSIC's Global Health Platform (PTI Salud Global) to Ron Geller. The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Camacho J, Albert E, Zulaica J, et al. A performance comparison of two (electro) chemiluminescence immunoassays for detection and quantitation of serum anti‐spike antibodies according to SARS‐CoV‐2 vaccination and infections status. J Med Virol. 2022;95:e28397. 10.1002/jmv.28397
Estela Giménez and David Navarro are senior authors who contributed equally to this work.
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1. Liu H, Wilson IA. Protective neutralizing epitopes in SARS‐CoV‐2. Immunol Rev. 2022;310(1):76‐92. 10.1111/imr.1308 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Kent SJ, Khoury DS, Reynaldi A, et al. Disentangling the relative importance of T cell responses in COVID‐19: leading actors or supporting cast? Nat Rev Immunol. 2022;22:387‐397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Castro Dopico X, Ols S, Loré K, Karlsson Hedestam GB. Immunity to SARS‐CoV‐2 induced by infection or vaccination. J Intern Med. 2022;291:32‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Cromer D, Steain M, Reynaldi A, et al. Neutralising antibody titres as predictors of protection against SARS‐CoV‐2 variants and the impact of boosting: a meta‐analysis. Lancet Microbe. 2022;3:e52‐e61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Khoury DS, Cromer D, Reynaldi A, et al. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS‐CoV‐2 infection. Nature Med. 2021;27:1205‐1211. [DOI] [PubMed] [Google Scholar]
- 6. Gilbert PB, Montefiori DC, McDermott AB, et al. Immune correlates analysis of the mRNA‐1273 COVID‐19 vaccine efficacy clinical trial. Science. 2022;375:43‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Recomendaciones de utilización de Evusheld para la prevención de Covid‐19. https://www.sanidad.gob.es/profesionales/saludPublica/prevPromocion/vacunaciones/covid19/docs/Recomendaciones_uso_Evusheld.pdf
- 8. Cesaro S, Ljungman P, Mikulska M, et al. Recommendations for the management of COVID‐19 in patients with haematological malignancies or haematopoietic cell transplantation, from the 2021 European Conference on infections in leukaemia (ECIL 9). Leukemia. 2022;36:1467‐1480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Misra A, Theel ES. Immunity to SARS‐CoV‐2: what do we know and should we be testing for it? J Clin Microbiol. 2022;60(60):e0048221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Higgins V, Fabros A, Kulasingam V. Quantitative measurement of anti‐SARS‐CoV‐2 antibodies: analytical and clinical evaluation. J Clin Microbiol. 2021;59:e03149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Theel ES, Johnson PW, Kunze KL, et al. SARS‐CoV‐2 serologic assays dependent on dual‐antigen binding demonstrate diverging kinetics relative to other antibody detection methods. J Clin Microbiol. 2021;59:e0123121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Saker K, Escuret V, Pitiot V, et al. Evaluation of commercial anti‐SARS‐CoV‐2 antibody assays and comparison of standardized titers in vaccinated healthcare workers. J Clin Microbiol. 2021;60:e017462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Swadźba J, Anyszek T, Panek A, Chojęta A, Wyrzykowska K, Martin E. Head‐to‐head comparison of 5 anti‐SARS‐CoV‐2 assays performance in one hundred COVID‐19 vaccinees, over an 8‐month course. Diagnostics. 2022;12:1426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Danese E, Montagnana M, Salvagno G, et al. Comparison of five commercial anti‐SARS‐ CoV‐2 total antibodies and IgG immunoassays after vaccination with BNT162b2 mRNA. J Med Biochem. 2021;40:335‐340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Salvagno GL, Lippi G. Correlation between anti‐SARS‐CoV‐2 total antibodies and spike trimeric IgG after BNT162b2 booster immunization. Vaccines. 2022;10:890. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Perkmann T, Perkmann‐Nagele N, Koller T, et al. Anti‐spike protein assays to determine SARS‐CoV‐2 antibody levels: a head‐to‐head comparison of five quantitative assays. Microbiol Spectrum. 2021;9:e0024721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Lukaszuk K, Kiewisz J, Rozanska K, et al. Usefulness of IVD kits for the assessment of SARS‐CoV‐2 antibodies to evaluate the humoral response to vaccination. Vaccines. 2021;9:840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Carta M, Marinello I, Cappelletti A, et al. Comparison of anti‐SARS‐CoV‐2 S1 receptor‐binding domain antibody immunoassays in health care workers before and after the BNT162b2 mRNA vaccine. Am J Clin Path. 2022;157:212‐218. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Mattiuzzo G, Bentley EM, Hassall M, et al. Openshaw PJMKBB. Establishment of the WHO International Standard and Reference Panel for anti‐SARS‐CoV‐2 antibody. World Health Organization; 2020:1‐60. [Google Scholar]
- 20. Valdivia A, Torres I, Latorre V, et al. Inference of SARS‐ CoV‐2 spike‐binding neutralizing antibody titers in sera from hospitalized COVID‐19 patients by using commercial enzyme and chemiluminescent immunoassays. Eur J Clin Microbiol Infect Dis. 2021;40:485‐494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Tang MS, Case JB, Franks CE, et al. Association between SARS‐CoV‐2 neutralizing antibodies and commercial serological assays. Clin Chem. 2020;66:1538‐1547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Jung K, Shin S, Nam M, et al. Performance evaluation of three automated quantitative immunoassays and their correlation with a surrogate virus neutralization test in coronavirus disease 19 patients and pre‐pandemic controls. J Clin Lab Anal. 2021;35:e23921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Sánchez‐Sendra B, Albert E, Zulaica J, et al. Neutralizing antibodies against SARS‐CoV‐2 variants of concern elicited by the comirnaty COVID‐19 vaccine in nursing home residents. Sci Rep. 2022;12:3788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Giménez E, Albert E, Zulaica J, et al. Severe acute respiratory syndrome coronavirus 2 adaptive immunity in nursing home residents following a third dose of the comirnaty coronavirus disease 2019 vaccine. Clin Infect Dis. 2022;75:e865‐e868. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Schober P, Boer C, Schwarte LA. Correlation coefficients: appropriate use and interpretation. Anesth Analg. 2018;126:1763‐1768. [DOI] [PubMed] [Google Scholar]
- 26. Myers L, Sirois MJ. Spearman correlation coefficients, differences between. In: Kotz S, Read CB, Balakrishnan N, Vidakovic B, Johnson NL, eds. Encyclopedia of Statistical Sciences. John Wiley & Sons, Inc; 2006. [Google Scholar]
- 27. Piñana JL, López‐Corral L, Martino R, et al. SARS‐CoV‐2 vaccine response and rate of breakthrough infection in patients with hematological disorders. J Hematol Oncol. 2022;15:54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Feng S, Phillips DJ, White T, et al. Correlates of protection against symptomatic and asymptomatic SARS‐CoV‐2 infection. Nature Med. 2021;27:2032‐2040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Greaney AJ, Loes AN, Gentles LE, et al. Antibodies elicited by mRNA‐1273 vaccination bind more broadly to the receptor binding domain than do those from SARS‐CoV‐2 infection. Sci Transl Med. 2021;13:eabi9915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Garrett ME, Galloway JG, Wolf C, et al. Comprehensive characterization of the antibody responses to SARS‐CoV‐2 spike protein finds additional vaccine‐induced epitopes beyond those for mild infection. eLife. 2022;11:e73490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Shrestha LB, Foster C, Rawlinson W, Tedla N, Bull RA. Evolution of the SARS‐CoV‐2 omicron variants BA.1 to BA.5: implications for immune escape and transmission. Rev Med Virol. 2022;32:e238. 10.1002/rmv.2381 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary information.
Supplementary information.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.