Abstract
Background
Many commercial assays, of different designs, detecting SARS-CoV-2-specific antibodies exist but with little experience with them.
Objectives
The aim of this study was to compare the performance of assays detecting IgG or total antibodies to N or S antigens, validated for routine use in France, with samples from subjects with more or less severe SARS-CoV-2 infection.
Methods
Eight assays were used: Abbott Architect, DiaSorin Liaison®, bioMérieux Vidas®, Roche Elecsys Cobas®, Siemens Atellica®, BioRad Platelia ELISA, Epitope Diagnostics ELISA, and Wantai ELISA. The tested population included 86 samples from 40 hospitalized subjects and 28 outpatients at different time from symptom onset.
Results
The positivity rate varied depending on the assay but was greater for all assays in hospitalized than non-hospitalized patients. Despite a good correlation between the assays, discrepancies occurred, without a systematic origin, even for samples taken more than 20 days after symptom onset. These discrepancies were linked to low antibody levels in pauci-symptomatic patients.
Conclusion
Whichever assay is chosen, a false negative result may need to be ruled out with another test in a risk situation.
Abbreviations: HCW, health care workers; SO, symptom onset; RBD, receptor binding domain; CLIA, chemiluminescent immunoassay; ELFA, enzyme linked fluorescent assay; CMIA, chemiluminescent microparticule immunoassay; ECLIA, electrochemiluminescent immunoassay; ELISA, enzyme linked immunoassay
Keywords: SARS-CoV-2, COVID-19, IgG, Total Ig, Immunoassay, Serology
1. Background
The SARS-CoV-2 (Severe acute respiratory syndrome coronavirus-2) responsible for COVID-19 (Coronavirus infectious disease 2019) emerged in December 2019 in Wuhan, China. The members of Coronaviridae family are enveloped positive RNA viruses; the 5’ part of the genome is encoding for a polyprotein cleaved in non-structural proteins forming the transcription and replication complex. The 3’ part of the genome in encoding for the structural proteins, spike (S), envelope (E), membrane (M) and the nucleoprotein (N). Serological assays target mainly the S and/or the N proteins which are considered as the target of neutralizing antibodies and immunodominant proteins, respectively.
While SARS-CoV-2 genome detection in respiratory specimens remain the only way to confirm current infection, serology might be useful for epidemiological studies and past SARS-CoV-2 infection determination in the absence of molecular diagnosis. Most of the companies producing immunodiagnostic assays have rapidly produced tests detecting SARS-CoV-2-specific IgG, IgM, IgA or total antibodies. Some of these assays have been evaluated by independent groups [[1], [2], [3]] but their comparison on the same samples has rarely been performed [[4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]] or only for a small number of commercial assays, with the exception of point-of-care systems.
2. Objective
The objective of this paper was to perform a performance comparison of eight commercial assays of which five use automated instruments with CLIA or ELFA technology (Abbott Architect, DiaSorin Liaison®, bioMérieux Vidas®, Roche Elecsys Cobas®, and Siemens Atellica®), and three are microplate ELISA (BioRad Platelia, Epitope Diagnostics EDI™, Wantai).
3. Materials and methods
The characteristics of each assay are presented in Table 1 . For comparison, only the assays detecting IgG or total antibodies are presented. Half of them are directed against the N protein (Abbott Architect, Roche Elecsys Cobas®, Biorad Platelia, Epitope Diagnostics) and the other half against parts of the S protein (S1 + S2: Diasorin Liaison®, S1: biomérieux Vidas® or the receptor binding domain (RBD): Siemens Atellica®, Wantai). All are CE marked and their sensitivity and specificity were evaluated by the manufacturer (Table 1). Antibody detection kits were used according to manufacturers' instructions.
Table 1.
Product description of the compared automated commercial antibody detection assays.
| Manufacturer assay name | DiaSorin Liaison® | bioMérieux Vidas® | Siemens Atellica® | Wantai | Abbott Architect | Roche Elecsys | BioRad Platelia | Epitope Diagnostics EDI™ |
|---|---|---|---|---|---|---|---|---|
| Antibody detected | IgG | IgG | Total Ig | Total Ig | IgG | Total Ig | Total Ig | IgG |
| recombinant labeled protein | S1 + S2 | S1 + peptide | RBD | RBD | N | N | N | N |
| methodology | indirect CLIA (2 steps) | Sandwich ELFA (2 steps) | Sandwich CLIA (1 step) | Sandwich ELISA | indirect CMIA (2 steps) | Sandwich ECLIA (2 steps) | Sandwich ELISA (1 step) | Indirect ELISA (2 steps) |
| specimen type | serum/plasma | serum/plasma | serum/plasma | serum/plasma | serum/plasma | serum/plasma | serum/plasma | serum/plasma |
| sample volume | 20 μL | 100 μL | 50 μL | 100 μL | 25 μL | 20 μL | 10 μL | 1 μL |
| instrument used | Liaison® XL | Vidas® | Atellica® | manual or automated ELISA instrument | Architect | Cobas® | manual or automated ELISA instrument | manual or automated ELISA instrument |
| turnaround time | 35 min | 27 min | 15 min | 1.5 h | 29 min | 18 min | 2 h | 2 h |
| cutoff calculation basis | calibrator | standard | calibrator | negative control (min 0.19) | calibrator | calibrator | cut-off control (mean) | negative control (mean+0.18) |
| threshold | AU/mL; 12 | ratio; 1 | ratio; 1 | ratio; 1.1 | ratio; 1.4 | ratio; 1 | ratio; 1 | ≥ 1,1x (NC + 0.18) |
| grey zone | >12 - <15 | ND | ND | ≥0.9 - ≤1.1 | ND | ND | ≥ 0.8 - < 1 | ≥ 0.9x (NC + 0.18) < 1.1x (NC + 0.18) |
| measuring interval | 3.8 - 400 | upper limit: 10 | upper limit 3.5 | |||||
| reported sensitivity: days from symptoms (number): % | ≤ 5 (44): 25.0 % 5–15 (52): 90.4% ≥16 (39): 97.4% |
≤7 (117): 45.3 % 8–15 (44): 88.6% ≥16 (29): 96.6% |
<7 (89): 60.7 % 7–13 (116): 97.5% ≥14 (42): 100% |
94.5 % | <3 (5): 0 % 3–7 (10): 50% 8–13 (34): 91.2% ≥14 (73): 100% |
<7 (116): 65.5 % 7–13 (59): 88.1% ≥14 (29): 100% |
≤8 (8): 73 % >8 (39): 97.4 % |
100 % |
| reported specificity | 98.5 % | 100 % | 99.8 % | 100 % | 99.6 % | 99.8 % | 99.6 % | 100 % |
RBD: Receptor Binding Domain, CLIA: ChemiLuminescence ImmunoAssay, ELFA: Enzyme Linked Fluorescent Assay, CMIA: Chemiluminescence Microparticule ImmunoAssay, ECLIA: ElectroChemiLuminescence ImmunoAssay, ND: not determined.
Characteristics of the commercial antibody detection assays classified according to the targeted antigen and the class of antibodies detected. The technical details of the assay as well of the sensitivity and specificity data reported by the manufacturer are listed.
In the present study 82 residual serum or plasma samples from a population of 68 SARS-CoV-2 infected patients, confirmed by a positive RT-PCR (reverse transcriptase polymerase chain reaction), were used.
4. Results
The patient population included 40 infected hospitalized patients (including 25 in intensive care units) and 28 non hospitalized infected health care workers (HCW) (including physicians, nurses, and lab staff). Age of the patients ranged from 7 to 81 years (median = 51). Median age of hospitalized patients was greater (64 years, range: 7–81) than that of HCW (36 years, range: 25–59). In the HCW group, women proportion (22/28; 78.6 %) was higher than in the hospitalized group (11/40; 27.5 %). Seven patients were tested at different time points (ranging from 2 to 5). The delays between the dates of samples used for serology testing, and the beginning of documented symptoms varied from 4 to 52 days (37 ≤ 20 days, 45 > 20 days). The data were compared between 3 groups of delays from symptom onset (SO) : ≤ 15 days (16 samples), 16–20 days (21 samples), > 20 days (45 samples). The results of the eight assays, grouped according to the delay from SO and the patient population (hospitalized patients versus HCW), are shown in Fig. 1 . The positivity rate is detailed in Table 2 . Regardless of the assay, antibody reactivity was higher in hospitalized patients compared to non-hospitalized HCW. The positivity rate increased with time from SO but did not reach 100 % in this population of HCW, probably because antibody titer declined relatively rapidly in patients with mild disease and weak antibody response. Of the 45 samples taken in 40 patients more than 20 days after SO, 12 (26.7 %) were negative for antibody detection in at least one assay. Eleven of these 12 samples were from HCW. A low reactivity of positive results was observed in most cases. Six were from two HCW with four and five follow-up samples, respectively. In these two HCW, antibody kinetics either increased just above the threshold, or increased then decreased, or never reached the threshold, depending on the assay. In a single case of a hospitalized person with a sample taken more than 20 days after SO, negative results were obtained with two assays (Abbott Architect and BioRad Platelia) while all other assays gave positive results. Whatever the delay after SO, discrepancies between assays occurred (29.3 % of the samples) without being able to be systematically attributed to the sensitivity, to the target antigen (N or S) or to the class of antibodies detected (IgG or total). These discrepancies occurred mainly in samples from HCW (19/24, 79 %). Despite this, concordance between the tests was quite good and similar for all of them, with no clear distinction depending on whether the tests being compared detected the same or another viral target (Table 3 ). In addition, there was no better agreement between the tests detecting total antibodies (Siemens Atellica®, Wantai, Roche Elecsys Cobas®, BioRad Platelia) than between these tests and those detecting only IgG.
Fig. 1.
Distribution of antibody detection values for the different assays according to the patient population and delay from symptom onset.
Each dot represents the value obtained with each serological assay. The assays’ results are presented according to the targeted antigens : S1 or S1+peptide ((A) Diasorin Liaison®, (B) bioMérieux Vidas®); Receptor Binding Domain ((C) Siemens Atellica®, (D) Wantai); or N protein ((E) Abbott Architect, (F) Roche Elecsys Cobas®, (G) Biorad Platelia, (H) Epitope Diagnostics EDI™). Results are expressed as a ratio of the sample signal to cut-off for all tests except the EDI™ assay which is expressed in optical density and the Diasorin Liaison® allowing the quantification of the antibodies in arbitrary units (AU/mL). Detection limit for each test is shown with a discontinued line. Black or red dots are used for samples from health care workers (HCW) or hospitalized (Hosp) patients respectively.
Table 2.
Positivity rate of each assay according to the delay from onset of symptoms.
| Days from symptom onset (N) |
DiaSorin Liaison® |
bioMérieux Vidas® |
Siemens Atellica® |
Wantai | Abbott Architect |
Roche Elecsys |
BioRad Platelia |
Epitope Diagnostics EDI™ |
|---|---|---|---|---|---|---|---|---|
| ≤ 15 (16) | 56.3 % | 81.3 % | 73.3 % | 100 % | 62.5 % | 68.8 % | 80 % | 81.3 % |
| 16 – 20 (21) | 71.4 % | 90.5 % | 81% | 95 % | 95.2 % | 85.7 % | 90.5 % | 100 % |
| > 20 (45) | 77.8 % | 90.7 % | 88.6 % | 97.6 % | 95.6 % | 88.9 % | 86.7 % | 95.3 % |
Percentage of antibody detection was expressed for each assay according to the delay expressed in days from symptom onset.
Table 3.
Concordance between each assay.
| DiaSorin Liaison® | bioMérieux Vidas® | Siemens Atellica® | Wantai | Abbott Architect | Roche Elecsys | BioRad Platelia | Epitope Diagnostics EDI™ | |
|---|---|---|---|---|---|---|---|---|
| DiaSorin Liaison® | 100 % | 80 % (80) | 82.5 % (80) | 72.7 % (77) | 78.05 % (82) | 80.5 % (82) | 81.5 % (81) | 74.7 % (79) |
| bioMérieux Vidas® | 80 % (80) | 100 % | 94.9 % (78) | 90.7 % (75) | 90 % (80) | 87.5 % (80) | 87.3 % (79) | 89.6 % (77) |
| Siemens Atellica® | 82.5 % (80) | 94.9 % (78) | 100 % | 85.3 % (75) | 86.1 % (79) | 87.3 % (79) | 82.5 % (80) | 84.2 % (76) |
| Wantai | 72.7 % (77) | 90.7 % (75) | 85.3 % (75) | 100 % | 90.9 % (77) | 87 % (77) | 89.5 % (76) | 93.5 % (77) |
| Abbott Architect | 78.05 % (82) | 90 % (80) | 86.1 % (79) | 90.9 % (77) | 100 % | 92.7 % (82) | 88.9 % (81) | 89.9 % (79) |
| Roche Elecsys | 80.5 % (82) | 87.5 % (80) | 87.3 % (79) | 87 % (77) | 92.7 % (82) | 100 % | 90.1 % (81) | 84.8 % (79) |
| BioRad Platelia | 81.5 % (81) | 87.3 % (79) | 82.5 % (80) | 89.5 % (76) | 88.9 % (81) | 90.1 % (81) | 100 % | 87.2 % (78) |
| Epitope Diagnostics EDI™ | 74.7 % (79) | 89.6 % (77) | 84.2 % (76) | 93.5 % (77) | 89.9 % (79) | 84.8 % (79) | 87.2 % (78) | 100 % |
We compared assays by pairs. Considering only the samples tested with both assays (N), we considered as concordant the samples for which antibodies were detected positive or detected negative by both assays; when one assay gave a positive value whereas the other gave a negative value, the sample result was considered as discordant between the assays. Concordance was expressed as a percentage of samples detected positive or negative by the two considered assays.
Data are repeated for horizontal or vertical reading.
5. Discussion
In the present study all assays showed similar overall performance although the most and least sensitive tended to be the ELISA from Wantai and the Diasorin Liaison® assay, respectively, as confirmed by previous reports with different patient populations [4,6,15]. For the other assays the differences were not sufficiently consistent to claim that one had a better sensitivity than the others. Comparison of the same samples with these different commercial antibody detection systems showed that samples from SARS-Cov-2 infected patients could be negative for antibody detection by any of them. Antibodies undetectability could be linked to patient characteristics such as variable kinetics of antibody production against each viral target, strength of antibody response, antibody affinity, or assay characteristics such as the antigen nature and preparation, or duration of antigen-antibody incubation. Each of observed discrepancies could have had a different origin leading to no obvious and consistent interpretation. Discrepant results occurred mainly in patients with mild or asymptomatic infection, as antibody response might be weak and decreasing over time. Such dissociated responses between assays have already been described by others [4,8,11].
Thus, in absence of molecular diagnosis, but with suggestive clinical features, a negative antibody result may need to be evaluated with another assay to ensure that it was not a false negative result.
One limitation of our study was the small size and heterogeneous sample population, limiting determination of a true sensitivity for each assay. Specificity of these assays was not addressed in this report but has been evaluated in other studies [[4], [5], [6],8,9,13]. Even if some cross reactivity was observed, it was low, except with SARS-CoV virus, and often with different samples with each test compared.
Despite good overall performance, commercial assays need to be further evaluated with longitudinal samples from well characterized patients. In the first published studies, data came from severe patients but serology would be more useful in individuals with mild illness. Discrepancies between assays occurring mainly in this patient category, they should be the target of future studies aimed at correlating the data with the kinetics of N and S-specific antibodies, as well as their neutralizing capacity.
Credit author statement
MAT selected and collected the samples. MAT and VE interpreted the data and wrote the manuscript.
VI contributed to the biobank collection, and MPM to the technical organization. They both helped at interpreting the data and contributed to the final version of the manuscript.
AB and BL contributed to the finalization of the manuscript.
This work did not receive any specific grant from funding agencies, in the public, commercial or not-for-profit sectors. The assay kits were kindly provided by the manufacturers.
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgments
We would like to thank all the technicians from the laboratory to make this evaluation possible by performing all the antibody assays, and especially, Laure Zanghellini, Geraldine Berthot, Catherine Girardi, Majdeline Lachgar, Christine Garrigou, and Marc Carteron. We also thank Khadija Sfouli for her precious help at the working organization.
References
- 1.Bryan A., Pepper G., Wener M.H., Fink S.L., Morishima C., Chaudhary A., et al. Performance characteristics of the Abbott Architect SARS-CoV-2 IgG assay and seroprevalence in Boise, Idaho. J. Clin. Microbiol. 2020;58(8) doi: 10.1128/JCM.00941-20. 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Tré-Hardy M., Wilmet A., Beukinga I., Dogné J.-M., Douxfils J., Blairon L. Validation of a chemiluminescent assay for specific SARS-CoV-2 antibody. Clin. Chem. Lab. Med. 2020;58(8):1357–1364. doi: 10.1515/cclm-2020-0594. [DOI] [PubMed] [Google Scholar]
- 3.Tang M.S., Hock K.G., Logsdon N.M., Hayes J.E., Gronowski A.M., Anderson N.W., et al. Clinical performance of the Roche sars-cov-2 serologic assay. Clin. Chem. 2020;66(8):1107–1109. doi: 10.1093/clinchem/hvaa132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Charlton C.L., Kanji J.N., Johal K., Bailey A., Plitt S.S., MacDonald C., et al. Evaluation of six commercial mid to high volume antibody and six point of care lateral flow assays for detection of SARS-CoV-2 antibodies. J. Clin. Microbiol. 2020 doi: 10.1128/JCM.01361-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Egger M., Bundschuh C., Wiesinger K., Gabriel C., Clodi M., Mueller T., et al. Comparison of the Elecsys® Anti-SARS-CoV-2 immunoassay with the EDITM enzyme linked immunosorbent assays for the detection of SARS-CoV-2 antibodies in human plasma. Clin. Chim. Acta. 2020;509:18–21. doi: 10.1016/j.cca.2020.05.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.GeurtsvanKessel C.H., Okba N.M.A., Igloi Z., Bogers S., Embregts C.W.E., Laksono B.M., et al. An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment. Nat. Commun. 2020;11(1):3436. doi: 10.1038/s41467-020-17317-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jääskeläinen A.J., Kuivanen S., Kekäläinen E., Ahava M.J., Loginov R., Kallio-Kokko H., et al. Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation. J. Clin. Virol. 2020;129:104512. doi: 10.1016/j.jcv.2020.104512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kohmer N., Westhaus S., Rühl C., Ciesek S., Rabenau H.F. Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays. J. Clin. Virol. 2020;129 doi: 10.1016/j.jcv.2020.104480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lassaunière R., Frische A., Harboe Z.B., Nielsen A.C., Fomsgaard A., Krogfelt K.A., et al. Evaluation of nine commercial SARS-CoV-2 immunoassays. Infect. Dis. (except HIV/AIDS) 2020 doi: 10.1101/2020.04.09.20056325. [DOI] [Google Scholar]
- 10.Nguyen N.N., Mutnal M.B., Gomez R.R., Pham H.N., Nguyen L.T., Koss W., et al. Correlation of ELISA based with random access serologic immunoassays for identifying adaptive immune response to SARS-CoV-2. Pathology. 2020 doi: 10.1101/2020.07.06.20145938. [DOI] [Google Scholar]
- 11.Schnurra C., Reiners N., Biemann R., Kaiser T., Trawinski H., Jassoy C. Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests. J. Clin. Virol. 2020;129 doi: 10.1016/j.jcv.2020.104544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tang M.S., Case J.B., Franks C.E., Chen R.E., Anderson N.W., Henderson J.P., et al. Association between SARS-CoV-2 neutralizing antibodies and commercial serological assays. Immunology. 2020 doi: 10.1101/2020.07.01.182220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Theel E.S., Harring J., Hilgart H., Granger D. Performance characteristics of four high-throughput immunoassays for detection of IgG antibodies against SARS-CoV-2. J. Clin. Microbiol. 2020 doi: 10.1128/JCM.01243-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Traugott M., Aberle S.W., Aberle J.H., Griebler H., Karolyi M., Pawelka E., et al. Performance of severe acute respiratory syndrome coronavirus 2 antibody assays in different stages of infection: comparison of commercial enzyme-linked immunosorbent assays and rapid tests. J. Infect. Dis. 2020;222(3):362–366. doi: 10.1093/infdis/jiaa305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Weidner L., Gänsdorfer S., Unterweger S., Weseslindtner L., Drexler C., Farcet M., et al. Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays. J. Clin. Virol. 2020;129 doi: 10.1016/j.jcv.2020.104540. [DOI] [PMC free article] [PubMed] [Google Scholar]