Abstract
Objectives
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serology tests are clinically useful to document prior SARS-CoV-2 infections. Data are urgently needed to select assays with optimal sensitivity at acceptable specificity for antibody detection.
Methods
A comparative evaluation was performed of 7 commercial SARS-CoV-2 serology assays on 171 sera from 135 subjects with polymerase chain reaction–confirmed SARS-CoV-2 infection (71 hospitalized patients and 64 paucisymptomatic individuals). Kinetics of IgA/IgM/IgG seroconversion to viral N and S protein epitopes were studied from 0 to 54 days after onset of symptoms. Cross-reactivity was verified on 57 prepandemic samples.
Results
Wantai SARS-COV-2 Ab ELISA and Orient Gene COVID-19 IgG/IgM Rapid Test showed superior overall sensitivity for detection of SARS-CoV-2 antibodies. Elecsys Anti-SARS-CoV-2 assay and EUROIMMUN Anti-SARS-CoV-2 combined IgG/IgA showed acceptable sensitivity (>95%) vs the consensus result of all assays from 10 days post onset of symptoms. Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, and Innovita 2019-nCoV Ab rapid test showed least cross-reactivity, resulting in an optimal analytical specificity greater than 98%.
Conclusions
Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assays are suitable for sensitive and specific detection of SARS-CoV-2 antibodies from 10 days after onset of symptoms.
Keywords: SARS-CoV-2, COVID-19, Serology, Antibody, Paucisymptomatic, Coronavirus
Key Points.
• The Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assays to N protein are suitable for sensitive and specific screening of a SARS-CoV-2 infection from 10 days after symptom onset.
• SARS-CoV-2 serology tests integrating various antibody isotypes show a higher sensitivity than assays measuring only IgG.
• No clear differences are seen the in the seroconversion kinetics of antibodies targeting SARS-CoV-2 S and N protein epitopes.
Coronavirus disease 2019 (COVID-19) is an infectious disease caused by a newly discovered coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Severe cases show excessive activity of proinflammatory immune cells, causing acute respiratory distress syndrome, septic shock, and bleeding and coagulation dysfunction.1-3 Due to high human-to-human transmissibility, COVID-19 rapidly became a pandemic threat in which clinical laboratory testing from diagnosis and treatment to epidemiologic surveillances are indispensable.
The gold standard for diagnosis of COVID-19 lung disease is nucleic acid amplification testing of SARS-CoV-2 virus-specific sequences coding for spike (S), envelope (E), and nucleocapsid (N) proteins; RNA dependent RNA polymerase (RdRP) gene; and open reading frame 1ab (ORF1ab) region.4 Diagnostic sensitivity of the most commonly used technique, reverse transcription polymerase chain reaction (PCR), on nasopharyngeal swabs is currently unknown. When compared to chest computed tomography analysis of lesions characteristic for viral pneumonia, estimates vary from lower than 70% to 90%,5,6 likely depending on COVID-19 disease stage, intensity of viral replication, sampling quality, and analytical properties of the amplification assay. In addition, insufficient PCR capacity during peak infection rate in overwhelmed health care systems left many patients with milder clinically suspected SARS-CoV-2 infections as well as asymptomatic infections untested.
Serology testing for COVID-19, comprising detection of IgM, IgA, or IgG antibodies to SARS-CoV-2–specific epitopes, might represent an interesting tool to document past SARS-CoV-2 infections, both in individual patients with suspected COVID-19 symptoms or late-stage complications who had no (conclusive) PCR test and at a population level to guide infection control policies. In addition, measuring SARS-CoV-2 antibodies might harbor prognostic value and convey information on protective immunity in vaccination trials.
SARS-CoV-2 shares a 80% overall nucleotide homology with SARS-CoV.1,7,8 In SARS-CoV, the S and N proteins contain the highest density of B-cell epitopes,9,10 and in silico analysis indicated that dominant B-cell epitopes share 69% to 100% homology to SARS-CoV-2. It was therefore a logical choice of many commercial developers of SARS-CoV-2 serology kits to target S and N proteins. Antibodies against S protein, composed of a S1 subunit with the receptor-binding domain (RBD) and a S2 subunit that mediates membrane fusion for viral entry, appear additionally interesting because of their proposed correlation with neutralizing antibodies and protective immunity to both SARS-CoV8 and, based on emerging data, also to SARS-CoV-2.11,12
Data on kinetics of humoral immune responses to SARS-CoV-2 are rapidly emerging, but relative sensitivity for detection of antibodies of various commercial assays is questioned. In our study, a cross-platform comparison of 7 commercially available SARS-CoV-2 serology assays was conducted. N and S protein epitopes and different combinations of antibody isotypes in PCR-confirmed COVID-19 patients, with critical and mild disease course at various time points, were targeted. Acceptable performance was defined as minimal sensitivity of 95% for detection of SARS-CoV-2 antibodies vs a consensus estimate and minimal cross-reactivity, defined as analytical specificity of 98%.
Materials and Methods
Patients
Serum samples were obtained from the following cohorts: (1) hospitalized COVID-19 patients—105 serum samples obtained at different time points from 71 patients with PCR-confirmed SARS-CoV-2 infection and admitted for severe COVID-19 pneumonia from March 1 to April 27, 2020, at our tertiary AZ Delta General Hospital in Roeselare, Belgium; and (2) patients with paucisymptomatic SARS-CoV-2 infections—66 serum samples from 64 health care workers with a SARS-CoV-2 infection, PCR-confirmed after developing fever and World Health Organization (WHO)-listed COVID-19 symptoms. These patients were home-quarantined without need for hospitalization. The study was approved by the AZ Delta ethical committee with a waiver of informed consent from the hospitalized COVID-19 patients and with written informed consent from participants with paucisymptomatic SARS-CoV-2 infections.
Cross-reactivity was evaluated on a panel composed of 57 prepandemic serum samples obtained from patients with PCR-confirmed infection by other HCoV respiratory viruses (HCoV 229E, n = 1; HCoV HKU1, n = 3; HCoV OC43, n = 2; HCoV OC43 + adenovirus, n = 1), other pathogens and viruses (n=42), or presence of autoimmune antibodies (n = 8). Serum samples from patients with other HCoV infections ranged from 0 to 39 days after PCR positivity Table 1.
Table 1.
Prepandemic Cross-Reactivity Panel
| Sample Source | Positivity to | No. of Samples Tested | Days After PCR Positivity | S-RBD Total Ab (Wantai) | S1 IgA (EI) | S1 IgG (EI) | S1/S2 IgG (DiaSorin) | N Total Ab (Roche) | N IgG (EI) | N/S IgM (Orient Gene) | N/S IgG (Orient Gene) | IgM (Innovita) | IgG (Innovita) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HCoV respiratory infections | HCoV HKU1 | 3 | 0 28 29 |
0/1 0/1 0/1 |
0/1 0/1 1/1 |
0/1 0/1 0/1 |
0/1 0/1 0/1 |
0/1 0/1 0/1 |
0/1 0/1 0/1 |
0/1 0/1 0/1 |
0/1 0/1 1/1 |
0/1 0/1 0/1 |
0/1 0/1 0/1 |
| HCoV 229E | 1 | 0 | 0/1 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | 0/0 | |
| HCoV OC43 | 2 | 19 39 |
0/1 0/1 |
0/1 0/1 |
0/1 0/1 |
0/1 0/1 |
0/1 0/1 |
0/1 0/1 |
0/1 1/1 |
0/1 0/1 |
0/1 0/1 |
0/1 0/1 |
|
| HCoV OC43 (+ adenovirus) | 1 | 0 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | 0/1 | |
| Non-CoV samples | Epstein-Barr virus IgM | 3 | 0/3 | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | |
| Cytomegalovirus IgM | 5 | 0/5 | 1/5 | 0/5 | 0/5 | 0/5 | 1/5 | 0/5 | 0/5 | 0/5 | 0/5 | ||
| Hepatitis B virus (HBsAg) | 5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | ||
| Hepatitis C virus | 5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 1/5 | 0/5 | 0/5 | ||
| Human immunodeficiency viruses | 5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 1/5 | 0/5 | 0/5 | ||
| Rubella virus | 3 | 0/3 | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | ||
| Borrelia burgdorferi | 4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | ||
| Mycoplasma pneumoniae | 4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | ||
| Toxoplasma gondii | 4 | 0/4 | 1/4 | 0/4 | 1/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | ||
| Treponema pallidum | 4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | 0/4 | ||
| Anti–nuclear factor | 5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | 0/5 | ||
| Rheumatoid factor | 3 | 0/3 | 0/3 | 0/3 | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | ||
| Total | 57 | 0/57 | 5/56 | 0/56 | 2/56 | 0/56 | 1/56 | 1/56 | 3/56 | 0/56 | 0/56 |
Ab, antibody; EI, EUROIMMUN; PCR, polymerase chain reaction.
SARS-CoV-2 Serology Assays
All serology assays were used according to manufacturers’ protocol using cutoffs specified in package inserts as detailed below.
Rapid Tests
COVID-19 IgG/IgM Rapid Test (Zhejiang Orient Gene Biotech) is a solid phase immunochromatographic assay for qualitative detection of IgM and IgG antibodies to recombinant N and S proteins. Innovita 2019-nCoV Ab Test (Innovita Biological Technology) is a colloidal gold lateral flow assay for qualitative detection of IgM and IgG antibodies to undisclosed SARS-CoV-2 epitopes. Rapid tests were considered positive if a line was observed for IgM, IgG, or both. Color intensity was not evaluated.
ELISA
Wantai SARS-COV-2 Ab ELISA (Beijing Wantai Biological Pharmacy Enterprise) is a double-antigen sandwich immunoassay for qualitative detection of all antibody isotypes (IgM, IgA, IgG) against RBD domain of S1 protein. Samples with a cutoff ratio (absorbance of the sample at 450 nm divided by 0.19) higher than 0.9 were considered positive (classifying gray zone results with cutoff ratio 0.9-1.1 as positive). Three indirect ELISAs from EUROIMMUN were tested: Anti-SARS-CoV-2 IgG and IgA assays for semiquantitative detection of IgA and IgG antibodies against S1 protein and Anti-SARS-CoV-2-NCP(IgG) assay for semiquantitative detection of IgG against N protein. (cutoff of 0.8 units, classifying gray zone results of 0.8-1.1 units as positive). All ELISAs were tested using the PhD system (Version EIA 0_16, Bio-Rad Laboratories).
Chemiluminescent Immunoassays
Elecsys Anti-SARS-CoV-2 assay for cobas e601 module (Roche Diagnostics) is a double-antigen sandwich assay for qualitative detection of all antibody isotypes (IgM, IgA, IgG) against N protein (cutoff of 1 cutoff index). LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin) is an indirect chemiluminescent immunoassay for quantitative detection of IgG antibodies against S1/S2 proteins (cutoff of 12 arbitrary units [AU]/mL, classifying gray zone results of 12-15 AU/mL as positive).
SARS-CoV-2 PCR
This was done using the Allplex 2019-nCoV assay (Seegene) for E/N/RdRP genes on nasopharyngeal swabs.
Statistical Analysis
Statistical analyses were performed using MedCalc (version 12.2.1). Sensitivities for detection of presence of SARS-CoV-2 antibodies were evaluated on samples obtained from SARS-CoV-2 PCR-positive patients as (1) total fraction of samples showing detectable antibodies and (2) by comparing each individual assay vs consensus outcome obtained by majority of all assays evaluated in this study. χ 2 test was used for comparing proportions for categorical variables. Not normally quantitative variables are expressed as medians (interquartile range [IQR]), and Mann-Whitney test was used to test for statistical differences between various timeframes after onset symptoms. Differences were considered statistically significant if P < .05. Kinetics of seroconversion in individual patients in Figure 1 were fitted to a scale from –1 to +1, with 0 representing each assays cutoff by subtracting each assay’s cutoff from its raw data signals, and dividing its absolute value by highest (lowest) cutoff-corrected signal for that assay obtained in our data set for positive (negative) samples.
Figure 1.
Kinetics of seroconversion in critically ill COVID-19 patients. The upper left panel shows the average kinetics of seroconversion in 13 intensive care unit patients. The other panels show the kinetics in 8 individual patients for whom 3 or more data points were available. Graphs represent for each of the indicated serology tests the normalized signal over time, fitted to a scale from –1 to +1 with 0 (black line) representing the assays’ cutoff, as described in the Statistical Analysis section.
Results
Cross-Reactivity (Analytical Specificity)
Analytical specificity was evaluated on 57 prepandemic samples from individuals infected with other HCoV viruses (229E/HKU1/OC43), other infectious agents, or with positivity to anti–nuclear factor or rheumatoid factor (Table 1). Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG, and Innovita 2019-nCoV Ab Test showed no cross-reactivity Table 2. EUROIMMUN Anti-SARS-CoV-2 IgA and Orient Gene COVID-19 IgG/IgM Rapid Test showed cross reactivity with common cold HCoV viruses, resulting in respective analytical specificities of 91.1% and 92.9%. LIAISON SARS-CoV-2 S1/S2 IgG (96.4% analytical specificity) was the only to show interference by rheumatoid factor (Table 1).
Table 2.
Performance Characteristics of Serology Kits vs the Result of PCR
| S-RBD Total Ab (Wantai) |
S1 IgA (EI) | S1 IgG (EI) |
S1 IgA + IgG (EI) |
S1/S2 IgG (DiaSorin) |
N Total Ab (Roche) | N IgG (EI) |
N/S IgM (Orient Gene) |
N/S IgG (Orient Gene) | N/S IgM + IgG (Orient Gene) |
IgM (Innovita) |
IgG (Innovita) |
IgM + IgG (Innovita) |
|||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall | SP | n/N % 95% CI |
57/57 100 93.7-100 |
51/56 91.1a 80.4-97.0 |
56/56 100 93.6-100 |
51/56 91.1a 80.4-97.0 |
54/56 96.4 87.7-99.6 |
56/56 100 93.6-100 |
55/56 98.2 90.4-99.9 |
55/56 98.2 90.4-99.9 |
53/56 94.6 85.1-98.9 |
52/56 92.9a 82.7-98.0 |
56/56 100 93.6-100 |
56/56 100 93.6-100 |
56/56 100 93.6-100 |
| SN | n/N % 95% CI |
146/169 86.4 80.3-91.2 |
134/169 79.3 72.4-85.1 |
122/169 72.2a 64.8-78.8 |
141/169 83.4 77.0-88.7 |
106/168 63.1a 55.3-70.4 |
132/170 77.6a 70.6-83.7 |
128/170 75.3a 68.1-81.6 |
128/171 74.8a 67.7-81.2 |
134/171 78.4 71.4-84.3 |
143/171 83.6 77.2-88.8 |
72/170 42.4a 34.8-50.2 |
113/170 66.5a 58.8-73.5 |
121/170 71.2a 63.7-77.8 |
|
| PPV | % 95% CI |
100 97.5-100 |
96.4 91.8-98.8 |
100 97.0-100 |
96.6 92.2-98.9 |
98.2 93.5-99.8 |
100 97.2-100 |
99.2 95.8-99.9 |
99.2 95.8-99.9 |
97.8 93.7-99.6 |
97.3 93.2-99.2 |
100 95.0-100 |
100 96.8-100 |
100 97.0-100 |
|
| NPV | % 95% CI |
71.2 60.0-80.8 |
59.3 48.2-69.8 |
54.4 44.3-64.2 |
64.6 53.0-75.0 |
46.6 37.2-56.0 |
59.6 49.0-69.6 |
56.7 46.2-66.7 |
56.1 45.7-66.1 |
58.9 48.0-69.2 |
65.0 53.5-75.3 |
36.4 28.8-44.5 |
49.6 40.0-59.1 |
53.3 43.3-63.1 |
|
| <10 dpos |
SN | n/N % 95% CI |
40/53 75.5 61.7-86.2 |
32/53 60.4 46.0-73.6 |
22/53 41.5a 28.1-55.9 |
32/53 60.4 46.0-73.6 |
16/51 31.4a 19.1-45.9 |
26/52 50.0a 35.8-64.2 |
29/52 55.8a 41.3-69.5 |
33/53 62.3 47.9-75.2 |
26/53 49.1a 35.1-63.2 |
34/53 64.2 49.8-76.9 |
20/52 38.5a 25.3-53.0 |
23/52 44.2a 30.5-58.7 |
28/52 53.8a 39.5-67.8 |
| 10-20 dpos |
SN | n/N % 95% CI |
36/40 90.0 76.3-97.2 |
40/42 95.2 83.8-99.4 |
37/42 88.1 74.4-96.0 |
40/42 95.2 83.8-99.4 |
34/42 81.0 65.9-91.4 |
39/42 92.9 80.5-98.5 |
38/42 90.5 77.4-97.3 |
38/42 90.5 77.4-97.3 |
39/42 92.9 80.5-98.5 |
39/42 92.9 80.5-98.5 |
25/42 59.5a 43.3-74.4 |
35/42 83.3 68.6-93.0 |
36/42 85.7 71.5-94.6 |
| >20 dpos |
SN | n/N % 95% CI |
70/76 92.1 83.6-97.0 |
62/74 83.8 73.4-91.3 |
63/74 85.1 75.0-92.3 |
69/74 93.2 84.9-97.8 |
56/75 74.7a 63.3-84.0 |
67/76 88.2 78.7-94.4 |
61/76 80.3a 69.5-88.5 |
57/76 75.0a 63.7-84.2 |
69/76 90.8 81.9-96.2 |
70/76 92.1 83.6-97.0 |
27/76 35.5a 24.9-47.3 |
55/76 72.4a 60.9-82.0 |
57/76 75.0a 63.7-84.2 |
| Hospitalized patients | SN | n/N % 95% CI |
86/103 83.5 74.9-90.1 |
81/105 77.1 67.9-84.8 |
68/105 64.8a 54.8-73.8 |
82/105 78.1 69.0-85.6 |
58/103 56.3a 46.2-66.1 |
75/104 72.1a 62.5-80.5 |
77/104 74.0 64.5-82.1 |
80/105 76.2 66.9-84.0 |
74/105 70.5a 60.8-79.0 |
83/105 79.0 70.0-86.4 |
53/104 51.0a 41.0-60.9 |
67/104 64.4a 54.4-73.6 |
73/104 70.2a 60.4-78.8 |
| Paucisymptomatic patients | SN | n/N % 95% CI |
60/66 90.9 81.3-96.6 |
53/64 82.8 71.3-91.1 |
54/64 84.4 73.1-92.2 |
59/64 92.2 82.7-97.4 |
48/65 73.8a 61.5-84.0 |
57/66 86.4 75.7-93.6 |
51/66 77.3a 65.3-86.7 |
48/66 72.7a 60.4-83.0 |
60/66 90.9 81.3-96.6 |
60/66 90.9 81.3-96.6 |
19/66 28.8a 18.3-41.2 |
46/66 69.7a 57.2-80.4 |
48/66 72.7a 60.4-83.0 |
Ab, antibody; CI, confidence interval; dpos, days post onset of symptoms; EI, EUROIMMUN; NPV, negative predictive value; PCR, polymerase chain reaction; PPV, positive predictive value; SN, sensitivities; SP, specificities.
aIndicates differences with the Wantai SARS-COV-2 Ab ELISA for which P < .05 were considered statistically significant.
Sensitivity for Detection of Presence of SARS-CoV-2 Antibodies
Study Participants
Sensitivities for detection of SARS-CoV-2 antibodies were compared on 171 samples obtained from 135 subjects, all with PCR-confirmed SARS-CoV-2 infections, pooled or grouped in two distinct cohorts: hospitalized and paucisymptomatic COVID-19 patients. Hospitalized patients included 105 samples from 71 patients hospitalized for severe COVID-19 disease, all with very high level of suspicion of COVID-19 pneumonia on chest computed tomography (COVID-19 Reporting and Data System [CO-RADS] score = 5)13: 48 males (median age, 65 years; IQR, 53-80) and 23 females (median age, 79 years; IQR, 67-86). Serum samples ranged from 0 to 39 days after patient-reported symptom onset. Paucisymptomatic patients included 66 samples from 64 health care workers with mild (n = 61) or no (n = 3) WHO-listed COVID-19 symptoms: myalgia (present in 62.5%), fever (60.9%), dry cough (56.2%), dyspnea (40.6%), severe fatigue (35.9%), headaches (30.0%), loss of smell or taste (26.6%), or diarrhea (18.8%). None of these patients were hospitalized. Serum samples ranged from 11 to 54 days after patient-reported symptom onset.
Sensitivity for Detection of SARS-CoV-2 Antibodies
Sensitivity was calculated for different patient groups (all patients, hospitalized and paucisymptomatic patients). First, vs SARS-CoV-2 PCR (100% of samples from PCR+ patients) as reference, by measuring the percentage of samples showing antibody titers above the respective assay’s cutoff (Table 2). Second, by comparing each individual assay to the consensus outcome of the majority of 7 tested assays Table 3. Wantai SARS-COV-2 Ab ELISA showed highest overall sensitivity for detection of SARS-CoV-2 antibodies: 86.4% (95% confidence interval [CI], 80.3%-91.2%) vs PCR and 100% (95% CI, 97.3%-100%) vs consensus at all time points in both patient cohorts. Its sensitivity was significantly higher (P < .05) than all other assays with the exception of Orient Gene COVID-19 IgG/IgM Rapid Test and EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined. In a real-world clinical setting, serology assays might be used at later time stages, eg, more than 20 days after onset of symptoms or to document past SARS-CoV-2 infection in paucisymptomatic patients. In these patients, 4 assays showed clinically acceptable sensitivity for detection of SARS-CoV-2 antibodies above 95% vs consensus result (Table 3): Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined, and Orient Gene COVID-19 IgG/IgM Rapid Test. In comparison with all other assays, LIAISON SARS-CoV-2 S1/S2 IgG showed significantly (P < .05) lower sensitivities of 83.6% (95% CI, 72.5%-91.5%) vs consensus (Table 3) at greater than 20 days post onset of symptoms and of 84.2% (95% CI, 72.1%-92.5%) in paucisymptomatic patients. Also, EUROIMMUN Anti-SARS-CoV-2-NCP(IgG) and Innovita 2019-nCoV Ab Test showed limited sensitivity at greater than 20 days post onset of symptoms.
Table 3.
Performance Characteristics of Serology Kits vs the Consensus Result of All Assays
| S-RBD Total Ab (Wantai) |
S1 IgA (EI) | S1 IgG (EI) |
S1 IgA + IgG (EI) |
S1/S2 IgG (DiaSorin) |
N Total Ab (Roche) | N IgG (EI) |
S/N IgM (Orient Gene) |
S/N IgG (Orient Gene) | S/N IgM + IgG (Orient Gene) |
IgM (Innovita) |
IgG (Innovita) | IgM + IgG (Innovita) |
|||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall | SN | n/N % 95% CI |
133/133 100 97.3-100 |
127/133 95.5a 90.4-98.3 |
119/133 89.5a 83.0-94.1 |
132/133 99.2 95.9-99.9 |
106/133 79.7a 71.9-86.2 |
129/134 96.3a 91.5 98.8 |
126/134 94.0a 88.6-97.4 |
123/135 91.1a 85.0-95.3 |
130/135 96.3a 91.6- 98.8 |
135/135 100 97.3-100 |
70/134 52.2a 43.4-60.9 |
112/134 83.6a 76.2-89.4 |
119/134 88.8a 82.2-93.6 |
| <10 dpos | SN | n/N % 95% CI |
29/29 100 88.1-100 |
29/29 100 88.1-100 |
22/29 75.9a 56.5-89.7 |
29/29 100 88.1-100 |
16/28 57.1a 37.2-75.5 |
25/28 89.3 71.8-97.7 |
27/28 96.4 81.6-99.9 |
28/29 96.6 82.2-99.9 |
25/29 86.2a 68.3-96.1 |
29/29 100 88.1-100 |
18/28 64.3a 44.1-81.4 |
22/28 78.6a 59.0-91.7 |
26/28 92.9 76.5-99.1 |
| 10-20 dpos | SN | n/N % 95% CI |
36/36 100 90.3-100 |
38/38 100 90.8-100 |
36/38 94.7 82.2-99.4 |
38/38 100 90.8-100 |
34/38 89.5a 75.2-97.1 |
38/38 100 90.8-100 |
38/38 100 90.8-100 |
38/38 100 90.8-100 |
38/38 100 90.8-100 |
38/38 100 90.8-100 |
25/38 65.8a 48.6-80.4 |
35/38 92.1 78.6-98.3 |
36/38 94.7 82.2-99.4 |
| >20 dpos | SN | n/N % 95% CI |
68/68 100 94.7-100 |
60/66 90.9a 81.3-96.6 |
61/66 92.4a 83.2-97.5 |
65/66 98.5 91.8-99.9 |
56/67 83.6a 72.5-91.5 |
66/68 97.1 89.8-99.6 |
61/68 89.7a 79.9-95.8 |
57/68 83.8a 72.9-91.6 |
67/68 98.5 92.1-99.9 |
68/68 100 94.7-100 |
27/68 39.7a 28.0-52.3 |
55/68 80.9a 69.5-89.4 |
57/68 83.8a 72.9-91.6 |
| Hospitalized patients | SN | n/N % 95% CI |
75/75 100 95.2-100 |
76/77 98.7 93.0-99.9 |
67/77 87.0a 77.4-93.6 |
77/77 100 95.3-100 |
58/76 76.3a 56.2-85.3 |
73/76 96.0 88.9-99.2 |
75/76 98.7 93.0-99.9 |
76/77 98.7 93.0-99.9 |
67/77 87.0a 77.4-93.6 |
77/77 100 95.3-100 |
51/76 67.1a 55.4-77.5 |
66/76 86.8a 77.1-93.5 |
71/76 93.4a 85.3-97.8 |
| Paucisymptomatic patients | SN | n/N % 95% CI |
58/58 100 93.8-100 |
51/56 91.1a 80.4-97.0 |
52/56 92.9a 82.7-98.0 |
55/56 98.2 90.4-99.9 |
48/57 84.2a 72.1-92.5 |
56/58 96.6 88.1-99.6 |
51/58 87.9a 76.7-95.0 |
48/58 82.8a 70.6-91.4 |
58/58 100 93.8-100 |
58/58 100 93.8-100 |
19/58 32.8a 21.0-46.3 |
46/58 79.3a 66.6-88.8 |
48/58 82.8a 70.6-91.4 |
Ab, antibody; CI, confidence interval; dpos, days post onset of symptoms; EI, EUROIMMUN; SN, sensitivities.
aIndicates differences with the Wantai SARS-COV-2 Ab ELISA for which P < .05 were considered statistically significant.
Kinetics of Seroconversion
We compared timing of detection of antibodies of the ELISA/CLIA assays on consecutive blood samples of 8 critically ill patients admitted to intensive care units (Figure 1). In all 8 patients, Wantai SARS-COV-2 Ab ELISA was first to exceed the predefined assay cutoff, followed by the EUROIMMUN Anti-SARS-CoV-2 IgA assay. In this cohort of intensive care patients, of the N-targeting assays, EUROIMMUN Anti-SARS-CoV-2-NCP(IgG) provided positive results more rapidly than Elecsys Anti-SARS-CoV-2 assay. LIAISON SARS-CoV-2 S1/S2 IgG was last to detect seroconversion. Seroconversion rates were additionally studied by a pooled analysis in samples from different patients, grouped according to the timeframe after symptom onset ranging from less than 10 days, 10 to 20 days, or more than 20 days post onset of symptoms (Tables 2 and 3). All tests except Wantai SARS-COV-2 Ab ELISA showed a significantly higher positivity rate between 10 and 20 days post onset of symptoms as compared to less than 10 days post onset of symptoms (P < .05). No significant differences were observed in positivity rates between 10 and 20 days post onset of symptoms and more than 20 days post onset of symptoms (Table 2), indicating that serology testing can be performed starting from 10 days after onset symptoms. In samples less than 10 days after onset of symptoms, all from hospitalized patients, the Wantai SARS-COV-2 Ab ELISA outperformed all other assays, with a sensitivity of 100% (95% CI, 88.1%-100%) vs consensus and 75.5% (95% CI, 61.7%-86.2%) vs PCR, which was significantly lower than its performance in samples from patients greater than 20 days post onset of symptoms (P < .05).
Concordance Analysis of Humoral Immune Response on Individual Samples
For the assays with acceptable overall sensitivity above 95% (Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined, and Orient Gene COVID-19 IgG/IgM Rapid Test) a good overall concordance was seen in samples from patients greater than 10 days post onset of symptoms, with 87.7% and 3.5% of samples positive or negative respectively with all 4 methods. No clear differences were observed in kinetics of appearance of antibodies to S or N epitopes. Beyond 10 days, only 1.4% (1/71) of hospitalized and 4.7% (3/64) paucisymptomatic patients developed no detectable antibodies.
Discussion
In this study we report on the clinical performance characteristics of 7 commercially available serology tests for detection of antibodies against SARS-CoV-2 S protein (S-RBD total antibodies, S1/S2 IgG, S1 IgA and IgG), N protein (N total antibodies, N IgG), and both proteins (N/S IgM and IgG). To our knowledge, this study is the first to report performance of Elecsys Anti-SARS-CoV-2 assay on the cobas e601 module. We specifically investigated their relative value as a complementary tool to screen for prior SARS-CoV-2 infection in individuals that were not (conclusively) tested by PCR in early stage of active viral replication up to 10 days after onset of symptoms. As a working definition for acceptable performance, we propose that such an assay should combine a minimal sensitivity for detection of SARS-CoV-2 antibodies of 95% vs a consensus estimate and a high analytical specificity above 98% in samples taken 20 days or more after onset of symptoms, also in subjects who experienced mild SARS-CoV-2 symptoms. Based on these criteria, Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, and Innovita 2019-nCoV Ab Test all showed acceptable analytical specificity. In terms of sensitivity for detection of SARS-CoV-2 antibodies vs consensus result obtained by all tests, Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG combined with IgA, and Orient Gene COVID-19 IgG/IgM Rapid Test were acceptable. Overall, only Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assay fulfilled the proposed acceptance criteria, with Wantai SARS-COV-2 Ab ELISA clearly outperforming all other evaluated assays.
A strength of this study is that the parallel evaluation of several kits allowed a reliable direct comparison of clinical performance using cutoffs provided by manufacturers. Also, our patient cohorts, including not only severe COVID-19 patients but also a sizeable cohort of mild SARS-CoV-2 infections, provide a good estimate on assays’ performances in the intended target population. We observed no notable differences in timing of seroconversion between severe and milder SARS-CoV-2 infections.
There are limitations to our study. Cross-reactivity analysis might require more extensive exploration. A higher number of sera from patients with PCR-confirmed HCoV infections and other common cold viruses need to be investigated. Our study did not include a sizeable cohort of fully asymptomatic SARS-CoV-2 infections, and it focused solely on qualitative analysis. Therefore, no investigation of differences in assays’ performance for quantification of antibody titers was performed.
In critically ill COVID-19 patients, SARS-CoV-2 antibody levels were reported to correlate to disease severity1 by triggering bradykinin and complement activation pathways. The assays evaluated here show large variations in their dynamic range, ranging from a good linearity for LIAISON SARS-CoV-2 S1/S2 IgG14 to a limited dynamic range with rapid signal saturation for Wantai SARS-COV-2 Ab ELISA. With a sample volume input of 100 μL that is 10 to 20 times higher than the other evaluated assays, Wantai SARS-COV-2 Ab ELISA is clearly designed toward high sensitivity for detection of SARS-CoV-2 antibodies by maximal antibody capture. Caution is needed when comparing (semi)quantitative estimates of antibody titers across platforms before certified standards with known titers become available.
Our data are compatible with other cross-platform evaluations15 indicating superior performance of Wantai SARS-COV-2 Ab ELISA as compared to EUROIMMUN Anti-SARS-CoV-2 IgG and IgA. The results are, however, discrepant with another study reporting a sensitivity of 100% and 99% specificity for LIAISON SARS-CoV-2 S1/S2 IgG14 obtained on a small set of 125 samples including only 40 PCR-confirmed patients and after receiver operating characteristic optimization of assay cutoffs. Since we observed considerable lot-to-lot variations in raw signals of the 2 LIAISON SARS-CoV-2 S1/S2 IgG kits tested, we believe that caution is warranted and cutoffs should only be optimized on better data sets and proper assessment of different lots.
It was reported that antibodies against S protein appear later in infection than antibodies against N protein.1,9 We also observed faster seroconversion of N vs S1 targeting IgG in EUROIMMUN assays. On the other hand, we observed a much faster seroconversion of total antibodies (IgA/IgM/IgG) against S-RBD (Wantai) than N protein (Elecsys). Within the same epitope/assay format (EUROIMMUN to S1), IgA antibodies clearly precede IgG. Overall, our data suggest that timing of seroconversion depends more on assay design, recombinant viral epitope, and antibody isotypes covered, and that overall sensitivity for detection of antibodies is likely enhanced when both IgA and IgG isotypes are measured.
In conclusion, this study supports clinical use of both Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assay for sensitive and specific screening of SARS-CoV-2 antibodies from 10 days after onset of symptoms.
Acknowledgments
This work was supported by a private donation by board members of Fagron (Nazareth, Belgium), a health care company, to RADar, the teaching and education initiative of AZ Delta General Hospital, to be used as an unconditional research grant for data collection, collaborative collaboration, and open access publication. The sponsor had no influence on the study design, data interpretation, and drafting of the manuscript. Clinical trial numbers IRB B1172020000009 and B1172020000006.
References
- 1. Grzelak L, Temmam S, Planchais C, et al. SARS-CoV-2 serological analysis of COVID-19 hospitalized patients, pauci-symptomatic individuals and blood donors. medRxiv. 2020. 10.1101/2020.04.21.20068858. [DOI] [Google Scholar]
- 2. De Smet D, De Smet K, Herroelen P, et al. Vitamin D deficiency as risk factor for severe COVID-19 : a convergence of two pandemics. medRxiv. 2020. 10.1101/2020.05.01.20079376. [DOI] [Google Scholar]
- 3. Qiu M, Shi Y, Guo Z, et al. Antibody responses to individual proteins of SARS coronavirus and their neutralization activities. Microbes Infect. 2005;7:882-889. 10.1016/j.micinf.2005.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Bohn MK, Lippi G, Horvath A, et al. Molecular, serological, and biochemical diagnosis and monitoring of COVID-19: IFCC taskforce evaluation of the latest evidence. Clin Chem Lab Med. 2020. 10.1515/cclm-2020-0722. [DOI] [PubMed] [Google Scholar]
- 5. Ai T, Yang Z, Hou H, et al. Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology. 2020. 10.1148/radiol.2020200642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. De Smet K, De Smet D, Demedts I, et al. Diagnostic power of chest CT for COVID-19: to screen or not to screen. medRxiv. 2020. 10.1101/2020.05.18.20097444. [DOI] [Google Scholar]
- 7. Chan JFW, Kok KH, Zhu Z, et al. Genomic characterization of the 2019 novel human-pathogenic coronavirus isolated from a patient with atypical pneumonia after visiting Wuhan. Emerg Microbes Infect. 2020;9:221-236. doi: 10.1080/22221751.2020.1719902. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Hachim A, Kavian N, Cohen CA, et al. Beyond the spike: identification of viral targets of the antibody responses to SARS-CoV-2 in COVID-19 patients. medRxiv. 2020. 10.1101/2020.04.30.20085670. [DOI] [Google Scholar]
- 9. Meyer B, Drosten C, Müller MA. Serological assays for emerging coronaviruses: challenges and pitfalls. Virus Res. 2014;194:175-183. doi: 10.1016/j.virusres.2014.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Grifoni A, Sidney J, Zhang Y, et al. A sequence momology and bioinformatic approach can predict candidate targets for immune responses to SARS-CoV-2. Cell Host Microbe. 2020;27:671-680. 10.1016/j.chom.2020.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. GeurtsvanKessel CH, Okba NMA, Igloi Z, et al. Towards the next phase: evaluation of serological assays for diagnostics and exposure assessment. medRxiv. 2020. 10.1101/2020.04.23.20077156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Okba NMA, Müller MA, Li W, et al. Severe acute respiratory syndrome coronavirus 2−specific antibody responses in coronavirus disease 2019 patients. Emerg Infect Dis J. 2020;26. doi: 10.3201/eid2607.200841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Prokop M, van Everdingen W, van Rees Vellinga T, et al. CO-RADS—a categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation. Radiology. 2020. 10.1148/radiol.2020201473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Tré-hardy M, Wilmet A, Beukinga I, et al. Validation of a chemiluminescent assay for specific SARS-CoV-2 antibody. Clin Chem Lab Med. 2020;5298 10.1515/cclm-2020-0594. [DOI] [PubMed] [Google Scholar]
- 15. Lassaunière R, Frische A, Harboe ZB, et al. Evaluation of nine commercial SARS-CoV-2 immunoassays. medRxiv. 2020. 10.1101/2020.04.09.20056325. [DOI] [Google Scholar]