Sensitivity and specificity of 14 SARS-CoV-2 serological assays and their diagnostic potential in RT-PCR negative COVID-19 infections

ABSTRACT

Background: Molecular detection of SARS-CoV-2 in respiratory samples is the gold standard for COVID-19 diagnosis but it has a long turnaround time and struggles to detect low viral loads. Serology could help to diagnose suspected cases which lack molecular confirmation. Two case reports are presented as illustration.

Objectives: The aim of this study was to evaluate the performance of several commercial assays for COVID-19 serology. We illustrated the added value of COVID-19 serology testing in suspect COVID-19 cases with negative molecular test.

Study design: Twenty-three sera from 7 patients with a confirmed molecular diagnosis of SARS-CoV-2 were tested using 14 commercial assays. Additionally, 10 pre-pandemic sera and 9 potentially cross-reactive sera were selected. We calculated sensitivity and specificity. Furthermore, we discuss the diagnostic relevance of COVID-19 serology in a retrospective cohort of 145 COVID-19 cases in which repetitive molecular and serological SARS-CoV-2 tests were applied.

Results: The interpretation of the pooled sensitivity of IgM/A and IgG resulted in the highest values (range 14–71% on day 2–7; 88–94% on day 8–18). Overall, the specificity of the assays was high (range 79–100%). Among 145 retrospective cases, 3 cases (2%) remained negative after sequential molecular testing but positive on final SARS-CoV-2 serology.

Conclusion: Sensitivity of COVID-19 serological diagnosis was variable but consistently increased at >7 days after symptom onset. Specificity was high. Our data suggest that serology can complement molecular testing for diagnosis of COVID-19, especially for patients presenting the 2^nd week after symptom onset or later.

Background

In December 2019, several cases of an unidentified pneumonia occurred in the province of Hubei, China. With the use of next-generation sequencing, the Wuhan Institute of Virology identified a new coronavirus as the etiological agent [1,2]. This new virus, which belongs to the genus betacoronavirus, was initially acknowledged as novel coronavirus 2019 (2019-nCOV) [3]. On 11 February 2020, the virus was officially renamed the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by the International Committee on Taxonomy of Viruses [4]. Since then the virus has rapidly spread and on the 18^th of July 2020, over 14 million confirmed cases and 600 000 direct casualties have been reported [5]. So far, implementation of strict quarantine measures have contained further exponential spreading of the virus in Europe and have flattened the first epidemic curve.

Considering the extent of the pandemic, its socio-economic impact, and the effect on healthcare systems worldwide, there is a high need to extend the diagnostic capacity. So far, the gold standard for the diagnosis of SARS-CoV-2, implemented at the start of the outbreak, is real-time reverse transcriptase polymerase chain reaction (RT-PCR) [6]. In contrast with a high specificity up to 98.8% [7], RT-PCR suffers from a rather long turnaround time when performed in batch testing (2 to 6 hours). Continuous PCR testing has a faster turnaround time (1 u) but a low capacity. False negative results in both the very early as well as the late phase of the disease due to low virus shedding at these stages of infection are reported [8–10]. Studies have shown discrepant results between RT-PCR and chest computed tomography (CT) findings [11,12]. CT imaging findings in 30 patients with suspicion of COVID-19 pneumonia were shown to have a sensitivity of 97.2% compared to 83.3% of initial nasopharyngeal RT-PCR [13].

Establishing the diagnosis of COVID-19 is sometimes very challenging and does not only rely on molecular tests. We report on two patients who presented at the emergency department with high suspicion of COVID-19 infection based on clinical findings and/or chest-CT findings. Nevertheless, SARS-CoV-2 RT-PCR testing was repeatedly negative on repetitive nasopharyngeal, throat and anal swabs.

Case 1 was a 66 year old male experiencing symptoms of shortness of breath, fever and muscle pain since 8 days. Lab findings showed thrombopenia (87,000/µL, ref. 149,000–31,9000/µL), lymphopenia (930/µL, ref. 1,133–3,105/µL), normal neutrophil count (2,980/µL, ref. 1,573–6,100/µL) low eosinophil count (0/µL, ref. 28–273/µL), normal basophil count (10/µL, ref. 6–50/µL), elevated D-dimers (1,480 ng/mL, ref. <500 ng/mL) and elevated C-reactive protein (29 mg/L, ref <5 mg/L). Chest CT on the day of admission showed bilateral ground glass opacities with a crazy paving pattern suspicious for COVID-19 (CT severity score of 10 or CO-RADS classification of 4: COVID-19 likely). Repetitive nasopharyngeal swabs on day 8, 9, 12 and 13 as well as anal swabs on day 9 and 13 after symptom onset were all SARS-CoV-2 RT-PCR negative using the protocol described by Corman and colleagues [6]. Serology on day 13 after symptom onset indicated negative IgM but positive IgG (Prima Professional®, Point-Of-Care antibody Tests). On day 13, the patient was admitted to the intensive care unit (ICU) because of type 1 respiratory insufficiency (pO2 60 mmHg, ref. 83–108 mm Hg; pCO2 31 mmHg, ref. 35–45 mm Hg; pH 7.46, ref. 7.35–7.45). He did not require mechanical ventilation. Based on these findings, he was diagnosed with COVID-19.

Case 2 was a 14 year old child presenting at the emergency room with 3 days of fever > 39°C, muscle pain, cough, sore throat, headache and fatigue. The girl’s father had suffered from fever and respiratory symptoms after contact with multiple COVID-19 patients 4 weeks earlier. On day 4 after symptom onset, there was a clinical deterioration to refractory shock and multi-organ failure with hypotension and respiratory and cardiac failure. Lab findings showed thrombopenia (89,000/µL, ref. 154,000–452,000/µL), lymphopenia (162/µL, ref. 1,500–6,500/µL), normal neutrophil count (7,564/µL, ref. 2,500–8,000), normal eosinophil count (283/µL, ref. 100–500/µL), low basophil count (0/µL, ref. 10–100/µL), elevated D-dimers (4,420 ng/mL, ref. <500 ng/mL) and high C-reactive protein (308 mg/L, ref. <5 mg/L). Chest CT did not show signs of viral pneumonia. Echocardiography revealed decreased left ventricular function. She responded well to supportive therapy and corticosteroid treatment. Throat swabs on day 3 and 5 after symptom onset were negative for SARS-CoV-2 with RT-PCR [6]. Serology on day 8 after symptom onset was negative for IgM but positive for IgG (Prima Professional®). She was diagnosed with pediatric inflammatory multisystem syndrome temporally associated with COVID-19 (PIMS-TS).

The case reports mentioned above point out the potential of serological assays to add to the diagnosis of COVID-19. So far, a large number of serological assays have been developed in response to the diagnostic need [14]. Data on the validation and comparison of the performance of these assays are emerging but are still limited [15–18]. Since antibody production by the specific immune system is subject to delay, a lower diagnostic performance of serological assays in the early phase of the disease is expected. So far, studies have revealed that antibodies could be detected as early as 3–6 days after symptom onset [9,10,19,20].

Objectives

The main goal of this study was to evaluate the performance of 5 five Point-Of-Care antibody Tests, one chemiluminescence microparticle immunoassay (CMIA), one chemiluminescence immunoassay (CLIA) and 7 enzyme-linked immunosorbent assays (ELISA) for COVID-19 serology in an attempt to validate and select the best performing ones. Secondly, we discuss the role of SARS-CoV-2 serology in the process of diagnosing COVID-19 disease in presumable COVID-19 cases with sequential negative SARS-CoV-2 RT-PCR.

Study design

The study was approved by the local ethics committee (BC-07829) of the Ghent University Hospital, a 1062-beds tertiary care teaching hospital in Belgium. From March 12^th to 31^st 2020, 23 consecutive sera from 7 different patients, admitted to the hospital with a confirmed diagnosis of COVID-19 based on positive RT-PCR [6], were collected on day 2 to 18 after symptom onset. The study group consisted of 2 females and 5 males. The median age was 58 years (range 39–81 years). The collected sera were divided into 2 groups: day 2–7 after symptom onset (7 samples) and day 8–18 after symptom onset (16 samples) in order to evaluate clinical sensitivity.

Specificity was evaluated using 10 pre-pandemic sera, i.e. clinical samples sent to the hospital laboratory for non-COVID-19-related serological investigations from August to September 2019. Furthermore, 9 potential cross-reactive sera were included, i.e. IgM positivity against Cytomegalovirus (CMV) (n = 2), Epstein-Barr virus (EBV) (n = 2) or Toxoplasma gondii (Toxo) (n = 1); samples positive for rheumafactor (RF) (n = 1) or anti-nuclear factor (ANF) (n = 1)) and sera from patients positive for non-SARS-CoV-2 endemic coronaviruses (OC43 and NL63) (n = 2).

The 14 serological assays listed in Table 1 were performed on each of the 23 sera from PCR confirmed COVID-19 positive patients and the 19 pre-pandemic/cross-reactive sera, according to the manufacturer’s instructions. Sensitivity and specificity were calculated for each test. We calculated both the overall sensitivity (day 2–18 after symptom onset) and the sensitivity of the two subgroups (day 2–7 and day 8–18), each time for IgM/A and IgG separately and for combined IgM/A and IgG testing.

Table 1.

Overview of the included assays

Test Assay	Company	Antibody detection	Recombinant antigen
Point-of-care test
Corona Virus (COVID-19) Combined IgM/IgG Rapid Test	Sol Scientifics	IgM+IgG	Not specified
COVID-19 IgG/igM Rapid Test Cassette	Zhejiang ORIENT GENE Biotech	IgM+IgG	Spike + nucleocapsid protein
Wantai SARS-COV-2 Ab Rapid Test	Beijing Wantai Biological Pharmacy Enterprise	IgM+IgG (combined)	Not specified
COVID-19 IgG/IgM RAPID TEST	PRIMA PROFESSIONAL	IgM+IgG	Not specified
Diagnostic Kit for IgG/IgM Antibody to SARS-CoV-2	WIZ BIOTECH	IgM+IgG	Not specified
Enzyme-Linked Immunosorbent Assay (ELISA)
Anti-SARS-CoV-2 ELISA (IgA)	EUROIMMUN	S1	Spike glycoprotein (S1domain)
Anti-SARS-CoV-2 ELISA (IgG)	EUROIMMUN	IgG	Spike glycoprotein (S1domain)
Novel Coronavirus COVID-19 IgM ELISA Kit	Epitope Diagnostics (EDI)	IgM	Not specified
Novel Coronavirus COVID-19 IgG ELISA Kit	Epitope Diagnostics (EDI)	IgG	Nucleocapsid protein
SARS-CoV-2 IgM ELISA Kit	Creative Diagnostics	IgM	Not specified
SARS-CoV-2 IgG ELISA kit	Creative Diagnostics	IgG	Not specified
Human Anti-2019 nCoV(N) IgG ELISA kit	Finetest, Wuhan Fine Biotech Co., Ltd.	IgG	Spike glycoprotein (S1domain)
Chemiluminescent microparticle immunoassay (CMIA)
SARS-CoV-2 IgG	Abbott	IgG	Nucleocapsid protein
Chemiluminescence immunoassay (CLIA)
Liaison SARS-CoV-2 S1/S2 IgG	DiaSorin	IgG	S1 and S2 protein

Sensitivity and specificity were calculated by means of Excel (version 16.0, Microsoft, Washington, USA) using the following definitions in Excel:

Sensitivity = 100 x [True Positive/(True Positive + False Negative)]

Specificity = 100 x [True Negative/(True Negative +False Positive)]

All patients consulting at the emergency department of our hospital from the 1^st of March till the 14^th of May, with symptoms suggestive of COVID-19 disease that required hospital admission and for which laboratory diagnosis was performed in our institution were retrospectively included in a database. In all patients a RT-PCR test on nasopharyngeal/throat swab was performed at the time of admission. If the initial nasopharyngeal swab was found to be negative, a second nasopharyngeal and additional anal swab were analyzed. If still negative, a bronchoalveolar lavage (BAL) was performed. Patients with repetitive negative molecular testing for SARS-CoV-2 but high clinical, epidemiological and/or radiological suspicion of COVID-19 disease were subjected to serological testing using COVID-19 IgG/IgM rapid test (Prima Professional®). Among this retrospective cohort of hospitalized patients clinically treated for COVID-19 disease, we calculated the number of patients that ultimately had a RT-PCR confirmed diagnosis. As such, an estimation of the added diagnostic value of SARS-CoV-2 serology in this cohort could be calculated.

Results

Sensitivity on day 2–7 after symptom onset ranged from 0% to 57% for IgM/A and from 0% to 71% for IgG. Sensitivity on day 8–18 after symptom onset ranged from 50% to 94% for IgM/A and from 81% to 94% for IgG.

If IgA/M and IgG were both available, the combined IgM/A and IgG sensitivity result was consistently higher than the separate interpretation (range 14–71% on day 2–7 and 88–94% on day 8–18; see Table 2).

Table 2.

Sensitivity and specificity of all test assays

Test Assay	Sensitivity day 2–7			Sensitivity day 8–18			Overall sensitivity day 2-d18			Specificity
	IgM*	IgG	IgM* + IgG	IgM*	IgG	IgM* + IgG	IgM*	IgG	IgM* + IgG	IgM*	IgG
Point-of-care test
Corona Virus (COVID-19) Combined IgM/IgG Rapid test (Sol scientifics)	57%	14%	57%	94%	88%	94%	83%	65%	83%	79%	100%
COVID-19 IgG/IgM Rapid test Cassette (Orient Gene)	43%	29%	43%	94%	88%	94%	78%	70%	78%	100%	100%
SARS-CoV-2 Ab rapid test (Wantai)	n.a.	n.a.	43%	n.a.	n.a.	94%	n.a.	n.a.	78%	n.a.	100%
COVID-19 IgG/IgM RAPID TEST (PRIMA PROFESSIONAL)	0%	29%	29%	50%	81%	88%	35%	65%	70%	95%	100%
Diagnostic Kit for IgG/IgM Antibody to SARS-CoV-2 (WIZ BIOTECH)	14%	14%	14%	94%	94%	94%	70%	70%	70%	95%	100%
Enzyme-Linked Immunosorbent Assay (ELISA)
Anti-SARS-CoV-2 (EUROIMMUN)*	57%	29%	57%	94%	88%	94%	83%	70%	83%	95%	100%
Novel Coronavirus COVID-19 (Epitope Diagnostics)	29%	71%	71%	69%	94%	94%	57%	87%	87%	95%	89%
SARS-CoV-2 IgG ELISA Kit (Creative Diagnostics)	29%	14%	29%	94%	88%	94%	74%	65%	74%	100%	100%
Human Anti-2019 nCoV(N) IgG ELISA kit (Finetest)	n.a.	29%	n.a.	n.a.	88%	n.a.	n.a.	70%	n.a.	n.a.	83%
Chemiluminescent microparticle immunoassay (CMIA)
SARS-CoV-2 IgG (Abbott)	n.a.	29%	n.a.	n.a.	88%	n.a.	n.a.	70%	n.a.	n.a.	100%
Chemiluminescence immunoassay (CLIA)
Liaison SARS-CoV-2 S1/S2 IgG (DiaSorin)	n.a.	0%	n.a.	n.a.	81%	n.a.	n.a.	59%	n.a.	n.a.	100%
Anti-SARS-CoV-2 (EUROIMMUN)*: IgA instead of IgM; n.a.: not applicable

Overall the specificity for SARS-CoV-2 IgG was excellent (100%) for all but two assays: The Novel Coronavirus COVID-19 (Epitope Diagnostics) kit showed cross reaction with Toxo IgM+ sample and one pre-pandemic sample and the Human Anti-2019 nCoV(N) IgG ELISA kit (Finetest) showed cross reaction with Toxo IgM+ sample and 2 pre-pandemic samples. For IgM, only two assays showed a specificity of 100%: COVID-19 IgG/IgM Rapid test Cassette (Orient Gene) and SARS-CoV-2 IgG ELISA Kit (Creative Diagnostics). The other assays cross reacted with one or more samples: the combined IgM/IgG Rapid Test IgM (Sol scientific®) showed cross-reaction with CMV IgM+ sample, ANF+ sample, RF+ sample and Tox IgM+ sample, the COVID-19 IgG/IgM RAPID TEST IgM (PRIMA PROFESSIONAL®) showed cross-reaction with EBV IgM+, the Diagnostic Kit for IgG/IgM Antibody to SARS-CoV-2 IgM (WIZ BIOTECH) showed cross-reaction with RF+ sera, the Anti-SARS-CoV-2 ELISA IgA (EUROIMMUN®) showed cross-reaction with CMV IgM+ sample and the Novel Coronavirus COVID-19 IgM (Epitope Diagnostics®) had a cross-reaction with one pre-pandemic sample.

In our institution a total of 145 patients were hospitalized with an in-house laboratory confirmed COVID-19 diagnosis. In this group, the presence of SARS-CoV-2 was confirmed using RT-PCR on the first nasopharyngeal and/or throat swab in 132 patients (91%). Another 4 patients were diagnosed with a second nasopharyngeal and 1 patient with an anal swab. Next, 5 patients were found to be positive on BAL samples. Three patients remained negative on repetitive swabs. These patients were not eligible for a BAL sampling procedure or did not present with respiratory symptoms and as such serological testing was performed. All 3 patients were found to be positive for IgG SARS-CoV-2 serology (Prima Professional®). Given the high pre-test probability for SARS-Cov-2 in this prospective cohort and repetitive testing, the added diagnostic value of SARS-CoV-2 serology in providing a diagnosis in this cohort was 2%.

Discussion

In this study, we evaluated the detection of SARS-CoV-2 antibodies (IgM/A or IgG) with 14 different serological assays. As expected, we found that sensitivity was higher if the test was performed starting from day 8 after symptom onset, as compared to earlier time points. With the exception of one assay (Liaison SARS-CoV-2 S1/S2 IgG (DiaSorin), all tests resulted in a sensitivity for IgG higher than 87% starting from day 8 after symptom onset. Specificity ranged from 79%-100%.

Recently, the first studies evaluating several commercial serology assays have been published. Geurts van Kessel et al [17]. studied three rapid tests, four ELISAs and a high throughput chemiluminescent assay. Sensitivity ranging from 81–100% was calculated by using a total of 187 sera from 107 RT-PCR confirmed COVID-19 patients. Specificity was determined using 147 serum and plasma samples from individuals exposed to human coronaviruses and other respiratory viruses with values ranging from 85–100%. Similar to our findings, the performance of the Liaison SARS-CoV-2 S1/S2 IgG (DiaSorin) was worst with a sensitivity of 81% (compared to 59% in our results) and specificity of 90% (compared with 100% in our results). The group of Haselmann et al [16]. evaluated the performance of two IgG ELISA assays and one IgG electrochemiluminescence immunoassay (ECLIA) based on 51 serum samples from 26 COVID-19 patients and another 51 serum samples from control patients. They report a sensitivity of 92–100% and specificity of 84–96%. The Austrian Red Cross Blood Service [15] evaluated 100 SARS-CoV-2 convalescent plasma donors and SARS-CoV-2 antibodies were characterized using three different IgG-ELISAs (EUROIMMUN IgG and NCP-IgG ELISA, Wantai ELISA), two CLIA (Elecsys, LIAISON) and two lateral flow tests (MEDsan IgM/IgG-Rapid-Test, Wantai Rapid Test). The Wantai ELISA and the Elecsys provided the highest sensitivities in this sample (98 and 95 % respectively). Serrano et al [18]. compared the performance of 3 lateral flow immunoassays (IgM/IgG combined) to 2 ELISAs (IgA and IgG) in serum samples from 109 RT-PCR confirmed patients in different weeks after symptom onset. The IgA ELISA was most sensitive the first week after symptom onset (71%). The sensitivity improved to 97% for IgA ELISA in the second week. In the third week IgA ELISA, IgG ELISA and 2 out 3 IgG lateral flow tests had a sensitivity of > 96%. The lateral flow immunoassays showed variable performances. Pieri et al. also found that IgA detected by the ELISA assay might be a more reliable and stable early serological marker than IgM. Instead, IgG, as expected, showed stable level after 10 days from symptoms onset [21]. These findings are in accordance with our study results.

Serological tests could be useful in some clinical settings. In our database, we found an added diagnostic value of serological assays in 2% of the COVID-19 suspect patients. Although the detection of SARS-CoV-2 antibodies does not provide information on the presence of the viral RNA, it confirms recent or past infection. This is especially relevant in clinical cases where high clinical suspicion for COVID-19 which cannot be confirmed with SARS-CoV-2 PCR.

The availability of these serological tests might avoid the need for more invasive sampling methods like BAL in the pursuit of diagnostic confirmation of COVID-19, especially. Furthermore, serology can aid in infection prevention management. Nuccetelli et al. suggest screening flowcharts for asymptomatic workers that is based on serological assays, workers that have already overcome the disease are subjected to molecular screening methods [22]. Next the development of antigen detection assays are underway, which have the potential to rapidly detect active SARS-CoV-2 infection. As the pandemic evolves, more and more insights will be gained in the value of serology as well as antigen detection [23].

The main limitation of the current study is the limited number of samples included. Nevertheless, this study has evaluated the performance of 14 different serological assays and therefore could provide useful information in this stage of the pandemic.

In conclusion, our findings show that sensitivity was variable but increased in a later stage of infection (at least 8 days after symptom onset). Interpretation of the combination of IgM or IgA and IgG resulted in the highest sensitivity. Our data suggest that virus-specific antibody detection for SARS-CoV-2 can complement molecular testing for diagnosis of COVID-19, especially for patients presenting in the 2^nd week or later after symptom onset. Larger datasets should be used to confirm current findings.

Disclosure statement

No potential conflict of interest was reported by the authors.

Author contributions

VHE: Conceptualization, Methodology, Investigation, Data Curation, Writing-Original draft preparation. CL, BJ, VBE, BF, DBL, SP, WJ: Writing-Reviewing and Editing. VB, VS, PE: Writing-Reviewing and Editing, Supervision.