A systematic review and meta-analysis of the accuracy of SARS-COV-2 IGM and IGG tests in individuals with COVID-19

Abstract

Introduction

: Active SARS-CoV-2 infection is confirmed mainly through the detection of viral nucleic acid via the reverse transcriptase polymerase chain reaction (RT-PCR) technique. Methods to assess humoral responses contribute to the monitoring of the disease and confirmation of exposure to the virus.

Objective

: To evaluate the accuracy of tests for IgM and IgG antibodies for SARS-CoV-2 infection confirmed by RT-PCR and utility as complementary data for immunosurveillance.

Methods

: Literature research was performed by searching the terms “COVID-19”, “COVID-19 diagnostic testing” and “test” in the databases MEDLINE, EMBASE, Cochrane Library, Web of Science and Cumulative Index to Nursing and Allied Health Literature to search for potentially eligible observational studies without language restrictions published up to September 2020.

Results

: The pooled sensitivity and specificity, regardless of collection moment, was 80.0% (CI 95% 72.0–86.0) and 97.0% (CI 95% 94.0–98.0) for “IgM and/or IgG”, respectively. Serology considering immunoglobulins M and G together had a high accuracy performance on “fifteenth day and after”: sensitivity and specificity was 91.0% (CI 95% 85.0–94.0) and 98.0% (CI 95% 95.0–99.0) respectively, DOR 461 and AUC 0.98.

Conclusion

: This study shows that serology is a group of tests with high accuracy, mainly following the second week after infection.

1. Introduction

In December 2019, a new variety of coronaviruses, officially called Severe Acute Respiratory Syndrome by Coronavirus-2 (SARS-CoV-2) was identified through three patients who had atypical pneumonia in Wuhan, China [1,2]. The rapid spread and sustained transmission of the virus, gave rise to a pandemic within three months of identification of the virus [3]. Worldwide, SARS-Cov-2 has infected more than 146 billion people, with more than 3 billion deaths (data obtained on April, 25, 2021) [4]. Due to the overlap of manifestations, clinical diagnosis is problematic, especially during seasonal flu [3].

Confirmation of infection occurs mainly through the detection of the SARS-CoV-2 nucleic acid via the reverse transcriptase polymerase chain reaction (RT-PCR) technique [3]. However, this test can only detect active infection and has variable accuracy through the course of infection and therefore does not provide a complete picture of the dynamics of infection. Additionally, RT-PCRs requires laboratories with specific equipment and highly trained technicians to perform the test [5]. Moreover, this technique requires the presence of sufficient quantities of the viral genome at the sample site, and therefore an incorrect sample collection can limit the usefulness of the quantitative method [6]. To date, a number of studies have shown that the RNA of the virus in swab samples from the upper respiratory tract is detectable by PCR until 7 days after first symptoms, on average [7]. Testing programs in large-scale, rapid diagnosis and isolation associated with rigorous tracking and preventive self-isolation of close contacts are essential measures to reduce the burden of the COVID-19 pandemic.

The quick and accurate diagnosis of COVID-19 contributes to the management of diseases and outbreaks, allowing health surveillance to act in preventing and generating new public health measures [7]. Under these conditions, additional screening methods that can detect convalescent infection contribute to monitoring of the virus by informing the attack rate in the population, and the infection fatality rate. Such screening is commonly performed via serological tests.

Serological tests are common within laboratory routine and identify infection through the detection of viral antigens or circulating antibodies. The test for the detection of specific SARS-CoV-2 antibodies in the blood of a patient offers a simpler, cheaper and faster diagnosis [5,7]. Antigen-specific immunoglobulin M (IgM) is the first subtype to be detected in the blood, followed by the production of immunoglobulin G (IgG). Thus, serological test for SARS-CoV-2 can be used in the rapid screening of Covid-19, identifying symptomatic or asymptomatic carriers, on a large scale [5]. Therefore, the objective of this systematic review is to evaluate the accuracy of serology for SARS-CoV-2 infections in cases confirmed or not by RT-PCR.

2. Methods

This is a systematic review and meta-analysis synthesizing scientific evidence of the accuracy of IgM and IgG antibody tests (serology) for SARS-CoV-2 infection diagnosis compared to RT-PCR.

A protocol with explicitly defined objectives and methods was registered in the International Platform of Registered Systematic Review and Meta-analysis Protocols (registration number INPLASY202040099, doi: 10.37766/inplasy2020.4.0099) available online. We have reported the systematic review and meta-analysis in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses – PRISMA statement [8].

2.1. Eligibility criteria

• •We included studies with patients who tested for SARS-Cov-19 using RT-PCR and who were evaluated using the IgM and IgG Antibody Test for SARS-CoV-2 Infection.

Population: patients tested by RT-PCR for SARS-CoV-2 and by a serologic test.

Intervention: IgM and IgG antibody test for SARS-CoV- 2 Infection.

• •Comparator: RT-PCR (reference test).

  Outcome: SARS-CoV-2 Infection.

• •Study designs: Cross-sectional, Cohort or Case-control.

2.2. Search strategy

A search strategy was developed using the following terms, both as text words and, as appropriate, Medical Subjects Heading (MeSH) or equivalent subject heading/thesaurus terms: “SARS-CoV” OR “COVID-19” OR “COVID” OR “COVID-19 diagnostic testing” AND “test”. The following databases were searched from their inception forwards for potentially eligible studies without language restrictions published up to September 2020: MEDLINE, EMBASE, Cochrane Library, Web of Science and Cumulative Index to Nursing and Allied Health Literature (CINAHL Database). Cross-referencing from retrieved studies was also conducted. We also performed searches for unpublished or ongoing trials. The search was performed on Octuber 2020. Before completing this review, we performed an additional search in each database and registration platform to guarantee that the most recent studies were included.

2.3. Studies selection

Two reviewers (ACM, GSP) screened all titles and abstracts for relevance in Rayyan (rayyan.qcri.org). Full texts of each potentially eligible study were retrieved and reviewed independently by the two reviewers (ACM, GSP). Disagreements between reviewers were resolved by a third reviewer (TC).

2.4. Data extraction

Data from all studies was independently extracted by two reviewers (ACM, GSP), and combined to construct a definitive dataset. Data was extracted from pre-piloted data extraction tables for study characteristics and placed on a 2 × 2 table for study results. However, when this was not possible, corresponding authors were contacted in order to obtain this data.

2.5. Quality assessment

All studies were assessed for their methodological quality using the QUADAS 2 (Quality Assessment of Diagnostic Accuracy Studies) criteria [9]. This was performed independently by two reviewers (TC and ACM). A figure representing each appraisal was provided in the results section.

2.6. Data synthesis

The primary analysis assessed the pooled sensitivity, specificity, diagnostic odds ratio (DOR), ROC curves, and negative and positive predictive values. Meta-analysis was conducted using the random effects model implemented with Metadisc 1.4 and STATA 16 softwares[10, 11]. We calculated the point estimate and 95% confidence interval of pooled sensitivity and pooled specificity data.

3. Results

3.1. Study identification and eligibility

Among the 1885 studies identified from electronic database searches and reference lists, 112 studies were excluded by duplication, leaving 1773 studies for reading titles and abstracts, and then we excluded 1703 published studies through title and abstract screening (Fig. 1 ). A total of 70 full-text studies were retrieved. Of those, 57 studies were excluded after further scrutiny: 9 studies included only confirmed cases, 31 were different design or editorial, 17 did not have serology and 1 did not have data for extraction. We included a total of 45 studies in the review (Li et al., 2020 [4], Guo et al., 2020[5], Jin et al., 2020[7], Adams et al., 2020 [12], Andrey et al., 2020 [13], Algaissi et al., 2020 [14], Ayouba et al., 2020 [15], Benavid et al., 2020 [16], Cassaniti et al., 2020 [17], Chan et al., 2020[18], Charlton et al., 2020 [19], Dohla et al. 2020 [20], Freeman et al., 2020 [21], Guedez-Lopez et al., 2020 [22], Halsemann et al., 2020 [23], Harb et al., 2020 [24], Herroelen et al., 2020 [25], Jia et al. 2020 [26], Lassauniere et al., 2020 [27], Lin et al. 2020 [28], Liu et al., 2020 (A) [29], Liu et al., 2020 (B)[30], Liu et al., 2020 (C) [31], Long et al., 2020 [32], Marinis et al., 2020[33], Naaber et al., 2020 [34], Pan et al., 2020 [35], Pancrazzi et al., 2020 [36], Paiva et al., 2020 [37], Paradisoet al., 2020 [38], Pegoraro et al., 2020 [39], Perera et al., 2020 [40], Perez-Garcia et al., 2020 [41], Pfluger et al., 2020 [42], Pieri et al., 2020 [43], Phipps et al., 2020 [44], Suhandynata et al., 2020 [45], Tan et al., 2020 [46], Tehrani et al., 2020 [47], Turbett et al., 2020 [48], Vásárhelyi et al., 2020 [49], Wang et al., 2020 [50], Yassine et al., 2020 [51], Zhao et al., 2020 [52], Zhang et al., 2020 [53], Zhong et al., 2020[54]). A complete list of excluded studies is available upon request.

Fig. 1.

PRISMA flow diagram of studies selections.

3.2. Study descriptions

Fourty-five primary studies were included in the analyses. A total of 13,650 patients met the inclusion criteria and were analyzed, including suspected cases, confirmed cases and healthy volunteers. The main characteristics of the included studies and samples are shown in Table 1 Table 2. shows the sum contingency table and analysis.

Table 1.

Characteristics of included studies.

Study	Country	Samples	Ntotal	N PCR+	N PCR-	Antibody test	Agemedia (range)years	Collect periodmedia (range)Days
Adams et al., 2020	United Kingdom	confirmed cases + pre-COVID samples	90	40	50	ELISA LFIA	NI	NI
Andrey et al., 2020	Switzerland	confirmed cases + healthy volunteers (not PCR tested)	91	46	45	Augurix RDT, rIFA Euroimmun (ELISA)	66 (50–76)	10 (5 −15) discriminated by (0–6) (7–14) (>15)
Algaissi et al., 2020	Saudi Arabia	confirmed cases + pre-COVID samples	135	10	125	ELISA S1 / ELISA N	24 a 75	discriminated by (0–6) (7–14) (>15)
Ayouba et al., 2020	France	confirmed cases + pre-COVID samples	138	61	77	xMAP assay IgG	72 (33–99)	>14
Benavid et al., 2020	USA	confirmed cases + pre-COVID samples	408	37	371	Premier Biotech (LFIA)	NI (Zero - >65)	NI
Cassaniti et al., 2020	Italy	suspected cases + healthy volunteers	110	68	42	Viva-DiagTM (LFIA)	IU 73.5 (38–86) ER 61.5 (33–97) healthy: 38.5 (25–69)	7 (4–11)
Chan et al., 2020	USA	confirmed cases + pre-COVID samples	155	99	56	Easy Check (LFIA)	NI	discriminated by (0–6) (7–13) (>14)
Charlton et al., 2020	Canada	confirmed cases + pre-COVID samples	92	42	50	Abbott IgG, Affinity, Bio-Rad (ELISA) BTNX, Biolidics, Deep blue, Getein, Genrui (LF (POC))	70.1 (34–102)	discriminated by (0–14) (14–21) (>21)
Dohla et al. 2020	Germany	suspected cases	49	22	27	rapid test	46	18,5
Freeman et al., 2020	USA	confirmed cases + pre-COVID samples with virus infections	99	519	618	ELISA	NI	>10
Guedez-Lopez et al., 2020	Spain	suspected cases	145	101	44	Sienna rapid Test (ICT) Wondfo (ICT) Prometheus (ICT)	50	Professionals 5 Patients 11
Guo et al., 2020	China	suspected cases	140	82	58	ELISA rN	NI	(1–39)
Halsemann et al., 2020	Germany	confirmed cases + healthy volunteers PCR negative	51	26	25	Epitope Ratio (ELISA) Euroimmun (ELISA) Elecsys	48 (20–73)	NI
Harb et al., 2020	EUA	confirmed cases + pre-COVID samples	447	65	382	Abbott Architect i2000 LIAISON® Elecsys on cobas® e601	NI	2–45
Herroelen et al., 2020	Belgium	confirmed cases + pre-COVID samples	225	169	56	Euroimmun (ELISA) Diasorin immunochromatographic Gene S/N and Innovita	71 (53–86)	0–39
Jia et al. 2020	China	suspected cases	57	24	33	ICT	NI	NI
Jin et al., 2020	China	suspected cases	66	43	33	chemiluminescence immunoassay, Shenzhen YHLO Biotech	47 (7–74)	18 (11–23)
Lassauniere et al. 2020	Denmark	confirmed cases + pre-COVID samples with virus infections	112	82	30	Euroimmun (ELISA) POC: Dynamiker, CTK Biotech Autobio diagnostics, Artron, Acro Biotech, Alltest biotech	Controls: 18–64 Cases: NI	discriminated by (0–6) (7–13) (14–20) (>21)
Li et al., 2020	China	suspected cases	525	397	128	Jiangsu Medomics (LFIA)	NI	(8–33)
Lin et al. 2020	China	confirmed cases + healthy volunteers + samples with pulmonar infection	159	79	80	CLIA and ELISA	42	13
Liu et al., 2020 (A)	China	confirmed cases + healthy volunteers	314	214	100	Lizhu (ELISA rN) Hotgen (ELISA rS)	NI	15 (0–55) discriminated by (0–5) (6–15) (>16)
Liu et al., 2020 (B)	China	suspected cases	179	90	89	ICT	controls: 56 cases: 76	NI
Liu et al., 2020 (C)	China	suspected cases	238	153	85	ELISA	55 (38–65)	0 - >16
Long et al., 2020	China	confirmed cases + close contacts	449	301	148	Bioscience (MCLIA)	47 (34–56)	(0–19)
Marinis et al., 2020	Sweden	confirmed cases + pre-COVID samples	76	37	39	LFIA	Controls: 37,4 ± 8,3 Cases: 71 ± 8	>14
Naaber et al., 2020	Estonia	confirmed cases + pre-COVID samples	197	97	100	SNIBE (CLIA) Euroimmun (ELISA) Abbott (CMIA) Epitope Ratio (ELISA) Diasorin (MCLIA) Biosensor (ICT) Lips-S-RBD Luciferase (ELISA) Lips-N Luciferase (ELISA)	59 (21–100)	>7
Pan et al., 2020	China (Hong Kong)	suspected cases	38	31	7	GICA	58 (20–96)	Discriminated by (1–7) (8–14) (>15)
Pancrazzi et al., 2020	Italy	suspected cases	516	73	413	Acro Biotech (POC)	53,7	NI
Paiva et al., 2020	USA	confirmed cases + pre-COVID samples + healthy volunteers	1212	28	1184	STANDARD Q LFIA Abbott (CMIA) Wondfo LFIA	59.5 ± 1.9	>14 dias
Paradisoet al., 2020	Italy	suspected cases	190	120	70	Viva-DiagTM (LFIA)	58.5	0 - >15
Pegoraro et al., 2020	Italy	suspected cases	226	159	67	Euroimmun (ELISA) Maglumi (CLIA) VivaDiag (ICT) PRIMA (ICT)	control 49 ± 9 cases 58 ± 20	Discriminated by Assyntomatic (0–5) (6–8) (9–10) (11–15) (>15)
Perera et al., 2020	China (Hong Kong)	confirmed cases + healthy volunteers	224	24	200	ELISA rS	(28–80)	Discriminated by (5–9) (11–18)
Perez-Garcia et al., 2020	Spain	confirmed cases + pre-COVID samples + samples with pulmonar infection	163	55	108	AllTest Biotech, ICT	62 (51–74)	Discriminated by (1–7) (8–14) (>15)
Pfluger et al., 2020	Germany	confirmed cases + pre-COVID samples	357	37	320	Euroimmun ELISA) Diasorin (MCLIA) Wantai (ELISA) Siemens (MCLIA)	18–70	Discriminated by (1–10) (>10)
Pieri et al., 2020	Italy	suspected cases	80	40	40	SNIBE (CLIA) Euroimmun (ELISA)	NI	Discriminated by (1–10) (11–45)
Phipps et al., 2020	USA	suspected cases	172	76	97	Abbott 06R86 (CMIA)	NI	Discriminated by (1–7) (8–14) (>15)
Suhandynata et al., 2020	USA	suspected cases	289	54	235	Diazyme (MCLIA)	(25–91)	Discriminated by (1–7) (8–14) (>15)
Tan et al., 2020	Singapure	confirmed cases + pre-COVID samples	336	173	163	Abbott (CMIA) Siemens (ELISA) Beckman (CMIA) Abbott (CMIA)	NI	Discriminated by (1–6) (7–13) (14–20) (21–64)
Tehrani et al., 2020	EUA	confirmed cases + pre-COVID samples	400	100	300	Chembio (LFIA)	NI	NI
Turbett et al., 2020	USA	confirmed cases + pre-COVID samples	1338	70	1268	Abbott (CMIA)	NI	Discriminated by (1–7) (8–14) (>15)
Vásárhelyi et al., 2020	Hungary	workers screening	1029	31	998	Clungene (LFIA)	NI	NI
Wang et al., 2020	China	confirmed cases + excluded cases with comorbidities	86	14	72	ELISA	NI	(3–7)
Yassine et al., 2020	Qatar	confirmed cases + pre-COVID sample	103	33	70	Epitope Ratio, AnshLabs DiaPro, Nova tec Lionex (ELISA)	controls: 36 (30–45) cases: 48 (40–57)	Discriminated by (1–6) (7–13) (>14)
Zhao et al., 2020	China	confirmed cases + healthy volunteers	481	69	412	ELISA	NI	NI
Zhang et al., 2020	China	not specified	782	122	660	GICA	NI	NI
Zhong et al., 2020	China	confirmed cases + healthy volunteers	347	47	300	ELISA rS- RBD CLIA	48 (18–82)	NI

Footnote: CLIA: Chemiluminescence ImmunoAssay; CMIA: Chemiluminescent Magnetic Immunological Assay; ELISA rN: enzyme-linked immunosorbent assay recombinant nucleocapsid protein-based kit; ELISA rS: enzyme-linked immunosorbent assay RBD of the recombinant S polypeptide-based kit; ELISA: Enzyme-linked immunossorbent assay; GICA: SARS-COV-2 ANTIGEN; ICT: immunochromatographic strip rapid serology tests; LFIA: Lateral Flow Immunoassay; MCLIA: magnetic chemiluminescence enzyme immunoassay; NI: Not Informed; PCR: Polymerase Chain Reaction; rIFA: rapid Immunofiltration Assay.

Table 2.

Accuracy of serology, reference standard: PCR. All studies pooled and discriminated by collect period and technique. Different techniques were not applied to the same sample.

All studies
Analysis	IgM and/or IgG% (CI 95%)	IgM only% (CI 95%)	IgG only% (CI 95%)
Sensitivity	80.0 (72.0–86.0)	48.5 (52.0–65.5)	64.9 (53.0–75.2)
Specificity	97.0 (94.0–98.0)	95.6 (90.8–98.0)	97.3 (95.2–98.5)
DOR	131 (56–303)	20.59 (0.10–41.09)	66.76 (8.94–124.58)
AUC	0.96 (0.93–0.97)	*	*
PPV	84.5 (83.3 – 85.7)	74.5 (71.9–76.9)	85.8 (83.9–87.4)
NPV	90.1 (89.5 – 90.7)	83.0 (82.0–84.0)	87.5 (86.7- 88.2)
TP	2806	861	1305
FP	516	299	220
FN	982	802	893
TN	9346	4102	6448
N total	13,650	6064	8866
Studies included	45	24	30

Discriminated by collect period
Analysis	Around 0–6th dayIgM and/or IgG% (CI 95%)	Around 7th-14th daysIgM and/or IgG% (CI 95%)	Around 15th day and afterIgM and/or IgG% (CI 95%)
Sensitivity	42.0 (24.0–63.0)	65.0 (64.9–77.3)	91.0 (85.0–94.0)
Specificity	97.0 (93.0–99.0)	97.0 (93.0–99.0)	98.0 (95.0–99.0)
DOR	25 (8–81)	72 (26–200)	461 (139–1534)
AUC	0.90 (0.87–0.92)	0.91 (0.88–0.93)	0.98 (0.96–0.99)
PPV	68.0 (61.5–73.9)	85.5 (81.7–88.6)	92.0 (90.2–93.5)
NPV	92.0 (90.9–92.9)	92.8 (91.7–93.7)	97.2 (95.7–96.7)
TP	144	340	946
FP	68	59	83
FN	199	181	102
TN	2324	2497	4156
N total	2735	3077	5287
Studies included	12	13	19

Discriminated by technique
Analysis	ELISAIgM and/or IgG% (CI 95%)	LFQIIgM and/or IgG% (CI 95%)	MCLIAIgM and/or IgG% (CI 95%)	Immunochroma-tographicIgM and/or IgG% (CI 95%)
Sensitivity	82.0 (72.0–89.0)	75.0 (59.0–87.0)	85.0 (71.0–93.0)	72.0 (48.0–88.0)
Specificity	96.0 (91.0–98.0)	97.0 (94.0–99.0)	98.0 (97.0–99.0)	88.0 (62.0–97.0)
DOR	112 (36–344)	115 (29–452)	334 (123–904)	19 (5–77)
AUC	0.95 (0.93–0.97)	0.96 (0.94–0.98)	0.99 (0.98–1.00)	0.86 (0.83–0.89)
PPV	86.0 (84.2–87.7)	94.1 (93.4–94.8)	96.6 (95.2–97.5)	78.4 (74.8–81.6)
NPV	85.1 (83.9–86.3)	91.3 (90.5–91.9)	87.6 (86.4–88.7)	77.2 (75.6–79.4)
TP	1330	834	906	410
FP	215	228	32	120
FN	531	277	371	256
TN	3048	4463	2634	972
N total	2072	5802	3943	758
Studies included	23	13	15	8

Discriminated by characteritics of patients included
Analysis	Confirmed cases + pre-COVID samples OR healthy volunteers(PCR tested or not)	Suspected cases
Sensitivity	83.0 (75.0–90.0)	74.0 (53.0–88.0)
Specificity	99.0 (98.0–99.0)	89.0 (73.0–96.0)
DOR	456 (214–972)	24 (14–57)
AUC	0.99 (0.98–1.00)	0.89 (0.86–0.92)
PPV	95.8 (94.6–96.7)	85.2 (83.3–86.9)
NPV	93.0 (92.3–93.6)	72.8 (70.8–74.8)
TP	1286	1217
FP	56	211
FN	414	531
TN	5682	1425
N total	7438	3384
Studies included	23	17
INCLUDED STUDIES: ALL STUDIES – IgM and/or IgG (Adams et al., 2020; Andrey et al., 2020; Algaissi et al., 2020; Ayouba et al., 2020; Benavid et al., 2020; Cassaniti et al., 2020; Chan et al., 2020; Charlton et al., 2020; Dohla et al. 2020; Freeman et al., 2020; Guedez-Lopez et al., 2020; Guo et al., 2020; Halsemann et al., 2020; Harb et al., 2020; Herroelen et al., 2020; Jia et al. 2020; Jin et al., 2020; Lassauniere et al. 2020; Li et al., 2020; Lin et al. 2020; Liu et al., 2020 (A); Liu et al., 2020 (B); Liu et al., 2020 (C); Long et al., 2020; Marinis et al., 2020; Naaber et al., 2020; Pan et al., 2020; Pancrazzi et al., 2020; Paiva et al., 2020; Paradisoet al., 2020; Pegoraro et al., 2020; Perera et al., 2020; Perez-Garcia et al., 2020; Pfluger et al., 2020; Pieri et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Tehrani et al., 2020; Turbett et al., 2020; Vásárhelyi et al., 2020; Wang et al., 2020; Yassine et al., 2020; Zhao et al., 2020; Zhang et al., 2020; Zhong et al., 2020); IgM only (Andrey et al., 2020; Algaissi et al., 2020; Charlton et al., 2020; Guedez-Lopez et al., 2020; Guo et al., 2020; Herroelen et al., 2020; Jia et al., 2020; Jin et al., 2020; Li et al., 2020; Lin et al., 2020; Liu et al., 2020 (A); Liu et al., 2020 (B); Marinis et al., 2020; Pan et al., 2020; Pancrazzi et al., 2020; Paiva et al., 2020; Pegoraro et al., 2020; Perez-Garcia et al., 2020; Pieri et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tehrani et al., 2020; Vásárhelyi et al., 2020; Wang et al., 2020); IgG only (Andrey et al., 2020; Algaissi et al., 2020; Charlton et al., 2020; Guedez-Lopez et al., 2020; Halsemann et al., 2020; Harb et al., 2020; Herroelen et al., 2020; Jia et al., 2020; Jin et al., 2020; Lassauniere et al., 2020; Li et al., 2020; Lin et al., 2020; Liu et al., 2020 (A); Liu et al., 2020 (B); Marinis et al., 2020; Naaber et al., 2020; Pan et al., 2020; Pancrazzi et al., 2020; Paiva et al., 2020; Pegoraro et al., 2020; Perez-Garcia et al. 2020; Pfluger et al., 2020; Pieri et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Tehrani et al., 2020; Turbett et al., 2020; Vásárhelyi et al., 2020; Yassine et al., 2020); DISCRIMINATED BY COLLECT PERIOD – Around 0–6th day (Andrey et al., 2020; Algaissi et al., 2020; Charlton et al., 2020; Liu et al., 2020 (A); Pan et al., 2020; Perez-Garcia et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Turbett et al., 2020; Wang et al., 2020; Yassine et al., 2020); Around 7th-14th days (Andrey et al., 2020; Algaissi et al., 2020; Cassaniti et al., 2020; Guedez-Lopez et al., 2020; Liu et al., 2020 (A); Pan et al., 2020; Perera et al., 2020; Perez-Garcia et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Turbett et al., 2020; Yassine et al., 2020); Around 15th day and after (Andrey et al., 2020; Algaissi et al., 2020; Ayouba et al., 2020; Chan et al., 2020; Charlton et al., 2020; Jin et al., 2020; Li et al., 2020; Liu et al., 2020 (A); Marinis et al., 2020; Pan et al., 2020; Paiva et al., 2020; Perera et al., 2020; Perez-Garcia et al., 2020; Pfluger et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Turbett et al., 2020; Yassine et al., 2020); DISCRIMINATED BY TECHNIQUE – ELISA (Adams et al., 2020; Andrey et al., 2020; Algaissi et al., 2020; Charlton et al., 2020; Freeman et al., 2020; Guo et al., 2020; Halsemann et al., 2020; Herroelen et al., 2020; Lassauniere et al., 2020; Lin et al., 2020; Liu et al., 2020 (A); Liu et al., 2020 (C); Naaber et al., 2020; Pegoraro et al., 2020; Perera et al., 2020; Pfluger et al., 2020; Pieri et al., 2020; Tan et al., 2020; Tehrani et al., 2020; Yassine et al., 2020; Wang et al., 2020; Zhao et al., 2020; Zhong et al., 2020); LFQI (Adams et al., 2020; Benavid et al., 2020; Cassaniti et al., 2020; Chan et al., 2020; Charlton et al., 2020; Guedez-Lopez et al., 2020; Li et al., 2020; Marinis et al., 2020; Paiva et al., 2020; Paradisoet al., 2020; Tehrani et al., 2020; Turbett et al., 2020; Vásárhelyi et al., 2020); MCLIA (Halsemann et al., 2020; Harb et al., 2020; Herroelen et al., 2020; Lin et al., 2020; Jin et al., 2020; Long et al., 2020; Naaber et al., 2020; Paiva et al., 2020; Pegoraro et al., 2020; Pfluger et al., 2020; Pieri et al., 2020; Phipps et al., 2020; Suhandynata et al., 2020; Tan et al., 2020; Zhong et al., 2020); Immunochromatographic (Herroelen et al., 2020; Jia et al., 2020; Liu et al., 2020 (B); Pan et al., 2020; Pegoraro et al., 2020; Perez-Garcia et al., 2020; Wang et al., 2020; Zhang et al., 2020); DISCRIMINATED BY CHARACTERITICS OF PATIENTS INCLUDED – Confirmed cases + pre-COVID samples OR healthy volunteers (Adams et al., 2020; Algaissi et al., 2020; Andrey et al., 2020; Ayouba et al., 2020; Benavid et al., 2020; Chan et al., 2020; Charlton et al., 2020; Halsemann et al., 2020; Harb et al., 2020; Herroelen et al., 2020; Liu et al., 2020 (A); Marinis et al., 2020; Naaber et al., 2020; Paiva et al., 2020; Perera et al., 2020; Pfluger et al., 2020; Tan et al., 2020; Tehrani et al., 2020; Turbett et al., 2020; Yassine et al., 2020; Zhao et al., 2020; Zhong et al., 2020); Suspected case (Cassaniti et al., 2020; Dohla et al., 2020; Guedez-Lopez et al., 2020; Guo et al., 2020; Jia et al., 2020; Jin et al., 2020; Li et al., 2020; Liu et al., 2020 (B); Liu et al., 2020 (C); Long et al., 2020; Pan et al., 2020; Pancrazzi et al., 2020; Paradisoet al., 2020; Pegoraro et al., 2020; Pieri et al., 2020; Phippse et al., 2020; Suhandynata et al., 2020).

Legend: FLQI: lateral flow qualitative immunoassay; ELISA rN: enzyme-linked immunosorbent assay recombinant nucleocapsid protein-based; MCLIA: magnetic chemiluminescence enzyme immunoassay.

3.3. Quality assessment

QUADAS-2 was performed considering the following categories: selection of patients, index and reference test, flow and timing (Fig. 2 ). Regarding selection, three studies was considered "unclear"[15, 16, 54]. Regarding index fourteen studies were considered "unclear" [12, 14, 15, [19], [20], [21], [22], 24, 43, [46], [47], [48], 51, 54], since these studies did not described blinding of workers involved in test collection and analyses, but it was considered as a source of "low concern".  Regarding the reference tests, all studies received the same evaluation; limitations of PCR as a reference test are mentioned in the discussion. Regarding flow and timing, twenty-two studies performed "unclear" [7, 12, 13, 15, [21], [22], [23], [24], [25], 27, [32], [33], [34], [40], [41], [42], 44, [46], [47], [48], 51, 54] because did not show enough information about intervals between tests.

Fig. 2.

Assessment of risk of bias.

3.4. Characteristics of included studies

Eighteen studies were from Europe [12, 13, 15, 17, 20, 22, 23, 25, 27, 33, 34, 36, 38, 39, [41], [42], [43], 49]; China was the origin of fifteen studies [4, 5, 7, 26, [28], [29], [30], [31], [32], 35, 40, 50, [52], [53], [54]], two of these from Hong Kong [35,40]; there were nine studies from USA (United States of America) [16, 18, 21, 24, 37, 44, 45, 47, 48] and one from each of follow: Canada [19], Singapore [16], Qatar [51] and Saudi Arabia [14]. Regarding technique, thirteen studies used lateral flow qualitative immunoassay (LFQI) [4,12,[16], [17], [18], [19], 22, 33, 37, 38, [47], [48], [49]] by different manufactures; Twenty-three studies applied Enzyme-Linked Immunosorbent Assay (ELISA), using recombinant nucleocapsid protein-based (rN) and recombinant S polypeptide-based kit (rS) [5,[12], [13], [14],19,21,23,25,[27], [28], [29],31,34,39,40,42,43,46,47,[50], [51], [52],54]. Fifteen used chemiluminescence immunoassay (MCLIA) [7, [23], [24], [25], 28, 32, 34, 37, 39, [42], [43], [44], [45], [46], 54], by different manufactures, and eight used Colloidal gold-based immunochromatographic (ICG) strip assay [41, 25, 26, 30, 35, 39, 50, 53].

3.5. Accuracy of serology – IGM and IGG

Results were discriminated, as shown in primary studies, by immunoglobulin (“IgM” and “IgG” separately and “IgM, IgG or both” reactivity) but some kits only tested both subclasses together (“IgM, IgG or both”). The pooled sensitivity and specificity, regardless of collection moment, was 80,0% (95% CI 72.0–86.0) and 97.0% (95% CI 94.0–98.0) for IgM and/or IgG, respectively. Serology considering immunoglobulins M and G together had a high accuracy performance, as a DOR (Diagnostic Odds Ratio) 131 and AUC (Area Under the Curve) 0.96. NPV on this sample was 90.1% (95% CI 89.5 – 90.7) and PPV, 84.5% (95% CI 83.3 – 85.7). This data and additional ones are described in Table2.

3.6. Sensitivity analysis

We considered the importance of describing the results divided by technique, manufacturer, sample selection, and mainly by period of blood collection, considering the number of days since the first symptoms. However, analysis by brand was not possible since primary studies were done using a variety of them.

These analyses are explained on Table 2. A comparison of the techniques employed was not possible since most primary studies did not report results from more than one test applied to the same sample results shown on Table 2 must be considered separated one from de other. Considering the blood collection period, in cases where it happened around the fifteenth day and after, the test presented a better performance, with pooled sensitivity of 91.0% (95% CI 85.0–94.0), specificity of 98.0% (95% CI 95.0–99.0), DOR 461 and AUC 0.98, for immunoglobulins M and G together – Fig. 3;. Areas Under the Curve comparing performance evolution by blood collection period are on Fig. 4 . Subanalyses of rapid diagnostics test for point-of-care were not performed since in primary studies that used these tolls the sample was not whole blood, as in practice, but serum or plasma.

Fig. 3.

Forest plot.

Fig. 4.

SROC curve: summary receiver operating characteristics curve.

Subanalyses of rapid diagnostics test for point-of-care were not performed since in primary studies that used these tolls the sample was not whole blood, as in practice, but serum or plasma.

An important find was the subanalyses by “characteristics of patients included”. Mostly studies included only “suspected cases”, a cohort that configures a diagnostic scenario, and “confirmed cases + pre-COVID samples, or healthy volunteers (PCR tested or not)”, characterizing a theorical scenario with “true positives” and “true negatives” considering clinical feature. The AUC rised from 0.89 in “suspected cases” to 0.99 in “confirmed cases + pre-COVID samples”. It suggests that well-established clinical criteria screening will improve tests performance. Complete analysis shown on Table2.

3.7. Heterogeneity

The I² index aims to quantify the dispersion of effect sizes in a meta-analysis.

For the main analyses, including all techniques, I2 was substantial heterogeneity in sensitivity and specificity was found among studies in most analyses. In the Main analysis, “IgM, IgG or both all techniques pooled”, I2 was 96.56 (95%CI 96.02–97.10) Q 1337.77 (df 46.0) for sensitivity and 99.07 (95%CI 98.98–99.17) Q 4949.38 (df 46.0) for specificity. Regarding the period of sample collection, from 7th to 14th days, less heterogeneity was found, I2 was 84.80 (95%CI 77.85–91.75) Q 85.52 (df 13.0) for sensitivity and 96.57 (95%CI 95.96–97.58) Q 378.92 (df 13.0) for specificity; and to 15th day onwards I2 was 89.01 (95%CI 85.21–82.81) Q 172.88 (df 19.0) for sensitivity and 98.81 (95%CI 98.61–99.02) Q 1599.86 (df 19.0) for specificity. Egger's regression identified p = 0.00 for Main analyses, as Begg's test performed, suggesting that there was a “small study effect” for publish bias Fig. 5. shows funnel plot the Main analysis, “IgM, IgG or both all techniques pooled”.

Fig. 5.

Funnel plot.

This heterogeneity could be explained by the difference in technique, antigen, manufacturer and by period of blood collection. Exclusion of any study from analysis did not significantly decrease heterogeneity in the analysis. Meta-regression per continent, technique and characteristics of sample did not decreased heterogeneity as well.

4. Discussion

In response to severe acute respiratory syndrome coronavirus 2 (SARS–CoV-2), approximately 3 billion people were placed under social distancing measures. Around the world, strategies have been deployed to break the spread of the virus. Identifying who is infected and contagious is a key point underpinning success of those interventions [55].

Development of an antibody response is influenced by a number of host-dependent factors, including age, severity of disease, among others [56]. In the case of SARS–CoV-2, and early studies suggest that the majority of patients seroconvert from 7 to 11 days after infection, although some patients may develop antibodies sooner [57,58]. Serum IgG amounts can rise at the same time or earlier than those of IgM against SARS-CoV-2. Cochrane COVID-19 Diagnostic Test Accuracy Group published a Systematic Review exploring serological tests accuracy for COVID-19 including people suspected of current or previous SARS-CoV-2 infection, or where tests were used to screen for infection, with TR-PCR as reference test or not (clinical diagnostic criteria as reference). Despite these differences regarding selection and reference criteria, they identified the same sensitivity of this study in “15th day onwards” sample, 91.4% (95% CI 87.0–94.4) previous and 91.0% (85.0–94.0) the present study; and different result for “7th – 14th days” sample: 72.2% (95% CI 63.5–79.5) previous and 65.0% (64.9–77.3) the present study [59].

One special challenge when evaluating serological tests is the huge diversity of techniques and manufactures. To exemplify, in the middle of April of 2020, 91 manufacturers had notified the Food and Drug Administration (FDA), of the United States of America, that they were offering serologic tests for commercial use, and four products have received FDA Emergency Use Approval [60]. The present study includes data resulting from four different techniques, from different kits, which allows for a current external validation of these tests but without enough information regarding reagents and overall diversity among the tests. An important consideration of the analysis is that rapid tests when performed as point-of-care tests, utilize whole blood as sample, while the investigated studies utilized methods for sample fractionation that increase performance of the test. In July 2020, Lisboa Bastos et al [61]. published a systematic review that showed similar results regarding ELISA and discrepant regarding other techniques performances. For ELISA, sensitivity has 84.3% (CI 95% 75.6 - 90.9) and specificity was 97.6% (CI 95% 93.2–99.4) and the present study identified 82.0% (CI 95% 72.0–89.0) and 96.0% (CI 95% 91.0–98.0), respectively, with a similar number of participants. For MCLIA (chemiluminescence assay), sensitivity has 97.8% (CI 95% 46.2–100) and specificity was not estimated, and the present study identified 85.0% (CI 95% (71.0–93.0) and 98.0% (CI 97% 91.0–99.0), respectively, probably because of the included number of participants on this study was four times fold, comparing to previous one. Immunochromatographic was not performed by that Systematic Review.

It is not known for certain whether individuals infected with COVID-19 who subsequently recover will be protected from future infection or how long protective immunity may last.

5. Limitations

Regarding weaknesses of this systematic review, the main point is the reference test. The PCR-RT test identifies the presence of the virus itself through virus RNA replication and because of that it is considered the golden standard. However, it can identify only the acute phase of the disease, which compromises its specificity and can influence serology evaluation. Primary studies did not include patients with COVID‐19 who may have had a false negative result on PCR, it may have affected test accuracy, but it is impossible to identify by how much.

This study was careful to analyze Negative and Positive Prediction Values to improve the information provided. However, it is important to remember that these measures are influenced by the disease prevalence in the population. Even though most studies only included suspected cases, characterized in a hospital scenario, some studies included healthy volunteers with the aim of calculating specificity, as described in Table 1. Therefore, the global sample of this systematic review is the reflection of primary study choices. Primary studies included in this systematic review did not report measures of immunoglobulin A (IgA). Both previous systematic Reviews cited in this study have not shown quantitative data about heterogeneity, so that was not possible to compare it [59,61].

The severe acute respiratory syndrome coronavirus 2 is an emerging infection with many unknowns. Currently, the World Health Organization (WHO) does not recommend the use of rapid diagnostics test for point-of-care, but such tests contribute for disease surveillance and epidemiologic research [62]. Our analysis supports the use of serological tests in suspected patients and provides scientific evidence for guiding infection control policies and therapies. Serological tests may have a role complementing other testing in individuals presenting later, when RT‐PCR tests are negative, or are not done, mainly after 7 days of symptoms. Results of present study cannot assess the utility of these tests for seroprevalence surveys for public health management purposes. These results can be applied mostly for suspected cases, in hospital scenario.

6. Future research directions

It is important to point out that results consist of the basis of preliminary analysis and that further investigation including consecutive sampling, standardization of methods, and well-defined use-case must be incorporated to generate robust evidence, simultaneous to a rapidly evolving epidemiology

7. Conclusion

This study supports that serology is a group of tests with high accuracy, mainly following the second week after infection, however the high heterogeneity cautions for careful application. Well-defined use-case for the tests (e.g. point-of-care versus immunosurveillance and technique) are necessary to be in place in order to benefit from the results of serological tests.

Role of the funding source

The present study has no funding sources.

Declaration of Competing Interest

The present study has no conflicts of interest.