Comparison of test performance of commercial anti-SARS-CoV-2 immunoassays in serum and plasma samples

Highlights

• •FDA-EUA approved anti-SARS-CoV-2 tests compare favorably with others.
• •Diagnostic sensitivity remains behind the manufacturer’s specifications.
• •Plasma and serum cannot both be used equally for anti-SARS-CoV-2 IgG detection.
• •Immune response relies on disease severity, thus need to be considered for validation.

Abstract

Background

For epidemiologic, social and economic reasons, assessment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection prevalence and immunity are important to adapt decisions to current demands. Hence, immunoassays for detection of anti-SARS-CoV-2 antibodies are introduced rapidly without requiring FDA emergency use authorization approval. Thus, evaluation of test performance predominantly relies on laboratories. This study aimed to evaluate the test performance of recently launched commercial immunoassays in serum and plasma samples.

Methods

51 serum samples from 26 patients with confirmed SARS-CoV-2 infection after end of quarantine and 25 control patients were analyzed using anti-SARS-CoV-2 IgG immunoassays from Roche, Euroimmun and Epitope to assess diagnostic sensitivity and specificity. 20 matching pairs of serum and plasma samples were included to analyze comparability between different specimens.

Results

Overall, a diagnostic sensitivity of 92.3%, 96.2–100% and 100% with a respective diagnostic specificity of 100%, 100% and 84–86% for the immunoassays from Roche, Euroimmun and Epitope were determined. In total, 84–96% of samples were correctly classified as negative and 92.3–95.2% as positive. The level of concordance between plasma- and serum-based testing diverged between the assays (Epitope r² = 0.97; Euroimmun r² = 0.91; Roche r² = 0.76).

Conclusions

The immunoassays from Euroimmun and Roche revealed a higher specificity than the Epitope assay without a substantial drop of diagnostic sensitivity. Significant differences between plasma- and serum-based testing highlights the need for determination of appropriate cut-offs per specimen type. Hence, there is an urgent need for test harmonization and establishment of quality standards for an appropriate use of COVID-19 serological tests.

1. Introduction

The infection with novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was declared world pandemic by the World Health Organization (WHO) on 12th of March 2020 [1]. Since then, the number of infections and the global spread continuously increased. As of May 21st 2020, over five million confirmed SARS-CoV-2 infections have been reported worldwide and more than 330,000 people died due to SARS-CoV-2 caused acute severe respiratory disease, termed Coronavirus disease 2019 (COVID-19) [2]. As long as no appropriate vaccination is available, the only possibility to reduce the rapid spread of SARS-CoV-2 represents quarantine of infected individuals along with social lockdown/restrictions and enhanced hygiene [3]. The diagnosis of acute infection relies on qRT-PCR based viral detection in respiratory material. As the identification of infected persons is hampered by the high percentage of oligo- or asymptomatic patients [4] and the shortage of test material [5], [6], the number of infections worldwide is thought to be substantially underestimated [7], [8], [9]. For epidemiologic, social and economic reasons, determination and surveillance of the immune status within the population to estimate SARS-CoV-2 infection prevalence and herd immunity are of upmost importance [10]. Hence, immunoassays for detection of anti-SARS-CoV-2 immunoreactivity are gaining growing attention.

Currently, over 100 SARS-CoV-2 antibody assays have been CE-marked under EU Directive 98/79/EC [11]. The available test systems can be discriminated into rapid diagnostic tests (RDT), either antigen- or antibody based, enzyme-linked immunosorbent assays (ELISA) and chemiluminescent immunoassays (CLIA). The global health non-profit Foundation for Innovative New Diagnostics (FIND) provides an overview of available test systems along with their respective market readiness as well as sensitivity and specificity data [12]. Additionally, information about immunological tests with approval in the United States can be obtained from the Center for Health Security of Johns Hopkins University [13]. As of May 21st 2020, 11 serology tests have received FDA emergency use authorization (EUA) with performance data provided by FDA [14]. In detail, three RDTs (Cellex Inc., ChemBio, Autobio Diagnostics Co. Ltd.), 5 ELISAs (Ortho-Clinical Diagnostics, Inc., Mount Sinai Laboratory COVID-19 ELISA IgG Antibody Test, DiaSorin Inc., Bio-Rad, Euroimmun AG) as well as three CLIAs (Roche, Abbott Laboratories Inc., Wadsworth Center, New York State Department of Health) received FDA EUA approval, respectively [13].

In agreement with EUA, regulatory requirements have been reduced and FDA stated that in contrast to qRT-PCR based tests for viral detection, EUA approval is not mandatory for serology-based test systems – neither commercial nor laboratory-developed ones [15]. Consequently, laboratories are now forced to perform appropriate validation studies. Taken into consideration the high diversity of available immunoassays [16], the different materials used for testing (e.g. serum, plasma, sputum), and the limited number of peer-reviewed publications addressing this issue, diagnostic accuracy and optimal use of serological anti-SARS-CoV-2 testing still need to be elucidated [17].

Therefore, we compared the test performance of recently FDA EUA approved immunoassays from Roche and Euroimmun with one for research and surveillance purposes only approved ELISA from Epitope Diagnostics.

2. Material and methods

2.1. Patient recruitment and specimen collection

26 patients with qRT-PCR confirmed COVID-19 disease after end of quarantine or hospitalization as well as 25 control patients were prospectively recruited at University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Germany. The study was approved by the Institutional Review Board and informed written consent was obtained from each subject prior to sample collection and analysis. For evaluation of the medical history, each subject answered a standardized questionnaire. Depending on results, control patients were assigned to five different cohorts: (i) atypical respiratory infection within last three months and either SARS-CoV-2 qRT-PCR negative or not performed (Control 1), (ii) other respiratory viral infection diagnosed (Control 2), (iii) chronic diseases (e.g. autoimmune disease) (Control 3), (iv) contact to a COVID-19 positive patient, but negative SARS-CoV-2 qRT-PCR and no clinical symptoms (Control 4), (v) healthy controls (Control 5). Patient characteristics are provided in Table 1 . From all patients, serum and lithium heparin blood samples were collected.

Table 1.

Patient characteristics.

		Total
Number of Patients		n = 51(%)
Sex	m	18 (32.3)
	f	33 (67.7)
Age	Median	48.0
	range	20–73
SARS-CoV-2 infection	yes	26.0
	no	25.0
Controls	Control group 1	11 (21.2)
	Control group 2	1 (1.9)
	Control group 3	7 (13.5)
	Control group 4	2 (3.8)
	Control group 5	4 (7.7)

Control group 1 = atypical respiratory infection with negative testing; Control group 2 = other confirmed viral infection; Control group 3 = chronic disease; Control group 4 = contact to COVID Positive, but negative testing; Control group 5 = no comorbidities.

2.2. Blood-collection

Serum samples were stored at room temperature for at least one hour to allow appropriate clotting. Clotted serum samples and lithium heparin blood samples were centrifuged at 2000 g for 10 min at 18 °C within 4 h after sample collection. Serum and lithium heparin plasma was aliquoted and stored at −80 °C.

2.3. Analysis of anti-SARS-CoV-2 antibodies

All samples were analyzed by three different commercially available immunoassays including anti-SARS-CoV-2 IgG ELISA (Lot:E200429AG, Euroimmun, Germany), EDI^TM Novel Coronavirus COVID-19 IgG ELISA (Lot:P745U, Epitope Diagnostics, United States) and Elecsys Anti-SARS-CoV-2 (Lot:496298, Roche, Germany). All tests were performed using the same lot.

Tests systems from Euroimmun and Epitope Diagnostics are enzyme-linked immunosorbent assays (ELISA) in a 96-well-plate format detecting IgG directed against the S1 domain of viral spike protein (Euroimmun) and IgG directed against full length nucleocapsid protein (Epitope), respectively. The Euroimmun assay reports the ratio of sample absorbance divided by calibrator absorbance and results are interpreted as positive (ratio ≥ 1.1), borderline (ratio ≥ 0.8 – < 1.1), or negative (ratio < 0.8). The Epitope assay reports the optical density (OD) of the sample corrected by OD of negative control. The cut-offs used for interpretation of assay results (positive, negative and borderline) have to be calculated according to a provided formula and therefore might differ in every run. All patient samples were analyzed in triplicates (if not indicated otherwise) according to manufactureŕs instructions with the provided positive and negative controls determined in duplicates. Read-out was done using the Victor3™ Plate Reader (Perkin Elmer).

The Elecsys Anti-SARS-CoV-2 is an electrochemiluminescence immunoassay (ECLIA) detecting antibodies including IgG using a recombinant protein representing the nucleocapsid antigen. Results are reported as a cutoff index (COI) and interpreted as negative (COI < 1.0) or positive (COI ≥ 1.0). Positive and negative controls were prepared using pooled patient samples according to manufactureŕs instructions. Controls were analyzed in duplicates, patient samples in triplicates (if not indicated otherwise) on a Cobas e411 instrument (Roche) according to manufactureŕs instructions.

2.4. Statistical analysis

All statistical analyses were carried out using Abacus 2.0 (LABanalytics GmbH, www.labanalytics.de, 2016, Germany) and R version 3.0.1 (The R Foundation for Statistical Computing).

Results of data analysis are presented as descriptive statistics by mean, 95% confidence interval (CI), standard deviation (SD), and coefficient of variation (CV) as appropriate. Verification studies were performed for all three test systems. Imprecision was determined in duplicates over 4 days, repeatability was assessed by analyzing a negative control, positive control and a patient sample near the positive cut-off for each assay twenty times. Linear and Passing Bablok regression as well as Bland-Altman analysis were determined for method comparison between serum and plasma samples. Between-group differences were assessed by Student’s t-test. For all statistical analyses, p-values < 0.05 were considered statistically significant.

3. Results

For this study, 51 patients were prospectively enrolled and their serum samples as well as 20 matching lithium heparin plasma samples were analyzed in order to evaluate the test performance of three commercially available test systems for detection of anti-SARS-CoV-2 immunoreactivity.

3.1. Assay verification

For method verification, within- and between-run imprecision were determined using a positive and negative control as well as a pooled patient sample near the positive cut-off of the respective assays. To assess within-run imprecision, all samples were analyzed as 20 replicates. Due to limited reagent availability, between-run imprecision was determined over 4 days exclusively for the positive and negative control. Results are provided in Table 2 . The Roche assay achieved the highest level of inter-assay precision, whereas the Euroimmun test revealed the highest intra-assay repeatability. Overall, all tests revealed an acceptable precision. Accuracy could not be determined as all three tests are qualitative assays without target values for the provided control samples. Importantly, agreement of qualitative results was 100% for all three immunoassays.

Table 2.

Statistical evaluation of within- and between-run imprecision.

		Between-run Imprecision		In-Run Imprecision
		Negative Control	Positive Control	Negative Control	Positive Control	Cut-off Control
Epitope	Mean (OD)	0.11	0.90	0.16	0.29	0.30
	SD	0.01	0.16	0.01	0.03	0.03
	CV (%)	9.18	18.34	8.39	10.94	11.34
Euroimmun	Mean (Ratio)	0.38	2.27	0.45	2.21	1.47
	SD	0.05	0.37	0.02	0.07	0.08
	CV (%)	13.27	16.27	5.20	3.34	5.63
Roche	Mean (COI)	0.09	3.61	0.09	3.43	1.41
	SD	0.00	0.24	0.02	0.10	0.02
	CV (%)	4.23	6.76	22.02	2.79	1.61

Abbreviations: SD = standard deviation; CV = coefficient of variation; OD = optical density; COI = cutoff index.

3.2. Comparison of SARS-CoV-2 antibody detection in serum and lithium heparin plasma

In total, anti-SARS-CoV-2 antibodies were determined in 20 matching pairs of serum and lithium heparin plasma samples from 13 patients with previous confirmed SARS-CoV-2 infection and variable clinical presentation and 7 control patients. Results are listed in Supplemental Table 1.

Linear regression analysis of anti-SARS-CoV-2 antibody detection revealed a high level of correlation with r² = 0.97 for the COVID-19 IgG ELISA from Epitope as depicted in Fig. 1 . A good to moderate linear correlation with r² = 0.91 and r² = 0.76 were revealed for the Euroimmun and Roche test, respectively.

Fig. 1.

Linear regression analysis of anti-SARS-CoV-2 Ig measures in serum and plasma. The scatter shows the relation between serum levels (x-axis) and plasma levels (y-axis) for (A) anti-SARS-CoV-2 IgG determination using the EDI^TM Novel Coronavirus COVID-19 IgG ELISA from Epitope Diagnostics (r² = 0.97), (B) anti-SARS-CoV-2 IgG determination using the anti-SARS-CoV-2 IgG ELISA from Euroimmun (r² = 0.91), and (C) anti-SARS-CoV-2 Ig determination using the Elecsys Anti-SARS-CoV-2 ECLIA from Roche (r² = 0.75). Values above or below the drawn regression line represent increase or decrease compared to serum, respectively.

In order to exclude a systematic error, a Passing Bablok regression and Bland-Altman analysis were conducted. Results are provided in Table 3 ; plots are displayed in Supplemental Fig. 1. Passing Bablok regression showed a strong to poor positive correlation for both testing specimens for the three different immunoassays with Kendalĺs tau ranging from 0.82 (Epitope) to 0.60 (Roche). Additionally, a systematic or proportional error could be excluded for the two ELISAs (95%CI y-intercept = −0.1–0.006; 95%CI slope = 0.997–1.047 for Epitope; 95%CI y-intercept = −0.5–0.2; 95%CI slope = 0.897–1.301 for Euroimmun), but not for the ECLIA from Roche (95%CI y-intercept = −0.6–0.1; 95%CI slope = 0.331–0.775). In agreement with these results, Student́s t-test revealed a significant difference between the COI determined in matching serum and lithium plasma samples for the Elecsys Anti-SARS-CoV-2 assay (p < 0.05). All tests were performed using the same lot for all three assays included in this study.

Table 3.

Comparison of plasma and serum based anti-SARS-CoV-2 antibody detection.

		Epitope		Euroimmun		Roche
		Plasma	Serum	Plasma	Serum	Plasma	Serum
Absolute result	Mean	0.90	0.90	4.00	4.40	14.00	7.50
	Range	−0.01–1.48	−0.023–1.48	0.35–8.63	0.35–9.69	0.08–56.7	0.08–42.7
	Median	1.30	1.40	2.60	4.50	8.40	1.40
	Kendalĺs tau	0.82		0.77		0.60
	Mean bias (95% CI)	−11.65% (-36.13% − 12.83%)		23.12% (-8.61% − 54.85%)		−28.57% (-45.18% - −11.96%)
Interpreted result	Overall Agreement (%)	100.00		90.0*/95.0#		100.00
	Positive Agreement (%)	100.00		92.3*/100#		100.00
	Negative Agreement (%)	100.00		85.7*/87.5#		100.00
	Coheńs kappa	1.00		0.78 (0.49–1.07)*/0.89 (0.69–1.10)#		1.00

As all three immunoassays are qualitative test systems, qualitative results (positive versus negative) were evaluated for both testing materials. This comparison revealed a 100% overall agreement for the tests system from Epitope and Roche, respectively. The overall agreement for the Euroimmun assay ranges from 90% to 95% depending on whether borderline results are considered positive or negative (Table 3). In detail, these discrepant results were seen in plasma samples, whereas no false-positive or false-negative result was revealed for serum.

3.3. Method comparison

For evaluation of diagnostic performance of all three immunoassays, 51 serum samples obtained from 26 patients with previous confirmed SARS-CoV-2 infection and 25 control patients were eligible. In detail, 11 patients with an atypical respiratory infection within the last three months and excluded COVID-19 infection, 1 patient with a confirmed influenza infection, 7 patients suffering from chronic diseases, 2 patients with contact to a COVID-19 positive patient, but negative SARS-CoV-2 qRT-PCR, and 4 healthy subjects without comorbidities served as controls. Results obtained by the three different serological test systems are listed in Table 4 , more detailed information is provided in Supplemental Table 2.

Table 4.

Evaluation of diagnostic test performance.

	Epitope	Euroimmun	Roche
Overall Agreement (%)	98.0*/92.2#	100*/98.0#	96.10
Positive Agreement (%)	100*/100#	100*/96.2#	92.30
Negative Agreement (%)	96.0*/84.0#	100*/100#	100.00
Coheńs kappa	0.96 (0.88–1.04)*/0.84 (0.69–0.99)#	1.00*/0.96 (0.88–1.04)#	0.92 (0.82–1.03)

For the EDI^TM Novel Coronavirus COVID-19 IgG ELISA a diagnostic sensitivity of 100% and a diagnostic specificity of 96% if borderline results are counted as positive and of 84% if considered negative were obtained. The diagnostic sensitivity using the anti-SARS-CoV-2 IgG ELISA was 100% with borderline results considered positive and 96.2% if taken negative. The diagnostic specificity was 100% as was for the Elecsys Anti-SARS-CoV-2 ECLIA. Here, the diagnostic sensitivity was 92.3%.

In detail, 21/25 samples (84% overall negative agreement) were found to be negative by all three immunoassay if borderline results considered positive and 24/25 (96% overall negative agreement) if taken positive. Only for 1/25 samples (4%) an unambiguous false-positive result was seen. This sample belonged to a control group 1 patient without known comorbidities. For patients with previous SARS-CoV-2 infection, 24/26 samples (92.3% overall positive agreement) were evaluated as positive by all three assay if borderline results counted as negative and 25/26 (95.2% overall positive agreement) if considered positive. For one of the two false-negative test results obtained by the Roche assay, the corresponding result of the Euroimmun test was borderline. For the other sample, the two other tests revealed a definite positive result. Importantly, in this case the COI of the Elecsys Anti-SARS-CoV-2 was 0.976 and thus marginally below the cutoff.

4. Discussion

The implementation of large-scale SARS-CoV-2 serological testing on a population level is heavily debated by governments and national regulators to adopt restriction regulations to current demands in countries where the number of infections is decreasing. Beside of supplementing primary diagnosis [18], monitoring of immune response to vaccine candidates and evaluation of immunity duration [10], seroepidemiologic studies represent the main application area of immunoassays. Here, serological tests promise to provide the greatest benefit as they may allow accurately assessing infection prevalence and establishing indicators of SARS-CoV-2 immunity. In this screening situation, oligo- or asymptomatic SARS-CoV-2 infection will represent the majority of cases.

For evaluation of test performance, the study cohort should represent the intended test population. This has not been adequately addressed so far, as the majority of published studies comparing different immunoassays were performed on hospitalized patients with ongoing infection [19], [20], [21], [22]. This might affect the diagnostic test performance on two ways. First, the time of blood-draw after symptom onset substantially impacts the number of positive test results as reliable test results for anti-SARS-CoV-2 IgG testing can be obtained with a median of 14 days after symptom onset and sometimes with a delay of several weeks [13], [17], [18], [23], [24], [25]. Second, it has been shown that the IgG level of hospitalized patients is enhanced compared to a moderate to absent IgG immune response in mildly affected individuals [11], [23], [26], [27]. Therefore, our study cohort included patients after hospitalization or end of home quarantine representing the different severity of the disease course.

All three commercial assays used in our method comparison study were verified according to our quality management requirements and in agreement with accreditation requirements according to ISO 15 189. All assays revealed an acceptable inter- and intraassay variability with the lowest coefficient of variation (CV) seen for the Elecsys test. Most importantly, the diagnostic accuracy for all test systems was 100% using quality control samples.

To the best of the authoŕs knowledge, this is the first study so far addressing systematically the impact of specimen type on the test performance of different immunoassays. For screening purposes or seroepidemiological studies, the available blood specimen types may vary. Often, only lithium heparin plasma is available as the most common residual material from patient samples archived in the laboratory. According to manufactureŕs instruction of Euroimmun and Roche, serum and lithium-heparin plasma can be used for anti-SARS-CoV-2 antibody testing and no different cut-offs have to be used for interpretation of test results. In contrast, the Epitope assay is restricted to serum as testing material. Our analysis of 20 matching serum and plasma samples revealed a high level of concordance between both testing modalities for assays from Euroimmun and Epitope by linear regression, Passing Bablok regression and Bland-Altman analysis demonstrating the interchangeability of anti-SARS-CoV-2 antibody detection in both blood matrices (r² = 0.97 and r² = 0.91). However, based on qualitative results the overall agreement was 100% for the Epitope test and 90–95% for the immunoassay from Euroimmun. Statistical evaluation of the Roche Elecsys assay demonstrated significantly higher values measured in plasma than in serum samples and correlation analysis proved that plasma- and serum-based testing are no interchangeable. In this context it is important to mention that all assays were performed with the same lot number and no technical problems from the manufacturer have been reported for this lot. Comparable results have been reported for neutralization antibody tests (NAT) [10]. Although no false-positive or false-negative qualitative result was noted for the Roche test, this might be the case for samples near the assays cut-off. Hence, our results for at least two FDA EUA approved assays demonstrate that optimal cut-offs and assay performance should be separately assessed for each testing material.

To estimate test performance, manufacturers refer to test sensitivity and specificity or positive and negative percent agreement indicating that a non-reference standard was used for test evaluation. In case of anti-SARS-CoV-2-IgG testing, the respective manufactureŕs information is summarized by the Center for Health Security of Johns Hopkins University [13]. Here, for all three immunoassays included in this study a sensitivity of 100% is reported if patients are tested from 14 days after diagnosis onwards. The specificity is indicated by 100% for the tests from Euroimmun and Epitope diagnostics and 99.81% for the Roche assay. In our study, we compared the diagnostic test performance of these immunoassays for detection of anti-SARS-CoV-2 IgG antibodies on a panel of 51 serum samples including 26 patients with confirmed SARS-CoV-2 infection and 25 controls. Our analysis revealed a diagnostic sensitivity of 92.3%, 96.2–100% and 100% with a respective diagnostic specificity of 100%, 100% and 84–86% for the immunoassays from Roche, Euroimmun and Epitope Diagnostics. In total, 84–96% of samples were correctly classified as negative and 92.3–95.2% as positive. For test sensitivity, these findings are clearly below manufactureŕs specifications. One explanation might be that we preferentially included patients with subclinical infections or a mild course of disease representing the majority of SARS-CoV-2 infections and for whom the IgG immune response is reported to be moderate [10], [23], [26], [27], [28]. In this context it is worth mentioning that one sample was classified as positive by two assays in agreement with the qRT-PCR confirmed SARS-CoV-2 infection, but was found negative by the Roche test. This underscores that, as expected, the high specificity of the Elecsys assay leads to a reduction in test sensitivity to a certain extent. Especially as no borderline results are reported by the Elecsys assay, patients with COIs marginally below the cut-off might need to be reevaluated after a certain time period to reduce false-negative test results. Egger et al. demonstrated that the diagnostic sensitivity strongly depends on time from symptom onset with a sensitivity ranging between 100% [29] and 89.36% [30] from day 15 onwards for the Elecsys test. Nevertheless, the false negative results revealed in our study are obtained 26 and 45 days after confirmation of SARS-CoV-2 infection. Overall, the diagnostic sensitivity of 92.3% revealed in our study is in line with those reported by others [29], [30], [31]. Comparable, findings for the assays from Euroimmun and Epitope diagnostics are in agreement with or even better than those reported in other studies [20], [21], [22].

The decision whether an assay is suitable for clinical use or which test might be preferred depends on the intended testing purpose. For seroepidemiologic studies and screening purposes, the identification of affected individuals is usually based on a screening test with high diagnostic sensitivity and a confirmatory test with high specificity. In case of anti-SARS-CoV-2, such a confirmatory test could be a neutralizing antibody test (NAT). However, these tests are time-consuming with a turn-around-time of 3–5 days [13], require a biosafety level-3 laboratory and thus cannot be performed on a broad extent or for population-based studies. Hence, the estimation of the infection prevalence relies on serological tests without further confirmatory test and thus priority must be given to test specificity over sensitivity. Arguments for prioritizing test specificity include more far-reaching consequences in case of a false positive test result. Second, due to currently low prevalence of SARS-CoV-2 infections within the population that is estimated at 0.32% in the United States [32], only with a test specificity approaching 100% an acceptable positive predictive value can be obtained [17], [21], [32]. Our study revealed that these requirements are fulfilled by the immunoassays from Euroimmun and Roche. However, as our study is limited by sample size further large-scale studies are warranted to address the specificity of the different assays in different subpopulations for whom a high level of test interference with immunoassays is reported, e.g. for patients suffering from chronic diseases. Another possibility to increase the positive predictive value of serological tests is to monitor individuals with a positive antibody test and only consider them positive in case of seroconversion defined as class-switch from IgM to IgG or a more than 4fold increased IgG titer formerly for SARS [33], [34].

Finally, it has to be mentioned that antibody detection might be due to cross reactivity with other coronaviruses [26] and even if specific does not necessarily equate with protective immunity [17], [35]. It remains to be elucidated whether antibodies against nucleocapsid or spike protein will prove superior in assessing immunity. True immunity studies will require direct comparison with NAT [17].

In summary, our study demonstrated a high specificity for both FDA EUA approved immunoassays and a good diagnostic sensitivity. Overall, these results were superior to the non-FDA EUA declared test included. This is underlining the suitability of the immunoassays from Roche and Euroimmun for seroepidemiologic studies and screening purposes. In case of positive results, either a follow-up of patient in order to confirm results by seroconversion or a NAT might be necessary. However, comparing different blood specimens significantly different results were obtained either on quantitative or qualitative evaluation. Hence, there is an urgent need for quality standards to be implemented, harmonization of tests allowing to compare test results and to guide further decisions. Appropriate cut-offs and deeper understanding of tests limitation are a prerequisite for the appropriate use of COVID-19 serological tests.

CRediT authorship contribution statement

Verena Haselmann: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Writing - original draft, Visualization. Maximilian Kittel: Investigation, Visualization, Writing - review & editing. Catharina Gerhards: Investigation, Writing - review & editing. Margot Thiaucourt: Investigation, Writing - review & editing. Romy Eichner: Investigation, Validation, Writing - review & editing. Victor Costina: Conceptualization, Methodology, Validation, Writing - review & editing. Michael Neumaier: Conceptualization, Project administration, Writing - review & editing.

Appendix A. Supplementary material

The following are the Supplementary data to this article:

Supplementary figure 1.