Performance evaluation of three automated quantitative immunoassays and their correlation with a surrogate virus neutralization test in coronavirus disease 19 patients and pre‐pandemic controls

Abstract

Background

SARS‐CoV‐2 pandemic is currently ongoing, meanwhile vaccinations are rapidly underway in some countries. The quantitative immunoassays detecting antibodies against spike antigen of SARS‐CoV‐2 have been developed based on the findings that they have a better correlation with the neutralizing antibody.

Methods

The performances of the Abbott Architect SARS‐CoV‐2 IgG II Quant, DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG, and Roche Elecsys anti‐SARS‐CoV‐2 S were evaluated on 173 sera from 126 SARS‐CoV‐2 patients and 151 pre‐pandemic sera. Their correlations with GenScript cPass SARS‐CoV‐2 Neutralization Antibody Detection Kit were also analyzed on 173 sera from 126 SARS‐CoV‐2 patients.

Results

Architect SARS‐CoV‐2 IgG II Quant and Elecsys anti‐SARS‐CoV‐2 S showed the highest overall sensitivity (96.0%), followed by LIAISON SARS‐CoV‐2 TrimericS IgG (93.6%). The specificities of Elecsys anti‐SARS‐CoV‐2 S and LIAISON SARS‐CoV‐2 TrimericS IgG were 100.0%, followed by Architect SARS‐CoV‐2 IgG II Quant (99.3%). Regarding the correlation with cPass neutralization antibody assay, LIAISON SARS‐CoV‐2 TrimericS IgG showed the best correlation (Spearman rho = 0.88), followed by Architect SARS‐CoV‐2 IgG II Quant and Elecsys anti‐SARS‐CoV‐2 S (all rho = 0.87).

Conclusions

The three automated quantitative immunoassays showed good diagnostic performance and strong correlations with neutralization antibodies. These assays will be useful in diagnostic assistance, evaluating the response to vaccination, and the assessment of herd immunity in the future.

The performances of three automated quantitative immunoassays detecting antibodies against SARS‐CoV‐2 spike protein, Abbott SARS‐CoV‐2 IgG II Quant; DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG; Roche Elecsys anti‐SARS‐CoV‐2 S were comparable, with superior sensitivity and specificity. The three immunoassays demonstrated strong correlations with GenScript cPass virus neutralization test.

1. INTRODUCTION

Severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), first reported in Wuhan, China in 2019, caused a worldwide outbreak that is currently ongoing.¹ Coronavirus disease 19 (COVID‐19), the infectious disease caused by SARS‐CoV‐2, became not only an unprecedented threat to public health worldwide but also a tragic shock to the economy across the globe.² The absence of specific treatment options proved to be effective against SARS‐CoV‐2 aggravated the affair.³ This has resulted in the heightened importance of SARS‐CoV‐2 diagnostic testing as quarantine and social distancing have become the primary strategies for control of COVID‐19.⁴

Molecular testing, especially RT‐PCR, a reliable tool detecting the active SARS‐CoV‐2 infection, is the first option for the COVID‐19 diagnosis.⁵, ⁶ And serologic testing, a secondary weapon in the diagnostic arsenal for COVID‐19, was used as complementary to the RT‐PCR in the area where RT‐PCR has its limitations. Because serologic testing for COVID‐19 has its advantages such as cost‐effectiveness, short turnaround time, high‐throughput, ability to detect past infection, and usefulness in resource‐limited areas.⁷

Recently, the countermeasure strategy against COVID‐19 has stepped up from detection and quarantine of infection to active achievement of herd immunity through vaccination, as several vaccines have been approved for emergency use by the government in Europe and the United States and vaccinations are rapidly underway in some countries.⁸, ⁹, ¹⁰ In line with these shifts, the importance of tests detecting antibodies for SARS‐CoV‐2, especially neutralizing antibodies representing the protective ability of host immunity, is further emphasized.

The majority of SARS‐CoV‐2 serologic assays used whole or some parts of spike protein (S1 subunit, S2 subunit, and receptor binding domain located in S1 subunit) or nucleocapsid (N) protein as target antigens.¹¹ Previous studies reported that assays targeting spike protein showed a better correlation with virus neutralization assay compared to nucleocapsid protein.¹², ¹³ Recently, commercial manufacturers launched the quantitative SARS‐CoV‐2 antibody assays using spike protein as a target antigen, which can be a pivotal tool assessing the effect of vaccination. Abbott Architect anti‐SARS‐CoV‐2 IgG II Quant, new quantitative SARS‐CoV‐2 IgG immunoassay released by Abbott, received CM mark approval from the EU government. Roche also released their new quantitative assay, Elecsys anti‐SARS‐CoV‐2 S, targeting receptor binding domain (RBD) of S1 subunit. And DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG has been developed based on the observation that trimer form of spike protein showed greater sensitivity for the detection of SARS‐CoV‐2 antibodies.¹⁴ Many studies are conducted regarding the clinical performances of the previous version of anti‐SARS‐CoV‐2 assays against N protein (Abbott or Roche) and against S1/S2 subunit (Diasorin).¹⁵, ¹⁶, ¹⁷, ¹⁸ However, clinical performances of newly launched SARS‐CoV‐2 antibody assays against S1 RBD (Abbott or Roche) or TrimericS (Diasorin) have not been evaluated thoroughly. To the best of our knowledge, the performance of Abbott Architect anti‐SARS‐CoV‐2 IgG II Quant has not been fully evaluated so far. The performances of Elecsys anti‐SARS‐CoV‐2 S have been reported in comparison with the previous version (Elecsys anti‐SARS‐CoV‐2 against N antigen), showed better sensitivity than Elecsys anti‐SARS‐CoV‐2 against N.¹⁹, ²⁰ The performance of LIAISON SARS‐CoV‐2 TrimericS IgG has been once reported²¹ and showed better performance than previous version of LIAISON SARS‐CoV‐2 against S1/S2.¹⁵, ¹⁶, ¹⁷ However, the superiority of clinical performance can be evaluated precisely when performed in the same population. We assessed the clinical performance of three newly developed anti‐SARS‐CoV‐2 assays in the same subjects.

Meanwhile, virus neutralization assay using live SARS‐CoV‐2 is the gold standard method for assessing neutralizing antibodies. But its utility is limited because it is labor‐intensive, time‐consuming, and requires specialized facilities such as biosafety level 3.⁴ For this reason, researchers tried to develop alternatives that are more appropriate for large‐scale use in clinical laboratories.²² GenScript cPass SARS‐CoV‐2 Neutralization Antibody Detection Kit is an enzyme‐linked immunosorbent assay (ELISA) based surrogate virus neutralization test (sVNT) that mimics the reaction of human ACE2 receptor and RBD. It has been reported that cPass SARS‐CoV‐2 Neutralization Antibody test presented an excellent correlation with cell‐culture‐based virus neutralization assays and could be a useful measure of virus‐neutralizing activity.²³, ²⁴ The correlations of cPass SARS‐CoV‐2 Neutralization Antibody test with three automated anti‐SARS‐CoV‐2 assays (Mindray CL‐900i against S and N, BioMerieux VIDAS 3 against RBD, and Diasorin LIAISON SARS‐CoV‐2 against S1/S2) have been reported with the best correlation in VIDAS 3 (r = 0.75), followed by LIAISON S1/S2 (r = 0.66) and Mindray CL‐900i (r = 0.57)²⁵. However, the correlations of cPass SARS‐CoV‐2 Neutralization Antibody test with Abbott SARS‐CoV‐2 IgG II Quant, Roche Elecsys anti‐SARS‐CoV‐2 S, and DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG have not been evaluated.

Therefore, we performed a comparative assessment of three fully automated quantitative assays detecting antibodies against spike protein: Abbott SARS‐CoV‐2 IgG II Quant, Roche Elecsys anti‐SARS‐CoV‐2 S, and DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG. We evaluated their clinical performance and quantitative correlation with cPass SARS‐CoV‐2 Neutralization Antibody test.

2. MATERIALS AND METHODS

2.1. Test specimens

This study was approved by the Institutional Review Board of Seoul National University Hospital (IRB no. 2011‐041‐1170). All subjects were admitted to Seoul National University Hospital, or Boramae Medical Center between February 2020 and January 2021. Leftover patient specimens obtained for routine serologic testing were used. A total of 173 sera from 126 COVID‐19 patients were included in this study. Clinical diagnosis for COVID‐19 were determined based on clinical symptoms, imaging diagnosis, and laboratory findings including RT‐PCR. Among 126 COVID‐19 patients (44 females and 82 males), two (1.6%), 20 (15.9%), 18 (14.3%), 40 (31.7%), and 46 (36.5%) patients were classified by disease severity as asymptomatic, mild, moderate, severe, and critical, respectively. For each subject, age, sex, the number of days from onset of the symptom to the day sample collected, and disease severity determined by WHO interim guidance²⁶ were acquired through electronic medical records. For specificity evaluation, 151 pre‐pandemic sera were tested. Out of the 151 sera, 98 were from healthy subjects and 53 were from patients with positive results of various infectious markers: 5 anti‐HAV IgG, 5 anti‐T. pallidum IgG, 5 anti‐HCV, 5 anti‐HBc IgG, 5 anti‐CMV‐IgG, 5 anti‐rubella IgG, 4 anti‐toxoplasma IgG, 3 anti‐HIV IgG, 2 anti‐mycoplasma IgG, 2 M. tubeculosis PCR, 2 RSV PCR, 5 rhinovirus PCR, 1 adenovirus PCR, 1 bocavirus PCR, 1 parainfluenza virus PCR, 1 coronavirus OC43 PCR, 1 coronavirus 229E PCR.

2.2. Automated anti‐SARS‐CoV‐2 IgG immunoassay

Three fully automated commercial immunoassays were evaluated. The specifications of the three immunoassays are summarized in Table 1. Abbott SARS‐CoV‐2 IgG II Quant (Abbott Laboratories, Sligo, Ireland; hereafter called Abbott Quant) is a chemiluminescent microparticle immunoassay designed for the quantitative determination of IgG antibodies to RBD of the S1 subunit of the spike protein of SARS‐CoV‐2. The assay was performed on Abbott Architect i2000SR system (Abbott Laboratories, Abbott Park). Roche Elecsys anti‐SARS‐CoV‐2 S (Roche Diagnostics; hereafter called Roche S) is an electrochemiluminescence immunoassay for the quantitative determination of antibodies to RBD of the spike protein of SARS‐CoV‐2. The assay was performed on Roche cobas e601 system (Roche Diagnostics). DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG (DiaSorin, Stillwater; hereafter called DiaSorin TrimericS) is a chemiluminescent immunoassay using magnetic particles coated with recombinant trimeric SARS‐CoV‐2 spike protein for the quantitative determination of IgG antibodies. The assay was performed on LIAISON XL analyzer (DiaSorin). All tests were performed according to the manufacturer's instructions.

TABLE 1.

Specifications of the four immunoassays claimed by each manufacturers

	Architect SARS‐CoV‐2 IgG II Quant	LIAISON SARS‐CoV‐2 TrimericS IgG	Elecsys anti‐SARS‐CoV‐2 S	cPass SARS‐CoV‐2 Neutralization Ab
Manufacturer	Abbott	DiaSorin	Roche	GenScript
Platform	Architect I system	LIAISON XL analyzer	cobas e system	ELISA system
Method	CLMIA	CLIA	ECLIA	ELISA
Target antigen	RBD	trimeric SP	RBD	RBD
Immunoglobulin class	IgG	IgG	Pan‐Ig	Pan‐Ig
Sensitivitya (%, 95% CI)	99.4 (96.5–100.0)	98.7 (94.5–99.6)	98.8 (98.1–99.3)	100.0 (87.1–100.0)d
Specificity (%, 95% CI)	99.6 (99.2–99.8)	99.5 (99.0–99.7)	100.0 (99.9–100.0)	100.0 (95.8–100.0)e
Unit	AU/ml	AU/ml	U/ml	%
Cut‐off	50	13	0.8	30
AMR	21.0–40,000.0b	1.85–800.0	0.4–250.0c	NAf

Abbreviations: AMR, analytical measuring range; CI, confidence interval; CLIA, chemiluminescent immunoassay; CLMIA, chemiluminescent microparticle immunoassay; ECLIA, electrochemiluminescent immunoassay; RBD, receptor binding domain; SP, spike protein.

^a Sensitivity were calculated from patient sample collected after 14 days or later from positive PCR results.

^b Measuring range extends up to 80,000 by 1:2 dilution.

^c Measuring range extends up to 2,500 by 1:10 dilution.

^d Positive percent agreement with plaque reduction neutralization test (PRNT)₅₀.

^e Negative percent agreement with PRNT₅₀.

^f Not available (approved for qualitative detection).

2.3. GenScript cPass SARS‐CoV‐2 Neutralization Antibody Detection assay

GenScript cPass SARS‐CoV‐2 Neutralization Antibody Detection Kit (GenScript, Piscataway; hereafter called cPass) is a blocking ELISA detection tool, which mimics the virus neutralization process. The detection kit utilizes the Horseradish peroxidase (HRP) conjugated recombinant SARS‐CoV‐2 RBD protein and the human angiotensin‐converting enzyme 2 (ACE2) receptor protein. The protein interaction between HRP‐RBD and ACE2 can be blocked by neutralizing antibodies against SARS‐CoV‐RBD. The assay was performed as below, according to the manufacturer's instruction.

Test samples, negative control, and positive control were 1:10 diluted with sample dilution buffer. HRP‐RBD was diluted 1:1000 with HRP dilution buffer. Diluted samples and diluted HRP‐RBD solution were mixed with a volume ratio of 1:1 and incubated at 37°C for 30 min. 100 μl of the mixture was then added to the capture plate coated with the human ACE2 receptor protein and incubated at 37°C for 15 min. After the incubation, the mixture was washed 4 times with 260 μl of wash buffer. Then, 100 μl of tetramethylbenzidine solution was added to the mixture and incubated at room temperature for 15 min. Finally, 50 μl of stop solution was added. The absorbance of the final solution was read at 450 nm in a microplate reader.

Signal inhibition was calculated as follow:

$$\text{Percent Signal Inhibition}=\left(1‐\text{average optical density value of sample}/\text{average optical density value of negative control}\right)\times 100\%$$

The test results were interpreted as positive when the percent signal inhibition was ≥30%, which is the cut‐off for signal inhibition claimed by the manufacturer.

2.4. Precision and linearity assessment

The precision assessment was performed on three quantitative assays, according to CLSI EP15‐A3 protocol,²⁷ using one quality control material and two pooled patient sera for five consecutive days, five times a day. Repeatability and within‐laboratory precision were estimated using ANOVA and compared to values claimed by the manufacturers.

Linearity assessment was performed on three quantitative assays, according to CLSI EP6‐A protocol.²⁸ Two patient sera with high (H) and low (L) concentration were mixed at ratios of 4H, 1L + 3H, 2L + 2H, 3L + 1H, and 4L. All levels are measured in duplicates.

2.5. Statistical analysis

For three immunoassays, sensitivity and specificity were calculated. The sensitivity of the subgroup sampled 14 days after the onset of symptoms was also calculated and compared with the manufacturer's claim. It is in line with the recommendation from infectious diseases society of America guidelines on the Diagnosis of COVID‐19 that suggests against using serologic testing to diagnose SARS‐CoV‐2 infection during the first two weeks (14 days) following symptom onset.²⁹ The concordances between the three immunoassays and cPass were assessed using overall, positive and negative percent agreement as well as Cohen's kappa statistics. Cohen's kappa is a robust statistic of inter‐rater reliability, useful for assessing the level of agreement between two diagnostic assays. Ranging between 0 and 1, a kappa value <0.40 represents poor agreement, 0.40–0.59 represents fair agreement, 0.60–0.74 represents good agreement, and ≥0.75 represents excellent agreement.³⁰ We evaluated the correlations of the quantitative value of three immunoassays with each other and with % inhibition value of cPass using Spearman's rank‐order correlation coefficient (rho). All statistical analyses were performed by using R version 4.0.5 (R foundation for statistical Computing).

3. RESULTS

3.1. Clinical performance

The clinical performances of three immunoassays are shown in Table 2. The sensitivity of Abbott Quant, DiaSorin TrimericS, and Roche S on 173 sera from 126 COVID‐19 patients was 96.0% (95% CI, 91.8%–98.4%), 93.6% (95% CI, 88.9%–96.8%) and 96.0% (95% CI, 91.9%–98.4%), respectively. The sensitivity calculated from the subgroup sampled 14 days after the onset of symptom was 97.6% (95% CI, 93.2%–99.5%), 96.8% (95% CI, 92.1%–99.1%) and 97.6% (95% CI, 93.2%–99.5%), respectively. The specificity of Abbott Quant, DiaSorin TrimericS, and Roche S on 151 pre‐pandemic sera were 99.3 (95% CI, 96.4–100.0), 100.0% (95% CI, 97.6–100.0%), and 100.0% (95% CI, 97.6%–100.0%), respectively. No positive result was observed in the cross‐reactivity panel.

TABLE 2.

Clinical performance of three immunoassays

		Clinical diagnosis				Sensitivity		Specificity
		Positive		Negative		(95% CI)		(95% CI)
		D ≥ 14	D < 14	Healthy	Other	Total	D ≥ 14	Total	Healthy	Other
		(n = 126)	(n = 47)	(n = 98)	(n = 53)	(n = 173)	(n = 126)	(n = 151)	(n = 98)	(n = 53)
Abbott Quant	Pos	123	43	1	0	96	97.6	99.3	99.0	100
	Neg	3	4	97	53	(91.8–98.4)	(93.2–99.5)	(96.4–100.0)	(94.4–100.0)	(93.3–100.0)
DiaSorin TrimericS	Pos	122	40	0	0	93.6	96.8	100	100	100
	Neg	4	7	98	53	(88.9–96.8)	(92.1–99.1)	(97.6–100.0)	(96.3–100.0)	(93.3–100.0)
Roche S	Pos	123	43	0	0	96	97.6	100	100	100
	Neg	3	4	98	53	(91.9–98.4)	(93.2–99.5)	(97.6–100.0)	(96.3–100.0)	(93.0–100.0)
Genscript cPass	Pos	123	41	NT	NT	94.8	97.6	NT	NT	NT
	Neg	3	6	NT	NT	(90.4–97.6)	(93.2–99.5)	NT	NT	NT

Abbreviations: Abbott Quant, Abbott SARS‐CoV‐2 IgG II Quant; D, days after onset of symptom; DiaSorin TrimericS, DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG; Genscript cPass, Genscript cPass SARS‐CoV‐2 Neutralization Antibody Detection Kit;NT, not tested; Other, other infection; Roche S, Roche Elecsys anti‐SARS‐CoV‐2 S.

3.2. Repeatability, within‐laboratory imprecision, and linearity

The repeatability and within‐laboratory imprecision for three immunoassays are shown in Table 3. The within‐laboratory precisions of Abbott Quant and Roche S were all <4.0%. The within‐laboratory precisions of DiaSorin TrimericS were 2.9–8.2%, which were slightly larger than that claimed by the manufacturer.

TABLE 3.

Repeatability and within‐laboratory imprecision for three quantitative immunoassays

Assay	Unit	Material	Mean Conc.	Repeatability (n = 25, % CV)	Within‐Lab precision (% CV)
Architect SARS‐CoV‐2 IgG II Quant	AU/ml	SARS‐CoV‐2 IgG II Quant Positive Control 2	667.63	2.1	2.3
		Low level PS	413.4	3.4	3.5
		High level PS	3733.14	2.9	2.9
LIAISON SARS‐CoV‐2 TrimericS IgG	AU/ml	LIAISON® SARS‐CoV‐2 TrimericS IgG Positive Control	34.89	2.2	2.9
		Low level PS	36.76	2.2	8.2
		High level PS	339.28	3.0	5.2
Elecsys anti‐SARS‐CoV‐2 S	U/ml	PreciControl Anti‐SARS‐CoV‐2 2	8.65	1.0	3.6
		Low level PS	4.37	0.7	2.5
		High level PS	141.65	0.8	3.2

Abbreviations: Conc., concentration; CV, coefficient of variation; Lab, laboratory; PS, pooled serum.

The linearity assessment of three immunoassays (Figure S1) revealed to be linear across the analytical measurement range (R² = 0.9992, 0.9947, 0.9966 for Abbott Quant, DiaSorin TrimericS, and Roche S, respectively). And % recovery was of Abbott Quant, DiaSorin TrimericS, and Roche S was all acceptable (criteria: 100% ± 10%), ranged as 96.4%–100.0%, 100.0%–109.7%, 92.8%–102.9%, respectively.

3.3. Correlation between results from three immunoassays

The correlations between results from three immunoassays were shown in Figure 1. The Roche S correlated well with Abbott Quant II (rho = 0.88) and DiaSorin Trimeric S (rho = 0.85). Abbott Quant II correlated well with DiaSorin Trimeric S (rho = 0.9).

FIGURE 1.

Spearman correlation between three immunoassays and cPass neutralization antibody. (A‐C). Abbott SARS‐CoV‐2 IgG II Quant (A), Roche Elecsys anti‐SARS‐CoV‐2 S (B), and DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG (C) demonstrated strong correlations with Genscript cPass surrogate virus neutralization test (Spearman's rho of 0.87, 0.87, and 0.88 for each). (D‐F). Three immunoassays demonstrated strong correlations with each other. Spearman's rho of 0.88 between Abbott SARS‐CoV‐2 IgG II Quant and Roche Elecsys anti‐SARS‐CoV‐2 (D); Spearman's rho of 0.9 between Abbott SARS‐CoV‐2 IgG II Quant and DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG (E); Spearman's rho of 0.85 between Roche Elecsys anti‐ SARS‐CoV‐2 and DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG (F)

3.4. Comparison to cPass SARS‐CoV‐2 neutralization antibody test

The concordances between the qualitative results of three immunoassays and cPass SARS‐CoV‐2 neutralization test in SARS‐CoV‐2 positive patients are shown in Table 4. The positive percent agreements of three immunoassays with cPass were 97.6%–99.4%. The negative percent agreements of three immunoassays with cPass were 55.6%–77.8%. The overall percent agreements of three immunoassays with cPass were 96.5%–97.7% with Cohen's kappa value of 0.61–0.74. The positive percent agreements between the three immunoassays were 98.8%–99.4% and the negative percent agreements between three immunoassays were 54.5%–71.4%. The overall percent agreements between three immunoassays were 96.5%–97.7% with Cohen's kappa value of 0.65–0.7.

TABLE 4.

Concordance between the qualitative results of three immunoassays and cPass SARS‐CoV‐2 Neutralization Antibody Detection kit in 173 SARS‐CoV‐2 patients

Assay	Compared to	OPA	PPA	NPA	Cohen's kappa
		% (95% CI)	% (95% CI)	% (95% CI)	(95% CI)
Abbott Quant	cPass	167/173	162/164	5/9	0.61
		96.5 (92.6–98.7)	98.8 (95.7–99.9)	55.6 (21.2–86.3)	(0.32–0.89)
DiaSorin TrimericS	cPass	167/173	160/164	7/9	0.68
		96.5 (92.6–98.7)	97.6 (93.9–99.3)	77.8 (40–97.2)	(0.44–0.92)
Roche S	cPass	169/173	163/164	6/9	0.74
		97.7 (94.2–99.4)	99.4 (96.6–100)	66.7 (29.9–92.5)	(0.49–0.98)
Abbott Quant	RocheS	169/173	164/166	5/7	0.70
		97.7 (94.2–99.4)	98.8 (95.7–99.9)	71.4 (29–96.3)	(0.43–0.98)
Abbott Quant	DiaSorin TrimericS	167/173	161/162	6/11	0.65
		96.5 (92.6–98.7)	99.4 (96.6–100)	54.5 (23.4–83.3)	(0.39–0.91)
Roche S	DiaSorin TrimericS	167/173	161/162	6/11	0.65
		96.5 (92.6–98.7)	99.4 (96.6–100)	54.5 (23.4–83.3)	(0.39–0.91)

Abbreviations: Abbott Quant, Abbott SARS‐CoV‐2 IgG II Quant; DiaSorin TrimericS, DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG; NPA, negative percent agreement; OPA, overall percent agreement; PPA, positive percent agreement; Roche S, Roche Elecsys anti‐SARS‐CoV‐2 S.

The correlations of quantitative results of three immunoassays with % inhibition values of cPass were very strong with Spearman's rho value of 0.87 for Abbott Quant and Roche S, and 0.88 for DiaSorin TrimericS (p < 0.001 for all) (Figure 1).

3.5. Receiver operating characteristics analysis

Receiver operating characteristics (ROC) curve analysis was performed on three immunoassays (Figure 2). Areas under the curve (AUC) of three immunoassays were 0.993 for Abbott Quant, 0.989 for DiaSorin TrimericS, and 0.983 for Roche S, which means excellent performances for all three immunoassays. Using the ROC curves, we assumed the optimized cut‐off values based on Youden's index. The optimized cut‐offs were 41.9 AU/ml, 3.985 AU/ml, and 0.544 U/ml for Abbott Quant, DiaSorin TrimericS, and Roche S, respectively. The corresponding manufacturer's recommended cut‐offs were 50.0, 13.0, and 0.8. Applying the optimized cut‐offs, the sensitivity of Abbott Quant improved from 96.0% to 97.1%, with no decrease in specificity (99.3%). The sensitivity of DiaSorin TrimericS improved from 93.6% to 98.3% with a slight decrease in specificity (from 100.0% to 97.1%). The sensitivity of Roche S improved from 96.0% to 96.5% with no decrease in specificity (100.0%).

FIGURE 2.

Receiver operating characteristics (ROC) curve analysis for three immunoassays. Area under the curve were 0.993 for Abbott Quant (A), 0.989 for DiaSorin TrimericS (B), and 0.983 for Roche S (C), which means excellent performances for all three immunoassays

4. DISCUSSION

In this study, we compared three commercially available automated quantitative immunoassays for the detection of antibodies to SARS‐CoV‐2 and cPass as a surrogate for viral neutralization. In the sensitivity test, all assays demonstrated excellent sensitivity greater than 90%, and Abbott Quant and Roche S showed slightly higher sensitivity than DiaSorin TrimericS. The sensitivity calculated from the subgroup sampled 14 days or later after the onset of symptom, which has been reported to be the window period of SARS‐CoV‐2 antibody formation³¹ were all slightly lower compared to that claimed by the manufacturer. (Abbott Quant 97.6% vs. 99.4%; DiaSorin TrimericS 96.8% vs. 98.7%; Roche S 97.6% vs. 98.8%). There were four sera from four patients with negative results in the subgroup sampled 14 days or later after the onset. Two samples were negative for all assays, and the other two were detected at very low quantitative values near the cut‐off only in Roche S and Abbott Quant, respectively. Out of four patients, two with underlying hematologic malignancy were suspected of having inhibition of antibody development by their immunocompromised status. The other two patients were asymptomatic or showed very mild symptoms. Previous studies have reported that antibodies were not detected in 10%–20% of mild cases of COVID‐19.³², ³³ Considering that 28 sera from mild cases were included in this study, the sensitivities of three immunoassays were satisfactory.

In prior reports evaluating the clinical performances of the previous version of anti‐SARS‐CoV‐2 assays against N protein (Abbott or Roche) and against S1/S2 subunit (DiaSorin),¹⁵, ¹⁶, ¹⁷, ¹⁸ the sensitivities of Abbott anti‐SARS‐CoV‐2 (against N) were 86.5%~90.8%. Those of Roche anti‐SARS‐CoV‐2 (against N) were 83.0%~93.0%. The sensitivities of DiaSorin LIAISON anit‐SARS‐CoV‐2 (against S1/S2) were 70.0%~85.3%, slightly lower than those of Abbott or Roche anti‐SARS‐CoV‐2 (N).¹⁵, ¹⁶, ¹⁷ In the report evaluating two Roche anti‐SARS‐CoV‐2 assays against N or S simultaneously,¹⁹ anti‐SARS‐CoV‐2 S showed higher sensitivity than anti‐SARS‐CoV‐2 (against N) (93.0% vs. 89.0%). In a recent report, the sensitivity of DiaSorin anti‐SARS‐CoV‐2 Trimeric S was 99.4%, which was higher than the previous version of DiaSorin anti‐SARS‐CoV‐2 against S1/S2.²¹ In our study, the sensitivity was highest in Roche S and Abbott Quant (96.0%), followed by DiaSorin Trimeric S (93.6%). All three immunoassays showed higher sensitivities than prior reports of the previous version of those assays (Abbott and Roche against N or DiaSorin anti‐S1/S2). The sensitivity of DiaSorin TrimericS in this study improved from 93.6% to 98.3% with a slight decrease in specificity (from 100.0% to 98.3%) by adjusting the cut‐off value from 13.0 AU/ml to 3.985 AU/ml, which implicates no substantial differences in sensitivity of the three immunoassays.

The sensitivity of cPass NT has been reported higher (93%) than those of Abbott anti‐SARS‐CoV‐2 (N) (89%) or Roche anti‐SARS‐CoV‐2 total (N) (83%).¹⁷ In our study, cPass showed similar sensitivity (94.8%) with the other three immunoassays (93.6%~96.0%), which is consistent with the previous reports, considering the lower sensitivity of Abbott or Roche anti‐SARS‐CoV‐2 (N) assays in prior studies.¹⁵, ¹⁶, ¹⁷, ¹⁸

In the specificity test conducted with 151 pre‐pandemic samples, all three immunoassays showed remarkable specificity with only one positive result from Abbott Quant. These results were consistent with manufacturer's claim and previous studies, which report superior sensitivity in the new version of anti‐SARS‐CoV‐2 assays (range 99.8%–100.0%).¹⁹, ²⁰, ²¹ In the previous version of three immunoassays, DiaSorin anti‐SARS‐CoV‐2 against S1/S2 showed slightly lower specificities than Abbott anti‐SARS‐CoV‐2 (N) or Roche anti‐SARS‐CoV‐2 total (N).¹⁵, ¹⁶, ¹⁷ In the new DiaSorin anti‐SARS‐CoV‐2 against trimeric S, specificity was excellent (99.8%).²¹ The sensitivities of both the previous and new version of Roche anti‐SARS‐CoV‐2 were excellent (100.0% for both).¹⁹, ²⁰

Imprecision of the new Roche anti‐SARS‐CoV‐2 against S has been reported as 1.06% at 9.06 U/ml.³⁴ The imprecision of the new DiaSorin anti‐SARS‐CoV‐2 against trimeric S was an average of 4.85% (3.6%~5.8% range) at values ranging from 5 to 591 AU/L,²¹ which was higher than that of Roche S. In our study, the imprecision at low and high level pooled serum was 8.2% and 5.2% in DiaSorin Trimeric S and 2.5% and 3.2% in Roche S. The higher imprecision in DiaSorin Trimeric S compared to Roche S in our study was consistent with previous reports.²¹, ³⁴ The imprecision of Abbott Quant II, firstly reported in this study, was similar to Roche S.

The correlations of the result from three immunoassays with each other were excellent (rho value 0.85–0.9). In comparison with cPass, all three immunoassays showed high percent agreement for overall patient samples above 95%. Cohen's kappa statistics were 0.61 for Abbott Quant, 0.74 for Roche S, and 0.68 for DiaSorin TrimericS, all denoting good agreement. Although cPass is not intended to use as a quantitative assay, strong correlations between quantitative values of three assays and % inhibition values of cPass were found in this study with Spearman rho value of 0.87–0.88. In previous reports, the Abbott SARS‐CoV‐2 IgG assay (targeting nucleocapsid antigen) and the Roche anti‐SARS‐CoV total antibody (targeting nucleocapsid antigen) showed weaker correlations with neutralizing antibody than DiaSorin SARS‐CoV‐2 IgG (targeting S1/S2 subunits), Euroimmune SARS‐CoV‐2 IgG ELISA (targeting S1 subunit), or Siemens SARS‐CoV‐2 total antibody (targeting RBD).¹², ¹³ Newly launched Abbott Quant and Roche S (both targeting RBD) showed strong correlations with cPass in this study as well as DiaSorin TrimericS (targeting trimeric spike protein). All three immunoassays can be applied to assess the immune response to vaccination because the SARS‐CoV‐2 vaccines use RBD as an immunogen.

Abbott Architect I and Roche cobas system provided the function for dilutional testing and DiaSorin LIAISON analyzer did not. In this study, Abbott Quant needed dilution and retest in four of the 173 samples (2.3%), Roche S in 58 samples (33.5%), so there was the inconvenience of needs for additional samples and reagents, especially in Roche S. DiaSorin TrimericS reported 11 out of 173 samples (6.4%) as above the limit of quantitation. Although direct comparison was unavailable because the quantitative units of the three assays are not uniform, Abbott Quant assay seems to have the highest upper limit of quantitation without dilution.

This study had some limitations. (1) Culture‐based virus neutralization test was not performed due to its requirement for very specialized facilities. Instead, cPass sVNT was utilized as a substitute, based on previous studies demonstrating excellent concordance between sVNT and conventional virus neutralization test.²³, ²⁴, ³⁵ (2) Few asymptomatic patients were included (2/173) because the patient group consisted mainly of inpatients. So, the results of this study should be carefully applied in populations that contain a large number of asymptomatic patients.

Nevertheless, we showed the performances of the new version of three automated quantitative immunoassays detecting antibodies against SARS‐CoV‐2 spike protein (Abbott and Roche) or trimeric S (DiaSorin). The improved sensitivities of Abbott SARS‐CoV‐2 IgG II Quant, DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG, and Roche Elecsys anti‐SARS‐CoV‐2 S compared to the previous version of three immunoassays were suspected. The total imprecision was slightly higher in DiaSorin Trimeric S than Roche S or Abbott Quant II. The correlations of the results of the three immunoassays were good. We also demonstrated the strong correlations of the three immunoassays with sVNT. These high‐throughput immunoassays are supposed to be valuable in diagnostic assistance of RT‐PCR, evaluating the response to vaccination, and the assessment of herd immunity in future.

Supporting information

Figure S1

Jung K, Shin S, Nam M, et al. Performance evaluation of three automated quantitative immunoassays and their correlation with a surrogate virus neutralization test in coronavirus disease 19 patients and pre‐pandemic controls. J Clin Lab Anal. 2021;35:e23921. 10.1002/jcla.23921

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.