Performance of the Roche/Snibe electrochemiluminescent anti-SARS-COV-2 spike assays compared to the Roche/Abbott IgG nucleocapsid and Abbott IgM spike assays

Abstract

Introduction

We evaluated the Roche Elecsys Anti-SARS-CoV-2 and Snibe SARS-CoV-2 S-RBD IgG spike chemiluminescent immunoassays and compared them to existing Roche/Abbott nucleocapsid and Abbott IgM spike assays.

Methods

We enrolled 184 SARS-CoV-2 RT-PCR positive samples and 215 controls (172 pre-pandemic, and 43 cross-reactivity) to evaluate the Roche spike antibody (anti-SARS-CoV-2-S) assay. For the Snibe evaluation, we included 119 RT-PCR positive samples and 249 controls (200 pre-pandemice, 49 cross-reactivity). 98 cases had been tested on three spike assays (Roche total antibody, Snibe IgG and Abbott IgM).

Results

The Roche anti-SARS-CoV-2-S assay had a CV of 0.5% (0.82U/mL) and 2.3% (8.72U/mL) and was linear from 1.16 to 240U/mL. The Snibe assay was linear from 6.43 to 77.7AU/mL, CV of 5.5% (0.43AU/mL) and 8.8% (0.18AU/mL). The Snibe spike assay was significantly more sensitive than the Abbott IgG assay at 0–6 days POS (35.2% vs 3.6%, mean difference 29.6%, 95% CI 17.5 to 41.8, p < 0.0001). Optimized LORs significantly improved the sensitivity of the Roche spike (48.1%–56.7%) and both nucleocapsid assays (Roche 43.3%–65.5%, Abbott 3.6%–18.5%) in early disease.

Conclusion

Although both spike assays showed higher sensitivity than their nucleocapsid counterparts, lower, optimized LORs provided the most significant improvements to sensitivity.

Highlights

• •We report the performance of the Roche and Snibe anti-SARS-CoV-2 spike assays.
• •The Snibe spike assay displayed the greatest sensitivity in early disease.
• •The Snibe assay showed cross-reactivity with dengue and hepatitis antibodies.
• •Optimized limits of reactivity improved the sensitivities of assays.

Abbreviatons

COVID-19

novel coronavirus disease 2019

SARS-CoV-2

severe acute respiratory syndrome coronavirus 2

Anti-SARS-CoV-2

SARS-CoV-2 antibodies

Anti-SARS-CoV-2-N

SARS-CoV-2 nucleocapsid antibodies

Anti-SARS-CoV-2-S

SARS-CoV-2 spike antibodies

CLIA

chemiluminescent immunoassay

LOR

limit of reactivity

RT-PCR

reverse-transcriptase polymerase chain reaction

COI

cut-off index

CV%

inter-assay precision

PPA

positive percentage agreement

NPA

negative percentage agreement

POS

Post-first positive RT-PCR

1. Introduction

The novel coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has swept across the globe causing a range of presentations ranging anywhere from a mild flu-like illness [1] to full-blown respiratory failure [2]. Spike proteins cover the entire SARS-CoV-2 virion and are essential for the viral life cycle, binding to the angiotensin-converting enzyme 2 receptor to enter cells to perpetuate the viral reproduction cycle [3]. The nucleocapsid protein, another essential component of the virion, can then enter host cells along with viral genetic material to assist in replication, virion assembly and release [4]. Antibody responses to SARS-CoV-2 take some time to develop, typically taking 7–14 days [5]. Thus, the Centers for Disease Control and Prevention have recommended that serologic testing can be used as a method to support the diagnosis of COVID-19 in those who present 9–14 days after disease onset [6]. Several preclinical trials testing antibodies directed against the spike protein have shown some efficacy in decreasing viral loads [7]. Together with previous knowledge from other coronaviruses [8], we know that the spike protein is the main target for viral neutralization. Manufacturers have thus recently created vaccines to induce an antibody response against SARS-CoV-2 spike proteins, with efficacies ranging from 94.1% [9] to 95% [10].

In the early stages of the pandemic, most assays for SARS-CoV-2 antibodies (anti-SARS-CoV-2) qualitatively assessed the presence of antibodies directed against the nucleocapsid of the virion (anti-SARS-CoV-2-N). The performance of these assays were satisfactory [11,12]. Recently, quantitative assays that assess antibodies directed against the spike protein (anti-SARS-CoV-2-S) have been introduced. Some studies demonstrate that these anti-SARS-CoV-2-S assays may have improved sensitivity over the previous nucleocapsid assays [13]. These anti-SARS-CoV-2-S assays would also be invaluable in the assessment of an effective antibody response to the COVID-19 vaccines. We thus evaluated the performance of two new anti-SARS-CoV-S assays, the Roche Elecsys Anti-SARS-CoV-2 S assay (total antibodies) on the Cobas 8000 analyser and the Snibe SARS-CoV-2 S-RBD IgG chemiluminescent immunoassay (CLIA) assay on the Maglumi analyser, comparing them to their respective nucleocapsid assay on the Roche and Abbott immunoassay analysers. We also compared their performance against an existing spike protein assay, the Abbott SARS-CoV-2 IgM assay (also run on the Architect immunoassay analyser). In addition, several studies [12,14,15] have reported that lower, optimized limits of reactivity (LORs) can greatly improve the sensitivity of anti-SARS-CoV-2 assays, especially in the detection of early disease. We proceeded to derive optimized LORs for each of the evaluated assays and observed how they might impact their test sensitivities.

2. Methods

Participants: All reverse-transcriptase polymerase chain reaction testing (RT-PCR) positive samples for COVID-19 (May to July 2020) with excess sera available were recruited. To evaluate the Roche anti-SARS-CoV-2-S assay, we enrolled 184 de-identified, anonymized serum samples (leftover from other routine tests, e.g., renal panels, liver function tests) that were positive for SARS-CoV-2 on RT-PCR testing as cases. For controls, we included 172 de-identified, anonymized samples from a staff health screening in 2018, and 43 cross-reactivity samples (Dengue 20, HBsAg 5, HBeAg 3, anti-HBe 4, HBcAg 6, anti-HBc 1, anti-HCV 3, syphilis 1), resulting in a total of 215 controls. All controls were negative on the Abbott Architect anti-SARS-CoV-2 IgG and IgM nucleocapsid assays and were deemed to be COVID-19-naive.

For the Snibe evaluation, we included 119 de-identified, anonymized serum samples (leftover from other routine tests, e.g., renal panels, liver function tests) that were positive for SARS-CoV-2 on RT-PCR as cases. We also recruited 200 de-identified, anonymous samples from a staff health screening in 2018, and 49 cross-reactivity samples (HBsAg 5, HBeAg 3, anti-HBe 4, HBcAg 6, anti-HBc 1, anti-HCV 3, syphilis 1, dengue 14, ANA 11, dsDNA 1), resulting in a total of 249 subjects as controls. All controls were negative on the Abbott anti-SARS-CoV-2 IgM assay and the Roche anti-SARS-CoV-2 nucleocapsid assay and deemed to be free of COVID-19.

The numbers of subjects tested on the Roche anti-SARS-CoV-2-S assay, Snibe assay, and Abbott IgM assay differ because the numbers of available case and control sera differed at different time points. Nevertheless, 98 of all the cases had been tested on all three anti-SARS-CoV-2-S assays (Roche total antibody, Snibe IgG and Abbott IgM), and these were analysed in our study as well.

Materials and methods: The Roche Elecsys Anti-SARS-CoV-2 S assay is a quantitative non-competitive (sandwich) CLIA. The sample is first incubated with biotinylated SARS-CoV-2 S-RBD-specific recombinant antigen and SARS-CoV-2 S-RBD-specific recombinant antigen labelled with a ruthenium complex to form a sandwich complex. This is then complexed with streptavidin-coated microparticles, and then magnetically captured onto an electrode. An electric current applied at the electrode induces a chemiluminescent signal that is directly proportional to the amount of anti-SARS-CoV-2-S. The results are reported in U/mL, a result of ≥0.80U/mL is positive for anti-SARS-CoV-2-S. The manufacturer's claimed assay precision is 2.9% (at 0.5U/mL) and 1.4% (at 184U/mL), specificity of 99.98% and a sensitivity of 98.8% ≥ 14 days after diagnosis with RT-PCR, reported limit of detection of 0.35U/mL and a measuring range of 0.4–250U/mL. The Roche anti-SARS-CoV-2 nucleocapsid assay is also a non-competitive CLIA which uses a similar assay method to the anti-SARS-CoV-2 S assay, except that it compares the sample signal to the mean chemiluminescent signal of a calibrator to derive an anti-SARS-CoV-2 index, with a reported cut-off index (COI) of 1.0 for positivity.

The Snibe SARS-CoV-2 S-RBD IgG CLIA is a quantitative indirect chemiluminescence immunoassay, where the sample, buffer and magnetic microbeads are first mixed and incubated to form immune-complexes. After precipitation in a magnetic field, the supernatant is decanted, and a wash cycle performed. Antibodies labelled with anti-human IgG antibody is then added to form complexes. After another cycle of precipitation, decanting and washing, starter material is added to initiate a chemiluminescent reaction. The result is expressed as AU/mL, and a result ≥1.0AU/mL is positive. The assay has a reported precision of 12.4% (0.4AU/mL) and 4.9% (5.1AU/mL), a limit of detection of 0.18AU/mL with a linear range of 0.18–100AU/mL. The assay has a claimed sensitivity of 100% ≥ 15 days post symptom onset and a claimed specificity of 99.6%. The Abbott SARS-CoV-2 IgG and IgM assay are qualitative CLIAs used for the detection of IgG (nucleocapsid) and IgM (spike) antibodies to SARS-CoV-2 respectively; the sample signal is compared to the mean chemiluminescent signal of a calibrator and a COI of ≥1.4 is deemed positive on the IgG assay, and a COI of ≥1.0 is deemed reactive on the IgM assay.

For RT-PCR testing, Changi General Hospital molecular laboratory employs a duplex real-time RT-PCR that targets the N and E genes using a Qiagen EZ1 extraction system and Rotor Gene Q amplification system.

Inter-assay precision (CV%) was evaluated by running 2 levels of control materials in replicates of 5 for 5 days. Linearity was assessed by using pooled patient sera to create a series of levels that covered the clinically relevant range. All samples used in this study were serum samples collected in Vacutainer Gel separator plain serum tubes (Becton Dickinson SST tubes with silica clot activator, polymer gel, silicone-coated interior). No samples with missing or indeterminate test results were used in this study.

Statistical analysis: As the Roche anti-SARS-CoV-2 and Abbott anti-SARS-CoV-2 IgG and IgM nucleocapsid assays are qualitative assays, sensitivity can be represented by positive predictive agreement (PPA) and specificity by negative predictive agreement (NPA) with the RT-PCR results. We compared the PPA/sensitivity of the spike and nucleocapsid assays by groups based on the days post-first RT-PCR (POS) as a surrogate for disease onset. ROC analysis was also performed between cases and controls for each assay to derive a recommended associated criterion. Thereafter, optimized LORs were derived for each assay by calculating the average of the recommended associated criterion and the 99th percentile of all negative controls. Data were presented in either mean ± standard deviation or median [inter-quartile range], as appropriate. 95% confidence intervals for sensitivity and specificity were calculated according to Clopper and Pearson (“exact” method) with standard logit confidence intervals for predictive values. To confirm statistically significant differences between results, paired categorical data were compared using the McNemar test and a p-value <0.05 was considered as statistically significant. Statistical analyses were performed using MedCalc® Statistical Software version 19.5.3 (MedCalc Software Ltd, Ostend, Belgium). Our IRB deemed this work exempt as this was part of routine laboratory evaluation of new assays and seroprevalence surveillance using de-identified leftover sera. Compliance with STARD guidelines is enclosed (see Supplementary Table 1).

3. Results

Complete data on all subject results is detailed in Supplementary Table 2.

Performance analysis: The Elecsys Anti-SARS-CoV-2 S assay had a CV of 0.5% at 0.82U/mL and 2.3% at 8.72U/mL and was linear from 1.16 to 240U/mL. Subjects with low values were reported as <0.4U/mL. For computation of results, values < 0.4U/mL were taken as 0.4U/mL. The Snibe SARS-CoV-2 S-RBD IgG assay was linear from 6.43 to 77.7AU/mL, with a CV of 5.5% (0.43AU/mL) and 8.8% (0.18AU/mL).

Specificity and cross reactivity analysis: On the Roche anti-SARS-CoV-2-S assay, all 215 controls were negative (0.4U/mL) resulting in a specificity of 100% (95% CI 98.3 to 100). On the other hand, 212/215 controls were negative on the Roche anti-SARS-CoV-2-N assay, with a specificity of 98.6% (95% CI 96.0 to 99.7). These 3 controls were positive for hepatitis antibodies, with COIs of 1.3, 1.59 and 1.51. None of the cross-reactivity samples were reactive on the Roche anti-SARS-CoV-2-S assay.

A total of 229 of 249 controls were negative on the Snibe anti-SARS-CoV-2-S IgG assay, with a specificity of 92.0% (95% CI 87.9 to 95.0). The Abbott anti-SARS-CoV-2-N IgG assay had 248/249 negative results, with a specificity of 99.6% (95% CI 97.8 to 100). Of note, the Snibe assay displayed cross-reactivity with 7 other antibody positive cross-reactivity test samples (HBeAg 2, Dengue 4, ANA 1).

Sensitivity analysis: In the 184 RT-PCR positive cases tested on the Roche total anti-SARS-CoV-2-S/anti-SARS-CoV-2-N assays, the sensitivities of both assays increased stepwise from 0 to 6 days POS to ≥14 days POS (see Table 1). In all cases, there was no significant difference between the sensitivities of both assays, even in the first week after disease onset (48.1% vs 43.3%, mean difference 13.0%, 95% CI 1.6 to 24.5, p = 0.30). In the 119 RT-PCR positive cases used to test the IgG anti-SARS-CoV-2-N/S assays, the sensitivity also progressively increased from 0 to 6 days to ≥14 days POS. Of note, the sensitivity of the Snibe IgG was significantly greater than the Abbott IgG in early disease (mean difference 29.6%, 95% CI 17.5 to 41.8, p < 0.0001).

Table 1.

Sensitivity analysis of the total and IgG anti-SARS-CoV-2 nucleocapsid and spike assays.

Days POS	Roche Nucleocapsid			Roche Spike			Abbott IgG Nucleocapsid			Snibe IgG Spike
	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)
0 to 6	45	59	43.3 (33.6–53.3)	50	54	48.1 (38.2–58.1)	2	52	3.6 (0.4–12.3)	19	35	35.2 (22.7–49.4)
7 to 13	29	10	74.4 (57.9–87.0)	31	8	79.5 (63.5–90.7)	23	8	74.2 (55.4–88.1)	25	6	80.6 (62.5–92.5)
≥14	39	2	95.1 (83.5–99.4)	39	2	95.1 (83.5–99.4)	33	1	97.1 (84.7–99.9)	34	0	100 (89.7–100)

Abbreviations: POS: post-first positive RT-PCR, PPA: positive percentage agreement, CI: confidence interval.

Sensitivity between anti-SARS-CoV-2-S assays: In the 98 cases that had been tested on all the Roche total antibody/Snibe IgG/Abbott IgM anti-SARS-CoV-2-S assays, the Snibe assay had the highest sensitivity at 0–6 days POS (see Table 2). However, the differences between the sensitivities was not statistically significant (Snibe vs Roche mean difference 15.6%, 95% CI 0.5 to 30.6, p = 0.09, Snibe vs Abbott mean difference 17.8%, 95% CI 1.2 to 34.4, p = 0.08).

Table 2.

Head-to-head Comparison of sensitivities between the anti-SARS-CoV-2 spike assays.

Days POS	Roche total antibody			Snibe IgG			Abbott IgM
	Pos	Neg	PPA (95% CI)	Pos	Neg	PPA (95% CI)	Pos	Pos/neg	PPA (95% CI)
0 to 6	8	37	17.8 (8.0–32.1)	15	30	33.3 (20.0–49.0)	7	38	15.6 (6.5–29.5)
7 to 13	18	6	75.0 (53.3–90.2)	20	4	83.3 (62.6–95.3)	17	7	70.8 (48.9–87.4)
≥14	28	1	96.5 (82.2–99.9)	29	0	100 (88.1–100)	23	6	79.3 (60.3–92.0)

Abbreviations: POS: post-first positive RT-PCR, PPA: positive percentage agreement, CI: confidence interval.

We performed Mann-whitney U testing between the Snibe and Roche assay results of the 98 cases to the 119 cases tested on the Snibe assay and 184 cases tested on the Roche assay, and found no significant difference between the groups (Snibe median difference 0, p = 0.97; Roche median difference 0U/mL, p = 0.43), which highlights the equivalency of the original case groups and the 98 cases that were tested on all assays.

Optimized limits for reactivity: The 99th percentile of the negative controls on the Roche anti-SARS-CoV-2-S assay was 0.4U/mL, and on the anti-SARS-CoV-2-N assay was COI 0.16. ROC analysis showed an associated criterion of 0.4U/mL (AUC 0.84, 95% CI 0.81–0.88) for the anti-SARS-CoV-2-S assay (see Supplementary Figure 1a) and an associated criterion of COI 0.1 (AUC 0.94, 95% CI 0.92–0.96) for the anti-SARS-CoV-2-N assay (see Supplementary Figure 1b). The average of these values gave an optimized LOR of >0.4U/mL for the anti-SARS-CoV-2-S assay and COI >0.13 for the anti-SARS-CoV-2-N assay.

The 99th percentile of the negative controls on the Snibe IgG anti-SARS-CoV-2-S assay and the Abbott IgG anti-SARS-CoV-2-N assay was 0.9AU/mL and COI 0.4 respectively. ROC analysis showed an associated criterion of 0.86AU/mL (AUC 0.86, 95% CI 0.82–0.90) for the Snibe assay (see Supplementary Figure 2a), with an associated criterion of COI 0.6 (AUC 0.88, 95% CI 0.84–0.91) for the Abbott assay (see Supplementary Figure 2b). The average of these values gives an optimized LOR of >0.9AU/mL for the Snibe assay, and COI >0.5 for the Abbott assay.

Sensitivity/Specificity analysis with optimized LORs: When applied to both Roche total antibody assays, the optimized LORs improved the assay sensitivity, particularly for the total anti-SARS-CoV-2-N assay (see Table 3). Although the mean difference between the nucleocapsid and spike assay sensitivities of cases at 0–6 days POS was 6.7% (95% CI -0.5 to 13.9), it was not statistically significant (p = 0.12). However, there was a statistically significant difference between the normal and optimized LOR sensitivities at 0–6 days for both the spike (mean difference 8.7%, 95% CI 3.3 to 14.1, p = 0.004) and nucleocapsid assays (mean difference 20.2%, 95% CI 12.5 to 27.9, p < 0.0001), with the nucleocapsid assay also having a significant difference pre-RT-PCR (mean difference 23.9%, 95% CI 11.6 to 36.2, p = 0.001) and at 7–13 days (mean difference 18.0%, 95% CI 5.9 to 30.0, p = 0.02). When the optimized cut-offs were applied to the control population, there was no change to the specificity of the anti-SARS-CoV-2-S assay (remained at 100%), but the specificity of the anti-SARS-CoV-2-N assay decreased slightly from 98.6% to 97.6% (95% CI 94.7 to 99.2) (210/215 controls negative). The optimized LORs applied to the Snibe/Abbott IgG assays also improved the sensitivities of both assays, especially in early disease. The difference between the sensitivities of both assays in 0–6 days POS remained significant even with optimized LORs (mean difference 22.2%, 95% CI 9.0 to 35.5, p = 0.004). Although there was no significant difference between the sensitivities of the normal and optimized LORs on the Snibe assay, the optimized LORs significantly improved the sensitivity of the Abbott IgG assay at 0–6 days POS (mean difference 14.8%, 95% CI 5.3 to 24.3, p = 0.008). The specificity of the Snibe assay decreased slightly from 92.0% to 90.8% (95% CI 86.5–94.1), the specificity of the Abbott assay decreased from 99.6% to 99.2% (95% CI 97.1–99.9%).

Table 3.

Sensitivity analysis of the total and IgG anti-SARS-CoV-2 nucleocapsid and spike assays with optimized LORs.

Days POS	Roche Nucleocapsid			Roche Spike			Abbott IgG Nucleocapsid			Snibe IgG Nucleocapsid
	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)	Positive	Negative	PPA (95% CI)
0 to 6	66	38	63.5 (53.4–72.7)	59	45	56.7 (46.7–66.4)	10	44	18.5 (9.3–31.4)	22	32	40.7 (27.6–55.0)
7 to 13	36	3	92.3 (79.1–98.4)	32	7	82.1 (66.5–92.5)	23	8	74.2 (55.4–88.1)	25	6	80.6 (62.5–92.5)
≥14	40	1	97.6 (87.1–99.9)	39	2	95.1 (83.5–99.4)	34	0	100 (89.7–100)	34	0	100 (89.7–100)

Abbreviations: POS: post-first positive RT-PCR, PPA: positive percentage agreement, CI: confidence interval.

4. Discussion

We have demonstrated that both the Roche total anti-SARS-CoV-2 S assay and the Snibe IgG anti-SARS-CoV-2 S assay have good precision and linearity. Both anti-SARS-CoV-2-S assays showed improved sensitivity compared to their anti-SARS-CoV-2-N counterparts when using the manufacturer recommended LORs, particularly in early disease, although the difference in sensitivity was statistically significant only between the Snibe/Abbott IgG assays. The Snibe assay had the highest sensitivity in the first week of disease onset when compared to the Roche total antibody and Abbott IgM anti-SARS-CoV-2-S assays, although the differences failed to achieve statistical significance. The optimized LORs significantly improved the sensitivities of the Roche spike/nucleocapsid assays and the Abbott nucleocapsid assay in the first week of infection, with the Roche nucleocapsid assay now having a greater sensitivity than the spike assay in days 0–6 POS (although the difference in sensitivity between the Roche spike/nucleocapsid assays was not significant). The Snibe assay was significantly more sensitive than the Abbott assay at 0–6 days POS.

There is some debate as to whether anti-SARS-CoV-2-S assays have better performance than anti-SARS-CoV-2-N assays. Some studies show that spike assays are more sensitive than nucleocapsid assays [16]. In a comparison of antibody detection using a Luminex bead-based assay in 296 serum samples (from both mild and severe cases of COVID-19), IgG anti-SARS-CoV-2-S assays had better sensitivities than anti-SARS-CoV-2-N assays (84–91% vs 85% in severe infections, 79–98% vs 70% in mild infections). Even after 5 months post-infection, the anti-SARS-CoV-2-S assays still had superior sensitivity (99-95% vs 95%). This may be because anti-SARS-CoV-2-S develops earlier than anti-SARS-CoV-2-N antibodies. In one study of 1850 hospitalized COVID-19 patients [17], the spike-specific IgM and IgG in both mild and severe cases were both higher than the nucleocapsid IgM and IgG on admission (e.g. spike IgG median 28.9 vs nucleocapsid IgG median 18.8 in mild/moderate cases) (Shenzhen YHLO Biotech magnetic CLIA run on an iFlash3000 analyser). However, other studies show the reverse [18]. In a study of 100 samples analysed by a luciferase immunoprecipitation assay, the sensitivity of anti-SARS-CoV-2-N was 100% more than 14 days after symptom onset, whereas the anti-SARS-CoV-2-S was only 91%. Although the anti-SARS-CoV-2-S assays in our study did show improved sensitivity compared to the anti-SARS-CoV-2-N assays, the difference was not statistically significant, and the 3 anti-SARS-CoV-2-S assays also did not show statistically significant differences in performance between themselves.

In spite of this, there are several benefits to assessing for anti-SARS-CoV-2-S over anti-SARS-CoV-2-N. Anti-SARS-CoV-2-S has been shown to persist longer than anti-SARS-CoV-2-N. In a cohort of 195 subjects [19], 78% had anti-SARS-CoV-2-S 4 months after a positive SARS-CoV-2 RT-PCR test, compared to only 22% for anti-SARS-CoV-2-N. In 3276 healthcare workers [20], anti-SARS-CoV-2-S levels were detected 180 days POS in 94% of the population, whereas anti-SARS-CoV-2-N antibodies peaked at 24 days POS before declining. Anti-SARS-CoV-2-S has also been associated with survival rates; in 509 patients with COVID-19 [21], IgG anti-SARS-CoV-2-S was associated with improved patient survival in regression analysis, with a hazard ratio of time to death of 0.6. Surprisingly, anti-SARS-CoV-2-N was not linked to survival. The development of IgG anti-SARS-CoV-2-S is associated with better survival even in patients with diabetes [22], whereas the nucleocapsid antibodies were not associated with patient survival rates. Furthermore, antibodies against SARS-CoV-2 spike proteins have been shown to be elevated by day 28 post-first dose vaccination, remaining elevated up to 56 days later [23], and can be elevated up to 119 days later after a booster vaccination 28 days after the first vaccine [24]. Anti-SARS-CoV-2-S assays will be undoubtedly useful in the assessment of patient immune responses to the new vaccines.

The Snibe assay displayed cross-reactivity with several other antibodies in samples positive for dengue, hepatitis and ANA antibodies, whereas the Roche assay showed no cross-reactivity. Some reports [25,26] indicate that there can be some cross-reactivity between dengue and SARS-CoV-2 antibodies, and this is important to note in countries where dengue fever is prevalent. Indeed, dengue fever can present with similar symptoms to COVID-19 or both infections may even occur concurrently. Other reports [27] have also demonstrated false-positive COVID-19 results in patients with hepatitis B and Epstein-Barr virus infection (Liaison SARS-CoV-2 S1/S2 IgG assay). Thus, it is pertinent to be cognizant of the patient's past/current medical history prior to antibody testing, and positive anti-SARS-CoV-2 results in patients with other antibodies may require further confirmatory testing.

Lower optimized LORs significantly improved the sensitivity of both Roche assays and the Abbott IgG anti-SARS-CoV-2-N assay, especially in early disease. The improvement was especially seen in early disease with Roche nucleocapsid assay (43.3%–65.5%) than the spike assay (48.1%–56.7%). In another study that compared the EUROIMMUN IgG nucleocapsid, Roche Elecsys nucleocapsid and LIAISON IgG spike assay [28], lower optimized LORs (EUROIMMUN COI >0.4, LIAISON >3.9AU/mL, Elecsys COI >0.17) improved the sensitivity of assays at 0–6 days post disease in the LIAISON assay (13.0%–26.1%) and the Elecsys assay (17.4%–26.1%). Furthermore, their optimized LOR for the Roche Elecsys nucleocapsid assay is fairly close to ours (COI >0.13). Although by themselves, the assay sensitivities are still insufficient for diagnosis even with optimized LORs, they can greatly improve the performance of antibody tests when used with another test in an orthogonal fashion. For example, when using lateral-flow immunoassays with sensitivities of 49.3% and specificities of 94.3%, the combined positive predictive value when used with an antigen test (sensitivity 56.2% and specificity of 99.5%) can be as high as 83.0% in a population with a disease prevalence of 0.5% [29]. As such, the use of optimized LORs in antibody testing will be a great boon should antibody tests be used in conjunction with RT-PCR testing, especially in areas of low disease prevalence.

Our study reports these new findings:

• ➢The performance of the new Roche anti-SARS-CoV-2 S and Snibe SARS-CoV-2 S-RBD IgG CLIA.
• ➢The anti-SARS-CoV-2-S assays display greater sensitivity in early disease compared to the previous anti-SARS-CoV-2-N assays.
• ➢The Snibe anti-SARS-CoV-2-S assay displayed the greatest sensitivity in early disease compared to the Roche total antibody and Abbott IgM anti-SARS-CoV-2-S assays.
• ➢Optimized LORs can significantly improve the sensitivities of assays and may be particularly useful when combining antibody assays with other tests in an orthogonal format.
• ➢Snibe SARS-CoV-2-S assays may show cross-reactivity with dengue and hepatitis antibodies.

One limitation of our study was that we were unable to test for cross-reactivity with other coronaviruses/other viruses that could cause symptoms of upper respiratory infection as samples from these groups were unavailable. The S2 domain of the spike protein and the nucleocapsid protein can be highly conserved across coronaviruses [30,31], and may cause additional false positives in patients with previous coronavirus infections. We also have relatively fewer patients who were POS ≥7 days, and further evaluations with larger populations in this group would be desirable. We have no data on the clinical severity of COVID-19 in these patients, as all sera was anonymized and deidentified, and data on the clinical history of patients was not available. It has been demonstrated that the titre of anti-SARS-CoV-2 corresponds to the severity to the severity of disease [32], and lower antibody titres in mild infections could have effects on assay sensitivity. The lack of statistical significance between anti-SARS-CoV-2-S and anti-SARS-CoV-2-N assays may be due to the small numbers in each category of days POS. Future studies with larger cohorts comparing these assays would be useful. The numbers of available sera for analysis for each assay was different, as the assay analysis was performed at different time points. However, Mann-Whitney U testing has found no significant difference between the results of samples tested on all assays vs the cohorts of all samples tested on the individual analytical platforms. This lack of difference demonstrates that the relationship observed between the total/IgG/IgM spike assays is still representative.

5. Conclusion

The new anti-SARS-CoV-2-S assays have demonstrated good performance, with sensitivities that are superior to the older nucleocapsid assays in early COVID-19. Optimized LORs can improve these sensitivities even further, and we await future studies that use these assays to assess the immune response of patients who have received the latest SARS-CoV-2 vaccinations.

Author statement

Conceptualization: TC Aw. Data curation: CS Lau, MS Wong, SP Hoo, PY Heng, SK Phua. Formal analysis: CS Lau. Funding acquisition: NOT APPLICABLE. Investigation: SP Hoo, PY Heng, SK Phua. Methodology: TC Aw. Project administration: SK Phua. Resources: TC Aw, MS Wong. Software: CS Lau. Supervision: TC Aw, MS Wong. Validation: SK Phua. Visualization: TC Aw. Writing - original draft: TC Aw, CS Lau. Writing - review and editing: TC Aw, CS Lau.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the supplementary data related to this article: