Surrogate test performance for SARS‐CoV‐2 neutralizing antibodies (nAbs) for convalescent plasma (CCP): How useful could they be?

Abstract

Background

COVID‐19 high‐titer CCP selection is a concern, because neutralizing antibody (nAb) testing requires sophisticated labs and methods. Surrogate tests are an alternative for measuring nAb levels in plasma bags, including those that are pathogen‐reduced.

Study design/methods

We studied a panel consisting of 191 samples from convalescent donors tested by nAb (CPE‐VNT), obtained from 180 CCP donations (collection: March 20–January 21) and 11 negative controls, with a total of 80 and 111 serum and plasma samples (71 amotosalen/UV treated), with nAb titers ranging from negative to 10,240. Samples were blindly tested for several surrogates: one anti‐RBD, two anti‐spike, and four anti‐nucleocapsid tests, either isolated or combined to improve their positive predictive values as predictors of the presence of high‐titer nAbs, defined as those with titers ≥160.

Results

Except for combined and anti‐IgA/M tests, all isolated surrogate tests showed excellent performance for nAb detection: sensitivity (98.3%–100%), specificity (85.7%–100%), PPV (98.9%–100%), NPV (81.3%–100%), and AUC (0.93–0.96), with a variable decrease in sensitivity and considerably lower specificity when using FDA authorization and concomitant nAb titers ≥160. All surrogates had AUCs that were statistically different from CPE‐VNT if nAb≥160, including when using combined, orthogonal approaches.

Conclusions

Surrogate tests (isolated or in combination) have an indirect good performance in detecting the presence of nAb, with lower sensitivity and specificity when high nAb titer samples are used, possibly accepting a considerable number of donors whose nAb titers are actually low, which should be evaluated by each laboratory responsible for CCP collection.

List of abbreviations

ACE2

angiotensin‐converting enzyme‐2 receptor

AUC

area under curve

CCP

Covid‐19 convalescent plasma

CPE‐VNT

cytopathic effect virus neutralization test

cVNT

conventional virus neutralization test

NAbs

neutralizing antibodies

NTD

N‐terminal domain

NP

nucleocapsid

pVNT

pseudotype virus neutralization test

RT‐PCR

real time polymearse chain reaction

RBD

receptor‐binding domain

1. INTRODUCTION

There are two main types of human antibodies: neutralizing (nAbs) and binding (ligand) antibodies. NAbs are “antibody markers of immunity against reinfection after an acute viral infection has been cleared, with capacity to reduce viral infectivity by binding to defined viral surface particles and blocking the viral replication cycle before the virally encoded transcription or synthesis in the host cell.” ¹ For SARS‐CoV‐2, nAbs are mostly directed against the receptor‐binding domain (RBD), and in a lower percentage (≅10%) against the N‐terminal domain (NTD) ² , ³ , ⁴ , ⁵ preventing viral binding to the angiotensin‐converting enzyme‐2 receptor (ACE2) in human cells. On the other hand, binding antibodies have the ability to bind to several other SARS‐CoV‐2 regions, signaling the current or past presence of the virus in the individual, although unable to interfere or prevent infectivity or evaluate its functionality; furthermore, they do not truly measure nAbs. ⁶ , ⁷ , ⁸ , ⁹ There are several licensed tests based on binding antibodies targeting the spike protein (S) or parts thereof (e.g., S1/S2 or RBD domains), or nucleocapsid (NP) antigens.

Despite the high accuracy and performance of available anti‐SARS‐CoV‐2 tests developed for clinical diagnosis, ⁶ , ¹⁰ little is known about their role when applied to screening high nAb titer CCP donors for therapeutic utilization. The FDA released an authorization for screening high‐titer CCP, ¹¹ currently followed by several centers and countries. There are still gaps concerning the correlation between commercial serological assays and nAb titers in CCP donors. ¹² , ¹³ , ¹⁴ Therefore, the simple adoption of commercial tests as nAb surrogates could result in doubtful results when CCP screening is concerned. Having the capacity to adopt nAb as a screening method for our small‐scale CCP program since its beginning, ¹⁵ and having collected a considerable number of CCP units and plasma/serum samples tested by nAb, we considered it appropriate to build a validation panel based on nAb titers in order to evaluate the surrogate role of some commercially available anti‐SARS‐CoV‐2 tests in our country, as a potential replacement for nAb tests for CCP screening programs.

2. MATERIAL AND METHODS

A panel consisting of 180 samples derived from convalescent donors tested by nAbs (VNT method) and additional 11 negative controls (known repeated donors collected before the pandemics and with no detectable SARS‐CoV‐2 antibodies) was selected. Informed consent was obtained from all participants in accordance with the institutional review board and the Helsinki Declaration.

All samples were collected between March 20 and January 21. There were a total of 80 and 111 serum and plasma samples, respectively, with 71 treated by amotosalen/UVA, with nAb titers ranging from negative to 10,240 (Figure 1). Samples were chosen in order to promote a homogeneous nAb titer distribution, tested by the cytopathic effect virus neutralization test (CPE‐VNT) ¹⁵ (Table S1); we used only serum as a negative control, not subject to pathogen reduction treatment. All convalescent donors had mild or moderate disease, and none required hospitalization. All donors had previously positive RT‐PCR results. Demographic data of the donors are presented in Table S2.

Samples were coded for surrogate testing, consisting of a surrogate virus neutralization test (sVNT), based on a competitive anti‐RBD inhibition test (cPass™ SARS‐CoV‐2 Neutralization Antibody Detection Kit, GenScript, Piscataway, NJ), two anti‐spike tests (Ortho Vitros COV2T Total, Ortho Clinical Diagnostics, Inc., Rochester, USA and Roche Elecsys anti‐SARS‐CoV‐2S, Roche Diagnostics GmbH, Mannheim, Germany), and four anti‐nucleocapsid tests (Roche Elecsys anti‐SARS‐CoV‐2, Roche Diagnostics GmbH, Mannheim, Germany), and an in‐house anti‐NP IgG, IgA, and IgM, developed by the Instituto de Ciências Biomédicas – ICB ‐ University of São Paulo, Brazil, as described elsewhere. ¹⁵ All tests were performed blindly, following the manufacturer's instructions, or according to the recent FDA authorization for CCP collection. ¹¹ High nAb samples were defined as those with nAb titers ≥160. We also evaluated an orthogonal approach in which reactive samples against any anti‐spike test were combined with the high inhibitory anti‐RBD test (≥68% inhibition) in order to improve their positive predictive values as predictors of the presence of high‐titer nAbs. The main characteristics of the surrogate tests are listed in Table 1. A more detailed description of all the tests used is provided in Appendix S1.

TABLE 1.

Main pattern of surrogate tests, including standard cut‐off (CO) and proposed FDA guidance for high‐titer donors

Test	Principle	Antigen(s)	Second stage	Antibodies	Standard cut‐off (CO)	FDA alternative CO (high‐titer)
c‐Pass	ELISA	Rec RBD (HRP)	Rec hACE2‐R (solid phase) + TMB	IgM, IgG	≥20%	≥68%
Ortho CoV‐2T	CLIA	Rec S1 (biotin)	Conjugated‐ recombinant S1 (HRP)	IgM, IgG, IgA	≥1.0	≥9.5
Elecsys anti‐ SARS‐CoV‐2S	E‐CLIA	Rec RBD (biotin); rec RBD (ruthenium); streptavidin‐coated magnetic particles	Binding to paramagnetic spheres + electric pulse (chemiluminescence)	IgG, IgM	≥ 0.8 U/ml	≥132 U/ml
Elecsys anti‐SARS‐CoV‐2	E‐CLIA	Rec N (biotin); rec NP (ruthenium); streptavidin‐coated magnetic particles	Binding to paramagnetic spheres + electric pulse (chemiluminescence)	IgG, IgM	≥1.0	≥109
ICB IgG anti‐NP	ELISA	nCoV‐PS‐Ag7	Conjugated goat anti‐human IgG (HRP)	IgG	S/CO a  = 1.0	S/CO ≥5.0 b
ICB IgM anti‐NP	ELISA	nCoV‐PS‐Ag7	Conjugated goat anti‐human IgM (HRP)	IgM	S/CO a  = 1.0	ND
ICB IgA anti‐NP	ELISA	nCoV‐PS‐Ag7	Conjugated goat anti‐human IgA (HRP)	IgA	S/CO a  = 1.0	ND

Abbreviation: ND, Not done.

^a Presented as signal/cut‐off (S/CO) ratio.

^b High S/CO based on previous Reference 1.

CCP plasma was collected via plasmapheresis (Trima Accel version 6, Terumo BCT, Lakewood, CO USA ‐ 600 ml plasma collection, using ACD‐A as an anticoagulant) and pathogen‐inactivated using amotosalen/UVA illumination (INTERCEPT®, Cerus Corporation, Concord, CA, USA), according to the manufacturer's instructions.

2.1. Statistical analysis

Normal distribution was checked using the Shapiro–Wilk or Kolmogorov–Smirnov test. Whenever possible, data were normalized by log transformation for parametric test statistical analysis; otherwise, nonparametric tests were used. Sensitivity, specificity, positive and negative predictive values (PPV and NPV), likelihood ratio (LR; i.e., the ratio of the probability/likelihood of a positive test result in an abnormal patient and in a normal patient, given by sensitivity/[1−specificity]), Cohen's kappa (agreement between tests defined as: slight [0–0.20]; fair [0.21–0.40]; moderate [0.41–0.60]; substantial [0.61–0.80], almost perfect [0.81–0.99] and perfect [1.00]), ¹⁶ Youden index (given by sensitivity + specificity −1), ¹⁷ nonparametric receiver operating curve (ROC), classified as excellent (0.9–1.0); good (0.8–0.9); fair (0.7–0.8) or poor (0.6–0.7), ¹⁸ and parallelism (relation between slopes of two curves, being accepted as parallel if the ratio falls within the limits between 0.8 to 1.25) ¹⁹ were calculated using Stata‐15 (College Park, TX) and JMP‐16 (Cary, NC) statistical packages. Statistical significance was set at p < .05.

3. RESULTS

From a total of 191 samples, 80 (41.9%) were sera (11 negative controls, NC, i.e., samples collected before the pandemics and with no detectable SARS‐CoV‐2 antibodies) and 111 (58.1%) plasma samples, with 40 (36%) and 71 (64%) samples derived from units before and after pathogen reduction treatment, respectively, as presented in Figure 1. A total of 74 donors provided 180 samples (ranging from 1 to 6 samples) based on sample availability and nAb titer distribution, as shown in Table S1. The mean (±SD), median (IQR), and remaining demographic data are shown in Table S2. Figure 2 shows the CCP donor sample nAb titer distribution (n = 180), based on the type of material (plasma/sera) or pathogen reduction treatment. There was no statistical difference in reactivity for each surrogate test based on plasma/sera or non‐treated/treated samples (Figures 3 and S1), based on the median reactivity of tests; however, parallelism was found only for the anti‐RBD test. Negative controls were not included in Figures 2 and 3 in order to avoid bias (none with the previous history of COVID and no SARS‐CoV‐2 antibodies).

FIGURE 1.

Diagram of samples distributed according to serum/plasma or pathogen reduction treatment. All samples were tested by eight isolated tests. anti‐RDB, competitive anti‐RBD inhibition test; nAb, neutralizing antibodies; NP, nucleoprotein

FIGURE 2.

Left: Box‐plot from nAb titer (natural log) from CCP donor samples, based on plasma or sera (top left) or pathogen reduction treatment (bottom left); with no statistical difference between groups (Mann–Whitney test); negative controls (NC; n = 11) were excluded. Right: Histogram density distribution of nAb titer (natural log), based on material (plasma or sera, top right), or pathogen reduction treatment (bottom right); both had a statistical difference in nAb titer distribution, (p = .048 and < .001, respectively, by the chi‐square test Χ²). The vertical dashed lines represent nAb titer ≥160 [Color figure can be viewed at wileyonlinelibrary.com]

FIGURE 3.

Median reactivity of anti‐SARS‐CoV‐2 tests from 180 samples, based on material (plasma and sera, upper quadrants) or pathogen reduction treatment (lower quadrants). Negative controls were not included, as they did not bear any SARS‐CoV‐2 Abs. There is no statistical difference in reactivity for each test based on plasma/sera or non‐treated/treated samples. Except for anti‐RBD test (given in % inhibition – left), all signal/cut‐off (S/CO) ratio were normalized into natural logarithm. ICB‐G, anti‐IgG NP; ICB‐A, anti‐IgA NP; ICB‐M, anti‐IgM NP; R, Roche; O, Ortho; Spk, Spike; NP, nucleoprotein [Color figure can be viewed at wileyonlinelibrary.com]

The agreement performance (kappa index) of each test is presented in Table 2. In general, there was a very good agreement between all tested methods when all samples were compared (regardless of the nAb titer), except for specific anti‐NP IgM and IgA or combined tests (Table 2, left). However, the same agreement pattern was remarkably changed once only high nAb titer samples (≥160) were compared under the FDA authorization for high‐titer samples (Table 2, right), with some improvements for certain procedures, although anti‐RBD was present in all improved combinations.

TABLE 2.

Kappa index for the agreement between all tested methods [Color table can be viewed at wileyonlinelibrary.com]

	nAb20	RBD	OSpk 1.0	R‐NP 1.0	R‐Spk 0.8	ICB‐G	ICB‐M	ICB‐A	OSpk1 + RBD	RSpk0.8 + RBD		nAb160	RBD68	OSpk 9.5	R‐NP 109	R‐SPk 132	ICB‐G 5.0	ICB‐M 5.0	ICB‐A 5.0	OSpk9.5 + RBD	RSpk132 + RBD
nAb20	1.00										nAb160	1.00									Slight
RBD	0.85	1.00									RBD68	0.66	1.00								Fair
OSpk 1.0	0.92	0.84	1.00								OSpk9.5	0.38	0.30	1.00							Mod
R‐NP 1.0	0.86	0.78	0.85	1.00							R‐NP109	0.29	0.29	0.10	1.00						Subs
R‐Spk 0.8	0.92	0.84	1.00	0.85	1.00						R‐Spk132	0.53	0.69	0.21	0.35	1.00					A/pft
ICB‐G	0.89	0.81	0.80	0.75	0.80	1.00					ICB‐G5.0	0.58	0.43	0.35	0.21	0.40	1.00				Pft
ICB‐M	0.11	0.11	0.09	0.13	0.09	0.12	1.00				ICB‐M5.0	0.03	0.03	0.01	0.08	0.06	0.03	1.00
ICB‐A	0.18	0.18	0.15	0.21	0.15	0.19	0.32	1.00			ICB‐A5.0	0.12	0.15	0.03	0.06	0.18	0.12	0.00	1.00
OSpk1+RBD	0.24	0.24	0.28	0.25	0.21	0.26	0.05	0.29	1.00		OSpk9.5+RBD	0.66	1.00	0.30	0.29	0.69	0.43	0.03	0.15	1.00
RSpk0.8+RBD	0.24	0.24	0.20	0.27	0.20	0.25	0.06	0.30	0.99	1.00	RSpk132+RBD	0.16	0.23	0.04	0.59	0.28	0.08	0.14	0.05	0.05	1.00

Note: The kappa results are classified as: <0 (poor); 0–0.20 (slight); 0.21–0.40 (fair); 0.41–0.60 (moderate); 0.61–0.80 (substantial); 0.81–0.99 (almost perfect); 1.00 (perfect). Results in left and right are based on manufacturer's instructions or using only high‐tittered sera compared with FDA authorization, respectively.

Abbreviations: anti‐RBD, competitive anti‐RBD inhibition test; anti‐RBD68, ≥68% competitive anti‐RBD inhibition test; ICB‐A, anti‐IgA NP; ICB‐G, anti‐IgG NP; ICB‐M, anti‐IgM NP; nAb20/160, neutralizing antibody titer ≥20 or ≥160, respectively; NP, nucleoprotein; O‐SPk, Ortho Spike; R‐NP, Roche NP; R‐Spk, Roche Spike; Spk0.8 and Spk1.0, anti‐Spike tests based on the manufacturer's cut‐off; SPk109 and SPk132, anti‐Spike tests based on FDA guidance for high‐titer.

Figure 4 shows the dispersion diagram of the isolated surrogate tests and nAb titers. The best fitted model was the quadratic regression, showing a higher adjusted R² (ranging from 0.6470 to 0.8079; all p < .001) than the ordinary linear regression. All quadratic models for surrogate tests showed a normal distribution of residuals with no signs of skewness, kurtosis, outliers, collinearity, heteroskedasticity (Breusch‐Pagan/Cook‐Weisberg test), and leverage.

FIGURE 4.

Quadratic regression between lnAb and anti‐RBD (% inhibition, upper left); ICB anti‐IgG NP (Ln S/CO, upper middle); Roche NP (Ln COI, upper right); Ortho Spike total (Ln Index, bottom left) and Roche Spike (Ln U/ml, bottom right). Horizontal red and black dashed lines represent reactive reactions based on manufacturer's instructions and according to the FDA authorization, respectively (except for ICB‐IgG, which was defined as S/CO ≥5.0); vertical black dashed line marks nAb titer ≥160 [Color figure can be viewed at wileyonlinelibrary.com]

With the exception of combined and anti‐NP IgA/M tests, the remaining surrogate tests performed well when all panel samples were tested (regardless of the nAb titer), including sensitivity (98.3%–100%), specificity (85.7%–100%), PPV (98.9%–100%), NPV (81.3%–100%), and AUC (0.93–0.96) using ROC analysis, as presented in Figure 5 (left) and Tables S3 and S4. In fact, there was no statistical difference (except for ICB‐A and‐M and the combined tests) when all surrogate tests were compared against the adopted “gold standard” (nAb≥20 by CPE‐VNT), showing similarities with those described by Patel et al. ²⁰ On the other hand, there was a substantially lower performance when simultaneously evaluating high nAb samples (≥160) and following FDA authorization for surrogate tests. Although some displayed good sensitivity, most presented lower specificity. In this case, all surrogate tests had AUCs statistically different from the high‐titer “gold standard” panel (nAb ≥ 160 by CPE‐VNT), although some approaches using anti‐RBD test inhibition ≥68% (isolated or in combination) had similar results, as shown in Figure 5 (right), Tables 3, and S5.

FIGURE 5.

ROC curve for surrogate tests compared against samples with nAb titers ≥20 (gold standard, left) or nAb titers ≥160 and FDA authorization for high‐titer samples (right). Except for ICB‐A, ICB‐M, and all combined tests (O‐Spk+anti‐RBD68 or R‐Spk+anti‐RBD68), all methods were statistically non‐significant whenever nAb titers ≥20 and manufacturer's instructions were followed (left); however, all surrogate methods following FDA authorization were statistically inferior (p < .0001) when compared to samples with nAb titers ≥160 (right). ICB‐A and ‐M were not shown in the right quadrant, because they already were statistically different by the former analysis. nAb20, neutralizing antibody titer ≥20; anti‐RBD, competitive anti‐RBD inhibition test; anti‐RBD68, ≥68% competitive anti‐RBD inhibition test; ICB‐G, anti‐IgG NP; ICB‐A, anti‐IgA NP; ICB‐M, anti‐IgM NP; R‐NP, Roche NP; O‐SPk, Ortho Spike; R‐Spk, Roche Spike; Spk‐0.8 and Spk1.0, anti‐Spike tests based on the manufacturer's cut‐off; SPk109 and SPk132, anti‐Spike tests based on FDA guidance for high‐titer; NP, nucleoprotein. *Both combinations for O‐Spk+ anti‐RBD68 presented the same result, being merged into a single line. **p < .0001 against reference (nAb20) [Color figure can be viewed at wileyonlinelibrary.com]

TABLE 3.

Performance of surrogate tests against a subset of 136 high‐titer sera (nAb titers ≥160) from convalescent donors

Surrogate test	Sens % (CI95%)	Spec % (95% CI)	PPV % (95% CI)	NPV % (95% CI)	AUC (95% CI)	LR + (95% CI)	Youden
Gen Script ≥68%	83.8 (76.5–89.6)	87.3 (75.5–94.7)	94.2 (88.4–97.6)	68.6 (56.4–79.1)	0.86 (0.80–0.91)	6.59 (3.28–13.2)	0.711
Vitros Spike ≥9.5	99.3 (96–100)	30.9 (19.1–44.8)	78.0 (71.1–84)	94.4 (72.7–99.9)	0.65 (0.59–0.71)	1.44 (1.2–1.72)	0.302
Roche Spike ≥132 UI/ml	71.3 (62.9–78.7)	90.9 (80–97)	95.1 (88.9–98.4)	56.2 (45.3–66.7)	0.81 (0.76–0.87)	7.85 (3.38–18.2)	0.622
Roche anti‐NP >109	45.6 (37–54.3)	94.5 (84.9–98.9)	95.4 (87.1–99)	41.3 (32.6–50.4)	0.70 (0.65–0.75)	8.36 (2.74–25.5)	0.401
ICB IgG anti‐NP S/CO ≥5.0	86.0 (79–91.4)	72.7 (59–83.9)	88.6 (82–93.5)	67.8 (54.4–79.4)	0.80 (0.73–0.86)	3.15 (2.04–4.88)	0.588
Comb OSp9 + RBD68	83.8 (76.5–89.6)	87.3 (75.5–94.7)	94.2 (88.4–97.6)	68.6 (56.4–79.1)	0.86 (0.80–0.91)	6.59 (3.28–13.2)	0.711
Comb RSp132 + RBD68	25.0 (18.0–33.1)	100 (93.5–100)	100 (89.7–100)	35.0 (27.6–43.0)	0.62 (0.50–0.66)	NE a	0.250

Note: All AUCs had p < .0001 when compared to the standard CPE‐VNT test.

^a NE, Not possible to be estimated, due to absence of reactive samples in one of the four categories: nAb neg/test neg; nAb neg/test pos; nAb pos/test neg or nAb pos/test pos.

4. DISCUSSION

There is a great concern in the selection of high‐titer COVID‐19 convalescent plasma (CCP) for COVID‐19 therapy, as nAb testing utilizing either conventional live virus (microneutralization methods, cVNT) or pseudotype virus (pVNT) requires sophisticated labs and methods that are not usually available. An alternative is to use tests designed to detect the RBD, spike S1‐protein subunit, or nucleocapsid protein (NP) antibodies, which must be adequate for CCP screening and preferentially capable of evaluating nAb levels in plasma bags, with different anticoagulant solutions, or undergoing pathogen‐reduction treatment.

We tried to establish a panel with an appropriate number of samples by selecting a number above the minimum required for validation of the testing methods. ²¹ Although nAb assays are not licensed in many countries, ²² their use has been a long‐standing practice in research laboratories.

In this study, we used both types of tests to promote a comprehensive validation for the evaluation of a suitable replacement test for nAb tests using a live virus or pseudovirus cultures, which are laborious, complex, and require specialized laboratories (BSL3 and BSL2 classes, respectively). For ligand tests, we used both an enzyme‐linked immunosorbent assay (ELISA; IgG, IgA, and IgM anti‐NP) and chemiluminescent assays (CLIA) directed against either the spike or NP region.

We attempted to assemble a panel with homogeneous nAb titer distribution. The observed unequal nAb titer distribution could be due to the selection of high‐titer only donations, initially selected for both pathogen‐reduction treatment and transfusion.

The date of the collection samples after the onset of symptoms (DOS) correlates to different sensitivities among several commercial anti‐SARS‐CoV‐2 antibody tests. ²⁰ , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ This panel is derived only from convalescent individuals, usually ≥28 days after DOS, posing some homogeneity as far as the time of collection is concerned.

Since all CCP donors must have had a previously proven infection (by RT‐PCR), there was no need to include very early samples in this panel, as none are expected to be in the window period of infection. Furthermore, all accepted donors had to collect an additional nasopharyngeal RT‐PCR at least 14, or preferably 28 days after the end of symptoms in order to confirm a true previous infection and to be accepted for CCP donation. Furthermore, nAbs tend to be stable for months ²⁸ , ²⁹ within the span period between infection and collected samples for this panel. Thus, false‐negative reactions are unlikely to occur.

It was not the scope of this study to measure the clinical or analytical performance of these tests, as described elsewhere, ²⁷ , ³⁰ , ³¹ but rather to validate its applicability as a surrogate test. There was no intention to correlate how high‐titer sera (nAb>160) would be compared to signal/cut‐off ratios, although partially demonstrated in Figure S1. In addition, this study did not define true positive and negative nAb samples (sensitivity and specificity, respectively); we attempted to define a relative sensitivity and specificity based on a pre‐defined high nAb titer cut‐off (≥160), in order to evaluate whether surrogates would have an acceptable performance for nAb test replacement. Naturally, a relatively low sensitivity would lead to rejection of true high nAb candidates, whereas a low specificity would result in accepting donors who actually do not have high nAb titers, with potential impairment of CCP therapeutic value. ³² , ³³

It is known that tests targeted at NP proteins might behave differently than those that react against spike‐based tests, which would also explain the variability found in this work. ²³ , ³⁴ In addition, when working only with a selected panel of previously known nAb status, one could consider creating an artificial population where a high prevalence rate would be expected. According to Bayes' theorem, ³⁵ the predictive value of a test (positive and negative) depends on both the test performance and the local prevalence of the marker. Since we artificially produced a high prevalence group of samples, one should expect that the PPV would be high.

The competitive surrogate neutralization test (sVNT) used in this study is a qualitative binding assay with the capacity to detect antibodies known to block the viral interaction with ACE2 receptor, by using a purified RBD molecule as a competing agent, measuring the RBD‐hACE2 receptor interaction, elucidating the neutralizing antibody's function (and probably other molecules), though not specific to any isotype (IgM, IgG, or IgA). ³⁶ It can be tested in large capacities and does not require sophisticated labs (BSL2/3). The test has shown a good correlation with PRNT methods, ³⁷ probably due to its better functional ability to correlate with real nAbs. Its limitation is that it is not capable of detecting those few individuals producing nAbs only against the NTD part of the Spike protein.

On the other hand, binding surrogate tests that use the S protein as a target might be capable of higher cross‐reactivity, since they can use either the complete S protein (S1 + S2) or parts of it (e.g., S1 domain or even RBD). Therefore, some variability in the behavior of S‐based tests can be expected when used as surrogates for nAbs. Given that this panel simulates a high‐prevalence population, we were surprised to find a low specificity with the CoV2T test; however, similar findings were already described by Goodhue Meyer et al. ³⁸ Finally, although the N protein is much more conserved across several coronaviruses, it produces earlier antibodies directed against epitopes that probably have no role in neutralizing behavior (different from nAbs). ³⁰ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴²

We adopted a quadratic model correlation for all surrogate tests, which gave superior R² when compared with linear regression models, including lower residuals, increasing the robustness of the model, which might explain the best results observed between the sVNT (GenScript) and the nAb tests, ⁸ a fact not observed by others. ³⁷

If CCP screening is applied for normal blood donors with a previous COVID‐19 infection, then a low prevalence in the screened population might be expected, resulting in low PPV values. Thus, the use of an orthogonal approach, using two independent tests, ²¹ , ²² would certainly enhance the estimated final PPV, perhaps reaching ≥99%. ⁴³ Although we have also explored this approach, our results were not superior to some isolated approaches, mainly because we already started with samples with a high post‐test positive probability.

Similar to the previous papers, ¹³ , ¹⁴ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ we found variable sensitivity based on nAb titers, albeit with a positive correlation between signal intensity of surrogate tests and nAb titers, corroborating the rationale for its use as a proxy method. In addition, we were following the fact that these samples had the potential to be already detected by all chosen antibody tests, since samples came from definitively proven diagnosed cases, where one expects that binding antibodies should have a high likelihood of reactivity.

With the exception of ICB anti‐IgG, ‐A, and ‐M tests, all chosen surrogates are “total antibody” tests, which show a higher sensitivity than a single isotype assay, ¹⁰ increasing the probability of nAb correlation, albeit at the cost of reduced specificity for some of them.

We chose not to include tests that allegedly present a decay in sensitivity after recovery of symptoms, ²⁶ , ³⁴ , ⁵⁰ , ⁵¹ , ⁵² avoiding confounders or mathematical adjustments for analysis. However, there is rarely a perfect test. This imperfection would probably be enhanced when using a panel like ours, moreover when the definition of a positive reaction would be the capacity for detecting samples with a nAb titer above a given threshold (≥160), which would cause more confusion in the interpretation of results derived from “imperfect tests.” ⁵³ We believe that the variability observed in our results does not pose any demerit on the quality of the studied tests; rather, it calls attention to the need to look at the available commercial tests in a different way than the one originally designed (i.e., a diagnostic purpose), since screening for high nAb titer convalescent donors would not be a priority for all companies responsible for test development. ⁵⁴ Therefore, each lab conducting their own screening program should implement validation procedures before adopting surrogate tests. ²¹ Perhaps it would be better to state that instead of claiming that the sample has “no high nAb titer” it would be more appropriate to recognize that there was no capacity for high nAb recognition from a given surrogate test; in other words, one has to play with probabilistic measures before reaching a final conclusion, given the wide result diversity from tests currently adopted in blood transfusion services. ⁵⁵ Regardless of the final decision to be made, one has to live with uncertainties in this world of still many unknown issues concerning SARS‐CoV‐2, particularly when there are hundreds of tests currently available or licensed ¹¹ , ⁵⁶ (available at http://www.finddx.org/covid-19/pipeline/) that have not been discussed a year ago. It is likely that with the newly released WHO anti‐SARS‐CoV‐2 standard, ⁵⁷ manufacturers will review their tests in order to achieve better standardization, both in terms of nAbs (as international standard unit, IU/ml) or binding assay formats (given in BAU/ml).

The design of this evaluation is somewhat different from that described earlier, ³⁶ since we did not define RT‐PCR tests as a reference. Since all individuals were convalescent, RT‐PCR was not deemed important; rather, we determined that nAbs should be the reference test. As expected, sVNT had an excellent sensitivity when compared to neat nAb samples, but also failed to detect some high nAb titer sera (22/136 sera or 16.2%). Whether this is due to the fact that a minority of nAbs are targeted against regions other than RBD, ⁵ , ⁴⁰ , ⁵⁸ , ⁵⁹ presence of anti‐ACE2 autoantibodies ⁶⁰ or impaired sensitivity of the test remains to be determined.

Although both nAb and anti‐spike antibodies maintain stable levels for several days at refrigerated temperatures, ⁶¹ we tried to use our samples as soon as possible after thawing to avoid any interference.

Another potential problem in the near future is the value of anti‐spike tests when screening vaccinated people (including booster doses), since infection by the virus will naturally lead to the development of antibodies against most of the main SARS‐CoV‐2 antigens, whereas most licensed vaccines are directed only against S or S subunits, including RBD. Thus, if a surrogate marker is used for a vaccinated‐only individual, it is likely that the test will be positive, although it does not necessarily indicate the presence of nAbs. By definition, these individuals cannot be considered convalescent (although possibly immune). We emphasize the importance of not accepting vaccinated‐only individuals for CCP collection, where a previous infection has not been carefully evidenced.

There are no benefits in using either IgA or IgM antibodies for CCP screening, since they show no adequate persistence after recovery, as evidenced by others. ⁶ , ⁴¹ , ⁶²

Our data also showed that pathogen reduction treatment did not interfere with the surrogate test results, which is an incentive for implementing pathogen‐reduction for CCP collection, posing no problem concerning the level of transfused nAb in the final pathogen‐treated CCP and probably did not interfere with their potential therapeutic effect. We believe that this point will be explored further in the future.

One limitation of this study is that there were no early seroconversion samples, since the purpose was to evaluate the behavior of several tests in order to screen high‐titer CCP donations, where all donors had to be, preferably, at least ≥28 days after the end of symptoms. All donors had mild or moderate disease and a mean (±SD) age of 37.7 ± 9.9, who might present lower nAb levels. ²⁹ , ⁶² , ⁶³ , ⁶⁴ In addition, some donors contributed more than one sample in the panel, albeit in different periods. Most samples were derived from the first pandemic wave, where the original Wuhan type was still predominant, before the introduction of variants of concern in Brazil – alpha, gamma, and, more recently, delta. Thus, one can argue that perhaps the same findings might differ after different variants have challenged our population. However, it seems that most surrogate tests have the capacity to detect antibodies against these new variants, ⁶⁵ although this might be a problem for molecular diagnostic tests. ⁶⁶ Nevertheless, this work reflects a snapshot of the moment, and continuous surveillance is required with the evolution of pandemics and global vaccination. Finally, all donors were from the São Paulo region, so we cannot define whether geographical regions might play a different role because of possible different viral types or immunological responses in people from different geographical or ethnic groups.

Although we defined one model for analysis, we agree with the famous quote from George Box that “all models are wrong, but some are useful”; in addition “one should not try to obtain a correct one through an excessive elaboration.” ⁶⁷ Furthermore, it is important to be cautious that if the presence of nAb in CCP donors might be a direct evidence of the potential therapeutic effect of plasma, adoption of a surrogate does not automatically mean the same, given that if an assumption A is true, it does not necessarily follow for a B one.

In conclusion, although surrogate tests have a good performance in detecting the presence of nAbs for clinical diagnosis on a qualitative basis, our data show that all surrogate tests used in this study are not perfectly correlated with the adopted “gold standard” method (CPE‐VNT nAb), particularly for sensitivity and specificity when high nAb titer samples are used, once they might also detect non‐nAbs, which might lead to the acceptance of a considerable number of donors whose nAb titers are actually low, although recognized as appropriate for CCP donation by the chosen surrogate test. This fact must be carefully evaluated by each lab responsible for CCP collection; after all, there is currently no perfect test for this purpose and whatever decision is taken, one must carefully evaluate their pros and cons.

CONFLICT OF INTEREST

The authors have disclosed no conflicts of interest.

Supporting information

Appendix S1. Supporting Information.

Wendel S, Fachini R, Fontão‐Wendel RCL, Mello R, Velasquez CV, Machado RRG, et al. Surrogate test performance for SARS‐CoV‐2 neutralizing antibodies (nAbs) for convalescent plasma (CCP): How useful could they be? Transfusion. 2021;61:3455–3467. 10.1111/trf.16714