High-Throughput Neutralization and Serology Assays Reveal Correlated but Highly Variable Humoral Immune Responses in a Large Population of Individuals Infected with SARS-CoV-2 in the US between March and August 2020

ABSTRACT

The ability to measure neutralizing antibodies on large scale can be important for understanding features of the natural history and epidemiology of infection, as well as an aid in determining the efficacy of interventions, particularly in outbreaks such as the current severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. Because of the assay’s rapid scalability and high efficiency, serology measurements that quantify the presence rather than function of serum antibodies often serve as proxies of immune protection. Here, we report the development of a high-throughput, automated fluorescence-based neutralization assay using SARS-CoV-2 virus to quantify neutralizing antibody activity in patient specimens. We performed large-scale testing of over 19,000 COVID-19 convalescent plasma (CCP) samples from patients who had been infected with SARS-CoV-2 between March and August 2020 across the United States. The neutralization capacity of the samples was moderately correlated with serological measurements of anti-receptor-binding domain (RBD) IgG levels. The neutralizing antibody levels within these convalescent-phase serum samples were highly variable against the original USA-WA1/2020 strain with almost 10% of individuals who had had PCR-confirmed SARS-CoV-2 infection having no detectable antibodies either by serology or neutralization, and ~1/3 having no or low neutralizing activity. Discordance between neutralization and serology measurements was mainly due to the presence of non-IgG RBD isotypes. Meanwhile, natural infection with the earliest SARS-CoV-2 strain USA-WA1/2020 resulted in weaker neutralization of subsequent B.1.1.7 (alpha) and the B.1.351 (beta) variants, with 88% of samples having no activity against the BA.1 (omicron) variant.

INTRODUCTION

The emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in December 2019 is responsible for the ongoing global pandemic of COVID-19 disease, with more than 600 million infections and over six million deaths (as of November 2022). In response to this unprecedented crisis, laboratories, universities, and companies worldwide raced to develop effective therapeutic and diagnostic methods. One important component of this response was the rapid development of serological assays to measure the presence of antibodies as a marker first of infection and later of vaccination (1). However, these serological measurements only serve as proxies of immune protection, as they measure the presence of antibodies rather than their functional ability to neutralize virus (2).

The ability to directly measure neutralizing antibodies on live SARS-CoV-2 virus in individuals can play an important role in understanding the efficacy of interventions. For example, the passive transfer of neutralizing antibodies is often considered the first available potential therapeutic approach against emerging novel pathogens. Assessment of levels of neutralizing antibodies in donor and recipient plasma may be necessary to determine the utility of COVID-19 convalescent plasma (CCP) in infection. One key factor likely to influence the impact of CCP therapy on clinical outcomes is the neutralizing antibody titer of the donor CCP, with high antibody titer CCP conferring the greatest benefit, and the titers of recipients, with individuals with low titers potentially deriving the most benefit from the CCP therapy.

The direct quantification of virus neutralizing antibody activity is commonly performed using a plaque reduction neutralization test (PRNT) and is considered the “gold standard” (3). The standard plaque assay is performed by infecting host cells with a suspension of virus, overlaying the infected cells with agar, and enumerating the number of PFU that result as a relative quantification of virus. The method is cumbersome, and thus not well suited to large-scale implementation, particularly on biosafety level-3 (BSL-3) pathogens such as SARS-CoV-2. Antibodies that neutralize SARS-CoV-2 bind the spike protein, which is the viral surface protein that engages the host cell’s angiotensin-converting enzyme 2 (ACE2) receptor upon infection (4). More specifically, antibodies against the receptor binding domain (RBD) of the spike protein are largely responsible for neutralization. As such, neutralization assays utilizing viral pseudotypes, typically lentivirus or vesicular stomatitis virus engineered to express the SARS-CoV-2 spike protein, were developed as convenient surrogate platforms, amenable to BSL-2 facilities, to detect and quantify SARS-CoV-2 neutralizing antibodies using a fluorescence or luminescence readout (5, 6). However, the widespread application and standardization of these assays may be limited when testing antibodies against SARS-CoV-2 variants with spike mutations or due to significant variability between different pseudotyped viruses, assays, and laboratories.

Here, we report the development of an automated high-throughput assay for measuring neutralizing antibody titers of human samples against fully replication-competent wild-type SARS-CoV-2 virus. We applied this assay to over 19,000 samples from individuals infected by SARS-CoV-2 between March and August of 2020 obtained from donors during an open-label, expanded access program of CCP for the treatment of SARS-CoV-2 infection early during the COVID-19 pandemic (7), and compared neutralization titers to either levels of IgG antibodies against RBD, or levels of IgG antibodies against nucleoprotein (N), another viral protein critical for viral replication and highly conserved among coronaviruses. While there was moderate correlation between the two measurements, we found that there was a wide range of antibody titers measured across serological and neutralization assays among individuals who had confirmed SARS-CoV-2 infection. There was very little cross-protection afforded to later viral variants such as the Omicron variant. Meanwhile, a small set of individuals that tested serologically negative for anti-RBD IgG, the most common target of serological measurements, did have CCP with neutralizing activity due to other immunoglobulin classes. These large-scale measurements were used to determine the clinical efficacy of the FDA’s emergency use authorization for CCP administration early during the pandemic.

RESULTS

Development of high-throughput fluorescent SARS-CoV-2 neutralization assay (NA).

We developed an assay using a TMPRSS2-expressing VeroE6 cell line (VeroE6/TMPRSS2). Transmembrane protease serine 2 (TMPRSS2) facilitates SARS-CoV-2 entry into host cells, thereby making cells highly susceptible to the virus (8, 9). We evaluated the reference strain SARS-CoV-2 USA-WA1/2020 in a range of multiplicity of infections (MOI) and found that an MOI of 0.002 yielded a reliable cell infection rate of approximately 60 to 70% after 40 h with minimal cytopathic effect. At 40 h postinfection, cells were fixed and infection was quantified using indirect immunostaining and imaging. We detected SARS-CoV-2 by visualization using a mouse anti-SARS-CoV-2 N protein antibody followed by a secondary Alexa Fluor 488 conjugated anti-mouse IgG antibody; cell nuclei were stained with Hoechst 33342 (Fig. 1a and b). We used rabbit anti-SARS-CoV-2 spike neutralizing antibody (5 μg/mL; Sino) as a positive control. The Z’-factor of the assay was 0.81 in a 384-well format.

FIG 1.

Workflow and validation of high-throughput fluorescent SARS-CoV-2 neutralization assay. (a) Overview of high-throughput neutralization assay for measuring neutralizing antibody titers against SARS-CoV-2. Patient serum samples are serially diluted and mixed with SARS-CoV-2. VeroE6/TMPRSS2 cells are then infected with the virus/serum mixture. The infected cells are fixed, immunostained and imaged. (b) Representative immunofluorescence images of VeroE6/TMPRSS2 cells infected with SARS-CoV-2 in the absence (left) or presence (right) of a positive neutralizing serum (right), 40 h postinfection. Scale bar, 200 μm. (c) Exemplary sigmoidal dose response curves across a dilution series of 4 different serum samples. The data are presented in mean ± SD (n = 3, technical replicates) of one representative experiment. (d) Correlation between ID₅₀ values of the PRNT and neutralizing assays. The Pearson correlation efficiency R² is shown. (e) Neutralization curves for the anti-RBD human IgG neutralizing monoclonal antibody in PRNT and neutralizing assays. The data are presented in mean ± SD (n = 3, independent experiments).

To test patient serum samples, we preincubated a series dilution of patient samples, ranging from 1:20 to 1:5,120, with virus at 37°C for 1 h to allow neutralization. The virus/sample mixtures were then added to VeroE6/TMPRSS2 cells in 384-well plates and infection was allowed to occur for 40 h. After washing, cells were immunostained and imaged. Data were fitted to a sigmoidal dose response curve across the dilution series (Fig. 1c). Neutralization titers (ID₅₀) were calculated as the reciprocal of the serum dilution required to obtain a 50% reduction in the percentage of infected cells compared to control untreated SARS-CoV-2 alone.

We compared the performance of the NA with the gold standard plaque reduction neutralization test (PRNT) on a panel of 11 serum samples from individuals confirmed to have been infected by SARS-CoV-2 and a control anti-RBD human IgG neutralizing monoclonal antibody (Acro). Neutralizing titers (ID₅₀) as measured by the NA and PRNT were strongly correlated (R² = 0.86; Fig. 1d). In both assays, the anti-RBD human IgG neutralizing MAb exhibited similar inhibitory concentration (IC₅₀) values of ~0.5 μg/mL (Fig. 1e). NA exhibited a steeper dose-response curve than PRNT, likely due to the different sensitivities of the two assays.

To determine the best dilution series to be used in the assay and the threshold to define negative samples, we tested a set of 51 serum samples collected before the start of the COVID-19 pandemic, and thus from presumably negative individuals and 124 samples from PCR-confirmed positive individuals obtained from the Massachusetts General Brigham (MGB) Biobank. All samples were tested in a series of 4-point, 4-fold dilutions starting at 1:40. All but one of the 51 prepandemic samples had no neutralizing activity at the 1:40 dilution, with the one sample having a calculated ID₅₀ value of 50. Using an ID₅₀ value of 40 as the breakpoint that distinguishes between positive and negative samples would thus define an assay specificity of 98%. Of the 124 samples from PCR-confirmed individuals, 85.5% had ID₅₀ cutoffs greater than 40 (Fig. S1a). The precision of the NA was assessed by testing a set of 418 samples in 4 replicates over 2 days. The replicates showed a significant correlation between two replicates on the same day with an R² value of 0.89 to 0.90 and on two separate days with an R² value of 0.86 (Fig. S1b).

FIG S1

High-throughput fluorescent neutralization assay (NA) validation. (a) Neutralization titers of 51 prepandemic samples (negative control) and 175 serum samples of PCR-confirmed COVID-19 patients from the Massachusetts General Brigham (MGB) Biobank. The data for the PCR-confirmed samples are further separated by the duration between date of PCR-positive test and date of sample collection. Prepandemic samples had almost no neutralizing activity, with 1 of 51 samples having an ID₅₀ just above the cutoff of 40 (>98.0% specificity). In contrast, 85.5% of PCR-positive samples had ID₅₀ values greater than 40; (b) Replicate-to-replicate and day-to-day variability of the NA assay. Samples from the validation set (n = 418) were tested in duplicate in the same batch on the same day to evaluate replicate-to-replicate variability (top two panels) and on two separate days to assess day-to-day variability (bottom panel). Each dot represents a single sample. Correlation analysis was performed on log-transformed values. The solid lines denote complete correlation, and the black dotted lines denote a 2-fold change in either direction. Download FIG S1, PDF file, 0.1 MB ^{(150.1KB, pdf)} .

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.

FIG S2

Abundance of RBD-specific IgG as measured by the serology assay. Serology of 492 serum samples from PCR-confirmed COVID-19 patients and 1,221 prepandemic samples were analyzed by RBD IgG. Each dot represents a single sample. The dotted lines indicate assay cutoffs for negative, indeterminate and positive samples, respectively. Download FIG S2, PDF file, 0.01 MB ^{(11.8KB, pdf)} .

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.

FIG S3

Comparison of 8-point to 4-point dilution series. Sample from the validation set (n= 418) were prepared at an 8-point 2-fold dilution series. Two median ID₅₀ values were calculated by fitting a four-parameter sigmoidal curve to data points from (i) all eight dilutions from 1:40 to 1:5,120 in 2-fold dilutions and (ii) four of the eight dilutions from 1:40 to 1:2,560 in 4-fold dilutions. Each dot represents a single sample. The solid line denotes the completed correlation between 8-point and 4-point analysis methods and dotted lines denote a 2-fold change in either direction. Download FIG S3, PDF file, 0.1 MB ^{(73KB, pdf)} .

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.

FIG S4

Correlation between anti-RBD IgG and anti-N IgG levels. Correlation analysis of the estimated abundance of anti-RBD IgG (y-axis) and Z-score abundance of anti-N IgG (x-axis) of the CCP sample set (n = 19,729). Each dot represents the IgG level of one sample. For visualization purposes, anti-N IgG Z-score abundance values less than or equal to 0 were assigned the value 0.005 in the plot. The nonparametric Spearman correlation coefficient (r_s) is shown. Download FIG S4, PDF file, 1.2 MB ^{(1.2MB, pdf)} .

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.

Development of high-throughput anti-RBD and anti-N protein serology assay.

In order to compare anti-SARS-CoV-2 antibody quantitation with functional neutralization, we also developed a sensitive and highly specific high-throughput enzyme-linked immunosorbent assay (ELISA) to quantify SARS-CoV-2 spike protein RBD- or N protein-specific antibody levels. We quantified antibody isotypes by measuring the colorimetric conversion of the substrate 3,3′,5,5′-tetramethylbenzidine (TMB) by horseradish peroxidase (HRP) conjugated to anti-Ig (IgG, IgA, or IgM) secondary antibodies (see Materials and Methods, Fig. 2a). Absorbance values for a dilution series of standards followed the expected sigmoidal curve and were highly correlated between replicates (R² = 0.997 for RBD and 0.994 for N; Fig. 2b). Moreover, ELISA results for independent aliquots analyzed on different days were highly reproducible (R² = 0.87 to 0.92; Fig. 2c).

FIG 2.

Workflow and validation of high-throughput anti-RBD and anti-N protein serology assay. (a) Overview of high-throughput serology assay for measuring anti-RBD and anti-N antibodies. Precoated SARS-CoV-2 RBD or nucleocapsid antigens capture IgG antibodies from the patient sample. Secondary anti-IgG antibodies that are conjugated to horseradish peroxidase (HRP) bind to the captured primary IgG antibodies and react with an absorbance-based substrate to generate a signal. (b) Representative standard curves of the anti-SARS-CoV-1-spike control antibody (CR3022) (left) and a patient sample that was previously shown to have a high anti-N IgG level (right) assessed in duplicate. The Pearson correlation efficiency R² values are shown. (c) Day-to-day reproducibility of the anti-RBD (left) and anti-N serology assay (right). Samples from the independent validation set (n = 420) were evaluated on two consecutive days and correlation analysis performed on log-transformed values. Pearson correlation efficiency R² values are shown.

To assess the performance of the serology assay, we measured anti-RBD and -N antibody abundances in 1,221 prepandemic negative serum samples and 492 positive samples from PCR-confirmed SARS-CoV-2 infected individuals obtained from the Massachusetts Consortium on Pathogen Readiness (MassCPR) and the Mass General Brigham (MGB) Biobank. To convert scaled absorbances into serology status classifications, we set the thresholds for “serology indeterminate” and “positive” samples as 3 to 6 and > 6 standard deviations (SD) above the mean scaled abundance of prepandemic negative controls, respectively. Applying these thresholds to the MassCPR and MGB samples, the ELISA had a sensitivity of 80% and a specificity of 97% (Fig. S2). Of note, this calculated diagnostic sensitivity includes samples from individuals who were PCR-confirmed infected but had not yet developed antibodies or had lost antibodies by the time of sample collection, as samples were collected between days 0 to 67 after diagnosis. As such, our 80% sensitivity may be underestimating the true sensitivity of our assay. Nonetheless, our assay’s sensitivity and specificity were comparable to those achieved when the same set of samples were tested with four other immunological assays (10).

Neutralization and serologic testing of a set of donor convalescent plasma.

We first applied the NA and ELISAs to an initial panel of 418 de-identified samples to compare the serology with functional neutralization assays. This panel, termed the “validation set,” was collected from donors as part of the U.S. Expanded Access Program (EAP) to CCP early in the COVID-19 pandemic (11). Samples were collected from individuals between 14 days and 6 months of COVID-19 diagnosis (12) and provided by blood banks across the country, as coordinated by the Biomedical Advanced Research and Development Authority (BARDA). Consistent with previous reports, the sample type (serum and plasma) did not affect the neutralizing activity and ELISA antibody titers (13, 14) (data not shown), so we tested both types of patient samples, depending on availability provided by participating blood banks. Using the ELISA to detect both anti-RBD and anti-N IgG abundance, 389 samples (93%) were positive for anti-RBD antibodies while 29 samples (17%) were negative or indeterminate. Meanwhile, only 162 samples (39%) were anti-N IgG antibody positive while 256 samples (63%) were negative or indeterminate (Fig. 3a). In the RBD ELISA, the median value of CCP samples was 56.9 μg/mL (95% CI, 52.5 to 64.2 μg/mL). In the N ELISA, the median relative abundance of CCP samples was 4.4 (95% CI, 3.9 to 5.1). Only 4.5% had neither anti-RBD nor N protein antibodies (Fig. 3b).

FIG 3.

Neutralizing activity and serological analysis of independent validation set (n = 418). (a) Serology of validation set showing proportion and number of samples with positive, indeterminate or negative result for anti-RBD and anti-N IgG antibody. (b) Frequency of samples that are anti-RBD and anti-N IgG antibody positive, indeterminate, and negative among the validation set. (c) Distribution of neutralization ID₅₀ values of the validation set. The left red bar indicates samples with neutralization titers below the limit of detection of the assay (ID₅₀ < 40). Samples with ID₅₀ values that exceeded the last dilution (1:5,120) were plotted as > 5,120. (d) Correlations between the estimated abundance of anti-RBD IgG (left) or Z-score abundance of anti-N IgG (right) and the SARS-CoV-2 neutralization titers, with each point representing an individual sample in the validation set. For visualization purposes, anti-N IgG Z-score abundance values less than or equal to 0 were assigned the value 0.005 in the plot. The nonparametric Spearman correlation coefficients (r_s) are shown. The vertical black dotted lines indicate assay cutoff values for indeterminate and positive samples. The horizontal red dotted line indicates an ID₅₀ value of 40. (e) Neutralization capacity of the validation set against the reference SARS-CoV-2 WA2020/1 strain and three other variants. Each point represents an individual sample in the validation set. Solid lines represent the median titer, and the whiskers show the 95% confidence interval. The red dotted line indicates an ID₅₀ value of 40. **, P < 0.01; ****, P < 0.0001.

We next measured the ID₅₀ values of this validation set using the NA in triplicate and found that the neutralization activities ranged from an ID₅₀ value (median from three replicates) of less than 40 to higher than 2,560, with a median of 525.7. The large majority of the samples (93.8%) showed neutralizing activity above the limit of detection (ID₅₀ > 40) (Fig. 3c). The Spearman correlation, which does not assume linearity, between serological results and neutralization activity was better for anti-RBD IgG (r_s = 0.62) than for anti-N IgG (r_s = 0.56). However, both antibody abundances showed moderate correlation with NA ID₅₀ values (Fig. 3d).

Testing of this validation set allowed us to explore the possibility of increasing the throughput of the NA by comparing the ID₅₀ values of these 418 samples calculated from an 8-point versus a 4-point dilution series. When we compared ID₅₀ values calculated by fitting a sigmoidal curve to all eight data points with those obtained from fitting to four data points in a 4-fold dilution series spanning the same dilution range, we found a linear relationship, with a Pearson correlation coefficient of 0.86 (Fig. S3), thus demonstrating that reducing the number of dilutions tested could be done without loss in performance. The reduction in datapoints did result in a banding pattern of fitted ID₅₀ values around the tested dilutions because the calculated ID₅₀ is based on fitting a sigmoidal curve with the four data points determining the resolution at which the ID₅₀ value could be determined. In cases where the transition from minimum to maximum infection rate occurred between two data points, without more data to define the features of the curve, the fitting algorithm estimated the ID₅₀ value to be close to one of the dilution points, thus producing a banding pattern with the 4-point dilution series that is not observed with the 8-point dilution series (Fig. 3d). Importantly, the high correlation between 4-point and 8-point ID₅₀ values suggested that accurate ID₅₀ values could be obtained using an assay format that generates data from a 4-point, 4-fold dilution method. This simplified assay was thus used in subsequent studies.

Neutralizing activity against viral variants.

In addition to measuring the neutralizing activity on the original WA1/2020 strain, we also tested the activity of 394 samples from the validation set against three clinically relevant SARS-CoV-2 variants, B.1.1.7 (alpha), B.1.351 (beta), and BA.1 (omicron) to explore the application of the assay to emerging variants. Of note, before and up to the time when the samples were collected, the dominant circulating strain was WA1/2020. In this sample set, the median neutralizing antibody titer was 532.6 against WA1/2020, 390.4 against B.1.1.7, 77.1 against B.1.351, and 20 against BA.1. The median neutralizing antibody titers against B.1.1.7, B.1.351, and BA.1 were reduced by 1.4-fold (P = 0.0069), 6.9-fold (P < 0.0001), and 26.6-fold, respectively (P < 0.0001). Notably, a majority of samples contained no neutralization activity against B.1.351 and BA.1, with 144 samples (36.5%) and 345 samples (87.5%), respectively, having below the limit of detection (ID₅₀< 40) (Fig. 3e).

Neutralization and serologic testing of convalescent plasma on a large-scale.

To characterize the antibody activity after SARS-CoV-2 infection on a large-scale, we tested 19,729 de-identified CCP samples provided by BARDA, with the vast majority collected between March and August of 2020 from across the United States by blood banks as part of the open label Expanded Access Program for CCP to treat SARS-CoV-2 infection (11). The time period between complete resolution of symptoms from SARS-CoV-2 acute infection and blood donation varied from 14 days to 6 months among the donors (12). All collected samples were evaluated for neutralizing activity against the WA1/2020 strain using the 4-point dilution assay and serology for SARS-CoV-2 specific anti-RBD and anti-N antibodies using a 1:100 sample dilution.

ELISA results showed that 91.2% and 80.3% of this large set of CCP samples were positive for antibodies against RBD and N IgG, respectively (Fig. 4a). In the RBD ELISA, the median value of CCP samples was 54.0 μg/mL (95% CI, 53.8 to 55.2 μg/mL). In the N ELISA, the median relative abundance of CP donor samples was 24.6 (95% CI, 23.9 to 25.3). In examining the correlation between anti-N and anti-RBD IgG antibody levels, we found that 77.5% of the samples were double-positive (positive for both anti-RBD and anti-N IgG), while 7.4% samples were single-positive (6% anti-RBD alone and 1.4% anti-N alone). Mirroring what was observed in the validation set, 3.5% of samples had no detectable IgG antibodies to either antigen (Fig. 4b). The anti-RBD and anti-N IgG abundance were moderately correlated (r_s = 0.61) (Fig. S4). Taken together, these data demonstrated that the majority of CCP samples had a wide range of antibody levels specific to immunogenic SARS-CoV-2 antigens RBD and N.

FIG 4.

Large scale CCP neutralization activity and serological analysis. (a) Serology of 19,729 samples showing proportion and number of samples with positive, indeterminate or negative result for anti-RBD and anti-N IgG antibody. (b) Frequency of samples that are anti-RBD and anti-N IgG antibody positive, indeterminate, and negative among the entire 19,729 CCP sample set. (c) Distribution of neutralization ID₅₀ values of the entire 19,729 CCP sample set. The left red bar indicates samples with neutralization titers below the limit of detection of the assay (ID₅₀ < 40). Samples with ID₅₀ values exceeded the last dilution (1:2,560) were plotted as > 2,560. The black dotted line indicates an ID₅₀ value of 160, which is the FDA recommended cutoff for use in CCP therapy. (d) Correlations between the estimated abundance of anti-RBD IgG (left) or Z-score abundance of anti-N IgG (right) and the SARS-CoV-2 neutralization titers, with each point representing an individual sample in entire 19,729 CCP sample set. For visualization purposes, Z-score abundance of anti-N IgG values less than or equal to 0 were assigned the value 0.005 in the plot. The nonparametric Spearman correlation coefficients (r_s) are shown. The vertical black dotted lines indicate assay cutoff values for indeterminate and positive samples. The horizontal red dotted line indicates an ID₅₀ value of 40. (e) Frequency of samples that are anti-RBD and anti-N IgG antibody positive, indeterminate, and negative among the NA positive samples (ID₅₀ > 40).

Ninety-two percent of the CCP samples had measurable neutralizing activity above the detection threshold of ID₅₀ > 40 (Fig. 4c) against WA1/2020, of which 95.8% and 84.5% tested positive for antibodies against RBD and N IgG, respectively. However, neutralizing activities of CCP donor samples were also highly variable, with a median ID₅₀ value of 435.4 (95% CI, 947.7 to 985.1), and a range from < 40 to > 2,560. 1,570 (8.0%) of the CP donor samples had no measurable neutralizing activity (ID₅₀ < 40), 4,961 (25.1%) had low titers (ID₅₀ 40 to 160), 6,145 (31.1%) had moderate titers (ID₅₀ 160 to 640), 4,930 (25.0%) had high titers (ID₅₀ 640 to 2,560), and 2,123 (10.8%) had very high titers > 2,560. Thus, about two-thirds (67.9%) of CP donor samples had moderate to high neutralizing activity, as quantified by an ID₅₀ ≥ 160 (which was the FDA’s initial recommended titer cutoff value for CCP therapy) (12), while one third (33.1%) had low to no activity, as quantified by an ID₅₀ < 160, and fell below the recommended threshold.

As was observed with the validation set, there was moderate correlation between serology and neutralization ID₅₀s in the CCP samples (anti-RBD IgG, r_s = 0.63; anti-N IgG, r_s = 0.50) (Fig. 4d), where, unsurprisingly, ID₅₀ values were better correlated with anti-RBD IgG levels than with anti-N IgG levels.

Discordance between neutralization and serology.

Although moderate correlations were observed between serological test results and neutralizing titers, we identified a small set of CCP samples that were discordant. Of the 18,159 CCP samples that were positive for neutralizing activity (ID₅₀ > 40), 82.4% were double positive for both anti-RBD and anti-N IgG, while 6.6% were single positive (5.7% anti-RBD only and 0.9% anti-N only) and 10.6% were indeterminate (Fig. 4e). The remaining 75 samples (0.4%) with neutralizing activity were double negative, having no detectable anti-RBD or anti-N IgG, with 57 of these samples having NA with an ID₅₀ ≥ 160. Considering that the neutralizing activity might result from the presence of other antibody isotypes against RBD or other neutralizing antibodies against the spike protein S1 domain outside RBD, we sought to measure levels of anti-RBD IgA and IgM and S1-specific IgG levels in a subset of these discordant samples. Due to limited sample volume and availability for a number of the discordant samples, we were able to evaluate only 25 out of 57 discordant samples with neutralizing antibody titers ≥ 160, but no detectable anti-RBD or anti-N IgG. Among these 25 samples, 16 were anti-RBD IgA positive, 6 were anti-RBD IgM positive, and 1 was anti-S1 IgG positive. Only 2 samples had neutralizing activity (ID50 values of 205.4 and 197.5) that could not be accounted for by other anti-RBD isotypes or anti-S1 antibodies.

DISCUSSION

Since the first cases of SARS-CoV-2 were reported in 2019, the COVID-19 pandemic continues to rage worldwide, though with lessened acuity. Significant efforts have been invested in developing reliable and specific diagnostic tests detecting either the virus or antibodies to the virus. Meanwhile, the development of high-throughput assays to detect function rather than quantitation of antibodies has been much more limited, even as they are necessary to understand the correlation more precisely and directly between the level of humoral immune response conferred by previous infection or vaccination and the degree of protection from reinfection. Neutralization assays directly measure function in comparison to serological assays which at best, quantitate antibodies as a surrogate for function; therefore, neutralization assays play a valuable role in benchmarking serology assays as an assessment of immune function (15). The advantage of a serological assay is its ease in execution, which have enabled large-scale serological studies (16, 17). In contrast, given the technically challenging nature of the gold standard PRNT assay, the scale at which neutralization studies can be performed has been limited. Even pseudotyped virus studies have been limited to tens to hundreds of samples (18, 19). Recombinant viruses expressing a variety of reporter genes, such as nanoluciferase and green fluorescent protein (GFP), have also been generated and used for neutralizing antibody detection and antiviral discovery (20, 21). However, adapting such methods to new variants requires extensive and challenging efforts to engineer the reporter strains. In the present study, we developed and validated a high-throughput, automated fluorescence-based neutralization assay using authentic SARS-CoV-2 virus, which was used by the Office of the Assistant Secretary for Preparedness and Response (ASPR) as the reference standard for diagnostic accuracy to which other neutralization assays were compared in their FDA EUA request for CCP as a therapeutic (22); and subsequently used to test CCP samples from donors and then distributed by the FDA national EAP and transfused to patients between April 4 and August 31, 2020 (7, 23).

Using our assay, we assessed a large set of ~19,000 samples from patients previously infected with SARS-CoV-2 and found that detection of anti-RBD IgG antibodies was more sensitive for previous infection (91.2%) than anti-N IgG antibodies (80.3%), with the combination of both antigens providing the greatest clinical performance (96.5%) and only 3.5% of CCP samples lacking either one. Meanwhile, neutralization activity was detected in 92% of samples from individuals infected between 14 days to 6 months before blood donation. Neutralization activity was moderately correlated with levels of anti-RBD IgG, with this correlation unsurprisingly being better than with anti-N IgG levels (r_s = 0.61 versus 0.50), given the role of RBD in spike protein engagement of the virus with the ACE2 receptor that is required for infection. However, despite the presence of antibodies and neutralizing activity, ~1/3 of the CCP samples had no to low neutralizing activity against the original circulating WA1/2020 strain (12). It has been previously reported, albeit from smaller sample size studies, that CCP from individuals infected with parental strain (WA1/2020) showed lower cross-neutralizing activity against the later emerged variants (24–26). Here, we show that CCP samples from individuals (n = 394) who recovered from early circulating strains (WA1/2020 and B.1.1.7, alpha variant) showed 6.9-fold and 26.6-fold reduction in neutralization potency against B.1.351 (beta) and BA.1 (omicron), respectively. Even more striking is the high percentage of samples that had no activity against B.1.351 and BA.1 variants, with 36.5% and 87.5% of samples having no detectable neutralizing activity, respectively.

Prior to the availability of alternative therapeutic options against an infectious pathogen, convalescent plasma (CP) is often the first option available. It has a long history of use against many different pathogens, resulting in the Nobel prize for its use against diphtheria (27). More recently, its efficacy has been suggested in infections such as SARS, MERS, and H1N1 influenza with some clinical support (28–31), even while high-quality data such as randomized, controlled studies may be variable or lacking to support its use in every case. The variability of its efficacy can be due to a variety of factors, such as the particular pathogen identity and its mechanisms of pathogenesis, the role of humoral immunity in controlling infection, the physiology and immune status of individual patients, the timing of administration relative to the window in which pathogen neutralization might be beneficial, and importantly, the neutralizing activity of the administered donor CP and that of the recipient.

In particular, unique to CP as a therapeutic compared to other therapeutic modalities is its nonstandardized formulation, as demonstrated by the highly variable neutralizing activity reported here in the ~19,000 CCP units collected for administration to SARS-CoV-2 patients. About one-third of tested samples had no to low levels of neutralizing activity. Efforts to determine efficacy are additionally complicated by variable levels of neutralizing antibodies in recipients of CP, with conceivably less benefit in patients with already high levels. Thus, the ability to assess levels of neutralizing antibodies accurately and functionally can be critical to determining the efficacy of CP, in general. Indeed, a retrospective analysis performed by the US Food and Drug Administration (FDA) on the outcomes of recipients of CCP in the early access program early in the pandemic in 2020, in the context of these data on 19,000 of the administered CCP samples, suggested that a modest clinical benefit was derived from the administration of CCP with higher antibody titers in nonintubated patients, presumably early in their course, and became the basis for the FDA’s emergency use authorization of CCP to treat COVID-19 on August 23, 2020 (23, 32). Subsequent studies that have analyzed efficacy have suggested a potential benefit when high-titer CCP is administered in certain populations who are treated early (33–40), while other studies have differed in conclusions (41–43). All of these studies were conducted during a pandemic and challenged by confounding variables that could impact determination of treatment efficacy, such as using undetermined the titer of plasma, late administration, or heterogenous patient populations. Therefore, the ability to rapidly measure neutralizing activity of CCP at scale could play a critical role in pathogen outbreaks, such as COVID-19, to determine the therapeutic efficacy of passive transfer of antibodies. Specifically, by pairing infection outcomes with activity measurements in both donors and recipients, one can determine the levels of activity required in donor plasma to benefit infected recipients with neutralizing activity below a determined threshold. Further, the ability to measure CCP activity at scale, can help define levels of immune protection, or lack thereof, within populations afforded by natural infection or vaccination. Importantly, a facile, rapid assay could enable monitoring of immune protection in the population over time and to emerging viral variants and the ability to perform large scale studies that may shed light on rare events, such as immune protection provided by different immunoglobulin isotypes.

Study limitations include the lack of detailed information of donors, particularly the between infection and CCP collection, which could provide a better understanding of the kinetics of immune protection in individuals; and the use of a limited number of viral antigens and immunoglobulin isotypes in the serology assay, which would have provided a more complete picture of humoral immunity to SARS-CoV-2. It is also important to note, however, that the humoral response does not fully define the immune state, with memory B and T cells contributing to protective immunity against SARS-CoV-2, independent of the presence or absence of neutralizing antibodies (44).

In conclusion, we developed an automated, high-throughput neutralization assay for SARS-CoV-2 to determine neutralizing antibody titer consistently and efficiently in large study populations. The ability to develop such capabilities and infrastructure rapidly enables clinical trials for vaccine and antiviral therapies. Quickly and accurately understanding the relationship between neutralizing activity and serology—that is, function versus quantitation of antibodies—is critical in determining the utility of different diagnostic tests as an indicator of immune protection. Any ongoing debate on the efficacy of convalescent plasma against SARS-CoV-2 does not diminish, and perhaps serves to amplify, the need for such neutralization assays to help provide clarity in the efficacy of CP in future infectious disease outbreaks. Taken together, the development of such assays on scale are an important component of pandemic responsiveness and preparedness and should be prioritized as new pathogens emerge and spread through populations. They are critical to enabling systematic, well-controlled studies to determine efficacy of interventions and generate clear guidelines in patient management.

MATERIALS AND METHODS

Cell culture.

All cell lines were routinely tested and certified as mycoplasma-free using the Universal Mycoplasma Detection Kit (ATCC, 30-1012K). African green monkey kidney clone E6 (Vero E6) cells were acquired from the American Type Culture Collection (ATCC, CRL-1586). VeroE6/TMPRSS2 cells, express TMPRSS2 constitutively, were provided by Nir Hacohen’s laboratory (Harvard/MGH) and maintained at 250 μg/mL hygromycin. Both Vero E6 cells and VeroE6/TMPRSS2 cells were cultured in Dulbecco's modified Eagle's medium (DMEM)—high glucose supplemented with 1% penicillin-streptomycin mixture and 10% fetal bovine serum (FBS) in a humidified, 5% CO₂ incubator at 37°C.

Human samples.

Convalescent plasma samples from recovered COVID-19 donors were collected in licensed blood establishments following FDA Guidance for donor eligibility in accordance with the FDA emergency use authorization for convalescent plasma (11, 12, 45). At the time of plasma collection, donors consented to use of de-identified donor information and test results for research purposes in conformance with the January 19, 2017, Final Common Rule, Federal Policy for the Protection of Human Subjects. This activity was determined to be exempt from human subjects research requirements, and self-certified by the participating blood establishments. For prepandemic samples, approval was obtained from the Mass General Brigham IRB.

SARS-CoV-2 virus and titration.

The SARS-CoV-2 stock USA-WA1/2020 (NR-52281), B.1.351 (NR-54011), and B.1.1.7 (NR-54009) were obtained from BEI Resources. Virus stocks were expanded in Vero E6 cells following a low MOI (0.01) inoculation and harvested after 3 days. All the experiments in this study were performed using the passage 2 virus stock. Viral titers were determined in Vero E6 cells by a fluorescent-focus assay (46). All procedures performed with infectious SARS-CoV-2 were conducted in the Biosafety Level 3 facility of the Broad Institute with approval from the Broad Environmental Health and Safety Office.

Plaque reduction neutralization test (PRNT).

Serial diluted anti-SARS-CoV-2 spike RBD neutralizing antibody (AcroBiosyntems) was mixed with an equal amount of virus suspension containing 40 to 50 plaque-forming units in 1 mL. After incubating the mixtures at 37°C for 1 h, each virus-antibody mixture (1 mL) was added to the one well of a 6-well plate containing a confluent monolayer of VeroE6/TMPRSS2 and the plate was further incubated for 1 h at 37°C in 5% CO₂ incubator. The cell monolayer was then overlaid with 1% agarose in DMEM medium. After 2 days of incubation, the plates were fixed with 4% PFA and stained with 0.1% crystal violet. Plaques were counted for IC₅₀ calculation.

Live SARS-CoV-2 neutralization assay.

VeroE6/TMPRSS2 were seeded at 10,000 per well the day prior to infection in CellCarrier-384 ultra microplate (Perkin Elmer). The validation set (418 samples) was tested at a starting dilution of 1:40 and serially diluted 2-fold up to eight dilution spots. All the other patient specimens were tested at a starting dilution of 1:40 and were serially diluted 4-fold up to four dilution spots. Serially diluted patient sera were mixed separately with diluted SARS-CoV-2 virus and incubated at 37°C with 5% CO₂ for 1 h. Sera-virus complexes were added to the cells. Plates were incubated at 37°C with 5% CO₂ for 40 h. After that, plates were fixed and inactivated using 4% paraformaldehyde in PBS for 2 h at room temperature. The fixed cell plates can be safely moved out of BSL-3. The immunostaining (described in the Automated immunostaining system section) and the image acquisition (described in the Fluorescence image acquisition and quantification section) were performed in BSL-2 environment.

Automated immunostaining system.

The automated immnunostaining system was composed of robotic arm, plate stackers, BioTek plate washers, dedicated Thermo Scientific Combi Multidrop dispensers for each assay reagent, and PlateLoc plate sealer. The fully automated screening system was controlled and connected by Cellario software. The immunostaining was all performed in a custom-designed light-protected hood (HighRes Biosolutions).

The fixed cells were washed twice with PBS, then permeabilized with 0.5% Triton X-100 (SigmaAldrich) in PBS for 10 min. The plates were then washed and blocked with 1% BSA in PBS for 30 min at room temperature. Plates were then incubated with diluted anti-SARS-CoV/SARS-CoV-2 nucleoprotein mouse antibody (Sino) for 90 min at room temperature. After washing four times with PBS, cells were stained with the diluted secondary antibody Alexa Fluor 488-conjugated goat anti-mouse (JacksonImmuno) and Hoechst 33342 stain (ThermoFisher) for 45 min at room temperature. After washing four times with PBS, plates were then sealed with a PlateLoc plate sealer and stored at room temperature until imaging.

Fluorescence image acquisition and quantification.

All fluorescence imaging was performed on an Opera Phenix high-content screening system (PerkinElmer). For fluorescence imaging of the fixed cell, four tiles were acquired per well covering 100% of the well using 10×/0.3 NA dry objectives in a confocal mode. Image analysis for all the imaging was carried out with the Harmony software (PerkinElmer). Cell nuclei were first identified using Hoechst 33342 staining, and the cell number was calculated. Cytoplasmic regions were then detected around each nucleus based on the Alexa488 channel. The cells from the edge of the field were eliminated from the analysis. To quantify the intracellular virus abundance, the total Alexa488 signal intensity was calculated in the cell cytoplasm and the average Alexa488 signal per live cell was calculated for each well.

Data analysis.

The average Alexa488 signal per live cell from each well on a given plate was normalized to fit into the dynamic range defined by the positive control and the negative control for that plate. Normalized data points for each sample were then analyzed for outliers, defined as points deviating from the monotonical decrease of the neutralization activity with increasing dilution. Normalized data points, excluding outliers (only if there is a single outlier point), were then used to fit a 4-parameter sigmoidal curve with the four parameters representing: minimum and maximum fraction of the infected cells, slope, and ID₅₀ where the minimum and the maximum were fixed at 0 and 1, respectively. The slope and ID₅₀ were optimized using a nonlinear curve fitting algorithm to minimize mean squared error (MSE) between the fitted curve and the data. The algorithm required initial values for the slope and ID₅₀.

Enzyme-linked immunosorbent assay (ELISA).

MaxiSorp 384-well microplates (Sigma) were coated with 50 μL/well of 2,500 ng/mL of recombinant SARS-CoV-2 spike protein receptor binding domain (RBD) (Aaron Schmidt, Ragon Institute) or nucleoprotein (N) (Sino Biological) in coating buffer (1 packet BupH carbonate-bicarbonate [ThermoFisher] in 500 mL H₂O) overnight at 4°C. Plates were then washed 3 times with 100 μL/well of wash buffer (0.05% Tween 20, 400 mM NaCl, and 50 mM Tris-HCl pH 8.0) using a BioTek 406 plate washer. Plates were blocked by adding 100 μL/well of blocking buffer (1% BSA, 140 mM NaCl, and 50 mM Tris-HCl pH 8.0) for 30 min at room temperature. Plates were then washed as described above. 50 μL of 1:100 diluted serum samples in dilution buffer (1% BSA, 0.05% Tween 20, 140 mM NaCl, and 50 mM Tris-HCl pH 8.0) were added to the wells and incubated for 30 min at 37°C. Plates were then washed 7 times as described above. 50 μL/well of 1:25,000 diluted detection antibody solution (HRP-anti human IgG and IgM, Bethyl Laboratory number A80-104P, A80-100P) was added to the wells and incubated for 30 min at room temperature. Plates were then washed 7 times as described above. Forty μL/well of Pierce TMB peroxidase substrate (ThermoFisher) was then added to the wells and incubated at room temperature for 3 min (IgG) or 5 min (IgM). The reaction was then stopped by adding 40 μL/well of stop solution (0.5 M H₂SO₄) to each well. The OD was read after 15 min at 450 nm and 570 nm on a BioTek Synergy HT. Each plate included duplicate 12 2-fold dilutions of standards (1 μg/mL of control antibodies CR3022 IgG1 and IgM [Absolute Antibody number Ab01680-10.0, Ab01680-15.0] for RBD or serum samples with high anti-N antibodies titers for N). Denoised absorbances (450 nm minus 570 nm values) of test samples were scaled using the absorbance curves of standard dilutions on each plate to normalize absorbance values between plates and estimate the abundance of anti-RBD antibodies.

Data availability.

De-identified individual participant-level neutralization and serology assay data will be made available to share after publication.