Skip to main content
PMC home page
Microbiology Spectrum logoLink to Microbiology Spectrum
. 2022 Jul 20;10(4):e01247-22. doi: 10.1128/spectrum.01247-22

SARS-CoV-2 Convalescent Sera Binding and Neutralizing Antibody Concentrations Compared with COVID-19 Vaccine Efficacy Estimates against Symptomatic Infection

Amy J Schuh a,✉, Panayampalli S Satheshkumar a, Stephanie Dietz a, Lara Bull-Otterson a, Myrna Charles a, Chris Edens a, Jefferson M Jones a, Kristina L Bajema a, Kristie E N Clarke a, L Clifford McDonald a, Sadhna Patel a, Kendra Cuffe a, Natalie J Thornburg a, Jarad Schiffer a, Kelly Chun b, Monique Bastidas b, Manory Fernando b, Christos J Petropoulos b, Terri Wrin b, Suqin Cai b, Dot Adcock b, Deborah Sesok-Pizzini b, Stanley Letovsky b, Alicia M Fry a, Aron J Hall a, Adi V Gundlapalli a
Editor: Rafael A Medinac
PMCID: PMC9430572  PMID: 35856710

ABSTRACT

Previous COVID-19 vaccine efficacy (VE) studies have estimated neutralizing and binding antibody concentrations that correlate with protection from symptomatic infection; how these estimates compare to those generated in response to SARS-CoV-2 infection is unclear. Here, we assessed quantitative neutralizing and binding antibody concentrations using standardized SARS-CoV-2 assays on 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 from COVID-19-unvaccinated persons with detectable anti-SARS-CoV-2 antibodies. Neutralizing and binding antibody concentrations were severalfold lower in the unvaccinated study population compared to published concentrations at 28 days postvaccination. In this convenience sample, ~88% of neutralizing and ~63 to 86% of binding antibody concentrations met or exceeded concentrations associated with 70% COVID-19 VE against symptomatic infection; ~30% of neutralizing and 1 to 14% of binding antibody concentrations met or exceeded concentrations associated with 90% COVID-19 VE. Our study not only supports observations of infection-induced immunity and current recommendations for vaccination postinfection to maximize protection against COVID-19, but also provides a large data set of pre-COVID-19 vaccination anti-SARS-CoV-2 antibody concentrations that will serve as an important comparator in the current setting of vaccine-induced and hybrid immunity. As new SARS-CoV-2 variants emerge and displace circulating virus strains, we recommend that standardized binding antibody assays that include spike protein-based antigens be utilized to estimate antibody concentrations correlated with protection from COVID-19. These estimates will be helpful in informing public health guidance, such as the need for additional COVID-19 vaccine booster doses to prevent symptomatic infection.

IMPORTANCE Although COVID-19 vaccine efficacy (VE) studies have estimated antibody concentrations that correlate with protection from COVID-19, how these estimates compare to those generated in response to SARS-CoV-2 infection is unclear. We assessed quantitative neutralizing and binding antibody concentrations using standardized assays on serum specimens collected from COVID-19-unvaccinated persons with detectable antibodies. We found that most unvaccinated persons with qualitative antibody evidence of prior infection had quantitative antibody concentrations that met or exceeded concentrations associated with 70% VE against COVID-19. However, only a small proportion had antibody concentrations that met or exceeded concentrations associated with 90% VE, suggesting that persons with prior COVID-19 would benefit from vaccination to maximize protective antibody concentrations against COVID-19.

KEYWORDS: SARS-CoV-2, COVID-19, standardized, quantitative, anti-SARS-CoV-2, IgG, neutralizing antibodies, correlation, antibody, immune, protection, correlate of protection, immunity

INTRODUCTION

As of 28 March 2022, 81.7% of the U.S. population 5 years of age and older has been fully vaccinated for COVID-19 (1). A Gallup survey conducted in July 2021 found that 18% of Americans would not agree to be vaccinated if a U.S. Food and Drug Administration (FDA)-approved COVID-19 vaccine were available to them immediately at no cost (2). One of the primary reasons cited for vaccine hesitancy was a history of SARS-CoV-2 infection and resultant antibodies.

Cumulative evidence indicates that SARS-CoV-2 antibodies are protective against SARS-CoV-2 reinfection (3). A series of nonhuman primate challenge studies demonstrated the central role of SARS-CoV-2-neutralizing antibodies in protection from reinfection (4–6). A randomized clinical trial involving the subcutaneous administration of REGEN-COV, a combination of two SARS-CoV-2-neutralizing monoclonal antibodies, or placebo within 96 h of SARS-CoV-2 exposure demonstrated that REGEN-COV prevented symptomatic COVID-19 and asymptomatic SARS-CoV-2 infection (7). A longitudinal study of >12,000 health care workers showed that SARS-CoV-2 infection-induced protective immunity lasts for at least 6 months (8).

The establishment of the first World Health Organization (WHO) international standard for anti-SARS-CoV-2 immunoglobulin (9) for quantitative assessment of neutralizing and binding antibody concentrations has made it possible for COVID-19 vaccine efficacy (VE) studies (10–13) to describe and propose standardized immune correlates of protection against symptomatic infection (or risk of symptomatic infection) in fully vaccinated persons (i.e., 2 weeks after their second dose in a 2-dose series, such as Pfizer-BioNTech [BNT162b2], Moderna [mRNA-1273], or AstraZeneca [ChAdOx1] vaccines, or 2 weeks after a single-dose vaccine, such as Johnson & Johnson’s Janssen [JNJ-78436735] vaccine) across vaccine trials that have used different antibody assays. Comparing antibody concentrations of unvaccinated persons with anti-SARS-CoV-2 antibodies to COVID-19-vaccinated cohorts and to estimated antibody concentrations associated with COVID-19 vaccine effectiveness would accomplish three goals: (i) improve understanding of the population distribution of antibody concentrations in response to infection and vaccination, (ii) help establish a relationship of quantitative antibody concentrations to those associated with protection against illness from published studies, and (iii) inform better-targeted messaging for universal COVID-19 vaccination.

Numerous SARS-CoV-2 serological assays have been developed throughout the COVID-19 pandemic to measure virus-specific antibody responses (14). Persons who recover from SARS-CoV-2 infection or receive a COVID-19 vaccine typically develop virus-specific neutralizing antibodies, with most of these antibodies directed against the immunodominant receptor binding domain (RBD) of the spike (S) protein (15, 16). Efficacy trials of the ChAdOx1 (11) and mRNA-1273 (12) vaccines showed that higher anti-SARS-CoV-2 S IgG, anti-SARS-CoV-2 RBD IgG, and SARS-CoV-2-neutralizing antibody concentrations were correlated with a reduced risk of symptomatic infection. Both trials determined antibody concentrations associated with varying levels of VE against symptomatic COVID-19 (11, 12). The ChAdOx1 vaccine trial estimated that 70% and 90% VE against symptomatic COVID-19 was associated with 50% neutralizing antibody titer (NT50) concentrations of 3.7 and 64.1 international units per milliliter (IU/mL), respectively, and 70% and 90% VE were associated with anti-RBD IgG antibody concentrations of 165.0 and 2,360.0 binding antibody units per milliliter (BAU/mL), respectively (11). The mRNA-1273 vaccine trial estimated that 70% and 90% VE against symptomatic COVID-19 were associated with 50% neutralizing antibody titer (NT50) concentrations of 4.0 and 83.0 IU/mL, respectively, and 70% and 90% VE were associated with anti-RBD IgG antibodies concentrations of 8.0 and 775.0 BAU/mL, respectively (12). However, it remains unclear how these antibody concentrations compare to those generated in response to SARS-CoV-2 infection in COVID-19-unvaccinated persons. To answer this question, we tested 3,067 prevaccination sera collected as part of a nationwide commercial laboratory seroprevalence study led by the U.S. Centers for Disease Control and Prevention (CDC) (17) from people who had previously tested positive for anti-SARS-CoV-2 antibodies using standardized anti-SARS-CoV-2 RBD IgG and SARS-CoV-2 pseudovirus neutralization antibody assays, and then we compared these antibody concentrations to concentrations measured in COVID-19-vaccinated cohorts (28 days postvaccination) and to those associated with 70% and 90% COVID-19 VE against symptomatic infection for two of the vaccines for which such data have been published to date.

RESULTS

Of the overall convenience sample of 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 with detectable anti-SARS-CoV-2 antibodies in a qualitative assay, 1,309 were from males (42.7%) and 1,758 (57.3%) were from females, 779 (25.4%) were from persons aged <18 years, 982 (32.0%) were from persons aged 18 to 49 years, 798 (26.0%) were from persons aged 50 to 64 years, and 508 (16.6%) were from persons aged ≥65 years.

Most of the serum specimens (n = 2,568, 83.7%) tested positive for both quantitative SARS-CoV-2-neutralizing antibodies and quantitative anti-SARS-CoV-2 RBD IgG; 271 sera (8.8%) tested negative for both neutralizing antibodies and anti-RBD IgG, 83 sera (2.7%) tested negative for neutralizing antibodies only, and 145 sera (4.7%) tested negative for anti-RBD IgG only. Geometric mean NT50 concentrations differed significantly according to the qualitative antibody test type that was used to screen sera for study inclusion (P < 0.0001) (see Fig. S1A in the supplemental material), while geometric mean anti-RBD IgG concentrations did not differ significantly according to the qualitative antibody test type that was used to screen sera for study inclusion (P = 0.02998) (see Fig. S1B).

The overall geometric mean NT50 concentration was 28.8 IU/mL (95% confidence interval [CI], 26.8, 31.0) and did not differ significantly according to sex (male, 30.9 IU/mL, 95% CI 27.4, 34.8; female, 27.3 IU/mL, 95% CI 25.0, 30.0; P = 0.1088) but differed significantly according to age category (<18 years, 31.3 IU/mL, 95% CI 28.4, 34.6; 18 to 49 years, 23.1 IU/mL, 95% CI 20.3, 26.2; 50 to 64 years, 38.0 IU/mL, 95% CI 32.5, 44.4; ≥65 years, 25.0 IU/mL, 95% CI 20, 31.3; P < 0.00010) (Fig. 1A). Persons aged <18 years had significantly higher NT50 concentrations than persons aged 18 to 49 years (P = 0.0108), and persons aged 50 to 64 years had significantly higher NT50 concentrations than persons aged 18 to 49 years (P < 0.0001) or persons aged ≥65 years (P = 0.0020) (Fig. 1A). The overall geometric mean anti-RBD IgG concentration was 162.5 BAU/mL (95% CI, 152.9, 172.7) and did not differ significantly according to sex (male, 171.3 BAU/mL, 95% CI 155.6, 188.5; female, 156.3 BAU/mL, 95% CI 144.4, 169.1; P = 0.1447) but did differ significantly according to age category (<18 years, 218.2 BAU/mL, 95% CI 199.2, 239.1; 18 to 49 years, 122.2 BAU/mL, 95% CI 109.8, 136.1; 50 to 64 years, 189.0 BAU/mL, 95% CI 167.2, 213.7; ≥65 years, 141.4 BAU/mL, 95% CI 117.8, 169.8; P < 0.0001) (Fig. 1B). Like the age-associated patterns of neutralizing antibody concentrations, persons aged <18 years had significantly higher anti-RBD IgG concentrations than persons aged 18 to 49 years (P < 0.0001) or persons aged ≥65 years (P < 0.0001), and persons aged 50 to 64 years had significantly higher anti-RBD IgG concentrations than persons aged 18 to 49 years (P < 0.0001) and persons aged ≥65 years (P = 0.0147) (Fig. 1B).

FIG 1.

FIG 1

Open in a new tab

SARS-CoV-2 50% neutralizing antibody titer (NT50) concentrations and anti-SARS-CoV-2 receptor binding domain (RBD) IgG concentrations for the overall convenience sample of 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 with detectable anti-SARS-CoV-2 antibodies on a qualitative assay and according to sex and age category. SARS-CoV-2 NT50 concentrations in international units per milliliter (IU/mL) (A) and anti-SARS-CoV-2 RBD IgG concentrations in binding antibody units per milliliter (BAU/mL) (B) are shown. Horizontal bars represent geometric means, vertical error bars represent 95% confidence intervals, and dashed horizontal lines represent assay cutoff values for seropositivity. P values from t tests (for sex) and post hoc Tukey tests (for age class) are shown for each sex and age class comparison. Bolded P values denote statistical significance (P < 0.05). Dotted horizontal lines represent antibody concentrations associated with 70% and 90% ChAdOx1 (11) and mRNA-1273 (12) vaccine efficacy (VE). The percentage of sera with antibody concentrations that met or exceeded the concentrations represented by each of the horizontal dotted lines are shown below the charts.

We found an overall significant, strong positive correlation between NT50 concentrations and anti-RBD IgG concentrations (r = 0.76, P < 0.0001) (Fig. 2A). The strength of the relationship remained consistent across sex (males, r = 0.76, P < 0.0001; females, r = 0.76, P < 0.0001) (Fig. 2B) and increased with increasing age category (<18 years, r = 0.67, P < 0.0001; 18 to 49 years, r = 0.69, P < 0.0001; 50 to 64 years, r = 0.79, P < 0.0001; ≥65 years, r = 0.88, P < 0.0001) (Fig. 2C).

FIG 2.

FIG 2

Open in a new tab

Correlations between SARS-CoV-2 50% neutralizing antibody titer (NT50) concentrations in international units per milliliter (IU/mL) and anti-SARS-CoV-2 receptor binding domain (RBD) IgG concentrations in binding antibody units per milliliter (BAU/mL) for the convenience sample of 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 with detectable anti-SARS-CoV-2 antibodies in a qualitative assay. (A) Overall comparison; (B) comparison by sex; (C) comparison by age class.

NT50 concentrations in the overall convenience sample of 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 with detectable anti-SARS-CoV-2 antibodies on a qualitative assay were 2.7-fold lower and 8.6-fold lower than concentrations reported from serum specimens collected 28 days post-COVID-19 vaccination with ChAdOx1 (11) or mRNA-1273 (12) VE study participants classified as not having COVID-19 during the follow-up period, respectively (Fig. 3A and B). Likewise, anti-RBD IgG concentrations in the overall convenience sample were 1.5-fold lower and 24.2-fold lower than concentrations reported from serum specimens collected 28 days post COVID-19 vaccination from ChAdOx1 and mRNA-1273 VE study participants classified as not having COVID-19 during the follow-up period, respectively (Fig. 3C and D). Overall, 88.3% of serum specimens in this study met or exceeded the concentration of SARS-CoV-2-neutralizing antibodies associated with 70% ChAdOx1 and mRNA-1273 VE, while 32.9% and 27.1% of sera met or exceeded the concentration of SARS-CoV-2-neutralizing antibodies associated with 90% ChAdOx1 and mRNA-1273 VE, respectively (Fig. 1A). For binding antibodies, we found that 63.1% and 86.4% of serum specimens from this study met or exceeded the concentration of anti-RBD IgG antibodies associated with 70% ChAdOx1 and mRNA-1273 VE, respectively, and 1.3% and 13.6% of sera met or exceeded the concentration of anti-RBD IgG antibodies associated with 90% ChAdOx1 and mRNA-1273 VE, respectively (Fig. 1B). A greater percentage of males and persons aged ≥50 years met or exceeded the concentrations of SARS-CoV-2-neutralizing antibodies and anti-RBD Ig associated with 70% ChAdOx1 and mRNA-1273 VE than females and persons aged <50 years, respectively.

FIG 3.

FIG 3

Open in a new tab

SARS-CoV-2 50% neutralizing antibody titer (NT50) concentrations in international units per milliliter (IU/mL) and anti-SARS-CoV-2 receptor binding domain (RBD) IgG concentrations in binding antibody units per milliliter (BAU/mL) for the overall convenience sample of 3,067 serum specimens collected during 27 July 2020 to 27 August 2020 with detectable anti-SARS-CoV-2 antibodies on a qualitative assay compared to 28 days post-COVID-19 vaccination concentrations in ChAdOx1 (11) and mRNA-1273 (12) VE study participants classified as not having COVID-19 during the follow-up period. (A) Median and interquartile range SARS-CoV-2 NT50 concentrations for the overall convenience sample compared to concentrations in VE study participants who received the ChAdOx1 vaccine and were classified as not having COVID-19 during the follow-up period. (B) Geometric mean and 95% confidence interval SARS-CoV-2 NT50 concentrations for the overall convenience sample compared to concentrations in VE study participants who received the mRNA-1273 vaccine and were classified as not having COVID-19 during the follow-up period. (C) Median and interquartile range anti-SARS-CoV-2 RBD IgG concentrations for the overall convenience sample compared to concentrations in VE study participants who received the ChAdOx1 vaccine and were classified as not having COVID-19 during the follow-up period. (D) Geometric mean and 95% CI anti-SARS-CoV-2 RBD IgG concentrations for the overall convenience sample compared to concentrations in VE study participants who received the mRNA-1273 vaccine and were classified as not having COVID-19 during the follow-up period. Horizontal bars represent medians or geometric means, and vertical error bars represent the interquartile range surrounding the median or 95% confidence intervals surrounding the geometric mean.

DISCUSSION

Similar to the ChAdOx1 (11) and mRNA-1273 (12) VE trial serologic testing results reported in the literature, we demonstrate a statistically significant, strong positive correlation between standardized anti-SARS-CoV-2 RBD IgG and SARS-CoV-2 NT50 concentrations, across all sex and age categories, using sera from unvaccinated persons with previous SARS-CoV-2 infection. Based on these broad correlations, our results suggest that high-throughput, quantitative anti-SARS-CoV-2 RBD IgG assays can be used as surrogates for SARS-CoV-2 pseudovirus neutralization assays that are considered mechanistic correlates of protection (12). Performing high-throughput, commercially available quantitative IgG assays is logistically feasible and practical on a large scale compared to performing time- and resource-intensive neutralization assays. Thus, there is a continued need for FDA-approved quantitative anti-SARS-CoV-2 S and RBD IgG assays in the management of the pandemic to facilitate the assessment of antibody concentrations associated with protection from infection and/or disease (14).

While our finding that most serum specimens in this study met or exceeded antibody concentrations associated with 70% COVID VE estimates supports the real-world observations of infection-induced immunity from reinfection, it is important to note that less than 33% of sera from unvaccinated persons with previous SARS-CoV-2 infection met or exceeded antibody concentrations associated with 90% ChAdOx1 and mRNA-1273 vaccine efficacy. This suggests that not all persons with a history of SARS-CoV-2 infection generate antibody responses of sufficient magnitude, as measured by a single, randomly timed blood test, to protect them against symptomatic reinfection. In addition to the circulation of the hypertransmissible Delta and Omicron variants of SARS-CoV-2, this finding may also partially explain the rise in COVID-19 cases that occurred in the United States in August 2021 despite a national serosurvey in May 2021 showing that 83.3% of the population had SARS-CoV-2 infection- and/or vaccine-induced antibodies (18). Meanwhile, at least in the short term (28 days), COVID-19 vaccination can provide 2.7- to 8.6-fold-higher neutralizing antibody concentrations and 1.5- to 24.2-fold-higher binding antibody concentrations. Thus, our results support current guidance that all eligible persons should consider COVID-19 vaccination to maximize their protection against symptomatic COVID-19 regardless of their SARS-CoV-2 infection history.

Our sampling approach of selecting specimens identified as positive using at least one qualitative anti-SARS-CoV-2 antibody assay would have selected for persons who had more severe disease and thus higher antibody concentrations and/or in whom antibodies had not waned. This selection bias may have led us to overestimate the percentage of persons with antibody concentrations meeting or exceeding those concentrations associated with 70% and 90% COVID-19 VE and would have missed persons infected with SARS-CoV-2 who did not mount a measurable postinfection immune response or had antibodies below detectable concentrations.

Our study has several limitations. First, due to the absence of information on whether persons had symptoms or if testing had been done or, if so, the date of COVID-19 symptom onset and/or a positive SARS-CoV-2 quantitative reverse transcription-PCR (qRT-PCR) or antigen test, we were unable to calculate the number of days that had elapsed between infection and collection of serum for antibody testing, and thus the measurements might not reflect peak antibody concentrations. The ChAdOx1 and mRNA-1273 VE trials measured serum antibody concentrations 28 days after the second vaccine dose, presumably at the peak of the measurable humoral immune response to the vaccine. Therefore, if the sera used in this study were collected >28 days after SARS-CoV-2 infection, it is possible that the percentage of persons with antibody concentrations meeting or exceeding those concentrations associated with 70% and 90% COVID-19 VE were underestimated. However, as a result of the sharp rise in COVID-19 cases in the United States in mid-June 2020 (see Fig. S2 in the supplemental material) (19), >50% of cases reported in the country prior to 27 July 2020 (the date first specimens used in this study were collected) occurred after 15 June 2020. This indicates that the majority of sera in this study were likely to have been collected within 73 days of SARS-CoV-2 infection, a time before IgG antibody concentrations would be expected to have significantly waned from peak postinfection concentrations (20).

Second, 13.6% of persons had anti-RBD IgG concentrations below the cutoff value of the assay (37.5 BAU/mL). However, the anti-RBD IgG concentration associated with 70% mRNA-1273 VE (8.0 BAU/mL) is below the cutoff value of the Cov2Quant IgG assay used in this study. Therefore, it is possible that some persons whose levels were below the assay cutoff value might have had anti-RBD IgG concentrations of ≥8.0 BAU/mL, leading us to underestimate the percentage of persons with antibody concentrations meeting or exceeding those concentrations associated with 70% mRNA-1274 COVID-19 VE.

Third, extrapolating these population-level results to individual-level clinical decision making with regard to immune protection based on a single antibody test poses a challenge. The sera tested here were untimed with regard to onset of infection, and we have no knowledge of the immune status of the individuals in terms of comorbidities and medications. These challenges are also applicable to postvaccination sera in terms of certain populations responding less well to vaccines in terms of neutralizing and binding antibodies compared to healthy volunteers in vaccine trials and limited availability of standardized quantitative assays (3).

Fourth, our findings, as well as the timing of the ChAdOx1 CoV-19 and mRNA-1273 VE trials, predate the circulation of the Delta and Omicron variants of SARS-CoV-2 that have exhibited 3- to 8-fold (21–23) and 43- to 122-fold decreases, respectively, in post-COVID-19 vaccine serum neutralizing antibody neutralization (i.e., sera collected 4 to 12 weeks after the second dose in a 2-dose vaccine series) compared to the original Wuhan-Hu-1 virus strain with and without the D614G mutation (23). Although no published COVID-19 VE studies have reported estimated neutralizing and/or binding antibody concentrations correlated with protection from symptomatic infection with SARS-CoV-2 Delta or Omicron variants, the dramatic decrease in post-COVID-19 vaccination serum neutralizing antibody concentrations against the Omicron variant suggests that the majority of unvaccinated persons with prior SARS-CoV-2 infection-induced antibodies only would be susceptible to symptomatic infection with the Omicron variant.

Finally, the efficiency and redundancy of the immune system, especially for the prevention of severe disease, in terms of humoral immunity (12), cellular immunity, and memory and anamnestic responses on second exposure to SARS-CoV-2 likely contribute to protection beyond an estimate indicated solely by single antibody concentrations (6, 24). Consistent with this philosophy, a recent study by Wei et al. (25) found that prior SARS-CoV-2 infection in COVID-19-unvaccinated study participants resulted in lower anti-S IgG concentrations but a greater level of estimated protection against Delta variant infection (33 BAU/mL associated with 67% protection) compared to study participants without prior SARS-CoV-2 infection that received two doses of ChAdOx1 (107 BAU/mL associated with 67% protection) or BNT62b2 (94 BAU/mL associated with 67% protection). These results indicate history of infection should be considered when evaluating quantitative SARS-CoV-2 antibody data.

In conclusion, we have demonstrated that in this sample, most unvaccinated persons with qualitative antibody evidence of prior infection had quantitative antibody concentrations against SARS-CoV-2 that met or exceeded antibody concentrations associated with 70% VE against symptomatic SARS-CoV-2 infection with previously circulating SARS-CoV-2 strains (e.g., Wuhan-Hu-1). However, only a small proportion had antibody concentrations that met or exceeded concentrations associated with 90% VE. Our findings suggest that persons with prior COVID-19 would benefit from vaccination to maximize protective antibody concentrations against symptomatic COVID-19. As the COVID-19 pandemic continues to evolve, standardized binding antibody assays that include spike protein-based antigens should be utilized to estimate antibody concentrations correlated with protection from COVID-19. Such data will be critical in informing public health guidance, such as the need for additional COVID-19 vaccine booster doses to prevent symptomatic infection.

MATERIALS AND METHODS

Specimen source and study design.

Sera were collected by two U.S.-based commercial laboratories for routine or acute clinical care (e.g., cholesterol screening or a sick visit) as part of a nationwide seroprevalence study; detailed methods have previously been described (17). Briefly, blood specimens collected for COVID-19-related reasons were excluded. Sera were tested (17) for anti-SARS-CoV-2 antibodies using one of the following three qualitative assays issued emergency use authorization by the FDA that were in use in the clinical laboratories: (i) Architect SARS-CoV-2 IgG assay (nucleocapsid [N] protein; Abbott, Chicago, IL); (ii) VITROS anti-SARS-CoV-2 IgG assay (S protein; Ortho-Clinical Diagnostics, Raritan, NJ); and (iii) Elecsys anti-SARS-CoV-2 assay (N protein; Roche, Indianapolis, IN) (see Fig. S3 in the supplemental material). Among 84,683 total serum samples collected during the period from 27 July 2020 to 27 August 2020 identified as anti-SARS-CoV-2 antibody positive and with linked age and sex information (as part of the larger serosurveillance study) (17), 3,067 serum specimens were selected by convenience sampling for reflex testing with anti-SARS-CoV-2 quantitative IgG and neutralizing antibody assays. Data on jurisdiction (see Fig. S4) and date (see Fig. S2) of blood collection were available. Data on race, ethnicity, and occurrence or date of SARS-CoV-2 infection or symptoms were not available; data on prior SARS-CoV-2 qRT-PCR test results were not available for most persons.

Ethics and consent to participate.

This activity was reviewed by the CDC and was conducted consistent with applicable federal law and CDC policy. Informed consent was waived, as all data were deidentified.

PhenoSense CoV neutralizing antibody assay.

Neutralizing antibodies against the SARS-CoV-2 S protein were measured using the PhenoSense CoV neutralizing antibody assay (26). To measure SARS-CoV-2 neutralizing antibodies, human immunodeficiency virus-1 pseudovirions expressing the SARS-CoV-2 S protein were prepared by cotransfecting HEK293 producer cells with an HIV-1 genomic vector and a SARS-CoV-2 envelope protein expression vector. Serial dilutions of sera were incubated with recombinant pseudovirions, and neutralizing activity was assessed by measuring the inhibition of luciferase activity in HEK293 target cells coexpressing the angiotensin-converting enzyme 2 and transmembrane serine protease 2 receptors. NT50 levels were expressed as the reciprocal of the serum dilution conferring 50% inhibition of pseudovirus infection. Calibration of the PhenoSense CoV neutralizing antibody assay with the first WHO international standard for anti-SARS-CoV-2 immunoglobulin (20/136; National Institute for Biological Standards and Controls, UK) (9) allowed us to generate a calibration factor of 0.0653 (S protein containing G614) and convert NT50 values from titers to IU/mL by multiplying by the calibration factor.

Cov2Quant IgG assay.

Anti-SARS-CoV-2 RBD IgG in sera were quantified using the electrochemiluminescent (ECL) Cov2Quant IgG assay (27). Briefly, sera were incubated with biotin-conjugated, recombinant SARS-CoV-2 RBD antigen bound to a streptavidin, carbon-coated microtiter well plate. After washing the plate, the wells were incubated with ruthenium-conjugated anti-IgG, and a final wash was performed to remove unbound material. Voltage was applied to the metallic electrode on the base of the plate, and a photomultiplier was used to measure the ECL signal generated from each well. The signal in relative light units (RLU) is directly proportional to the anti-RBD IgG concentration in sera. Each assay batch included a dilution series of affinity-purified human IgG standard that was used to calculate the anti-SARS-CoV-2 RBD IgG concentration of each serum. Calibration of the Cov2Quant IgG assay with the WHO international standard allowed us to generate a calibration factor of 25 and convert anti-SARS-CoV-2 RBD IgG concentrations from micrograms per milliliter to BAU/mL by multiplying by the calibration factor.

Comparison of study antibody data to antibody concentrations associated with COVID-19 VE.

Of the COVID-19 vaccines in use around the world currently, serological correlates of protection thresholds associated with VE have been estimated for ChAdOx1 (11) and mRNA-1273 (12) using the first WHO international standard for anti-SARS-CoV-2 immunoglobulin. As the quantitative antibody assays used in our study were calibrated to the same standard, we were able to compare antibody concentrations in our study to those estimated to be associated with 70% and 90% COVID-19 VE to ChAdOx1 and mRNA-1273.

Statistical analyses.

Data management tasks, and statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC) and GraphPad Prism 9.0.0 (GraphPad Software, San Diego, CA). A t test on log10-transformed data was used to determine if there were statistically significant differences between geometric mean anti-SARS-CoV-2 RBD IgG and NT50 concentrations by sex. Analyses of variance (ANOVAs) on log10-transformed data were used to determine if there were statistically significant differences between geometric mean anti-SARS-CoV-2 RBD IgG and NT50 concentrations according to previous qualitative antibody test and age category. Post hoc Tukey’s tests were performed to identify previous qualitative antibody tests and age categories with significantly different anti-SARS-CoV-2 RBD IgG and NT50 concentrations; P values were adjusted to account for multiple comparisons. Pearson’s correlation was used to assess the overall relationship between log10 anti-RBD IgG and log10NT50 concentrations, as well as the relationships between the two variables across sex and age categories. Two-sided P values of <0.05 were considered statistically significant.

ACKNOWLEDGMENTS

We thank Marla Williams, Morgan Herle, Wade Tanico, and Paul Theobald from Labcorp for help received in planning and execution of sample logistics and data acquisition. A special thanks goes to the reference laboratory operators that performed Cov2Quant IgG and PhenoSense CoV-neutralizing antibody assays. We also thank members of the Multistate Assessment of SARS-CoV-2 Seroprevalence (MASS) steering group, including Lyle Peterson, Margaret Honein, Adam MacNeil, Eduardo Azziz-Baumgartner, Francisco Averhoff, Sridhar Basavaraju, Tina Benoit, Carla Black, Kevin Berney, Ryan Weigand, Michele Owen, Ruchi Pancholy, and Lisa Grohskopf.

This work was supported by the U.S. Centers for Disease Control and Prevention. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Footnotes

[This article was published on 20 July 2022 with errors in the supplemental material. The supplemental material was updated in the current version, posted on 26 July 2022.]

Supplemental material is available online only.

Supplemental file 1
Supplemental material. Download spectrum.01247-22-s0001.pdf, PDF file, 1.4 MB (1.4MB, pdf)

Contributor Information

Amy J. Schuh, Email: aschuh@cdc.gov.

Rafael A. Medina, Pontificia Universidad Católica de Chile

REFERENCES

  • 1.US Centers for Disease Control and Prevention. 2022. COVID-19 vaccinations in the United States. https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-total-admin-rate-total. Accessed 29 March 2022.
  • 2.Jones JM. 6 August 2021. About One in Five Americans Remain Vaccine-Resistant. https://news.gallup.com/poll/353081/one-five-americans-remain-vaccine-resistant.aspx.
  • 3.US Centers for Disease Control and Prevention. 2021. Science brief: SARS-CoV-2 infection-induced and vaccine-induced immunity. https://www.cdc.gov/coronavirus/2019-ncov/science/science-briefs/vaccine-induced-immunity.html. Accessed 5 November 2021. [PubMed]
  • 4.Chandrashekar A, Liu J, Martinot AJ, McMahan K, Mercado NB, Peter L, Tostanoski LH, Yu J, Maliga Z, Nekorchuk M, Busman-Sahay K, Terry M, Wrijil LM, Ducat S, Martinez DR, Atyeo C, Fischinger S, Burke JS, Slein MD, Pessaint L, Van Ry A, Greenhouse J, Taylor T, Blade K, Cook A, Finneyfrock B, Brown R, Teow E, Velasco J, Zahn R, Wegmann F, Abbink P, Bondzie EA, Dagotto G, Gebre MS, He X, Jacob-Dolan C, Kordana N, Li Z, Lifton MA, Mahrokhian SH, Maxfield LF, Nityanandam R, Nkolola JP, Schmidt AG, Miller AD, Baric RS, Alter G, Sorger PK, Estes JD, Andersen H, Lewis MG, Barouch DH. 2020. SARS-CoV-2 infection protects against rechallenge in rhesus macaques. Science 369:812–817. doi: 10.1126/science.abc4776. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.McMahan K, Yu J, Mercado NB, Loos C, Tostanoski LH, Chandrashekar A, Liu J, Peter L, Atyeo C, Zhu A, Bondzie EA, Dagotto G, Gebre MS, Jacob-Dolan C, Li Z, Nampanya F, Patel S, Pessaint L, Van Ry A, Blade K, Yalley-Ogunro J, Cabus M, Brown R, Cook A, Teow E, Andersen H, Lewis MG, Lauffenburger DA, Alter G, Barouch DH. 2021. Correlates of protection against SARS-CoV-2 in rhesus macaques. Nature 590:630–634. doi: 10.1038/s41586-020-03041-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Corbett KS, Nason MC, Flach B, Gagne M, O'Connell S, Johnston TS, Shah SN, Edara VV, Floyd K, Lai L, McDanal C, Francica JR, Flynn B, Wu K, Choi A, Koch M, Abiona OM, Werner AP, Moliva JI, Andrew SF, Donaldson MM, Fintzi J, Flebbe DR, Lamb E, Noe AT, Nurmukhambetova ST, Provost SJ, Cook A, Dodson A, Faudree A, Greenhouse J, Kar S, Pessaint L, Porto M, Steingrebe K, Valentin D, Zouantcha S, Bock KW, Minai M, Nagata BM, van de Wetering R, Boyoglu-Barnum S, Leung K, Shi W, Yang ES, Zhang Y, Todd JM, Wang L, Alvarado GS, Andersen H, Foulds KE, Edwards DK, Mascola JR, Moore IN, Lewis MG, Carfi A, Montefiori D, Suthar MS, McDermott A, Roederer M, Sullivan NJ, Douek DC, Graham BS, Seder RA. 2021. Immune correlates of protection by mRNA-1273 vaccine against SARS-CoV-2 in nonhuman primates. Science 373:eabj0299. doi: 10.1126/science.abj0299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.O’Brien MP, Forleo-Neto E, Musser BJ, Isa F, Chan K-C, Sarkar N, Bar KJ, Barnabas RV, Barouch DH, Cohen MS, Hurt CB, Burwen DR, Marovich MA, Hou P, Heirman I, Davis JD, Turner KC, Ramesh D, Mahmood A, Hooper AT, Hamilton JD, Kim Y, Purcell LA, Baum A, Kyratsous CA, Krainson J, Perez-Perez R, Mohseni R, Kowal B, DiCioccio AT, Stahl N, Lipsich L, Braunstein N, Herman G, Yancopoulos GD, Weinreich DM, Covid-19 Phase 3 Prevention Trial Team . 2021. Subcutaneous REGEN-COV antibody combination to prevent Covid-19. N Engl J Med 385:1184–1195. doi: 10.1056/NEJMoa2109682. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Lumley SF, O'Donnell D, Stoesser NE, Matthews PC, Howarth A, Hatch SB, Marsden BD, Cox S, James T, Warren F, Peck LJ, Ritter TG, de Toledo Z, Warren L, Axten D, Cornall RJ, Jones EY, Stuart DI, Screaton G, Ebner D, Hoosdally S, Chand M, Crook DW, O'Donnell AM, Conlon CP, Pouwels KB, Walker AS, Peto TEA, Hopkins S, Walker TM, Jeffery K, Eyre DW, Oxford University Hospitals Staff Testing Group . 2021. Antibody status and incidence of SARS-CoV-2 infection in health care workers. N Engl J Med 384:533–540. doi: 10.1056/NEJMoa2034545. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Mattiuzzo G, Bentley EM, Hassall M, Routley S, Richardson S, Bernasconi V, Kristiansen P, Harvala H, Roberts D, Semple M, Turtle LC, Openshaw PJ, Baillie K, Nissen-Meyer LS, Brantsæter AB, Baxendale H, Atkinson E, Rigsby P, Padley D, Almond N, Rose NJ, Page M. 2020. Establishment of the WHO International Standard and Reference Panel for anti-SARS-CoV-2 antibody (WHO/BS/2020/2403). World Health Organization, Geneva, Switzerland. https://www.who.int/publications/m/item/WHO-BS-2020.2403. [Google Scholar]
  • 10.Earle KA, Ambrosino DM, Fiore-Gartland A, Goldblatt D, Gilbert PB, Siber GR, Dull P, Plotkin SA. 2021. Evidence for antibody as a protective correlate for COVID-19 vaccines. Vaccine 39:4423–4428. doi: 10.1016/j.vaccine.2021.05.063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Feng S, Phillips DJ, White T, Sayal H, Aley PK, Bibi S, Dold C, Fuskova M, Gilbert SC, Hirsch I, Humphries HE, Jepson B, Kelly EJ, Plested E, Shoemaker K, Thomas KM, Vekemans J, Villafana TL, Lambe T, Pollard AJ, Voysey M, Oxford CVTG . 2021. Correlates of protection against symptomatic and asymptomatic SARS-CoV-2 infection. Nat Med 27:2032–2040. doi: 10.1038/s41591-021-01540-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Gilbert PB, Montefiori DC, McDermott AB, Fong Y, Benkeser D, Deng W, Zhou H, Houchens CR, Martins K, Jayashankar L, Castellino F, Flach B, Lin BC, O’Connell S, McDanal C, Eaton A, Sarzotti-Kelsoe M, Lu Y, Yu C, Borate B, van der Laan LWP, Hejazi NS, Huynh C, Miller J, El Sahly HM, Baden LR, Baron M, De La Cruz L, Gay C, Kalams S, Kelley CF, Andrasik MP, Kublin JG, Corey L, Neuzil KM, Carpp LN, Pajon R, Follmann D, Donis RO, Koup RA, Immune Assays Teams, Moderna, Inc. Team, Coronavirus Vaccine Prevention Network/Coronavirus Efficacy Team, United States Government/COVPN Biostatistics Team . 2022. Immune correlates analysis of the mRNA-1273 COVID-19 vaccine efficacy clinical trial. Science 375:43–50. doi: 10.1126/science.abm3425. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Khoury DS, Cromer D, Reynaldi A, Schlub TE, Wheatley AK, Juno JA, Subbarao K, Kent SJ, Triccas JA, Davenport MP. 2021. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection. Nat Med 27:1205–1211. doi: 10.1038/s41591-021-01377-8. [DOI] [PubMed] [Google Scholar]
  • 14.Gundlapalli AV, Salerno RM, Brooks JT, Averhoff F, Petersen LR, McDonald LC, Iademarco MF, Response CC . 2021. SARS-CoV-2 serologic assay needs for the next phase of the US COVID-19 pandemic response. Open Forum Infect Dis 8:ofaa555. doi: 10.1093/ofid/ofaa555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Ju B, Zhang Q, Ge J, Wang R, Sun J, Ge X, Yu J, Shan S, Zhou B, Song S, Tang X, Yu J, Lan J, Yuan J, Wang H, Zhao J, Zhang S, Wang Y, Shi X, Liu L, Zhao J, Wang X, Zhang Z, Zhang L. 2020. Human neutralizing antibodies elicited by SARS-CoV-2 infection. Nature 584:115–119. doi: 10.1038/s41586-020-2380-z. [DOI] [PubMed] [Google Scholar]
  • 16.Robbiani DF, Gaebler C, Muecksch F, Lorenzi JCC, Wang Z, Cho A, Agudelo M, Barnes CO, Gazumyan A, Finkin S, Hagglof T, Oliveira TY, Viant C, Hurley A, Hoffmann HH, Millard KG, Kost RG, Cipolla M, Gordon K, Bianchini F, Chen ST, Ramos V, Patel R, Dizon J, Shimeliovich I, Mendoza P, Hartweger H, Nogueira L, Pack M, Horowitz J, Schmidt F, Weisblum Y, Michailidis E, Ashbrook AW, Waltari E, Pak JE, Huey-Tubman KE, Koranda N, Hoffman PR, West AP, Jr, Rice CM, Hatziioannou T, Bjorkman PJ, Bieniasz PD, Caskey M, Nussenzweig MC. 2020. Convergent antibody responses to SARS-CoV-2 in convalescent individuals. Nature 584:437–442. doi: 10.1038/s41586-020-2456-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Bajema KL, Wiegand RE, Cuffe K, Patel SV, Iachan R, Lim T, Lee A, Moyse D, Havers FP, Harding L, Fry AM, Hall AJ, Martin K, Biel M, Deng Y, Meyer WA, 3rd, Mathur M, Kyle T, Gundlapalli AV, Thornburg NJ, Petersen LR, Edens C. 2021. Estimated SARS-CoV-2 seroprevalence in the US as of September 2020. JAMA Intern Med 181:450–460. doi: 10.1001/jamainternmed.2020.7976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Jones JM, Stone M, Sulaeman H, Fink RV, Dave H, Levy ME, Di Germanio C, Green V, Notari E, Saa P, Biggerstaff BJ, Strauss D, Kessler D, Vassallo R, Reik R, Rossmann S, Destree M, Nguyen KA, Sayers M, Lough C, Bougie DW, Ritter M, Latoni G, Weales B, Sime S, Gorlin J, Brown NE, Gould CV, Berney K, Benoit TJ, Miller MJ, Freeman D, Kartik D, Fry AM, Azziz-Baumgartner E, Hall AJ, MacNeil A, Gundlapalli AV, Basavaraju SV, Gerber SI, Patton ME, Custer B, Williamson P, Simmons G, Thornburg NJ, Kleinman S, Stramer SL, Opsomer J, Busch MP. 2021. Estimated US infection- and vaccine-induced SARS-CoV-2 seroprevalence based on blood donations, July 2020–May 2021. JAMA 326:1400–1409. doi: 10.1001/jama.2021.15161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.US Centers for Disease Control and Prevention. YEAR. Trends in number of COVID-19 cases and deaths in the US reported to CDC, by state/territory. https://covid.cdc.gov/covid-data-tracker/#trends_dailycases. Accessed October 14, 2021.
  • 20.Egbert ER, Xiao S, Colantuoni E, Caturegli P, Gadala A, Milstone AM, Debes AK. 2021. Durability of spike immunoglobin G antibodies to SARS-CoV-2 among health care workers with prior infection. JAMA Netw Open 4:e2123256. doi: 10.1001/jamanetworkopen.2021.23256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Planas D, Veyer D, Baidaliuk A, Staropoli I, Guivel-Benhassine F, Rajah MM, Planchais C, Porrot F, Robillard N, Puech J, Prot M, Gallais F, Gantner P, Velay A, Le Guen J, Kassis-Chikhani N, Edriss D, Belec L, Seve A, Courtellemont L, Pere H, Hocqueloux L, Fafi-Kremer S, Prazuck T, Mouquet H, Bruel T, Simon-Loriere E, Rey FA, Schwartz O. 2021. Reduced sensitivity of SARS-CoV-2 variant Delta to antibody neutralization. Nature 596:276–280. doi: 10.1038/s41586-021-03777-9. [DOI] [PubMed] [Google Scholar]
  • 22.Mlcochova P, Kemp SA, Dhar MS, Papa G, Meng B, Ferreira IATM, Datir R, Collier DA, Albecka A, Singh S, Pandey R, Brown J, Zhou J, Goonawardane N, Mishra S, Whittaker C, Mellan T, Marwal R, Datta M, Sengupta S, Ponnusamy K, Radhakrishnan VS, Abdullahi A, Charles O, Chattopadhyay P, Devi P, Caputo D, Peacock T, Wattal C, Goel N, Satwik A, Vaishya R, Agarwal M, Mavousian A, Lee JH, Bassi J, Silacci-Fegni C, Saliba C, Pinto D, Irie T, Yoshida I, Hamilton WL, Sato K, Bhatt S, Flaxman S, James LC, Corti D, Piccoli L, Barclay WS, Rakshit P, CITIID-NIHR BioResource COVID-19 Collaboration , et al. 2021. SARS-CoV-2 B.1.617.2 Delta variant replication and immune evasion. Nature 599:114–119. doi: 10.1038/s41586-021-03944-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Garcia-Beltran WF, St Denis KJ, Hoelzemer A, Lam EC, Nitido AD, Sheehan ML, Berrios C, Ofoman O, Chang CC, Hauser BM, Feldman J, Roederer AL, Gregory DJ, Poznansky MC, Schmidt AG, Iafrate AJ, Naranbhai V, Balazs AB. 2022. mRNA-based COVID-19 vaccine boosters induce neutralizing immunity against SARS-CoV-2 Omicron variant. Cell 185:457–466.e4. doi: 10.1016/j.cell.2021.12.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Sette A, Crotty S. 2021. Adaptive immunity to SARS-CoV-2 and COVID-19. Cell 184:861–880. doi: 10.1016/j.cell.2021.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Wei J, Pouwels KB, Stoesser N, Matthews PC, Diamond I, Studley R, Rourke E, Cook D, Bell JI, Newton JN, Farrar J, Howarth A, Marsden BD, Hoosdally S, Jones EY, Stuart DI, Crook DW, Peto TEA, Walker AS, Eyre DW, Thomas T, Ayoubkhani D, Black R, Felton A, Crees M, Jones J, Lloyd L, Sutherland E, Pritchard E, Vihta K-D, Doherty G, Kavanagh J, Chau KK, Hatch SB, Ebner D, Ferreira LM, Christott T, Dejnirattisai W, Mongkolsapaya J, Cameron S, Tamblin-Hopper P, Wolna M, Brown R, Cornall R, Screaton G, Lythgoe K, Bonsall D, Golubchik T, Fryer H, Cox S, COVID-19 Infection Survey Team . 2022. Antibody responses and correlates of protection in the general population after two doses of the ChAdOx1 or BNT162b2 vaccines. Nat Med 28:1072–1082. doi: 10.1038/s41591-022-01721-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Markmann AJ, Giallourou N, Bhowmik DR, Hou YJ, Lerner A, Martinez DR, Premkumar L, Root H, van Duin D, Napravnik S, Graham SD, Guerra Q, Raut R, Petropoulos CJ, Wrin T, Cornaby C, Schmitz J, Kuruc J, Weiss S, Park Y, Baric R, de Silva AM, Margolis DM, Bartelt LA, Pasetti MF. 2021. Sex disparities and neutralizing-antibody durability to SARS-CoV-2 infection in convalescent individuals. mSphere 6:e0027521. doi: 10.1128/mSphere.00275-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Kappelman MD, Weaver KN, Boccieri M, Firestine A, Zhang X, Long MD, Group P-CS . 2021. Humoral immune response to messenger RNA COVID-19 vaccines among patients with inflammatory bowel disease. Gastroenterology 161:1340–1343.e2. doi: 10.1053/j.gastro.2021.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental file 1

Supplemental material. Download spectrum.01247-22-s0001.pdf, PDF file, 1.4 MB (1.4MB, pdf)

Articles from Microbiology Spectrum are provided here courtesy of American Society for Microbiology (ASM)

RESOURCES