Skip to main content
PMC home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Nov 4;145:105024. doi: 10.1016/j.jcv.2021.105024

Development of a rapid point-of-care test that measures neutralizing antibodies to SARS-CoV-2

Douglas F Lake a,1,⁎, Alexa J Roeder a,1, Erin Kaleta b, Paniz Jasbi c, Kirsten Pfeffer a, Calvin Koelbela a, Sivakumar Periasamy d,e, Natalia Kuzmina d,e, Alexander Bukreyev d,e,f, Thomas E Grys b, Liang Wu g, John R Mills g, Kathrine McAulay b, Maria Gonzalez-Moa h, Alim Seit-Nebi h, Sergei Svarovsky h
PMCID: PMC8567411  PMID: 34781240

Abstract

Background

After receiving a COVID-19 vaccine, most recipients want to know if they are protected from infection and for how long. Since neutralizing antibodies are a correlate of protection, we developed a lateral flow assay (LFA) that measures levels of neutralizing antibodies from a drop of blood. The LFA is based on the principle that neutralizing antibodies block binding of the receptor-binding domain (RBD) to angiotensin-converting enzyme 2 (ACE2).

Methods

The ability of the LFA was assessed to correctly measure neutralization of sera, plasma or whole blood from patients with COVID-19 using SARS-CoV-2 microneutralization assays. We also determined if the LFA distinguished patients with seasonal respiratory viruses from patients with COVID-19. To demonstrate the usefulness of the LFA, we tested previously infected and non-infected COVID-19 vaccine recipients at baseline and after first and second vaccine doses.

Results

The LFA compared favorably with SARS-CoV-2 microneutralization assays with an area under the ROC curve of 98%. Sera obtained from patients with seasonal coronaviruses did not show neutralizing activity in the LFA. After a single mRNA vaccine dose, 87% of previously infected individuals demonstrated high levels of neutralizing antibodies. However, if individuals were not previously infected, only 24% demonstrated high levels of neutralizing antibodies after one vaccine dose. A second dose boosted neutralizing antibody levels just 8% higher in previously infected individuals, but over 63% higher in non-infected individuals.

Conclusions

A rapid, semi-quantitative, highly portable and inexpensive neutralizing antibody test might be useful for monitoring rise and fall in vaccine-induced neutralizing antibodies to COVID-19.

KEYWORDS: Neutralizing antibodies, Covid-19, Sars-cov-2, Lateral flow assay, RBD, ACE2

1. Introduction

Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) causes COVID-19 and originated in Wuhan, China in December 2019 [1, 2, 3]. Vaccines continue to be tested [4,5] with the goal of preventing COVID-19 via induction of neutralizing antibodies (NAbs) and anti-viral T cells. Vaccine trials show that RNA vaccines elicit protective immunity, but durability of natural and vaccine-induced immunity is not fully known [5]. Several groups reported that up to one-third of serum samples from individuals who recovered from COVID-19 do not neutralize SARS-CoV-2 [6, 7, 8]. Whether previously infected or vaccinated, it is informative for individuals to learn if they generated high levels of NAbs so that they can resume normal activities without fear of re-infection and transmitting the virus [9, 10, 11].

Viral neutralization assays measure antibodies that block infection of host cells. The gold standard of neutralization for SARS-CoV-2 measures reduction of viral plaques or foci in microneutralization assays. These assays are slow, laborious, require highly trained personnel and a BSL3 facility. Another challenge is that neutralization assays require careful titration of virus and depend on host cells for infection, both of which add variability to the assay. These limitations prevent use of SARS-CoV-2 neutralization assays for clinical applications.

SARS-CoV-2 uses receptor binding domain (RBD) on spike protein to bind angiotensin converting enzyme 2 (ACE2) on host cells; RBD appears to be the principal neutralizing domain [12,13]. Using this knowledge, we developed a lateral flow assay (LFA) that measures levels of NAbs which block RBD from binding to ACE2. Other groups have developed RBD-ACE2-based competition ELISAs[18,19] but none have developed a rapid, highly portable, semi-quantitative test that can easily be incorporated into clinical settings or research studies where traditional laboratory or neutralization tests are not practical.

2. Materials and methods

2.1. Human subjects and samples

Serum and finger-stick blood samples were collected for this study under an Arizona State University institutional review board (IRB)-approved protocol #0,601,000,548 and Mayo Clinic IRB protocol #20–004,544. Serum samples obtained from excess clinical samples at Mayo Clinic were left over from normal workflow. COVID-19 samples ranged from 3 to 84 days post PCR positive result.

Twenty-seven control serum samples from patients with non-COVID-19 respiratory illnesses as determined by the FilmArray Respiratory Panel 2 (Biofire Diagnostics) were collected from patients from 2/14/17 – 4/6/20 as part of routine clinical workflow. All residual clinical samples were stored at 2–8 °C for up to 7 days, and frozen at −80 °C thereafter.

2.2. SARS-CoV-2 microneutralization assay

A microneutralization assay was performed using a recombinant SARS-CoV-2 expressing mNeonGreen (SARS-CoV-2 ng) as previously described [16]. Inhibitory concentrations for which 50% of virus is neutralized by serum antibodies (IC50 values) were obtained on a set of 38 COVID-19 sera. Sixty µl aliquots of SARS-CoV-2 ng were pre-incubated for 1 h in 5% CO2 at 37 °C with 60 µl 2-fold serum dilutions in cell culture media, and 100 µl were inoculated onto Vero-E6 monolayers in black polystyrene 96-well plates with clear bottoms (Corning) in duplicate. The final amount of the virus was 200 PFU/well, the starting serum dilution was 1:20 and the end dilution was 1:1280 unless an IC50 was not reached in which case serum was diluted to 1:10,240. Cells were maintained in Minimal Essential Medium (ThermoFisher Scientific) supplemented by 2% FBS (HyClone) and 0.1% gentamycin in 5% CO2 at 37 °C. After 2 days of incubation, fluorescence intensity of infected cells was measured at 488 nm using a Synergy 2 Cell Imaging Reader (Biotek). Signal was normalized to virus alone with no serum added and reported as percent neutralization. IC50 was calculated with GraphPad Prism 6.0 software. Work was performed in a BSL-3 biocontainment laboratory of the University of Texas Medical Branch, Galveston, TX.

2.3. Serologic antibody assay

The Ortho Vitros Anti-SARS-CoV-2 IgG test (Ortho Vitros test) was performed on an Ortho Clinical Diagnostics Vitros 3600 Immunodiagnostics System at the Mayo Clinic. This assay is approved for clinical testing under FDA Emergency Use Authorization to qualitatively detect antibody to the S1 subunit of SARS-CoV-2 spike protein. Results are reported as reactive (S/CO ≥ 1.0) or nonreactive (S/CO <1.0). Specimens were tested within 7 days of collection and stored at 2–8 °C. The same 38 serum samples were run in the Ortho Vitros test, microneutralization assay, and the LFA.

2.4. Lateral flow neutralizing antibody assay

The Lateral Flow NAb assay was developed to measure levels of antibodies that compete with ACE2 for binding to RBD. The LFA single port cassette (Empowered Diagnostics) contains a test strip composed of a sample pad, blood filter, conjugate pad, nitrocellulose membrane striped with test and control lines, and an absorbent pad (Axim Biotechnologies Inc). The LFA also contains a control mouse antibody conjugated to red gold nanospheres and corresponding anti-mouse IgG striped at the control line.

LFAs were run at room temperature on a flat surface for 10 min prior to reading results. To perform the test, 6.7 µl of serum or 10 µl whole blood were added to the sample port followed by 60 µl of chase buffer. After 10 min, densities of both test and control lines were recorded in an iDetekt RDS-2500 density reader.

The test leverages the interaction between RBD-conjugated green-gold nanoshells (Nanocomposix) that bind ACE2 at the test line when RBD-neutralizing antibodies (RBD-NAbs) are absent or low. Test line density is inversely proportional to RBD-NAbs present within the sample. As a semi-quantitative test, the results of the LFA can be interpreted using a scorecard or a densitometer. A red line across from the “C” indicates that the test ran properly. An absent or faint test line indicates high levels of RBD-NAbs, whereas a dark test line suggests low or lack of RBD-NAbs.

Precision testing was performed using sera from one highly, and one non-neutralizing donor in replicates of 10. Density values were recorded as above and%CVs calculated using the formula: (Standard Deviation/Mean) * 100%.

2.5. Data analysis

Pearson's correlation (r) was conducted to assess the strength and significance of associations between the LFA, the Ortho Vitros test and IC50 values. Regression analysis using IC50 values evaluated consistency [14] while Bland-Altman plots assessed agreement and bias [17,18]. Correlation analysis was conducted using IBM SPSS. For two-group analysis, IC50 values corresponding to >240 were categorized as titer of ≥1:320 (neutralizing), whereas IC50 values ≤240 were categorized as ≤1:160 (low/non-neutralizing). Receiver operating characteristic (ROC) analysis was performed to assess accuracy, sensitivity, and specificity of the LFA and Ortho Vitros tests in assessing neutralization; optimal cutoffs for each method were established to maximize area under curve (AUC) [19,20]. ROC analysis was conducted using R language in the RStudio environment (version 3.6.2; RStudio PBC). All analyses were conducted using raw values; data were not normalized, transformed, or scaled.

3. Results

As shown at the bottom in Fig. 1 A, serum containing high levels of NAbs results in a weak or ghost test line because NAbs bind RBD on green-gold beads, preventing RBD from binding to the ACE2 receptor at the test line. Serum with low levels of NAbs results in a strong test line because little to no antibodies prevent RBD on beads from binding to ACE2. Fig. 1 B demonstrates results of the test using COVID-19 sera with different levels of NAbs.

Fig. 1.

Fig. 1

Open in a new tab

(A) Schematic of Neutralization LFA. Below each graphic is a representative image of a lateral flow strip demonstrating actual line density. Addition of non-COVID19-immune serum or plasma (top) does not block binding of RBD-beads (green particles) to ACE2 resulting in the RBD-bead–ACE2 complex creating a visible line. Addition of patient serum with moderate titer NAbs to the sample pad creates a weak line (middle). Addition of patient serum with high titer NAbs (> 1:640) blocks binding of RBD-beads to ACE2 such that no line is observed at the test location on the strip (bottom). Red control line represents capture of a mouse monoclonal antibody coupled to red beads. (B) Scorecard for measuring levels of NAbs . Red control line across from the “C” on the cassette indicates that the test ran properly and the green test line across from the “T” can be used to measure the ability of plasma or serum to block RBD on gold nanoshells from binding to ACE2. (0) represents patient serum producing a visually non-existent line with density units of 10,095 and an IC50>500 (IC50=1151); (1) represents patient serum with a line density of 132,503 and an IC50 of 396; (2) represents patient serum with a line density of 317,156 and an IC50 of 243; (3) represents patient serum with a line density of 645,040 6 and an IC50 of 96.

To support the application of the LFA to measure NAb levels to SARS-CoV-2, we tested 38 serum samples that were assigned IC50 values in a SARS-CoV-2 microneutralization assay [16]. The experiment was performed in a blinded manner such that personnel running either the LFA or the microneutralization assay did not know the results of the comparator test. When line densities from the LFA were plotted against IC50 values determined in the microneutralization assay, serum samples with strong neutralization activity demonstrated low line densities; this indicates that NAbs inhibited RBD from binding to ACE2 (Fig. 2 ).

Fig. 2.

Fig. 2

Open in a new tab

Comparison of RBD-ACE2 competition LFA density values with IC50 values determined in a SARS-CoV-2 microneutralization assay on 38 samples (collected 3 to 90 days after PCR positive result). Ranges of IC50 values are shown on the X-axis plotted against LFA line density units on the Y-axis.

Next, we determined if the LFA detected neutralization activity in serum samples collected from patients with other PCR-confirmed respiratory viruses including seasonal coronaviruses (Fig. 3 A) and for serum samples collected prior to December 2019 (Fig. 3 B). Neither seasonal respiratory virus sera, nor pre-December 2019 samples showed neutralizing activity.

Fig. 3.

Fig. 3

Open in a new tab

A) Serum samples collected with PCR-confirmed diagnosis of seasonal respiratory viruses (Coronavirus OC43, blue; Coronavirus HKU-1, green; Coronavirus NL-63, pink; influenza A, orange, influenza B, red; parainfluenza, purple; rhinovirus, teal; respiratory syncycial virus, yellow; and adenovirus, black were run on the LFA as described in Methods. A positive control serum from a convalescent COVID-19 patient is shown on the far right of the bar graph in white. B) Serum samples collected pre-December 2019. Cutoff value of 263,000 density units was calculated based on receiver operating characteristic curves (see Fig. 6).

We then compared both the Ortho Vitros test and our LFA to sera with IC50 values determined in the SARS-CoV-2 microneutralization assay using 38 COVID-19 sera. To assess agreement between our LFA and the Ortho Vitros test, density units from the LFA and values from the Ortho test were regressed onto IC50 values (Supplemental Figure S1).

LFA values accounted for roughly 52% of observed variance in IC50 values, while the Ortho Vitros test accounted for approximately 27% of IC50 variance. LFA showed significant negative correlation with IC50 values (r = −0.720, p < 0.001), while the Ortho Vitros test values showed a significant positive correlation to IC50 values (r = 0.522, p = 0.001). Additionally, the LFA and Ortho Vitros test values correlated with each other (r = −0.572, p < 0.001).

To evaluate bias, mean differences and 95% confidence intervals (CIs) were calculated and plotted alongside limits of agreement (Fig. 4 ). Both LFA and Ortho Vitros test values showed strong agreement with titer, although the Ortho Vitros test showed a tendency to underestimate neutralizing capacity while the LFA method showed no bias.

Fig. 4.

Fig. 4

Open in a new tab

Bland-Altman plots showing bias (mean difference and 95% CI) and computed limits of agreement (mean difference ± 2SD) between (A) Ortho Vitros Anti-SARS-CoV-2 IgG test and IC50 values and (B) our LFA and IC50 values. Thirty-eight samples were tested.

ROC analysis was performed to assess the ability of the LFA and the Ortho Vitros test to classify low/non-neutralizing (Neg, <1:160), and highly neutralizing groups (≥1:320) (Fig. 5 ). As shown in Fig. 5 B and 5 D, the LFA misclassified one non-neutralizing sample (Neg, <1:160) as neutralizing (≥1:320) which the Ortho Vitros test also misclassified as neutralizing. The Ortho test also incorrectly classified five additional neutralizing samples as non-neutralizing.

Fig. 5.

Fig. 5

Open in a new tab

(A) Univariate ROC analysis of Ortho Vitros Anti-SARS-CoV-2 IgG test for discrimination of neutralizing samples (≥1:320) [AUC: 0.856, 95% CI: 0.697—0.953, sensitivity = 0.9, specificity = 0.7]. (B) Box plot of Ortho Vitros Anti-SARS-CoV-2 IgG test values between neutralizing (≥1:320) and non-neutralizing (Neg—1:160) groups. (C) Univariate ROC analysis of LFA for discrimination of neutralizing samples (≥1:320) [AUC: 0.978, 95% CI: 0.908—1.0, sensitivity = 0.9, specificity = 1.0]. (D) Box plot of LFA values between neutralizing (≥1:320) and non-neutralizing (Neg—1:160) groups.

Our LFA showed high accuracy for classification of neutralizing samples (AUC = 0.978), while the Ortho Vitros test showed modest accuracy (AUC = 0.856). Notably, while both methods showed roughly 90% sensitivity, the Ortho Vitros test showed only 70% specificity. In contrast, the LFA showed perfect specificity (100%) in this analysis of 38 samples.

Optimal cutoffs were computed to maximize AUC. For the LFA, density unit values below 263,000 classify samples as neutralizing and correspond to titers ≥1:320. Density values above this LFA cutoff classify samples in the non-neutralizing group. For the Ortho Vitros test, values between 0 and 23.3 were representative of non-neutralizing capacity, whereas values above 23.3 were reflective of the neutralizing group.

Precision studies were performed on replicate samples (n = 10) and showed a CV of ∼9% from a serum sample in the high neutralizing range and ∼6% CV in a serum sample from the low neutralizing range (Supplemental Table 1).

Precision studies were performed on replicate samples (n = 10) and showed a CV of ∼9% from a serum sample in the high neutralizing range and ∼6% CV in a serum sample from the low neutralizing range (Supplemental Table 1).

Since NAb levels may be considered correlates of protection, we tested sera from RNA vaccine recipients (mRNA-1273 and BNT162b2) in “previously infected” and “not previously infected” individuals using finger-stick blood in the rapid LFA (Fig. 6 ). In previously infected individuals at baseline (within 3 months of PCR-based diagnosis), 38% demonstrated high levels of NAbs. After the first vaccine dose, 87% of previously infected individuals demonstrated high NAb levels, while only 24% of not previously infected individuals developed high levels of NAbs. After the second vaccine dose, levels of NAbs increased to 95% in the previously infected cohort, while NAb levels increased to 87% in the not previously infected cohort. This data suggests that a second vaccine dose is important for highest levels of NAbs.

Fig. 6.

Fig. 6

Open in a new tab

NAb levels in prior infection and vaccine-induced individuals. (A) Baseline indicates within one week of first vaccine dose; Post-1st Dose indicates withing one week of 2nd vaccine dose; Post 2nd Dose indicates 10–20 days after 2nd vaccine dose. High and Low indicates density ranges of Test lines shown in (B). Densities were read in a reader as described in Methods. Serum titers that correspond to high range densities are >1:1280 to ≥1:160. Serum titers corresponding to low range densities are <1:160.

4. Discussion

We developed a rapid test that measures levels of NAbs in serum and whole blood. As shown in Fig. 2, the LFA correlates well with serologic titers determined using a SARS-CoV-2 microneutralization assay, especially when serum sample IC50 values are >250. Advantages of the LFA test are that it can be inexpensively and rapidly deployed to determine levels of NAbs in vaccine recipients. Moreover, the test can be used longitudinally to evaluate duration of protective immunity in naturally infected and vaccinated individuals–many more than could ever be evaluated using BSL2 or BSL3-based neutralization assays.

The LFA and Ortho Vitros test showed a significant correlation with each other (r = −0.572, p < 0.001), displaying good linear relation (r = −0.720, p < 0.001)[21]. The LFA accounts for 52% of observed IC50 variance (R2 = 0.5187), while the Ortho Vitros test accounts for 27% (R2 = 0.2725). Although absolute quantitation demands an excellent coefficient of determination (R2 ≥ 0.99)[22], variables with R2 ≥ 0.5 are highly predictive in univariate regression models while measures with R2 < 0.5 are recommended for use in multivariate models with complementary measures to increase predictive accuracy [23,24]. Bland-Altman analysis (Fig. 4) showed the Ortho Vitros test to be prone to underestimation of IC50 values, while the LFA method did not exhibit over- or underestimation bias. Furthermore, across mean values for both methods, the LFA showed discrete differential values while the Ortho Vitros test struggled to differentiate high neutralizing samples.

Using our rapid test to measure NAbs in previously infected vaccine recipients and those who were not infected agrees with other studies in BSL3 facilities using serum from venipuncture blood [5,25, 26, 27, 28, 29]. Natural infection may not elicit high levels of NAbs [6, 7, 8], but a first dose of vaccine induces high levels of NAbs in the majority of recipients similar to 2 doses of vaccine in non-previously infected individuals, suggesting natural infection primes the immune system[30]. In naïve individuals, a single dose of vaccine elicits high NAb levels (Titers >1:160) in only 24% of vaccine recipients, leaving 76% of vaccine recipients with titers lower than 1:160 which would not qualify for convalescent plasma donation according to FDA memo of March 9,2021. After a second vaccine dose, the LFA indicated high levels of NAbs in 87% of recipients, identical to levels observed in previously infected individuals after the first vaccine dose. These findings might suggest that a booster (3rd vaccine dose) in non-infected individuals could induce the highest levels of NAbs in the most people.

Limitations of the LFA are that it uses only the RBD portion of spike protein. Although the vast majority of reports indicate that the principle neutralizing domain is the RBD portion of spike protein, mAbs have been reported that neutralize SARS-CoV-2 by binding to the N-terminal domain of spike protein [31,32]. Also, since the spike protein assumes multiple conformations during viral binding and entry [33], neutralizing epitopes exist on the quaternary structure of spike [32]. Although RBDs on the nanoparticles may associate, it is not known if they assume a native conformation.

Other limitations are the binary nature of this data analysis (high and low neutralizing) of a continuous assay. NAb levels should be evaluated longitudinally to assess rise and fall in NAb levels; this rapid test is well-suited for that role. Another limitation is that the LFA does not differentiate high affinity anti-RBD NAbs from an abundance of lower affinity anti-RBD NAbs.

This test may prove useful in monitoring COVID-19 vaccine recipients as a correlate of protection. It would be logistically difficult to obtain a tube of blood from every vaccine recipient for BSL3 work. However, since this LFA requires only a drop of blood, individual use of this test might lead to more comprehensive longitudinal monitoring of protective humoral immunity and indicate when boosters might be required.

Funding

This study was funded in part by Axim Biotech, San Diego, CA

Author contributions

DL, SS and AS-N developed the LFA. AJ performed the experiments and interpreted the results. PJ performed statistical analysis. EK, TG and KM contributed samples. CK, AJ and KP tested vaccine recipients and interpreted results. SP, NK and AB performed SARS-CoV-2 microneutralization assays and interpreted results LW and JM contributed samples for preliminary experiments. MM-G designed the strip layout and produced the strips for the vaccine study.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: DL and SS are co-founders of Sapphire, the research division of Axim Biotech. SS, MM-G and AS-N are employees of Axim Biotech. Other authors have no conflicts regarding this work.

Footnotes

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jcv.2021.105024.

Appendix. Supplementary materials

mmc1.docx (80.2KB, docx)

References

  • 1.Ghinai I., McPherson T.D., Hunter J.C., Kirking H.L., Christiansen D., Joshi K., Rubin R., Morales-Estrada S., Black S.R., Pacilli M., Fricchione M.J., Chugh R.K., Walblay K.A., Ahmed N.S., Stoecker W.C., Hasan N.F., Burdsall D.P., Reese H.E., Wallace M., Wang C., Moeller D., Korpics J., Novosad S.A., Benowitz I., Jacobs M.W., Dasari V.S., Patel M.T., Kauerauf J., Charles E.M., Ezike N.O., Chu V., Midgley C.M., Rolfes M.A., Gerber S.I., Lu X., Lindstrom S., Verani J.R., Layden J.E., Brister S., Goldesberry K., Hoferka S., Jovanov D., Nims D., Saathoff-Huber L., Hoskin Snelling C., Adil H., Ali R., Andreychak E., Bemis K., Frias M., Quartey-Kumapley P., Baskerville K., Murphy E., Murskyj E., Noffsinger Z., Vercillo J., Elliott A., Onwuta U.S., Burck D., Abedi G., Burke R.M., Fagan R., Farrar J., Fry A.M., Hall A.J., Haynes A., Hoff C., Kamili S., Killerby M.E., Kim L., Kujawski S.A., Kuhar D.T., Lynch B., Malapati L., Marlow M., Murray J.R., Rha B., Sakthivel S.K.K., Smith-Jeffcoat S.E., Soda E., Wang L., Whitaker B.L., Uyeki T.M. First known person-to-person transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the USA. The Lancet. 2020;395:1137–1144. doi: 10.1016/S0140-6736(20)30607-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Huang C., Wang Y., Li X., Ren L., Zhao J., Hu Y., Zhang L., Fan G., Xu J., Gu X., Cheng Z., Yu T., Xia J., Wei Y., Wu W., Xie X., Yin W., Li H., Liu M., Xiao Y., Gao H., Guo L., Xie J., Wang G., Jiang R., Gao Z., Jin Q., Wang J., Cao B. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020;395:497–506. doi: 10.1016/S0140-6736(20)30183-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Li R., Pei S., Chen B., Song Y., Zhang T., Yang W., Shaman J. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2) Science. 2020;368:489–493. doi: 10.1126/science.abb3221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Polack F.P., Thomas S.J., Kitchin N., Absalon J., Gurtman A., Lockhart S., Perez J.L., Pérez Marc G., Moreira E.D., Zerbini C., Bailey R., Swanson K.A., Roychoudhury S., Koury K., Li P., Kalina W.V., Cooper D., Frenck R.W., Hammitt L.L., Türeci Ö., Nell H., Schaefer A., Ünal S., Tresnan D.B., Mather S., Dormitzer P.R., Şahin U., Jansen K.U., Gruber W.C. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N. Engl. J. Med. 2020;383:2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Widge A.T., Rouphael N.G., Jackson L.A., Anderson E.J., Roberts P.C., Makhene M., Chappell J.D., Denison M.R., Stevens L.J., Pruijssers A.J., McDermott A.B., Flach B., Lin B.C., Doria-Rose N.A., O'Dell S., Schmidt S.D., Neuzil K.M., Bennett H., Leav B., Makowski M., Albert J., Cross K., Edara V.-.V., Floyd K., Suthar M.S., Buchanan W., Luke C.J., Ledgerwood J.E., Mascola J.R., Graham B.S., Beigel J.H. Durability of Responses after SARS-CoV-2 mRNA-1273 Vaccination. N Engl J Med. 2020:4. doi: 10.1056/NEJMc2032195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.D.F. Robbiani, C. Gaebler, F. Muecksch, J.C.C. Lorenzi, Z. Wang, A. Cho, M. Agudelo, C.O. Barnes, A. Gazumyan, S. Finkin, T. Hagglof, T.Y. Oliveira, C. Viant, A. Hurley, H.-.H. Hoffmann, K.G. Millard, R.G. Kost, M. Cipolla, K. Gordon, F. Bianchini, S.T. Chen, V. Ramos, R. Patel, J. Dizon, I. Shimeliovich, P. Mendoza, H. Hartweger, L. Nogueira, M. Pack, J. Horowitz, F. Schmidt, Y. Weisblum, E. Michailidis, A.W. Ashbrook, E. Waltari, J.E. Pak, K.E. Huey-Tubman, N. Koranda, P.R. Hoffman, A.P. West, C.M. Rice, T. Hatziioannou, P.J. Bjorkman, P.D. Bieniasz, M. Caskey, M.C. Nussenzweig, Convergent Antibody Responses to SARS-CoV-2 Infection in Convalescent Individuals, Immunology, 2020. 10.1101/2020.05.13.092619.
  • 7.J.A. Juno, H.-.X. Tan, W.S. Lee, A. Reynaldi, H.G. Kelly, K. Wragg, R. Esterbauer, H.E. Kent, C.J. Batten, F.L. Mordant, N.A. Gherardin, P. Pymm, M.H. Dietrich, N.E. Scott, W.-.H. Tham, D.I. Godfrey, K. Subbarao, M.P. Davenport, S.J. Kent, A.K. Wheatley, Immunogenic profile of SARS-CoV-2 spike in individuals recovered from COVID-19, Infectious Diseases (except HIV/AIDS), 2020. 10.1101/2020.05.17.20104869.
  • 8.Wu F., Wang A., Liu M., Wang Q., Chen J., Xia S., Ling Y., Zhang Y., Xun J., Lu L., Jiang S., Lu H., Wen Y., Huang J. Neutralizing antibody responses to SARS-CoV-2 in a COVID-19 recovered patient cohort and their implications. MedRxiv Prepr. 2020:20. doi: 10.1101/2020.03.30.20047365. [DOI] [Google Scholar]
  • 9.Han Z., Battaglia F., Terlecky S.R. Discharged COVID-19 Patients Testing Positive Again for SARS-CoV-2 RNA: a Minireview of Published Studies from China. J. Med. Virol. 2020:jmv.26250. doi: 10.1002/jmv.26250. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Ye G., Pan Z., Pan Y., Deng Q., Chen L., Li J., Li Y., Wang X. Clinical characteristics of severe acute respiratory syndrome coronavirus 2 reactivation. J. Infect. 2020;80:e14–e17. doi: 10.1016/j.jinf.2020.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Hoang V.T., Dao T.L., Gautret P. Recurrence of positive SARS-CoV-2 in patients recovered from COVID-19. J. Med. Virol. 2020:jmv.26056. doi: 10.1002/jmv.26056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Wang Q., Zhang Y., Wu L., Niu S., Song C., Zhang Z., Lu G., Qiao C., Hu Y., Yuen K.-.Y., Wang Q., Zhou H., Yan J., Qi J. Structural and Functional Basis of SARS-CoV-2 Entry by Using Human ACE2. Cell. 2020 doi: 10.1016/j.cell.2020.03.045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Premkumar L., Segovia-Chumbez B., Jadi R., Martinez D.R., Raut R., Markmann A., Cornaby C., Bartelt L., Weiss S., Park Y., Edwards C.E., Weimer E., Scherer E.M., Rouphael N., Edupuganti S., Weiskopf D., Tse L.V., Hou Y.J., Margolis D., Sette A., Collins M.H., Schmitz J., Baric R.S., de Silva A.M. The receptor binding domain of the viral spike protein is an immunodominant and highly specific target of antibodies in SARS-CoV-2 patients. Sci. Immunol. 2020;5:eabc8413. doi: 10.1126/sciimmunol.abc8413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.C.W. Tan, W.N. Chia, X. Qin, P. Liu, M.I.-C. Chen, C. Tiu, Z. Hu, V.C.-W. Chen, B.E. Young, W.R. Sia, Y.-.J. Tan, R. Foo, Y. Yi, D.C. Lye, D.E. Anderson, L.-.F. Wang, A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2–spike protein–protein interaction, Nat. Biotechnol. (2020). 10.1038/s41587-020-0631-z. [DOI] [PubMed]
  • 16.Xie X., Muruato A., Lokugamage K.G., Narayanan K., Zhang X., Zou J., Liu J., Schindewolf C., Bopp N.E., Aguilar P.V., Plante K.S., Weaver S.C., Makino S., LeDuc J.W., Menachery V.D., Shi P.-.Y. An Infectious cDNA Clone of SARS-CoV-2. Cell Host Microbe. 2020;27:841–848. doi: 10.1016/j.chom.2020.04.004. e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Giavarina Davide. Understanding Bland Altman analysis. Biochem. Medica. 2015;25:141–151. doi: 10.11613/BM.2015.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Doğan N.Ö. Bland-Altman analysis: a paradigm to understand correlation and agreement. Turk. J. Emerg. Med. 2018;18:139–141. doi: 10.1016/j.tjem.2018.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.N.A. Obuchowski, J.A. Bullen, Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine, Phys Med Biol. (2018) 29. [DOI] [PubMed]
  • 20.Nakas C.T., Yannoutsos C.T. Ordered multiple-class ROC analysis with continuous measurements. Med. Stat. 2021;23(November 30):3437–3449. doi: 10.1002/sim.1917. [DOI] [PubMed] [Google Scholar]
  • 21.Schober P., Boer C., Schwarte L.A. Correlation Coefficients: appropriate Use and Interpretation. Anesth. Analg. 2018;126:1763–1768. doi: 10.1213/ANE.0000000000002864. [DOI] [PubMed] [Google Scholar]
  • 22.Rights J.D., Sterba S.K. Quantifying explained variance in multilevel models: an integrative framework for defining R-squared measures. Psychol. Methods. 2021;24:309–338. doi: 10.1037/met0000184. n.d. [DOI] [PubMed] [Google Scholar]
  • 23.Rights J.D., Sterba S.K. A framework of R-squared measures for single-level and multilevel regression mixture models. Psychol. Methods. 2021;23:434–457. doi: 10.1037/met0000139. n.d. [DOI] [PubMed] [Google Scholar]
  • 24.Rights J.D., Sterba S.K. New Recommendations on the Use of R-Squared Differences in Multilevel Model Comparisons. Multivar. Behav. Res. 2021;55:568–599. doi: 10.1080/00273171.2019.1660605. n.d. [DOI] [PubMed] [Google Scholar]
  • 25.Jackson L.A., Anderson E.J., Rouphael N.G., Roberts P.C., Makhene M., Coler R.N., McCullough M.P., Chappell J.D., Denison M.R., Stevens L.J., Pruijssers A.J., McDermott A., Flach B., Doria-Rose N.A., Corbett K.S., Morabito K.M., O'Dell S., Schmidt S.D., Swanson P.A., Padilla M., Mascola J.R., Neuzil K.M., Bennett H., Sun W., Peters E., Makowski M., Albert J., Cross K., Buchanan W., Pikaart-Tautges R., Ledgerwood J.E., Graham B.S., Beigel J.H. An mRNA Vaccine against SARS-CoV-2 — Preliminary Report. N. Engl. J. Med. 2020;383:1920–1931. doi: 10.1056/NEJMoa2022483. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Lombardi A., Bozzi G., Ungaro R., Villa S., Castelli V., Mangioni D., Muscatello A., Gori A., Bandera A. Mini Review Immunological Consequences of Immunization With COVID-19 mRNA Vaccines: preliminary Results. Front. Immunol. 2021;12 doi: 10.3389/fimmu.2021.657711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Sahin U., Muik A., Derhovanessian E., Vogler I., Kranz L.M., Vormehr M., Baum A., Pascal K., Quandt J., Maurus D., Brachtendorf S., Lörks V., Sikorski J., Hilker R., Becker D., Eller A.-.K., Grützner J., Boesler C., Rosenbaum C., Kühnle M.-.C., Luxemburger U., Kemmer-Brück A., Langer D., Bexon M., Bolte S., Karikó K., Palanche T., Fischer B., Schultz A., Shi P.-.Y., Fontes-Garfias C., Perez J.L., Swanson K.A., Loschko J., Scully I.L., Cutler M., Kalina W., Kyratsous C.A., Cooper D., Dormitzer P.R., Jansen K.U., Türeci Ö. COVID-19 vaccine BNT162b1 elicits human antibody and TH1 T cell responses. Nature. 2020;586:594–599. doi: 10.1038/s41586-020-2814-7. [DOI] [PubMed] [Google Scholar]
  • 28.Anderson E.J., Rouphael N.G., Widge A.T., Jackson L.A., Roberts P.C., Makhene M., Chappell J.D., Denison M.R., Stevens L.J., Pruijssers A.J., McDermott A.B., Flach B., Lin B.C., Doria-Rose N.A., O'Dell S., Schmidt S.D., Corbett K.S., Swanson P.A., Padilla M., Neuzil K.M., Bennett H., Leav B., Makowski M., Albert J., Cross K., Edara V.V., Floyd K., Suthar M.S., Martinez D.R., Baric R., Buchanan W., Luke C.J., Phadke V.K., Rostad C.A., Ledgerwood J.E., Graham B.S., Beigel J.H. Safety and Immunogenicity of SARS-CoV-2 mRNA-1273 Vaccine in Older Adults. N. Engl. J. Med. 2020;383:2427–2438. doi: 10.1056/NEJMoa2028436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Doria-Rose N., Suthar M.S., Makowski M., O'Connell S., McDermott A.B., Flach B., Ledgerwood J.E., Mascola J.R., Graham B.S., Lin B.C., O'Dell S., Schmidt S.D., Widge A.T., Edara V.-.V., Anderson E.J., Lai L., Floyd K., Rouphael N.G., Zarnitsyna V., Roberts P.C., Makhene M., Buchanan W., Luke C.J., Beigel J.H., Jackson L.A., Neuzil K.M., Bennett H., Leav B., Albert J., Kunwar P. Antibody Persistence through 6 Months after the Second Dose of mRNA-1273 Vaccine for Covid-19. N. Engl. J. Med. 2021 doi: 10.1056/NEJMc2103916. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.J.E. Ebinger, J. Fert-Bober, I. Printsev, M. Wu, N. Sun, J.C. Prostko, E.C. Frias, J.L. Stewart, J.E. Van Eyk, J.G. Braun, S. Cheng, K. Sobhani, Antibody responses to the BNT162b2 mRNA vaccine in individuals previously infected with SARS-CoV-2, Nat. Med. (2021). 10.1038/s41591-021-01325-6. [DOI] [PMC free article] [PubMed]
  • 31.Chi X., Yan R., Zhang J., Zhang G., Zhang Y., Hao M., Zhang Z., Fan P., Dong Y., Yang Y., Chen Z., Guo Y., Zhang J., Li Y., Song X., Chen Y., Xia L., Fu L., Hou L., Xu J., Yu C., Li J., Zhou Q., Chen W. A neutralizing human antibody binds to the N-terminal domain of the Spike protein of SARS-CoV-2. Science. 2020:eabc6952. doi: 10.1126/science.abc6952. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Liu L., Wang P., Nair M.S., Yu J., Rapp M., Wang Q., Luo Y., Chan J.F.-W., Sahi V., Figueroa A., Guo X.V., Cerutti G., Bimela J., Gorman J., Zhou T., Chen Z., Yuen K.-.Y., Kwong P.D., Sodroski J.G., Yin M.T., Sheng Z., Huang Y., Shapiro L., Ho D.D. Potent neutralizing antibodies against multiple epitopes on SARS-CoV-2 spike. Nature. 2020;584:450–456. doi: 10.1038/s41586-020-2571-7. [DOI] [PubMed] [Google Scholar]
  • 33.Cai Y., Zhang J., Xiao T., Peng H., Sterling S.M., Walsh R.M., Rawson S., Rits-Volloch S., Chen B. Distinct conformational states of SARS-CoV-2 spike protein. Science. 2020:eabd4251. doi: 10.1126/science.abd4251. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (80.2KB, docx)

Articles from Journal of Clinical Virology are provided here courtesy of Elsevier

RESOURCES