A Test-Negative Design for Immune Correlates Approximates a Traditional Exposure-Proximal Design but Requires Far Fewer Blood Samples

Abstract

Traditional vaccine clinical trials sample blood from all participants. In contrast, the test-negative immune correlates (TNIC) design only samples blood from participants who develop symptoms. We compared traditional to test-negative immune correlates methods in the mRNA-1273 severe acute respiratory syndrome coronavirus 2 vaccine efficacy clinical trial. Using a neutralizing antibody assay, hazard ratios were 0.48 (95% confidence interval [CI], .29–.73) and 0.55 (95% CI, .28–1.06) for traditional and test-negative methods, respectively. Analogous ratios for binding antibody assay were 0.69 (95% CI, .52–.94) and 0.78 (95% CI, .50–1.20). The results support use of the logistically simpler TNIC design.

A test-negative immune correlates (TNIC) design that measures antibody gives similar results to the usual immune correlates design but requires far few blood samples. The TNIC design is attractive for outbreak setting and may be useful for T-cell correlates.

In vaccine development, there is substantial interest in understanding the relationship between antibody kinetics and risk of disease both to delineate vaccine mechanism and for licensure of new vaccines. Traditional exposure-proximal correlates analysis evaluates how current antibody levels vary with the instantaneous risk of disease and with instantaneous vaccine efficacy [1–3]. Traditional exposure-proximal designs involve collecting and storing serial blood specimens for the entire vaccinated group so that antibody levels can be measured in participants who subsequently become disease cases and a subset of participants who do not become cases.

A more efficient alternative is to sample the relatively few vaccinated participants who present with symptoms. Using a test-negative immune correlates (TNIC) design, antibody levels at symptom onset are compared between those who test negative for the virus and those who test positive [4]. If antibodies prevent disease, those who test negative will have higher antibody levels than those who test positive (Figure 1). TNIC studies can be embedded into conventional test-negative studies that assess vaccine effectiveness against symptomatic disease [5] and are attractive in outbreak settings where repeated blood sampling is difficult.

Figure 1.

Schematic of the test-negative immune correlates design. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Mathematically, traditional and TNIC approaches estimate the same correlate of risk curve, given certain assumptions. These include the standard assumptions applied to vaccinated participants in conventional test-negative studies [6–9], plus prompt presentation upon symptom onset and the lack of a rapid anamnestic (memory) response at the time of blood collection (Supplementary Figure 1). In this study, we reanalyze the mRNA-1273 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccine efficacy clinical trial (COVE Study) data to empirically compare the estimated correlate of risk curves based on traditional and TNIC approaches.

METHODS

Study Design and Population

The COVE Study (ClinicalTrials.gov identifier NCT04470427) randomized approximately 30 000 SARS-CoV-2–naive adults 1:1 to 2 doses of 100 µg of mRNA-1273 or placebo administered 4 weeks apart on days 1 and 29 [10]. Blood was drawn on days 1, 29, and 57 and at the onset of symptoms (disease day 1 [DD1]) for immunological assessment. The per-protocol analysis set was defined as participants who were baseline SARS-CoV-2 negative, received both doses, and had no major protocol violations. Coronavirus disease 2019 (COVID-19) diagnosis required participants to exhibit 2 systemic symptoms and 1 respiratory symptom, and to test positive for SARS-CoV-2 by reverse-transcription polymerase chain reaction [10] within 2 weeks of symptom onset.

During the blinded phase of the trial from July 2020 through March 2021, there were 38 per-protocol vaccinated participants who developed COVID-19 starting 7 days after day 57 (to match the exposure-proximal analysis set) who were also negative for anti-nucleocapsid (anti-N) immunoglobulin G (IgG) binding antibody (to remove cases where antibody level might be impacted by an anamnestic response). For each case, we identified 3 controls (except for 1 case that inadvertently had 5 controls) who had symptoms consistent with COVID-19 but who tested negative for SARS-CoV-2 and were anti-N seronegative. Controls were matched by region and calendar day of the case DD1 ± 7 days. Supplementary Figure 2 provides a schematic of the sampling design within the COVE Study. Supplementary Table 1 provides a demographic breakdown of the analysis groups.

Assays

Binding antibody (bAb) was measured with a Meso Scale Discovery 4-plex assay [11], which measures binding IgG to the ancestral SARS-CoV-2 spike (S-2P version), receptor-binding domain, and N antigens with a readout in World Health Organization (WHO) international units (binding antibody units [BAU]/mL).

Neutralizing antibody (nAb) was measured using the Monogram Biosciences PhenoSense pseudovirus D614G neutralization assay with readouts for number of virus particles at 50% inhibition (ID₅₀) titers in WHO international units (IU₅₀/mL) . The nAbs for the traditional design were measured using the Duke University neutralization assay expressed in IU₅₀/mL. Limits of detection for the assays are given in the Supplementary Materials.

Statistical Methods

We estimated a traditional exposure-proximal correlate of risk curve based on a Cox model with a time-varying covariate given by the predicted antibody level. a score for the risk of acquiring COVID-19, an indicator for high risk of severe COVID-19, and an indicator of minority status (Supplementary Materials). These variables were chosen to increase precision of the estimated curve and to control for variables (confounders) that could impact both the risk of acquiring COVID-19 and antibody level. A case-cohort design was used where all cases and a stratified random sample of noncases were used (immunogenicity cohort) [11]. The predicted antibody level was given by the day 57 measured antibody level plus B × d, where B was an estimated slope of decay, and d the number of days since day 57. Antibody decay was estimated using a linear mixed-effects model with a random intercept and terms for age and sex using data from an immunogenicity study of mRNA-1273 [2].

For the TNIC analysis, conditional logistic regression was used with the above covariates and predicted antibody replaced with measured antibody using samples drawn on symptom presentation. The TNIC model estimates the exposure-proximal hazard ratio (HR) provided the cases and associated controls are like random samples of cases and controls from the exposure-proximal analysis, and the measured antibody at symptom onset reflects the antibody level at exposure (see Supplementary Materials). Supplementary Table 2 compares the 2 designs.

RESULTS

The geometric mean titer of nAbs at DD1 was 80.7 IU₅₀/mL for the 38 TNIC cases and 132.3 IU₅₀/mL for the 116 TNIC controls. The case:control geometric mean ratio (GMR) was 0.61 (95% confidence interval [CI], .36–1.01). The geometric mean concentration of bAbs to spike at DD1 was 922.2 BAU/mL for cases and 1211.8 BAU/mL for controls. The GMR was 0.78 (95% CI, .50–1.20).

Using a traditional exposure-proximal correlate of risk modeling approach, the estimated HR for nAbs was 0.48 (95% CI, .29–.73) compared to an estimated HR of 0.55 (95% CI, .28–1.06) using the TNIC approach (Figure 2, left panel). Analogous estimates for bAbs were 0.69 (95% CI, .52–.94) using the traditional approach and 0.78 (95% CI, .50–1.20) using the TNIC approach (Figure 2, right panel). One control had a low log₁₀ binding spike value of −0.367 and was extremely influential in the parameter estimate. When this point was excluded from the model, the estimated TNIC HR for bAbs was 0.62 (95% CI, .27–1.46), more similar to the traditional approach (Supplementary Figure 3).

Figure 2.

Correlate of risk curves for neutralizing antibody titers (left) and binding antibody concentration (right). The test-negative curve is black. The exposure-proximal curve is blue. Dashed lines provide 95% confidence intervals (CIs) for the test-negative immune correlates of risk curve. Red dashes on the top denote log₁₀ titer or concentration values of the cases. Black dashes on the bottom are log₁₀ titer or concentration values of the controls. Dashed lines are 95% CIs. Abbreviations: BAU, binding antibody units; COVID-19, coronavirus disease 2019; ID₅₀, 50% inhibition; IgG, immunoglobulin G; nAb, neutralizing antibody.

A key assumption for the validity of the TNIC design is that antibody level measured on DD1 approximates the vaccine-induced antibody level at time of exposure. Supplementary Figure 4 provides the predicted and observed values for bAb and nAb levels for the 38 per-protocol cases, 8 anti-N–seropositive cases that were excluded from the primary analysis, and 8 controls that were part of the immunogenicity cohort. Tests of the paired difference between predicted and measured antibody on DD1 for the 38 cases and 8 controls were not significant for any comparison, though the variance of the TNIC assays was larger (Supplementary Table 3A and 3B). To examine the assumption that the TNIC controls are like a random sample of the exposure-proximal controls, we graphed the distribution of predicted antibody for the 1154 exposure-proximal controls with the measured antibody for the 3 TNIC controls. For each case, the distribution of antibody levels is similar for the 2 control groups, though with wider variation for the TNIC controls (Supplementary Figure 5, Supplementary Table 3C).We excluded the 8 anti-N positive cases, reasoning that their DD1 spike and nAb levels may be partly due to an anamnestic response. To investigate this hypothesis, we reran the model including these cases, which resulted in HR estimates of 0.61 for nAb and 0.91 for bAb, compared to 0.55 and 0.78 without the 8 cases. Thus, inclusion of the anti-N–positive cases weakens the correlation, supporting the hypothesis that the anti-N–positive cases represent an anamnestic response and justifying their removal.

DISCUSSION

To our knowledge, our analysis represents the first comparison of the traditional exposure-proximal immune correlates HR with those estimated from a TNIC design. Using data from a randomized, blinded controlled COVID-19 vaccine efficacy trial, we found that the TNIC estimates for both nAb and bAb were within 15% of the traditional exposure-proximal analysis. A data-driven exploratory analysis that removed an outlier had closer alignment. The traditional exposure-proximal approach had a large immunogenicity cohort and more cases, contributing to greater precision of the exposure-proximal immune correlates estimates.

The TNIC design has a substantial logistical advantage compared to traditional designs in terms of the number of blood samples required. Traditional immune correlates designs draw blood from all vaccine recipients at “peak” antibody response. Once cases are identified, those blood samples are assayed along with samples from a random subset of controls. Some traditional designs sample all vaccine recipients at multiple time points to better estimate antibody levels near exposure. For example, the COVE Study planned for 6 blood draws from each of approximately 30 000 participants, a total of 180 000 blood draws. In the traditional design, antibody levels were measured in 46 vaccinated cases and 1145 vaccinated controls. In contrast, the TNIC design required samples to be collected and assayed only at the onset of symptoms, resulting in blood being drawn from only 46 vaccinated cases and 118 vaccinated controls. Additional efficiencies can be achieved if the TNIC is embedded within a traditional test-negative design (TND) to assess vaccine effectiveness by leveraging the TND infrastructure.

While the estimated TNIC HRs were within 15% of the exposure-proximal estimates, the CIs are wider for the TNIC estimates, especially for the bAb. This is partly because of the reduced sample sizes for cases and controls: 38 and 46 and 116 and 1154 for the TNIC and traditional exposure-proximal design, respectively. For bAb, there was an outlier that also impacted CI width. Antibody measured at the time of COVID-19 symptom has been used before for immune correlates analyses. A TNIC proof-of-concept study was conducted in the Dominican Republic and estimated correlate of risk and protection curves for multiple variants [12]. A TNIC study was also used in the US Flu Vaccine Effectiveness Network. While these studies demonstrated the utility of breakthrough infection antibody [5], they could not compare the estimates with the predicted antibody from an exposure-proximal design. Our work shows that the TNIC design approximates traditional methods. Further, we demonstrated (1) that only a small and identifiable fraction of test-positive cases had an anamnestic response and (2) that the TNIC controls reflected a random sample of the population of controls.

Despite the importance of understanding the role of T cells and disease risk, they are rarely studied as correlates of risk due to the prohibitive cost and burden of whole blood sampling and storage for all participants as required for traditional methods. Sampling cell populations under a TNIC design is more feasible, given it requires a relatively small number of blood samples to be collected. The TNIC approach could be embedded in traditional vaccine trial designs to allow exploration of cell-mediated correlates of risk or used as a standalone clinical trial design when logistical simplicity is paramount.

The study has limitations. The study was run within a clinical trial where participants were encouraged to present promptly for testing following the onset of symptoms. If participants do not present promptly, the correlates of risk curve may be flatter, though exclusion of anti-N–positive cases should mitigate this bias. Further, our study was performed in SARS-CoV-2–naive individuals; validation of the TNIC design as a proxy for the traditional exposure-proximal design for nonnaive individuals and for other diseases remains to be done.

In summary, we provide proof of concept that TNIC provides estimates of correlates of COVID-19 risk curves similar to those obtained by the traditional exposure-proximal approach. This study provides a rationale to evaluate the COVID-19 TNIC approach in SARS-CoV-2–experienced populations, and in analyzing correlates of risk against other viruses of public health impact.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online (http://jid.oxfordjournals.org/). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.