Prediction of serum HIV-1 neutralization titers of VRC01 in HIV-uninfected Antibody Mediated Prevention (AMP) trial participants

ABSTRACT

VRC01 is being evaluated in the AMP efficacy trials, the first assessment of a passively administered broadly neutralizing monoclonal antibody (bnAb) for HIV-1 prevention. A key analysis will assess serum VRC01-mediated neutralization as a potential correlate of protection. To prepare for this analysis, we conducted a pilot study where we measured longitudinal VRC01 serum concentrations and serum VRC01-mediated neutralization in 47 and 31 HIV-1 uninfected AMP participants, respectively. We applied four different statistical approaches to predict serum VRC01-mediated neutralization titer against Env-pseudotyped viruses, including breakthrough viruses isolated from AMP placebo recipients who became HIV-1 infected during the trial, using VRC01 serum concentration and neutralization potency (IC50 or IC80) of the VRC01 clinical lot against the same virus. Approaches 3 and 4, which utilized pharmacokinetics/pharmacodynamics joint modeling of concentration and neutralization titer, generally performed the best or comparably to Approaches 1 and 2, which, respectively, utilized only measured and model-predicted concentration. For prediction of ID80 titers against breakthrough viruses, Approaches 1 and 2 rendered comparable performance to Approaches 3 and 4, and could be reasonable approaches to adopt in practice as they entail reduced assay cost and less complicated statistical analysis. Our results may be applied to future studies of other bnAbs and bnAb combinations to maximize resource efficiency in serum neutralization titer measurement.

The ongoing phase 2b Antibody Mediated Prevention (AMP) trials (HVTN 704/HPTN 085 and HVTN 703/HPTN 081; ClinicalTrials.gov NCT02716675 and NCT02568215) are the first proof-of-concept studies assessing whether a broadly neutralizing monoclonal antibody (bnAb) – VRC01, which targets the CD4 binding site of the HIV-1 envelope (Env) glycoprotein^1,2 – can prevent HIV-1 acquisition in humans. The trials are being conducted in two at-risk cohorts: 1924 women in sub-Saharan Africa (HVTN 703/HPTN 081) and 2699 men and transgender persons who have sex with men (MSM/TG) in the Americas and Switzerland (HVTN 704/HPTN 085). Participants were randomized to receive ten 8-weekly intravenous infusions of 10 mg/kg VRC01, 30 mg/kg VRC01, or placebo; further details of the study design and statistical considerations are given in Gilbert et al.³ The results of the primary efficacy analysis have recently been published.⁴

Attaining high serum neutralization titers against circulating HIV-1 strains is hypothesized to be important for achieving high prevention efficacy. In support of this hypothesis, day-of-challenge serum neutralization titer of passively immunized bnAbs has been shown to associate strongly with protection of nonhuman primates from simian-human immunodeficiency virus infection.⁵ A key secondary objective of the AMP trials is to conduct correlates analyses to assess VRC01 markers, including serum neutralization titers at the estimated time of infection against a representative panel of HIV-1 viruses, as predictors of protection against HIV-1 infection via a case–control study. If validated, such a marker could provide an important benchmark that bnAb-inducing vaccines would need to meet in phase 1/2 trials in order to be further considered for advancement into efficacy trials.

In preparation for the AMP correlates analysis, we conducted a pilot study in a subset of VRC01 recipients in the AMP trials who remained HIV-uninfected until the end of the study and were not taking pre-exposure prophylaxis (PrEP) during the study. We measured a total of 1003 VRC01 serum concentrations from n = 47 participants at up to 22 time-points, and a total of 246 50% or 80% inhibitory dose (ID50 or ID80) serum neutralization titers from n = 31 of the 47 participants at up to 8 of the 22 time-points. As it is resource-intensive to longitudinally measure serum neutralization titer, especially against multiple viruses of interest, an important question is whether serum neutralization titer can be predicted using less resource-intensive measurements, e.g. observed serum concentration, or pharmacokinetics model-estimated serum concentration. The latter would allow prediction of serum neutralization titer at the estimated time of infection, with important implications for the correlates analysis. We previously showed⁶ that, given longitudinally measured serum VRC01 concentrations and the 50% or 80% inhibitory concentration (IC50 or IC80) of the VRC01 clinical lot against a given Env-pseudotyped virus (hereafter, “virus”), we could apply statistical methods to accurately predict serum neutralization titers in HVTN 104,⁷ a phase 1 trial of VRC01 in healthy HIV-1 uninfected adults. Here we refine the three approaches used in Huang et al.⁶ and also propose a new approach incorporating instantaneous inhibitory potential (IIP), a measure of antiviral drug activity at a given concentration of the drug that takes into account the slope of the dose–response neutralization curve.⁸ In addition, we predict VRC01 serum neutralization titer in AMP participant serum samples against breakthrough viruses recovered from AMP participants. This characterization of neutralization titers against contemporaneously transmitted viruses provides valuable data for the interpretation of the primary trial efficacy results.

All AMP participants provided written informed consent. VRC01 recipients who remained HIV-1 negative through at least the week 88 study visit, did not permanently discontinue infusions during follow-up, and were inferred to not have used PrEP as described in Huang et al.⁹ were eligible for sampling into the pilot study, irrespective of the number of infusions received and the timing of infusions. For each sampled participant, serum concentrations measured at five days post-second infusion (Day 61), every four weeks from Week 4 until Week 80, and at Week 88, along with serum neutralization titer measured at Day 61, Weeks 24 (infusion #4 visit), 28, 48 (infusion #8 visit), 52, 72 (infusion #10 visit), 76, and 88 were included in the analyses. In HVTN 704/HPTN 085, n = 12 participants fitting the criteria for non-PrEP users from each VRC01 dose group were randomly sampled. Within each of those non-PrEP user groups of 12 participants, n = 8 were randomly sampled for serum VRC01 neutralization measurements. In HVTN 703/HPTN 081, n = 12 participants from each VRC01 dose group were sampled. One participant in the high-dose group did not fit the criteria for non-PrEP users; serum VRC01 neutralization data were collected from n = 8 and n = 7 participants randomly chosen from the 12 participants in each of the low and high-dose groups, respectively.

VRC01 concentrations in serum samples of study participants were quantified by an anti-idiotype enzyme-linked immunosorbent assay (ELISA).¹⁰ Neutralization activity against HIV-1 viruses by VRC01 (using both the clinical lot of VRC01 in vitro and post-administration VRC01 in serum samples) was measured by the TZM-bl target cell neutralization assay.^11,12 IC50 and IC80 concentrations of the clinical lot of VRC01, as well as ID50 and ID80 titers in VRC01 serum samples, were assessed against: one negative control virus, SVA-MLV (for QC purposes); one virus highly sensitive to VRC01-mediated neutralization, CH0505TF.gly4; one tier 2 reference virus, PVO.4;¹³ and four breakthrough viruses, two from each of the two AMP studies, derived from placebo recipients who became infected during the study. The breakthrough viruses (H703_1471_190s, H703_1750_140Es, H704_1535_030sN, and H704_2544_140eN01) were randomly sampled among all placebo breakthrough viruses to be at least moderately sensitive (IC50 < approximately 1 µg/ml) to VRC01-mediated neutralization. Further characterization of these viruses’ sensitivity to bnAb and non-bnAb neutralization is provided in Table S1.

Four approaches (summarized in Table 1) were investigated for predicting serum ID50 titers against each virus, as well as the geometric mean serum ID50 titer against the four breakthrough viruses. Approaches 1 and 2 do not require observed serum neutralization titers, whereas Approaches 3 and 4 do. Specifically, Approach 1 predicted serum ID50 titers as the measured VRC01 serum concentration divided by the IC50 of the VRC01 clinical lot against each virus, where concentrations below the lower limit of quantification (LLoQ) were replaced by half of the LLoQ. Approaches 2 and 3 are similar to what was described before,⁶ except that serum concentration (for both approaches) and neutralization titers (for Approach 3 only) below the respective assay LLoQs were not excluded, but contributed to the estimation of parameters in the population pharmacokinetics (popPK) model for Approach 2 and in the joint pharmacokinetics/pharmacodynamics (PK/PD) model for Approach 3 via computing the maximum likelihood estimate of the population parameters based on all the information provided by data both above and below the LLoQ. For the calculation of the likelihood, this method for imputation of data below the LLoQ is equivalent to the M3 method in NONMEM.¹⁴ For Approach 2, the popPK model-predicted^9,15 VRC01 serum concentration was divided by the IC50 of the VRC01 clinical lot against each virus as the predicted neutralization titer, where unlike Approach 1 predicted concentrations below the LLoQ were taken as is without any truncation. For Approach 3, the relationship between the popPK model-predicted concentration and the observed neutralization titers (i.e. PD outcome) over multiple time-points was modeled using a linear mixed effects PD model with a random intercept and a random slope to account for variabilities across individuals. Approach 4 expanded upon Approach 3 by using longitudinal IIP values instead of serum concentration as the predictor in the PD modeling; IIP was calculated as IIP = -log₁₀(1- c^m/(k^m+ c^m), where c = popPK model-predicted VRC01 serum concentration, k = IC50 and m = log₁₀(4)/[log₁₀(IC80) – log₁₀(IC50)]. For both Approach 3 and Approach 4, serum concentration data from all 47 participants were used in the construction of the popPK model, whereas a leave-one-out cross-validation procedure on the PD portion of the joint modeling was employed for an unbiased assessment of the prediction performance based on “left-out” test data not used to train and optimize the PD model. Specifically, within each iteration data from one participant were “left out” as test data and data from the remaining 30 participants were used as training data to construct the PD model; prediction of neutralization titers were obtained from the model for the “'left-out” participant. This procedure was repeated 31 times until every participant was “left out” once and the performance of approaches was evaluated based on the prediction from all 31 iterations. The same four approaches were used for predicting serum ID80 titers, except that IC80 and ID80 values were used in the analysis. All PK/PD modeling was performed in Monolix¹⁶ and other analyses in R version 3.5.1.¹⁷

Table 1.

Summary table of the four approaches for predicting serum neutralization titer of VRC01. For concreteness, serum ID50 titers are considered but all descriptions apply to the prediction of either ID50 or ID80 titers

Approach for Predicting SerumID50 Titers	Summary*	Compared to Approaches 1–3 in ref #5	Pros	Cons
Approach 1	$\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{I}\mathrm{D}50_{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{d}}=\frac{\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{c}}_{\mathrm{M}\mathrm{e}\mathrm{a}\mathrm{s}}}{\mathrm{C}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{t}\mathrm{I}\mathrm{C}50}$	Same as Approach 1 in;5 Serum ConcMeas < LLoQ replaced by LLoQ/2 in both articles.	Less resource-intensive: does not require ID50 measured on any serum sample; requires serum concentration measured in the given sample only Simplest among all four approaches	Requires the availability of clinical lot IC50 In general, prediction performance not as high as Approaches 3 and 4
Approach 2	$\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{I}\mathrm{D}50_{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{d}}=\frac{\mathrm{S}\mathrm{e}\mathrm{r}\mathrm{u}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{c}}_{\mathrm{p}\mathrm{o}\mathrm{p}\mathrm{P}\mathrm{K}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}}{\mathrm{C}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{t}\mathrm{I}\mathrm{C}50}$	Previous work in Ref5: Serum ConcMeas < LLoQ replaced by LLoQ/2 in the popPK model. This current work: Likelihood method used to account for serum ConcMeas < LLoQ in the popPK model.	Less resource-intensive: does not require ID50 measured on any serum sample Generally, as good as or better performance than Approach 1	Requires the availability of clinical lot IC50 Compared to Approach 1, Approaches 2–4 require serum concentrations measured at multiple time-points to permit popPK modeling In general, prediction performance not as high as Approaches 3 and 4
Approach 3	popPK modeling of measured longitudinal serum concentrations from all individuals; PD modeling of measured longitudinal serum concentrations and ID50 titers from individuals who have both types of data available; Serum ID50Pred obtained by plugging in popPK model-predicted serum concentration from step 1 into the PD model of step 2.	Previous work in Ref5: Serum ConcMeas < LLoQ replaced by LLoQ/2 in the popPK model; ID50Meas < LLoQ replaced by LLoQ/2 in the PD model. This current work: Likelihood method used to account for serum ConcMeas <LLoQ, and ID50Meas <LLoQ in the popPK and PD models.	Compared to Approaches 1 & 2, does not require availability of the clinical lot IC50 Generally, better prediction performance than Approaches 1 and 2	More resource-intensive: in addition to serum concentrations measured at multiple time-points to permit popPK modeling, requires ID50 measured on a subset of those PK serum samples to permit PD modeling
Approach 4	popPK modeling of measured longitudinal serum concentrations from all individuals; calculate serum IIP for all individuals; PD modeling of calculated longitudinal serum IIP and ID50 titers from individuals who have both types of data available; Serum ID50Pred obtained by plugging in calculated IIP from step 1 into the PD model of step 2.	NA (newly described here)	Accounts for not only serum concentrations but also the slope of the dose–response neutralization curve of the clinical lot product Generally, better prediction performance than Approaches 1 and 2	More resource-intensive: in addition to serum concentrations measured at multiple time-points to permit popPK modeling, requires ID50 measured on a subset of those PK serum samples to permit PD modeling Compared to Approach 3, additionally requires the availability of both IC50 and IC80 to permit the calculation of IIP as the predictor of ID50 titers

*Intended as an overview summary of each Approach; please see Methods for the full statistical description.

Conc = concentration; IIP = instantaneous inhibitory potential; Meas = measured; Pred = predicted; PK/PD = pharmacokinetics/pharmacodynamics.

These four approaches were applied to the analyses of all available serum concentration (Figure S1) and serum neutralization titers (Figures S2-S5) in the pilot study. Of note, not all samples had detectable serum VRC01 concentration (defined as serum concentration > 1.0 µg/mL, the LLoQ of the ELISA assay) and not all samples had detectable VRC01 serum neutralization titers (defined as serum neutralization titer >10, the LLoQ of the TZM-bl target cell neutralization assay). Because the precision of the measurements is expected to impact the performance of the prediction approaches, we evaluated the four approaches separately in the following four sample categories:

1. No detectable concentration AND no detectable neutralization titer (-, -; “double-negative”);

2. Detectable concentration BUT no detectable neutralization titer (+, -);

3. No detectable concentration BUT detectable neutralization titer (-, +); and

4. Detectable concentration AND detectable neutralization titer (+, +; “double-positive”).

The prediction performance of each of the four approaches was evaluated as follows. As none of the samples in category (1) or category (2) had their observed neutralization titer above the LLoQ of the assay, we reported the classification accuracy in terms of the percent of samples out of total in each category with titers that were correctly predicted to be ≤LLoQ. As all of the samples in category (3) had their observed neutralization titer >LLoQ, we report the classification accuracy in terms of the percent of samples out of the total in this category with titers that were correctly predicted to be >LLoQ. For samples in category (4) because both the concentration and titers were above the LLoQs of the respective assays, we report how well the predicted neutralization titer agrees with the observed neutralization titer, where the extent of agreement is quantified according to the same metrics as in Huang et al.⁶ [fold difference (FD): the ratio of the observed titer over the predicted titer; relative fold difference (RFD): the difference of the log_e-transformed observed and predicted titers divided by the log_e-transformed observed titer; relative mean squared error (RMSE): the average of the squared difference between the observed and predicted log_e-transformed titers of each approach relative to Approach 1; and concordance correlation coefficients for assessing agreement of repeated measurements (CCC_rm)^18,19 of the longitudinal observed vs. predicted log_e-transformed titers]. Using these metrics, a given prediction is more accurate the closer its FD is to 1, the closer its RFD is to 0, the smaller its RMSE value is, and the closer its CCC_rm is to 1.

We first discuss the results for predicting ID50 titer. Approach 3 generally showed the best accuracy in predicting the detectability of neutralization titer against all viruses for samples in categories (1)–(3), and the best performance in predicting the magnitude of neutralization titer for samples in category (4), though with overlapping 95% confidence intervals (CIs) across all four approaches in most cases (Table 2a); Approaches 1–4 had a similar performance in terms of the binary classification accuracy when all four sample categories were combined (Table S2A). Specifically, for samples in category (1), perfect (100%) classification accuracy was achieved in nearly all of the predictions, with the lowest accuracy still excellent at 86% (Approach 2, H704_1535_030sN) (Table 2a). For samples that were deemed negative or positive by only one of the two assays [i.e., categories (2) and (3)], the prediction performance of all approaches was generally dampened. Approach 3 was nearly always the best-performing method, with classification accuracies ranging from 69% to 74% across viruses, with the exception of CH0505TF.gly4 where Approach 3 showed lower, albeit still highly accurate, prediction (96% vs 100% by the other three approaches) (Table 2a). For samples in category (4), the agreement between predicted vs. observed ID50 titers was high (CCC_rm ranging from 0.74 to 0.89 across viruses) by Approaches 3 and 4, and intermediate (CCC_rm ranging from 0.57 to 0.80 across viruses) by Approaches 1 and 2, with generally slightly improved results for Approach 3 compared to Approach 4 across viruses (Figure 1a). Approaches 3 and 4 also tended to render more consistent prediction performance, exhibited by smaller FD and RFD ranges, and smaller RMSEs compared to those obtained by Approaches 1 and 2, with perhaps slightly better performance by Approach 3 compared to Approach 4 (Figure 2a). These findings suggest that the popPK modeling – and especially the joint PK/PD modeling – best accommodates the entire data range of concentration and neutralization titers, smoothing out the few extreme predictions by Approach 1.

Table 2.

Performance of approaches 1–4 for predicting ID50 (Panel A) and ID80 (Panel B) neutralization titers broken down by the status (detectable vs. undetectable, + vs. -) of measured serum concentration and serum neutralization titer

	ID50
		Detectable
A Virus	IC50 (µg/ml)*	Serum conc.a	Serum neut. Titerb	# of samples	Appr. 1	Appr. 2	Appr. 3	Appr. 4	Reported Values
H704_2544_140eN01	1.02	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)c
		+	−	95	0.72 (0.62, 0.80)	0.73 (0.63, 0.81)	0.74 (0.64, 0.82)	0.74 (0.64, 0.82)	Classification accuracy (95% CI)
		+	+	127	0.68 (0.48, 0.82)	0.65 (0.42, 0.80)	0.77 (0.61, 0.87)	0.76 (0.6, 0.86)	CCCrmd (95% CI)
H703_1750_140Es	0.92	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	94	0.66 (0.56, 0.75)	0.66 (0.56, 0.75)	0.69 (0.59, 0.77)	0.68 (0.58, 0.77)	Classification accuracy (95% CI)
		+	+	128	0.70 (0.52, 0.82)	0.69 (0.48, 0.83)	0.85 (0.76, 0.91)	0.85 (0.76, 0.91)	CCCrm (95% CI)
H703_1471_190s	0.19	-	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	9	NAe	NAe	NAe	NAe	Classification accuracy
		+	+	213	0.80 (0.68, 0.88)	0.77 (0.63, 0.86)	0.87 (0.8, 0.92)	0.85 (0.77, 0.91)	CCCrm (95% CI)
H704_1535_030sN	0.12	−	−	22	1.00 (0.85, 1.00)	0.86 (0.66, 0.95)	1.00 (0.85, 1.00)	0.91 (0.72, 0.98)	Classification accuracy (95% CI)
		−	+	2	NAe	NAe	NAe	NAe	NAe
		+	−	6	NAe	NAe	NAe	NAe	NAe
		+	+	216	0.75 (0.61, 0.85)	0.74 (0.59, 0.84)	0.86 (0.78, 0.92)	0.82 (0.71, 0.89)	CCCrm (95% CI)
Breakthrough virus geometric meanf	0.38	−	−	22	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	1.00 (0.85, 1.00)	Classification accuracy (95% CI)
		−	+	2	NAe	NAe	NAe	NAe	NAe
		+	-	4	NAe	NAe	NAe	NAe	NAe
		+	+	218	0.78 (0.65, 0.86)	0.72 (0.57, 0.82)	0.85 (0.75, 0.91)	0.86 (0.77, 0.92)	CCCrm (95% CI)
PVO.4	0.57	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	64	0.50 (0.38, 0.62)	0.52 (0.40, 0.64)	0.70 (0.58, 0.80)	0.59 (0.47, 0.70)	Classification accuracy (95% CI)
		+	+	158	0.59 (0.36, 0.76)	0.57 (0.33, 0.74)	0.89 0.81, 0.93)	0.86 (0.78, 0.92)	CCCrm (95% CI)
CH0505TF.gly4	0.002	−	+	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	0.96 (0.80, 0.99)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	+	222	0.62 (0.43, 0.76)	0.65 (0.47, 0.78)	0.82 (0.74, 0.88)	0.74 (0.59, 0.84)	CCCrm (95% CI)
	ID80
		Detectable
B Virus	IC80 (µg/ml)*	Serum conc.a	Serum neut. Titerb	# of samples	Appr. 1	Appr. 2	Appr. 3	Appr. 4
H704_2544_140eN01	2.98	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	156	0.88 (0.82, 0.92)	0.88 (0.82, 0.92)	0.87 (0.81, 0.91)	0.91 (0.85, 0.95)	Classification accuracy (95% CI)
		+	+	66	0.80 (0.62, 0.9)	0.83 (0.69, 0.91)	0.73 (0.54, 0.85)	0.81 (0.67, 0.9)	CCCrm (95% CI)
H703_1750_140Es	2.64	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	157	0.83 (0.76, 0.88)	0.84 (0.77, 0.89)	0.85 (0.79, 0.90)	0.90 (0.84, 0.94)	Classification accuracy (95% CI)
		+	+	65	0.83 (0.67, 0.91)	0.86 (0.71, 0.94)	0.78 (0.61, 0.88)	0.86 (0.74, 0.92)	CCCrm (95% CI)
H703_1471_190s	0.57	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	53	0.68 (0.55, 0.79)	0.70 (0.57, 0.81)	0.91 (0.80, 0.96)	0.74 (0.61, 0.84)	Classification accuracy (95% CI)
		+	+	169	0.85	0.82	0.89	0.87	CCCrm (95% CI)
H704_1535_030sN	0.33	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	-	34	0.38 (0.24, 0.55)	0.38 (0.24, 0.55)	0.74 (0.57, 0.86)	0.44 (0.29, 0.60)	Classification accuracy (95% CI)
		+	+	188	0.81 (0.69, 0.89)	0.78 (0.65, 0.87)	0.87 (0.8, 0.92)	0.84 (0.75, 0.9)	CCCrm (95% CI)
Breakthrough virus geometric meanf	1.10	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	34	1.00 (0.90, 1.00)	1.00 (0.90, 1.00)	1.00 (0.90, 1.00)	0.97 (0.85, 0.99)	Classification accuracy (95% CI)
		+	+	188	0.70 (0.57, 0.8)	0.63 (0.48, 0.75)	0.83 (0.71, 0.91)	0.85 (0.74, 0.92)	CCCrm (95% CI)
PVO.4	1.74	−	−	24	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	−	122	0.77 (0.69, 0.84)	0.75 (0.67, 0.82)	0.75 (0.67, 0.82)	0.82 (0.74, 0.88)	Classification accuracy (95% CI)
		+	+	100	0.68 (0.46, 0.82)	0.71 (0.51, 0.84)	0.88 (0.79, 0.93)	0.90 (0.83, 0.94)	CCCrm (95% CI)
CH0505TF.gly4	0.005	−	−	1	NAe	NAe	NAe	NAe	NAe
		−	+	23	1.00 (0.86, 1.00)	1.00 (0.86, 1.00)	0.96 (0.80, 0.99)	1.00 (0.86, 1.00)	Classification accuracy (95% CI)
		+	+	222	0.84 (0.71, 0.91)	0.87 (0.77, 0.92)	0.89 (0.83, 0.93)	0.86 (0.78, 0.92)	CCCrm (95% CI)

*IC50 and IC80 values are for the clinical lot of VRC01, calculated as the geometric mean of all replicates.

^aDetectable serum concentration (+) = Serum concentration > 1.0 µg/mL;

No detectable serum concentration (-) = Serum concentration ≤ 1.0 µg/mL.

^bDetectable serum neutralization titer (+) = Serum neutralization titer > 10;

No detectable serum neutralization titer (-) = Serum neutralization titer ≤ 10.

^cClassification accuracy = percent of samples correctly predicted to have neutralization titer > 10 or ≤ 10 for samples with empirically measured neutralization titer > 10 or ≤ 10, respectively.

^dCCC_rm = Concordance Correlation Coefficient for repeated measurements, expressing how well the predicted and measured neutralization titers agree with each other. Only calculated for samples with detectable VRC01 serum concentration and VRC01 serum neutralization titer.

^eNA = Not applicable. Results are only shown in a row if n ≥ 10.

Appr. = Approach.

^fGeometric mean: predict the geometric mean titers based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

Figure 1.

Plots of observed vs. predicted serum neutralization titer for each approach among double positives [category (4) samples]. (a) ID50, (b) ID80. Values in the lower right-hand corner of each plot correspond to CCCrm agreement values, with 95% confidence intervals shown in parentheses. Double positive = participant with detectable (>LLoQ) serum concentration of VRC01 and detectable (>LLoQ) serum VRC01-mediated neutralization. Breakthrough virus geometric mean: predict the geometric mean titer based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

Figure 2.

Plots of fold difference (Fold diff.), relative fold difference (Rel. fold diff.), and relative mean squared error (Rel. MSE) for each approach predicting (a) serum ID50 neutralization titers and (b) serum ID80 neutralization titers. Serum neutralization titers were predicted among double positives [category (4) samples] for each of the six tested viruses and for the breakthrough virus geometric mean. Double positive = participant with detectable (>LLoQ) serum concentration of VRC01 and detectable (>LLoQ) neutralization of VRC01. Breakthrough virus geometric mean: predict the geometric mean titer based on the geometric mean of the observed titers across the 4 breakthrough viruses, and the geometric mean of the IC50 values (Panel A) or of the IC80 values (Panel B) of the 4 breakthrough viruses (as if the geometric mean corresponded to a single virus).

We next discuss the results for predicting ID80 titer, which showed a few differences from the ID50 results. For samples in category (1), perfect (100%) classification accuracy was achieved in all predictions (Table 2b). Samples in category (3) only existed for one virus, CH0505TF.gly4, and classification accuracy was again comparable across the four approaches (100% accuracy for all approaches except for Approach 3 at 96%). However, for samples in category (2) or (4), the performance of Approaches 1 and 2 tended to be more comparable to that of Approaches 3 and 4, and the relative prediction performance of the four approaches appeared to depend somewhat on the neutralization sensitivity of the viruses: for viruses that are less resistant to VRC01 (IC80 < ~1.0 μg/mL), Approach 3 generally showed the best accuracy in predicting the detectability of serum neutralization and the magnitude of serum neutralization titer, followed by nearly indistinguishable differences in performance among Approaches 1, 2 and 4; for viruses that are more resistant against VRC01 (IC80 > ~1.0 μg/mL), Approach 4 generally showed comparable or better performance than the other approaches, with Approaches 1 and 2 being slightly better or comparable to Approach 3 except for PVO.4 in category (4) (Table 2b). For example, for samples in category (4), similar results were seen: CCC_rm for Approaches 1, 2, 3 and 4 against the three viruses with IC80 < ~1 μg/mL was 85%, 82%, 89% and 87%, 81%, 78%, 87% and 84%, and 84%, 87%, 89% and 86%; against the three viruses with IC80 > ~1.0 μg/mL, CCC_rm for Approaches 1, 2, 3 and 4 was 80%, 83%, 73% and 81%, 83%, 86%, 78% and 86%, and, 68%, 71%, 88% and 90%. For the breakthrough virus geometric mean, CCC_rm was again comparable between Approaches 3 and 4 (83% vs 85%, respectively), followed by Approaches 1 and 2 (70% and 63%, respectively) (Figure 1b). Lastly, compared to the prediction of ID50 titers, the relative merit of Approaches 3 and 4 to Approaches 1 and 2 in terms of smaller FD and RFD ranges, and smaller RMSEs was less obvious for the prediction of ID80 titers except for the breakthrough virus geometric mean (Figure 2b). Similarly, approaches 1–4 had comparable performance in classification accuracy when all four sample categories were combined (Table S2B).

In summary, our results show that incorporating longitudinal measurements of serum concentration and neutralization titers of VRC01 considerably improves post-administration prediction of both the detectability of serum neutralization, and the magnitude of serum neutralization titer. This finding holds true for predicting neutralization against not only reference viruses but also contemporaneously circulating strains of HIV-1. We also found that for samples with detectable concentration and neutralization titer, joint PK/PD modeling using neutralization titers and IIP (i.e., Approach 4) did not show improvement in predicting ID50 titers compared to joint modeling using neutralization titers and serum concentration (i.e., Approach 3), and only incremental improvement in predicting ID80 titers against more resistant viruses. Even though IIP has been proposed to more accurately quantify antiviral activity (compared to IC50 or IC80 alone), our findings suggest that the additional information in IIP is likely only more valuable for viruses that are harder to be neutralized by VRC01 or that serum neutralization titers and IIP may exhibit a relationship that is not best modeled linearly with the number of paired data available in our pilot study. As it is resource-intensive to longitudinally measure serum neutralization titers, it is likely that the latter implication applies to most clinical bnAb studies. In addition, because Approaches 1 and 2 generally also rendered reasonable prediction accuracy, and the merit of Approaches 3 and 4 relative to Approaches 1 and 2 was less obvious for the prediction of ID80 titers against both reference viruses and circulating strains of HIV-1, our analyses suggest that Approaches 1 and 2 could be reasonable approaches in practice as they do not require running neutralization titer assays on serum samples and the statistical analyses are simpler. Lastly, although neither Approach 1 nor Approach 2 obviously outperformed the other (likely due to the relatively small sample size) and both approaches are empirical-based, Approach 2 (which utilizes model-predicted concentration) is generally preferred over Approach 1 when there are sufficient serum concentration data to build a robust PK model. These observations provide valuable support in the choice of approaches for the prediction of serum neutralization titers at the estimated time of infection in case-control studies.

One constraint of our previous study⁶ was that the comparison of Approach 3 vs. Approaches 1 and 2 was based on a single tier 2 virus. To increase the scope of the study, here we also predicted serum neutralization titer against four contemporaneously circulating HIV-1 viruses, strengthening the relevance of our conclusion regarding the advantage of Approach 3. Additionally, building on our previous work where we predicted post-administration serum neutralization titer in adults in the US at low risk for HIV infection,⁶ here we used data from the first efficacy trials of an HIV bnAb (VRC01), which are being conducted in at-risk cohorts in sub-Saharan Africa and the Americas and Switzerland, with important demographic differences from the US cohort. Lastly, our previous study was unable to predict neutralization titers for samples that were not detectable by both the concentration and neutralization titer assays due to its restricted way in handling readouts below the LLoQ of the assays. Here we accommodated the modeling of all samples regardless of whether their readouts were above or below the LLoQs, and the proposed approaches all achieved satisfactory prediction accuracy in classifying the detectability of VRC01 via neutralization activity.

We next discuss some limitations of our study. First, it is unknown whether our prediction methods will be applicable to other bnAbs besides VRC01, or perhaps more importantly, combinations of bnAbs. The latter question is particularly important to address, given the possibilities of synergistic or antagonistic neutralization interactions among multiple bnAbs infused in combination. These possibilities should be thoroughly investigated, for each considered bnAb combination, in both in vitro and animal models prior to clinical consideration. In vitro studies have already identified a number of double²⁰ - and triple²¹ -bnAb combinations with synergistic neutralization potency; these findings await validation in vivo in passive immunization/SHIV challenge studies. When data become available from ongoing and future trials of other single bnAb or combination bnAb infusions,²² similar approaches can be adapted to analyze data from such studies. Another potential consideration is that resource barriers associated with testing both serum concentrations and serum neutralization titers of multiple bnAbs infused simultaneously may arise, which could lead to a more resource-sparing approach (Approach 1 or 2) outweighing the demonstrated advantages (in terms of prediction accuracy) of the resource-intensive approach (Approach 3 or 4). Second, it is unknown how well our approaches would predict serum neutralization titers in people living with HIV. Given that our approaches are developed using data collected prior to HIV infection, further research is needed to understand the relationship between serum concentration and serum neutralization post acquisition of HIV infection. Lastly, our proposed approaches could be readily applicable to the prediction of neutralization titers in mucosal samples; however, further research is needed to understand whether comparable predictive accuracy is achieved across different biological samples and whether the incorporation of serum data could be used to improve prediction when such data become available.

In conclusion, it is encouraging to confirm that neutralization titers in serum against both reference viruses and breakthrough viruses could be reasonably well predicted. The performance of the proposed approaches achieved satisfactory levels based on serum concentration data from 47 participants, but neutralization titer data from only 31 participants in two different dose groups and two different study cohorts. These findings further solidified assay planning for the AMP marker studies, which will include measurements of serum concentration at all available time-point and neutralization titers at a subset of time-points to enable Approaches 3 and 4 to be used. These results also provide important guidance for the planning of other marker studies in future bnAb and combination bnAb trials.

Funding Statement

This work was supported by the National Institute of Allergy and Infectious Disease of the National Institutes of Health under Grants UM1 AI068614 [LOC: HIV Vaccine Trials Network], UM1 AI068619 [LOC: HIV Prevention Trials Network], UM1 AI068635 [HVTN SDMC FHCRC], UM1AI068617 [HPTN SDMC], UM1 AI068618 [HVTN Laboratory Center FHCRC], and UM1AI068613 [HPTN Laboratory Center]. The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Disclosure of potential conflicts of interest

The authors have no conflicts of interest to disclose.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2021.1908030.