A meta-analysis of passive immunization studies shows an association of serum neutralizing antibody titer with protection against SHIV challenge

SUMMARY

Passively administered broadly neutralizing antibodies (bNAbs) targeting the HIV envelope glycoprotein (Env) have been shown to protect nonhuman primates against chimeric simian-human immunodeficiency virus (SHIV) infection. With data from multiple nonhuman primate SHIV challenge studies that used single bNAbs, we conducted a meta-analysis to examine the relationship between predicted serum 50% neutralization titer (ID50) against the challenge virus and infection outcome. In a logistic model that adjusts for bNAb epitopes and challenge viruses, serum ID50 had a highly significant effect on infection risk (p < 0.001). The estimated ID50 to achieve 50%, 75%, and 95% protection was 91 (95% CI: 55, 153), 219 (117, 410), and 685 (319, 1471), respectively. This analysis indicates that serum neutralizing titer against the relevant virus is a key parameter of protection and that protection from acquisition by a single bNAb may require substantial levels of neutralization at the time of exposure.

Graphical Abstract

eTOC:

Pegu et al. present a meta-analysis of nonhuman primate studies that administered a single broadly neutralizing antibody (bNAb) before SHIV challenge. They find that serum neutralizing titer against the challenge virus strongly correlates with protection. Further, protection mediated by a single bNAb may require substantial levels of neutralization at exposure.

Introduction

Close to 2 million new HIV-1 infections occur annually and there remains an urgent global public health need for an effective HIV-1 vaccine that can prevent infection (UNAIDS, 2018). In the absence of an effective vaccine, alternative strategies to prevent HIV-1 infection are being actively pursued (Eakle et al., 2018). Human broadly neutralizing antibodies (bNAbs) against HIV-1 present an attractive strategy for prevention of viral infection because of their favorable safety profile and potential for long-term persistence in the circulation.

HIV-1 bNAbs are derived from the B-cells of selected HIV-1 infected donors and bind to relatively conserved regions of the envelope glycoprotein (Env) expressed on the surface of infectious virions – mediating potent virus neutralization. Detailed virologic and structural studies have determined that bNAbs target specific epitopes in Env, including the CD4 binding site, the V3-glycan supersite, the V1V2 apex, the gp 120/140 interface region and the membrane proximal region of gp41, to potently neutralize many diverse strains of HIV-1 (McCoy and Burton, 2017; Sok and Burton, 2018). Moreover, bNAbs have been shown to protect against chimeric simian-human immunodeficiency virus (SHIV) infection in nonhuman primate (NHP) models of HIV-1 infection, providing proof of concept for their potential to prevent HIV-1 infection in humans [reviewed in (Hessell et al., 2018; Pegu et al., 2017)]. Most commonly, protection in NHPs has been assessed by intravenous administration of a single bNAb, followed one day, or several days later, by a single mucosal challenge with a SHIV that would result in infection of all control animals (Hessell et al., 2007; Hessell et al., 2016; Hessell et al., 2009b; Hessell et al., 2010; Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Rudicell et al., 2014; Shingai et al., 2014). A variety of bNAbs have been tested, often at varying infusion doses – allowing an assessment of the amount of antibody required to achieve protection (Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Shingai et al., 2014).

Several HIV-1 bNAbs have entered clinical trials – including antibodies to four distinct epitopes on the HIV-1 Env (Caskey et al., 2019; Cohen and Caskey, 2018; Sok and Burton, 2018). VRC01 is a bNAb that targets the CD4 binding site of the HIV-1 envelope that has demonstrated protection against SHIV infection after passive administration in nonhuman primate studies (Gautam et al., 2016; Ko et al., 2014; Rudicell et al., 2014). It is the first bNAb to be tested for its ability to protect against HIV-1 acquisition after passive administration in humans. Two parallel randomized placebo controlled studies, the Antibody Mediated Prevention (AMP) trials, were launched in 2016, with the primary objective to assess whether intravenously-delivered VRC01 prevents HIV-1 infection in adults at high risk of HIV-1 acquisition (Gilbert et al., 2017). While the field awaits the results of the AMP trials, modeling studies have further supported the potential of passive administration of bNAbs in preventing HIV-1 infection (Huang et al., 2018).

To better understand the titers needed to achieve protection, we performed a meta-analysis of the data from passive immunization studies in NHP, as described above, that use a single infusion of a bNAb, followed by a mucosal SHIV challenge. Since bNAbs vary in their neutralization potency, and challenge SHIVs vary in their neutralization sensitivity to a given bNAb, we used the measured bNAb antibody level present in the sera of animals on the day of SHIV challenge and the in vitro neutralization sensitivity of the challenge virus to the bAb to infer the serum neutralization titer against the challenge SHIV. Predicted serum neutralization titers were then assessed for association with protective efficacy, considering other variables that included the challenge virus and the bNAb epitope targeted. We found a strong association between serum neutralization titers and protection against SHIV challenge with the estimated serum 50% neutralization titer (ID50) needed to achieve 50%, 75%, and 95% protection being 91 (95% CI: 55, 153), 219 (95% CI: 117, 410), and 685 (95% CI:319, 1471), respectively.

Results

Description of study data

This analysis combined data from 13 published studies (Hessell et al., 2007; Hessell et al., 2016; Hessell et al., 2009b; Hessell et al., 2010; Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Rudicell et al., 2014; Shingai et al., 2014) and 5 unpublished studies (Table 1, Figure 1). Criteria for inclusion were: the infusion of a single dose of one bNAb, followed by a single high-dose SHIV challenge; and the availability of bNAb level data on the day of SHIV challenge. The serum bNAb level was used to predict the serum neutralization ID50 titer against the SHIV used for challenge, based on the known in vitro IC50 of the bNAb against the SHIV.

Table 1.

Passive immunization studies. See also Tables S1, S2, S6.

HIV-1 Env Epitope	Antibody	Challenge virus (route)	Antibody dose (mg/kg)	No. of animals (infected/total)	Reference
CD4 binding site	b12	SHIVSF162P4 (vaginal)	25	0/4	(Parren et al., 2001)
			5	2/4
			1	4/4
		SHIVSF162P3 (vaginal)	25	1/9	(Hessell et al., 2007)
	VRC01	SHIVSF162P3 (vaginal)	20	0/4	(Ko et al., 2014)
		SHIVSF162P3 (rectal)	20	0/4	(Ko et al., 2014)
		SHIVSF162P3 (oral)	20	0/4	(Hessell et al., 2009a)
			5	1/5
		SHIVBaL.P4 (rectal)	20	0/6	(Ko et al., 2014)
			5	0/6	(Ko et al., 2014)
			1.25	0/4	Unpublished
			0.3	16/22	(Ko et al., 2014)
		SHIVAD8EO (rectal)	50	1/2	(Shingai et al., 2014)
			20	2/2
	VRC01LS	SHIVSF162P3 (rectal)	20	1/6	Unpublished
		SHIVBaL.P4 (rectal)	0.3	5/12	(Ko et al., 2014)
		SHIVBG505 (rectal)	20	0/4	Unpublished
	VRC07–523LS	SHIVBaL.P4 (rectal)	0.2	1/4	(Rudicell et al., 2014)
			0.05	4/4
		SHIVSF162P3 (rectal)	20	0/6	Unpublished
	N6LS	SHIVBG505 (rectal)	20	0/4	Unpublished
			10	1/4
			2.5	2/4
	3BNC117	SHIVAD8EO (rectal)	5	0/2	(Shingai et al., 2014)
			1	2/2
		SHIV327c (rectal)	10	3/4	(Julg et al., 2017)
			2	3/3
V1V2	PGDM1400	SHIV325C (rectal)	2	0/5	(Julg et al., 2017)
			0.4	0/5
			0.08	3/4
	CAP256.VRC26.25LS	SHIV325C (rectal)	2	0/4	(Julg et al., 2017)
			0.4	0/4
			0.08	0/4
	PG9	SHIVBaL.P4 (rectal)	20	2/6	(Ko et al., 2014)
			5	3/6
			0.3	6/6
V3-glycan	PGT121	SHIVSF162P3 (vaginal)	5	0/5	(Moldt et al., 2012)
			1	0/5
			0.2	2/5
		SHIVAD8EO (rectal)	20	0/2	(Shingai et al., 2014)
			5	1/2
			1	0/2
			0.2	2/2
		SHIV327c (rectal)	10	0/4	(Julg et al., 2017)
			2	1/3
	PGT126	SHIVSF162P3 (vaginal)	10	0/5	(Moldt et al., 2016)
			2	3/5
			0.4	4/5
		SHIVSF162P3 (rectal)	10	1/4	(Moldt et al., 2016)
			2	2/4
			0.4	4/4
	10–1074	SHIVAD8EO (rectal)	20	0/2	(Shingai et al., 2014)
			5	0/2
			1	2/2
High mannose glycan	2G12	SHIVSF162P3 (vaginal)	40	2/5	(Hessell et al., 2009b)
MPER	4E10	SHIVBa-L (rectal)	50	0/6	(Hessell et al., 2010)
	2F5	SHIVBa-L (rectal)	50	0/6	(Hessell et al., 2010)
	10E8	SHIVBaL.P4 (rectal)	20	0/6	(Ko et al., 2014)
			5	0/6
			0.3	3/6

Figure 1. Passive immunization studies in NHPs.

Information on the passive immunization NHP studies used for the meta-analysis, organized by research group.

Overall, data from 274 animals were available, with 242 NHPs from 13 SHIV challenge studies published between 2001 and 2017, and an additional 32 NHPs from unpublished studies (Table 1, Figure 1). Data were collected by five groups (Table 2): Mascola and colleagues from VRC, NIAID (n=130); Burton and colleagues from Scripps (n=80); Barouch and colleagues from Harvard (n=35); Martin and colleagues from the Viral Pathogenesis and Vaccine Section (VPVS), NIAID (n=22); and Haigwood and colleagues from the Oregon National Primate Research Center (ONPRC), Oregon Health & Science University (OHSU) (n=7). Information about sex is available for 198 animals (64% male, 36% female). The study dataset (Table 2) includes 16 bNAbs targeting four Env regions: the CD4 binding site, V1V2, V3-glycan, and MPER; the bNAb infusion dose ranged from 0.05 to 50 mg/kg. Eight different SHIVs were used, including three tier 1 viruses (highly sensitive to neutralization) and five tier 2 viruses (less sensitive to neutralization). Regarding route of challenge, 76%, 22%, and 2.6% of study animals were challenged by the rectal, vaginal, and oral route, respectively. The viral challenge dose ranged from 300 to 12800 TCID50, with challenges implemented at day 1 (53%), day 2 (26%), or day 5 (22%) post bNAb transfer.

Table 2.

Descriptive statistics of study dataset.

Categorical Variable	Category		N (no. of animals)	% (% among animals with information observed)
Study Group	Mascola		130	47.4%
	Burton		80	29.2%
	Barouch		35	12.8%
	Martin		22	8.0%
	Haigwood		7	2.6%
HIV-1 Env Epitope	CD4 Binding Site	b12	21	7.7%
		VRC01	57	20.8%
		VRC01LS	22	8%
		VRC07–523LS	14	5.1%
		N6LS	12	4.4%
		3BNC117	11	4%
	V1V2	PGDM1400	9	3.3%
		CAP256.VRC26.25LS	12	4.4%
		PG9	18	6.6%
	V3-glycan	PGT121	30	10.9%
		PGT126	27	9.9%
		10–1074	6	2.2%
		2G12	5	1.8%
	MPER	4E10	6	2.2%
		2F5	6	2.2%
		10E8	18	6.6%
Virus	Tier 1	SHIVBaL	12	4.4%
		SHIVBaLP4	94	34.3%
		SHIVSF162P4	12	4.4%
	Tier 2	SHIV325C	21	7.7%
		SHIV327C	14	5.1%
		SHIVAD8EO	22	8%
		SHIVBG505	16	5.8%
		SHIVSF162P3	83	30.3%
Challenge Route	Oral		7	2.6%
	Vaginal		60	21.9%
	Rectal		207	75.5%
Challenge dose (TCID50)	300		85	40.3%
	500		26	12.3%
	1000		22	10.4%
	2000		12	5.7%
	12800		66	31.3%
	Not available		63	-
Sex	Female		72	36.4%
	Male		126	63.6%
	Not available		76	-
Target Cell for Measuring in vitro	TZM-bl		245	89.40%
Neutralization Sensitivity	PBMC		29	10.60%
Day of Challenge post Ab transfer	1		144	52.60%
	2		70	25.50%
	5		60	21.90%
Continuous Variable			Median	Range
Ab Dose (mg/kg)			5	(0.05,50)
Ab Conc at day of challenge (ug/ml)			25.02	(0.11,1122.33)
ID50 titers*			113.67	(2.17,8010.58)
ID80 titers*			29.05	(0.003, 4005.29)
IIP**			2.42	(0.34, 8.46)

^*ID50 (ID80) titers calculated as bNAb serum concentration at day of challenge divided by IC50 (IC80) of the bNAb against the challenge virus.

^**The instantaneous inhibitory potential (IIP) = −log₁₀(1 − f(c)), where f(c) = c^m/(IC50^m + c^m) = fraction neutralization at concentration c, with slope m = log(4)/(log IC80 - log IC50), and c being bNAb concentration at the day of challenge.

The in vitro neutralization potency IC50 of each bNAb against the challenge SHIV was measured using replication-competent SHIV infection of TZM-bl cells (89%) or PBMCs (11%) (Table SI). Serum bNAb level on the day of viral challenge was measured by quantitative ELISA and ranged from 0.11 to 1122 μg/mL (median 25.0 μg/mL). The predicted serum neutralization ID50 titers against the SHIV challenge virus used ranged from 2.17 to 8011 (median 114) (Table 2). Full information (including dose, challenge virus, bNAb concentration at day of challenge, protection status, and bNAb IC50 and IC80 against the challenge SHIV) for the 32 NHPs from unpublished studies is provided in Table S2.

Logistic regression modeling of infection risk as a function of ID50 titer and other covariates

The risk of infection was modeled as a function of the log 10 transformed predicted serum neutralization ID50 titer and additional covariates. Since the observed infection risk was zero for several bNAbs (CAP256.VRC26.25, 4E10, 2F5), bNAb information was modeled by the Env epitope category they target. In addition, since the observed infection risk was also zero for the SHIV_BaL virus, in our analysis it was pooled together with SHIV_BaLP4 virus in the logistic model. Among various combinations of covariates to adjust for, the best fitting model selected by minimum AIC models infection risk as a linear function of log10(ID50) and adjusts for epitope and virus type, with a significant interaction (p=0.005) between log10(ID50) and challenge virus (SHIV_SF162P3 vs. other) (Table 3). Within this best fitting model, the overall contribution of ID50 to infection risk is highly significant (p<0.001). In particular, an increase in serum ID50 titer was associated with a significant reduction in infection risk, with an odds ratio of 0.012 per 10-fold increase in ID50 titer (p<0.001) for all SHIV challenge viruses except for SHIV_SF162P3, for which an odds ratio of 0.17 per 10-fold increase in ID50 titer (p=0.002) was observed. Also, after accounting for ID50 level and challenge virus type, animals injected with bNAbs targeting MPER tend to have a significantly lower risk of infection compared to animals injected with bNAbs targeting other epitope categories (Table 3), with the caveat that these MPER bNAbs were studied against only one SHIV, which has a tier 1 neutralization phenotype but is not highly sensitive to MPER bNAbs compared to bNAbs against other epitopes.

Table 3.

Odds ratio (OR) estimates for log₁₀(ID50) and covariates based on the model of infection risk conditional on log₁₀(ID50), epitope of the bNAb, challenge virus, and the interaction between log₁₀(ID50) and challenge virus (SF162P3 vs. other). See also Table S3.

	OR	95% CI	p-value
log10(ID50) for virus other than SHIVSF162P3	0.012	(0.003,0.054)	<0.001
log10(ID50) for virus SHIVSF162P3	0.17	(0.057,0.51)	0.002
Epitope relative to MPER
CD4 Binding Site	801	(58, 16243)	<0.001
V1V2	1030	(43, 24887)	< 0.001
V3-glycan	4120	(173, 97943)	< 0.001
Virus relative to SHIVBaL/SHIVBaLP4
SHIVSF162P4	1.98	(0.29,13.43)	0.49
SHIV325C	0.008	(0,0.18)	0.002
SHIV327C	0.83	(0.18,3.79)	0.81
SHIVAD8EO	0.52	(0.12,2.17)	0.37
SHIVBG505	0.095	(0.016,0.57)	0.01
SHIVSF162P3	0	(0,0.016)	<0.001

Based on the selected logistic regression risk model, the curve for protection level based on ID50 titer was constructed for various epitope and virus combinations. As shown in Table S3, there are 12 different epitope and virus combinations in the study dataset. Serum ID50 titer was a consistently strong correlate of protection across combinations, with some variability in the specific ID50 titer needed to achieve a given protection level across combinations (Figure 2 and Figure S1). For example, when challenged with SHIV_BaL/SHIV_BaLP4, the estimated required serum ID50 titer to achieve 50% protection was 114 (95% CI: 85, 152) for bNAbs that target the CD4 binding site and 118 (95% CI: 30, 463) for bNAbs that target the V1V2 region, but only 3 (95% CI: 1, 8) for bNAbs that target MPER; the corresponding serum ID50 titers required to achieve 95% protection were 529 (95% CI: 274, 1020), 547 (95% CI: 108, 2765), and 15 (95% CI: 5, 42) for bNAbs that target the CD4 binding site, VIV2, and MPER, respectively.

Figure 2. Association of protection with serum ID50 titer for different SHIVs.

The curves for each indicated SHIV strain depict the estimated level of protection as a function of serum ID50 titer for bNabs targeting various epitopes in the study dataset. The 95% bootstrap confidence intervals for ID50 titers required to achieve 50%, 75%, and 95% protection are represented by the three colored horizontal lines on each curve. See also Figures S1 and S2.

Based on the selected risk model, we estimated the overall level of protection as a function of ID50 titer, averaging over all epitope and virus combinations in the study dataset (Figure 3). The required ID50 titers to achieve 50%, 75%, and 95% protection were estimated at 91 (95% CI: 55, 153), 219 (95% CI: 117, 410), and 685 (95% CI: 319, 1471), respectively.

Figure 3. Association of protection with serum ID50 titers averaging over all epitope and virus combinations.

The 95% bootstrap confidence intervals for ID50 titers required to achieve 50%, 75%, and 95% protection are represented by the three horizontal lines on the curve. See also Figure S3.

Logistic regression modeling of infection risk as a function of instantaneous inhibitory potential (IIP), ID50 titer, and ID80 titer

Data were available from 218 animals in studies for which both IC50 and IC80 information was available. In this subset, highly significant (p<0.001) pairwise spearman correlations are observed among log10(ID50), log10(ID80) and IIP, with a strong correlation between log10(ID50) and log10(ID80) (cor = 0.84) and between log10(ID80) and IIP (cor = 0.83) and a modest correlation between log10(ID50) and IIP (cor=0.55) (Figure S2). Adjusting for epitope and challenge virus, IIP correlated with infection risk when ID50 and ID80 titers were left out of the model, with an estimated 90.6% reduction in the odds of infection per SD increase in IIP. However, IIP was not associated with infection risk when either ID50 or ID80 titer was included in the model. In models with ID50 or ID80 titer in addition to epitope and challenge virus, the reductions in the odds of infection by SHIVsfi62P3 or a challenge virus other than SHIV_SF162P3 were 57.9% and 94.0% per SD increase in log10(ID50) and 75.1% and 98.5% per SD increase in log10(ID80). The model with ID80 titers has only slightly better fit compared to the model with ID50 (AIC=175.4 and 177.2 respectively); combining ID50 and ID80 titers did not further improve model fitting. The average estimated levels of protection as a function of ID50 titer, ID80 titer, and IIP are presented in Figure S3. The curve of protection vs IIP (Figure S3c) allows one to compute the fraction of neutralization to achieve a desired protection level. This analysis suggests that it is possible to achieve 50% protection with 93.7% neutralization (95% CI: 68.4%, 98.4%), 75% protection with 99.8% neutralization (95% CI: 96.8%, 99.9%) and 95% protection with >99.9% neutralization.

Super Learner analysis to identify best classification models

We conducted Super Learner analysis to further compare the predictive capacity of models based on ID50, ID80, IIP, or combinations thereof. Table S4 shows model performance in predicting level of protection for various candidate learning algorithms and the resulting Super Learner ensemble model fitted using ID50, ID80, IIP or a combination thereof, in addition to epitope and challenge virus. The top 10 best performing models and their CV-AUCs along with the predictors used to fit them are presented in Table S5. The Super Learner model fit using ID80, epitope, and challenge virus as predictors was the best performing model with a CV-AUC of 0.879 (95% CI: 0.827, 0.931). Addition of IIP as a predictor did not seem to enhance model performance. Replacing ID80 with ID50 in the Super Learner model led to similar performance with a CV-AUC of 0.875 (95% CI: 0.822, 0.928), which again was not enhanced by the addition of IIP. Combining ID50 titer with ID80 titer in the Super Learner did not improve performance compared to using either one of the two measures alone. The best performing model (random forest) with IIP and without ID50 and ID80 titers had a CV-AUC of 0.833 (95% CI: 0.772, 0.894), smaller than those for models using either ID50 or ID80 titer. The addition of any one of the three neutralization activity measures led to a substantial improvement of the CV-AUC compared to the best performing model based on epitope and challenge virus alone (CV-AUC of 0.719, 95% CI: 0.638, 0.800) (Table 4).

Table 4:

Top performing models and their CV-AUCs based on models with one of ID50, ID80 and IIP as predictors, in addition to epitope and challenge virus, based on the subset of 218 animals with IC80 information. See also Tables S4 and S5.

Neutralization Activity Measure*	Algorithm**		CV-AUC	95% CI
None	SL.glm		0.719	0.638–0.800
ID50	SuperLearner		0.875	0.822–0.928
ID80	SuperLearner		0.879	0.827–0.931
IIP	SL. randomF orest		0.833	0.772–0.894

^*In addition to epitope and challenge virus

^**Algorithms are listed by the functions used in the SuperLearner R package. An exception is “SL.naivebayes”, which was a custom wrapper designed to use the naiveBayes function from the e1071 package. The SL.glmnet package was used with the lasso penalty. All tuning parameters are set to the default values of the SuperLearner package, except SL.xgboost, which we modified to fit decision stumps rather than trees.

Discussion

This meta-analysis of a comprehensive dataset of published and unpublished data from passive immunization studies using bNAbs and mucosal SHIV challenges confirms that serum neutralization titer on the day of SHIV challenge is strongly associated with the protective efficacy against the challenge virus. This finding supports the hypothesis that the serum neutralization titers against prevalent HIV-1 strains will be an important determinant of protective efficacy in human passive immunization studies, including the ongoing AMP trials. Moreover, our models allow the estimation of serum neutralization titers needed to achieve 50%, 75%, and 95% protection from infection in the SHIV challenge model, which can be compared to data from ongoing and future human studies to provide a better understanding of the predictability of NHP protection models.

The passive transfer studies in this meta-analysis all used a single mucosal SHIV challenge that aimed to produce infection in 100% of control animals. We chose this study format because it represented the largest body of NHP data, and because it allowed us to most precisely associate serum antibody level, and the inferred neutralization titer, with protection. Compared to the relative inefficiency of human sexual transmission, this NHP challenge format may represent a more stringent test of the ability of bNAbs to protect – therefore the protective sera titers arrived at in this meta-analysis may be an overestimate of the minimal titers required to mediate protection against viral challenge in the setting of mucosal exposure to HIV-1.

This meta-analysis included bNAbs to four different Env epitopes and eight different strains of SHIV. Serum ID50 titer was identified as a consistently strong correlate of protection across various combinations of epitope and virus type, although the odds ratio is less extreme for SHIVSF162P3 compared to other virus types. The less significant effect estimate of ID50 on infection risk for SHIVSF162P3 was driven by a few studies with bNAbs targeting the CD4 binding site (including b12, VRC01, VRC01LS, VRC07–523LS) and SHIV_SF162P3, which had very low overall infection rate (3 infections out of 33 animals) with little association with ID50 levels. Additional data are warranted to further investigate this finding.

Comparing different epitopes included in the study, a significantly lower risk of infection was observed with bNAbs targeting MPER relative to bNAbs targeting other epitope categories, with the caveat that these MPER bNAbs were studied using only one SHIV strain, which is known to be generally neutralization sensitive, though not unusually sensitive to neutralization by MPER bNAbs. These results indicate that for MPER targeting bNAbs, effector functions other than neutralization may also contribute significantly to their protective efficacy in this model. Additional studies using challenge viruses displaying different sensitivities to MPER bNAbs would be needed to better understand the protective efficacy of this category of bNAbs.

One limitation of our analyses is that the SHIV stocks used for these mucosal challenges have generally been noted to have limited diversity, e.g. 0.4–0.5% Env diversity in the SHIVSF162P3 challenge stock (Jayaraman et al., 2007; Ng et al., 2010) and 0.3% mean genetic Env diversity in the SHIV_BaLP4 challenge stock (Santra et al., 2015). It may be difficult to extrapolate these results to protection of humans from mucosal HIV exposure, especially if the HIV-1 virions present at the exposure event in humans exhibit greater genetic diversity than is observed in the challenge stocks in the NHP studies. Despite this limited diversity in the challenge stocks, divergence of viral sequences from the challenge virus sequence in each infected animal has been observed (Jayaraman et al., 2007; Ng et al._, 2010; Santra et al., 2015). In addition, enumeration of Transmitter/Founder (T/F) env performed in these studies has shown that the number of T/F envs are very similar to that observed in HIV infection in humans (Keele et al., 2008). This provides some support for the use of mucosal SHIV challenge as a model for natural HIV transmission.

The levels of antibody at mucosal surfaces, where the SHIV challenge was performed, were not determined in most of these studies. This is an important issue to address since antibodies at mucosal surfaces, or in mucosal tissues, represent the first barrier to the establishment of infection. However, it is challenging to accurately and reproducibly determine mucosal antibody levels, and most NHP studies have not done this. Another important caveat is that most studies performed mucosal SHIV challenge within a day after antibody administration, while recent studies have shown that mucosal accumulation of passively transferred antibodies can take more than 2 days to peak in mucosal tissues (Ko et al., 2014; Welles et al., 2018). It has also been shown that the protective efficacy of bNAbs against mucosal SHIV challenge can involve clearance of infectious virus from distal sites of virus exposure, suggesting that antibody-mediated protection may not be solely related to action at the mucosal surface (Hessell et al., 2016; Liu et al., 2016). Overall, systemic antibody levels appear to be a robust corelate of antibody-mediated protection against infection.

The instantaneous inhibitory potential (IIP) measures the log 10 reduction in a single round of infection in the presence of a bNAb, and in capturing information about the neutralization curve slope, measures a different feature of neutralization than ID50 or ID80 titers (Jilek et al., 2012). Among a subset of studies with ID80 and IIP available, we found IIP to be positively correlate with protection against SHIV infection, as also reported previously in a repeated low-dose challenge study (22). We did find that IIP was a strong independent predictor of protection by itself but adding IIP to models with ID50 or ID80 titers did not further enhance the performance of the model to predict the level of protection compared to the performance of the model with ID50 or ID80 titer alone.

The protective serum ID50 titers reported here are similar to the titers that were observed to be protective for 5 different bNAbs in passive transfer experiments using repetitive mucosal SHIV challenge (Shingai et al., 2014) and this lends additional support to the protected titer predicted from this meta-analysis. Overall, these data suggest that the protective titers from NHP SHIV challenge studies may be predictive of efficacy in human trials of bNAbs and therefore a useful metric to measure in such trials. Of note, many published studies report neutralization using 293T-derived Env-pseudoviruses, which can be more sensitive to neutralization than the PBMC derived SHIV neutralization data that were used in this analysis (Cohen et al., 2018; Louder et al., 2005). However, the assay using 293T-derived pseudoviruses is reproducible with low technical measurement error and is therefore advantageous for developing a potential statistical correlate of prevention efficacy; this assay is being used in the first planned correlates analyses in the AMP trials. In addition, a recent study of vaccination with recombinant native-like Env trimers to induce nAbs found that a serum ID50 titer of 500 was correlated with >90% protection against mucosal SHIV challenge (Pauthner et al., 2019). This finding is similar to our result that a serum ID50 titer of 685 corresponds to 95% protection. These findings suggest that achieving such high titers should be one of the key goals of a successful prophylactic vaccine.

Our analysis suggests that serum neutralization titer is a strong predictor of protective efficacy in the NHP model of HIV infection and could be an important determinant of protective efficacy against HIV. Our findings may also inform the development of Env protein-based vaccination strategies that aim to elicit bNAbs and suggest benchmarks for the bNAb titers that a successful preventative vaccine may need to achieve. Our results further highlight the importance of evaluating serum neutralizing titers against relevant viruses in efficacy trials of bNAbs and vaccine candidates.

STAR Methods

LEAD CONTACT AND MATERIALS AVAILABILITY

Materials Availability Statement: Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Ying Huang (yhuang@fredhutch.org). This study did not generate new unique reagents.

EXPERIMENTAL MODEL AND SUBJECT DETAILS

Rhesus macaques

Details on the 242 NHPs used in the 13 published studies can be found in the original papers (Hessell et al., 2007; Hessell et al., 2016; Hessell et al., 2009b; Hessell et al., 2010; Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Rudicell et al., 2014; Shingai et al., 2014). The 32 NHPs (20 males, 12 females) in the 5 unpublished studies were Outbred Indian rhesus macaques (Macaca mulatto). All rhesus monkeys were adults (> 3 years old) that did not express the class I alleles Mamu-A*01, Mamu-B*08, and Mamu-B*17. All animal experiments were reviewed and approved by the Animal Care and Use Committee of the Vaccine Research Center, NIAID, NIH, and all animals were housed in a fully AAALAC (Association for Assessment and Accreditation of Laboratory Animal Care International) accredited facility with stringent standard operating procedures and compliant with U.S. Animal Welfare Act (AWA) and Regulations, the Public Health Service (PHS) Policy on Humane Care and Use of Laboratory Animals, the Guide for the Care and Use of Laboratory Animals and all applicable NIH Policies on in vivo research.

METHOD DETAILS

All method details for the 242 NHPs from the 13 published studies can be found in the original papers (Hessell et al., 2007; Hessell et al., 2016; Hessell et al., 2009b; Hessell et al., 2010; Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Rudicell et al., 2014; Shingai et al., 2014). Method details for the 32 NHPs in the 5 unpublished studies follow below:

Antibodies

bNAbs were prepared as previously described, as follows: N6LS: Walker et al., 2011; VRC01 and VRC01LS: Ko et al., 2014; VRC07–523LS: Rudicell et al., 2014.

Virus strains

SHIV challenge stocks were prepared as previously described, as follows: SHIV_BG505: Li et al., 2016; SHIV_BALP4: Pal et al., 2003; SHIV_SF162P3: Harouse et al., 2001; Tan et al., 1999.

Rhesus macaque immunization and challenge

NHPs were intravenously infused with the appropriate bNAb 2–5 days before challenge. Each infusion consisted of the appropriate dose of the stock antibody diluted in PBS. For challenge, animals were inoculated intra-rectally with the dose of the SHIV challenge stock that was determined to lead to infection in control animals after a single challenge.

Measurement of antibody levels

Similar quantitative ELISA methods as those used in the published studies (Hessell et al., 2007; Hessell et al., 2016; Hessell et al., 2009b; Hessell et al., 2010; Julg et al., 2017; Ko et al., 2014; Moldt et al., 2016; Moldt et al., 2012; Parren et al., 2001; Rudicell et al., 2014; Shingai et al., 2014) were used to measure antibody levels (μg/mL) in plasma or serum samples on the day of challenge. Briefly, anti-idiotype antibodies or HIV-1 Env-based proteins were used to capture the bNAbs; complexes were detected by incubation with HRP-conjugated anti-human IgG conjugates, followed by incubation with a colorimetric substrate and spectrophotometric measurement (Ko et al., 2014).

Neutralization assays

Neutralization of replication-competent SHIV by the studied bNAbs was evaluated in vitro using TZM-bl (Li et al., 2005; Montefiori, 2009) or PBMC (Parren et al., 2001) target cells (NIH AIDS Research and Reference Reagent Program, catalog number 8129) as described. The in vitro 50% and 80% inhibitory concentration (IC) neutralization titers, IC50 and IC80, determined previously in published studies were used for analysis. For SHIVs used in unpublished studies, similar methods were used to determine the neutralization titers. For the TZM-bl target cell assay (Li et al., 2005; Montefiori, 2009), replication-competent SHIV viruses were incubated with antibody for 30 min at 37°C, after which TZM-bl cells were added. The HIV protease inhibitor indinavir was added to wells at a final concentration of 1 μM to limit infection of target cells to a single round of viral replication. Luciferase expression was quantified 48 hours after infection upon cell lysis and the addition of luciferin substrate (Promega). For the PBMC-based assay, phytohemagglutinin (PHA)-activated peripheral blood mononuclear cells (PBMC) were used as target cells; neutralization assessment was carried out as described previously (Parren et al., 2001).

QUANTIFICATION AND STATISTICAL ANALYSIS

All analyses were based on data from individual animals that received bNAb infusion. Descriptive analyses were performed to characterize the distribution of various covariates, including study group, bNAb and epitope-targeting region, challenge virus, route and dose, sex of animal, day of challenge post-bNAb administration, and target cell type for measuring neutralization sensitivity.

Predicted ID50 titer against the challenge virus was calculated by dividing serum concentration of the bNAb at the day of challenge by the in vitro IC50 titer of the bNAb against the SHIV challenge virus. In support of this approach, predicted ID50 titers were shown to be tightly correlated with experimental ID50 titers in a repeated low dose SHIV challenge model (Gautam et al., 2016; Wagh et al., 2018) and in a phase 1 study using bNAb VRC01 (Ledgerwood et al., 2015). We also analyzed a subset of data from a previously unpublished passive transfer study of the N6LS bNAb to further establish the validity of this prediction approach. The comparison between the experimentally determined and predicted ID50 titers in plasma samples of rhesus macaques five days after intravenous administration is presented in Table S6.

Logistic regression was used to model the association of log10-transformed serum ID50 titer with infection outcome, and to assess possible effect modification by other covariates as well as to adjust for possible confounding variables. The Akaike information criterion (AIC) was used to compare the fit of different models. All p-values are two-sided and based on a Wald test. Statistical significance is declared with p-value < 0.05.

Since all control animals were infected in each of the included studies (infection rate = 100%), based on the selected model of P(infection|ID50, covariate), the % protection conditional on ID50 titer at given covariate level was computed as % protection = [1-P(infection | ID50, covariate)] × 100%. Serum ID50 titer for achieving 50%, 75%, and 95% protection was estimated conditional on covariates. A nonparametric bootstrap procedure with 1000 resamples was conducted to construct 95% confidence intervals about these estimates.

Based on the selected model of P(infection | ID50, covariate), the overall % protection at a given ID50 titer, averaging over values of all other covariates, was estimated as 1- P(infection | ID50)= 1- ΣP(infection | ID50, covariate group) ×P(covariate group | ID50), where the sum is over all distinct covariate groups and P(covariate group | ID50) was modeled by multinomial logistic regression with a cubic spline basis for log10(ID50) with a node at its median. Again, a nonparametric bootstrap with 1000 resamples was used to construct 95% confidence intervals for ID50 titers required to achieve 50%, 75%, and 95% protection in the study dataset.

The instantaneous inhibitory potential (IIP) measures the log10 reduction in infectious events in the presence of drugs/antibodies (Montefiori 2009). In a subset of studies for which in vitro IC80 titer of the bNAb against the challenge virus was available (n=218), IIP at the day of challenge was computed using estimated Hill curves based on IC50 and IC80 values (Wagh et al., 2016) as IIP = −log₁₀(1 − f(c)), where f(c) = c^m/(IC50^m + c^m) = fraction neutralization at concentration c, with slope m = log(4)/(log IC80 − log IC50), and c being bNAb concentration at the day of challenge; serum 80% neutralization titer (ID80) against the challenge virus was also predicted by dividing serum concentration of the bNAb at the day of challenge by the IC80 titer of the bNAb against the challenge virus. Spearman rank correlation was assessed among ID50 titer, ID80 titer, and IIP. Logistic models were fitted with log10 transformed ID50 titer, ID80 titer, and/or IIP to evaluate and compare their performance in prediction.

ID50 titer, ID80 titer and IIP were also assessed as predictors of infection by building convex ensemble models using regression stacking (Breiman, 1996; Wolpert, 1992), also known as Super Learning (van der Laan et al., 2007), with the SuperLearner R package (van der Laan et al., 2007; Polley et al., 2011). This approach applies convex weights to combine candidate learning algorithms with the aim of minimizing the negative log-likelihood loss as a cross-validated prediction criterion. The candidate learning algorithms included lasso (Tibshirani, 1996) [with logistic link as implemented in the glmnet R package (Friedman J, 2018)], random forests (Liaw and Wiener, 2002) [implemented in the randomForest R package (Liaw and Wiener, 2018-03-25)], Naïve Bayes (John and Langley, 1995) [implemented in the e1071 R package (Meyer et al., 2017-02-02)], and boosted decision stumps via extreme gradient boosting (Chen and Guestrin, 2016) [implemented in the xgboost R package (Chen et al., 2018-01-23)]. We also included classical statistical techniques: generalized linear models and stepwise-selected generalized linear models (again with logistic link as implemented in the glmnet R package). Each candidate learning algorithm and the resulting Super Learner ensemble model was fit with log10 transformed ID50 titer, log10 transformed ID80 titer, IIP, or a combination thereof with additional covariates. The candidate learners and the Super Learner were evaluated via external 10-fold cross-validation to guard against overfitting. The cross-validated area under the receiver operating characteristics curve (CV-AUC) was used as the prediction metric. Wald-type 95% confidence intervals about CV-AUC were computed using influence function-based standard error estimates (LeDell et al., 2015).

DATA AND CODE AVAILABILITY

Data and Code Availability Statement: All data analyzed during this study is included in this or other published studies. All code used during this study is included in other published studies.

Supplementary Material

2

Table S2. Detailed animal, challenge, antibody, and protection status information for the n=32 NHPs from the 5 unpublished studies. Related to Table 1.

KEY RESOURCES TABLE

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Antibodies
N56LS	Generated in-house as previously described (Walker et al., 2011, Huang et al., 2016)	N/A
VRC01	Generated in-house as previously described (Ko et al., 2014)	N/A
VRC01LS	Generated in-house as previously described (Ko et al., 2014)	N/A
VRC07–523LS	Generated in-house as previously described (Rudicell et al., 2014)	N/A
Bacterial and Virus Strains
SHIVBG505	Generated in-house as previously described (Li et al., 2016)	N/A
SHIVBaL P4	Generated in-house as previously described (Pal et al., 2003)	N/A
SHIVSF162P3	Generated in-house as previously (Harouse et al., 2001; Tan et al., 1999)	N/A
Biological Samples
N/A	N/A	N/A
Chemicals, Peptides, and Recombinant Proteins
N/A	N/A	N/A
Critical Commercial Assays
N/A	N/A	N/A
Deposited Data
N/A	N/A	N/A
Experimental Models: Cell Lines
TZM-bl cells (also called JC53BL-1)	NIH AIDS Research and Reference Reagent Program	Cat. no. 8129
Experimental Models: Organisms/Strains
Indian origin, outbred, young adult male and female rhesus monkeys (Macaca mulatta) that did not express the class I alleles Mamu-A*01, Mamu-B*08, and Mamu-B*17	Alphagenesis Inc., Yermasse, SC	N/A
Oligonucleotides
N/A	N/A	N/A
Recombinant DNA
N/A	N/A	N/A
Software and Algorithms
glmnet R package	Friedman et al., 2010	https://cran.r-project.org/web/packages/glmnet/glmnet.pdf
randomForest R package	Liaw and Weiner, 2002	Version 46–14 at https://www.stat.berkeley.edu/~breiman/RandomForests/
e1071 R package	N/A	https://cranr-projectorg/web/packages/e1071/indexhtml
xgboost R package	Chen and Guestrin, 2016	https://cran.r-project.org/web/packages/xgboost/xgboost.pdf
SuperLearner R package	Van der Laan, Polley, and Hubbard, 2007; Polley et al., 2011	https://cran.r-project.org/web/packages/SuperLearner/index.html
Other
N/A	N/A	N/A

Highlights.

• Meta-analysis of NHP studies administering a single bNAb before SHIV challenge

• Serum neutralizing titers correlate strongly with protection against SHIV infection

• ID50, ID80, and IIP all predict protection, with ID50 and ID80 stronger predictors

• Substantial levels of neutralization may be needed for protection by a single bNAb