Estimating Efficacy in a Randomized Trial With Product Nonadherence: Application of Multiple Methods to a Trial of Preexposure Prophylaxis for HIV Prevention

Abstract

Antiretroviral preexposure prophylaxis (PrEP) for persons at high risk of human immunodeficiency virus infection is a promising new prevention strategy. Six randomized trials of oral PrEP were recently conducted and demonstrated efficacy estimates ranging from 75% to no effect, with nonadherence likely resulting in attenuated estimates of the protective effect of PrEP. In 1 of these trials, the Partners PrEP Study (Kenya and Uganda, 2008–2011), participants (4,747 serodiscordant heterosexual couples) were randomized to receipt of tenofovir (TDF), coformulated TDF/emtricitabine (FTC), or placebo. Intention-to-treat analyses found efficacy estimates of 67% for TDF and 75% for TDF/FTC. We applied multiple methods to data from that trial to estimate the efficacy of PrEP with high adherence, including principal stratification and inverse-probability-of-censoring (IPC) weights. Results were further from the null when correcting for nonadherence: 1) among the strata with an estimated 100% probability of high adherence (TDF hazard ratio (HR) = 0.19, 95% confidence interval (CI): 0.07, 0.56; TDF/FTC HR = 0.12, 95% CI: 0.03, 0.52); 2) with IPC weights used to approximate a continuously adherent population (TDF HR = 0.18, 95% CI: 0.06, 0.53; TDF/FTC HR = 0.15, 95% CI: 0.04, 0.52); and 3) in per-protocol analysis (TDF HR = 0.18, 95% CI: 0.06, 0.53; TDF/FTC HR = 0.16, 95% CI: 0.05, 0.53). Our results suggest that the efficacy of PrEP with high adherence is over 80%.

Editor's note: An invited commentary on this article appears on page 857, and the authors’ response appears on page 861.

Human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome is a leading cause of death globally. As of 2013, 35 million persons were estimated to be living with HIV, and 2 million adults acquire HIV each year (1). Effective prevention strategies are needed to reduce HIV incidence. Antiretroviral medications used as preexposure prophylaxis (PrEP) by HIV-uninfected persons who are at high risk of acquiring the virus is a promising new prevention strategy. Recently, 4 randomized clinical trials of daily oral PrEP demonstrated moderate to high efficacy (44%–75%) (2–5), while 2 randomized trials failed to demonstrate a protective effect (6, 7). This wide range of efficacy results was probably primarily driven by varying levels of adherence across trials. Substudies nested within the active arms of each PrEP trial measured PrEP medication concentrations in stored plasma samples and indicated that adherence levels were less than or equal to 30% in the trials with null findings (6, 7) and between approximately 50% and 80% in the trials that demonstrated efficacy; the level of adherence in the trials generally correlated with HIV prevention efficacy results (2–5).

Intention-to-treat (ITT) analyses from randomized clinical trials are widely accepted as the gold standard for causal inference. However, when adherence is imperfect, ITT analyses provide an attenuated estimate of the causal effect of the intervention. The range of efficacy results across PrEP trials has led to considerable interest in quantifying the protective effect of PrEP when adherence is high. Investigators in each PrEP trial reported findings from secondary analyses aiming to identify highly adherent participants, and in 3 trials they estimated the reduction in HIV risk associated with detection of PrEP medication in plasma from case-control or case-cohort studies nested within the active arms of the studies. These nested studies compared participants with tenofovir detected in plasma to those without tenofovir detected, without adjustment for joint predictors of adherence and HIV risk. Results suggested a stronger association between PrEP use and HIV protection than the ITT results—up to a 92% reduction in risk (2, 3, 5). However, these nonrandomized comparisons are subject to bias, because persons who choose to adhere to their assigned medication regimen may differ systematically from those who are nonadherent with regard to baseline HIV risk (known as self-selection).

Randomization, by design, produces groups that are comparable in terms of their baseline risk. As such, the observed outcomes for participants in the study's placebo arm, on average, represent the potential outcomes for participants in the active arm had they been assigned to placebo (8, 9). Several methods have been developed in recent years that aim to estimate efficacy from randomized trials with nonadherence by structuring adherent comparison groups that are comparable in terms of their potential outcomes (10–15). Using data from the Partners PrEP Study (2), the largest clinical trial of PrEP for HIV prevention conducted to date (to our knowledge), we aimed to estimate PrEP efficacy with high adherence using multiple approaches, including the principal stratification framework (10), inverse-probability-of-censoring weighting (IPCW) (11), and per-protocol analysis.

METHODS

Study population

The Partners PrEP Study was a randomized, double-blind, placebo-controlled, 3-arm clinical trial of daily oral PrEP which enrolled and followed 4,747 HIV-serodiscordant heterosexual couples (1 partner HIV-infected and 1 uninfected) in Kenya and Uganda between July 2008 and July 2011 (2). HIV-uninfected partners were randomized 1:1:1 to receive tenofovir disoproxil fumarate (TDF), TDF coformulated with emtricitabine (TDF/FTC), or placebo and were followed monthly for up to 36 months. At each study visit, HIV-uninfected partners were tested for HIV seroconversion and questionnaires were administered, including an assessment of sexual behavior in the prior month. Bottles of study medication were dispensed and the prior month's bottles were returned at each visit, along with any unused pills. At quarterly visits, plasma samples were collected and cryopreserved.

Incident HIV seroconversion after randomization was confirmed in 52 participants in the placebo arm, in 17 among those assigned to TDF, and in 14 among those assigned to TDF/FTC; an additional 14 subjects seroconverted to HIV but were subsequently determined to have been infected at the time of randomization and were not included in the primary ITT analysis. The primary trial results estimated 67% efficacy (95% confidence interval (CI): 44, 81) for TDF alone and 75% efficacy (95% CI: 55, 87) for TDF/FTC (2).

The study protocol was approved by the University of Washington (Seattle, Washington) Human Subjects Review Committee and ethical review committees at each of the study sites.

Time of HIV acquisition event

In the primary trial results, the time of the HIV acquisition event was defined as the month in which HIV seroconversion was identified at the study site. In the course of early HIV infection, HIV RNA is often detectable prior to seroconversion. Therefore, after study closure, stored plasma samples from all seroconverters were tested for HIV RNA at seroconversion and at each quarterly visit prior to seroconversion, until a negative RNA result was reached. For the analyses presented here, we defined the time of HIV acquisition as the month of first evidence of infection (RNA or seroconversion), which more closely approximates the time of the transmission event. As in the primary trial analysis (2), we excluded from our analyses persons who seroconverted during follow-up but were later determined by HIV RNA detection to have been HIV-infected at enrollment.

Adherence definition and measures

Two objective measures of adherence to the study product were used in these analyses: monthly counts of returned study medication (number of pills) and detection of tenofovir in archived plasma samples. When using pill counts to classify adherence, study pills not returned were assumed to have been consumed, and pill coverage was estimated as the proportion of pills taken out of the number expected to have been taken. Pill coverage between 80% and 107% was defined as high adherence, allowing up to 2 excess pills per month, and proportions outside of this range were defined as poor adherence, because excess pill consumption may be associated with “pill dumping” (16). Tenofovir, a component of both active arms in the trial, was measured in plasma using an ultraperformance liquid chromatography–mass spectrometry assay (17) in a randomly selected cohort that provided an estimate of study medication use for the entire trial (17) and among a sample of participants negative for herpes simplex virus type 2 (HSV-2) at enrollment, who were selected to evaluate the effect of PrEP on HSV-2 acquisition (18). In these 2 studies, tenofovir was measured in samples collected at months 1, 3, and 6 and biannually thereafter. High adherence was defined as tenofovir concentrations greater than 40 ng/mL, since this level has been shown to be the lower bound consistent with daily use (17). Among persons selected into these cohorts, we estimated the proportion who were adherent by each measure and the concordance from months 1 through 24.

Principal stratification approach

In the principal stratification framework, “strata” are identified by postrandomization variables, such as compliance behavior, which are both predictive of the outcome (e.g., HIV infection) and can be predicted by baseline covariates. Strata defined by baseline covariates are essentially baseline subgroups (10). We aimed to estimate the effect of PrEP on HIV acquisition among the strata of participants who would adhere to their designated regimen if assigned to one of the active arms. In blinded, placebo-controlled randomized trials, when the intervention has few adverse effects and is not available to control participants—as is the case with PrEP—adherence behaviors are not expected to differ by randomization arm; thus, the average causal effect of PrEP among participants who would adhere if assigned to active PrEP is akin to the “complier average causal effect” (15). Because the most objective measure of adherence, plasma detection of tenofovir, was applied in only a subset of participants, we first built a model to predict this measure in the remaining participants. Following the approach of Follmann (19), we built a logistic regression model with plasma tenofovir detection >40 ng/mL at the 6-month study visit as the outcome and only baseline characteristics as predictors and then estimated PrEP efficacy among participants with a high probability of adherence. We assumed that adherence behaviors would generally be established by 6 months.

Prediction model

The logistic model used to predict high adherence included the 2 cohorts described above with tenofovir measured in plasma: participants selected completely at random (n = 200) and participants who were HSV-2-negative at enrollment (n = 112). When tenofovir concentration measures were expected at 6 months but were missing, values were imputed as undetectable when pill coverage was known to be 0 and the last measured value was carried forward when pill coverage was not 0. Similarly, because we relied on tenofovir concentrations at 6 months to characterize adherence throughout the study, for persons with low or undetectable tenofovir concentrations due to protocol-defined study medication holds (e.g., because of pregnancy or laboratory abnormalities) who might otherwise be adherent, the last measured value was carried forward. Because tenofovir was a component of both active randomization arms and randomization arm was confirmed to not be associated with tenofovir detection, we built the prediction model by combining data from both the TDF and TDF/FTC arms (n = 297 with existing values and n = 8 with carried-forward values).

All models included HSV-2 status at enrollment and study site as covariates. Additional potential predictors included enrollment characteristics of 1) the HIV-infected partner (CD4-positive T-lymphocyte count, plasma HIV RNA levels, and World Health Organization stage of HIV infection), 2) the uninfected partner (age, sex, years of education, alcohol use, travel time to the clinic, disclosure of serodiscordant status to anyone, source of income, and socioeconomic status as measured by a prior principal components analysis (20)), and 3) the partnership (number of children engendered together, frequency of sex in the prior month, unprotected sex in the prior month, any sex with someone other than the enrolled study partner in the prior month, number of years for which serodiscordance had been known, and duration of cohabitation (years of living together)). Transformations of continuous variables were evaluated individually to identify the best fit based on Akaike's Information Criterion (21). Predictors significant at P < 0.2 in univariate analysis were included in the multivariate model. In the multivariate context, plausible interaction terms were considered and transformations of continuous variables were reevaluated, again using Akaike's Information Criterion. The area under the receiver operating characteristic curve was estimated for the final model using 10-fold cross-validation (22).

The predicted probability of adherence for all participants in the trial was calculated from the coefficients in the final prediction model. We examined the distribution of baseline characteristics across arms within quintiles of the predicted probabilities to assess the exchangeability (i.e., comparability of baseline HIV risks) of participants within strata.

Estimating PrEP efficacy

The efficacy of PrEP was estimated in a Cox regression model including randomization arm, the predicted probability of adherence as a linear term, and the interaction of the 2 variables. Following Follmann (19), the sum of the coefficients for arm and the interaction term would equate to the log hazard of HIV acquisition for participants receiving PrEP versus placebo when the probability of adherence is 100%. The coefficient for arm alone represents the log hazard of HIV acquisition for persons on PrEP versus placebo when adherence is predicted to be zero. We note that 100% and 0% probabilities are extrapolations from the model. Confidence intervals were estimated from 1,000 bootstrapped samples, using stratified sampling (23). We first sampled seroconverters and nonseroconverters separately from the 2 cohorts with tenofovir measures and estimated the prediction model among them; we then sampled seroconverters and nonseroconverters separately from outside the cohorts and calculated the predicted probability of high adherence in the full combined bootstrapped sample. Finally, we estimated PrEP efficacy in the Cox model. The 95% confidence interval was constructed with the 2.5th and 97.5th percentiles of the estimated hazard ratios.

This stratified bootstrap also allowed us to assess the sensitivity of the prediction model to participants selected into the cohorts. If the cohort members were not representative of the full study sample, we would expect classification of adherence (calculated predicted probabilities) to vary widely across bootstrapped samples, potentially resulting in a skewed distribution of effect estimates. We examined whether the distribution of the hazard ratios centered around our point estimate, and we report the median and interquartile range in addition to the constructed confidence interval.

IPCW approach

IPCW can be used in randomized clinical trials with nonadherence or loss to follow-up to estimate the treatment effect that would be observed if all participants remained in the trial and adhered to their assigned treatment (11). In this approach, participants are artificially censored at the first missed visit, when treatment is switched, or when adherence is estimated to be poor. Uncensored participants with characteristics similar to those of censored participants are weighted to represent those who were lost, with the goal of eliminating the selection bias that results from informative censoring.

Using IPCW, we aimed to estimate the effect of PrEP that would be observed if all participants maintained continuously high adherence. In our analysis, participants were censored either at their first missed visit or at the first study month in which pill coverage indicated poor adherence (<80% or >107%). In this approach, we were interested in estimating efficacy with consistent daily use; therefore, participants on study medication hold were also classified as “nonadherent.”

Stabilized inverse-probability-of-censoring (IPC) weights were estimated according to published methods (11, 24, 25) in pooled logistic regression models including time since randomization modeled as a restricted cubic spline with knots at the 5th, 35th, 65th, and 95th percentiles (26). The weight numerator included baseline covariates: study site, age, sex, years of education, any reported symptoms of sexually transmitted infection, the infected partner's CD4 cell count, and randomization arm. The weight denominator included the same baseline covariates and, additionally, time-varying covariates with data reported in the prior month, including the number of sex acts, any unprotected sex, and study medication pill coverage (in the range 80%–107%), as well as any symptoms of sexually transmitted infection in the prior 3 months and the infected partner's last measured CD4 cell count during follow-up. Variables that were evaluated for inclusion in the numerator of the weights but were not found to be associated with censoring (P > 0.2) in models adjusted for study site and visit month included sexual behavior and the infected partner's plasma HIV RNA levels. Time-varying partner initiation of antiretroviral therapy was evaluated for inclusion in the denominator but was not associated with censoring (P > 0.2) in a model that adjusted for all variables selected for the numerator. In contrast to our principal stratification approach, where the number of observations in the prediction model was limited, here we modeled all continuous covariates as finely as possible as restricted cubic splines with 3 or 4 knots (26), with the number of knots selected based on Akaike's Information Criterion. Finally, the protective effect of PrEP with continuously high adherence was estimated in a weighted pooled logistic regression model with robust variance estimation to approximate a Cox model, including all variables in the numerator of the weights as adjustment variables (24, 25).

Per-protocol approach: censoring poor adherence, unweighted

We estimated PrEP efficacy in a Cox model censoring all study visits occurring on and after the first missed visit or poor-pill-coverage visit (as in IPCW, but without weights). The Cox model stratified the results by study site and included the same covariates as those selected for IPC weights.

All analyses were conducted in SAS, version 9.3 (SAS Institute, Inc., Cary, North Carolina).

RESULTS

Among 4,733 HIV-serodiscordant couples enrolled in the Partners PrEP Study and not later determined to have already acquired HIV at the time of enrollment, 38% of HIV-uninfected partners were female, and the median age at enrollment was 33 years (interquartile range (IQR), 28–40); these distributions were similar across trial randomization arms (Table 1). The median duration of cohabitation was 7 years (IQR, 3–14), and the median number of years for which couples had known their serodiscordant status was 0.4 (IQR, 0.1–2.0). Among participants with plasma tenofovir concentrations measured longitudinally, over time 59%–77% had concentrations consistent with daily use, while 80%–91% were highly adherent, as measured by pill counts (Table 2).

Table 1.

Baseline Characteristics of HIV-Serodiscordant Heterosexual Couples (n = 4,733 Couples) in Kenya and Uganda, Partners PrEP Study, 2008–2011

Characteristic	Assigned Treatment
	TDF (n = 1,579 Couples)			TDF/FTC (n = 1,576 Couples)			Placebo (n = 1,578 Couples)
	No. of Persons	%	Median (IQR)	No. of Persons	%	Median (IQR)	No. of Persons	%	Median (IQR)
HIV-uninfected partner
Female sex	595	37.7		566	35.9		619	39.2
Age, years			33 (28–39)			33.5 (28–40)			34 (28–40)
Education, years			7 (4–10)			7 (4–10)			7 (4–10)
Travel time to clinic >2 hours	846	53.6		841	53.4		812	51.5
Income derived principally from farming	732	46.4		715	45.4		691	43.8
No. of sex acts in prior month			4 (2–8)			4 (3–8)			4 (2–8)
Any unprotected sex in prior month	441	27.9		415	26.3		408	25.9
HIV-infected partner
Age, years			32 (26–39)			32 (26–39)			33 (26–39)
CD4-positive T-lymphocyte count, cells/μL			491 (370–661)			497 (380.5–664.5)			498.5 (375–663)
Plasma HIV RNA level, log10 copies/mL			3.9 (3.2–4.5)			3.9 (3.1–4.5)			3.9 (3.2–4.5)
Partnership characteristics
No. of years serodiscordance had been known			0.5 (0.1–2.0)			0.4 (0.1–2.0)			0.5 (0.1–2.0)
Duration of cohabitation, years			7.0 (3.0–14)			7.2 (3.0–14)			7.3 (3.0–14)

Abbreviations: FTC, emtricitabine; HIV, human immunodeficiency virus; IQR, interquartile range; PrEP, preexposure prophylaxis; TDF, tenofovir disoproxil fumarate.

Table 2.

Percentage of Participants Determined to Be Highly Adherent to Their Assigned Treatment (as Measured by Pill Counts and Plasma Tenofovir Concentrations^a), by Study Month, Partners PrEP Study, 2008–2011

Study Month	No. of Persons	% Adherent		Concordance Between Measures, %
		Pill Coverage 80%–107%	Plasma Tenofovir Level >40 ng/mL
1	299	80	77	74
3	301	81	70	73
6	305	84	68	72
12	262	87	65	71
18	188	86	59	65
24	120	91	68	70

^a Among participants with tenofovir concentrations measured in plasma. Participants were from 2 randomly selected cohorts from the study's active arms: 1) persons randomly selected from all participants at enrollment; 2) persons randomly selected from participants negative for herpes simplex virus type 2 at enrollment.

ITT analysis

The ITT results obtained using the first evidence of HIV infection as the outcome were nearly identical to the previously reported primary results from the trial that used seroconversion as the outcome (2). Using first evidence of HIV infection, the efficacy of PrEP versus placebo was 67% for TDF alone (hazard ratio (HR) = 0.33, 95% CI: 0.19, 0.56) and 75% for combined TDF/FTC (HR = 0.25, 95% CI: 0.13, 0.45) (Figure 1).

Figure 1.

Estimates of the efficacy of human immunodeficiency virus preexposure prophylaxis (PrEP) derived from multiple approaches correcting for treatment nonadherence. Data were obtained from the Partners PrEP Study, 2008–2011. CI, confidence interval; FTC, emtricitabine; HR, hazard ratio; IPC, inverse probability of censoring; TDF, tenofovir disproxil fumarate.

Principal stratification

The final logistic prediction model for high adherence using the cohorts with measured tenofovir concentrations included study site and the following characteristics of the uninfected partner at enrollment: HSV-2 status, age (years; continuous), years of education (continuous and quadratic), a principal component for socioeconomic status, travel time to the clinic of >2 hours, primary income derived from farming, an interaction term for travel time and farming income, number of sex acts in the prior month, any unprotected sex in the prior month, and the HIV-infected partner's CD4 cell count (cells/μL; continuous). The area under the receiver operating characteristic curve estimated in cross-validation was 0.65. The median predicted probability of high adherence was 0.74 (IQR, 0.59–0.87; range, 0.02–1.00), which did not differ across randomization arms (data not shown). Across randomization arms, baseline characteristics were comparable within quintiles of the predicted probabilities (Table 3).

Table 3.

Characteristics of Participants Within Quintiles of the Predicted Probability of Treatment Adherence, by Randomization Arm, Partners PrEP Study, 2008–2011

Characteristic and Quintile	Assigned Treatment
	TDF		TDF/FTC		Placebo
	% Within Quintile	Median (IQR)	% Within Quintile	Median (IQR)	% Within Quintile	Median (IQR)
Female sex
5 (highest quintile)	44		44		43	43
4	31		32		40	40
3	40		39		37	37
2	42		32		36	36
1 (lowest quintile)	33		33		42	42
Age, years
5 (highest quintile)		39 (33–46)		39 (33–45)		39 (34–46)
4		35 (30–42)		35 (30–42)		35 (30–41)
3		33 (28–38)		34 (29–40)		34 (28–40)
2		31 (26–36)		32 (28–37)		33 (29–37)
1 (lowest quintile)		29 (25–33)		28 (24–32)		28 (24–33)
Education, years
5 (highest quintile)		2 (0–7)		2 (0–7)		3 (0–7)
4		7 (5–10)		7 (4–10)		6 (4–10)
3		7 (5–11)		7 (5–10)		7 (6–10)
2		8 (6–10)		8 (6–11)		8 (6–11)
1 (lowest quintile)		8 (7–11)		8 (7–10)		8 (7–10)
No. of sex acts in prior month
5 (highest quintile)		3 (2–5)		3 (2–6)		3 (2–6)
4		4 (2–6)		4 (2–7)		4 (2–6)
3		4 (3–8)		3 (2–6)		4 (2–8)
2		4 (3–8)		4 (3–8)		4 (3–10)
1 (lowest quintile)		6 (3–12)		6 (4–12)		6.5 (3–12)
Any unprotected sex in prior month
5 (highest quintile)	18		18		13
4	18		18		25
3	29		19		20
2	29		32		25
1 (lowest quintile)	46		44		46
Duration of cohabitation, years
5 (highest quintile)		12.0 (5.2–20)		10.0 (4.0–18)		11.0 (5.0–19)
4		7.0 (3.0–14)		8.5 (3.0–15)		8.3 (3.3–15)
3		5.7 (2.1–12)		9.2 (3.3–16)		7.3 (3.0–13)
2		7.0 (2.7–13)		7.0 (3.0–12)		6.0 (2.4–13)
1 (lowest quintile)		5.0 (2.0–9.9)		4.2 (2.0–8.4)		5.0 (2.0–10)

Abbreviations: FTC, emtricitabine; IQR, interquartile range; PrEP, preexposure prophylaxis; TDF, tenofovir disproxil fumarate.

The estimated effect of PrEP was further from the null than the ITT estimate when the probability of adherence was high. Among participants with an estimated 100% probability of high adherence, the reduction in the hazard of HIV acquisition was 81% for TDF (HR = 0.19, 95% CI: 0.07, 0.56) and 88% for TDF/FTC (HR = 0.12, 95% CI: 0.03, 0.52) relative to placebo. Among those with a 90% probability of high adherence, the estimated reduction in the hazard of HIV acquisition for TDF was 78% (HR = 0.22, 95% CI: 0.10, 0.54) and 84% for TDF/FTC (HR = 0.16, 95% CI: 0.05, 0.45) relative to placebo. Among those predicted to have poor adherence (an estimated probability of 0), the hazard ratio was 1.14 (95% CI: 0.11, 6.47; P = 0.89) for TDF and 0.85 (95% CI: 0.05, 8.74; P = 0.88) for TDF/FTC, consistent with an expected lack of protective effect when no study medication is consumed.

To address the concern that the cohorts used in the prediction model may not have been representative of the full sample, we examined the distribution of the estimated hazard ratios in the bootstrapped samples. The median hazard ratio for the effect of PrEP at a predicted probability of 100% was 0.22 (IQR, 0.15–0.30) for TDF and 0.16 (IQR, 0.10–0.25) for TDF/FTC.

IPCW analysis

Visits with continuously high adherence (pill coverage 80%–107%) retained in the IPCW analysis accounted for 49% of all visits in the ITT analysis and included 27 of all 82 seroconversions in the trial. The mean IPC weight was 0.99, and the range was 0.60–4.01 (standard deviation, 0.12). The estimated reduction in the hazard of acquiring HIV in a pseudopopulation with continuously high adherence was 82% for TDF (HR = 0.18, 95% CI: 0.06, 0.53) and 85% for TDF/FTC (HR = 0.15, 95% CI: 0.04, 0.52) relative to placebo.

Per-protocol analysis: censoring poor adherence, unweighted

In the per-protocol analysis, the estimated effect of TDF relative to placebo was an 82% reduction in the hazard of acquiring HIV (HR = 0.18, 95% CI: 0.06, 0.53), and the estimated effect of TDF/FTC was 84% (HR = 0.16, 95% CI: 0.05, 0.53).

DISCUSSION

Using multiple rigorous methods to estimate the efficacy of PrEP for HIV prevention when adherence was high, the results from these analyses indicated stronger protection than was estimated by ITT and together suggest that protection is greater than 80% for either TDF alone or TDF/FTC. Both the principal stratification approach and the IPCW approach relied on random assignment to placebo or PrEP and thus aimed to compare highly adherent participants in groups that were comparable in terms of their baseline HIV risk, though in using postrandomization variables, IPCW cannot be viewed as fully maintaining randomization (8, 9). In the principal stratification approach, the adherent population was a baseline subgroup with a high probability of adherence, and in IPCW this was a pseudopopulation with perfect adherence and follow-up. We would expect equivalent results from these approaches when the effect among compliers is the same as the effect among noncompliers had they complied. The per-protocol analysis, which is not inherently structured around exchangeable groups (i.e., groups comparable in terms of their baseline HIV risk), also generally agreed with these approaches, probably because poor adherence in this double-blind placebo-controlled trial did not differ by randomization arm, thus maintaining reasonably exchangeable groups among those who adhered. We expect that any misclassification of actual adherence would have been independent of randomization arm, and we therefore believe that these are attenuated estimates of PrEP efficacy.

These analyses include both time-fixed and time-varying classifications of adherence. While time-varying classification has the potential to more accurately describe the effects of PrEP use over time, the time-fixed classification provides a simple baseline subgroup interpretation and is supported by a prior analysis carried out within the Partners PrEP Study which showed that individuals generally had consistent adherence behavior over time (17). Across approaches, our estimated results assumed that adherence was correctly classified, an assumption that probably was not fully met. Pill counts in the Partners PrEP Study have some inaccuracies (17), and while tenofovir concentration in plasma is a more objective measure of adherence, it reflects only recent use. Although plasma tenofovir concentrations greater than 40 ng/mL are consistent with daily use, this level could be achieved with a single dose taken within the last day (17). Further, because tenofovir measurements were available only for a sample of participants, our ability to predict adherence was limited; the cross-validated area under the receiver operating characteristic curve suggested only a moderate discriminatory ability of the model. The similarity of results across methods suggests that time-varying measures of PrEP use did not substantially improve the classification of daily adherence in comparison with principal stratification.

In the principal stratification approach, a central assumption is that within strata, treatment arms are exchangeable. Although this assumption cannot be confirmed in the data, it is plausible given the comparable distribution of covariates within quintiles of the strata. In this blinded placebo-controlled trial, where placebo participants did not have access to active medication, we assumed only 2 potential compliance behaviors: adherence to the assigned treatment if assigned to active treatment and nonadherence if assigned to active treatment (27). Our assumption that subjects assigned to receive placebo did not have access to active treatment was probably valid, since PrEP was not available in the study communities during the study time period, and while HIV treatment containing TDF could have been accessed by HIV-infected partners, transference of that medication to their HIV-uninfected partner seems unlikely.

In the IPCW approach, a primary assumption is that all common causes of censoring (i.e., adherence) and HIV acquisition are accurately measured. Predictors of adherence are not well understood, and sexual risk behavior, potentially a common cause, is prone to underreporting and thus may not have been accurately captured in the weights. Nonetheless, we do not expect the reasons for censoring (including sexual behavior, reported or not) or the degree of censoring to have differed by randomization arm. Notably, the range of the weights was narrow, probably due to a lack of strong predictors of censoring. As the weights approach 1, the IPCW analysis increasingly resembles the per-protocol analysis, which explains the nearly identical results derived from these 2 approaches.

One other group of investigators has conducted a principal stratification analysis of 2 PrEP trials of TDF/FTC, using an alternate approach within this framework (28). They estimated the effect of PrEP in the adherent subgroup to be approximately 90% protective in both trials (28). Several other approaches, including Bayesian methods, have been applied within the principal stratification framework to other randomized trials with nonadherence (13, 29–32). In our application of this framework, as well as the IPCW and per-protocol methods, we observed wider confidence intervals than those seen in the ITT analysis, since we essentially evaluated fewer events among adherent participants.

In conclusion, our application of multiple methods to estimate the efficacy of PrEP when adherence is high suggests that protection from HIV infection is greater than 80% with either TDF alone or TDF/FTC. The overall agreement across these distinct approaches, each with different assumptions, provides strength for this conclusion. While ITT analysis remains the gold standard for randomized trials, secondary analyses that account for nonadherence, when structured around exchangeable groups, are recommended. In our data, adherence measurement error was probably a greater limitation with the principal stratification approach (due to limited strong predictors of adherence) than with the IPCW or per-protocol approach. In other PrEP trials, investigators have reported poor validity of pill counts (3, 6, 7). When application of multiple methods is not feasible, researchers should consider the limitations of the data when selecting a method. Our results illustrate the contribution that methods derived within the potential-outcomes framework can make to the interpretation of clinical trial data for HIV prevention interventions.