Mosaic effectiveness: measuring the impact of novel PrEP methods

Summary:

A number of ongoing trials seek to evaluate long-acting PrEP agents in by demonstrating it is non-inferior to daily oral TDF/FTC. A trial comparing oral PrEP to a new method compares effectiveness in a setting where only one or the other is provided; however, a new product will likely be delivered in a context where oral PrEP is also available. Its impact is best measured by its potential contribution in a context that also includes oral TDF/FTC as an option. Will it extend the reach of effective biomedical HIV prevention?

We offer an alternative standard for long-acting products—a measure of the effectiveness of the new product in addition to oral TDF/FTC as compared to oral TDF/FTC alone. We term this quantity “mosaic effectiveness”—a compelling measure of impact. We illustrate scenarios where a novel product can fail the to show non-inferiority but demonstrates substantial mosaic effectiveness, thus implying the public health utility of the novel product even if it is less effective than oral PrEP. Regulatory standards should take into account mosaic effectiveness not just comparative effectiveness. Thus, standards for products that combine rigor with public health relevance can accelerate progress against the HIV epidemic.

1·0. Pre-Exposure Prophylaxis and Its Promise

Oral co-formulated tenofovir disoproxil fumarate and emtricitabine (TDF/FTC) is highly efficacious for HIV pre-exposure prophylaxis (PrEP) with an excellent safety profile.¹ The scale-up to translate PrEP efficacy to public health impact (effectiveness) has proven to be challenging. In the United States, which leads the world in the number of PrEP initiations,² recent data revealed that only 18% of those with indications for PrEP have actually received TDF/FTC.³ Moreover, in key populations within the United States, the uptake is lower. African Americans make up 45% of those with indications for PrEP⁴ but only 1% of these African Americans, who might benefit from PrEP, have been started PrEP.⁵

Most individuals with substantial HIV risk will either decline an offer of PrEP, indicate interest without initiation, or discontinue TDF/FTC shortly after starting.^6,7 Surveys of potential PrEP users suggest that many do not find a daily oral medication acceptable and would prefer an alternative PrEP delivery method (e.g., an implant, a microbicide, or an injectable).^8,9 This implies that the use of, and delivery method for, PrEP is a preference-sensitive decision. A variety of non-oral PrEP product are in the development pipeline. Public health implementers and communities hope that a broader variety of options will engage a greater percentage of the at-risk population in effective biomedical HIV prevention, and thereby curb the HIV epidemic, especially in populations with the greatest need. In this article, we discuss standards for the development of antiretroviral-based non-daily and/or non-oral PrEP products, in particular, long-acting injections and implants. We abbreviate these as “LA PrEP” products. Products such as vaccines and neutralizing antibodies have other considerations and, hence, are not our focus here.

At present, the roadmap for the development of new PrEP products involves demonstrating non-inferiority to oral TDF/FTC in randomised trials. Non-inferiority is a natural criterion for proving that a new product is an adequate substitute for TDF/FTC. However, the public health impact of PrEP LAs will stem from retaining and engaging a new population, non-users of TDF/FTC, in another effective prevention method and, thus, increase the reach of biomedical HIV prevention. Therefore, we seek an alternative product for TDF/FTC non-users rather than a replacement product for existing TDF/FTC users. This paper proposes a novel criterion, mosaic effectiveness, as an alternative index of impact because it is more aligned with this objective. Heuristically, it measures the reductions in HIV infections between a context in which TDF/FTC is the only available PrEP product and one in which TDF/FTC and the PrEP LA are both available as PrEP options. To show this kind of impact, requires that the PrEP LA have a new delivery modality (e.g., injectable, ring, douche) to be likely to engage a new set of users for which TDF/FTC is not acceptable.

2·0. Trials for PrEP LA Product: Non-inferiority

Three phase III clinical trials have been launched to evaluate next generation agents for antiretroviral-based pre-exposure prophylaxis. HPTN083¹⁰ and HPTN084¹¹ evaluate cabotegravir long-acting injections, and the DISCOVER¹² study evaluates co-formulated tenofovir alafenamide with emtricitabine (TAF/FTC).

The HPTN083 and DISCOVER trials are designed as non-inferiority (NI) studies with the hypothesis that a PrEP LA can refute loss of greater than 50% of the effectiveness of TDF/FTC at the 95% confidence level^13,14. This standard is typical in NI studies and conceptually compelling if the new product is designed as a replacement for the control treatment.

However, Snappin and Jiang¹⁵ render an insightful critique of the preservation of effect criteria. They demonstrate that effect preservation translates into a higher standard for the new product than the initial criterion (better than placebo) that placebo-controlled trials typically apply to the control product (e.g., TDF/FTC). Fig 1 compares the planned years of follow-up for HPTN083 and DISCOVER with the achieved follow-up in the primary analysis of iPrEx study. The studies enrol subjects from similar populations with a similar objective but the planned follow-up, which tends to track the cost and resource requirements, are 2–3-fold larger for DISCOVER and HPTN083 than for iPrEx. The preservation of effect criterion is a high-cost resource-intensive standard for a PrEP LA to reach.

Figure 1. Person Years of Follow-Up in Three HIV PrEP Trials.

Note:

iPrEx:	TDF/FTC v. placebo	superiority study
HPTN083:	TDF/FTC v. Injectable cabotegravir	non-inferiority study
DISCOVER:	TDF/FTC v. TAF/FTC	non-inferiority study

TDF/FTC: tenofovir disoproxil fumarate and emtricitabine

TAF/FTC: tenofovir alafenamide with emtricitabine

We contend that active controlled trial designs will not address the public health impact of a new PrEP product. Specifically, a comparative effectiveness trial with a non-inferiority design demonstrates the effectiveness of either the oral PrEP alone or the novel agent alone. The ultimate impact of the new product will depend, not on its effect alone, but on its ability to add to a milieu of biomedical preventions by engaging at risk populations that do not use or desire oral TDF/FTC. Non-inferiority does not address this and is poorly aligned to reveal the value of novel products. It would be a natural criterion for a similar, presumably replacement, product. Daily oral TAF/FTC has the same regimen and delivery as TDF/FTC, is not a PrEP LA, and should be required to meet conventional non-inferiority through demonstrating preservation of effectiveness.

Consider a randomised, active controlled trial of an PrEP LA with a TDF/FTC control group. Using standard methods,^13,14 a NI margin of about 1·23 can be justified¹⁰, leading to a sample size of approximately 172 incident HIV seroconversions. The data in Table 1 mimics the potential results of a trial with that sample size. Note, the upper limit of the 95% confidence interval for the rate ratio of (1·53) falls outside the NI margin and thus the PrEP LA fails to demonstrate non-inferiority. Therefore, the product would fail its primary objective and likely not be licensed for use. Accordingly, should the PrEP LA be abandoned? We consider alternative perspectives on the evaluation of this question.

Table 1:

Hypothetical Results from an Active Controlled Trial of a Novel Prevention Method

Arm	N	Person Years	HIV+	Rate Per (100 PY***)
TDF/FTC**	2500	5000	81	1.62
PrEP LA*	2500	5000	91	1.82

3·0. Standards for a Preference Sensitive Decision

We imagine that a candidate agent would be evaluated in a double-blind/double dummy randomised active controlled trial of TDF/FTC v. an PrEP LA. We propose the following standards for a candidate agent:

1. Safety: The product is very safe and well-tolerated.

2. Efficacy: The PrEP LA demonstrates high efficacy/effectiveness when used as directed. An ideal PrEP LA would have an efficacy of 90% or higher when used as directed (like TDF/FTC). The efficacy estimation should be done using principled causal methods^16,17 based on the trial data when compared to the inferred background HIV incidence rate. This efficacy estimate should then be used to rule out products with low to moderate efficacy.

3. Effectiveness: Evidence of a public health impact after the introduction of the PrEP LA within the set of available prevention options. This will require evidence of effective use of the PrEP LA in a population that either declines to use or does not effectively use TDF/FTC.

If the PrEP LA is safe, criterion (i) is satisfied. The criterion (ii) ensures that we are confident that the product works if used properly (efficacy) and would use as-treated analysis of trial data to infer efficacy. Criterion (iii) is a novel requirement, which maps the public health impact of the deployment of the PrEP LA in a context in which TDF/FTC is already an existing option. It could be defined as a comparison of the HIV incidence in a trial population where TDF/FTC is the only PrEP option with the HIV incidence in the same trial population where users are offered both TDF/FTC and the PrEP LA as PrEP options. We term “control effectiveness” as the effectiveness (relative to background HIV incidence) in a population provided with TDF/FTC PrEP. We define “choice effectiveness” as the effectiveness (relative to background incidence) when the user and/or provider can choose between the TDF/FTC or the PrEP LA. We term the comparison of control and choice effectiveness conditions as “mosaic effectiveness”. The PrEP LA will demonstrate mosaic effectiveness only members of the population who are TDF/FTC non-users at risk for HIV adopt the effective use of the PrEP LA. This would be unlikely to occur if the new product is similar in its delivery system – for instance, a daily oral tablet (e.g., TAF/FTC).

Choice effectiveness, and thus mosaic effectiveness, are not directly identifiable from the effectiveness observed in an active controlled trial randomising between TDF/FTC vs. a PrEP LA. Both mosaic and choice effectiveness measures depend intimately on the acceptability of and adherence to the PrEP LA among TDF/FTC non-users. We illustrate this by revisiting the data in Table 1.

4·0. Illustrating Mosaic Effectiveness

Table 2 further elaborates on the scenario that underlies the mock trial data in Table 1. Let 40% of the population adhere to TDF/FTC on a daily basis, 30% adhere to the PrEP LA as directed, and both TDF/FTC and the PrEP LA reduce HIV infection risk by 90% when used as directed (efficacy). Here, we treat adherence to both products as binary and fixed for an individual.

Table 2:

Scenario Underlying the Results of the Trial of a Novel PrEP agent

	PrEP LA*	TDF/FTC**
Person Years	5,000	5,000
HIV Rate per 100 PY*** (background)	2.5	2.5
Efficacy	90%	90%
Proportion adherent	30%	40%
Effectiveness	27%	36%
Observed HIV+	91	81
Averted HIV+	34	44

^*PrEP LA: non-daily and/or non-oral pre-exposure prophylaxis agent

^**TDF/FTC: daily oral tenofovir disoproxil fumarate and emtricitabine

^***Person years of follow-up

The impact of implementation is not established using the relative effectiveness of the two strategies in competition but rather using the effectiveness of the two combined as options—since the rationale is that individuals and providers (and other stakeholders) will likely be choosing between them. Mosaic effectiveness depends on the relative adherence to PrEP LA and TDF/FTC in the population—an idea intimately connected with choice and preference.

Table 3 considers the 5,000-person cohort in the scenario illustrated in Table 2. We expect approximately 125 HIV infections in the absence of any PrEP product. Depending on the individual’s assignment, some of these infections will be observed (because of non-adherence or imperfect efficacy) and others will be prevented (averted) by the PrEP method. Within these 125, consider four latent strata¹⁸, defined by whether their infections will be prevented by the use of TDF/FTC and/or the PrEP LA, if offered: (1: flexible stratum)—they will have their infections prevented by either method offered to them; (2: pill-preferred stratum)—these will have their infections prevented if the TDF/FTC is provided but will become infected if the PrEP LA is offered. Individuals in this stratum do not benefit from the addition of a PrEP LA in the clinical milieu, rather these individuals will be harmed by the addition of a PrEP LA. The stratum (3: adopters) of PrEP LA adopters will become infected if only TDF/FTC was offered but their infections are averted due to the availability of the PrEP LA. This is the population that will allow the reach of PrEP to be increased and one that the PrEP LA is designed to engage. The final stratum, (4: unreached stratum), will become infected irrespective of whether they are offered TDF/FTC or the PrEP LA, and, hence, they remain in need of additional HIV prevention options.

Table 3:

Number of Infections in the Hypothetical Trial Distributed by the Four Strata of Infections Defined by Whether HIV Infections Would Occur or Not Between the TDF/FTC Only Stratum or Novel PrEP Agent Only

	TDF/FTC** Only
PrEP LA* Only	Averted	Observed
Averted	Δ1	Δ3	34
Observed	Δ2	Δ4	91
	44	81	125

Distribution of the 125 individuals who would become HIV+ in the absence of any PrEP in four principal strata. The numbers in each cell are latent.

1. Flexible stratum – those with infections prevented by offer of either TDF/FTC or PrEP LA, number of infections = Δ₁

2. Pill-preferred stratum – those with infections prevented if offered TDF/FTC but not prevented if offered the PrEP LA, number of infections = Δ₂

3. PrEP LA adopters stratum – those infected if offered TDF/FTC but prevented an offer of the PrEP LA, number of infections = Δ₃

4. Unreached stratum – infected if provided TDF/FTC or PrEP LA. number of infections = Δ₄

The shaded cells indicate strata in which infections could be prevented by the offer of the PrEP LA and/or TDF/FTC. The numbers in these cells are latent.

^*PrEP LA: non-daily and/or non-oral pre-exposure prophylaxis agent

^**TDF/FTC: daily oral tenofovir disoproxil fumarate and emtricitabine

Table 3 breaks down the anticipated number of HIV infections within each of the strata. The number of HIV infections by treatment arm are set using the values in Table 1. The shaded cells indicate HIV infections that are avertable by offering one or the other PrEP method. The (control) effectiveness of the delivery of TDF/FTC alone is defined as (Δ₁+Δ₂)/125 = 44/125 = 35%. The choice effectiveness is defined as (Δ₁+Δ₂+Δ₃)/125, which depends on Δ₃—where, Δ₃ the number of infections that would be preventable by offering PrEP LA to the “PrEP LA adopters” stratum who would not use TDF/FTC. The number of these infections cannot be estimated although some constraints (e.g., Δ_{1 +} Δ₃ = 34) are apparent from Table 3. They depend on scenarios of product preference. Let us consider two scenarios.

4·1. A high efficacy/low effectiveness scenario: redundant preferences

Suppose all of those who are adherent to the PrEP LA are also adherent to TDF/FTC. From Table 2, the population then sub-stratifies into the following: 30% PrEP LA and TDF/FTC users (will adopt/adhere to either), 10% TDF/FTC users but PrEP LA non-users, and 60% are non-users of both PrEP LA and TDF/FTC. None are adherent to PrEP LA but not to TDF/FTC. This is evident from the regions where infections have been averted using TDF/FTC [Fig 2(a) (in blue)] and using the PrEP LA [Fig 2(b) (in red)], which overlap considerably. This yields the results as shown in Fig 2(c), where the “PrEP LA adopters” have zero infections (Δ₃ =0), and the size of the region of observed infections (in black) is not reduced between Fig 2(a) and Fig 2(c). Hence, the control effectiveness (of TDF/FTC alone) and choice effectiveness (of offering a choice between TDF/FTC and PrEP LA) are both 35% with 95% CI (14% to 52%). Mosaic effectiveness can be defined as the difference or the ratio between the choice and control effectiveness estimates. The difference in effectiveness would be 0·00 with, 95% CI (−0·20 to +0·20) and, p=1·00. The ratio of the effectiveness is equivalent to the averted infections ratio (AIR), a measure previously proposed for active controlled trials;¹⁹ its value here is 1·00 with 95% CI (0·57 to 1·76).

Figure 2. Population+ of 125 Who Would Seroconvert without PrEP. (a) TDF/FTC* Alone (b) PrEP LA** Alone (c) TDF/FTC and PrEP LA Offered as Choice: Redundant Scenario.

Note:

Panel 2 (a) Blue: Infections averted by TDF/FTC

Panel 2 (b) Red: Infections averted by PrEP LA

Latent Strata in Panel 2 (c)

Purple: Flexible (infections prevented by use of either TDF/FTC or PrEP LA), n=33

Blue: Pill-Preferred (infections averted by TDF/FTC but not by the PrEP LA), n=11

Red: PrEP LA Adopters (infected if offered TDF/FTC but averted by the PrEP LA), n=0

Black: Unreached (infected if provided TDF/FTC or PrEP LA), n=81

⁺ In this Figure, the same 125 individuals are displayed under three scenarios. Figures in the same position across the graphs identify the same individual.

*TDF/FTC: co-formulated tenofovir disoproxil fumarate and emtricitabine

**PrEP LA: non-daily and/or non-oral pre-exposure prophylaxis agent

In this setting, all those who would use the PrEP LA would use TDF/FTC anyway; hence, any infections that are averted due to individuals choosing the PrEP LA would be averted by offering TDF/FTC [indicated in purple in Fig 2(c)]. Hence, PrEP LA does not play a role in preventing infections against a background of TDF/FTC, and mosaic effectiveness is null. A different preference configuration can yield very different results.

4·2. A high efficacy/high effectiveness scenario: synergy of choices

Suppose, instead, that the PrEP LA and TDF/FTC represent distinct choices and that individuals strongly prefer one and would not use the other. The population would then stratify to: 0% adherent to both the PrEP LA and/or TDF/FTC, 40% TDF/FTC users but PrEP LA non-users, 30% PrEP LA users but TDF/FTC non-users, and 30% non-users of both PrEP LA and TDF/FTC. These strata totals are also consistent with Tables 1 and 2. Fig 3(a) and 3(b) show that the regions of averted infection for TDF/FTC (in blue) and the PrEP LA (in red) are non-overlapping. Hence, in Fig 3(c), there are 34 HIV infections among the adopters (in red, Δ₃ = 34) – i.e., infections that could be averted by the delivery of PrEP LA, but not by delivery of TDF/FTC to individuals in this stratum.

Figure 3. Population+ of 125 Who Would Seroconvert without PrEP. (a) TDF/FTC* Alone (b) PrEP LA** Alone (c) TDF/FTC and PrEP LA Offered as a Choice: Synergistic Scenario.

Note:

Panel 3 (a) Blue: Infections averted by TDF/FTC

Panel 3 (b) Red: Infections averted by PrEP LA

Latent Strata in Panel 3 (c)

Purple: Flexible (infections prevented by use of either TDF/FTC or PrEP LA), n=0

Blue: Pill-Preferred (infections averted by TDF/FTC but not by the PrEP LA), n=44

Red: PrEP LA Adopters (infected if offered TDF/FTC but averted by the PrEP LA), n=34

Black: Unreached (infected if provided TDF/FTC or PrEP LA), n=47

⁺ In this Figure, the same 125 individuals are displayed under three scenarios. Figures in the same position across the graphs identify the same individual.

*TDF/FTC: co-formulated tenofovir disoproxil fumarate and emtricitabine

**PrEP LA: non-daily and/or non-oral pre-exposure prophylaxis agent

Here the control effectiveness remains 36%, whereas the choice effectiveness increases to (44+34)/125 = 62%, due to the availability of the PrEP LA. There is higher choice effectiveness in this setting because the PrEP LA will be used by 50% = (750/1500) of those would not use TDF/FTC. The additive mosaic effectiveness comparison would be (44+34)/125 – 44/125 = 0·27, with 95% CI (+0·09 to +0·45) and p = 0·003. Thus, 27% of all background infections in the population would be prevented solely by the introduction of the PrEP LA as an option. The averted infections ratio would be ((44+34)/125)/(44/125)) = 1·77, with 95% CI (1·09 to 2·89). This suggests that the introduction of the choice of using PrEP LA into the synergy scenario will increase the number of averted infections by 1·77 times. It is also possible to define the additive rate by standardizing to person years ((44+34)-44)/5,000 = 0·7 HIV infections avoided per 100 person years. Such a measure may be helpful in estimating the absolute number of infections prevented by incorporating the PrEP LA.

The head-to-head analysis in Table 1 suggests that the PrEP LA is a marginal product, presumably with no public health benefit. However, the mosaic framework addresses the question of whether introduction of the PrEP LA as an option prevents infections against a background of TDF/FTC alone (or any other existing standard of care) and that the answer depends on PrEP LA use in TDF/FTC non-users. The example illustrates that mosaic effectiveness is demonstrated when the PrEP LA expands the pool of PrEP users by the adoption of the PrEP LA among TDF/FTC non-users. For instance, a new daily oral tablet (e.g., TAF/FTC) for PrEP would not be expected to show mosaic effectiveness.

5·0. Discussion

A new PrEP agent should be safe, efficacious, and effective. Efficacy is critical because motivating users will require messaging that conveys that the new PrEP product is highly protective if taken as directed. Even if a novel product is proven effective, it will be difficult for any normative agency to license the product without the confidence that it provides very high protection if used as directed. Confounding remains a risk in estimating the efficacy, but this is considerably reduced if principled methods^16,17 are combined with a thorough sensitivity analysis. Any evaluation of a new prevention product requires the complimentary perspective of effectivenss, demonstrated by intent to treat analysis, and efficacy, inferred though adherence analyses. Our novel criterion replaces proof of effect preservation tests. We believe that effect preservation as a standard is misaligned with the objectives of developing novel PrEP products.

Successful prevention of HIV in a population with individuals at risk requires matching individuals with PrEP technologies that fit their preferences, needs, desires and, therefore, methods that they will use. Active controlled trials examine relative overall effectiveness but make no account of choice or preference and they may attract populations which are amenable to both of the products being compared. We have numerically illustrated a case where a product may fail the non-inferiority test but may have a strong public health impact. Adopting the new criteria will increase our power for detecting products that fit a similar profile. The increase in power can translate into more efficient trials. Here, we have focused on trials in HIV prevention; however, this framework is relevant in trials that compare any technology with preference sensitivity in adoption or adherence.

Research about desires, preference, fit, and, therefore, use, require new kinds of data on the acceptability and/or suitability of the PrEP LA among individuals who won’t initiate or sustain the use of TDF/FTC. It may be possible to demonstrate criteria (i)-(iii) in a single trial but (iii) may be best evaluated in a study which focuses on user preferences. Fortunately, study designs exist that are able to obtain this information. These are typically smaller, focused studies conducted outside of pivotal randomised trials. Qualitative research using inductive methods can bring the complexities and nuances of preference into clear focus and direct efforts towards the development of such medications. Discrete choice experiments are widely used in marketing and have been used to examine PrEP method preference^20,21. This approach offers the respondent a series of comparisons between two goods or services in which the attributes are repeatedly varied. By indicating their desired product iteratively, the respondent reveals a quantifiable metric for the strength of their preferences. Quantifying desire can help understand the kinds of trade-offs patients are willing to make to get what they want. Other approaches can be mimicking the TRIO study²², which had HIV- participants try a variety of delivery systems (delivering placebo rather than antiretrovirals) and asked them to rate their preferences and experiences. To date, acceptability and preference are sometimes considered to be problems to defer because they are distal in the translational pipeline (i.e., in the domain of implementation science) but we feel these considerations should be incorporated into the earliest phases of product development and phase III trial design.

Mosaic effectiveness requires counterfactual estimation of the HIV background incidence in the trial population and also the estimation of the HIV incidence when participants are matched to the product (TDF/FTC vs. PrEP LA) to which they would be adherent. Estimation approaches could use the transportability framework²³ on the markers of product use. This is a framework similar to one that has been proposed for HIV vaccine bridging studies.²⁴ Formal theoretical development of this framework is an area of open research. In our illustrations, we also treat adherence as being static. But methods must be developed for handling time-varying adherences. This will be particularly difficult for “on-demand” products, where it may be difficult to document product use at the time of HIV exposure. The statistical details for incorporating preference data to estimate the counterfactual scenario where participants are matched with a PrEP product are a work in progress. In addition, the example considered a scenario in which a perfect match could be made between an individual and the product they would adhere to. This preference may be unknown at the time of PrEP initiation; therefore, statistical methods should elucidate a choice process that will be guided by pre-PrEP characteristics and the attitudes of participants.

Impact of new products will depend on engaging and sustaining PrEP users and motivating health systems to offer and promote them. TDF/FTC PrEP efficacy trials vividly demonstrated this when the trials were reported and the gap between stated adherence and documented adherence was apparent²⁵. Thus, we advocate greater emphasis on the study of product preference and choice (on the part of users and other stakeholders), particularly revealed preferences, and the incorporation of this information into the interpretation of trials of next generation PrEP agents.