Estimands—What they are and why they are important for pharmacometricians

We provide a brief introduction to estimands and share our perspective on why we think estimands are helpful for the practicing pharmacometrician. The discussion is motivated by the recent release of the International Council for Harmonisation (ICH) E9(R1) guideline on estimands, which describes an aligned framework for planning, conducting, and analyzing clinical trials as well as interpreting sensitivity analyses. We also draw connections to earlier work by Lewis Sheiner.

WHAT IS AN ESTIMAND?

In the early 1990s, Lewis Sheiner put much thought into an appropriate epistemologic approach to clinical sciences in general and clinical drug evaluation in particular. ¹ Among other topics, he drew attention to the properties and shortcomings of intention‐to‐treat analyses and how these may be addressed using model‐based methods. Later, Sheiner and Rubin ² promoted the term “use‐effectiveness” for the causal effect of prescribing a drug and “method‐effectiveness” for the causal effect of taking a drug (see Bernstein ³ for an early use of these terms in the context of contraceptives). Focusing on these two terms, the authors emphasized the importance of the scientific question, the what to estimate (i.e., the “estimand”) versus the analytical strategy (how to estimate the estimand).

These concepts have reemerged with the recent publication of the ICH E9(R1) ⁴ guideline on the choice of estimands in clinical drug development. In general terms, an estimand is a precise description of the quantity one tries to estimate, reflecting a particular clinical question underlying a clinical trial. To that effect, the ICH E9(R1) guideline emphasizes the importance of defining the estimand attributes: population, variable, treatment, and population‐level summary. The framework focuses on causal treatment effects that are particularly important in drug development and approval, which assess the following question: “How does the outcome of the treatment compare with what would have happened to the same subjects under an alternative treatment (i.e., if they had not received the treatment or had received another treatment)?” As we cannot travel back in time and observe the outcome of the same patients under both treatments, clinical trials usually rely on randomization to construct two comparable groups, with the only difference being the treatment that patients receive, which allows the estimation of causal treatment effects at a population level. ⁵ , ⁶

Unfortunately, in real clinical trials, patients may not stay on the initially assigned treatment regimen; intervening events may occur, such as dose adjustment or treatment discontinuation (due to lack of efficacy or adverse event [AE]), rescue medication intake or treatment switches, changes in the background therapy, or death. These events are labeled as “intercurrent events” (IEs) in ICH E9(R1) and occur after treatment initiation and affect either the interpretation or the existence of the measurements associated with the clinical question of interest (see Figure 1). ICH E9(R1) emphasizes the need to address IEs at an estimand level: different strategies to handle IEs change the quantity one tries to estimate. For instance, different ways of addressing nonadherence to the initially assigned treatment changes the estimand and thus the scientific question one is answering, very similar to how “use‐effectiveness” and “method‐effectiveness” answer different clinical questions of interest.

FIGURE 1.

Journeys of four patients illustrating intercurrent events that occur after randomization and affect either the interpretation or the existence of the measurements associated with the primary variable. Patient 1 reduces the dose (e.g., due to an adverse event) and remains in the efficacy follow‐up. Patient 2 also reduces the dose (e.g., due to an adverse event) but withdraws at a later timepoint from further follow‐up. Patient 3 dies while on treatment

HOW DOES THE ICH E9(R1) PROPOSE TO ADDRESS INTERCURRENT EVENTS?

ICH E9(R1) introduces five strategies to address IEs. To illustrate these, consider a trial where the variable of main interest is continuous and assessed at a fixed timepoint at the end of the trial (e.g., change from baseline in glycated hemoglobin at week 52 in type 2 diabetes mellitus). Patients in this trial may experience an AE and because of that may have to reduce the initially assigned dose. This AE and the subsequent change in dose constitute an IE. Any efficacy measurement collected after the occurrence of this IE may be influenced by it and as such may also affect the overall results and the interpretation of the patient's outcome at trial end. Table 1 outlines the five strategies and illustrates their use for this example.

TABLE 1.

The five IE strategies mentioned in the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use E9(R1) guideline illustrated with an example of an IE of dose reduction as a result of an AE

Intercurrent event handling strategy	​ General description of strategy
	Strategy applied to the intercurrent event “dose reduction due to an AE”
Treatment policy strategy	Occurrence of an IE is considered irrelevant in defining the treatment effect of interest: the value for the variable of interest is used regardless of whether the IE occurs.
	The treatment is changed to the treatment policy “Start with initially assigned dose, but allow for dose reduction if required due toan AE.”
Hypothetical strategies	A setting is envisaged in which the IE would not occur: the value of the variable to reflect the clinical question of interest is the value that the variable would have taken in the hypothetical scenario defined.
	One hypothetical scenario of interest could be the treatment effect in a setting where dose adjustment due to an AE would not occur.
Composite variable strategy	An IE is considered in itself to be informative about the patient's outcome and incorporated into the definition of the variable, usually as an unfavorable outcome. The newly defined variable is then a composite of efficacy and the occurrence of the IE.
	Occurrence of an AE leading to dose reduction could be considered a negative outcome. A binary variable could be created that is 1 if a patient has a “change from baseline larger than a cut‐off” and “does not have a dose reduction due to an AE” and 0 otherwise. This modifies the variable to a composite variable in some ways similar to a patient specific utility function.
While on treatment strategy	For this strategy, the response to treatment before the occurrence of the IE is of interest. This strategy hence modifies the variable.
	For patients who have a dose reduction due to an AE a function of the values before that event (e.g., the average) would be used.
Principal stratum strategy	The target population is the subpopulation (“principal stratum”) in which the IE would not occur.
	Here the subpopulation of interest is defined to be those patients who would not experience dose reduction due to an AE regardless of whether they were initially assigned to investigational or control treatment.

Estimands are described in terms of the attribute treatments, population, and variable and the chosen IE strategies will typically be reflected in these estimand attributes. For example, the treatment policy and hypothetical strategies have an impact on the treatment attribute. The composite and while‐on‐treatment strategy modify the variable, while the principal stratum strategy modifies the population.

Abbreviations: AE, adverse event; IE, intercurrent event.

The appropriate choice of an IE strategy (and thus the estimand) depends on the specific IE, the context, and more generally the question of interest. For an actual analysis/estimation of the estimand, further steps need to be taken. For example, additional modeling assumptions need to be made for the analysis that may be varied using suitable sensitivity analyses.

ARE ESTIMANDS AND THE ICH E9(R1) GUIDELINE RELEVANT FOR PHARMACOMETRICIANS?

Understanding estimands should be of interest to anyone performing estimation, including pharmacometricians. Otherwise, estimates derived out of a given estimation process may not concisely answer the question of interest; without prior consideration, this may not be obvious to the analyst him/herself. Although ICH E9(R1) was primarily written with statistical analysis traditions in mind, the principles outlined in ICH E9(R1) apply more broadly and are relevant whenever a treatment effect is to be estimated.

What type of estimand do pharmacokinetic (PK)/pharmacodynamic (PD) analyses typically address?

A pharmacometrician would most typically approach an IE of nonadherence (such as dosing modification or treatment discontinuation) using a standard pharmacometric workflow consisting of a two‐step approach of building dose‐exposure (PK) and exposure‐response (PD) models followed by simulations of PK and PD profiles. Nonadherence to a defined dosing regimen is typically taken into account when constructing the dose‐exposure model by considering the actual dosing, resulting in modified drug concentrations for those patients. The model can then be used to simulate a large number of fully adherent patient PK profiles, and based on those profiles simulate the response profiles for each patient. Under specific conditions, the population average difference in the response between treatment and control then constitutes a causal estimate of the treatment effect. This corresponds in the ICH E9(R1) language to the hypothetical scenario “had all patients adhered to the intended treatment regimen,” which targets method effectiveness. This connection between standard pharmacometric workflows and causal inference techniques has also recently been pointed out elsewhere. ⁷ A similar approach may be also used for other types of IEs such as intake of rescue medication, where a hypothetical strategy could target the treatment effect “had rescue not been available.”

What scientific questions can such hypothetical estimands address?

The decision on which IE strategy or which hypothetical scenario to consider must take into account the scientific question, the needs of stakeholders, and the target audience for the analysis. A hypothetical treatment effect estimand of “had all patients adhered” allows to estimate the efficacy under full adherence, and thus the loss of efficacy due to the actual nonadherence. Together with an understanding for the reasons of nonadherence (e.g., forgetting to take medication, perceived lack of drug efficacy, or toxicities and adverse events) the drug developer may seek to reduce the efficacy loss via development of regimen alternatives using alternative dose amounts, dosing frequencies, administration forms, formulations, or mobile applications with automated reminders. This is a drug developer's perspective. For health authorities or health technology assessment agencies, a hypothetical strategy “had all patients adhered” may only be of primary interest in situations where there is already a mechanism in place that could make patients more likely to adhere in medical practice. For example, for IEs such as dose adjustments due to AEs, AEs leading to treatment discontinuation, or death, the relevance and practical estimability of hypothetical estimands such as “if dose were not adjusted due to AEs,” “if treatment were continued irrespective of AEs occurring,” or “had patients not died” need to be carefully considered in situations where the IE cannot be avoided in medical practice.

What is the role of assumptions and sensitivity analyses?

The two‐step method discussed previously yields an estimator for the hypothetical estimand outlined previously under specific modeling assumptions, for example, related to the relationship between the process that leads to the IE and PD, conditional on the PK and other covariates used in the models. The ICH E9(R1) encourages transparency about the plausibility of these assumptions and the use of sensitivity analyses. These sensitivity analyses should target the same estimand but use different modeling assumptions.

Should and can pharmacometric analyses address estimands other than hypothetical estimands?

The choice of estimand (and selection of IE strategy) depends on the scientific question and on the target audience. We think the pharmacometric community should be open toward strategies beyond the hypothetical. For example, pharmacometric models that implement the pharmacological understanding of the system are particularly adequate to evaluate the effect of a nontested drug regimen. A treatment policy strategy estimand may be required by the health authorities when such a new regimen is under consideration for approval. With this purpose, the aforementioned two‐step approach could be extended by modeling the effect of previous drug concentrations on IE incidence (e.g., the AE incidence that leads to a lack of adherence), for related ideas, see Hu and Sale ⁸ , ⁹ and Sheiner et al ⁸ , ⁹ ). The simulation step would involve simulating drug concentration profiles based on the intended drug regimen and the incidence of AE from those concentration profiles, then adapting the dose administration and the concentration profiles upon incidence of simulated AEs, and finally simulating the PD response based on the possibly adapted concentration profiles. Other IE strategies can be similarly implemented.

CONCLUSION

Any pharmacometric analysis implicitly targets an estimand, and we should be explicit about it. Vice versa, many estimands may be best addressed with (pharmacometric) modeling and simulation techniques. To that end, the ICH E9(R1) introduces a common language that can be used in collaboration with statisticians, clinicians, and other drug development stakeholders. This facilitates discussion and alignment on a commonly understood estimand and IE handling strategies first, and only then, corresponding analyses and assumptions. Although the ICH E9(R1) focuses on the causal effects of treatments, the same framework may also be used to evaluate estimands not related to treatment comparisons. The estimand framework can be understood as an attempt to restore “the intellectual primacy to the questions we ask, not the methods by which we answer them.” ¹ It is our firm belief that “opening the box” on the what questions in drug development through the estimand framework will ultimately lead to wider use of model‐informed drug development methodologies ¹⁰ because they are often the most effective or the only way to address key questions such as method effectiveness.

CONFLICT OF INTEREST

The authors declared no competing interests for this work.

AUTHOR CONTRIBUTIONS

B.B., F.B., T.D., O.S., and M.L. wrote the manuscript; All authors designed the research and performed the research.