Incorporating Estimands into Clinical Trial Statistical Analysis Plans

Abstract

Background:

International Council for Harmonisation (ICH) E9 Statistical Principles for Clinical Trials was developed as a consensus guidance document to encourage worldwide harmonization of the principles of statistical methodology in clinical trials. Addendum E9 (R1) clarified and extended ICH E9 with a focus on estimands and sensitivity analyses. Since the release of E9 (R1), clinical trial protocols have included estimands, but there is variation in how they are presented. Statistical Analysis Plans (SAPs) are increasingly becoming publicly available (e.g. posting on ClinicalTrials.gov) and present an opportunity to link estimands with planned analyses to present the alignment of trial objectives, design, conduct and analysis.

Methods:

A table format was used to create a template for inclusion in SAPs that satisfies ICH E9 (R1) guidance to align statistical analysis to the estimand. The template provides a consistent structure for presentation of estimands and the associated analysis, and is applicable to a wide range of trial designs. We illustrate use of the template with a hypothetical clinical trial in HIV-1.

Results:

The estimand-to-analysis table template starts with the study objective describing the clinical question of interest as written in the trial protocol. The remainder of the table describes each attribute of the estimand (treatment, target population, variable, intercurrent events, and population-level summary) in the left column (ESTIMAND), while the right column describes how each attribute will be handled using the data collected in the clinical trial (ANALYSIS). The template was applied to a hypothetical, early phase single-arm trial, modeled after a pediatric trial in HIV, where the objective was to determine the safety of a new antiretroviral drug as part of a combination antiretroviral treatment regimen in the pediatric population. Three intercurrent events were illustrated in the table: death, premature treatment discontinuation before 24 weeks, and pregnancy. An estimand-to-analysis table from a grant application that addresses the primary objective of a placebo-controlled randomized trial is also presented to demonstrate an alternative usage.

Conclusion:

We found the template to be useful in study design, providing a snapshot of the objective, target population, potential intercurrent events, analysis plan, and considerations for missing data in one place and facilitating discussion among stakeholders. The proposed standardized presentation of estimand attributes and analysis considerations in SAPs will provide guidance to SAP authors and consistency across studies to facilitate reviews.

Background

International Council for Harmonisation (ICH) E9 Statistical Principles for Clinical Trials was developed as a consensus guidance document to encourage worldwide harmonization of the principles of statistical methodology in clinical trials.¹ It has been adopted by regulatory bodies around the world.² In November 2019, E9 (R1) clarified and extended ICH E9 with a focus on estimands and sensitivity analyses.³ The addendum aligns and strengthens the formulation of clinical trial objectives, design, conduct, analysis, and interpretation of the treatment effects addressed in a clinical trial. Recommendations include defining estimands, which are precise descriptions of the treatment effect that address the clinical question posed by the trial objectives. Since the release of E9 (R1), estimands have been discussed in both statistical and clinical literature.^4–11 Clinical trial protocols increasingly include estimands,^12,13 but there is variation in how they are presented.

Planned analyses for a clinical trial are typically outlined briefly in the protocol and described in more detail in a separate Statistical Analysis Plan (SAP). A guideline for clinical trial SAPs states that “[k]ey initiatives that may influence SAP content include the addendum to ICH E9 on estimands and sensitivity analyses”.¹⁴ SAPs are increasingly becoming publicly available documents. For example, the National Institutes of Health (NIH) Final Rule for Clinical Trials Registration and Results Information Submission now requires posting of SAPs in ClinicalTrials.gov in addition to study results.¹⁵ SAPs provide an opportunity to link estimands with more detailed planned analyses.

We present a template for inclusion in SAPs that satisfies ICH E9 (R1) guidance that “[t]he statistical analysis of clinical trial data should be aligned to the estimand.” The template provides a consistent structure for presentation of estimands and decisions on the associated analysis applicable to a wide range of trial designs. While there are a number of publications that address the contents of estimands, the purpose of our paper is to share a simple table that is useful for inclusion in SAPs. Most publications about estimands have been in the confirmatory setting of Phase III randomized trials, and only about one third of Phase II trials move on to the next phase.¹⁶ Therefore, we illustrate use of the template with a hypothetical example addressing a safety objective in a single-arm, early-phase trial. We also present an estimand-to-analysis table that was recently included in an NIH grant proposal for a placebo-controlled, confirmatory clinical trial to demonstrate an alternate application.

Methods

A table format was used to create a template to present estimand attributes and corresponding analysis components.

Results

Estimand-to-analysis table

The table template (Table 1) starts with the study objective describing the clinical question of interest as written in the trial protocol. The next section states the estimand addressing the objective. The remainder of the table describes each attribute of the estimand (treatment, target population, variable, intercurrent events, and population-level summary) in the left column (ESTIMAND), while the right column describes how each attribute will be handled using the data collected in the clinical trial (ANALYSIS). The ESTIMAND side does not include details that are only defined in the context of the trial. For example, the ESTIMAND side refers to people or individuals, not study participants, and “no treatment” rather than placebo. Information in the ANALYSIS column should provide sufficient detail for programmers and statisticians to identify which participants will be included in the analysis set, how their data will be handled, and which statistical methods will be used in the analysis. Sensitivity analyses associated with estimation can be described in the ANALYSIS column and more fully in other sections of the SAP, as needed. Supplementary analyses address different estimands, so they would lead to separate tables.

Table 1.

Estimand-to-Analysis Table Template

Objective: Study objective (clinical question of interest) as written in the trial protocol
Estimand: Succinctly describes the estimand attributes outlined in more detail below
Treatment: Treatments or interventions of interest
ESTIMAND	ANALYSIS
Target population	Analysis set
Population targeted by the clinical question of interest	Participants within the clinical trial contributing to estimation of the population-level summary measure b
Variable	Outcome measure
Measure obtained for each individual required to address the clinical question of interest	How the variable will be determined for analysis
Handling of intercurrent events	Handling of missing data
Intercurrent Event 1 (Strategya) Intercurrent Event 2 (Strategya) (Repeat for each intercurrent event)	Imputation approach for missing data and underlying assumptions Sensitivity analyses: Different ways of handling missing data
Population-level summary measure	Analysis approach
Summary measure for the variable that serves as a basis for treatment effect, without the treatment or for comparison between treatments	How the statistic (estimator) will be calculated Sensitivity analyses: Different assumptions/methods for calculating the estimator

^aStrategies defined in E9 (R1) include Treatment Policy, While on Treatment, Composite, Principal Stratum, and Hypothetical

^bIf there are multiple analysis groups (e.g. comparison groups in multi-arm trials), the groups can be defined here.

Target population (ESTIMAND) and analysis set (ANALYSIS).

The target population is the population targeted by the clinical question of interest. In some early phase trials, the target population may not be the same as the group in which the drug or intervention under study will ultimately be used. For example, new treatments and pharmacologic studies to identify drug concentrations or drug-drug interactions are often conducted in healthy volunteers. Within the clinical trial, the analysis set provides additional details defining which enrollees should be included, as not all participants may contribute to the analysis. For studies complying with Clinical Data Interchange Standards Consortium standards, these analysis set details can guide assignment of population analysis flags in the ADaM Subject Level Analysis Dataset.¹⁷

Variable (ESTIMAND) and outcome measure (ANALYSIS).

The variable is what is measured on each individual in the target population. While the variable may be defined for an ideal setting (e.g. laboratory measurement upon treatment completion), the outcome measure collected in a clinical trial is constricted by the conduct of the trial (e.g. laboratory measurements collected at a study visit within a time window after treatment completion). Additional details may be needed in the outcome measure description for the programmer or statistician to determine when and how that measurement will be identified for each participant within the structured visit schedule of the clinical trial. These details are also required when results are reported to ClinicalTrials.gov.

Handling of intercurrent events (ESTIMAND).

Intercurrent events are identified during study design. They are events potentially occurring after an individual starts treatment that, if they occur, could affect either the interpretation or the existence of the variable. How each intercurrent event is handled is described in this section. If using one of the five strategies defined in E9 (R1),³ the strategy can be referenced.

Handling of missing data (ANALYSIS).

Within the clinical trial, additional details are needed to address how missing outcome measures will be handled in the analysis.^18,19 Planned sensitivity analyses to assess robustness of conclusions to different missing data assumptions and imputation approaches can be included in this section.

Population-level summary measure (ESTIMAND) and Analysis approach (ANALYSIS).

The population-level summary measure specifies how treatment effects will be summarized. The analysis approach provides additional details needed to conduct the analysis, including how estimation of the summary measure of interest will be conducted and which statistical test will be used for inference. Sensitivity analyses to assess the impact of differing assumptions or different statistical methods for estimation can be included in this section.

Illustrative example for a safety objective

We illustrate use of the template for a hypothetical, early phase single-arm trial, modeled after an HIV-1 pediatric trial. Suppose the objective is to determine the safety of a new antiretroviral drug (ARVTRT) as part of a combination antiretroviral treatment regimen in the pediatric population. In this hypothetical trial, the population of interest is individuals aged 13–18 years with well-controlled HIV. One drug in their combination daily antiretroviral regimen will be switched with the study drug ARVTRT, hypothesized to have a better safety profile. Participants are followed on the new ART combination for 24 weeks. As with most trials involving experimental drugs, the trial under consideration excludes pregnant individuals. If pregnancy occurs after enrollment, the protocol stipulates that the ARVTRT study treatment should be discontinued. Table 2 begins with the safety objective as stated in the trial protocol, followed by the estimand description.

Table 2.

Estimand-to-Analysis Table Example for a Safety Objective

Objective: To assess the safety of ARVTRT once daily for 24 weeks.
Estimand: Probability of at least one severe or life-threatening adverse event occurring up to 24 weeks after initiation of ARVTRT in a pediatric population with controlled HIV-1
Treatment: ARVTRT # mg once daily for 24 weeks
ESTIMAND	ANALYSIS
Target population	Analysis set
Individuals aged 13 – 18 years on combination ART regimen with controlled HIV-1 (HIV-1 RNA < 50 copies/ml for at least 6 months) who initiate ARVTRT as a replacement for one of the drugs.	All participants who meet trial eligibility criteria and initiate treatment.
Variable	Outcome measure
Occurrence of at least one severe or life-threatening adverse event up to 24 weeks after treatment initiation	Grade ≥3 adverse event (graded according to the NIAID DAIDS Table for Grading the Severity of Adult and Pediatric Adverse Events20) occurring after initiation of treatment and up to and including Week 24 study visit, per protocol-defined visit window.
Handling of intercurrent events	Handling of missing data
Death is the most serious adverse event and is included in the variable (Composite Variable Strategy). Premature treatment discontinuation for any reason, including pregnancy: Events through 24 weeks used to determine the variable irrespective of ARVTRT discontinuation for any reason (Treatment Policy Strategy).	For participants discontinuing study prior to Week 24, assume no additional adverse events occur. Sensitivity analysis: For participants discontinuing study prior to Week 24, censor follow-up at the last study visit.
Population-level summary measure	Analysis approach
Proportion of individuals with at least one adverse event as specified in the variable	Proportion calculated with a two-sided 95% confidence interval (CI) using Clopper-Pearson exact method for binomial data. Sensitivity analysis: Proportion calculated using the Kaplan-Meier method with Greenwood’s formula for the variance to calculate two-sided 95% CI.

On the ESTIMAND side of the table, the target population highlights the key characteristics of the population of interest. For this study, the intended population is aged 13–18 years with virologically controlled HIV. It also specifies they must have initiated ARVTRT, the treatment of interest. The variable is occurrence of at least one severe or life-threatening adverse event²⁰ experienced by an individual over the 24 weeks after treatment initiation. Three intercurrent events are considered: death, premature treatment discontinuation before 24 weeks, and pregnancy. Death is the worst adverse event and application of a composite variable strategy is illustrated. For individuals not completing the full 24 weeks of treatment, the treatment policy strategy is applied, and the variable will continue to be monitored up to 24 weeks, irrespective of treatment disposition. Because pregnancy after enrollment leads to treatment discontinuation, handling of pregnancy as an intercurrent event can be folded into handling of premature treatment discontinuation. While we handled premature treatment discontinuation using a treatment policy strategy, there are other approaches. If consideration up to treatment discontinuation is of interest or if the summary measure is a quantity that is independent of time (e.g. rate of adverse events per unit exposure), a while on treatment strategy may be more appropriate for a safety analysis.¹¹ The ESTIMAND concludes with the population-level summary measure defined as a proportion.

On the ANALYSIS side, the analysis set notes that participants who enter the trial in error with eligibility violations or who fail to initiate treatment will be excluded. This type of exclusion is not uncommon in Phase I or II studies. The outcome measure clarifies that adverse events reported up to the Week 24 study visit are included, using the visit window specified in the protocol. Handling of missing data states that for the primary analysis based on simple proportions, participants lost to follow-up before study Week 24 are assumed to have had no additional adverse events. As a sensitivity analysis, missing data are handled by censoring at the last study visit, with the proportion calculated using a time-to-event approach. Because this is a small study, the simpler analytic approach is chosen as the primary method of analysis. The analysis approach describes the statistical method for calculating the proportion and confidence intervals: proportions using exact methods for binomial data for the primary analysis and Kaplan-Meier with Greenwood’s variance for the sensitivity analysis.

Application for a placebo-controlled clinical trial

We also present an estimand-to-analysis table in Table 3 that was recently included in an NIH grant proposal, with minor modifications from the actual submission for simplicity. The template was applied to succinctly describe important design and analysis details for the primary objective of a placebo-controlled, confirmatory clinical trial. The trial will determine the benefit of pitavastatin for prevention of major adverse cardiovascular events among people living with HIV with low to moderate cardiovascular risk. Details of the clinical trial are in Grinspoon et al.²¹ The target of estimation is the cause specific hazard ratio of major adverse cardiovascular events between pitavastatin and no treatment. Non-cardiovascular deaths are intercurrent events and are handled with a while on treatment strategy (“while alive” when the events are deaths) to be censored as competing events in the analysis. In this instance, the estimand-to-analysis table is helpful to succinctly tie together estimand, handling of intercurrent events, and the analysis approach.

Table 3.

Estimand-to-Analysis Table for a Placebo-Controlled Clinical Trial

Primary Objective: To determine the effects of pitavastatin as a primary prevention strategy for major adverse cardiovascular events (MACE) in HIV.
Estimand: The cause-specific relative hazard of major adverse cardiovascular events (MACE) following prescribed pitavastatin (relative to no treatment) among people with HIV (PWH) on stable antiretroviral treatment (ART) who have low to moderate cardiovascular disease (CVD) risk according to 2013 American College of Cardiology/American Heart Association (ACC/AHA) guidelines22 and no current indication for a statin.
Treatment a : Pitavastatin 4 mg PO daily
ESTIMAND	ANALYSIS
Target population	Analysis set
PWH on stable ART with low to moderate CVD risk according to 2013 ACC/AHA guidelines and no current indication for a statin.	All randomized participants. Participants assigned to pitavastatin through randomization will be the active treatment group. Participants assigned to placebo will be the comparator group.
Variable	Outcome measure
Time to the occurrence of any of the following MACE outcomes: atherosclerotic or other CVD death, nonfatal myocardial infarction, unstable angina hospitalization, coronary or peripheral arterial revascularization, nonfatal stroke or transient ischemic attack, urgent peripheral artery disease ischemic event, death of undetermined cause.	Time to first MACE outcome or non-CV death where time is measured from randomization to the date of the event. Participants lost to follow-up or who otherwise discontinue study prematurely without experiencing any event will be censored at the date of last contact; deaths from known non-CV causes will be considered as competing risk events in the primary analysis.
Handling of intercurrent events	Handling of missing data
Premature treatment discontinuation: All follow-up included regardless of attribution to treatment (Treatment policy strategy). Deaths from known non-CVD cause: Follow-up is censored (While on treatment strategy, or “while alive”)b.	Unless censoring patterns appear otherwise, censoring will be assumed non-informative for the primary analysis. Sensitivity analyses: Inverse probability of censoring weights (IPCW) will be used to adjust for potentially informative censoring of participants discontinuing the study prematurely for reasons other than death. Information from vital status follow-up will be included, and event times will be imputed for any remaining unobserved follow-up times based on a range of event rates for each treatment group.
Population-level summary measure	Analysis approach
Cause-specific hazard ratio relative to without pitavastatin	The relative cause-specific hazard of pitavastatin versus placebo for MACE will be estimated using Cox proportional hazards model, where non-CVD deaths are competing events. Modification of the statin effect over time (non-proportional hazards) will be evaluated with treatment by time interaction; piecewise hazards will be used in case of failure of the proportional hazards assumption.

^aAn option is to present the Treatment attribute separately in the ESTIMAND and ANALYSIS columns for placebo-controlled trials to show that placebo, while not a treatment of interest in the estimand, is a treatment arm in the context of a trial for the analysis. In this option, Treatment under ESTIMAND would state the treatment of interest, and Treatment under ANALYSIS would include both the treatment and the placebo.

^bBecause response to treatment prior to the occurrence of the intercurrent event is of interest for while on treatment strategy, this is essentially “while alive” when considering death as an intercurrent event.

Discussion

We have introduced a table template to include in SAPs, which connects the estimand addressing a trial objective with how the associated statistical analysis will be conducted. We are using the template in SAPs at the Center for Biostatistics in AIDS Research (CBAR), the statistical center for two large NIH-sponsored clinical trials networks (the AIDS Clinical Trials Group and the International Maternal Pediatric Adolescent AIDS Clinical Trials). CBAR statisticians are responsible for a wide variety of treatment and strategy trials in infectious diseases (HIV-1 and co-infections including tuberculosis and viral hepatitis, and more recently, SARS-CoV2), populations (adult and pediatric), and study designs (pharmacokinetic, Phase I/II, and multi-arm Phase III trials).

We have found the template helpful when collaborating with study teams during study design. Providing a snapshot of the objective, target population, potential intercurrent events, analysis plan and considerations for missing data in one place facilitates team discussion and decision-making. We highlighted pregnancy in the safety objective example because it is an important consideration that requires discussion in many clinical trials.²² The streamlined snapshot of the key trial elements clearly shows how the team’s decisions can affect the target and analysis populations, strategy and data collection requirements. For instance, in discussion of the analysis set, investigators may recommend exclusion of participants under specific conditions that necessitates clarification to the target population in the estimand.

While developed primarily for SAPs, the estimand-to-analysis table may be useful in other documents to provide a concise snapshot of the estimand and analysis plan for a trial objective, as demonstrated by the example from a grant proposal. The standardized presentation of estimand attributes provides guidance to document authors and consistency across studies that facilitates reviews. Prospectively documenting each estimand attribute in this structured format ensures study objectives and estimands are consistent, required data are collected according to the strategies associated with the intercurrent events, and consensus is reached on how missing data will be handled in the analysis.