Early phase clinical trials extension to guidelines for the content of statistical analysis plans

Abstract

This paper reports guidelines for the content of statistical analysis plans for early phase clinical trials, ensuring specification of the minimum reporting analysis requirements, by detailing extensions (11 new items) and modifications (25 items) to existing guidance after a review by various stakeholders.

This paper details guidelines for the content of statistical analysis plans (SAPs) for early phase clinical trials, presenting an extension to the guidelines for the content of SAPs in clinical trials by Gamble et al.¹ Early phase clinical trials (phase I and non-randomised phase II) aim to determine the safety and initial indicators of efficacy of interventions before conducting phase III clinical trials, which have the potential to change practice. The undertaking of definitive late phase clinical trials is often a lengthy and costly process, because they ensure full scale evaluation of the interventions and can also include cost effectiveness analyses. Definitive clinical trials are predicated on accurate and robust conclusions from early phase clinical trials, with flaws in design and analysis potentially leading to interventions failing to demonstrate a benefit in phase III clinical trials. Consequently, the design, conduct, and analysis of early phase clinical trials does not solely affect that specific study. Conclusions from early phase clinical trials have implications for all related subsequent clinical trials. As such, these studies must be performed to the highest standards of rigour and quality, to ensure that correct decisions are taken forward.

Historically, phase I clinical trials were conducted without extensive statistical involvement and conformed to rule based designs (eg, the 3+3 design to determine the maximum tolerated dose).^{2 3} Recent recommendations propose that phase I studies should use model based designs⁴ such as the continual reassessment method,^{5 6 7 8} or model assisted designs such as a modified toxicity probability interval design.⁹ Randomised dose finding phase I clinical trials (eg, that randomise to attain the optimal doses or dose schedules once safety has been assured¹⁰) and single arm phase II designs¹¹ are also being used, all of which require considerable statistical input before, during, and at the analysis stage of the clinical trial. Use of these trial designs with greater statistical involvement has become more prevalent, accelerated by oncology clinical trials,¹² however, examples are emerging across other disease areas.¹³

The International Council for Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use has provided the E9 guidelines (ICH E9), which state that “although the early phases of drug development consist mainly of clinical trials that are exploratory in nature, statistical principles are also relevant.”¹⁴ Because early phase clinical trials use statistical model based designs, the requirement for good quality SAPs, including additional statistical parameters and progression criteria to later phase research, becomes an even greater necessity,^{15 16} with trial statisticians having a key role in designing, and undertaking analysis of, early phase clinical trials.

Guidelines for the content of SAPs by Gamble et al were published in 2017¹ and highlighted the need for a detailed SAP to improve transparency, clinical trial quality, and accuracy. These guidelines were developed with the primary intention of being applicable to the analyses of randomised controlled trials at later phases, and acknowledged that despite some recommendations being transferable, specific consideration and guidance are needed for early phase clinical trials. These guidelines were discussed at a statisticians operational group meeting of the UK Clinical Research Collaboration (UKCRC) registered Clinical Trials Unit (CTU) network in April 2018, confirming that specific consideration and guidance for early phase clinical trials was an area of unmet need. This need was based on the fact that early phase clinical trials often are not randomised, use adaptive designs, and otherwise have statistical considerations and requirements that differ in character from those of later phase, randomised controlled trials. This discussion led to this extension of those 2017 guidelines to fulfil the needs and considerations of SAPs for early phase clinical trials. Given the drug development pathway, early phase trials are more prevalent than late phase trials,¹⁷ highlighting the importance and impact of this guidance.

In this article, we describe the development of an extension to published guidelines for SAP content to broaden their applicability to early phase clinical trials. These recommendations aim to guide the authors of SAPs for all early phase studies, irrespective of the study design used (rule based, model based, model assisted, or randomised phase I trials; or single arm phase II designs). Beyond the scope of this extension are randomised phase II trials, given that they are covered by the existing SAP guidelines for randomised trials.

Summary points.

• Guidance for the content of statistical analysis plans (SAPs) for clinical trials was published in 2017 and focused on late phase, randomised controlled trials

• The existing guidelines have been extended to broaden their applicability to early phase (phase I and non-randomised phase II) clinical trials

• This extension is based on existing guidance; a comprehensive search to identify existing published protocols, SAPs, and SAP guidance; a survey of clinical trial funders and regulators; a survey of current practice by statisticians within Clinical Trials Units registered with the UK Clinical Research Collaboration; a critical appraisal and expert review meeting; and a pilot of the proposed guidelines

• Of 55 original items in the current SAP content guidance, 30 have remained unchanged, 25 have been modified, and a further 11 new items have been proposed to ensure comprehensive and appropriate guidance for early phase clinical trials

Development of the guidance extension

This guidance document encapsulates the findings of a comprehensive search to identify existing published protocols, SAPs, and SAP guidance; a survey of clinical trial funders and regulators; a survey of current practice by statisticians within CTUs registered with the UKCRC; a critical appraisal and expert review meeting; and a pilot of the proposed guidelines. An overview of this process is given in appendix 1.

Literature review of existing guidance

A literature review was undertaken to identify peer reviewed publications of applicable guidelines, and example clinical trial protocols and SAPs (appendix 2 lists the search terms). Two statisticians searched the EQUATOR (enhancing the quality and transparency of health research) network repository for existing guidance, and PubMed for published SAPs and for protocols of early phase clinical trials (at phases I and II). Protocols were included in the search for statistical analysis information, as SAPs might not always have been written for some early phase clinical trials.

These searches were performed independently by two statisticians, with all papers categorised as a relevant and methodological paper, relevant and published protocol or SAP, or not relevant. Papers were classified as not relevant because they included late phase trials, cluster randomised trials, epidemiological studies, meta-analyses and systematic reviews, results papers, editorial publications, or other reasons of non-relevance decided at the discretion of the reviewers (eg, methodological papers, and SAPs for literature reviews). Any discrepancies regarding relevance categorisation were discussed and resolved by mutual agreement.

The literature review was initially performed in November 2019 and updated in October 2020. Of 610 papers returned by the literature review, 500 were excluded owing to non-relevance. Figure 1 shows the number of papers categorised according to each exclusion. The other reasons for exclusion included statistical and clinical trials methodology and literature reviews, SAPs for substudies of randomised controlled trials, and SAPs for substudies. The SAP and protocol search identified two papers perceived to be SAP guidelines: the first being the original guidelines for the content of SAPs,¹ and the second being an overview to the typical content of SAPs for various study types (eg, randomised controlled trials and observational studies) rather than proposing recommended content for SAPs.¹⁸ The literature review found three published SAPs for early phase trials^{19 20 21} and 105 published early phase protocols containing some statistical content. The three published early phase SAPs were all single arm, phase II clinical trials. Of the 105 published protocols, one included an SAP as an appendix,²² seven indicated that a separate SAP had been written (but did not make it available), and the remainder typically contained varying but limited statistical content. These findings are consistent with the notion that while the publication of SAPs has increased, these plans are overwhelmingly for late phase clinical trials.

Fig 1.

Results of literature review (as of 22 October 2020) of existing guidance for content of statistical analysis plans (SAPs). *Other reasons of non-relevance include statistical and clinical trials methodology and literature reviews, statistical analysis plans for substudies of randomised controlled trials, and statistical analysis plans for substudies

Survey of clinical trial funders and regulators

The same clinical trial funders and regulators contacted during the original SAP guidance development¹ were contacted via email in January 2020. Funders were initially contacted to gauge whether they fund early phase clinical trials. If a response was not received, up to two further reminder emails were sent. Surveys were sent to all regulators and those organisations who confirmed the scope of their funding considerations would extend to early phase clinical trials. Consultation with clinical trial funders led to the identification of two additional dedicated early phase clinical trial funders who were also approached. A list of organisations contacted is provided in appendix 3. The goal of these surveys was to ascertain funding and regulatory requirements of design, analysis, publication, and SAP contents for early phase clinical trials.

Of the 39 funding institutions contacted, 28 responded to our request for information, of which 15 indicated that they would fund early phase research. The European Medicines Agency and Medicines and Healthcare products Regulatory Agency provided their regulatory requirements for early phase clinical trials. Additionally, guidance was reviewed that pertained to the running and conduct of early phase clinical trials, such as multiple ICH documents,^{14 23 24 25} the Royal Statistical Society working party report on statistical issues in first-in-human studies,²⁶ various consolidated standards of reporting trials (CONSORT) statements,^{27 28 29 30} and publicly available regulatory guidance.³¹ The predominant documents that funders referred to were ICH E9¹⁴ and existing late phase guidance.¹ The prevailing findings were that:

• Dose escalation decisions, stopping criteria, and interim go-no-go criteria are often poorly documented, potentially resulting in ambiguous and non-robust decisions to escalate

• The statistical design and analysis plan are often not clearly justified

• Where model based approaches are used, the choice of model should be clearly justified, and the risk of overdosing must be quantified and justified to be acceptable, supported by simulation where applicable

• Appropriate sample sizes for early phase clinical trials can be better justified statistically (eg, by simulation) as opposed to by mere custom and historical practice, and that sufficient detail regarding the sample size should be included (with supplemented code to help facilitate this where appropriate) to allow for full replication

• The programming code for modelling should either be suitably referenced or made available in the SAP so that escalation decisions can be replicated and reproduced

• Standard statistical principles (eg, the implication of interim analyses on the overall integrity of the clinical trial and type I error control) are still applicable.

Survey of Clinical Trials Units

The survey was developed based on the original SAP guidance survey¹ and tailored to early phase clinical trials. The aim was to identify CTUs conducting early phase clinical trials and the current practice within those units for developing SAPs. The survey (appendix 3) was circulated to CTUs in the UKCRC network. A list of the 53 registered CTUs was accessed (May 2020) from the UKCRC website³² to cross check responders.

A senior statistician at each CTU was asked to complete the survey to reflect practices and majority opinion within the statistician’s CTU in May 2020. If no response was received then two reminder emails were sent via the mailing list to encourage responses, and then contacts at the unit were approached directly for a response. Example SAPs shared by CTUs conducting early phase clinical trials were collated and reviewed for content to establish the current level of detail provided. To ensure as much coverage for study design types and disease areas, examples were sought from multiple scenarios, including design based (eg, rule based, model based, or single arm phase II) and disease based (oncology or non-oncology) scenarios.

Of the 53 CTUs, 40 (75%) responded to the survey about their experiences of SAPs for early phase clinical trials. Of the 40 responders, 21 (53%) declared to design and analyse early phase clinical trials. On enquiry, the remaining 13 units who did not respond did not conduct early phase clinical trials and so were not pursued further. The prevailing practice of the 21 units statistically designing and conducting early phase clinical trials at the time of survey circulation was to have a generic template (or set of instructions) applicable to all phases of clinical trial, without specific instructions or sections for early phase clinical trials (n=15, 71%). Three (14%) units reported having no template. Importantly, only three (14%) units reported having a template specific to early phase clinical trials. Most units (n=20, 95%) highlighted the desire for early phase SAP guidance, and all indicated that they would use the guidance extension if it existed.

Critical appraisal and expert review meeting

The first draft of the guidance was produced using the literature reviews, funding and regulatory requirements, CTU examples, and authors’ experiences. An expert review panel from the United Kingdom and United States was convened, consisting of academic, pharmaceutical, NHS (UK only), and regulatory representatives (appendix 4).

The panel met virtually on 26 October 2020 with contribution and attendance from 16 statisticians from 14 organisations. Participants critically appraised the first draft of the extension in advance of the meeting in preparation of wider discussion with the group. Considerable discussion centred on the level of detail required from the guidelines and areas that required expansion or clarification. Comments were received from meeting attendees, and a further five statisticians who were unable to attend the meeting. Consensus was reached at this meeting regarding several areas for inclusion of content and recommendations. After incorporation of comments, attendees reviewed and provided feedback before finalising the draft guidance extension for piloting within CTUs.

Piloting of the early phase trial extension

After the expert review meeting, the guidelines extension was updated and piloted at six CTUs in the UK. The aim of the pilot was to ensure the guidance extension produced was fit for purpose, was appropriate to the needs of statisticians writing early phase clinical trial SAPs, and to identify any items requiring further clarification. The pilot was conducted between December 2020 to March 2021 whereby participating CTUs were invited to give formal feedback via email. Piloting covered both phase I and phase II clinical trial designs, and different disease areas. Feedback was universally positive, with minimal amendments (improved wording and clarifications) made to the guidelines extension, which were then finalised.

Ethics

Consistent with the development of the original SAP guidance, ethical approval was not sought for the distribution of surveys; instead, consent to take part was indicated by survey participation.

Early phase extension to existing SAP guidance

The resulting recommendations for the extension to existing SAP content guidance for early phase clinical trials are presented in table 1 (and supplementary checklist 1). Of 55 items proposed in the original SAP content guidance,¹ 30 items have remained unchanged, 25 items have been modified to better reflect early phase clinical trials, and a further 11 new items have been proposed. Substantial modifications and new items include:

Table 1.

Extension of content guidance for statistical analysis plans (SAP) from early phase clinical trials

Section/item	Original SAP content guidance			Recommended early phase clinical trials extension guidance
	Item No	Description		Item No	Description
Section 1: Administrative information
Title and trial registration	1a	Descriptive title that matches the protocol, with SAP either as a forerunner or subtitle, and trial acronym (if applicable)
	1b	Trial registration number
SAP version	2	SAP version number with dates
Protocol version	3	Reference to version of protocol being used
SAP revisions	4a	SAP revision history
	4b	Justification for each SAP revision
	4c	Timing of SAP revisions in relation to interim analyses, etc
Roles and responsibility	5	Names, affiliations, and roles of SAP contributors
Signatures of:	6a	Person writing the SAP
	6b	Senior statistician responsible
	6c	Chief investigator or clinical lead
Section 2: Introduction
Background and rationale	7	Synopsis of trial background and rationale including a brief description of research question and brief justification for undertaking the trial
Objectives	8	Description of specific objectives or hypotheses		8	Description of specific question, objectives, or hypotheses. It should be made clear what the key objectives are (eg, primary and secondary objectives that encompasses toxicity, efficacy, pharmacokinetics, pharmacodynamics, or some combination)
Section 3: Study methods
Trial design	9	Brief description of trial design including type of trial (eg, parallel group, multiarm, crossover, factorial) and allocation ratio and might include brief description of interventions		9a	Brief description of trial design, including the trial phase and the design method (dose escalation (eg, CRM); or single arm phase II (eg, Simon’s two stage)). If the trial has a randomised element to it, summary information regarding the randomisation, including the allocation ratio, should be specified
				9b	Treatment information, including the dose levels of intervention(s). Where appropriate, and if multiple doses are used, the following should also be reported: the ordering and combination (in the instance of multiple agents under investigation) of dose levels, and the dose level to start at
				9c	Details regarding the statistical methodology underpinning the trial, including the choice of the number of parameters in the model if applicable, its empirical form and all formulas. Ensure that all model parameters are given, including the weights of the model, where appropriate
				9d	Rules of the trial design and model: information on the target objective (toxicity, response, pharmacodynamics, pharmacokinetics, either singularly or in combination), classification of overdosing, and any stopping boundaries should be given. This information can include the desired certainty in these estimates. Moreover, for dose decisions (eg, escalation, de-escalation, remain at current dose or stop early), details regarding dose transitions and dose skipping should be given
				9e	Experimental details and design specifics: for dose escalation trials, information regarding cohort size, including whether this is fixed or flexible should be given. Indication of the stopping rules for interim and final evaluations: for model based and model assisted designs, details of the prior including full skeleton (if applicable) and its elicitation should be given; for single arm phase II trials, the target sample size and, where appropriate, the timing of any interim analyses should be given
Randomisation	10	Include randomisation details (eg, whether any minimisation or stratification occurred (including stratifying factors used or the location of that information if it is not held within the SAP))		10	Where appropriate, include randomisation details (eg, whether any minimisation or stratification occurred (including stratifying factors used or the location of that information if it is not held within the SAP)) and, where applicable, details on blinding.
Sample size	11	Full sample size calculation or reference to sample size calculation in protocol (instead of replication in SAP)		11	Full sample size determination or justification or reference to relevant section in protocol (instead of replication in SAP)
Framework	12	Superiority, equivalence, or non-inferiority hypothesis testing framework, including which comparisons will be presented on this basis		12	If applicable, specify whether trial is to be performed under hypothesis testing or bayesian framework
Statistical interim analyses and stopping guidance	13a	Information on interim analyses specifying what interim analyses will be carried out and listing of time points		13a	Information pertaining to interim dose decisions (eg, escalation, de-escalation, remain at current dose or stop early)
	13b	Any planned adjustment of the significance level due to interim analysis		13b	Information on other interim analyses specifying what and when interim analyses will be conducted.
	13c	Details of guidelines for stopping the trial early		13c	Any planned adjustment of the significance level due to interim analysis
				13d	Details of guidelines for stopping the trial early
Timing of final analysis	14	Timing of final analysis (eg, all outcomes analysed collectively or timing stratified by planned length of follow-up)
Timing of outcome assessments	15	Time points at which the outcomes are measured, including visit periods
Section 4: Statistical principles
Indications of uncertainty†	16	Level of statistical significance		16*	Level of statistical significance
	17	Description and rationale for any adjustment for multiplicity and, if so, detailing how the type 1 error is to be controlled		17	Description of any planned adjustment for multiplicity, and if so, including how the type 1 error is to be controlled
	18	Confidence intervals to be reported		18	Either confidence or credible intervals to be reported (appropriately picked dependent on the trial methodology)
Adherence and protocol deviations	19a	Definition of adherence to the intervention and how this is assessed including extent of exposure
	19b	Description of how adherence to the intervention will be presented
	19c	Definition of protocol deviations for the trial
	19d	Description of which protocol deviations will be summarised
Analysis populations	20	Definition of analysis populations (eg, intention to treat, per protocol, complete case, safety)		20	Clear definition of the trial or dose cohort(s) including how cohorts will be referred to, how patients enter cohorts, the minimum number of patients needed to be in a cohort (and how long they have been in) before dose escalation decisions can be made. Trial level definitions of patient populations (eg, per protocol, intention to treat, safety) should be given. Details regarding evaluable patients and specify what happens to unevaluable patients should also be made. These definitions should also be provided for any interim analysis populations
Section 5: Trial populations
Screening data	21	Reporting of screening data (if collected) to describe representativeness of trial sample
Eligibility	22	Summary of eligibility criteria
Recruitment	23	Information to be included in the CONSORT flow diagram
Withdrawal/follow-up	24a	Level of withdrawal (eg, from intervention or from follow-up)
	24b	Timing of withdrawal/lost to follow-up data
	24c	Reasons and details of how withdrawal/lost to follow-up data will be presented
Baseline patient characteristics	25a	List of baseline characteristics to be summarised		25a*	List of baseline characteristics to be summarised
	25b	Details of how baseline characteristics will be descriptively summarised
Section 6: Analysis
Estimand definition (list and describe each primary and secondary outcome including details of each item)‡	26a	Specification of outcomes and timings. If applicable include the order of importance of primary or key secondary end points (eg, order in which they will be tested)		26a	Details of the treatment (including treatment combinations), and any alternative treatments to which comparisons will be made (where appropriate). For dose finding trials, information on whether analysis will be performed per cohort, per dose received, pooled across all dose levels, or in some combination of these
	26b	Specific measurement and units (eg, glucose control, HbA1c (mmol/mol or %))		26b	The trial population, defined with reference to item 20, pertinent to each estimand
	26c	Any calculation or transformation used to derive the outcome (eg, change from baseline, quality-of-life score, time to event, logarithm)		26c	The variable of interest to be obtained for each patient that is required to answer the scientific question. For outcomes recorded at multiple time points, distinction as to which of these time points are required for the estimand
				26d	Intercurrent events and their handling strategy, including adjustment to analysis
				26e	Detail the population level summary measure for each estimand
Analysis methods	27a	What analysis method will be used and how the treatment effects will be presented		27a	What estimator and analysis method will be used and how the results will be presented
	27b	Any adjustment for covariates		27b*	Any adjustments for covariates
	27c	Methods used for assumptions to be checked for statistical methods		27c*	Methods used to check assumptions of the underlying statistical methods and goodness of fit for the model
	27d	Details of alternative methods to be used if distributional assumptions do not hold (eg, normality, proportional hazards)		27d*	Details of alternative methods to be used if distributional assumptions do not hold
	27e	Any planned sensitivity analyses for each outcome where applicable		27e	Any planned sensitivity analyses for each estimand where applicable
	27f	Any planned subgroup analyses for each outcome including how subgroups are defined		27f	Any planned subgroup analyses for each estimand including how subgroups are defined
Missing data	28	Reporting and assumptions or statistical methods to handle missing data (eg, multiple imputation)		28*	Reporting and assumptions or statistical methods to handle missing data (eg, multiple imputation)
Additional analyses	29	Details of any additional statistical analyses required (eg, complier-average causal effect analysis)
Harms	30	Sufficient detail on summarising safety data (eg, information on severity, expectedness, and causality); details of how adverse events are coded or categorised; how adverse event data will be analysed (that is, grade 3/4 only, incidence case analysis, intervention emergent analysis)		30	Sufficient detail on summarising safety data outside of those used for dose escalation (eg, non-DLT safety data), such as information on severity, expectedness, and causality; details of how adverse events are coded or categorised; how adverse event data will be analysed (that is, by grade, incidence case analysis, intervention emergent analysis)
Statistical software	31	Details of statistical packages to be used to carry out analyses		31	Details of statistical packages to be used to carry out design, simulation, and analyses
References	32a	References to be provided for non-standard statistical methods
	32b	Reference to data management plan
	32c	Reference to the trial master file and statistical master file
	32d	Reference to other standard operating procedures or documents to be adhered to
Section 7: Suggested SAP appendices
Simulation report				33	Operating characteristics of the trial design to assess the probability of trial success under different plausible scenarios
Dose transition pathways				34	For dose escalation trials, indication of the dose transition pathways (either using tables or trees/graphs) under different DLT scenarios
Code				35	Full model specification and programming code used for evaluation of dose escalation decisions
Reports template				36	Optional section detailing example tables, graphs, and report templates

Supplementary checklist 1 provides a printable version of the checklist. CONSORT=consolidated standards of reporting trial; CRM=continual reassessment method; DLT=dose limiting toxicity; HbA1c=glycated haemoglobin. Blank table cells in the right hand column indicate that the original item (in the left hand column) is appropriate and covers the necessary content for early phase trials, with examples for all items given in appendix 5.

^*Items for which the table description has remained the same, but the explanation, as detailed in appendix 5, has been amended.

^†This item was originally labelled as “Confidence intervals and P values” in the Gamble et al paper.¹ It has been changed to “Indications of uncertainty” to reflect that many early phase trials designs are underpinned by bayesian methodology.

^‡This item was originally labelled as “Outcome definitions” in the Gamble et al paper.¹ It has been changed to “Estimand definition” following the wider adoption of the International Council for Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use’s E9 guidelines (ICH E9 R1; addendum on estimands and sensitivity analyses).

• Increased details regarding statistical design methodology, and model choice where appropriate

• Update of outcome definitions to include definition of estimands in line with the principles outlined in ICH E9 (R1)

• Inclusion of simulation reports incorporating operating characteristics, to justify statistical design or sample size where applicable

• Inclusion of code required for novel methodology

• Inclusion of dose transition pathways, where appropriate

• Amendments to wording to be more neutral to both frequentist and bayesian methodology, to reflect that some early phase clinical trials designs, particularly phase I, are underpinned by bayesian methods.

Minor changes were made to items including updates to the descriptions to ensure pertinence to early phase clinical trials. Items with no comments in the “Recommended early phase clinical trials extension” column in table 1 (and supplementary checklist 1) indicates that the original item is appropriate and covers the necessary content for early phase clinical trials. An elaboration of each item within the extension guidelines is included in appendix 5, with example text covering various early phase clinical trial designs and disease areas. These examples are intended to be illustrative and are not an endorsement of the methods described.

Discussion

Critical appraisal of clinical trials is only possible if their design, conduct, and analysis are predefined thoroughly and clearly described. Planned trial analyses must be suitably predefined, typically in an SAP. Predefinition of analyses increases the credibility of results by minimising the opportunity of making data driven decisions, or by selecting estimands or methodology to produce a more positive trial outcome. With increased focus on transparency and concerns regarding reproducibility of results alongside the ability to reconstruct clinical trial design and analyses, guidance has been produced for the prospective reporting of SAPs for randomised controlled trials,³³ which advocates that SAPs should be made publicly available. However, this guidance has generally focused on late phase clinical trials and the needs of early phase clinical trials have not explicitly been considered, acknowledging the need for extensions.¹ The absence of transparency for early phase clinical trials and guidance tailored to these designs could result in biased results, which could in turn misinform decision making in clinical development. This SAP guidance extension has been developed to enable statisticians, triallists, and clinical investigators conducting early phase clinical trials to produce clear and relevant SAPs for those clinical trials. The desired outcome is that these clinical trials would be run with increased transparency and increase the likelihood of accurate conclusions.

A major update of the extension is the development of wording on outcome measures and estimands (appendix 5, section 6, item 26), following the ICH E9 (R1) addendum on estimands and sensitivity analyses in clinical trials.²⁵ Although ICH E9 (R1) focuses on the analysis and interpretability of late phase trials, it clearly states that the same principles apply to single arm trials, therefore encompassing early phase designs. At the time of publication, estimands are not widely used in early phase clinical trials with working groups convened specifically to advise on this issue (eg, the Oncology Estimand Working Group³⁴ and Estimand Implementation Working Group³⁵). These groups will provide or publish guidance and examples pertaining to estimands for early phase clinical trials, which should be incorporated into the writing of SAPs in the future, and which might also necessitate revisions to this document.

Another important component to the extension is the simulation report template. All phases of clinical trials need to ensure that the trial will yield an accurate, unbiased result. For fixed or late phase clinical trials, this component typically manifests as a formal sample size calculation. However, for model assisted and model based designs of early phase trials, this calculation is not appropriate. Instead, simulations to assess the designs operating characteristics are needed to ensure that the clinical trial will yield a result and provide sufficient overdose control.³⁶ A template for the suitable simulation report has not been developed as part of this project as they depend on multiple variables including disease area, trial question, trial design, and methodology. Instead, suggestions for content are provided in appendix 5, item 33. The same rationale applies to relevant code and a reports template, with suggestions provided in appendix 5, items 35 and 36, respectively.

In early phase clinical trials, critical decisions regarding the trial progression often need to be made at multiple time points, potentially as early as after the first patient has completed a specified evaluation period. Therefore, compared to late phase clinical trials, SAPs for early phase clinical trials will generally need to be written earlier. At the expert review meeting, the panel acknowledged that the first version of the SAP should be signed off before the trial opening, but this is not always feasible. The panel then recommended finalising the first version of the SAP before the first analysis of clinical trial data, for example, before evaluation of a potential first dose escalation or any interim analysis.

The panel acknowledged that the SAP should encompass all relevant points from table 1 (and supplementary checklist 1) but recommended that signposting should be used to indicate location of details captured elsewhere in documents such as in the clinical trial protocol or a simulation report, to avoid replication. The SAP is not a standalone document and should be read in conjunction with other trial related documentation.

In some scenarios (eg, rule based designs), an SAP might not be required for early phase clinical trials—for instance, if all the applicable content according to these guidelines is sufficiently detailed in the protocol. However, any minor change to that content as the clinical trial progresses would then necessitate a protocol amendment. The combination of running an efficient clinical trial and the level of detail appropriate for analysis of early phase clinical trials provided within this content justifies a standalone SAP.

Although not all UKCRC registered CTUs responded to our survey, this extension captures the opinions of all CTUs who design and analyse early phase clinical trials. Our work built on the original SAP guidelines, which included a Delphi survey.³⁷ In developing the extension to early phase clinical trials, an additional Delphi survey was not considered necessary because our aim was to build on existing knowledge rather than requiring repetition of the process. Instead, we ensured that all relevant stakeholder groups (including CTUs, funders, and regulators) were included in the elicitation of information with the expert multidisciplinary panel assuming responsibility of ensuring detailed considerations of the produced guidelines.

Our expert panel comprised of more academic statisticians than regulatory or pharmaceutical statisticians. This under-representation was identified and more representatives from these areas were invited to participate on the panel. In addition, some individuals provided comments and responded to surveys but were not able to participate in the expert panel. Importantly, every expert panel member was invited to provide their opinions outside of the meeting and review the proposed guideline extension independently and share with their colleagues. When these views were combined, all views were weighted equally regardless of further participation in the extension’s development, and consideration was taken in ensuring that the opinions of all stakeholder groups were reflected in the final version, which all authors have contributed to and agreed on.

Given the multitude and complexity of designs for early phase clinical trials, this guidance was developed to be as generic and applicable as possible across all designs. A proportionate approach was taken, striking a balance between an increased number of items but not increasing beyond the minimum number of items needed to cover adequately the diverse features of early phase clinical trial designs and analysis requirements. Methodological developments within early phase clinical trials are increasingly prevalent, and therefore continued progress in this area is expected. The guidance might need to be updated when developments in statistical techniques emerge, and as such methodology and guidance documentation around early phase clinical trials might need to be reviewed periodically for necessary updates. Certain types of trials, such as bayesian or adaptive trials, are not covered by this extension or the original guidance and will require additional considerations incorporating available regulatory and published guidance. This guidance document provides a necessary extension to the SAP guidelines paper published by Gamble et al¹ for early phase clinical trials. Adherence to this extended guidance will support those individuals working in early phase clinical trials in producing robust conclusions to ensure that correct decisions are taken forward.

Web extra.

Extra material supplied by authors

Web appendix 1: Supplementary materials

Web appendix 2: Survey

Web appendix 3: Checklist 1: SAP Content Guidance Extension for Early Phase Clinical Trials