Do we consider scalability from the outset? A methodological review of pilot randomised controlled health trials

Abstract

Objectives

Health interventions should be designed to be appropriate for scaling from the outset, but the extent to which factors that are important for scalability are reported on in pilot randomised trials is unclear. This review assesses the extent to which pilot randomised trials report on 15 domains of intervention scalability.

Design

Methodological review.

Data sources

Four journals were searched: BMJ Open, BMC Pilot and Feasibility Studies, BMC Trials and PLoS One for articles published between January 2023 and October 2024.

Eligibility criteria

We included pilot randomised trials of health interventions.

Data extraction and synthesis

Data relevant to 15 scalability domains, derived from the Intervention Scalability Assessment Tool and wider implementation science literature, were extracted. Data were double-extracted for 20% of the included studies. Two authors scored all studies from 0 to 3 (0=Not at all; 1=Small extent; 2=Moderate extent; 3=Large extent) on the extent to which each of the 15 scalability domains was reported. For each scalability domain, we calculated the mean score and the frequency of each categorical score across the included studies.

Results

Titles and abstracts screening (521 publications) resulted in 132 full-text publications for review. Of the 104 eligible studies, a random sample of 50 studies were selected for detailed review. Through snowballing, an additional 49 associated publications were identified (eg, protocols), resulting in 99 publications across 50 studies. Most studies reported the Problem (30/50, 60%; mean=2.4 ± 0.8) and the Intervention (37/50, 74%; mean=2.7 ± 0.4) to a large extent (ie, scored 3), with the Delivery Setting and Workforce domain most often receiving a score of 2 (28/50, 56%; mean=2.2 ± 0.7). For eight of the scalability domains, the majority of studies scored 0.

Conclusions

The extent of scalability domain reporting in pilot trials of health interventions is limited. Explicit consideration of scalability in pilot trials could improve the design of fully powered randomised controlled trials and enhance the potential for effective interventions to be translated into practice. Future research should consider if and how to incorporate scalability considerations into pilot trials, and whether pilot trial and intervention reporting guidelines should be expanded to include scalability considerations.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This review provides a methodological snapshot of scalability reporting in recently published pilot randomised trials of health interventions rather than an exhaustive review.

• Data extraction and scalability scoring protocols were developed a-priori and based on established methods.

• Two authors independently scored all included studies across scalability domains.

Background

If evidence-based health interventions are not scaled-up, the desired health impact of these interventions will not be achieved. Evidence-based interventions are often initially tested in pilot and feasibility trials (here, pilot trials). The primary objective of pilot trials is to assess the feasibility of conducting a future definitive trial that is powered to assess intervention effectiveness.^{1 2} Typically, a-priori progression criteria (eg, related to recruitment, intervention acceptability and retention) support researchers to decide if, and how, to proceed to a future definitive trial.³ However, focusing on establishing feasibility for progression to effectiveness trials may inadvertently result in the research and intervention design neglecting to consider factors critical for the future scaling of interventions.

Most public health⁴ and clinical^{5 6} interventions are not scaled-up or effectively integrated into practice.⁷ To address this, researchers and policymakers frequently suggest designing interventions with implementation and scaling in-mind8,12 and thinking politically¹³—to improve the real-world suitability of interventions.² For example, designing an intervention with the end in mind is a key component of the Designing for Dissemination and Sustainability logic model.¹⁴ Scalability can be defined as ‘the ability of a health intervention shown to be efficacious on a small scale and/or under controlled conditions to be expanded under real-world conditions to reach a greater proportion of the eligible population, while retaining effectiveness’¹⁵ (pg.289). Without an explicit focus on scalability from the outset, interventions could be designed and shown to be effective in trials that may not be suitable for delivery outside of trial conditions.

Several scalability domains have been identified based on factors deemed important for scaling health interventions to benefit more people while retaining intervention effectiveness.¹⁶ The Intervention Scalability Assessment Tool (ISAT)¹⁶ outlines 10 scalability domains including: the problem being addressed; the intervention approach; the strategic and political context; intervention effectiveness; intervention costs and benefits; fidelity and adaptation; acceptability and target population reach; intended delivery setting and workforce; sustainability (long-term effectiveness); and implementation infrustructure.¹⁶ This tool is designed to support policymakers and practitioners to make decisions about the suitability of interventions for scale-up after they have been tested. However, if some of these scalability domains are also considered during the design and early stage evaluation of interventions, this may increase the real-world relevance of new health interventions and their potential for scaling.¹⁶

Despite the potential value of considering scalability during early stage trials, scalability is not explicit consideration in the Consolidated Standards of Reporting Trials (CONSORT) extension reporting guidelines for pilot and feasibility trials.¹ However, some domains within the CONSORT extension for pilot and feasibility trials are also important for scaling interventions into practice (eg, intervention acceptability).¹² Equally, some domains that are important for scaling interventions into practice are less relevant for pilot trials. For example, testing intervention effectiveness is not recommended in pilot trials,¹ but evidence of effectiveness is critical to inform policymaker and practitioner decisions for scaling interventions to achieve positive health impact.^{17 18} Overall, little is currently known about the extent to which domains that are important for scaling interventions are reported in pilot trials of health interventions, and thus the extent to which scalability is explicitly considered from the outset in early-stage trials is unclear.

Aim

This methodological review assesses the extent to which recent pilot randomised trials report on intervention scalability across 15 scalability domains.

Methods

Protocol and registration

A protocol for this methodological review is registered on the Open Science Framework.¹⁹ This review is reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.²⁰

Eligibility criteria

Inclusion criteria were external randomised controlled pilot and/or feasibility trials of health interventions (including but not limited to pharmaceutical, surgical, public health and allied health interventions), with no restriction on study feasibility objectives or outcomes. Studies published in English between 1 January 2023 and 18 October 2024 were eligible. Exclusion criteria were non-randomised pilot or feasibility studies, protocol papers and pilot and feasibility studies embedded into a larger randomised controlled trial.

Information sources

Drawing on methods employed by Mellor et al,²¹ four journals were searched through PubMed: British Medical Journal (BMJ) Open, Pilot and Feasibility Studies (PAFS), Trials and Public Library of Science (PLoS) One. These four journals were selected because they have the most PubMed indexed publications that included the terms ‘pilot’ or ‘feasibility’ and ‘trial’ in their title in 2023 and 2024 and because they direct authors to align their reporting against the CONSORT statement.²² BMJ Open and Trials advise authors to use the most appropriate statement extension, and PAFS directs authors to the CONSORT Extension for Pilot and Feasibility Trials.¹ The search terms are detailed in online supplemental file 1. Search terms included ‘pilot’ or ‘feasibility’ in the title and ‘trial’ or ‘study’ in the title or abstract.

Snowballing for associated papers

Three methods were used to identify additional publications associated with the included studies, such as publications related to intervention development, trial protocol and additional outcomes. First, the trial registration webpage for the study was searched for associated publications. Second, we hand-searched reference lists of included articles for related papers. Third, for outcomes published subsequently, a forward citation search was conducted. Where two or more publications were found for the same study (eg, a protocol paper and one or more outcome paper), we assessed the extent to which scalability was considered across all associated papers and report these as a single study.

Study selection

The review process, from title and abstract screening through to data extraction, was conducted in Covidence.²³ Titles and abstracts of identified publications were screened against inclusion criteria by MMcL. Those that met inclusion criteria proceeded to full text review, conducted by MMcL. Of the studies that met inclusion criteria during full text review, we selected a random sample of 50 studies from which to extract data for detailed review. A researcher (HR) who was not involved in screening, data extraction or analysis, used Microsoft Excel’s random number generator to randomly assign each eligible study a unique number from 1 to 104. Studies numbered 1–50 progressed to data extraction.

Data extraction

Data extraction forms (see online supplemental file 2) produced in Covidence²³ were prepiloted on the first 10 trials ordered alphabetically to ensure usability and completeness. One researcher (MMcL) extracted the data for all included publications, with support from a student. Another team member (KT) conducted a second data extraction for the first 20% of studies (n=10 studies). As we found minimal differences between the two data extractions, we decided not to conduct double data extraction for the remaining studies.

We extracted first author last name; study year; study title; study country; study aims and objectives; intervention setting; intervention type; target population; and intervention duration. We also extracted data relating to the 15 scalability domains outlined in table 1. The first 10 scalability domains were derived from the ISAT.¹⁶ Scalability domains included in the ISAT were identified via a literature review, expert input and iterative engagement with policymakers and practitioners.¹⁶ Application of the ISAT involves reviewing the evidence for each domain and assigning a score that indicates the extent to which that domain is addressed. Scores range from 0 (not at all) to 3 (to a large extent), with higher scores indicating greater readiness for intervention scaling.¹⁶ Results are summarised in a radar plot to visually represent relative strengths and domains that may require strengthening for intervention scaling.¹⁶

Table 1. Scalability domain descriptions and inclusion in reporting guidelines.

Scalability domain*	Description	Inclusion in CONSORT-Extension for pilot and feasibility trials1
The Problem	A description of the impact of the problem on the health of the population.	Aligns to Checklist Item 2a ‘Scientific background and explanation of rationale for future definitive trial, and reasons for randomised pilot trial’
The Intervention	A description of the proposed programme/intervention to address the problem described, including the key elements and modes of delivery.	Checklist Item 5. ‘The interventions for each group with sufficient details to allow replication, including how and when they were actually administered’ Also mentions ‘The template for intervention description and replication (TIDieR) guidelines should be followed and the checklist completed’.29
Political Context	Any reporting or discussion of the intervention’s strategic alignment with national, state or regional policy directions or priorities. Any discussion of how scaling-up the health intervention will be strategically useful to funders.	Not explicitly mentioned. Possibly aligns to Checklist Item 2a ‘Scientific background and explanation of rationale for future definitive trial, and reasons for randomised pilot trial.’
Evidence of effectiveness	Any reporting of impact on health outcomes.	Methodological consideration and principle 7: ‘Formal hypothesis testing for effectiveness (or efficacy) is not recommended. The aim of a pilot trial is not to assess effectiveness (or efficacy) and it will usually be underpowered to do this’
Costs versus Benefits	Any reporting or discussion of costs of development, delivery or maintenance (including income, if applicable) and/or cost-effectiveness.	Not explicitly mentioned.
Fidelity and Adaptation	Any reporting of adherence to intervention delivery protocol, or adaptations to the intervention that may affect fidelity to the original intervention.	Not explicitly mentioned. Possibly aligns to Checklist Item 3b: ‘Important changes to methods after pilot trial commencement (such as eligibility criteria), with reasons’. The TIDieR intervention reporting template is recommended to use, which refers to Fidelity and adaptation.29
Acceptability	Any reporting of the extent of stakeholder (including participant) satisfaction with the intervention, including discussion of cognitive and emotional responses to the intervention.	Mentioned in several examples provided in the guidelines. Links to Checklist Item 2b ‘Specific objectives or research questions for pilot trial’
Reach	Any reporting of extent of intervention uptake by participants, and/or discussion of potential intervention uptake by participants at scale.	Checklist Item 7a ‘Rationale for numbers in the pilot trial’ and 13a ‘For each group, the numbers of participants who were approached and/or assessed for eligibility, randomly assigned, received intended treatment, and were assessed for each objective’
Delivery Setting and Workforce	Any reporting of who delivered the intervention and/or where it was delivered and/or in what setting.	Checklist Item 5: ‘Details should include who administered the treatment, as well as what it comprised and how often and where it was delivered.’ The TIDieR intervention reporting template is recommended to use, which refers to the items ‘Where’ and ‘Who Provided’.29
Implementation Infrastructure	Any reporting or discussion of pathways or considerations for real-world delivery/ongoing implementation and resources required, including feasibility of real-world delivery, ‘practicality’, ‘ease of delivery’ and ‘possible to undertake’.	Not explicitly mentioned. Relates to Checklist Item 4C: ‘…it is important to know of any specific aspects that might not be easy to implement in the future definitive RCT.’
Sustainability (long-term effectiveness)	Any reporting of longer-term outcomes of health outcomes, beyond the end of the intervention and/or beyond 12 months of intervention (ie, medium-to-long term).	Not explicitly mentioned.
Implementation Barriers and Enablers	Any reporting of barriers and facilitators to implementation or scalability.	Not explicitly mentioned.
End-user Engagement	Any reporting or discussion of engagement of end-users in the development process of the intervention and/or implementation support strategies (eg, co-design, advisory groups).	Not explicitly mentioned.
Stakeholder partnerships (excluding end-users)	Any reporting or discussion of project stakeholders or partners engaged, including discussions and engagement with stakeholders, through to listed in-kind and cash partners.	Not explicitly mentioned.
Recruitment strategies (real-world)	Any reporting or discussion of the applicability of the intervention recruitment methods for use in the real world.	Recruitment mentioned throughout, but in relation to future trial and no explicit mention of real world.

^*Definitions of each scalability domain adapted from the ISAT16 and outcomes for implementation research.24

CONSORT, Consolidated Standards of Reporting Trials; ISAT, Intervention Scalability Assessment Tool; RCT, randomised controlled trial.

In this review, we split the ISAT domain of reach and acceptability into two separate domains. Reach and acceptability are distinct concepts,²⁴ with acceptability more akin to satisfaction and reach relating to uptake of the intervention and population coverage. Further, we included four additional domains, which according to the implementation science literature are important for scaling interventions: understanding implementation barriers and enablers,²⁵ end-user engagement (eg, co-design),²⁶ stakeholder partnerships²⁷ and recruitment strategies relevant in the real world.²⁸

Scoring of scalability domains

Using an approach adapted from the ISAT,¹⁶ we rated the extent to which studies reported on each scalability domain on a scale from 0 to 3 (0=Not at all; 1=To a small extent; 2=To a moderate extent; 3=To a large extent). Studies were scored on all 15 domains (acknowledging that some domains such as effectiveness and sustainability may be less relevant to pilot randomised trials). Data scoring was completed independently by two authors (MMcL and KT). An operational definition for each domain was generated (see online supplemental file 3) to provide guidance and ensure consistency of scoring. Discrepancies of one or more points between scorers (eg, KT scored 1 and MMcL scored 2) were discussed between MMcL and KT to reach consensus. If consensus was not achieved, a third author (MDMcD) provided a score. Then, the final score was derived by consensus discussion between three authors.

Synthesis of results

Descriptive statistics (frequencies and mean, median and IQR for trial sample sizes) were produced to describe trial characteristics and scores for each scalability domain. Data were analysed in Microsoft Excel. As scores were not normally distributed, categorical frequency data for each score (ie, 0, 1, 2, 3) per domain across included studies were summarised and a stacked bar chart generated showing the percentage distribution of scores across scalability domains visually. To align with the instructions for using the ISAT tool¹⁶ mean score per scalability domain across all studies was also calculated and a radar plot with visual representation of reporting across scalability domains produced. We also conducted a narrative synthesis of the findings for each domain.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of this methods-focused review.

Results

Study selection

Our search strategy identified 521 publications. We screened their titles and abstracts, then assessed the full texts of 132 publications for eligibility, resulting in 104 eligible studies. A random sample of 50 (of the 104) studies were selected for inclusion. From these 50 studies, an additional 49 related publications were identified, resulting in 99 publications for inclusion across 50 studies. Online supplemental file 4 provides a full list of included publications and their corresponding studies. Figure 1 shows the PRISMA flow chart of publications included and excluded at each stage. The main reason for exclusion at full text screening was no control group (n=16).

Figure 1. PRISMA flow chart. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomised controlled trial.

Study characteristics

Table 2 summarises the characteristics of the included studies. Most studies were two-arm (47/50, 94%), non-industry-funded (36/50, 72%) and the most common intervention types were ‘public health’ (15/50, 30%).

Table 2. Characteristics of the included randomised pilot trials.

	N (%)
Journal
British Medical Journal Open	11 (22)
Pilot and Feasibility Studies	18 (36)
Trials	0 (0)
PLoS One	21 (42)
Country
Australia	5 (10)
Canada	4 (8)
UK	13 (26)
USA	8 (16)
Other	20 (40)
Funder
Industry	0 (0)
Non-industry	36 (72)
A combination	4 (8)
Trial did not receive funding	9 (18)
Unknown	1 (2)
Intervention type
Surgical	0 (0)
Pharmaceutical	5 (10)
Public health	15 (30)
Mental health and psychological well-being	13 (26)
Chronic disease management	8 (16)
Allied health and rehabilitation	9 (18)
Sample size
Median (IQR)	48 (33–76)
Range (min–max)	610 (10–620)
Number of arms
2	47 (94)
>2	3 (6)
Intention to proceed to a full trial
Yes (including with adaptations)	40 (80)
No (do not proceed)	1 (2)
Unsure (unclear)	9 (18)

Percentages may not sum up to 100 due to rounding.

Scalability domain scores across studies

Table 3 summarises the extent to which included studies reported on each of the scalability domains. The majority of studies received a score of 3 for the Problem (30/50; 60%, mean=2.4 ± 0.8) and the Intervention (37/50, 74%; mean=2.7 ± 0.4), indicating that information relating to these domains was reported to a large extent. Delivery Setting and Workforce was the next highest scoring domain with most studies receiving a score of 2 (28/50, 56%; mean=2.2 ± 0.7). Four domains scored an average of more than one but less than two. These were: Effectiveness; Fidelity and Adaptation; Acceptability; and Reach. The scores for the effectiveness domain reflect pilot trial recommendations not to undertake effectiveness testing and our scoring method with no trials reporting powered inferential statistics on health outcomes (ie, scoring 3). All included studies considered reach to a small extent (eg, how many people received the intervention) and scored 1, but it was usually in the context of trial recruitment rather than in the context of potential intervention reach if implemented and scaled in usual care settings.

Table 3. Extent of reporting of scalability domains across studies, in randomised pilot trials.

	Reporting of scalability domains
Scalability domain	Score=0 n (%)	Score=1 n (%)	Score=2 n (%)	Score=3 n (%)	Mean (SD) score if reported (ie, 1–3)	Mean (SD) overall score (ie, 0–3)
The Problem	0 (0)	8 (16)	12 (24)	30 (60)	2.4 (0.8)	2.4 (0.8)
The Intervention	0 (0)	0 (0)	13 (26)	37 (74)	2.7 (0.4)	2.7 (0.4)
Political Context	35 (70)	11 (22)	4 (8)	0 (0)	1.3 (0.4)	0.4 (0.6)
Effectiveness	7 (14)	14 (28)	29 (58)	0 (0)	1.7 (0.5)	1.4 (0.7)
Costs versus Benefits	35 (70)	9 (18)	2 (4)	4 (8)	1.7 (0.9)	0.5 (0.9)
Fidelity and Adaptation	21 (42)	15 (30)	8 (16)	6 (12)	1.7 (0.8)	1.0 (1.0)
Acceptability	18 (36)	6 (12)	15 (30)	11 (22)	2.2 (0.7)	1.4 (1.2)
Reach	0 (0)	47 (94)	3 (6)	0 (0)	1.1 (0.2)	1.1 (0.2)
Delivery Setting and Workforce	1 (2)	5 (10)	28 (56)	16 (32)	2.2 (0.6)	2.2 (0.7)
Implementation Infrastructure	37 (74)	12 (24)	1 (2)	0 (0)	1.1 (0.3)	0.3 (0.5)
Sustainability	47 (94)	2 (4)	1 (2)	0 (0)	1.3 (0.5)	0.1 (0.3)
Implementation Barriers and Enablers	32 (64)	7 (14)	8 (16)	3 (6)	1.8 (0.7)	0.6 (1.0)
End-user Engagement	29 (58)	9 (18)	8 (16)	4 (8)	1.8 (0.7)	0.7 (1.0)
Stakeholder Partnerships	42 (84)	7 (14)	1 (2)	0 (0)	1.1 (0.3)	0.2 (0.4)
Recruitment Strategies (Real World)	41 (82)	6 (12)	3 (6)	0 (0)	1.3 (0.5)	0.2 (0.5)

Percentages may not sum up to 100 due to rounding.

Figure 2 visually depicts the percentage distribution of scores across scalability domains. The majority of the included pilot trials did not report information (ie, scored 0) for eight scalability domains (Political Context; Costs versus Benefits; Implementation Infrastructure; Sustainability; Implementation Barriers and Enablers; End-user Engagement; Stakeholder Partnerships; and Recruitment Strategies (Real World)). For example, most studies (35/50; 70%) did not report information relating to Political Context, including alignment of the health intervention with government policy directions, priority health areas or broader strategic context. Although reporting intervention cost-effectiveness is beyond the scope of pilot trials, the majority did not report any information related to the cost to develop or deliver the intervention (35/50; 70%). Most did not report information related to End-user Engagement (29/50; 58%), Implementation Infrastructure (37/50; 74%) or Stakeholder Partnerships (42/50; 84%). The low score for the scalability domain Sustainability (long-term effectiveness) reflects that this is beyond the scope of pilot trials.

Figure 2. Distribution of scores across 15 scalability domains.

Figure 3 presents a radar plot of mean scalability domain scores across included studies. The plot visually depicts variation in the extent of reporting across domains, with higher scores (>2) observed for the Intervention, the Problem, and Delivery Setting and Workforce, and lower scores (<1) for domains such as Stakeholder Partnerships, Recruitment Strategies (Real World), Political Context and Costs versus Benefits.

Figure 3. Radar plot showing 15 scalability domains and mean overall scores for extent of reporting across included studies.

Discussion

Summary of main findings

Our study provides the first assessment of the extent to which pilot and feasibility randomised controlled trials of health interventions report on key domains that are important for intervention scalability. Findings from this review suggest that intervention scalability domains are not well reported in pilot and feasibility trials, with included studies largely limiting their consideration of intervention scalability to three of the 15 domains, namely: the Intervention, the Problem, and Delivery Setting and Workforce. These findings suggest that some domains that are important for the future scalability of health interventions could be more explicitly considered from the outset in pilot trials and reported in associated publications.

Comparison to other studies

The goal of pilot and feasibility trials is to assess the potential for progression to a definitive full trial,¹ whereas scalability domains are factors important for scaling health interventions in real world contexts to benefit more people while retaining intervention effects.^{12 16} There is overlap between some scalability domains¹⁶ and what is recommended in CONSORT-pilot trial reporting guidelines.¹ The CONSORT checklist of information to include when reporting a pilot trial¹ includes describing the problem that the trial proposes to address in the background; describing the intervention in accordance with the Template for intervention description and replication (TIDieR)²⁹; and providing details of who administered the intervention and where it was delivered. Respectively, these checklist items recommend pilot trials report information related to the scalability domains the Problem, the Intervention, and the Delivery Workforce and Setting. Correspondingly, the pilot trials included in our analysis largely reported these scalability domains comprehensively.

Recent research by Pfledderer et al³⁰ developed a consolidated set of 20 considerations for the design, conduct, implementation and reporting of pilot and feasibility studies (all study design types) of behavioural interventions. Some of the scalability domains considered in the current review strongly align with these considerations including ‘Stakeholder engagement and co-production’, ‘Adaptations and tailoring’, ‘Fidelity’ and ‘Cost and Resources’.³⁰ In our analysis of pilot trials, reporting of the linked scalability domains for Stakeholder Partnerships, End-user Engagement and Costs and Benefits was limited. Two (of six) core elements that underpin each phase outlined in the Medical Research Council framework for developing and evaluating complex interventions are ‘stakeholder engagement’ and ‘economic considerations’.³¹ This suggests that while these domains are critical for developing, evaluating and scaling interventions, they are not well reported from the outset in pilot trials.

Our analysis showed limited reporting of concepts related to Fidelity and Adaptation. Recognising the extent to which interventions are delivered as intended (ie, fidelity), and if any changes are made to the intervention (ie, adaptation and modifications), is important to understand what was delivered.²⁵ There can be tensions between traditional trial approaches that emphasise controlled settings for intervention delivery, and less controlled research designs that embrace evaluating interventions delivered in diverse real-world contexts. Adaptations during randomised controlled trials are common³² and can be considerable, so understanding the extent, type and reasons for adaptation can inform scale-up.³³ These tensions mirror broader challenges in bridging gaps between evidence generation and real-world implementation.

A recent JAMA Special Issue highlighted that the two limbs of knowledge generation and implementation largely operate as a ‘house divided’, and that better integration between clinical trials and practice is needed to accelerate improvements in the delivery of health interventions.⁶ Our findings reflect this disconnect, with the Implementation Infrastructure scalability domain given little consideration in the reporting of pilot trials. However, even where implementation infrastructure is robust, and evidence for intervention effects are well-established in later stage research, the decision to adopt or scale-up an intervention is largely determined by the health policy process and political context.^{8 11} For example, there has been a proliferation of research trials of physical activity interventions,³⁴ but many fail to articulate the political context¹³ (eg, congruence with health policy priorities). The limited (reported) considerations by researchers positioning the intervention in political context could be a factor in why so few of these interventions are translated into practice.³⁵ Most pilot trials in our analysis did not report information related to Political Context.

Taken together, our findings show that reporting of scalability domains in pilot trials is mixed but overall limited to three domains that correspond with information captured in the CONSORT extension for Pilot and Feasibility Trials. Implementation science has grown as a discipline to address failures in the translation of effective health interventions in a systematic and coordinated manner^{25 36} and funders internationally have increasingly mandated explicit translation and impact planning from the outset.³⁷ However, findings from our review suggest that such principles have not been explicitly integrated into pilot trial reporting guidelines. As articulated in the recent elaboration and explanation publication for the CONSORT 2025 guideline update for reporting randomised trials,³⁸ reporting issues is one aspect of research waste that is relatively easy to fix, and updating reporting guidelines periodically is essential to ensure their contemporary value and usefulness. To ensure that the potential for future scaling and translation of health interventions into practice is explicitly and systematically considered in the development, conduct and reporting of pilot trials, we suggest that the CONSORT extension for Pilot and Feasibility Trial reporting guidelines is updated.

The anticipated future costs of interventions in the real-world, stakeholder engagement, political context and the implementation infrastructure are some examples of scalability domains that could be reported from the outset in pilot trials of health interventions. However, the relevance and relative importance of specific scalability domains may differ by intervention stage, trial design and research field, and views on their incorporation into pilot trial reporting guidelines are likely to vary across methodological specialisations. Thus, we are unable to make strong recommendations on which specific domains could or should be reported, and in what ways. Any specific recommendations should be developed through international and interdisciplinary consensus and align with best practice for the development of reporting guidelines.³⁹

A second suggestion is for explicit consideration of scalability within recommendations for progression criteria in randomised pilot trials.³ This would ensure that progression decisions are informed not only by the feasibility of conducting a full trial to test intervention effectiveness, but also by the wider context surrounding the intervention’s potential for future translation into practice. A final suggestion is to include explicit consideration of scalability within the TIDieR intervention reporting template.²⁹ While the primary purpose of TIDieR is to support the completeness and clarity of intervention reporting for replication, we suggest that explicitly capturing key scalability considerations within the template would support the systematic consideration of scalability for interventions spanning pilot trial to later stage research. Ultimately, such changes would help support researchers to plan for scalability from the outset and position interventions optimally for future health impact, as recommended by implementation researchers.8,11

Strengths and limitations of the study

The scoring of scalability domains was underpinned by the implementation science literature on domains important for scalability^{25 27 28} including the ISAT tool.¹⁶ The ISAT tool has previously been used to assess the extent to which scalability is considered in publications related to certain types of interventions40,43 and as a reflective tool for researchers to assess the potential scalability of their own interventions.⁴⁴ However, no reviews have explored the extent of consideration across scalability domains for pilot and feasibility trials.

An exhaustive review was not feasible due to the high number of published pilot trials—the CONSORT extension for pilot and feasibility studies has been cited over 4000 times.¹ In line with guidance for methodological reviews,⁴⁵ our approach to selecting studies for inclusion balanced decisions about feasibility, analytical depth and the usefulness of information obtained. Meaningful conclusions have been drawn from reviews with smaller sample sizes that assess scalability across interventions.^{43 46} A strength of this review is that information was extracted from all available linked published studies of the included pilot trials, including intervention development and protocol papers, as well as outcomes publications. This approach allowed for a comprehensive assessment of whether the included studies reported scalability.

This study investigated external randomised pilot trial publications; therefore, it is unclear if the findings can be generalised to other feasibility study designs such as non-randomised pilot trials. The scoring of the extent of scalability domain reporting was informed by clear definitions with example scoring interpretations, developed a priori for each scalability domain. However, inevitably the process involves subjectivity, hence why two researchers independently scored 100% of papers. We only included recent pilot trials published since January 2023. While this is a strength, it may also be a limitation that some associated papers (eg, subsequent process evaluations) that may further increase the scores for each scalability domain assessed, may not yet be published. Finally, we also acknowledge that the inclusion of all types of health interventions led to heterogeneity of interventions. It is possible that the reporting of some scalability domains may vary by intervention type.

Conclusion

Despite calls for interventions to be designed for scale from the outset, our findings highlight that the extent to which scalability is reported in recent pilot trials is limited. One way to help ensure that the potential for the future scaling and translation of health interventions into practice is explicitly and systematically considered in the development, conduct and reporting of pilot trials, is to update the CONSORT extension for Pilot and Feasibility Trial reporting guidelines. Updated guidance to this effect may benefit researchers, peer reviewers, journal editors and funders of external randomised pilot trials, and inform the design of subsequent definitive trials and eventual real-world delivery of interventions found to be effective. Future research should consider what scalability domains are most important and relevant for pilot trials to report, and how pilot trial or intervention reporting guidelines could be expanded to include scalability considerations. Any specific recommendations should be developed with international and interdisciplinary consensus, reflecting a diversity of expertise, contexts, research agendas and intervention types. In the meantime, researchers conducting pilot and feasibility trials may increase the real-world relevance of their studies by considering their specific intervention context and the potential relevance of reporting information relating to specific scalability domains such as political context, stakeholder partnerships, end-user engagement and potential intervention costs.

Data availability statement

Data are available upon reasonable request.