REporting quality of PilOt randomised controlled trials in surgery (REPORTS): a methodological survey protocol

Abstract

Introduction

The aim of this methodological review is to evaluate the completeness of reporting of surgical pilot and feasibility randomised trials as per the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials. Moreover, we aim to assess for the presence of spin reporting and inconsistency between abstract and main text reporting in surgical pilot and feasibility randomised trials.

Methods and analysis

A comprehensive, electronic search strategy will be used to identify studies indexed in Medline, Embase and Cochrane Central Register of Controlled Trials (CENTRAL) databases. Studies will be included if they are pilot or feasibility randomised trials of surgical interventions. The primary outcome will be overall CONSORT statement extension to randomised pilot and feasibility trials checklist completeness. This will be defined as trials reporting each of the 40 items in the CONSORT statement extension to randomised pilot and feasibility trials checklist. Secondary outcomes will include the reporting of individual studies as per the CONSORT extension to randomised pilot and feasibility trials, the use of spin reporting strategies, trial factors associated with reporting quality and spin strategy use, and consistency between abstract and main text reporting. Poisson and logistic regressions will be performed to explore the association between trial factors and completeness of reporting as measured by the number of reported CONSORT items.

Ethics and dissemination

This is a methodological survey that has been registered a priori on the International Prospective Register for Systematic Reviews (PROSPERO) (CRD42023475512). Local ethics approval is not required. We plan to disseminate study results through peer-reviewed publication and conference presentations.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• These methodological surveys will comprehensively review surgical pilot and feasibility study reporting.

• This study will explore research characteristics associated with surgical pilot and feasibility study reporting.

• This study may be at risk of sampling bias due to the inclusion of only pilot and feasibility studies published in 2011 and 2021.

• None of the outcome measures are validated tools for reporting the completeness and quality of pilot and feasibility studies.

Introduction

Surgical volume continues to grow around the world. In 2004, it was estimated that approximately 234.2 million surgical procedures were performed across the globe.¹ A similar analysis estimated that the number of surgical procedures performed worldwide had grown to 312.9 million in 2012.² According to North American data, the number of surgical procedures was halved throughout the height of the COVID-19 pandemic, but surgical case volumes have since rebounded to prepandemic levels.³ As the global burden of surgical diseases continues to grow, it is estimated that the number of surgical procedures performed worldwide far exceeds these most recent estimates.⁴ Correspondingly, surgical research has increased exponentially. While most surgical research is in the form of observational studies, the use of surgical randomised trials to generate surgical evidence has become increasingly common.^{5 6} From 1999 to 2009, there was a 50% increase in the volume of published surgical randomised trials (1999, 300; 2009, 450).⁶ Other reviews from the early 2000s have suggested increased volumes of published randomised trials by as much as 100%–250% from the 1990s.^{7 8}

Despite the uptake in volume, completion of surgical randomised trials remains a monumental task. There are several methodological challenges in surgical randomised trials that are often not present in randomised trials pertaining to medical interventions, such as blinding, standardisation of surgical procedures, recruitment and the impact of surgeon preference/technique.⁹ As such, in order to reach the goal of completing an impactful randomised trial that provides unmistakable evidence pertaining to certain interventions, a significant amount of time and money must be invested.¹⁰ Given the large commitment associated with definitive randomised trials, investigators will often conduct pilot randomised trials with the goal of demonstrating that a larger, definitive randomised trial aimed at elucidating the effectiveness of an intervention is feasible.¹¹ They have been increasingly common in surgical research and have become a part of the recommendations set forth by the Medical Research Council framework for developing and evaluating complex interventions as well as the Idea, Development, Exploration, Assessment and Long-Term Follow-Up framework for the evaluation of surgical interventions.^12–14

The Consolidated Standards of Reporting Trials (CONSORT) statement was first published in 1996 with the objective to provide guidance to authors in terms of improving the reporting of their randomised trials.¹⁵ It has since undergone two major rounds of expert-consensus revisions, and the most recent CONSORT statement for the reporting of randomised trials was published in 2010.¹⁶ With the recent increase in breadth of randomised trial methodology, extensions to the original CONSORT statement have been created to address specific nuances pertinent to the different types of randomised trials. In 2016, the CONSORT extension to randomised pilot and feasibility trials was published.^{17 18} It is a 40-item checklist that applies to any randomised trial in which a future definitive randomised trial, or some component of it, is scaled down to assess the feasibility of the future randomised trial, regardless of the design (eg, factorial and cluster).¹⁷ Previous methodological reviews have evaluated pilot randomised trials as per the CONSORT extension to randomised pilot and feasibility trials in chronic kidney disease and psychiatric illness; however, a review of this nature has yet to be conducted with surgical pilot randomised trials.^{19 20} Two additional important concepts of reporting quality include spin reporting and consistency between abstract and full-text reporting. Spin reporting was first introduced in 1995 by Junger but has since been popularised as a methodological issue following a publication by Boutron et al in the Journal of the American Medical Association in 2010.^{21 22} In the context of health research, spin reporting refers to the selective or biased presentation of study findings, often aimed at enhancing the perceived significance or impact of the results. It has never been studied in the context of randomised pilot and feasibility trials.²¹ Similarly, while inconsistency between abstract and main text reporting is a well-known methodological issue in randomised trials, this concept has yet to be evaluated in randomised pilot and feasibility trials.²³

Our aim is to perform a methodological review evaluating the completeness of reporting of surgical randomised pilot and feasibility trials as per the CONSORT extension to randomised pilot and feasibility trials. Additionally, we will aim to assess the quality of reporting by assessing the presence of spin reporting (ie, highlighting beneficial results and suppressing detriments) and consistency between abstract and main text reporting.

Materials and methods

Study design

This study is a methodological survey of completeness of reporting of full articles of pilot or feasibility randomised trials for surgical interventions. See table 1 for a summary of all the objectives, corresponding outcomes, explanatory variables and corresponding hypotheses, and methods of analysis. This is a methodological survey that has been registered a priori on the International Prospective Register for Systematic Reviews (PROSPERO) (CRD42023475512). The methodology for this review has been reported according to Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (Supporting information in online supplemental file 3). We have commenced the preliminary work for this study and the planned end date for this research programme is June 2024.

Table 1.

Summary of all the objectives, corresponding outcomes, explanatory variables and corresponding hypotheses, and methods of analysis

Protocol summary
Objective(s)	Outcome(s)	Explanatory variables	Hypothesis	Methods of analysis
Primary: to assess the completeness of reporting and evaluate if the items of the CONSORT extension for pilot and feasibility trials have been reported in surgical pilot and feasibility trials Secondary: identify aspects of the CONSORT checklist consistently reported in pilot and feasibility trials; identify factors associated with complete reporting of pilot and feasibility trials as per the CONSORT checklist; assess the presence and level of spin reporting in surgical pilot and feasibility trials; identify factors associated with spin reporting in surgical pilot and feasibility trials; assess the presence of inconsistent reporting between abstracts and full texts in surgical pilot and feasibility trials; identify factors associated with inconsistent reporting in surgical pilot and feasibility trials	Overall completeness of the reporting of pilot and feasibility trials as per the CONSORT checklist. Completeness of reporting of the individual trials according to the CONSORT checklist. Completeness of reporting of the individual items in the CONSORT checklist. The presence of spin reporting. The level of spin reporting (ie, high, moderate and low). The presence of inconsistent reporting between abstracts and main texts	Year of publication Industry funding Registration in a clinical trial database Multisite study Journal endorsement of CONSORT Published a pilot RCT protocol Number of patients randomised *Publications between 1 January 2011 and 31 December 2011 and 1 January 2021 and 31 December 31 2021	Pilot and feasibility randomised trials in surgery will be poorly reported as per the CONSORT extension for pilot trials. We hypothesise that the following trial factors will be associated with improved reporting: publication after creation of the CONSORT extension to randomised pilot and feasibility trials, industry funding, multicentre studies, journal endorsement of CONSORT and a larger sample size will be associated with an increased number of reported CONSORT items	The completeness of reporting will be summarised with descriptive statistics. We will conduct a Poisson regression to explore the association between the explanatory variables and completeness of reporting as measured by the number of reported CONSORT items

*Note

CONSORT, Consolidated Standards of Reporting Trials; RCT, randomised controlled trial.

Primary research question

Are pilot and feasibility randomised trials in surgery reported as per the CONSORT extension to randomised pilot and feasibility trials?

Secondary research question(s)

1. Are there trial factors associated with more complete reporting as per the CONSORT extension to randomised pilot and feasibility trials?

2. Are there certain domains of the checklist that are more consistently reported than others?

3. Is the reporting of pilot and feasibility trials published following the creation of the CONSORT extension to randomised pilot and feasibility trials higher quality than the reporting of pilot and feasibility trials published before the creation of the CONSORT extension to randomised pilot and feasibility trials (ie, are they more complete as per the CONSORT extension to randomised pilot and feasibility trials checklist)?

4. Are authors of randomised pilot and feasibility trials adequately reporting and classifying outcomes? Are ‘spin’ strategies commonly used when reporting outcomes from randomised pilot and feasibility trials? If so, what section(s) of the published randomised pilot and feasibility trials are being affected by spin?

5. Are there trial factors associated with spin reporting?

6. Is the reporting of outcomes from randomised pilot and feasibility trial abstracts consistent with reporting of outcomes in the main text of manuscripts?

7. Are there trial factors associated with inconsistent abstract and main text reporting?

Study hypothesis

Pilot and feasibility randomised trials in surgery will be poorly reported as per the CONSORT extension for pilot trials. We predict that the majority of included studies will report less than half of the items included in the CONSORT extension to randomised pilot and feasibility trials (ie, >50% of pilot randomised controlled trials (RCTs) will report <50% of the CONSORT items).^{19 20 24}

Based on previous literature examining the reporting of pilot and feasibility randomised trials as per the CONSORT extension to randomised pilot and feasibility trials, we hypothesise that the following trial factors will be associated with improved reporting: publication after creation of the CONSORT extension to randomised pilot and feasibility trials, industry funding, multicentre studies, journal endorsement of CONSORT and a larger sample size will be associated with an increased number of reported CONSORT items.^24–26 Additionally, other covariates that are anticipated to impact the reporting of pilot and feasibility trials include the publication of a study protocol and registration in a clinical trial database.^27–29

Lastly, we hypothesise that spin techniques for reporting outcomes from randomised pilot and feasibility trials will be commonly used by study authors, the interpretation of findings will often be inconsistent with the outcomes and the reporting of outcomes in abstracts will be inconsistent with the reporting of outcomes in the main text of the manuscripts. These have previously been reported as an issue with reporting of definitive, parallel randomised trials in large medical journals as well as surgical trials.^{21 23 30}

Intervention details

The CONSORT statement was first published in 1996 with the objective to provide guidance to authors in terms of improving the reporting of their randomised trials.¹⁵ It has since undergone two major rounds of expert-consensus revisions, and the most recent CONSORT statement for the reporting of randomised trials was published in 2010.¹⁶ With the recent expansion of randomised trial methodology, extensions to the original CONSORT statement have been created to address specific nuances pertinent to the different types of randomised trials. In 2016, the CONSORT extension to randomised pilot and feasibility trials was published.¹⁷ It is a 40-item checklist that applies to any randomised trial in which a future definitive randomised trial, or some component of it, is scaled down to assess the feasibility of the future randomised trial, regardless of the design (eg, factorial and cluster).¹⁷ The included studies will be assessed by two independent reviewers for completion of reporting as per the CONSORT extension to randomised pilot and feasibility trials.

Comparator details

There was no comparator for this methodological survey.

Eligibility criteria

The inclusion criteria will be as follows:

1. Pilot or feasibility randomised trials. This will be defined as any study that self-describes as a pilot or feasibility trial in their title, abstract or introduction that randomised patients prospectively to trial interventions.

2. Intervention of the pilot/feasibility randomised trials was a surgical intervention. A surgical intervention will be defined as follows for the purposes of this survey: any invasive intervention performed by a physician in the operating room or angiography suite requiring regional or general anaesthesia.

3. Published between 1 January 2011 and 31 December 2011 or 1 January 2021 and 31 December 2021. These timeframes were determined based on their relation to the publication of the CONSORT extension to randomised pilot and feasibility trials (ie, 2016). Altman’s data suggest that it takes 4–6 years for a publication to become highly cited.³¹ Therefore, it was assumed that the CONSORT extension to randomised pilot and feasibility trials would have been integrated into clinical trials by 2021. The year 2011 was chosen to keep a constant interval between the date of CONSORT extension to randomised pilot and feasibility trials publication and the years that the studies were analysed.

   The exclusion criteria will be as follows:

   1. Studies described as definitive randomised trials or studies that were not explicitly defined as a pilot or feasibility randomised trial by the study authors as above

   2. Conference abstracts of pilot or feasibility randomised trials

   3. Observational or quasi-randomised pilot studies

   4. Single-armed, non-comparative studies

   5. Systematic reviews, meta-analyses, case-control, case series, case study, surveys, letters, editorials, RCT protocols, secondary analyses of randomised trial data or any other type of study not reporting primary data

   6. Studies not evaluating a surgical intervention as defined above

   7. Studies not published in the time periods defined above

Information sources

The following databases will be searched from 1 January 2011 to 31 December 2011 and 1 January 2021 to 31 December 2021:

1. Medline

2. Embase

3. CENTRAL

The references of studies meeting inclusion criteria were searched manually to ensure that all relevant articles were included.

Search strategy

The search was designed and conducted by a medical research librarian with input from study investigators. Search terms included ‘Pilot Projects’, ‘Feasibility Studies’, ‘Randomized Clinical Trial’, ‘Surgery’ and more (complete search strategies for each of the individual database searches are available in online supplemental files 1 and 2).

Study selection

Two reviewers will independently evaluate the systematically searched titles and abstracts using a standardised, pilot-tested form. Discrepancies that occur at the title and abstract screening phases will be resolved by the inclusion of the study. At the full-text screening stage, discrepancies will be resolved by consensus between the reviewers. If disagreement persists, an additional reviewer will be consulted.

Data management

Two reviewers will independently conduct data extraction into a data collection form designed a priori. Discrepancies will be reviewed in detail by a third reviewer who will resolve the conflict. The extracted data will include:

1. Trial characteristics: author, year of publication, journal of publication, journal impact factor, study period, study design, location of study, number of included centres, study inclusion criteria, study exclusion criteria, intervention details, control details, primary outcomes, secondary outcomes, statistical analyses, study funding, study protocol publication, study registration in a clinical trial database, number of patients assessed for eligibility, number of patients randomised and number of patients completing follow-up

2. Journal characteristics: journal impact factor and explicit mention of CONSORT endorsement on the journal website

3. Reporting details: number of items reported in the study as per the CONSORT extension to randomised pilot and feasibility trials

4. Spin reporting: the use of spin strategies and location of spin strategies (ie, abstract, results section and discussion section)

5. Abstract reporting: described methodology (ie, type of study, inclusion/exclusion criteria, primary and secondary outcomes and methods of analysis), reported outcomes (ie, number of included patients, demographic information, primary outcome data, secondary outcome data, outcome measures and measures of certainty) and conclusion(s)

6. Full-text reporting: described methodology (ie, type of study, inclusion/exclusion criteria, primary and secondary outcomes and methods of analysis), reported outcomes (ie, number of included patients, demographic information, primary outcome data, secondary outcome data, outcome measures and measures of certainty) and conclusion(s)

Outcome definitions

The primary outcome will be overall CONSORT statement extension to randomised pilot and feasibility trials checklist completeness. This will be defined as trials reporting each of the 40 items applicable to all pilot RCTs in the CONSORT statement extension to randomised pilot and feasibility trials checklist. This will result in 40 proportions.

The secondary outcomes will include the completeness of reporting according to the CONSORT statement extension to randomised pilot and feasibility trials checklist for each individual trial defined as the number of items reported (out of 40) for each included study and categorised based on the reporting item (eg, title and abstract, trial design and outcomes). We will also aim to identify research factors associated with proper reporting of randomised pilot and feasibility trials. With regard to spin, this will be defined as the ‘use of specific reporting strategies…to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome’.²¹ The different spin strategies that we will assess in this survey will follow the original classification described by Boutron et al ²¹:

‘(1) A focus on statistically significant results (within-group comparison, secondary outcomes, subgroup analyses, modified population of analyses); (2) interpreting statistically nonsignificant results for the primary outcomes as showing treatment equivalence or comparable effectiveness; and (3) claiming or emphasizing the beneficial effect of the treatment despite statistically nonsignificant results’.

However, these definitions are not specifically applicable to randomised pilot and feasibility trials, and thus, we have adapted the classification as follows: (1) primary focus on efficacy as opposed to feasibility, (2) focus on statistically significant findings as opposed to feasibility (including statistical significance of secondary outcomes) and (3) presentation of results as feasible despite not actually being feasible (ie, criteria for feasibility as described in the given paper have not been met, but study reports it as being met). The level of spin will be assessed in each study conclusion. High spin was defined as the presence of at least two of the three spin strategies: focusing on efficacy as opposed to feasibility, focusing on statistically significant findings as opposed to feasibility and/or presenting results as feasible despite not actually being feasible. Moderate spin was defined as the presence of at least one of the three spin strategies (eg, focusing on efficacy as opposed to feasibility without only focusing on statistically significant findings or presentation of an outcome as feasible despite not actually being feasible). Low spin was defined as the absence of all three spin strategies. This classification of the level of spin is exploratory and not validated.

Lastly, in terms of consistency between abstract and main text reporting, the reporting will be said to be inconsistent if: (1) conclusions stronger (ie, more definitive) than in the main text, (2) omission of negative results found in the main text (ie, results not favouring the intervention group), (3) different primary and/or secondary outcomes than in the main text and (4) presence of methods, results and/or conclusions not in the main text.^{23 32–34} Conclusions stronger than in the main text will be evaluated and defined according to criteria previously published by Bramer et al. ³⁵ The abstract and main text conclusions will be graded as ‘favourable’, ‘promising’ and ‘unfavourable’. A conclusion will be deemed ‘favourable’ if the authors recommended the use of the intervention in clinical practice without reservation. A conclusion will be deemed ‘promising’ if the intervention is described positively with some element of reserve (eg, further research is needed prior to clinical implementation). A conclusion will be deemed ‘unfavourable’ if an intervention is not recommended for clinical use. If the conclusion in the abstract is different and towards the favourable extreme of this spectrum, then the study will be counted as having a conclusion in the abstract that was stronger than the main text.

Sample size

Using a 95% CI approach, the number of required pilot randomised trials (n) for the survey will be given by as follows: n=1.96⋅2(P₀⋅(1 P0)/E²).³⁶ With a margin of error (E) of 0.15, and assuming that the proportion of studies with adequate reporting as per the CONSORT statement extension to randomised pilot and feasibility trials checklist is 0.50 (P₀), 44 pilot and feasibility trials would need to be included in the present survey (table 2).^{20 24} The 95% CI approach was also used to determine the number of randomised pilot and feasibility trials required to adequately assess the use of spin reporting. With a set E of 0.15, and assuming that the proportion of studies with spin reporting is 0.60, 42 pilot and feasibility trials would need to be included.²¹ Lastly, in terms of consistency between abstract and main test reporting, the 95% CI approach determined that 40 pilot and feasibility trials would need to be included (E=0.15, P₀=0.65).²³

Table 2.

Estimation of sample size based on 95% CIs

	P0
	0.50	0.55	0.60	0.65	0.70
E 0.05	392	388	377	357	330
0.10	98	97	94	89	82
0.11	81	80	78	74	68
0.12	68	67	65	62	57
0.13	58	57	56	53	49
0.14	50	50	48	46	42
0.15	44	43	42	40	37

P₀, prior estimate of the proportion of studies with adequate reporting according to the CONSORT statement extension for randomised pilot and feasibility trials checklist.

E, margin of error.

Statistical analysis

The completeness of reporting will be summarised with descriptive statistics (ie, proportion of articles reporting each CONSORT statement item with corresponding 95% CIs estimated using Agresti-Coull intervals) for the primary outcome. For items 6 c, 7b, 11 a, 11 b, 18 and 19 a, the proportions will be calculated based on the total number of studies for which the item is applicable.

The secondary outcomes will be evaluated with descriptive statistics. The mean and SD of the total number of CONSORT items reported will be calculated. The proportion of studies using the different spin strategies highlighted above along with the respective locations (ie, abstract, results and discussion) in which the spin strategies were used will be reported along with corresponding 95% CIs estimated using Agresti-Coull intervals. The proportion of studies with inconsistencies between abstract reporting and main text reporting will be reported along with corresponding 95% CIs estimated using Agresti-Coull intervals. To calculate the mean number of CONSORT items reported, ‘not applicable’ responses to reporting items 6 c, 7b, 11 a, 11 b, 18, and 19 a will be excluded. We will conduct a Poisson regression to explore the association between the following factors and completeness of reporting as measured by the number of reported CONSORT items per study (‘not applicable’ items will be excluded as outlined above):

• Year of publication (dichotomous: 2011/2021)

• Industry funding (dichotomous: yes/no)

• Registration in a clinical trial database (dichotomous: yes/no)

• Multisite study (dichotomous: yes/no)

• Journal endorsement of CONSORT (dichotomous: yes/no)

• Published a pilot RCT protocol (dichotomous: yes/no)

• Number of patients randomised (dichotomous: ≥50/<50)

Based on previously published data, it was hypothesised that publication after creation of the CONSORT extension to randomised pilot and feasibility trials, industry funding, multicentre studies, journal endorsement of CONSORT and larger sample size will be associated with an increased number of reported CONSORT items.^24–26 Additionally, other covariates that are anticipated to impact the reporting of pilot and feasibility trials include publication of a study protocol and registration in a clinical trial database.^27–29 The results of the Poisson regression will be reported with unadjusted and adjusted incidence rate ratios including the associated 95% CIs.

A logistic regression will be conducted to assess the association between the presence of spin, as well as inconsistency between abstract and main text reporting, and the same research factors highlighted above for the Poisson regression. The results of the logistic regression will be reported with unadjusted and adjusted ORs including the associated 95% CIs.

All statistical analyses will be performed on STATA V.18 (StataCrop, College, TX).

Risk of bias assessment

Risk of bias for randomised trials will be assessed using the Cochrane Risk of Bias Tool for Randomised Controlled Trials 2.0.³⁷ Two reviewers will assess the risk of bias independently. Discrepancies will be reviewed in detail by a third reviewer who will resolve the conflict. Risk-of-bias figures will be created with RoBvis.³⁸

Ethics and dissemination

Local ethics approval is not required. We plan to disseminate study results through peer-reviewed publication and conference presentations.

Patient and public involvement

There was no patient or public involvement in the development of this protocol.

Discussion

The CONSORT guidelines have become the standard by which randomised trials are reported.³⁹ The CONSORT extension to randomised pilot and feasibility trials was published in 2016 and has since defined a new standard for reporting of randomised pilot and feasibility trials.¹⁷ However, the ubiquity of their use and reference is much less than the 2010 CONSORT guidelines for parallel randomised trials.^{17 39} This reduced adherence to the extension to randomised pilot and feasibility trials is further evidenced by methodological surveys evaluating the reporting of pilot and feasibility trials from a number of different clinical areas, including chronic kidney disease and psychiatry. Kosa et al reported that the mean number of CONSORT extension to randomised pilot and feasibility trials reported by pilot and feasibility studies investigating chronic kidney disease was 18.4, equating to 53% of the number of items listed in the extension.²⁰ Similarly, Bhatt et al found that the mean reporting score according to the extension for pilot and feasibility RCTs investigating behavioural interventions was 51.6%.¹⁹ Comparable investigation of pilot and feasibility trials pertaining to surgery has never been performed. As such, this methodological survey was designed to assess the completeness of reporting of full articles of pilot or feasibility randomised trials for surgical interventions. We hypothesise that reporting according to the CONSORT extension to randomised pilot and feasibility trials will be poor. By highlighting these discrepancies, we ultimately anticipate that this study will provide information to surgeon scientists that will help improve the reporting of surgical pilot and feasibility randomised trials in the future.

Potential challenges and limitations associated with this work include large quantities of data extraction, difficulty applying a novel definition of spin reporting and sampling bias. While there are a large number of proposed data points that will be extracted, we have assembled a large and experienced team of health method researchers who will ensure the feasibility of this data extraction process. Spin reporting has never been defined and studied in the context of pilot and feasibility trials; thus, our proposed definitions and methods of study may not be entirely applicable. We have extensively reviewed the literature and queried pilot and feasibility trial experts to make these definitions as applicable as possible, yet we will iteratively assess whether the definitions need to be altered throughout the conduct of the study and report these processes thoroughly in our subsequent publications. Lastly, sampling bias will be a limitation as we are only evaluating studies published in 2011 and 2021. These years were chosen intentionally, however, due to previous data published by Altman who suggests that it takes 4–6 years for research to disseminate and become highly cited.³¹ Therefore, these 5 years before and after the publication of the CONSORT extension to randomised pilot and feasibility trials are of particular interest.

Ethics statements

Patient consent for publication

Not applicable.