Inclusivity of the target population in orthopaedic surgical randomised trials: a review of high impact journals

Stephen D Brealey; Lucy Atha; Catherine Knowlson; Elizabeth Cook; Kate Hicks; Joanne Newman; Arabella Scantlebury; Joy Adamson; Caroline Fairhurst; Nick A Johnson; Joseph J Dias

doi:10.3310/nihropenres.13781.2

. 2025 Apr 3;5:6. Originally published 2025 Jan 24. [Version 2] doi: 10.3310/nihropenres.13781.2

Inclusivity of the target population in orthopaedic surgical randomised trials: a review of high impact journals

Stephen D Brealey ^1,^a, Lucy Atha ¹, Catherine Knowlson ¹, Elizabeth Cook ¹, Kate Hicks ¹, Joanne Newman ¹, Arabella Scantlebury ¹, Joy Adamson ¹, Caroline Fairhurst ¹, Nick A Johnson ^1,², Joseph J Dias ²

¹Department of Health Sciences, York Trials Unit, University of York, York, England, UK

²University Hospitals of Leicester NHS Trust, Leicester, UK

Email: stephen.brealey@york.ac.uk

No competing interests were disclosed.

Roles

Stephen D Brealey: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Lucy Atha: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Catherine Knowlson: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Elizabeth Cook: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Kate Hicks: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Joanne Newman: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Arabella Scantlebury: Conceptualization, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Joy Adamson: Conceptualization, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Caroline Fairhurst: Conceptualization, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Nick A Johnson: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Joseph J Dias: Conceptualization, Data Curation, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

PMCID: PMC12640489 PMID: 41287628

Version Changes

Revised. Amendments from Version 1

The reporting on place of residence for a study has been amended with reference to Table 6S. The fourth paragraph of the Discussion has had a sentence added that acknowledges the restrictions of data capture for different jurisdictions. The final paragraph of the Discussion has been updated to explain the focus on high impact journals, to acknowledge that different journals have different policies about what can be reported and uploaded with an article, and acknowledged the limitations of the search strategy.

Abstract

Background

This review examines whether randomised controlled trials (RCTs) of surgery in orthopaedics are inclusive of their target populations, including under-served populations.

Methods

The BMJ, Journal of the American Medical Association, The Lancet, and The New England Journal of Medicine were electronically searched in February 2022 for eligible RCTs published from 1 January 2014. Screening, key baseline patient characteristics, the inclusion of under-served groups and whether patient recruitment was pragmatic in design were key data extracted. The findings were tabulated and reported narratively.

Results

There were 26 RCTs included that were parallel in design and conducted across a range of countries in different hospital settings. Four RCTs did not report the complete CONSORT statement. There was variation in the percentage of the screened population who were randomised into the studies ranging from 5.8% to 74.7%. Most RCTs were pragmatic in design regarding patient selection but this did not necessarily translate to an inclusive trial population. Only two RCTs reported the age and gender of all screened patients. All 26 RCTs reported the age and gender of randomised patients but only four studies reported ethnicity. Reporting about the consideration and inclusion of under-served populations was limited.

Conclusions

There is variation in the exclusion of patients of the target population. Reporting of key patient characteristics during screening and attention given to under-served populations in the design, conduct and reporting of these trials is limited. Training and education on inclusivity is required along with practical guidance about how to implement this. To improve inclusivity in the screening and recruitment of patients there should be a focus on (i) screening and eligibility criteria, (ii) collection and reporting on attributes to ensure no section of the eligible population is inadvertently excluded, and (iii) embedding mechanisms to allow all eligible patients the opportunity to participate.

Keywords: Inclusivity; Under-served; Orthopaedics; Surgery; Randomised trials; Review

Plain Language Summary

Orthopaedic surgical trials often aim to evaluate surgery in real-world settings, including a wide range of participants to reflect the target population. This approach helps ensure the research is relevant to clinical practice and that the findings can be widely applied. The National Institute for Health and Care Research (NIHR) in the United Kingdom is focusing on improving the inclusion of under-represented groups in healthcare research, such as those of different ages, educational backgrounds, and language abilities. Including diverse participants in trials is crucial to avoid missing important findings and to prevent discrimination. This review looked at whether orthopaedic surgical trials are inclusive of their target populations, including under-served groups. It included 26 trials from various countries and hospital settings. The review found that there is variation in how patients are excluded, limited reporting of key patient characteristics, and insufficient attention to under-served populations in the design and conduct of these trials. Training and education on inclusivity is required along with practical guidance about how to implement this. To make trials more inclusive and representative, there should be a focus on the following (i) the criteria for screening and eligibility, (ii) collecting and reporting information to ensure no eligible group is left out, and (iii) create ways for all eligible patients to participate.

Introduction

Orthopaedic surgical randomised controlled trials (RCTs) are often pragmatically designed to evaluate whether surgery is an effective intervention in a realistic clinical setting ^1,
2. The evaluation of surgery, lends itself to a pragmatic approach, by its complex nature ¹. A critical aim of a pragmatic approach is to be inclusive of a broad sample of participants that will reflect the target population in clinical practice and maximise the generalisability of findings.

Differences in recruitment across sites, however, is common in pragmatic RCTs ³. This can result in recruited patients differing from those who are not recruited across characteristics such as age, sex, ethnicity, severity of disease, educational status, social class, and place of residence ⁴. Reviews show that trials consistently fail to report participant flow accurately, particularly before informed consent and randomisation ^5,
6. As it is not always clear how many patients were screened for inclusion and why they were not randomised, the results may not be accepted by the surgical community. The National Institute for Health and Care Research (NIHR), the United Kingdom’s (UK) largest public funder of trials, has begun to focus on the inclusion of under-represented groups in health care research. Various grouping have been suggested for consideration and include the following: demographic factors (e.g. age, sex, ethnicity); social and economic factors (e.g. employment, socio-economic status, geographic location, language, digitally excluded); and health status (e.g. mental health condition, cognitive impairment, physical disabilities) ^7,
8. Inclusivity in trials is important for improving representation of the target population so important findings specific to different populations are not missed and to avoid potential discrimination towards historically under-served populations. More money, time and effort may be required to be more inclusive, but may lend itself to research that is representative of the whole patient population and as a result, is more informative for patient and clinical decision-making ⁸.

For orthopaedic surgical trials, the focus has been on improving internal validity ^1,
9 rather than on their applicability to practice ¹⁰. A criticism of such trials has been that the screening, choice, and application of eligibility criteria has meant many patients are excluded. This may affect whether clinicians accept the results of a trial if not considered to be reflective of their usual patients. Treatments may then not be used that could benefit patients and optimise the efficient use of NHS resources, or alternatively, are continued to be used when they have limited clinical and/or cost-effectiveness. This lack of representation has been interpreted as limiting the applicability of the findings of orthopaedic surgical trials ^11,
12 and can delay their translation into practice and increase research waste ¹³.

In the absence of literature exploring the applicability of orthopaedic surgical trials, that such trials have been criticised for a lack of patient representation, and the emerging policy to be inclusive of underserved groups we judged it was timely to conduct a review on this topic. We chose to focus on orthopaedic trials that are published in high impact medical journals that are likely to have high visibility and potential to influence key stakeholders and clinical practice ¹². The aim of this review of high impact journals was to examine whether published findings of RCTs of surgery in orthopaedics are inclusive of their target populations and suggest practical recommendations for encouraging inclusivity by design in future orthopaedic surgical trials.

Methods

Patient and Public Involvement

There was no Patient and Public involvement in this research.

We adapted systematic review methodology to robustly review current methodological practice in orthopaedic surgical trials. To optimise study design and transparency in our reporting the protocol and the findings are aligned with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) checklist and the PRISMA guidance ^14,
15. The review was prospectively registered with Research on Research hub ( https://ror-hub.org/study/1955/).

Eligibility criteria

Individually randomised trials that included an orthopaedic patient population defined as involving bone or joint disorders were eligible for inclusion. Trials must have included surgery compared with: other surgical intervention(s); non-operative (i.e. did not involve surgery) interventions; or a placebo-control. Surgery was defined as any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. Placebo refers to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention. This includes when a scope is inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual procedure ¹⁶. RCTs could be conducted anywhere but only articles written in the English language were included.

Information sources and search strategy

The BMJ, Journal of the American Medical Association (JAMA), The Lancet, and The New England Journal of Medicine (NEJM) were chosen as examples of top-ranking medical journals and electronically searched in February 2022 for eligible RCTs published from 1 January 2014. That year was chosen as it is when concerns were raised about the DRAFFT trial and the number of patients excluded ^11,
17. For the BMJ, the date to filter from 01/01/2014 to 06/02/2022 (defaults to today’s date) was selected, “Research” as the type of article and by “trial” in the title. For the Lancet, the publication range could be customised from January 2014 to February 2022. The search terms “randomised” and “trial” were selected to be in the title of “The Lancet” journal and then results selected on Research Article. For JAMA, the search term used was “randomized clinical trial”, then for article type “research” was selected and for content type “article” with a customised date range of 1 January 2014 to 6 February 2022. Finally, for the NEJM, the term “randomized” was used to search within the abstract, “research” for article category and a date range of 2014/01/01 to 2022/02/28).

Study selection

One reviewer screened all titles and abstracts to identify potentially eligible studies. Full manuscripts of potentially relevant studies were assessed by the reviewer against the eligibility criteria and independently checked by a second reviewer (LA, CK, EC, KH, JN, JA). Disagreements over eligibility were resolved through discussion or recourse to a third reviewer (SB).

Data extraction

A data extraction form was developed in Microsoft Excel and piloted using six studies to assess eligibility and two studies for data extraction. Data collected from the piloting of the form was not included in the review. Data were extracted from the main publication and supplementary files by one reviewer and checked by a second reviewer (LA, CK, EC, KH, JN, JA). Disagreements over data extraction were resolved through discussion or recourse to a third reviewer (SB).

Data items

Information extracted included author, year, study design (e.g. parallel, factorial), country, setting (e.g. number of trauma hospitals/major trauma centres), target patient population (e.g. top level of the CONSORT ¹⁸ statement flowchart or equivalent in text), eligibility criteria, recruitment period, intervention/comparator(s), number of patients (screened, excluded, not consented, randomised), reasons for exclusion, recruitment (i.e. where e.g. clinic, ward, intensive care units; how e.g. search medical databases, media advertising, use of incentives) and age (years), gender and ethnicity of patients.

Data extracted about under-served populations and patients being able to consent included: language barriers (e.g. translation, literacy); allowance for disability (e.g. visual/hearing impairment); electronic data collection (i.e. digital disadvantage); and lack of capacity to consent for themselves ¹⁹.

Domains of the PRECIS-2 tool were used to rate whether recruitment of patients was pragmatic on a scale of 1 to 5 (i.e. very explanatory, rather explanatory, equally pragmatic and explanatory, rather pragmatic, very pragmatic) for eligibility, recruitment and setting ²⁰. This was undertaken by one reviewer (NJ) and checked by a second reviewer (JD) and, if necessary, recourse to a third reviewer (SB).

Quality appraisal was not undertaken as it was not an effectiveness review.

Data synthesis

A narrative and tabular summary of key study characteristics is provided, including the target patient population and eligibility criteria.

The following numbers of patients are presented: (i) screened for enrolment; (ii) excluded based on eligibility criteria; (iii) did not consent; and (iv) randomised. These numbers are presented for all included trials and stratified by the type of comparator.

Age, gender and ethnicity of patients at baseline are summarised descriptively for (i) screened (entire sample); (ii) ineligible (excluded patients); (iii) non-consenting; and (iv) randomised patients. This is presented for all trials and stratified by the type of comparator.

A fixed effects meta-analysis to explore heterogeneity in baseline characteristics (i.e. age, gender, ethnicity) between patients screened but not randomised and those randomised using the I ² statistic was planned ²¹. This was not feasible as there were too few studies, nor for this reason was the planned subgroup analyses about how pragmatic was the trial design or type of comparator.

Finally, whether under-served patient populations were considered including facilitators to consent and whether the trials were pragmatic in the selection and recruitment of patients is tabulated.

Results

Study selection

3,030 potentially eligible studies were identified. After screening the title and abstract, there was full retrieval of journal articles for 27 studies; one was subsequently excluded that did not include surgery ²². Therefore 26 studies were included ^17,
23–
47. Figure 1 summarises the study selection process. Table 1S in the extended data gives a detailed description of the eligibility criteria and patient population.

Study characteristics

Table 2S summarises the study characteristics. All studies were parallel in design, four included a sham or placebo-control ^{25,
41,
42,
47}, and were conducted across a wide range of countries in different hospital settings. Nine studies did not clearly report where recruitment was undertaken ^{17,
23,
24,
32,
34,
36,
37,
39,
47}; among those that did, recruitment took place in locations including out-patient clinics, fracture clinics, wards, and emergency departments. Twelve studies did not report how recruitment was conducted ^{17,
23,
24,
28,
31,
33,
34,
39–
42,
47}. For the remaining studies, recruitment included screening by trial co-ordinators/research associates, review by individual or expert panel of surgeons, or new admissions by surgical teams. The detailed eligibility criteria of the included trials are available in Figshare repository for which further details are in the Data Availability section.

Completion of the CONSORT statement

Table 1 describes the number of patients screened for enrolment, excluded, eligible, non-consenting and randomised as reported in the CONSORT flowchart. Four studies did not fully complete the reporting of patients in the study ^{17,
28,
29,
36}. There was variation in the percentage of the screened and eligible population who were randomised ranging from 5.8% up to 74.7% and 30.1% up to 92.4%, respectively. This occurred within studies of different comparators with the surgical intervention.

Table 1. CONSORT statement flowchart.

Author (year)	Surgery comparator	Number of patients screened	Number of patients excluded	Number of patients eligible	Number of patients not consenting	Number of patients randomised (% of screened & % of eligible)
Rangan et al., 2020	Multiple comparators	914	116	798	295	503 (55.0% & 63.0%)
Skou et al., 2015	Non-operative intervention	1475	1348	127	27	100 (6.8% & 78.7%)
Bailey et al., 2020	Non-operative intervention	790	622	168	40	128 (16.2% & 76.2%)
Willett et al., 2016	Non-operative intervention	2015	1344	671	51	620 (30.8% & 92.4%)
Rangan et al., 2015	Non-operative intervention	1250	687	563	313	250 (20.0% & 44.4%)
van de Graaf et al., 2018	Non-operative intervention	Not reported	Not reported	Not reported	Not reported	321
Rämö et al., 2020	Non-operative intervention	321	181	140	58	82 (25.5% & 58.6%)
Costa et al., 2022	Non-operative intervention	2636	1936	700	196	504 (19.1% & 72.0%)
Griffin et al., 2014	Non-operative intervention	2006	1504	502	351	151 (7.5% & 30.1%)
Kise et al., 2016	Non-operative intervention	341	115	226	85	140 ^a (41.0% & 61.9%)
Palmer et al., 2019	Non-operative intervention	495	145	350	128	222 (44.8% & 63.4%)
Reijman et al., 2021	Non-operative intervention	Not reported	Not reported	282	115	167 (N/A & 59.2%)
Dias et al., 2020	Non-operative intervention	1047	272	775	336	439 (41.9% & 56.6%)
Griffin et al., 2018	Non-operative intervention	6028	5380	648	268 ^b	348 ^c (5.8% & 53.7%)
Ghogawala et al., 2016	Other surgery	Not reported	Not reported	130	64 ^d	66 (N/A & 50.8%)
Försth et al., 2016	Other surgery	358	59	299	52	247 (69.0% & 82.6%)
Ghogawala et al., 2021	Other surgery	458	168	290	127 ^e	163 (35.6% & 56.2%)
Costa et al., 2017	Other surgery	537	131	406	85	321 (59.8% & 79.1%)
Costa et al., 2014	Other surgery	Not reported	Not reported	639	178	461 (N/A & 72.1%)
Faith investigators, 2018	Other surgery	7306	5609	1697 ^f	589	1108 (15.2% & 65.3%)
HIP ATTACK investigators, 2020	Other surgery	27701	19921	7780	532 ^g	2970 (10.7% & 38.2%)
Beard et al., 2019	Other surgery	962	121	841	310	531 ^h (55.2% & 63.1%)
Beard et al., 2018	Placebo control/sham	2975	2235	740	427 ⁱ	313 (10.6% & 42.3%)
Paavola et al., 2018	Placebo control/sham	281	68	213	3	210 (74.7% & 98.6%)
Firanescu et al., 2018	Placebo control/sham	1280	944	336	156	180 (14.1% & 53.6%)
Clark et al., 2016	Placebo control/sham	302	148	154	34	120 (39.7% & 77.9%)

Open in a new tab

^a A further patient was not randomised as they incurred another injury following screening

^bA further 29 eligible patients were not invited to randomisation consultation

^cThree patients were randomised in error and did not receive treatment and were not followed-up

^dThis is 24 eligible who declined all participation and 40 who declined randomisation but included in an observation cohort

^eOf the 127, 91 enrolled into a non-randomised cohort and 15 withdrew prior to randomisation and 21 did not wish to enrol at all/did not wish to have surgery or had surgery at another facility

^f The study reports that 1843 were eligible patients; however, of these 146 were potentially eligible but missed so were not confirmed as eligible patients

^g A further 4278 were eligible but not randomised for the following reasons: operating room board could not accommodate (n=1643), not identified before surgery (n=1009), surgeon not available (n=396), family did not consent (n=374), physician declined (n=231), other (n=625)

^h Of 531, three were randomised twice so excluded

ⁱ Of these 427, 232 took part in an observational cohort for patients with a strong preference and 195 did not partake in the trial or cohort

Description of key baseline characteristics

Tables 3S, 4S and 5S (refer to extended data) describe the key baseline characteristics of age, gender and ethnicity for (i) screened; (ii) ineligible; (iii) non-consenting; and (iv) randomised patients. Only two studies described the characteristics of both screened (age and gender only) and randomised patients ^43,
46, so heterogeneity was not statistically explored. The same two studies did this for ineligible patients and four studies for non-consenting patients ^{17,
36,
43,
46}. All 26 studies reported the age and gender of randomised patients and only four studies for ethnicity ^{32,
34,
43,
46}.

Inclusion of under-served populations

Table 6S (refer to extended data) describes the trial populations for characteristics relevant to under-served populations. There is considerable variation in the choice of lower age limit and ten of the 26 studies (39%) specified an upper age limit (for seven this was ≤75 years) ^{24,
25,
28,
29,
32,
33,
36,
39,
42,
44}. Four studies described the ethnicity of trial participants ^{32,
34,
43,
46}. Six studies described the education of the trial participants ^{28,
29,
33,
40,
43,
46}. Four of those studies, did not describe the entire sample, for example, only whether college education or equivalent was met ^{28,
29,
33,
40}. Eight studies reported the employment status of participants ^{26,
28,
32,
37,
40,
43,
44,
46}. No studies reported using deprivation scores to help recruit the target population, such as selecting a sample of recruiting sites to reflect a range of geographical populations that are historically under-served by research activity. One study reported on place of residence (e.g. independent or nursing home) ²³.

Methods to facilitate consent

Methods to facilitate consent are detailed in Table 7S (refer to extended data). One study reported that language barriers were addressed with the availability of translators ⁴⁶. Two studies reported that study materials were available to potential participants in formats other than written, including the use of a DVD or verbal explanation ^38,
40. Most studies required written consent and completion of paper questionnaires. No studies reported the use of electronic or verbal consent and only three studies referred to electronic collection of questionnaires ^28,
29,
45. Few studies mentioned whether patients without capacity to consent were included in their target population (n=7) or patients being excluded for this reason (n=15). Of the remaining four studies, consent was taken by, for example, a legal guardian or was a decision made by the clinical team in the context of the Mental Health Capacity Act 2005 ^{23,
30,
34,
36}.

Pragmatic selection of trial participants

Table 2 summarises how pragmatic studies were in their selection of patients and includes questions about eligibility, recruitment and setting. For eligibility, 19 of the 26 studies (73%) were agreed to be ‘very pragmatic’ or ‘rather pragmatic’ in design, deeming trial participants similar to those patients in usual care. For recruitment, and the extra effort to do this beyond how patients would be identified in usual care, 24 (92%) studies were ‘very pragmatic’ or ‘rather pragmatic’ in design. Then for the setting in which patients were recruited, 16 (62%) studies were ‘very pragmatic’ or ‘rather pragmatic’ in design.

Table 2. Pragmatic selection of patients into the included studies.

Author (year)	Eligibility—To what extent are the participants in the trial similar to those who would receive this intervention if it was part of usual care?	Recruitment—How much extra effort is made to recruit participants over and above what would be used in the usual care setting to engage with patients?	Setting—How different are the settings of the trial from the usual care setting?
Ghogawala et al., 2016	Very explanatory	Rather explanatory	Rather explanatory
Försth et al., 2016	Rather pragmatic	Rather pragmatic	Rather pragmatic
Skou et al., 2015	Equally pragmatic and explanatory	Rather pragmatic	Rather explanatory
Bailey et al., 2020	Rather pragmatic	Rather pragmatic	Very explanatory
Willett et al., 2016	Rather pragmatic	Rather pragmatic	Very pragmatic
Ghogawala et al., 2020	Rather pragmatic	Rather pragmatic	Very pragmatic
Rangan et al., 2015	Very pragmatic	Very pragmatic	Very pragmatic
van de Graaf et al., 2018	Rather pragmatic	Rather pragmatic	Rather pragmatic
Rämö et al., 2020	Very explanatory	Rather pragmatic	Rather explanatory
Costa et al., 2017	Very pragmatic	Very pragmatic	Very pragmatic
Costa et al., 2014	Rather pragmatic	Very pragmatic	Very pragmatic
Costa et al., 2022	Rather pragmatic	Very pragmatic	Very pragmatic
Firanescu et al., 2018	Rather explanatory	Equally pragmatic and explanatory	Equally pragmatic and explanatory
Griffin et al., 2014	Very pragmatic	Very pragmatic	Equally pragmatic and explanatory
Kise et al., 2016	Rather explanatory	Rather pragmatic	Rather explanatory
Paavola et al., 2018	Rather explanatory	Rather pragmatic	Rather explanatory
Palmer et al., 2019	Equally pragmatic and explanatory	Rather pragmatic	Equally pragmatic and explanatory
Reijman et al., 2021	Rather pragmatic	Rather pragmatic	Rather pragmatic
Beard et al., 2018	Rather pragmatic	Very pragmatic	Very pragmatic
Faith investigators, 2018	Very pragmatic	Very pragmatic	Very pragmatic
HIP ATTACK investigators, 2020	Very pragmatic	Rather pragmatic	Very pragmatic
Dias et al., 2020	Very pragmatic	Very pragmatic	Very pragmatic
Beard et al., 2019	Very pragmatic	Very pragmatic	Very pragmatic
Griffin et al., 2018	Rather pragmatic	Rather pragmatic	Rather pragmatic
Rangan et al., 2020	Very pragmatic	Very pragmatic	Very pragmatic
Clark et al., 2016	Rather pragmatic	Rather pragmatic	Very explanatory

Open in a new tab

Discussion

This review of orthopaedic surgical trials published in high impact journals illustrates considerable variation in how patients are recruited that could affect clinical applicability and acceptability of trial findings. There is marked variation in patients initially screened, who meet all the eligibility criteria, provide consent and are randomised into the study. Limited data were collected about key baseline characteristics of patients who pass through the different phases of patient selection. Notably only four studies (15%) reported on ethnicity which is similar to a recent review that found only 9.3% (38 of 407) of NIHR trials demonstrated exactly how they both recorded, and reported, ethnicity ⁴⁸. Critical to understanding the selection of patients into RCTs is describing their enrolment in the flowchart of the CONSORT statement ¹⁸. Most studies reported the different steps of enrolment but there is considerable selectivity of patients from the screened target population to who were randomised into the study. Within included studies we found limited data about ethnicity, education or employment status of patients. The methods did not explain how language barriers were addressed, and what alternative methods of data collection and enrolment of patients without capacity to consent were used. Studies were mostly pragmatic in recruitment of patients, which by definition should have clinical applicability. However, whilst judged to be pragmatic in design ²⁰, these findings suggest the need to think beyond what is traditionally considered to be pragmatic and truly be inclusive of all eligible patients and that of under-served populations.

Recently, a lack of patient representation in health care research has become the focus of the NIHR, the largest public funder of trials in the UK. Consequently, there has been an emphasis on including under-served groups. It is known, for example, that for musculoskeletal conditions some minority and ethnic groups are disproportionately represented because of risk factors such as levels of physical activity, vitamin D deficiency, poverty, and pre-existing long term conditions such as diabetes ⁴⁹. A recent national survey from a representative sample of 5,030 people from across the UK found nine in ten people (88%) think a diverse mix of participants in health care and research is important even if the research costs more money (70%) or takes more time (74%) ⁵⁰. Both leading funding bodies and the public expect to have inclusivity in research.

Sometimes trial teams may deliberately widen their screening to ensure every possible patient is considered for the study. Although there may be legitimate reasons for this, several included studies specified an upper age limit of ≤75 years which could proactively exclude eligible patients. When designing studies around the inclusive selection of patients and optimising the flow of patients careful consideration should be given to: defining the target population, the choice of eligibility criteria, who is involved in the screening of patients and the training they have and methods used to screen ⁵¹, methods to minimise patient and/or surgeon preferences ⁵² and optimise patient recruitment ⁵³ and involvement of patient and public collaborators ⁵⁴.

Reporting key characteristics of patients who are screened, excluded because of eligibility criteria and who are not approached or do not consent may help reassure clinicians and policy makers about the representativeness of the trial sample. The General Data Protection Regulation in the UK provides the lawful basis for processing such data and the common law of confidentiality allows the collection of data without a legal basis as long as the patient cannot be identified ⁵⁵. As an example, age could be collected in years (or age bands) or only the first part of a postcode to inform measures of deprivation. This allows the lawful and feasible collection of key characteristics of the screened population without the need for consent. A consistent approach from Research Ethics Committees/Health Research Authority and subsequent Information Governance professionals undertaking local site review is required as to what is acceptable to collect that ensures anonymity but permits reporting about inclusivity. This type of data capture may not be permissible, however, due to restrictions of different jurisdictions.

In the UK and the NIHR focus on improving inclusivity in research, frameworks are being or have been developed as to how this may be achieved ^7,
8,
56. This is part of the NIHR workstream called “Innovations in Clinical Trial Design and Delivery for the Under-served” (INCLUDE) project. This includes a roadmap that defines under-served groups and barriers to their inclusion ⁸. The Ethnicity Framework launched on 1 October 2020 ( https://www.trialforge.org/trial-forge-centre/include/) aims to help trial teams think about the inclusion of ethnic groups in their trial ⁷. Multiple approaches to address the barriers to inclusive participation in research include: translation of recruitment and patient questionnaires subject to appropriate validation; and provision of materials in braille, audio-recorded, or animation and apps to help those with low literacy, learning or sensory difficulty. For these tools to be universally adopted into standard trial practice, a coordinated and consistent approach is required to their implementation with a greater understanding of their resource implications to be considerate of the workforce and pressures facing the NHS.

A strength of this review was applying the PRISMA guidelines ^14,
15. The protocol was registered prospectively. It was conducted by a multi-disciplinary team of methodologists, orthopaedic surgeons and trial coordinators. The review is limited to RCTs of orthopaedic surgical trials in high impact journals that are amongst the most cited medical journals and known to the authors as having published large-scale orthopaedic surgical trials. We chose to focus on these RCTs as they are likely to be the best resourced to deliver research and influence key stakeholders and clinical practice. The review team focussed on reporting what was presented in the main publication and supplementary material available on the journal website. It is possible that details not published by the journal are available in a full monograph, in trial registries or in the published protocol; although being described in a registry or protocol does not necessarily mean it was implemented. Moreover, different journals may have different word count policies and what is permitted to be uploaded alongside an article. This review focused on the journal publication and supplementary material as that is most likely to be read by surgeons and to influence decisions in clinical practice. Several reviewers checked study inclusion and undertook data extraction that could contribute to variability in decision-making. This was to make the review feasible with the lack of resources to support it. Maintaining the standard of a second reviewer checking a first reviewer with recourse to always the same third reviewer should mitigate this limitation. Finally, identifying studies with simple search terms were undertaken of the journal website rather than an electronic database such as PUBMED as this was more feasible with the latter lacking specificity in the searches ⁵⁷. Whilst this search strategy may not meet the standards of a systematic review of effectiveness we have nevertheless undertaken a comprehensive review with the resources available to us and makes an original and timely contribution to the literature.

Conclusion

Patient selection and recruitment is a key challenge for RCTs. Different clinical pathways and differences between participating sites and resources available add to the complexity of achieving this. However, the enrolment of a highly selective sample of patients may impact on the clinical applicability and acceptability of study findings. Trials often purport to be pragmatic in design. The limited data available about who and how patients are included in these studies, questions whether they truly are pragmatic and inclusive of the target population. This review is not a criticism of existing high impact orthopaedic surgical trials that are an important contribution to the evidence-base as only recently has there been this attention towards inclusivity and improving external validity. The challenge now is to address this and ensure every person eligible to take part has the same opportunity and are not excluded whether consciously or not. This is a requirement of leading funding bodies and an expectation of the public. This may be difficult and complex to implement as it requires time, resources and funding for which there can be an opportunity cost and needs to be integrated into efficient trial design and delivery ⁵⁸. Change will also not happen on its own and needs initiatives that provide training and education on inclusivity in clinical trials ⁵⁹ and practical guidance about how to implement strategies to achieve this ⁶⁰. The NIHR is starting initiatives to provide training in inclusive research design ⁶¹ and regulatory bodies are developing guidance on increasing diversity of people taking part in clinical trials ⁶². The promotion of decentralised clinical trials away from trial sites could also improve inclusivity in recruitment allowing participants to overcome geographical, financial, family and work constraints ⁶³.

Finally, the following practical guidance could improve inclusivity in the screening and recruitment of patients into orthopaedic surgical trials:

(i) screening and eligibility criteria – including collection of data to allow complete reporting of the CONSORT flowchart ¹⁸, careful consideration in the definition of eligibility criteria ⁶⁴, and efficient data capture methods to record data on those patients screened, eligible, approached and randomised ⁵¹.

(ii) collection and reporting on attributes to ensure no section of the eligible population is inadvertently excluded – including, for example, collecting data during screening on age, sex, ethnicity and (first part of) postcode to inform measures of deprivation which are often not reported yet known to influence patient outcomes ^4,
65,
66.

(iii) embedding mechanisms to allow all eligible patients the opportunity to participate – including making information accessible in a variety of formats such as the translation of recruitment materials; provision of materials in braille, audio-recorded, or animation (that allows captions in different languages); and direction to apps to help read printed materials.

Ethics and consent

This study did not require any form of ethical approval or consent

Acknowledgements

There are no acknowledgements as this review was solely undertaken by the authors.

Funding Statement

This project is funded by the National Institute for Health Research (NIHR) under its [‘Health Technology Assessment Programme’ (Grant Reference Number NIHR133418)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

[version 2; peer review: 1 approved, 1 approved with reservations, 1 not approved]

Data availability

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Figshare: Supporting materials for review of high impact journals about inclusivity in orthopaedic surgical randomised trials

An additional file including Tables 1S to 7S of extended data is available at Figshare repository along with the trial protocol and PRISMA checklist ( https://doi.org/10.6084/m9.figshare.27074599) ⁶⁷.

This project contains the following underlying data:

Tables 1S to 7S extended data
Applicability of orthopaedic surgical trials review protocol_2023.02.02.docx

Reporting guidelines

Figshare: PRISMA checklist for “Inclusivity of the target population in orthopaedic surgical randomised trials: a review of high impact journals”. Doi: https://doi.org/10.6084/m9.figshare.27074599 ⁶⁷

This data is available under the terms of the Creative Commons Zero “No rights reserved” data waiver.

Author contributions

CRediT author statement: All authors contributed to conceptualization, visualization, methodology, writing – original draft, writing – review and editing. SB, LA, CK, EC, KH, JN, NAJ and JJD also contributed to data curation, investigation and validation.

References

1. Cook JA: The challenges faced in the design, conduct and analysis of surgical Randomised Controlled Trials. Trials. 2009;10: 9. 10.1186/1745-6215-10-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
2. Hanzlik S, Mahabir RC, Baynosa RC, et al. : Levels of evidence in research published in The Journal of Bone and Joint Surgery (American volume) over the last thirty years. J Bone Joint Surg Am. 2009;91(2):425–8. 10.2106/JBJS.H.00108 [DOI] [PubMed] [Google Scholar]
3. Knowlson C, Dean A, Doherty L, et al. : Recruitment patterns in multicentre randomised trials fit more closely to price's law than the Pareto principle: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. Contemp Clin Trials. 2022;113: 106665. 10.1016/j.cct.2021.106665 [DOI] [PubMed] [Google Scholar]
4. Rothwell PM: External validity of Randomised Controlled Trials: “to whom do the results of this trial apply?”. Lancet. 2005;365(9453):82–93. 10.1016/S0140-6736(04)17670-8 [DOI] [PubMed] [Google Scholar]
5. Hopewell S, Hirst A, Collins GS, et al. : Reporting of participant flow diagrams in published reports of randomized trials. Trials. 2011;12: 253. 10.1186/1745-6215-12-253 [DOI] [PMC free article] [PubMed] [Google Scholar]
6. Toerien M, Brookes ST, Metcalfe C, et al. : A review of reporting of participant recruitment and retention in RCTs in six major journals. Trials. 2009;10: 52. 10.1186/1745-6215-10-52 [DOI] [PMC free article] [PubMed] [Google Scholar]
7. Treweek S, Banister K, Bower P, et al. : Developing the INCLUDE ethnicity framework—a tool to help trialists design trials that better reflect the communities they serve. Trials. 2021;22(1): 337. 10.1186/s13063-021-05276-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
8. Witham MD, Anderson E, Carroll C, et al. : Developing a roadmap to improve trial delivery for under-served groups: results from a UK multi-stakeholder process. Trials. 2020;21(1): 694. 10.1186/s13063-020-04613-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Blencowe NS, Cook JA, Pinkney T, et al. : Delivering successful randomized controlled trials in surgery: methods to optimize collaboration and study design. Clin Trials. 2017;14(2):211–8. 10.1177/1740774516687272 [DOI] [PubMed] [Google Scholar]
10. Sterne JAC, Savović J, Page MJ, et al. : RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: I4869. 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]
11. Fullilove S, Gozzard C: Dorsally displaced fractures of the distal radius: a critical appraisal of the DRAFFT (Distal Radius Acute Fracture Fixation Trial) study. Bone Joint J. 2016;98-B(3):298–300. 10.1302/0301-620X.98B3.36771 [DOI] [PubMed] [Google Scholar]
12. Nwachukwu BU, Kahlenberg CA, Lehamn JD, et al. : Characteristics of orthopedic publications in high-impact general medical journals. Orthopaedics. 2017;40(3):e405–e12. 10.3928/01477447-20170223-04 [DOI] [PubMed] [Google Scholar]
13. Schmidtke KA, Evison F, Grove A, et al. : Surgical implementation gap: an interrupted time series analysis with interviews examining the impact of surgical trials on surgical practice in England. BMJ Qual Saf. 2023;32(6):341–56. 10.1136/bmjqs-2022-015077 [DOI] [PMC free article] [PubMed] [Google Scholar]
14. Shamseer L, Moher D, Clarke M, et al. : Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350: g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
15. Moher D, Liberati A, Tetzlaff J, et al. : Preferred Reporting Items for Systematic reviews and Meta-Analyses: the PRISMA statement. PLoS Med. 2009;6(7): e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Wartolowska K, Judge A, Hopewell S, et al. : Use of placebo controls in the evaluation of surgery: systematic review. BMJ. 2014;348: g3253. 10.1136/bmj.g3253 [DOI] [PMC free article] [PubMed] [Google Scholar]
17. Costa ML, Achten J, Parsons NR, et al. : Percutaneous fixation with Kirschner wires versus volar locking plate fixation in adults with dorsally displaced fracture of distal radius: Randomised Controlled Trial. BMJ. 2014;349: g4807. 10.1136/bmj.g4807 [DOI] [PMC free article] [PubMed] [Google Scholar]
18. Moher D, Hopewell S, Schulz KF, et al. : CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340: c869. 10.1136/bmj.c869 [DOI] [PMC free article] [PubMed] [Google Scholar]
19. NIHR: Improving inclusion of under-served groups in clinical research: guidance from INCLUDE project.2020; [Accessed date 18 January 2022]. Reference Source
20. Loudon K, Treweek S, Sullivan F, et al. : The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350: h2147. 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
21. Hicks A, Fairhurst C, Torgerson DJ: A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;95:55–62. 10.1016/j.jclinepi.2017.10.001 [DOI] [PubMed] [Google Scholar]
22. Prestmo A, Hagen G, Sletvold O, et al. : Comprehensive geriatric care for patients with hip fractures: a prospective, randomised, controlled trial. Lancet. 2015;385(9978):1623–33. 10.1016/S0140-6736(14)62409-0 [DOI] [PubMed] [Google Scholar]
23. HIP ATTACK Investigators: Accelerated surgery versus standard care in hip fracture (HIP ATTACK): an international, randomised, controlled trial. Lancet. 2020;395(10225):698–708. 10.1016/S0140-6736(20)30058-1 [DOI] [PubMed] [Google Scholar]
24. Palmer AJR, Ayyar Gupta V, Fernquest S, et al. : Arthroscopic hip surgery compared with physiotherapy and activity modification for the treatment of symptomatic Femoroacetabular Impingement: multicentre Randomised Controlled Trial. BMJ. 2019;364:l185. 10.1136/bmj.l185 [DOI] [PMC free article] [PubMed] [Google Scholar]
25. Beard DJ, Rees JL, Cook JA, et al. : Arthroscopic Subacromial Decompression for subacromial shoulder pain (CSAW): a multicentre, pragmatic, parallel group, placebo-controlled, three-group, randomised surgical trial. Lancet. 2018;391(10118):329–38. 10.1016/S0140-6736(17)32457-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
26. Beard DJ, Davies LJ, Cook JA, et al. : The clinical and cost-effectiveness of total versus Partial Knee Replacement in patients with medial compartment osteoarthritis (TOPKAT): 5-year outcomes of a Randomised Controlled Trial. Lancet. 2019;394(10200):746–56. 10.1016/S0140-6736(19)31281-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
27. Willett K, Keene DJ, Mistry D, et al. : Close contact casting vs surgery for initial treatment of unstable ankle fractures in older adults: a Randomized Clinical Trial. JAMA. 2016;316(14):1455–63. 10.1001/jama.2016.14719 [DOI] [PubMed] [Google Scholar]
28. Reijman M, Eggerding V, Van Es E, et al. : Early surgical reconstruction versus rehabilitation with elective delayed reconstruction for patients with Anterior Cruciate Ligament rupture: COMPARE Randomised Controlled Trial. BMJ. 2021;372:n375. 10.1136/bmj.n375 [DOI] [PMC free article] [PubMed] [Google Scholar]
29. van de Graaf VA, Noorduyn JCA, Willigenburg NW, et al. : Effect of early surgery vs Physical Therapy on knee function among patients with nonobstructive meniscal tears: the ESCAPE Randomized Clinical Trial. JAMA. 2018;320(13):1328–37. 10.1001/jama.2018.13308 [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Costa ML, Achten J, Griffin J, et al. : Effect of locking plate fixation vs intramedullary nail fixation on 6-month disability among adults with displaced fracture of the distal tibia: the UK FixDT Randomized Clinical Trial. JAMA. 2017;318(18):1767–76. 10.1001/jama.2017.16429 [DOI] [PMC free article] [PubMed] [Google Scholar]
31. Rämö L, Sumrein BO, Lepola V, et al. : Effect of surgery vs functional bracing on functional outcome among patients with closed displaced humeral shaft fractures: the FISH Randomized Clinical Trial. JAMA. 2020;323(18):1792–801. 10.1001/jama.2020.3182 [DOI] [PMC free article] [PubMed] [Google Scholar]
32. Ghogawala Z, Terrin N, Dunbar MR, et al. : Effect of ventral vs dorsal spinal surgery on patient-reported physical functioning in patients with cervical spondylotic myelopathy: a Randomized Clinical Trial. JAMA. 2021;325(10):942–51. 10.1001/jama.2021.1233 [DOI] [PMC free article] [PubMed] [Google Scholar]
33. Kise NJ, Risberg MA, Stensrud S, et al. : Exercise therapy versus Arthroscopic Partial Meniscectomy for degenerative meniscal tear in middle aged patients: Randomised Controlled Trial with two year follow-up. Br J Sports Med. 2016;50(23):1473–80. 10.1136/bjsports-2016-i3740rep [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Fixation using Alternative Implants for the Treatment of Hip fractures (FAITH) Investigators: Fracture fixation in the operative management of hip fractures (FAITH): an international, multicentre, Randomised Controlled Trial. Lancet. 2017;389(10078):1519–27. 10.1016/S0140-6736(17)30066-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
35. Griffin DR, Dickenson EJ, Wall PDH, et al. : Hip arthroscopy versus best conservative care for the treatment of Femoroacetabular Impingement syndrome (UK FASHIoN): a multicentre Randomised Controlled Trial. Lancet. 2018;391(10136):2225–35. 10.1016/S0140-6736(18)31202-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
36. Ghogawala Z, Dziura J, Butler WE, et al. : Laminectomy plus fusion versus laminectomy alone for lumbar spondylolisthesis. N Engl J Med. 2016;374(15):1424–34. 10.1056/NEJMoa1508788 [DOI] [PubMed] [Google Scholar]
37. Rangan A, Brealey SD, Keding A, et al. : Management of adults with primary frozen shoulder in secondary care (UK FROST): a multicentre, pragmatic, three-arm, superiority randomised clinical trial. Lancet. 2020;396(10256):977–89. 10.1016/S0140-6736(20)31965-6 [DOI] [PubMed] [Google Scholar]
38. Griffin D, Parsons N, Shaw E, et al. : Operative versus non-operative treatment for closed, displaced, intra-articular fractures of the calcaneus: Randomised Controlled Trial. BMJ. 2014;349: g4483. 10.1136/bmj.g4483 [DOI] [PMC free article] [PubMed] [Google Scholar]
39. Försth P, Ólafsson G, Carlsson T, et al. : A randomized, controlled trial of fusion surgery for lumbar spinal stenosis. N Engl J Med. 2016;374(15):1413–23. 10.1056/NEJMoa1513721 [DOI] [PubMed] [Google Scholar]
40. Skou ST, Roos EM, Laursen MB, et al. : A randomized, controlled trial of total knee replacement. N Engl J Med. 2015;373(17):1597–606. 10.1056/NEJMoa1505467 [DOI] [PubMed] [Google Scholar]
41. Clark W, Bird P, Gonski P, et al. : Safety and efficacy of vertebroplasty for acute painful osteoporotic fractures (VAPOUR): a multicentre, randomised, double-blind, placebo-controlled trial. Lancet. 2016;388(10052):1408–16. 10.1016/S0140-6736(16)31341-1 [DOI] [PubMed] [Google Scholar]
42. Paavola M, Malmivaara A, Taimela S, et al. : Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: randomised, placebo surgery controlled clinical trial. BMJ. 2018;362: k2860. 10.1136/bmj.k2860 [DOI] [PMC free article] [PubMed] [Google Scholar]
43. Dias JJ, Brealey SD, Fairhurst C, et al. : Surgery versus cast immobilisation for adults with a bicortical fracture of the scaphoid waist (SWIFFT): a pragmatic, multicentre, open-label, randomised superiority trial. Lancet. 2020;396(10248):390–401. 10.1016/S0140-6736(20)30931-4 [DOI] [PubMed] [Google Scholar]
44. Bailey CS, Rasoulinejad P, Taylor D, et al. : Surgery versus conservative care for persistent sciatica lasting 4 to 12 months. N Engl J Med. 2020;382(12):1093–102. 10.1056/NEJMoa1912658 [DOI] [PubMed] [Google Scholar]
45. Costa ML, Achten J, Ooms A, et al. : Surgical fixation with K-wires versus casting in adults with fracture of distal radius: DRAFFT2 multicentre randomised clinical trial. BMJ. 2022;376: e068041. 10.1136/bmj-2021-068041 [DOI] [PMC free article] [PubMed] [Google Scholar]
46. Rangan A, Handoll H, Brealey S, et al. : Surgical vs nonsurgical treatment of adults with displaced fractures of the proximal humerus: the PROFHER randomized clinical trial. JAMA. 2015;313(10):1037–47. 10.1001/jama.2015.1629 [DOI] [PubMed] [Google Scholar]
47. Firanescu CE, De Vries J, Lodder P, et al. : Vertebroplasty versus sham procedure for painful acute osteoporotic vertebral compression fractures (VERTOS IV): randomised sham controlled clinical trial. BMJ. 2018;361(361): k1551. 10.1136/bmj.k1551 [DOI] [PMC free article] [PubMed] [Google Scholar]
48. Wallace N, O'Keeffe S, Gardner H, et al. : Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability. J Clin Epidemiol. 2023;162:81–9. 10.1016/j.jclinepi.2023.08.008 [DOI] [PubMed] [Google Scholar]
49. Jeraj S, Butt J: Musculoskeletal conditions and Black, Asian and minority ethnic people: addressing health inequalities.Race Equality Foundation;2020. Reference Source
50. Health Research Authority: Public perceptions of research. 2023; [updated 21 Nov 2023]. Reference Source
51. Wilson C, Rooshenas L, Paramasivan S, et al. : Development of a framework to improve the process of recruitment to Randomised Controlled Trials (RCTs): the SEAR (Screened, Eligible, Approached, Randomised) framework. Trials. 2018;19(1): 50. 10.1186/s13063-017-2413-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
52. Davies L, Beard D, Cook JA, et al. : The challenge of equipoise in trials with a surgical and non-surgical comparison: a qualitative synthesis using meta-ethnography. Trials. 2021;22(1): 678. 10.1186/s13063-021-05403-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
53. Scantlebury A, McDaid C, Brealey S, et al. : Embedding qualitative research in Randomised Controlled Trials to improve recruitment: findings from two recruitment optimisation studies of orthopaedic surgical trials. Trials. 2021;22(1): 461. 10.1186/s13063-021-05420-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
54. Crocker JC, Ricci-Cabello I, Parker A, et al. : Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ. 2018;363: k4738. 10.1136/bmj.k4738 [DOI] [PMC free article] [PubMed] [Google Scholar]
55. Health Research Authority: Types of health and care information and the legal frameworks protecting them. 2022; [updated 5 Dec 2022]. Reference Source
56. Shepherd V, Joyce K, Flynn S, et al. : INCLUDE impaired capacity to consent framework.2022; [Accessed date 12 July 2023]. Reference Source
57. Hopewell S, Clarke M, Lefebvre C, et al. : Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev. 2007;2007(2): MR000001. 10.1002/14651858.MR000001.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
58. NIHR: Best research for best health: the next chapter. National Institute of Health Research,2021. Reference Source
59. Shiely F, Rychlíčková J, Kubiak C, et al. : Training and education on inclusivity in clinical trials—the SENSITISE project. Trials. 2024;25(1): 318. 10.1186/s13063-024-08150-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
60. Biggs K, Hullock K, Dix C, et al. : Time to STEP UP: methods and findings from the development of guidance to help researchers design inclusive clinical trials. BMC Med Res Methodol. 2024;24(1): 227. 10.1186/s12874-024-02342-y [DOI] [PMC free article] [PubMed] [Google Scholar]
61. NIHR: Inclusive research design to become an NIHR condition of funding. [Accessed date 16 October 2024]. Reference Source
62. Health Research Authority: HRA and MHRA draft inclusion and diversity guidance. [Accessed date 16 October 2024]. Reference Source
63. Vayena E, Blasimme A, Sugarman J: Decentralised clinical trials: ethical opportunities and challenges. Lancet Digit Health. 2023;5(6):e390–e4. 10.1016/S2589-7500(23)00052-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
64. Dawson S, Banister K, Biggs K, et al. : Trial forge guidance 3: randomised trials and how to recruit and retain individuals from ethnic minority groups—practical guidance to support better practice. Trials. 2022;23(1): 672. 10.1186/s13063-022-06553-w [DOI] [PMC free article] [PubMed] [Google Scholar]
65. Averbuch T, Mohamed MO, Islam S, et al. : The association between Socioeconomic Status, sex, race / ethnicity and in-hospital mortality among patients hospitalized for heart failure. J Card Fail. 2022;28(5):697–709. 10.1016/j.cardfail.2021.09.012 [DOI] [PubMed] [Google Scholar]
66. Cheng AL, Bradley EC, Brady BK, et al. : The influence of race, sex, and social disadvantage on self-reported health in patients presenting with chronic musculoskeletal pain. Am J Phy Med Rehabil. 2022;101(3):211–6. 10.1097/PHM.0000000000001774 [DOI] [PMC free article] [PubMed] [Google Scholar]
67. Brealey S: Supporting materials for review of high impact journals about inclusivity in orthopaedic surgical randomised trials. figshare. 2024; [cited 2025 Jan 6]. https://figshare.com/articles/journal_contribution/Detailed_description_of_the_eligibility_criteria_and_patient_population/27074599/3 [Google Scholar]

NIHR Open Res. 2025 Aug 30. doi: 10.3310/nihropenres.15170.r36215

Reviewer response for version 2

Usman Ali ¹

This is a timely and highly relevant review examining the inclusivity of target populations in orthopaedic surgical RCTs published in high-impact journals. This addresses a critical aspect of clinical trial methodology and generalizability, aligning with contemporary research policy and public interest in diverse representation. The manuscript is generally well-structured, clearly presented, and adheres to key systematic review reporting guidelines, making its methods largely reproducible. One thing of note is the type of study, as the authors mentioned previously that this is not a systematic review. I would suggest expanding on the type of article in the methodology section and clarify what sort of review article it is, if not a systematic review

Some of the Key strengths of the article are its high relevance, clear and detailed methods, and adherence to reporting standards

The research question is particularly significant, addressing a crucial gap in the literature and in understanding patient representation within orthopaedic surgical trials, which has direct implications for clinical applicability and equity in healthcare. The article provides a transparent account of its methodology, including eligibility criteria, data extraction processes, and study selection, which supports the technicality of the work. Although minor limitations are present in the methods section, which are explained below but the overall methodology is robust, clear, and efficient.

Major Points for Authors' Consideration:

Study Selection Methodology and Reviewer Independence:

The manuscript states that " One reviewer screened all titles and abstracts to identify potentially eligible studies. Full manuscripts of potentially relevant studies were assessed by the reviewer against the eligibility criteria and independently checked by a second reviewer."

While the full-text assessment involved independent checking by a second reviewer, the initial screening of titles and abstracts was performed by a single reviewer. Standard review practices typically recommend two independent reviewers for both the initial title/abstract screening and the full-text assessment to minimize bias and human error throughout the selection process. Although this approach is sometimes adopted for pragmatic reasons, strengthening this initial step would enhance the robustness of the review.

Information Sources and Search Strategy, and the Scope of Journals:

The review's search strategy was confined to four top-ranking general medical journals (The BMJ, JAMA, The Lancet, and The NEJM). The authors have answered a prior report previously and justified this choice by citing the high visibility and influential nature of these journals, arguing that trials published therein are most likely to inform surgical practice and are well-resourced.

However, despite this rationale, it remains a significant limitation that the search did not include prominent orthopaedic-specific journals (e.g., JBJS, BJJ, JAAOS, CORR, JOR, Arthroscopy).

Rationale for Inclusion: Orthopaedic surgery is a highly specialized field, and a substantial body of relevant, impactful, and prominent orthopaedic surgery related research is exclusively published in specialty journals. The exclusion of these sources risks to potentially missing a large and relevant portion of the orthopaedic surgical trial landscape, which could influence the overall findings on inclusivity.

Orthopaedic surgeons and residents predominantly rely on orthopaedic specialty journals for most of their focused, clinically relevant knowledge and updates. Specialty journals are highly valued for surgical techniques, subspecialty advances, and research that directly applies to orthopaedic practice. Surveys of orthopaedic residents consistently show that specialty journals like JAAOS and JBJS are considered very important resources for primary scientific literature, clinical updates, and examination studies. (1, 2)

While general medical journals such as JAMA or NEJM have broad appeal and are respected for broader medical knowledge, they might be less frequently prioritized compared to orthopaedic-specific journals for day-to-day clinical practice and specialized research. A review of orthopaedic trials should ideally reflect the literature directly accessed by the orthopaedic community. Also the findings, while valuable for trials in high-impact general journals, may not fully represent or generalize to the broader spectrum of orthopaedic surgical trials, which often appear in dedicated specialty publications.

I would urge the authors to reconsider the implications of this scope limitation more deeply, perhaps by acknowledging that their "snapshot" primarily reflects inclusivity in orthopaedic trials published in general medical journals, rather than the full breadth of orthopaedic surgical research.

If the authors are unable to re-conduct the search to include orthopaedic-specific journals, I strongly recommend they enhance their acknowledgment of this limitation. This can be achieved by explicitly stating in the 'Limitations' section that the findings are primarily generalizable to orthopaedic surgical RCTs published in high-impact general medical journals, and may not fully reflect practices or issues prevalent in trials published within orthopaedic journals. Furthermore, throughout the 'Discussion' and 'Conclusion' sections, statements should be consistently qualified to remind the reader of this specific journal scope, for instance, by stating 'orthopaedic surgical trials published in high-impact general medical journals often demonstrate limitations in inclusivity.' Finally, when discussing implications for orthopaedic practice, it should be acknowledged that direct applicability might be primarily for surgeons who rely more on general medical literature, rather than those who primarily consult specialty orthopaedic journals for updates. This consistent and explicit framing will prevent misinterpretation and clearly delineate the boundaries of the review's generalizability; however, I strongly recommend the inclusion of orthopaedic-specific journals. Given the excellence and timeliness of the research question, the authors should not miss the opportunity to incorporate such valuable information for a truly comprehensive understanding of inclusivity in orthopaedic surgical research.

Coherence of the Discussion Section:

The first paragraph of the Discussion effectively summarizes the direct findings of the review, outlining variations in recruitment and limitations in baseline characteristic reporting.

However, the subsequent paragraphs sometimes lean towards a more general narrative about inclusivity policies (e.g., NIHR initiatives, public opinion) and broad recommendations for trial design. While these elements are crucial for contextualizing the findings and providing a path forward, a clearer and more explicit link back to the specific aggregated findings from the screened articles could be maintained throughout the discussion. This would help ensure that the entire discussion remains firmly anchored in the review's empirical results and avoids appearing overly generalized.

I thank the Journal for the opportunity to review this manuscript. The authors are to be commended for identifying such a pertinent and interesting topic within orthopaedic surgical research and for undertaking this review. Please note that the points raised in this report reflect my personal assessment and professional opinion, which may, of course, differ from those who are experts in the field.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

My primary areas of research include orthopaedic research in low- and middle-income countries, health equity, health disparity, and orthopaedics in underserved communities. This allow me to particularly assess aspects of this article related to patient inclusivity and representation, and the implications for equitable healthcare

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References

1. : Educational resource utilization by current orthopaedic surgical residents: a nation-wide survey. JAAOS Global Research &.2019; [DOI] [PMC free article] [PubMed]
2. : Orthopedic surgery residents' study habits and performance on the orthopedic in-training examination. Am J Orthop.2007; [PubMed]

NIHR Open Res. 2025 Sep 12.

Stephen Brealey ¹

Response: Thank you for your thoughtful and constructive comments about our review; we are grateful to have the opportunity to respond. The study design could be described as a “narrative review” of the literature, which was conducted as robustly as was feasible by adapting key methodological principles from systematic reviews with the resources available. It wasn’t an effectiveness review, which allowed a more flexible application of systematic review methodology. However, characterising this study as a “narrative review” would not fully represent the methodological rigour undertaken. Given the extensive adaptations of systematic review methodology to enhance the study's robustness, we prefer to keep to the current description of the review in the Methods section that is both accurate and appropriate.

Some of the Key strengths of the article are its high relevance, clear and detailed methods, and adherence to reporting standards

Major Points for Authors' Consideration:

Study Selection Methodology and Reviewer Independence:

Response: We agree that a more robust study selection process would have enhanced its rigour. However, as explained in the Discussion, a more pragmatic approach was taken with the limited resources available to complete this review. It is unlikely these additional steps would alter the clear message found from this review. Nonetheless, to acknowledge this limitation, it has been added to the Discussion.

Information Sources and Search Strategy, and the Scope of Journals:

However, despite this rationale, it remains a significant limitation that the search did not include prominent orthopaedic-specific journals (e.g., JBJS, BJJ, JAAOS, CORR, JOR, Arthroscopy).

Response: We focussed on high-impact journals for the reasons given in the publication. We understand that surgeons will consider speciality journals for day-to-day practice. However, policy and practice, such as that informed by NICE guidance, will be very much influenced by large-scale well-funded clinical trials. We argue that if well-funded studies with teams of collaborators across multiple institutions and sometimes across multiple countries do not appropriately address inclusivity it is unlikely that this will be different for evidence published in specialty journals. It was not feasible to include evidence from orthopaedic-specific journals. We have, as has been suggested, emphasised the limited scope and generalisability of the review in the Discussion.

Coherence of the Discussion Section:

The first paragraph of the Discussion effectively summarizes the direct findings of the review, outlining variations in recruitment and limitations in baseline characteristic reporting.

Response: The authors have carefully considered this comment. The Discussion is anchored to the evidence found in this review and has been discussed in the context of the current guidance and literature on this topic to optimise inclusivity into trials. Notably the Discussion ends with practical guidance to improve inclusivity into orthopaedic surgical trials. Therefore, no further edits to the Discussion are considered necessary.

NIHR Open Res. 2025 Jun 12. doi: 10.3310/nihropenres.15170.r35122

Reviewer response for version 2

Ameeta Retzer ¹

Thank you for addressing the comments I had previously outlined. I have no further comments.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

- systematic evaluation of clinical trial protocols and reports, inclusive and equitable research design. I am not an expert in surgical or orthopaedic research.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

NIHR Open Res. 2025 Mar 3. doi: 10.3310/nihropenres.14969.r34610

Reviewer response for version 1

Kim Madden ¹

Thank you for the opportunity to review this paper. Inclusivity is a key issue in orthopaedics and the topic is very important. However, I have several comments about the paper.

1. Several studies have been missed. There are more than 26 orthopaedic trials in major journals in the past decade. E.g. [Ref 1] [Ref 2].

The search strategy is flawed because the exact term "randomized" is not always used. Sometimes it is "randomly assigned" or "randomization" as in the examples given here.

2. This type of analysis that involves describing the excluded sample is challenging because each site or local ethics committee will have different requirements/processes for screening and recording who was screened. Our site is not allowed to keep any data on patients who do not consent. Researchers in my jurisdiction are not even allowed to know about patients who declined to be screened or declined to talk to research staff so those are never reported. I have noticed vastly different numbers of patients screened at different sites, not because of different actual numbers of patients, but different screening processes and privacy laws.

3. Why use a fixed effects model? Random effects seems more appropriate here.

4. "One study reported including participants living at an alternative place of residence, that is a nursing home [cites the HIP ATTACK trial]". The HIP ATTACK trial didn't recruit from nursing homes, it included patients presenting to level 1 trauma centres from anywhere, including nursing homes. Many hip fracture trials would do the same. This is a bit misleading and could be worded better. Something like "one study reported on place of residence (e.g. independent or nursing home).

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

research methodology, clinical trials, systematic reviews, orthopaedic trauma, arthroplasty

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

References

1. : Total Hip Arthroplasty or Hemiarthroplasty for Hip Fracture. N Engl J Med .2019;381(23) : 10.1056/NEJMoa1906190 2199-2208 10.1056/NEJMoa1906190 [DOI] [PubMed] [Google Scholar]
2. : A Trial of Wound Irrigation in the Initial Management of Open Fracture Wounds. N Engl J Med .2015;373(27) : 10.1056/NEJMoa1508502 2629-41 10.1056/NEJMoa1508502 [DOI] [PubMed] [Google Scholar]

NIHR Open Res. 2025 Mar 31.

Stephen Brealey ¹

Thank you for your comments on our article.

1. We comment in the last paragraph of the Discussion that it was more feasible to search by journal and we adapted the terms we used specific to the journal being searched.

2. We comment in the fourth paragraph of the Discussion that a consistent approach is required by committees to allow collection of these data and what is agreed locally by IG professionals. We have added an extra sentence to this paragraph to recognise your important point about how this data capture may be limited by different jurisdictions.

3. The planned meta-analysis to explore heterogeneity in baseline characteristics (i.e. age, gender, ethnicity) between patients screened but not randomised and those randomised was not ultimately feasible as there were too few studies. This analysis was planned to explore the assumption that if the trials were truly inclusive then the participants randomised would be a representative sample of those screened, and as such there should be no difference between the baseline characteristics of these two groups, rather like if they had been formed by random allocation. Therefore, a fixed effects meta-analysis was planned rather than a random effects on the assumption that there was a common treatment estimate (i.e. zero) for the baseline characteristics between these two groups across the trials and the only legitimate source of between study variation is due to chance, which is accounted for in a fixed effects model. Whereas a random effects model assumes and allows for heterogeneity i.e. in this case that the difference between the baseline characteristics of the two groups would vary across the different trials. Following the fixed effects meta-analysis, we would have interpreted the resulting I ²value of heterogeneity in line with the Cochrane handbook guidelines (i.e. 0%–40% might not be important; 30%–60% may represent moderate heterogeneity; 50%–90% may represent substantial heterogeneity; and 75%–100% considerable heterogeneity).

Ref. Hicks A, Fairhurst C, Torgerson DJ. A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;95:55-62.

4. Thank you, we have modified the sentence as suggested.

We understand your concerns about potentially missing some studies, however, our aim was not to undertake a systematic review like that for effectiveness. For the reasons given, we purposefully focused on high impact journals and to keep it feasible with no funding. We have undertaken a comprehensive review and used the PRISMA principles where appropriate to bring rigour to its design and conduct. Inclusivity is an important topic and our take home message is that, even among the highest quality journals, it is difficult for surgeons to make an assessment of whether a trial population is representative of their clinical population. We think this is an important message that would not have been impacted by the methods selected. Therefore, in our opinion, this remains an original and timely contribution to the literature.

NIHR Open Res. 2025 Feb 18. doi: 10.3310/nihropenres.14969.r34609

Reviewer response for version 1

Ameeta Retzer ¹

1. This article describes a review of randomised controlled trials in orthopaedic surgery and whether they are inclusive of their target populations, including under-served groups. This is a timely and interesting study of research inclusion in orthopaedic surgery.

2. The article is clearly presented and appears to cite the appropriate literature.

3. Study design and description of methods - I am not familiar with the methods used to identify the literature through searching journal databases. In the discussion, Hopewell et al are cited for their study on using handsearching versus electronic searching to identify RCT reports to justify use of this method over the use of electronic databases. This reference is from 2007 and I am not sure if this justification still stands as the usual practice to systematically identify literature. While the journals searched are high-impact, it is not clear on what basis these were chosen and it may be that surgery-specific journals or orthopaedics journals would have been more appropriate. I am also not familiar with how journals index their articles so not clear on whether the terms used were appropriate. I would suggest expanding on this in the limitations section of the paper as the methods as they are currently described and justified do not assure that this was the most systematic way to identify the relevant literature to address this research question. Further due to the variation in journalistic styles of each of the journals, there is likely to be variation in the volume of detail in each of the articles e.g one journal may have a more restrictive wordcount than another or a policy of sharing the protocol as a supplementary appendix may have been introduced as a different time. As such, the comparison between two articles from different journals may not be like with like nor would they necessarily yield the necessary information to answer the research question.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

- systematic evaluation of clinical trial protocols and reports, inclusive and equitable research design. I am not an expert in surgical or orthopaedic research.

NIHR Open Res. 2025 Mar 31.

Stephen Brealey ¹

Thank you for your comments on our article. We agree that the method of handsearching has limitations, however, this is not a systematic review like would be done to assess effectiveness, and the focus on specific journals meant we chose to search them directly. The final paragraph of the Introduction and the Discussion explains why we focus on high impact journals rather than surgery-specific journals i.e. these trials are most likely to inform surgical practice and to be the most resourced to address all the aspects of inclusivity of the target population. We chose the specific journals as they are amongst the most cited medical journals and to the authors of this review, they are journals known to have published large scale surgical trials that are likely to influence practice. Similarly, we acknowledge in the last paragraph of the Discussion why we focused on the journal article and Supplementary material, again this is what is likely to be read by surgeons to inform their practice. Journals will have different policies about word count and the extent to which material is available; this, however, is what surgeons access to inform decision-making. We have elaborated a little further on these points in the final paragraph of the Discussion.

Essentially, we were trying to provide a ‘snapshot’ of what could be considered best practice in trial publication, in order to highlight issues relating to inclusivity. We do not expect that wider inclusion criteria would have changed the findings from this review. What we have learned is that, based on trials published in, arguably, the most prestigious and influential journals, it is difficult for clinicians to assess whether the characteristics of those participating in the trial reflects the demographics of the patient population. We think this is an important finding to make known to the research community.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Figshare: Supporting materials for review of high impact journals about inclusivity in orthopaedic surgical randomised trials

This project contains the following underlying data:

Tables 1S to 7S extended data
Applicability of orthopaedic surgical trials review protocol_2023.02.02.docx

Reporting guidelines

This data is available under the terms of the Creative Commons Zero “No rights reserved” data waiver.

[ref-1] 1. Cook JA: The challenges faced in the design, conduct and analysis of surgical Randomised Controlled Trials. Trials. 2009;10: 9. 10.1186/1745-6215-10-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-2] 2. Hanzlik S, Mahabir RC, Baynosa RC, et al. : Levels of evidence in research published in The Journal of Bone and Joint Surgery (American volume) over the last thirty years. J Bone Joint Surg Am. 2009;91(2):425–8. 10.2106/JBJS.H.00108 [DOI] [PubMed] [Google Scholar]

[ref-3] 3. Knowlson C, Dean A, Doherty L, et al. : Recruitment patterns in multicentre randomised trials fit more closely to price's law than the Pareto principle: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. Contemp Clin Trials. 2022;113: 106665. 10.1016/j.cct.2021.106665 [DOI] [PubMed] [Google Scholar]

[ref-4] 4. Rothwell PM: External validity of Randomised Controlled Trials: “to whom do the results of this trial apply?”. Lancet. 2005;365(9453):82–93. 10.1016/S0140-6736(04)17670-8 [DOI] [PubMed] [Google Scholar]

[ref-5] 5. Hopewell S, Hirst A, Collins GS, et al. : Reporting of participant flow diagrams in published reports of randomized trials. Trials. 2011;12: 253. 10.1186/1745-6215-12-253 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-6] 6. Toerien M, Brookes ST, Metcalfe C, et al. : A review of reporting of participant recruitment and retention in RCTs in six major journals. Trials. 2009;10: 52. 10.1186/1745-6215-10-52 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-7] 7. Treweek S, Banister K, Bower P, et al. : Developing the INCLUDE ethnicity framework—a tool to help trialists design trials that better reflect the communities they serve. Trials. 2021;22(1): 337. 10.1186/s13063-021-05276-8 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-8] 8. Witham MD, Anderson E, Carroll C, et al. : Developing a roadmap to improve trial delivery for under-served groups: results from a UK multi-stakeholder process. Trials. 2020;21(1): 694. 10.1186/s13063-020-04613-7 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-9] 9. Blencowe NS, Cook JA, Pinkney T, et al. : Delivering successful randomized controlled trials in surgery: methods to optimize collaboration and study design. Clin Trials. 2017;14(2):211–8. 10.1177/1740774516687272 [DOI] [PubMed] [Google Scholar]

[ref-10] 10. Sterne JAC, Savović J, Page MJ, et al. : RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: I4869. 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]

[ref-11] 11. Fullilove S, Gozzard C: Dorsally displaced fractures of the distal radius: a critical appraisal of the DRAFFT (Distal Radius Acute Fracture Fixation Trial) study. Bone Joint J. 2016;98-B(3):298–300. 10.1302/0301-620X.98B3.36771 [DOI] [PubMed] [Google Scholar]

[ref-12] 12. Nwachukwu BU, Kahlenberg CA, Lehamn JD, et al. : Characteristics of orthopedic publications in high-impact general medical journals. Orthopaedics. 2017;40(3):e405–e12. 10.3928/01477447-20170223-04 [DOI] [PubMed] [Google Scholar]

[ref-13] 13. Schmidtke KA, Evison F, Grove A, et al. : Surgical implementation gap: an interrupted time series analysis with interviews examining the impact of surgical trials on surgical practice in England. BMJ Qual Saf. 2023;32(6):341–56. 10.1136/bmjqs-2022-015077 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-14] 14. Shamseer L, Moher D, Clarke M, et al. : Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350: g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]

[ref-15] 15. Moher D, Liberati A, Tetzlaff J, et al. : Preferred Reporting Items for Systematic reviews and Meta-Analyses: the PRISMA statement. PLoS Med. 2009;6(7): e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-16] 16. Wartolowska K, Judge A, Hopewell S, et al. : Use of placebo controls in the evaluation of surgery: systematic review. BMJ. 2014;348: g3253. 10.1136/bmj.g3253 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-17] 17. Costa ML, Achten J, Parsons NR, et al. : Percutaneous fixation with Kirschner wires versus volar locking plate fixation in adults with dorsally displaced fracture of distal radius: Randomised Controlled Trial. BMJ. 2014;349: g4807. 10.1136/bmj.g4807 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-18] 18. Moher D, Hopewell S, Schulz KF, et al. : CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340: c869. 10.1136/bmj.c869 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-19] 19. NIHR: Improving inclusion of under-served groups in clinical research: guidance from INCLUDE project.2020; [Accessed date 18 January 2022]. Reference Source

[ref-20] 20. Loudon K, Treweek S, Sullivan F, et al. : The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350: h2147. 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]

[ref-21] 21. Hicks A, Fairhurst C, Torgerson DJ: A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;95:55–62. 10.1016/j.jclinepi.2017.10.001 [DOI] [PubMed] [Google Scholar]

[ref-22] 22. Prestmo A, Hagen G, Sletvold O, et al. : Comprehensive geriatric care for patients with hip fractures: a prospective, randomised, controlled trial. Lancet. 2015;385(9978):1623–33. 10.1016/S0140-6736(14)62409-0 [DOI] [PubMed] [Google Scholar]

[ref-23] 23. HIP ATTACK Investigators: Accelerated surgery versus standard care in hip fracture (HIP ATTACK): an international, randomised, controlled trial. Lancet. 2020;395(10225):698–708. 10.1016/S0140-6736(20)30058-1 [DOI] [PubMed] [Google Scholar]

[ref-24] 24. Palmer AJR, Ayyar Gupta V, Fernquest S, et al. : Arthroscopic hip surgery compared with physiotherapy and activity modification for the treatment of symptomatic Femoroacetabular Impingement: multicentre Randomised Controlled Trial. BMJ. 2019;364:l185. 10.1136/bmj.l185 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-25] 25. Beard DJ, Rees JL, Cook JA, et al. : Arthroscopic Subacromial Decompression for subacromial shoulder pain (CSAW): a multicentre, pragmatic, parallel group, placebo-controlled, three-group, randomised surgical trial. Lancet. 2018;391(10118):329–38. 10.1016/S0140-6736(17)32457-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-26] 26. Beard DJ, Davies LJ, Cook JA, et al. : The clinical and cost-effectiveness of total versus Partial Knee Replacement in patients with medial compartment osteoarthritis (TOPKAT): 5-year outcomes of a Randomised Controlled Trial. Lancet. 2019;394(10200):746–56. 10.1016/S0140-6736(19)31281-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-27] 27. Willett K, Keene DJ, Mistry D, et al. : Close contact casting vs surgery for initial treatment of unstable ankle fractures in older adults: a Randomized Clinical Trial. JAMA. 2016;316(14):1455–63. 10.1001/jama.2016.14719 [DOI] [PubMed] [Google Scholar]

[ref-28] 28. Reijman M, Eggerding V, Van Es E, et al. : Early surgical reconstruction versus rehabilitation with elective delayed reconstruction for patients with Anterior Cruciate Ligament rupture: COMPARE Randomised Controlled Trial. BMJ. 2021;372:n375. 10.1136/bmj.n375 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-29] 29. van de Graaf VA, Noorduyn JCA, Willigenburg NW, et al. : Effect of early surgery vs Physical Therapy on knee function among patients with nonobstructive meniscal tears: the ESCAPE Randomized Clinical Trial. JAMA. 2018;320(13):1328–37. 10.1001/jama.2018.13308 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-30] 30. Costa ML, Achten J, Griffin J, et al. : Effect of locking plate fixation vs intramedullary nail fixation on 6-month disability among adults with displaced fracture of the distal tibia: the UK FixDT Randomized Clinical Trial. JAMA. 2017;318(18):1767–76. 10.1001/jama.2017.16429 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-31] 31. Rämö L, Sumrein BO, Lepola V, et al. : Effect of surgery vs functional bracing on functional outcome among patients with closed displaced humeral shaft fractures: the FISH Randomized Clinical Trial. JAMA. 2020;323(18):1792–801. 10.1001/jama.2020.3182 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-32] 32. Ghogawala Z, Terrin N, Dunbar MR, et al. : Effect of ventral vs dorsal spinal surgery on patient-reported physical functioning in patients with cervical spondylotic myelopathy: a Randomized Clinical Trial. JAMA. 2021;325(10):942–51. 10.1001/jama.2021.1233 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-33] 33. Kise NJ, Risberg MA, Stensrud S, et al. : Exercise therapy versus Arthroscopic Partial Meniscectomy for degenerative meniscal tear in middle aged patients: Randomised Controlled Trial with two year follow-up. Br J Sports Med. 2016;50(23):1473–80. 10.1136/bjsports-2016-i3740rep [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-34] 34. Fixation using Alternative Implants for the Treatment of Hip fractures (FAITH) Investigators: Fracture fixation in the operative management of hip fractures (FAITH): an international, multicentre, Randomised Controlled Trial. Lancet. 2017;389(10078):1519–27. 10.1016/S0140-6736(17)30066-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-35] 35. Griffin DR, Dickenson EJ, Wall PDH, et al. : Hip arthroscopy versus best conservative care for the treatment of Femoroacetabular Impingement syndrome (UK FASHIoN): a multicentre Randomised Controlled Trial. Lancet. 2018;391(10136):2225–35. 10.1016/S0140-6736(18)31202-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-36] 36. Ghogawala Z, Dziura J, Butler WE, et al. : Laminectomy plus fusion versus laminectomy alone for lumbar spondylolisthesis. N Engl J Med. 2016;374(15):1424–34. 10.1056/NEJMoa1508788 [DOI] [PubMed] [Google Scholar]

[ref-37] 37. Rangan A, Brealey SD, Keding A, et al. : Management of adults with primary frozen shoulder in secondary care (UK FROST): a multicentre, pragmatic, three-arm, superiority randomised clinical trial. Lancet. 2020;396(10256):977–89. 10.1016/S0140-6736(20)31965-6 [DOI] [PubMed] [Google Scholar]

[ref-38] 38. Griffin D, Parsons N, Shaw E, et al. : Operative versus non-operative treatment for closed, displaced, intra-articular fractures of the calcaneus: Randomised Controlled Trial. BMJ. 2014;349: g4483. 10.1136/bmj.g4483 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-39] 39. Försth P, Ólafsson G, Carlsson T, et al. : A randomized, controlled trial of fusion surgery for lumbar spinal stenosis. N Engl J Med. 2016;374(15):1413–23. 10.1056/NEJMoa1513721 [DOI] [PubMed] [Google Scholar]

[ref-40] 40. Skou ST, Roos EM, Laursen MB, et al. : A randomized, controlled trial of total knee replacement. N Engl J Med. 2015;373(17):1597–606. 10.1056/NEJMoa1505467 [DOI] [PubMed] [Google Scholar]

[ref-41] 41. Clark W, Bird P, Gonski P, et al. : Safety and efficacy of vertebroplasty for acute painful osteoporotic fractures (VAPOUR): a multicentre, randomised, double-blind, placebo-controlled trial. Lancet. 2016;388(10052):1408–16. 10.1016/S0140-6736(16)31341-1 [DOI] [PubMed] [Google Scholar]

[ref-42] 42. Paavola M, Malmivaara A, Taimela S, et al. : Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: randomised, placebo surgery controlled clinical trial. BMJ. 2018;362: k2860. 10.1136/bmj.k2860 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-43] 43. Dias JJ, Brealey SD, Fairhurst C, et al. : Surgery versus cast immobilisation for adults with a bicortical fracture of the scaphoid waist (SWIFFT): a pragmatic, multicentre, open-label, randomised superiority trial. Lancet. 2020;396(10248):390–401. 10.1016/S0140-6736(20)30931-4 [DOI] [PubMed] [Google Scholar]

[ref-44] 44. Bailey CS, Rasoulinejad P, Taylor D, et al. : Surgery versus conservative care for persistent sciatica lasting 4 to 12 months. N Engl J Med. 2020;382(12):1093–102. 10.1056/NEJMoa1912658 [DOI] [PubMed] [Google Scholar]

[ref-45] 45. Costa ML, Achten J, Ooms A, et al. : Surgical fixation with K-wires versus casting in adults with fracture of distal radius: DRAFFT2 multicentre randomised clinical trial. BMJ. 2022;376: e068041. 10.1136/bmj-2021-068041 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-46] 46. Rangan A, Handoll H, Brealey S, et al. : Surgical vs nonsurgical treatment of adults with displaced fractures of the proximal humerus: the PROFHER randomized clinical trial. JAMA. 2015;313(10):1037–47. 10.1001/jama.2015.1629 [DOI] [PubMed] [Google Scholar]

[ref-47] 47. Firanescu CE, De Vries J, Lodder P, et al. : Vertebroplasty versus sham procedure for painful acute osteoporotic vertebral compression fractures (VERTOS IV): randomised sham controlled clinical trial. BMJ. 2018;361(361): k1551. 10.1136/bmj.k1551 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-48] 48. Wallace N, O'Keeffe S, Gardner H, et al. : Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability. J Clin Epidemiol. 2023;162:81–9. 10.1016/j.jclinepi.2023.08.008 [DOI] [PubMed] [Google Scholar]

[ref-49] 49. Jeraj S, Butt J: Musculoskeletal conditions and Black, Asian and minority ethnic people: addressing health inequalities.Race Equality Foundation;2020. Reference Source

[ref-50] 50. Health Research Authority: Public perceptions of research. 2023; [updated 21 Nov 2023]. Reference Source

[ref-51] 51. Wilson C, Rooshenas L, Paramasivan S, et al. : Development of a framework to improve the process of recruitment to Randomised Controlled Trials (RCTs): the SEAR (Screened, Eligible, Approached, Randomised) framework. Trials. 2018;19(1): 50. 10.1186/s13063-017-2413-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-52] 52. Davies L, Beard D, Cook JA, et al. : The challenge of equipoise in trials with a surgical and non-surgical comparison: a qualitative synthesis using meta-ethnography. Trials. 2021;22(1): 678. 10.1186/s13063-021-05403-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-53] 53. Scantlebury A, McDaid C, Brealey S, et al. : Embedding qualitative research in Randomised Controlled Trials to improve recruitment: findings from two recruitment optimisation studies of orthopaedic surgical trials. Trials. 2021;22(1): 461. 10.1186/s13063-021-05420-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-54] 54. Crocker JC, Ricci-Cabello I, Parker A, et al. : Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ. 2018;363: k4738. 10.1136/bmj.k4738 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-55] 55. Health Research Authority: Types of health and care information and the legal frameworks protecting them. 2022; [updated 5 Dec 2022]. Reference Source

[ref-56] 56. Shepherd V, Joyce K, Flynn S, et al. : INCLUDE impaired capacity to consent framework.2022; [Accessed date 12 July 2023]. Reference Source

[ref-57] 57. Hopewell S, Clarke M, Lefebvre C, et al. : Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev. 2007;2007(2): MR000001. 10.1002/14651858.MR000001.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-58] 58. NIHR: Best research for best health: the next chapter. National Institute of Health Research,2021. Reference Source

[ref-59] 59. Shiely F, Rychlíčková J, Kubiak C, et al. : Training and education on inclusivity in clinical trials—the SENSITISE project. Trials. 2024;25(1): 318. 10.1186/s13063-024-08150-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-60] 60. Biggs K, Hullock K, Dix C, et al. : Time to STEP UP: methods and findings from the development of guidance to help researchers design inclusive clinical trials. BMC Med Res Methodol. 2024;24(1): 227. 10.1186/s12874-024-02342-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-61] 61. NIHR: Inclusive research design to become an NIHR condition of funding. [Accessed date 16 October 2024]. Reference Source

[ref-62] 62. Health Research Authority: HRA and MHRA draft inclusion and diversity guidance. [Accessed date 16 October 2024]. Reference Source

[ref-63] 63. Vayena E, Blasimme A, Sugarman J: Decentralised clinical trials: ethical opportunities and challenges. Lancet Digit Health. 2023;5(6):e390–e4. 10.1016/S2589-7500(23)00052-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-64] 64. Dawson S, Banister K, Biggs K, et al. : Trial forge guidance 3: randomised trials and how to recruit and retain individuals from ethnic minority groups—practical guidance to support better practice. Trials. 2022;23(1): 672. 10.1186/s13063-022-06553-w [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-65] 65. Averbuch T, Mohamed MO, Islam S, et al. : The association between Socioeconomic Status, sex, race / ethnicity and in-hospital mortality among patients hospitalized for heart failure. J Card Fail. 2022;28(5):697–709. 10.1016/j.cardfail.2021.09.012 [DOI] [PubMed] [Google Scholar]

[ref-66] 66. Cheng AL, Bradley EC, Brady BK, et al. : The influence of race, sex, and social disadvantage on self-reported health in patients presenting with chronic musculoskeletal pain. Am J Phy Med Rehabil. 2022;101(3):211–6. 10.1097/PHM.0000000000001774 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref-67] 67. Brealey S: Supporting materials for review of high impact journals about inclusivity in orthopaedic surgical randomised trials. figshare. 2024; [cited 2025 Jan 6]. https://figshare.com/articles/journal_contribution/Detailed_description_of_the_eligibility_criteria_and_patient_population/27074599/3 [Google Scholar]

PERMALINK

Inclusivity of the target population in orthopaedic surgical randomised trials: a review of high impact journals

Stephen D Brealey

Lucy Atha

Catherine Knowlson

Elizabeth Cook

Kate Hicks

Joanne Newman

Arabella Scantlebury

Joy Adamson

Caroline Fairhurst

Nick A Johnson

Joseph J Dias

Roles

Version Changes

Revised. Amendments from Version 1

Abstract

Background

Methods

Results

Conclusions

Plain Language Summary

Introduction

Methods

Patient and Public Involvement

Eligibility criteria

Information sources and search strategy

Study selection

Data extraction

Data items

Data synthesis

Results

Study selection

Figure 1. Flowchart of included studies.

Study characteristics

Completion of the CONSORT statement

Table 1. CONSORT statement flowchart.

Description of key baseline characteristics

Inclusion of under-served populations

Methods to facilitate consent

Pragmatic selection of trial participants

Table 2. Pragmatic selection of patients into the included studies.

Discussion

Conclusion

Ethics and consent

Acknowledgements

Funding Statement

Data availability

Underlying data

Extended data

Reporting guidelines

Author contributions

References

Reviewer response for version 2

Usman Ali

Roles

References

Stephen Brealey

Reviewer response for version 2

Ameeta Retzer

Roles

Reviewer response for version 1

Kim Madden

Roles

References

Stephen Brealey

Reviewer response for version 1

Ameeta Retzer

Roles

Stephen Brealey

Associated Data

Data Availability Statement

Underlying data

Extended data

Reporting guidelines

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles