Evaluation of methodology, reporting and transparency of 50 highly cited randomised trials in neurology: a cross-sectional analysis

Abstract

Background

Highly cited randomised controlled trials (RCTs) have a high impact on medical decision making. However, their quality has not been systematically evaluated. This cross-sectional study evaluates the methodological and reporting quality, and aspects of transparency of 50 highly cited RCTs in the field of neurology published between 2018 and 2022.

Methods

We searched Web of Science to identify the 10 most-cited neurology RCTs per year from 2018 to 2022. Methodological quality was assessed using the Revised Cochrane Risk-of-Bias tool (RoB 2), and reporting quality was evaluated based on the Consolidated Standards of Reporting Trials (CONSORT) 2010 Statement. Additionally, we analysed data sharing statements, document availability and trial registration details according to current recommendations and regulations. Two independent reviewers evaluated each study.

Results

Included studies were mostly multicentred multinational (36/50) RCTs; 25/50 investigated stroke. We rated 16% (8/50) as low risk of bias, 72% (36/50) with some concerns and 12% (6/50) as high risk. Reporting quality was generally adequate for intervention description and primary outcome analysis (>80% reported), but lacked detail on randomisation and trial design (<50% reported). Data sharing statements were provided in 36/50 publications. Most RCTs provided protocol (48/50) and statistical analysis plan (46/50) while analytic code was rarely available (8/50) for most publications. Most studies were prospectively registered (41/50). Yet, only a minority (18/50) documented results in the registry within 12 months.

Conclusions

Most studies had some concerns in risk of bias evaluation and a generally acceptable reporting quality. However, quality of methods reporting, results documentation in registries and analytic code access need improvement.

Trial registration

Open Science Framework (OSF) (DOI 10.17605/OSF.IO/4RV7A).

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study followed a standardised approach throughout all steps of this methodological study according to a prospectively registered protocol and study selection, data extraction and assessments were performed independently by two reviewers.

• Methodological quality, reporting quality and several facets of transparency, including trial registration, results posting, and protocol, analytic code and data availability, were assessed.

• Risk of bias, completeness of reporting and transparency were assessed using standardised instruments, such as the revised Cochrane risk-of-bias tool (RoB 2) and a checklist based on the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement and its extension for abstracts.

• The 10 most-cited eligible trials per year from 2018 to 2022 were identified based on citation rank.

• All assessments were based on publicly accessible sources without contacting trial authors, so that risk-of-bias judgements and evaluation of reporting and transparency reflect the information available in the published record.

Background

Neurological disorders, such as stroke and neurodegenerative diseases, pose a substantial global health burden because of their high prevalence and significant impact on morbidity and mortality.¹ Consequently, clinical research in the neurological field plays a critical role in advancing therapeutic strategies. High-quality randomised controlled trials (RCTs) are the recognised methodological standard for evaluating the efficacy and safety of interventions, and form the basis of evidence-based guidelines and clinical decision-making.²

Nevertheless, the scientific and clinical value of an RCT is intrinsically linked to the quality of its methods and reporting, and its transparency. The risk of bias of results from RCTs is frequently rated using the revised Cochrane risk-of-bias tool for randomised trials (RoB 2).³ However, risk-of-bias assessment is often hampered by incomplete or inadequate reporting and interpretation of study results requires adequate quality of reporting of study design and context.⁴ Reporting guidelines like the Consolidated Standards of Reporting Trials (CONSORT) Statement⁵ provide checklists on information that authors should disclose in journal publications of RCTs. Inadequate trial conduct or insufficient reporting practices may result in misleading interpretation and inappropriate application, which could, in turn, compromise patient safety and care quality.⁶

In addition, preregistration of clinical trials transparently ensures a clear, predefined study plan⁷ and grants public access to all important trial information.⁸ Properly maintained registry entries thereby prevent unnecessary redundant research, waste of resources, imprecision and distortion of results in systematic reviews by publication bias.^7–10 Although many guidelines and committees (eg, the Declaration of Helsinki¹¹ and the International Committee of Medical Journal Editors (ICMJE)^{10 12}) require prospective registration of studies, implementation varies greatly.^13–16

Sharing of data and analytic code on the other hand allows independent re-analysis of data and thereby contributes to transparency. Lack of reproducibility of study results has led to controversial discussions referred to as the replication crisis.^17–22 Individual participant data can also be used to test post hoc hypotheses and has advantages when used for meta-analysis of different clinical trials.²³

Within the scientific community, citation counts are often used as a proxy for the impact or relevance of a publication.^24–27 Rankings are established and recognition is awarded according to citation scores.^25–28 The ‘impact factor’, a metric based on average citation of articles of a journal,²⁹ is often used as a proxy for research impact, also in the field of neurology.^{30 31} Some institutions even use bibliometric analysis as a marker for scientific quality comparable to peer review.³² As highly cited trials are by definition disproportionately often represented in subsequent literature—they appear in systematic and narrative reviews as well as clinical practice guidelines—it can be assumed that they shape clinical decision making or future research directions in particularly relevant fields.

However, there is growing concern that citation frequency of a study does not necessarily correlate with research quality.³³ To our knowledge, no comprehensive evaluation examining reporting, risk of bias and aspects of transparency in contemporary, highly cited neurological RCTs has been conducted so far. Previous research has addressed selected aspects of methodological quality, reporting practices or transparency aspects in neurological RCTs, focusing predominantly on older studies or restricted samples of journals.^{14 34–36}

This methodological cross-sectional study aims to evaluate the risk of bias for the primary outcome and the implementation of current transparency and reporting standards in 50 highly cited RCTs in neurology, defined as the 10 most-cited trials per year published from 2018 to 2022.

Methods

The protocol of this study was prospectively registered in the Open Science Framework (OSF)³⁷ and is disclosed in the online supplemental appendix.

Search

On 17 February 2025, we conducted separate searches restricted to publications of each year from 2018 to 2022 in the Web of Science Database. We used an RCT filter³⁸ but applied no restriction to category or journal and sorted the results by citations (highest first) and exported the first search results with title and abstract in batches of 500–1000 publications. We included human RCTs with primary focus on treatment or prevention of a neurological disease and focusing on a clinical outcome. Studies on conditions with multiple manifestations (eg, tumour metastasis without focus on brain), secondary analyses of RCTs or dose-finding studies were excluded. After piloting with results of a previous year, two reviewers (AB, ACB or ES) independently screened titles and abstracts and retrieved full texts of potentially relevant publications sequentially in descending order of citation until the 10 eligible most-cited RCTs of each year (from 2018 to 2022, date of print-publication) had been identified. Discrepancies were resolved by discussion or consultation of a third reviewer (JS or JJM). Details on inclusion and exclusion criteria as well as the search strategy are provided in the protocol in the online supplemental appendix (p. 32, p. 36).

Data extraction

Two independent reviewers (AB, ACB or ES) extracted data into standardised and piloted Microsoft Excel tables. We extracted data on authors and journals, characteristics on study design, setting, population and results, and information on funding and conflicts of interest. The exact data items extracted are specified in the online supplemental appendix (p. 33). Discrepancies were resolved by discussion or consultation of a third reviewer. All assessments were based exclusively on publicly accessible sources.

Assessment of risk of bias

Following a piloting phase, the risk of bias for one primary outcome was assessed for each study using the revised Cochrane risk-of-bias tool for randomised trials (RoB 2)³ by two independent reviewers who had expertise and had received structured training in this tool (AB, ACB, ES or JS). Assessment was based on the publication and directly linked supplementary material including protocols and statistical analysis plans. Domain and overall risk of bias assessment followed the supplied algorithms after using the RoB 2 template with signalling questions. Signalling questions were answered with response options ‘Yes’, ‘Probably yes’, ‘No’, ‘Probably no’ and ‘No information’ according to the RoB 2 guidance document and cribsheet. For domain 2, the effect of assignment to intervention was evaluated. In case of multiple primary outcomes, the first-mentioned primary outcome was chosen. Discrepancies were resolved by discussion or consultation of a third reviewer.

Assessment of completeness of reporting

We used the CONSORT 2010 Statement³⁹ and CONSORT for reporting randomised trials in journal and conference abstracts⁴⁰ to assess completeness of reporting. Each item was evaluated according to a standardised and piloted checklist built on the description of the items in CONSORT 2010 explanation and elaboration,⁴ CONSORT for reporting in abstracts explanation and elaboration⁴¹ and the COBPeer-Tool.⁴² Each item was rated either as ‘completely reported’, ‘partly reported’ or ‘not reported’. For most items (all except for 1a–2b, 8a, 20–23 and 25), supplementary material as well as directly linked and accessible protocols were included. For items 6a, 6b, 12a, 16, 17a and 17b, emphasis was placed on the primary outcome. Assessment was performed by two reviewers independently (AB, ACB, ES or JS), with discrepancies resolved by discussion or consultation of a third reviewer.

Assessment of aspects of transparency

Two independent reviewers (AB, ACB or ES) assessed study registration dates and result documentation in the referenced trial register entries for completeness and timing of documentation in accordance with best practices as defined by regulation and current standards.^{10 43–45} Availability of additional documents (protocol, statistical analysis plan, analytic code, clinical study report, informed consent form) and data sharing statements in the publication and registry were examined according to the requirements of the ICMJE.¹⁰ Results were documented in standardised piloted Microsoft Excel tables. Discrepancies were resolved by discussion or consultation of a third reviewer. All data items analysed are specified in the protocol provided in the online supplemental appendix (p. 34).

Statistical analysis

Statistical analysis was conducted in R V.4.5.0 (R Foundation for Statistical Computing, Vienna, Austria) with the use of the packages tidyverse and readxl and in Microsoft Excel 2016 with the functions COUNTIF, MEDIAN, MAX and MIN. Numbers and percentages were calculated for extracted or evaluated items and displayed in bar charts. For stratified analysis, results were displayed side by side or in multi-panel plots. Due to the descriptive nature of our analyses, no formal hypothesis testing was applied. The R code for analysis is disclosed in the online supplemental appendix (p. 14 ff).

Specifications to protocol

We excluded phase 2 RCTs that primarily focus on dose finding and safety since they do not focus on treatment or prevention of a neurological disease. If one publication reported on more than one RCT, the trial with the largest number of randomised participants was included. Further specifications to the protocol are disclosed in the online supplemental appendix (p. 13).

Results

Study selection

Figure 1 shows the process of study selection according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria⁴⁶ for the separate searches for the years 2018–2022. In total, from 763 658 search results, 2001 were screened on title-abstract and 88 on full-text level. Numbers needed to screen increased in 2020 and 2021 due to highly cited COVID-19-research in these years.^{47 48} Finally, we included 50 publications, which are listed in online supplemental table S1.^49–98 Publications excluded in the full-text screening are listed in online supplemental table S2.

Figure 1. Study selection according to the PRISMA 2020 flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCTs, randomised controlled trials.

Study characteristics

Key characteristics of included studies are summarised in table 1. The most represented journals are the New England Journal of Medicine (n=28) and The Lancet (n=11). First and last authors had affiliations to institutions in North America (60%), Europe (44%), Asia (22%) or Australia (2%). Most RCTs investigated treatment for ischaemic stroke (n=25). Interventions were mostly pharmacological (n=38) or endovascular (n=8). The vast majority of studies (96%) restricted study inclusion to adults (≥18 years).

Table 1. Characteristics of included studies.

Study characteristics	N (%)/Median (range)
Treated conditions
Ischaemic stroke	25 (50)
Alzheimer’s disease	4 (8)
Neuromyelitis optica spectrum disorder	4 (8)
Glioblastoma	3 (6)
Amyotrophic lateral sclerosis	2 (4)
Multiple sclerosis	2 (4)
Hereditary transthyretin amyloidosis with polyneuropathy	2 (4)
Others	8 (16)
Participants
Adults (≥18 years)	48 (96)
Children (<18 years)	1 (2)
Children and adults	1 (2)
Type of intervention
Pharmacological	38 (76)
Endovascular	8 (16)
Surgery	1 (2)
Other	3 (6)
Type of RCT
Parallel	49 (98)
Factorial	1 (2)
Number of study arms
2	46 (92)
≥3	4 (8)
Allocation ratio
1:1/1:1:1/1:1:1:1	40 (80)
2:1	9 (18)
3:1	1 (2)
Study design
Superiority	43 (86)
Non-inferiority†	7 (14)
Placebo-/Sham-controlled
Yes	30 (60)
No	20 (40)
Recruitment
Number of centres	47 (7–564)
Multicentre studies	50 (100)
Number of participating countries	8 (1–42)
Multinational studies	36 (72)
Randomised participants	329 (83–12 737)
Analysed participants	320 (60–11 016)
Analysed participants of all randomised participants	99.7 (47.5–100)
Geographic region of participating centres*
North America	35 (70)
Europe	35 (70)
Asia	30 (60)
Australia	20 (40)
South/central America	12 (24)
Africa	5 (10)
Geographic region of first/last author affiliation*
North America	30 (60)
Europe	22 (44)
Asia	11 (22)
Australia	1 (2)
Type of funding
Industry/private	26 (52)
Public	13 (26)
Both (industry and public)	11 (22)
Involvement of funder in study
Yes	27 (54)
No	20 (40)
Unclear	3 (6)
Potential conflict of interest disclosed
To industry	18 (36)
To public donors	1 (2)
To both (industry and public donor)	30 (60)
None	1 (2)
Potential conflict of interest to study funder
Yes	38 (76)
No	12 (24)
Premature termination of study (reason)
No	31 (62)
Yes	18 (36)
Study success	6 (12)
Futility/safety concern	4 (8)
Poor recruitment	3 (6)
Lack of funding	2 (4)
Other reason	3 (6)
Unclear	1 (2)
Primary study outcome
Positive	33 (66)
Negative	17 (34)

^*Studies might be conducted in more than one geographic region and authors might be affiliated to different institutions from different world regions. That is why the below-mentioned options do not add up to 100%.

^†Testing for non-inferiority can include a second test for superiority if non-inferiority is shown.

All studies were multicentred (median: 47 centres) and 36 were multinational studies (median: 8 countries). Studies were predominantly conducted in North America (70%), Europe (70%), Asia (60%) and Australia (40%). The median number of participants randomised was 329 (range: 83–12 737). About one third of the studies (36%) were terminated early, mostly for study success, but also for futility/safety concerns, poor recruitment or lack of funding. About two thirds of the studies (66%) had a statistically significant positive primary outcome.

In total, 46 of the included RCTs had two study arms (92%) and 49 a parallel design (98%). Moreover, 43 RCTs were primarily designed to test superiority, the other seven assessed non-inferiority. Most studies (60%) were placebo-controlled/sham-controlled. Funding originated from industry (n=26), public donors (n=13) or both (n=11). In 27 studies, the funder was involved in trial design and conduct and in 38 studies the authors disclosed potential conflicts of interest with regard to the study funder.

Risk of bias

Overall, 8/50 (16%) studies were rated with a low risk of bias, 36/50 (72%) with some concerns and 6/50 (12%) with high risk of bias for the primary outcome (figure 2). In each domain, 60%–70% of the studies were rated with low risk of bias, 26%–38% with some concerns and 0%–4% with high risk of bias. In domain 1, differences in group characteristics were the main drivers of ‘some concerns’ rating, while in domains 2 and 4, open-label design with issues linked to blinding led to some concerns in many studies. Where systematic differences between the groups were likely to have influenced the outcome, the judgement was even ‘high risk of bias’. Risk of bias for domain 3 was largely shaped by missing outcome data or imbalances in outcome data. Absence of or deviations from protocol and statistical analysis plan were reasons for some concerns in domain 5. Important changes from protocol or statistical analysis plan that may probably reflect a retrospective selection of the primary outcome definition resulted in high risk of bias judgements in two cases.

Figure 2. Risk of bias of included studies.

Looking at risk-of-bias ratings stratified by type of intervention (pharmacological vs non-pharmacological), there was little difference in overall risk of bias (figure 3). However, for domains where blinding is relevant, pharmacological interventions were rated with a lower bias risk (domain 2: 79% low risk, domain 4: 82% low risk) compared with non-pharmacological interventions (domain 2: 8% low risk, domain 4: 25% low risk). In contrast, in domain 3 (Risk of bias due to missing outcome data), pharmacological interventions performed worse (53% low risk) compared with non-pharmacological interventions (83% low risk). In domains 1 and 5, only minor differences between intervention types were observed.

Figure 3. Risk of bias stratified by type of intervention.

The distribution of overall risk of bias ratings was similar across industry and public funded RCTs. In the risk of bias domains, similar patterns as in the comparison of pharmacological versus non-pharmacological treatment occur when comparing industry versus public funding (online supplemental figure S1).

Quality of reporting

The quality of reporting of the CONSORT items is presented in figure 4.

Figure 4. Quality of reporting according to the CONSORT 2010 statement. CONSORT, Consolidated Standards of Reporting Trials.

The vast majority of studies sufficiently reported items regarding description of interventions (item 5: 48/50), (primary) outcome measurement (item 6a: 47/50; item 6b: 26/29), outcome analysis (items 12a: 50/50; item 12b: 41/47), results of the outcome (item 15: 49/50; item 16: 46/50; item 17a: 49/50) and results interpretation (item 22: 45/50). Also, the vast majority of included studies shared enough details about sample size calculation (item 7a: 47/50), defined period of recruitment and follow-up (item 14a: 40/50), allowed assumptions about trial termination (item 14b: 47/50) and provided information on trial registration (item 23: 50/50), study protocols (item 24: 43/50) and funding (item 25: 44/50).

In contrast, only a minority of the included studies reported enough details on trial design and explicitly named the analysis framework (superiority vs non-inferiority, parallel vs factorial) (item 3a: 19/50) or provided details on participants including methods of recruitment (item 4a: 20/50). Randomisation details, like restriction to randomisation sequence with stratifications, block sizes, etc (item 8b: 19/50), or the detailed steps of allocation concealment (item 9: 19/50) and implementation (item 10: 21/50) were rarely described. Most studies did not give a complete study hypothesis or objective containing information on participants, intervention, comparison and outcome (item 2b: 20/50), and most studies with binary outcomes failed to report both relative and absolute effect measures (item 17b: 4/20). Also, many studies did not completely address potential harms and benefits of the intervention in the introduction (item 2a: 12/50). Additionally, study limitations and especially clinically important differences (item 20: 22/50) were not sufficiently addressed in the discussion.

Less than half of the studies identified the trial as ‘randomised’ in the title (item 1a: 22/50) and none provided all information required by the CONSORT Checklist for abstracts⁴⁰ (item 1b: 0/50). Especially trial design (2/50), participants with age and setting (12/50), the objective with information on participants, intervention and comparator (11/50), the method of randomisation and allocation (6/50) and blinding of participants, caretakers and outcome assessors (10/50) were poorly described in the abstract. On the other hand, trial registration (47/50) and authors with contact details (50/50) were well reported. Figure 5 shows the completeness of reporting within the journal abstracts according to the criteria from the CONSORT Checklist for abstracts.⁴⁰

Figure 5. Quality of reporting in the abstracts according to the CONSORT statement for reporting in abstracts. CONSORT, Consolidated Standards of Reporting Trials.

In online supplemental figure S2, implementation of CONSORT items in title and abstract is presented stratified by the two journals with the most included publications (The New England Journal of Medicine and The Lancet) and showed some variation with tendency towards better reporting in The Lancet. When looking at the full text, in the other items from the CONSORT Checklist³⁹ most ratings are conserved between the journals (online supplemental figure S3).

Aspects of transparency

In total, 36/50 of the included studies provided a detailed data sharing statement, and two studies referred to the sponsor’s website for the data sharing policy (figure 6). Of these, 28/38 assured access to individual deidentified participant data, while three did not plan to share individual participant data at all. Seven studies planned to grant access to all collected data, 12 studies only to data underlying reported outcomes, 10 studies to other subsets of individual participant data and 6 studies did not report the kind of individual participant data they would share (figure 6). In the registry, half (25/50) of the studies provide a data sharing statement, but only one quarter specifies how and what data they would share in a data sharing plan (figure 6). A data sharing statement was provided in 80%–100% of the publications from 2019 to 2022, but none of the publications from 2018 (online supplemental figure S4). In the registry, however, data sharing statement/data sharing plan availability did not change much over time and remains lower than in the publications (online supplemental figure S4).

Figure 6. Data sharing statement and data availability.

Study protocols could be retrieved for 48/50 of the included studies. Of those, 40 provided a prospective protocol version or a detailed summary of changes in the current protocol version. A statistical analysis plan was available for 46/50 of the included studies, and the analytic code for 8/50. Also, availability of clinical study reports (1/50) and informed consent forms (3/50) is low. Details on document availability are provided in figure 7.

Figure 7. Document availability.

All included RCTs were registered (figure 8), mostly in ClinicalTrials.gov (n=47) and/or EudraCT (n=5). Overall, 82% (41/50) were prospectively registered before the first participant was recruited. In 33/50 of the included RCTs, results were documented in the registry, in 18 studies of these within 12 months after primary completion (figure 8). Reference to the publication was provided in 96% (48/50) and reference to the study protocol in 80% (40/50) of the registry entries.

Figure 8. Completeness of study registration.

Compared with non-pharmacological interventions, studies on pharmacological interventions were slightly more often prospectively registered (33/38 vs 8/12) and had results documented (28/38 vs 5/12) (online supplemental figure S5). If results were documented in the registry, this was within 12 months in all non-pharmacological trials (5/5), but only in about half of the pharmacological trials (13/28) (online supplemental figure S5). Studies financed by industry/private donors included study protocol references and reported results more often than publicly funded RCTs, but not more often within 12 months (online supplemental figure S6).

Discussion

Our study offers a comprehensive overview of the methodological quality of highly cited RCTs in the field of neurology published from 2018 to 2022, focusing on risk of bias, reporting quality, data and document availability and study registration. Most of the included studies were rated with some concerns regarding the overall risk of bias. Quality of reporting varied across the items of the CONSORT Checklist: reporting of interventions, analysis and results as well as funding, protocol and registry was good, but reporting of randomisation details, background and discussion was largely insufficient. No study reported all items of the CONSORT for reporting randomised trials in journal and conference abstracts.⁴⁰ Data sharing statements were provided in most journal publications published in or after 2019 and for most RCTs protocols and statistical analysis plans were available, while only few studies provided parts of analytic code. Most studies were prospectively registered and about two thirds had results documented in the registry—but only a minority within 12 months after primary completion.

In comparison to average RCTs included in Cochrane Reviews from 2011 to 2014⁹⁹ or a sample of RCTs from leading neurology journals in 2008 and 2013,³⁴ the studies in our sample appear to have lower risk of bias. In non-pharmacological trials, the above-average concerns in bias due to deviations from the intended interventions and in measurement of the outcome might be a result of difficulties in blinding participants, healthcare personnel and outcome assessors—an important aspect in these domains.³ Of note, for participant-reported outcomes, participants are regarded as the outcome assessors.¹⁰⁰ The higher risk of bias due to missing outcome data in pharmacological trials might be explained by longer time spans until outcome measurement—non-pharmacological trials with fixed follow-up times had (with few exceptions) primary outcome assessment time points of 90 days post-stroke while nearly half of non-event-driven pharmacological trials measured the primary endpoint after 24 weeks and ranging up to 104 weeks. Additionally, primary outcomes in pharmacological trials were more often event-driven than in non-pharmacological trials.

Although most high-impact neurology journals require or recommend CONSORT as a reporting guideline for RCTs,¹⁰¹ meta-research studies on quality of reporting of neurological RCTs are scarce and limited to for example, specific disease conditions¹⁰² or samples of Spanish and Latin American journals.³⁵

Previous cross-sectional studies also evaluated reporting quality of RCTs in other medical fields.^103–107 Though better reported than in trials in other medical fields (COVID-19,^{103 104} acupuncture^{105 106} and hepatitis¹⁰⁷), our results suggest that improvements are needed in the description of study methods—especially regarding randomisation details that are needed for risk of bias evaluation.³ Also, information on funding, protocol and registry was better reported than in trials published on COVID-19^{103 104} or in critical care medicine.¹⁰⁸ Reporting quality in abstracts was better in our sample than in COVID-19 studies¹⁰⁹ and similar to other older studies from high-impact journals.¹¹⁰ The observed differences in title and abstract reporting between journals could be explained by word count restrictions.^{111 112}

Data sharing statements were made mandatory by the ICMJE in full text publication and registry submitted from July 2018/January 2019,¹⁰ which explains the high rates of data sharing statements in publications (but not in registries) from 2019 onwards. Other studies also suggested high rates of declared data sharing after implementation of ICMJE regulation,¹¹³ though a gap to actual data availability remains.^{113 114} Rates of data sharing statements and sharing of protocols and statistical analysis plans were higher in our sample than among COVID-19 RCTs.¹¹⁵ Most studies were prospectively registered and about two thirds had results documented in the registry. Though required by European and US regulation for applicable medical trials,^{43 44} only a minority of the included studies documented results within 12 months after primary completion. However, applicability of the included RCTs to European and US regulation was not assessed in this study.

Although citation count does not necessarily reflect clinical and scientific impact, several of the included trials directly changed clinical management. Studies like DAWN,⁸¹ DEFUSE-3,⁵⁰ POINT,⁶⁹ WAKE-UP,⁹⁰ EXTEND,⁷⁶ THALES⁶⁸ changed clinical practice in stroke.^{116 117} Others, such as APOLLO,⁴⁹ NEURO-TTR,⁵² CHERISH⁷⁸ or N-MOmentum⁵⁵ and PREVENT⁸⁴ provided pivotal evidence for new pharmacological agents for treatment of hereditary transthyretin amyloidosis with polyneuropathy,^{118 119} spinal muscular atrophy¹²⁰ or neuromyelitis optica spectrum disorder.¹²¹ Conversely, some studies also reported neutral results and thereby contested current understanding of disease mechanisms: results from the CheckMate 143 trial⁸⁵ questioned the understanding of immune tumour microenvironment in glioblastoma,¹²² EXPEDITION3⁶⁶ and EMERGE/ENGAGE⁵⁴ led to a fundamental discussion about targeting amyloid-β for treatment of Alzheimer’s disease.^{123 124} Highly cited trials shape the field by developing new treatments and changing practice as well as by driving scientific and methodological debate, which underscores the importance of methodological rigour and high transparency standards assessed in this study.

This study exhibits methodological strengths that contribute to robustness and reliability of its findings. The assessment was grounded in high-quality methods, including a highly sensitive search strategy, the use of two independent reviewers with experience in evaluation of RCTs and the application of a rigorous and standardised evaluation process. Additionally, the selection of an up-to-date sample comprising recent, highly cited neurological RCTs allowed for a relevant analysis of current practices in methodology, reporting and transparency.

Nevertheless, several limitations need to be acknowledged. This study focused mostly on primary outcomes as the most important outcomes by definition, potentially overlooking issues related to secondary outcomes. Furthermore, the reliance on registry entries as reported in trial publications may have missed additional registry data. The use of a commercial citation tracking introduces the possibility of incomplete or biased citation data. Although a number of included studies influenced clinical decision making, citation counts alone may not necessarily reflect scientific or clinical impact. Additionally, our pre-specified approach of identifying the 10 most-cited RCTs per year 2018–2022 in the field of neurology is not equivalent to the 50 most-cited trials in this time span. However, it represents a robust sample of highly cited trials and avoids bias due to accumulating citation counts. To avoid bias due to possible systematic differences in responsiveness and to assess risk of bias, reporting and transparency from a reader’s perspective, assessment was based exclusively on publicly accessible material. Evaluation of reporting quality was based on the CONSORT 2010 statement,³⁹ which was standard at the time that the included trials were published. However, with the recent release of the CONSORT 2025 update,⁵ certain aspects of reporting may need to be re-evaluated in light of evolving standards in RCTs published from nowadays onward.

Overall, we could show a better methodological quality than other samples of RCTs, but still readers of highly cited neurological RCTs should be aware of some concerns in risk of bias evaluation in most studies. While quality of reporting for intervention and analysis was generally good, authors of RCTs should ensure reporting of sufficient detail on methods. Most studies were pre-registered, but authors should also be encouraged to share analytic code and update registry entries with results soon after primary completion.

Our study provides a comprehensive summary of methodological quality, reporting practices and transparency aspects in highly cited RCTs in neurology from 2018 to 2022. It describes risk of bias and identifies areas of improvement for reporting and transparency. Thereby this study can serve as an orientation for authors, reviewers, readers and the scientific community in the field of neurology.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.