Reporting guidance considerations from a statistical perspective: overview of tools to enhance the rigour of reporting of randomised trials and systematic reviews

Abstract

Objective

Research waste has received considerable attention from the biomedical community. One noteworthy contributor is incomplete reporting in research publications. When detailing statistical methods and results, ensuring analytic methods and findings are completely documented improves transparency. For publications describing randomised trials and systematic reviews, guidelines have been developed to facilitate complete reporting. This overview summarises aspects of statistical reporting in trials and systematic reviews of health interventions.

Methods

A narrative approach to summarise features regarding statistical methods and findings from reporting guidelines for trials and reviews was taken. We aim to enhance familiarity of statistical details that should be reported in biomedical research among statisticians and their collaborators.

Results

We summarise statistical reporting considerations for trials and systematic reviews from guidance documents including the Consolidated Standards of Reporting Trials (CONSORT) Statement for reporting of trials, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Statement for trial protocols, the Statistical Analyses and Methods in the Published Literature (SAMPL) Guidelines for statistical reporting principles, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement for systematic reviews and PRISMA for Protocols (PRISMA-P). Considerations regarding sharing of study data and statistical code are also addressed.

Conclusions

Reporting guidelines provide researchers with minimum criteria for reporting. If followed, they can enhance research transparency and contribute improve quality of biomedical publications. Authors should employ these tools for planning and reporting of their research.

Introduction

It has been established that there is a significant amount of preventable waste in biomedical research.¹ This is important given the crucial role that biomedical research has in informing patient care and policies and decisions around our population's health. Waste due to incomplete, unusable and inaccessible research is a major concern and one has several available solutions.^{2 3}

Optimal principles on which to base the preparation of biomedical research publications are those of completeness and transparency, an underlying rationale being that of the potential for usability and reproducibility by other researchers. In a recent editorial, Goodman et al address the issue of inconsistent views on what the term ‘reproducibility’ means.⁴ In short, it refers to the transparency and reliability of research, and whether published methods can be repeated to yield the same results and conclusions as originally reported. While there has long existed an established ordering of research designs in clinical research (commonly dubbed the ‘evidence hierarchy’), there remains a vital need for better reporting to enable readers to accurately grasp the rigours of a given research study beyond its labelled design to inform determination of faith in the findings it generated.

Unfortunately, accumulating evidence shows that the research community often fails to meet standards for transparent reporting. Empirical explorations into the reporting of randomised controlled trials (RCTs), systematic reviews and meta-analyses, as well as other study designs have demonstrated these deficiencies. For example, evaluations of trial reporting show that key details such as outcome definitions, sample size calculation, allocation concealment and sequence generation are incompletely reported in more than half of published trials.⁵ In a 2014 sample of systematic reviews, more than half did not identify a primary outcome.⁶ Mental health researchers have found that the challenges of inadequate reporting in the context of randomised trials and systematic reviews also exist in this domain. Regarding RCTs, Thornley and Adams⁷ demonstrated inadequate reporting of blinding and allocation concealment in a total of more than 2000 trials of interventions for schizophrenia. Patel et al⁸ performed a systematic review that assessed the completeness of phase 2/3 studies of antipsychotic agents, and found several reporting limitations related to the description of design, indication of hypotheses, documentation of sample size calculation, and description of randomisation and blinding. de Vries et al⁹ performed a meta-analysis of more than 100 trials of second-generation antidepressant drugs for management of major depressive and anxiety disorders, and demonstrated that a majority of included trials provided little to no information on the incidence of serious adverse events. Melander et al¹⁰ found that there was considerable evidence of selective outcome reporting and selective publication based on inspection of 42 placebo-controlled trials of selective serotonin reuptake inhibitors provided to the Swedish drug regulatory authority, and suggested that any efforts to recommend a ‘best’ therapy in practice based on only public data are seemingly limited by biased data. Regarding the completeness of reporting of systematic reviews, Spineli et al¹¹ conducted a systematic review of Cochrane reviews to evaluate the extent to which they indicated methods to address missing study data and acknowledged their impact on the review, and found that in both cases there remains a need for improvement in transparency. There is an urgent need to address the challenge of poor reporting of many forms.

Of particular importance, and focused on in the current overview, is the reporting of statistical considerations in research. Writing descriptions of statistical methods and quantitative findings may be challenging for authors, particularly those with limited training in statistical methods, and given the word limitations of journal articles. In an era where considerable attention has been placed on the needs for completeness and transparency in research, it is important for researchers to ensure they are doing their part to adhere to these principles when writing up their research. Thankfully, there are tools available to facilitate this, in the form of reporting guidelines.

Over the past 20 years, we have seen the development of reporting guidance for biomedical research; the EQUATOR (Enhancing the QUAlity and Transparency Of health Research; http://www.equator-network.org) Network library currently lists more than 350 reporting guidelines for biomedical research. Reporting guidelines are typically developed through a consensus-based and evidence-based process, and often include a checklist of minimum reporting recommendations for a given study design.¹² Some reporting guidelines include template diagrams to help describe the flow of participants through the study process.¹²

Given the frequency with which RCTs and systematic reviews are carried out by biomedical researchers and their importance in informing healthcare and decisions, reporting guidelines of particular importance are the CONSORT (Consolidated Standards of Reporting Trials) Statement for parallel group RCTs^{13 14} and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Statement for systematic reviews.¹⁵ Evidence is accumulating that shows that the endorsement and use of reporting guidelines in the editorial process, peer review and by authors are associated with more completely reported research.^{6 16–21} This article provides an overview of the most widely used reporting guidelines for reporting of protocols and completed reports of randomised trials and systematic reviews/meta-analyses, focusing primarily on the statistical considerations they address.

Methods

In the current overview, we begin with discussion of sources of core guidance for the reporting and planning of RCTs, and next address those of key relevance for systematic reviews and meta-analyses. To conclude, we discuss considerations regarding two topics of ongoing debate of relevancy to statistical analysts and the discussion of reproducibility of research, namely those of open data sharing and provision of statistical code.

Results

Randomised trials

RCTs represent the gold standard source of primary data for the evaluation of healthcare interventions;²² they are vital to the continued evolution of medical interventions for patients. However, the design, analysis and reporting of RCTs are complex, and require careful planning and collaboration among clinicians, methodologists and statisticians alike. As additional focus has been placed on the importance of study protocol development, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Statement²³ has also been developed. Also based on a consensus framework, SPIRIT provides readers with core components to be described for the development of clinical trial protocols and aligns well with guidance from CONSORT. Core statistical considerations addressed by these tools are discussed next.

Reporting completed trials: the CONSORT Statement

The CONSORT Statement for parallel group trials was originally published in 1996²⁴ (with updates in 2010^{13 14}), its content established through a consensus process that included meetings and a modified Delphi process.

Table 1 presents a summary of devoted items from the CONSORT Statement which addresses statistical considerations (we also note those directly addressed by the SPIRIT Statement for study protocols); details of what authors should aim to report are described within the table text. While researchers new to the use of reporting guidance may at first find the time required for preparation of study protocols and reports to be extended, using these tools will ultimately enhance the contributions of their research to the literature.

Table 1.

Core CONSORT and SPIRIT elements addressing statistical considerations for trials

Statistical considerations for reporting, with guidance regarding core details	Defined SPIRIT item?	Defined CONSORT item?
Description of trial design. Specify clearly the type (eg, parallel, crossover, factorial, cluster randomised, etc), the framework (eg, superiority, non-inferiority, exploratory/pilot) and the ratio of allocation to intervention groups for the study.	√	√
Estimation of sample size. Specify the estimated number of patients for enrolment based on calculations performed. Provide clear mention of the outcome used as the basis for the estimation, as well as the assumptions made and/or additional information used (eg, assumed event rates or means and SDs per group, along with source or rationale for these values, type 1 error rate, chosen power level, test to be used to compare groups, adjustments for anticipated losses to follow up and other information which may be required for sample size estimations for trial designs beyond the standard parallel groups design) to arrive at the planned sample size.	√	√
Assignment of patients to intervention groups. Describe the approach to generate the randomisation scheme for patients with additional details as necessary (eg, computer generation of random sequence, stratification factors with boundary criteria, size of blocks to be implemented, use of simple randomisation or adaptive randomisation or other modified approaches).	√	√
Description of elements of the study's primary analysis. Specify the primary and secondary end points for comparison between groups, and the main analytic approach of focus for primary interpretation of findings (eg, statistical test). If multiple groupings of patients are to be considered in separate analyses (eg, related to adherence), clearly describe these groups. Describe the measures of effect (eg, risk ratio, mean difference) that will be used to summarise comparisons between groups, and indicate chosen type 1 error rates and reporting of CIs. Describe adjustments to be made for multiple testing, and any additional design-specific information required for analysis (eg, for cross-over or cluster-randomised studies).	√	√
Performance of additional data analyses. Efforts to consider additional analyses of the study data, for example, subgroups (either clinically or methods oriented), should be described. Clear details regarding any boundaries to group patients should be indicated. Statistical techniques for additional comparisons should be clarified, and a priori hypotheses regarding subgroups should be reported. Analyses incorporating adjustments (eg, via regression) should also be outlined. Specific approaches for missing data (including imputation or other approaches) should be described.	√	√
Interim analyses of study data. Specify details regarding interactions with a data monitoring committee and statistical analyses performed at key points during the trial (eg, timing of analyses, analytic methods), as well as criteria that will be considered for early study stopping (related to benefits, harms or futility), sample size adjustments or modifications to allocation ratios.	√	√
Assessment of harms observed during the study. Specify anticipated harms and whether data were actively solicited from patients, duration of time for which end points were recorded and how frequently. Make clear any formally planned statistical comparisons of harms to be made between groups, and/or descriptive techniques to be used to summarise the data.	√	√
Describe protocol changes made during the study. All changes made after protocol finalisation and study start should be clearly described. This includes the addition, removal and reordering of clinical end points, modifications to planned study sample size, changes to the data analysis plan and other changes of relevance for readers which represent changes from the study protocol.		√
Describe patient flow observed during the study. The numbers of patients assigned to intervention groups, the numbers receiving interventions and the numbers contributing to data analysis should be provided. Patient attrition should be clearly described. Use of a flow diagram is common and recommended.		√
Document baseline demographics across groups. A clear description of the distribution of all important patient demographics and clinical features should be provided. This addresses the comparability of groups, establishes the success of randomisation in balancing confounders and instructs readers on features of the study population.		√
Document the numbers of patients included in analyses. Report the numbers of study participants that were included for formal analysis for each end point assessed. Make clear whether analyses reflect the groups as originally randomised, or whether certain differences are present.		√

CONSORT addresses a variety of statistical considerations of importance for consumers of research, and ensuring the completeness of reports on these details will enhance the ability of readers to assess study rigour and establish their trust in its findings; through a consensus process, items were carefully identified with this consideration in mind. For example, considering items from table 1, clear description of sample size estimation provides readers the opportunity to assess the assumptions made by the research team (including, eg, hypothesised group event rates or means informing calculations) and to consider the degree of statistical power to test hypotheses specified a priori while also affording researchers themselves the chance to assess study feasibility and increase the likelihood of performing an appropriately powered study. Regarding specification of endpoints assessed and underlying methods used (including tests used, modelling methods and the proper inclusion of patients), description of these details provides informed readers with a clear grasp of the analytic approach to assess the collected data. This represents both a vital step towards reproducibility of findings as well as a core consideration that will inform readers when establishing their level of trust in study findings. Clear description of a priori and post hoc secondary data explorations such as subgroup analysis can appropriately frame the planning and thought process of study authors for readers, alleviating concerns in establishing the extent of data fishing which may have been undertaken during data analysis. Finally, while table 1 highlights additional elements, which are much lesser in terms of statistical complexity, inclusion of basic details such as patient flow, specification of trial design and randomisation mechanism, and compilation of harms may often fall to a study statistician to perform. As there remains room for improvement in author reporting on near all elements of CONSORT, even these fundamental details warrant mention.

Provision of details related to the statistical aspects of trials mentioned in table 1 by an increasing number of researchers will continue to benefit the quality of randomised trials in the biomedical literature as a whole.

Reporting trial protocols: the SPIRIT Statement

Past research has shown that the completeness of protocols for trials is also sometimes insufficient, including deficiencies in statistical aspects such as sample size estimation and reporting of findings for all a priori end points.^{25 26} This is unfortunate, as protocols provide the basis for rigorous planning and performance of trials, and provide a public record of the study's methodological plan.

Also based on a consensus framework, SPIRIT provides readers with core components to be described for the development of clinical trial protocols. In essence, while CONSORT provides instruction to authors at the time of study reporting, SPIRIT provides guidance at the important stage of study planning, prior to initiation. Thus, in addition to improving completeness of protocols, it may also help researchers better consider all key details during study design. Intuitively, much of its content reinforces recommendations of CONSORT, as can be seen in table 1.

We refer readers to the full checklists and websites for CONSORT (http://www.consort-statement.org) and SPIRIT (http://www.spirit-statement.org) as well as the key guidance publications for review of the broader list of items to be considered to maximise transparency of clinical trial protocols; the related Explanations and Elaborations reports for both documents also provide examples of suitable reporting to inform readers, as well as educational content regarding terminology and core concepts.^{14 27}

CONSORT Statement extensions

In addition to core guidance for parallel group trials, panels of experts have developed extensions for different trial designs with novel considerations in terms of design and analysis. These include extensions for non-inferiority/equivalence trials,²⁸ cluster-randomised trials,²⁹ pilot trials,³⁰ N-of-1 studies,³¹ pragmatic trials³² and the reporting of harms.³³ All of these guidance documents address aspects of statistical relevance for researchers working on the description of methods and findings from RCTs, and draw attention for non-statisticians to additional analytic topics of which they should be aware during the study design, analysis and reporting phases. From a statistical perspective, reports of cluster-randomised studies must provide additional clarity regarding at what level (cluster vs patient) study hypotheses will be assessed; considerations related to cluster size, number of clusters and intracluster correlation with regard to sample size estimation; how clustering was accounted for in data analyses; and other such nuances. Description of non-inferiority and equivalence trials requires additional attention to statistical details including specification of the chosen non-inferiority margin, with rationale.²⁸ The extension for feasibility studies provides mild updates of CONSORT items while placing focus on considerations for moving towards a definitive trial.³⁰ The extension for N-of-1 studies addresses the need for several additional statistical considerations including the accounting for carry-over/period effects in analyses, reporting the allocation and sequence of treatment periods, and a host of additional details related to the summary of findings from analysis.³¹ For these and further details, we refer interested readers to the CONSORT website to locate the corresponding publications.

Reporting of statistical findings in primary research: the SAMPL Guideline

While many reporting guidelines exist, few provide specific instructions on how to report data. Rather, they tend to provide general recommendations for which type of data to report. An important topic is how such data is ‘best’ provided to readers. From the perspectives of transparency and in consideration of the needs of researchers (including systematic reviewers) who may have specific interests for key data, there is a need to provide sufficient information. For instance, incomplete descriptions of the numbers of events, SDs and measures of precision and other information can limit the usability of published research.

The SAMPL (Statistical Analyses and Methods in the Published Literature) Guidelines, developed by Altman and Lang, provide recommendations for reporting statistical methods and findings in biomedical journals.³⁴ Their intent is to be broad reaching, being relevant for all primary study designs. Reporting recommendations include how to report: basic numbers and descriptive statistics; risks, rates and ratios; hypothesis testing; analyses of association and correlation; regression, analysis of variance and analysis of covariance; analysis of time-to-event end points; and Bayesian analyses. In discussion of these different types of analyses, Altman and Lang address the specific data that authors should aim to provide. The specific guidance is detailed and beyond the scope of this summary, and we encourage readers to review this guidance and to reinforce its content to their research teams' statisticians during the planning, analysis and reporting phases of their research.

Systematic reviews

Systematic reviews have become increasingly prevalent in the biomedical literature, with at least 8000 published annually as of 2014.⁶ Intuitively, as many are focused on the synthesis of interventions evaluated in RCTs, systematic reviews are a highly valued, gold standard component in the practice of evidence-based medicine. They are an important conduit of knowledge used to inform clinical decision-making and the development of clinical guidelines. Thus, rigour in their design, conduct and reporting is critical. In addition to ‘traditional’ systematic reviews of aggregated patient data to compare two medical interventions, during the past 20 years, there has been an evolution of several more complex forms of syntheses including that of individual patient data (IPD meta-analysis),³⁵ and an analytic framework for the comparison of multiple interventions (network meta-analysis; NMA³⁶). The complexity of statistical considerations for meta-analyses continues to rise. Guidance for the reporting of systematic reviews of several forms, again focusing on statistical aspects, is discussed next.

Reporting completed systematic reviews: the PRISMA Statement

Preceded by the QUOROM (Quality Of Reporting Of Meta-analyses) Statement,³⁷ the PRISMA Statement was developed via face-to-face meetings and a consensus process that involved 29 methodologists, authors, physicians and journal editors. Past research has documented the positive impact of this reporting guidance on the completeness of reported systematic reviews.^{6 17} Its 27 checklist items address all aspects of reviews, maximising its efforts to ensure authors adequately cover background and rationale, study methods for data gathering and data analysis, and subsequently presentation of results and documentation of primary findings. Table 2 summarises the important items to be addressed from the perspective of statistical methods for systematic reviews and meta-analyses.

Table 2.

Core PRISMA and PRISMA-P elements addressing statistical considerations for systematic reviews

Statistical considerations for reporting, with guidance regarding core details	Defined PRISMA-P item?	Defined PRISMA core item?
Describe criteria under which study data will be quantitatively synthesised. Outline considerations that will be undertaken by the research team to make decisions regarding whether formal meta-analyses of end point data will be performed. This should include clear mention of factors related to study population, intervention, comparators and outcomes of included studies.	√	√
Describe planned summary measures for meta-analyses and evaluations of heterogeneity. Specify considerations for the combining of study data using meta-analysis including type of model (random vs fixed effects), approach (eg, generic inverse variance, Mantel Haensel), effect measures (eg, mean differences, ORs) as well as plans for the assessment of statistical heterogeneity (eg, I2), handling of multiarm studies, and considerations for missing data. Any data transformations to be undertaken should be mentioned.	√	√
Address performance of additional analyses (eg, subgroup/sensitivity analyses). Describe additional analyses beyond the primary analysis that explore the robustness of findings and/or assess the potential for different effects in certain types of patients or studies. This includes subgroup analyses, sensitivity analyses, metaregressions and other such data analyses performed secondary to the primary analyses. Distinctions between prespecified and post hoc analyses should be made.	√	√
Describe exploration of metabiases. Describe any evaluations performed to address the potential for risk of bias across studies (eg, publication bias via funnel plots or statistical tests, selective reporting via inspection of trial protocols). Statistical details of the evaluations performed (eg, graphical components or statistical tests performed) should be described, while reasons any planned analyses were not performed should also be clearly stated (eg, minimal studies available, etc).	√	√
Describe study selection process. For transparency of the systematic process followed to identify studies, document the numbers of studies screened for eligibility at the title/abstract level and the full-text level. Provide a summary of the final number of studies included qualitatively and quantitatively, as well as a description of reasons for exclusion from full-text screening. Provision of an illustrative flow diagram is recommended.		√
Describe study characteristics. Provide characteristics of relevance extracted from included studies. A narrative description capturing the extent of similarities and differences in key features of study methods, interventions and patient populations should also be provided, including tabulations of the numbers of studies possessing characteristics of key relevance to the research question.		√
Present results of individual studies. Present both simple summary data (eg, number of events and patients for dichotomous measures or means, SD and sample size for continuous end points) as well as summary effect estimates (including CIs) for each study and end point. If feasible, use of forest plots is recommended. This enhances reproducibility, inspection of variations across studies, and enhances the ability to identify data collection errors. Specifics to report for different end point types and strategies to make use of web appendices are discussed in the PRISMA elaborations document.		√
Present findings from meta-analyses. Report summary estimates from all meta-analyses performed in the review along with CIs estimates of between-study heterogeneity assessed. Ensure that the numbers of studies and patients contributing to each synthesis are clear. Use of forest plots is recommended.		√

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analysis; PRISMA-P, PRISMA for Protocols.

Similar to CONSORT for reporting of trials, added time to complete drafting of manuscripts which researchers encounter early during their adoption of PRISMA will improve the contributions of their research to the literature.

PRISMA content includes several statistical considerations of relevance to readers which directly impact their ability to accurately assess the rigour of systematic reviews and their corresponding degree of faith in findings. A priori specification of primary considerations of the research team in the decision to synthesise data (in relation to both variation between clinical study features as well as benchmarks of statistical variability of effect sizes) provides clear documentation of influential factors regarding this important judgement. Clear description of the model used for analysis, the weighting scheme for studies and the chosen measures of treatment effect and uncertainty provide readers with a simple and clear description clarifying assumptions about study differences and the details for summary estimates that will be provided. Similar to reporting of trials, clear specification of a priori and post hoc secondary analyses again can minimise concerns regarding potential data dredging. Finally, clear presentation of study-level data in addition to summary data from meta-analyses allows readers to assess differences in study data more closely in terms of variations in observed effect sizes, event rates within intervention groups and other such factors of importance. Optimal reporting of the details outlined in table 2 by authors will continue to benefit the quality of reviews in the biomedical literature as a whole.

Reporting systematic review protocols: the PRISMA-P Statement

Only 16% of systematic reviews report having a publicly accessible protocol; these are dominated by reviews carried out within Cochrane.⁶ The importance of documentation of and access to systematic review protocols has grown during the past decade based on increased interest in several notions including: (1) a need for increased focus on developing plans reviews to avoid arbitrary decision-making, as well as anticipating potential obstacles during review conduct; (2) consideration of reproducibility of research by others; (3) avoiding unintentional duplication of systematic reviews and (4) the ability to identify biases related to protocol deviations and selective reporting.³⁸ As with primary research, prospective registration of systematic reviews can facilitate greater transparency of the review process. The PROSPERO prospective register for systematic review protocols, housed with the University of York, was developed in 2011 (http://www.crd.york.ac.uk/PROSPERO/). In addition to registration, documentation of detailed methods and analytical plans for systematic reviews is important. Similar in intent to SPIRIT for trial protocols, the PRISMA for Protocols (PRISMA-P) Statement³⁹ was published in 2015. PRISMA-P is intended for reviews of therapeutic efficacy of medical interventions, and includes a checklist of 17 items that align closely with content from the PRISMA Statement (see table 2) to enable seamless integration.

We refer readers to the full checklists and websites for PRISMA and PRISMA-P (http://www.prisma-statement.org), as well as the key guidance publications for review of the broader list of items to be considered to maximise transparency of clinical trial protocols; the related Explanations and Elaborations reports for both tools also provide examples of strong reporting, and additionally include educational content.^{38 40}

PRISMA Statement extensions

In addition to core guidance for systematic reviews comparing the therapeutic efficacy of pairs of medical interventions using aggregate patient data, extensions of PRISMA have been developed for reviews of different structures and purpose. An extension for NMA⁴¹ was developed to help researchers address reporting issues for nuances of such analyses, including presentation of the expanded evidence base of interventions using network diagrams and statistical evaluation of the agreement between sources of direct and indirect evidence (commonly called consistency or coherence). Certain modifications of other core PRISMA items were also made in relation to how to summarise findings from analysis, consideration of Bayesian applications and strategic use of supplements to present all information of relevance to readers. For readers unfamiliar with NMA, the Explanations and Elaborations document also provides educational content on key methodological concepts and illustrative examples of good reporting.⁴¹ Recent research⁴² has put forward a structured set of suggestions for extensions of the standard protocol structure for traditional systematic reviews addressed by PRISMA-P guidance.³⁸

The PRISMA Extension Statement for Individual Patient Data (IPD) Meta-Analysis added three additional reporting items for consideration by authors, and also incorporated wording changes for 23 checklist items.⁴³ Newly added items included specification of methods followed to confirm the integrity of the IPD (including baseline balance and other factors), reporting of any vital findings from these assessments and exploration for differences in benefits and harms in different types of patients (including evaluation of relevant interaction terms and other potential effect modifiers). Modifications addressed changes necessary for items that included provision of additional details of the approach to analysis necessary for IPD meta-analysis (including accounting for clustering of patients, methods to synthesise aggregate and patient-level data together, and other details with a focus towards complete transparency of methods for analysis).

Few additional nuances from a statistical perspective are noted by the PRISMA Extension Statement for Harms,⁴⁴ though of note is the need for authors to clearly stipulate the handling of zero cells in meta-analyses as these are not uncommon when reviewing data for end points associated with lower event rates; ensuring clarity regarding criteria employed for supplemental meta-analyses related to harms grades, varying end point definitions and potentially alternative models is also important. Finally, the PRISMA Extension for equity-focused reviews⁴⁵ mentions a small number additional considerations of statistical relevance related to providing clear specification of subgroup (or other) analyses performed to assess health inequities, with consideration of both relative and absolute measures of effect. Additional PRISMA extensions for reviews involving children, reviews of diagnostic accuracy, scoping reviews and rapid reviews are forthcoming.

Additional topics of relevance

Sharing of study data and statistical code

Regarding reproducibility of biomedical research, an additional consideration for statisticians and investigators involved in the conduct of trials is the sharing of clinical trial data. In 2016, the International Committee of Medical Journal Editors (ICMJE)^{46 47} expressed their position on ethical requirements for trialists to openly share data derived from randomised trials, a perspective shared by an increasing number of funders and other organisations. As per Goodman, the reproducibility of research requires that other researchers may replicate original findings using the same materials evaluated by the original study authors, which naturally requires access to full study data. While a growing number of journals continue to support the sharing of clinical trial data, research suggests that journals mandating this principle are not yet consistently following this policy.⁴⁸ From a transparency perspective, research units involved in the design, performance and reporting of clinical trials should give strong consideration to optimising the transparency of their research via public data sharing. The same consideration should also be undertaken by researchers performing systematic reviews and meta-analyses; while traditional meta-analyses comparing two interventions commonly provide access to raw study data as a component of standard graphical output from many meta-analysis packages, this is not the case for meta-analyses of other forms. For RCTs and systematic reviews alike, open provision of study data is likely to elevate the trust and confidence of readers in study findings. While researchers' interests to openly adopt data sharing has been limited to some degree by concerns over impact on their careers, there is reason to believe positive impacts including increased public attention for their research as well as enhanced career prospects may follow.⁴⁹

For RCTs and systematic reviews, formal guidance does not currently exist suggesting that researchers should provide access to the statistical/computer code used to generate study findings; however, this issue has been discussed in the literature.^50–52 Based on the premise of reproducibility, the potential for identification of errors in original analyses (during peer review or postpublication) and the ability to consider additional analyses of clinical relevance, there is appeal in the provision of code for readers and reviewers. However, as Ioannidis suggests, concern may remain as to whether code provided is a complete representation of all analyses carried out or only a subset thereof, limited to the study's most interesting findings.⁴⁶ As most biomedical journals are online, the use of online supplements enables authors make additional content available with their research, including relevant statistical code. Other initiatives such as the Open Science Framework (https://osf.io/) facilitate open sharing of research documents as well as online collaboration between researchers. While the difficulty of ‘spin’ in biomedical research remains a challenge, continued improvement in statistical reporting may enhance our ability to identify it more easily in our readings of evolving literature.

Discussion and conclusions

Efforts to maximise the transparency and completeness of reporting can go a long way towards reducing waste in research, within the realm of mental health publications as well as the broader realm of all biomedical publications. Statistical aspects, in terms of description of methods used and results achieved, represent an important dimension in which the clarity of reporting is critical. During the past 20 years, reporting guidelines have been developed which offer researchers an excellent resource, easy to follow and to reflect in journal submissions via appropriate citation and completion of checklists. Continued endorsement of such guidance by biomedical journals paired with increased awareness and implementation by researchers is key steps towards further enhancing the quality of reporting of clinical trials and systematic reviews. Other instrumental stakeholders who can facilitate optimal reporting include institutions and funders who can mandate adherence to guidelines and open access principles.^{2 3}

While the existence of poor reporting of clinical trials and systematic reviews may not necessarily directly correlate to poor design and methodological quality, a relationship of this nature may often be presumed. Therefore, complete reporting can facilitate assessments of methods, quality and the risk of bias in published research. This overview only addresses core guidance documents related to randomised trials and systematic reviews, and corresponding research for other study designs is also available. Interested readers will find comprehensive access to existing reporting guidance tools from the website of the EQUATOR Network, and we encourage readers to explore them for use in their future research.