Abstract
Purpose
Outcomes of randomized controlled trials (RCTs) have been shown to influence clinical decision making. Thus, the quality and reliability of these outcomes are essential for both patients and medical care providers. To date, no study has assessed the quality of intervention reporting of RCTs in orthopaedics. The aim of this study was to evaluate the quality of intervention reporting of published RCTs in the field of orthopaedics using the Template for Intervention Description and Replication (TIDieR) checklist.
Methods
In this cross-sectional analysis, we applied the TIDieR checklist to assess the quality of intervention reporting in orthopaedic RCTs. Additionally, we evaluated the TIDieR checklist’s influence on intervention reporting by comparing overall adherence to checklist items in trials published before the TIDieR checklist’s release versus trials published after its release. Finally, we assessed whether certain factors were associated with the quality of intervention reporting.
Results
From a random sample of 300 publications in orthopaedic journals, 175 parallel-arm and cluster RCTs were identified. The overall rate of adherence to TIDieR items was 58.4%. Only 31.4% of orthopaedic RCTs adhered to at least 6 of the 12 TIDieR checklist items, whereas 0% adhered to all 12 items. We found no significant improvement in the quality of intervention reporting in studies published after the TIDieR checklist’s release compared with studies published before its release (P = .97). Additionally, preregistered trials were associated with more complete intervention reporting.
Conclusions
Our results suggest suboptimal reporting of orthopaedic RCT interventions. In addition, the TIDieR checklist’s intended effect—to better the quality of RCT intervention reporting—appears to have fallen short of its goal.
Clinical Relevance
Because outcomes of RCTs are used to guide clinical decision making, it is essential that orthopaedic surgeons and clinical practice guideline panels are equipped with high-quality published research. Increasing the accuracy of intervention reporting may lead to more accurate clinical application. Thus, adoption of more stringent reporting of trial interventions by researchers, authors, and journal editors may improve the quality of orthopaedic research, as well as improve patient outcomes.
Randomized controlled trials (RCTs) are critical to the advancement of evidence-based medicine and are essential resources for clinical decision making. In orthopaedic surgery, RCTs are considered Level I evidence by some orthopaedic surgery journals1 and are used by clinical practice guideline panels to establish robust practice recommendations. Considering the implications of trial findings in orthopaedics, the quality and completeness of reporting methodologic details in trial publications are imperative for readers to effectively evaluate research quality and interpret study outcomes. Accurate extrapolation of such outcomes is crucial to make informed judgments when applying these outcomes in the clinical setting. Despite the utility of orthopaedic trials, significant gaps remain in their reporting, which is problematic for both readers and researchers.2, 3, 4, 5
Previous studies using the Consolidated Standards of Reporting Trials (CONSORT) statement—a 25-item checklist outlining a set of recommendations to increase the transparency of RCT reporting6—have indicated that methodologic reporting of orthopaedic trials needs improvement.3,4 For example, Chess and Gagnier7 noted inadequate reporting in 42% of orthopaedic RCTs. Chen et al.2 found that 66% of trials did not sufficiently report blinding and nearly 60% failed to report allocation concealment. Montané et al.4 reported that 75% of RCTs scored 12 or less overall on CONSORT checklist items, 56.5% failed to report the estimated effect size, and 25% did not report the interpretation of potential bias or lack of precision of results. These findings suggest that key methodologic components, as detailed in the CONSORT Statement, are consistently under-reported in orthopaedic RCTs.
The CONSORT Statement has achieved much in elucidating areas in need of improvement. However, this checklist includes only a single item dedicated to intervention reporting. Complete intervention reporting is necessary for the reproducibility of interventions in future studies or the application of trial interventions to patient care. To combat this problem, the Template for Intervention Description and Replication (TIDieR) was created to better explain the “who, what, when, where, and why” regarding trial interventions.8 Although the overall completeness of trial reporting using the CONSORT guidelines has been extensively evaluated, little is known about the quality of intervention reporting in particular. Thus, we evaluated the quality of intervention reporting in orthopaedic trials and assessed whether the publication of the TIDieR checklist has influenced intervention reporting.
Methods
This cross-sectional analysis did not involve human subjects and, thus, was not subject to institutional review board oversight. Our study is reported using a modified version of the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.9 All versions of the study protocol, analysis scripts, materials, and data are available on the Open Science Framework (Center for Open Science) to foster study reproducibility and transparency.10
Search
Two authors (S.J. and M.V.) searched PubMed (including the MEDLINE collection) for RCTs published in orthopaedic surgery journals. To perform this search, we used PubMed’s clinical trial filter (Clinical Trial[ptyp]) and restricted returns by publication date. The following journals were included: The American Journal of Sports Medicine; The Journal of Bone and Joint Surgery; Clinical Orthopaedics and Related Research; Spine; Knee Surgery, Sports Traumatology, Arthroscopy; The Journal of Arthroplasty; Arthroscopy: The Journal of Arthroscopic & Related Surgery; The Spine Journal; European Spine Journal; and The Bone & Joint Journal. The selection of these journals was based on the 2019 Google Scholar h5-index “Orthopedic Medicine and Surgery” subcategory, which ranked these journals as the top 10 journals in orthopaedic surgery. S.J. and M.V. searched these journals in PubMed to identify all RCTs published from January 1, 2011, to December 31, 2013, and from January 1, 2016, to December 31, 2018. Our aim was to select cohorts of studies published before and after the March 2014 publication of the TIDieR checklist and assess its influence, if any, on RCT intervention reporting. The period between TIDieR publication and the start date of the second search permitted 22 months for the TIDieR checklist to take effect. We randomly sampled 150 records from each search (300 total) using Microsoft Excel’s random number function.
Eligibility
The following types of RCTs were included: parallel-arm, crossover, and cluster trials. Exclusions are listed in Figure 1.
Fig 1.
Flowchart of included and excluded randomized controlled trials (RCTs). (TIDieR, Template for Intervention Description and Replication.)
Primary and Secondary Outcomes
Our primary outcome was the percentage of completion of the TIDieR checklist.8 Our secondary outcomes were (1) to compare the completeness of intervention reporting before and after the publication of the TIDieR checklist and (2) to evaluate factors associated with intervention reporting.
Training
Prior to data extraction, S.J. and M.V. developed a standardized training module. Two investigators (M.A. and A.S.) completed the training module prior to study commencement. Training was led by S.J. and comprised the following: (1) an overview of the study, (2) a review of the article by Hoffmann et al.8 and a brief review of all 12 TIDieR checklist items, and (3) data extraction for 3 example RCTs. This module was recorded and is available online for reference.10
Screening
Rayyan (Qatar Computing Research Institute), a systematic review application, was used to screen PubMed records for eligibility. J.M.A. and A.S. screened all records in a double, masked fashion. After screening, these investigators held a consensus meeting to resolve any disagreements. S.J. and M.V. were available for third-party adjudication.
Data Extraction
Data extraction was performed by J.M.A. and A.S. As before, each investigator was masked to the other investigator’s responses. After data extraction and scoring, both investigators held a final consensus meeting to resolve discrepancies. S.J. and M.V. were available for third-party adjudication. The investigators used a Google Form developed by S.J. that contained all 12 checklist items, as outlined by Hoffmann et al.,8 as well as other variables that previous studies found to be related to completeness of reporting when using the CONSORT guidelines.11, 12, 13, 14, 15, 16 This form is available online for reference.10
Data Analysis
For our primary objective, we evaluated the completeness of intervention reporting of RCTs in orthopaedic surgery. For each trial, the overall completeness of reporting (i.e., number of TIDieR checklist items reported) was measured and assigned a point value: 1 point for complete compliance with an item, 0.5 points for partial compliance, and 0 points for items not reported. To summarize adherence to individual checklist items, we used frequencies, percentages, and 95% confidence intervals. We also summarized data by item to evaluate which items were sufficiently reported and which were under-reported. We characterized the distribution of trials by the number of reported TIDieR items using a histogram. For this graph, we considered an item to be reported only if it was completely reported by the authors; partially completed items were considered not reported.
We conducted an interrupted time-series analysis to investigate whether publication of the TIDieR checklist improved intervention reporting in orthopaedic RCTs. To perform this analysis, TIDieR checklist completion was scored as a percentage of completion at the trial level. Interrupted time-series analysis has been used in previous studies to evaluate the effect of reporting guidelines on the completeness of reporting.17 Because this analysis requires only 1 data point per period, we calculated the mean adherence for all trials published in the same month and year. Standard errors were estimated using the Newey-West method.18 We used Stata’s “itsa” command (StataCorp).
We used generalized estimating equations (GEEs) to evaluate whether particular trial characteristics (or demographic characteristics) were associated with intervention reporting in orthopaedic RCTs.19 We chose GEEs because we expected clustering in our data, such that trials published within the same journal would likely adhere to the same reporting guidance. We specified a Poisson distribution (because our outcome variable was count data), a log link function, and an exchangeable correlation structure. Our resulting coefficients were exponentiated and presented as incidence rate ratios with 95% confidence intervals and P values. Because of disproportionate sample sizes, we made the post hoc decision to cluster some descriptive trial characteristics into categories, where applicable. Stata’s “xtgee” command was used for the analysis. For all analyses, the type I error (α) was set at .05 a priori. We used MATLAB (The MathWorks)20 to prepare data for the final analysis and Stata (version 15.1) for all analyses.
Results
Our search returned 1,603 publications, from which a random sample of 300 publications was generated and further analyzed. Of these, 175 randomly sampled publications were included whereas 125 were excluded (Fig 1).
Sample Characteristics
Our final sample included 175 RCTs, with the largest percentages being published in The Journal of Arthroplasty (n = 40) and Knee Surgery, Sports Traumatology, Arthroscopy (n = 32). Of the journals included in our final sample, 7 of 9 (77.8%) either require or recommend adherence to the CONSORT guidelines in the “instructions for authors” sections on their websites whereas 0 of 9 explicitly endorse or require adherence to the TIDieR checklist (Table 1). Only 48 publications explicitly mentioned following the CONSORT guidelines, whereas no trial mentioned adhering to the TIDieR checklist. Table 2 provides further sample characteristics.
Table 1.
Summary of Journal Endorsement of CONSORT and/or TIDieR Guidelines for RCT Intervention Reporting
| Journal Name | Endorsement of CONSORT | Endorsement of TIDieR |
|---|---|---|
| The Journal of Arthroplasty | Yes | No |
| Knee Surgery, Sports Traumatology, Arthroscopy | No | No |
| Spine | Yes | No |
| The American Journal of Sports Medicine | Yes | No |
| Clinical Orthopaedics and Related Research | Yes | No |
| European Spine Journal | Yes | No |
| Arthroscopy: The Journal of Arthroscopic & Related Surgery | Yes | No |
| The Bone & Joint Journal | No | No |
| The Spine Journal | Yes | No |
| The Journal of Bone & Joint Surgery | Yes | No |
CONSORT, Consolidated Standards of Reporting Trials; RCT, randomized controlled trial; TIDieR, Template for Intervention Description and Replication.
Table 2.
Baseline Characteristics of Sampled RCTs from Top Orthopaedic Surgery Journals
| Characteristic | Google-Form Response | n (%) |
|---|---|---|
| Name of journal | The Journal of Arthroplasty | 40 (22.9) |
| Knee Surgery, Sports Traumatology, Arthroscopy | 32 (18.3) | |
| Spine | 21 (12) | |
| The American Journal of Sports Medicine | 20 (11.4) | |
| Clinical Orthopaedics and Related Research | 15 (8.6) | |
| European Spine Journal | 14 (8.0) | |
| Arthroscopy: The Journal of Arthroscopic & Related Surgery | 14 (8.0) | |
| The Bone & Joint Journal | 12 (6.8) | |
| The Spine Journal | 7 (4.0) | |
| Source of funding | Not mentioned | 64 (36.6) |
| Industry | 36 (20.6) | |
| None | 30 (17.1) | |
| Public | 20 (11.4) | |
| Mixed | 11 (6.3) | |
| Private | 9 (5.1) | |
| Hospital | 5 (2.9) | |
| Hypothesis | Superiority | 91 (47.9) |
| Not sure | 64 (33.7) | |
| Equivalence | 21 (11.1) | |
| Other | 8 (4.2) | |
| Noninferiority | 4 (2.1) | |
| Mixed (superiority and equivalence) | 2 (1.1) | |
| Study design | Parallel arm | 172 (98.3) |
| Cluster | 3 (1.7) | |
| Intervention | Procedure | 69 (39.4) |
| Drug | 39 (22.3) | |
| Other | 36 (20.6) | |
| Device | 25 (14.3) | |
| Mixed | 6 (3.4) | |
| Blinding | No blinding | 76 (43.4) |
| Single blind | 39 (22.3) | |
| Double blind | 60 (34.3) | |
| No. of participants per trial | <60 | 63 (36.0) |
| 60-170 | 88 (50.3) | |
| 171-280 | 15 (8.6) | |
| >280 | 9 (5.1) | |
| Country where trial was conducted | Outside of United States | 133 (76) |
| United States | 40 (22.8) | |
| Both | 1 (0.6) | |
| Not mentioned | 1 (0.6) | |
| Conducting center | Single center | 148 (84.6) |
| Multicenter | 27 (15.4) | |
| RCTs with CONSORT endorsement | Yes | 48 (27.4) |
| No | 127 (72.6) | |
| RCTs with TIDieR endorsement | Yes | 0 (0) |
| No | 175 (100) | |
| Trial prospectively registered | Yes | 63 (36) |
| No | 112 (64) |
NOTE. A total of 175 RCTs were included in the analysis.
CONSORT, Consolidated Standards of Reporting Trials; RCT, randomized controlled trial; TIDieR, Template for Intervention Description and Replication.
Adherence to TIDieR Checklist Items
The mean number of TIDieR items reported was 7.0 (standard deviation, 1.48) out of a possible 12. Six items or fewer were reported in 31.4% of RCTs (55 of 175). Furthermore, no trial adhered to all 12 TIDieR checklist items. All 175 trials included a name or phrase describing the intervention, as well as the rationale, theory, or goal of the intervention. Other items with greater than 75% adherence included items 4 (“What procedures?”), 6 (“How?”), and 8 (“When and how much?”). Items with poor adherence (<20%) included items 9 (“Tailoring?”), 10 (“Modifications?”), 11 (“How well: planned?”), and 12 (“How well: actual?”).
Secondary Analysis
As a secondary outcome, our study analyzed whether publication of the TIDieR checklist affected intervention reporting. Figure 2 displays the mean percentage of completion by month for trials published before and after the TIDieR checklist’s publication. The results suggest that the publication of the TIDieR checklist did not result in a statistically significant trend in the percentage of completion of the TIDieR checklist items (P = .97). Table 3 shows the overall percentage of adherence to each TIDieR item for the 2 groups of orthopaedic trials, those published prior to and after the TIDieR checklist’s release.
Fig 2.
Interrupted time-series analysis of mean percentage of adherence to Template for Intervention Description and Replication (TIDieR) items.
Table 3.
Percentage of Adherence to TIDieR Checklist Items by Prespecified Period and Overall
| TIDieR Checklist Item∗ | Trials Published Between 2011 and 2013 (n = 73) |
Trials Published Between 2016 and 2018 (n = 102) |
Trials Published Between 2011 and 2018 (N = 175) |
|||
|---|---|---|---|---|---|---|
| Adherence to Checklist Item, n (%) | 95% CI | Adherence to Checklist Item, n (%) | 95% CI | Adherence to Checklist Item, n (%) | 95% CI | |
| 1. Brief name | 73 (100) | 1.000-1.000 | 102 (100) | 1.000-1.000 | 175 (100) | 1.000-1.000 |
| 2. Why/rationale? | 73 (100) | 1.000-1.000 | 102 (100) | 1.000-1.000 | 175 (100) | 1.000-1.000 |
| 3. What materials? | 50 (68.5) | 0.578-0.791 | 85 (83.3) | 0.761-0.906 | 135 (77.1) | 0.709-0.833 |
| 4. What procedures? | 53 (72.6) | 0.624-0.828 | 81 (79.4) | 0.716-0.873 | 134 (76.6) | 0.703-0.828 |
| 5. Who provided? | 40 (54.8) | 0.434-0.662 | 55 (53.9) | 0.442-0.636 | 95 (54.3) | 0.469-0.617 |
| 6. How? | 68 (93.2) | 0.874-0.989 | 100 (98.0) | 0.953-1.007 | 168 (96) | 0.931-0.989 |
| 7. Where? | 46 (63) | 0.519-0.741 | 72 (70.6) | 0.617-0.794 | 118 (67.4) | 0.605-0.744 |
| 8. When and how much (dose)? | 68 (93.2) | 0.874-0.990 | 100 (98.0) | 0.953-1.007 | 168 (96) | 0.931-0.989 |
| 9. Tailoring or personalization of intervention? | 8 (10.9) | 0.038-0.0.181 | 16 (15.7) | 0.086-0.227 | 24 (13.7) | 0.086-0.188 |
| 10. Modifications? | 5 (6.8) | 0.011-0.126 | 0 (0.0) | 0.000-0.000 | 5 (2.9) | 0.004-0.053 |
| 11. How well (planned) was intervention delivered? | 5 (6.8) | 0.011-0.126 | 10 (9.8) | 0.040-0.156 | 15 (8.6) | 0.044-0.127 |
| 12. To what extent (actual) was intervention delivered as planned? | 3 (4.1) | –0.004 to 0.087 | 11 (10.7) | 0.047-0.168 | 14 (8.0) | 0.040-0.120 |
CI, confidence interval; TIDieR, Template for Intervention Description and Replication.
Items were included in the calculation of the percentage of adherence if “completely” compliant with the TIDieR item.
Additionally, we sought to determine whether specific characteristics influenced the likelihood that an RCT intervention was more completely reported. The results from our GEE model suggest that preregistered trials were associated with better TIDieR adherence than trials that were not preregistered. In addition, multicenter trials and trials conducted within the United States were associated with poorer TIDieR adherence (Table 4).
Table 4.
Characteristics Associated With Intervention Reporting
| Characteristic | Variable |
||
|---|---|---|---|
| IRR | P Value | 95% CI | |
| No. of authors | |||
| <4 | 1 | Reference | Reference |
| 4-7 | 0.97 | 0.313 | 0.92-1.03 |
| >7 | 1.99 | 0.971 | 0.92-1.09 |
| Funding source | |||
| Industry and private | 1 | Reference | Reference |
| Public and mixed funding sources | 1.05 | 0.059 | 1.00-1.10 |
| Not mentioned | 1.02 | 0.565 | 0.95-1.10 |
| None | 0.99 | 0.688 | 0.92-1.05 |
| Hypothesis | |||
| Superiority | 1 | Reference | Reference |
| All other hypotheses | 0.97 | 0.267 | 0.92-1.02 |
| Study design | |||
| Parallel arm | 1 | Reference | Reference |
| Crossover | 0.99 | 0.811 | 0.88-1.10 |
| Intervention type | |||
| Procedure | 1 | Reference | Reference |
| Drug | 1.02 | 0.471 | 0.96-1.09 |
| Non-procedure and non-drug | 1.05 | 0.084 | 0.99-1.10 |
| Blinding | |||
| No blinding | 1 | Reference | Reference |
| Blinding (single or double) | 0.99 | 0.489 | 0.96-1.02 |
| No. of participants | |||
| <60 | 1 | Reference | Reference |
| 60-170 | 0.96 | 0.069 | 0.92-1.00 |
| 171-280 | 0.97 | 0.669 | 0.84-1.12 |
| >280 | 1.00 | 0.984 | 0.86-1.16 |
| Location of trial | |||
| Outside of United States, multiple locations including United States, or not mentioned | 1 | Reference | Reference |
| United States | 0.95 | 0.011 | 0.92-0.99 |
| Conducting center | |||
| Single center | 1 | Reference | Reference |
| Multicenter | 0.92 | <0.001 | 0.89-0.95 |
| CONSORT endorsement by trial authors | |||
| No | 1 | Reference | Reference |
| Yes | 0.99 | 0.806 | 0.92-1.06 |
| TIDieR endorsement by trial authors | |||
| No | 1 | Reference | Reference |
| Yes | — | — | — |
| Trial registry | |||
| None | 1 | Reference | Reference |
| Yes, listed | 1.06 | 0.009 | 1.01-1.10 |
| No. of sources in which information was found | |||
| 1 | 1 | Reference | Reference |
| ≥2 | 1.02 | 0.304 | 0.98-1.06 |
CI, confidence interval; CONSORT, Consolidated Standards of Reporting Trials; IRR, incidence rate ratio; TIDieR, Template for Intervention Description and Replication.
Discussion
Our analysis of 175 RCTs in orthopaedics showed suboptimal intervention reporting of TIDieR checklist items. We found that only 58.4% of TIDieR items were adequately reported, which indicates that there is room for improvement of intervention reporting in orthopaedic trials. Commonly omitted checklist items included intervention modifications, whether the intervention was personalized or tailored, and assessment of intervention fidelity. Our results are similar to those of previous studies in other fields of medicine.21, 22, 23, 24 For example, a 2019 systematic review of 98 RCTs reporting 134 interventions for major depressive disorder found that only 55.2% of “core items” were completely reported.25 Moreover, a study assessing intervention reporting of 200 physiotherapy RCTs found that nearly 25% of trials did not adequately describe at least 6 TIDieR items for the intervention groups.26 Furthermore, we found no significant improvement in the percentage of adherence to TIDieR checklist items in studies published after the checklist’s release compared with studies published prior to its release. These findings suggest inadequate completeness of intervention reporting in orthopaedic trials, even after the publication of the TIDieR checklist. Taken together, the findings of our study and previous studies shed light on significant gaps in intervention reporting across multiple areas of medicine. Because the feasibility of reproducing a study largely depends on intervention reporting, we address areas of weakness of TIDieR adherence specific to orthopaedic RCTs that, if corrected, can promote an environment that facilitates reproducible research.
Our study found consistent gaps in intervention reporting in orthopaedic trials that may limit reproducibility and have downstream effects on patient care.21,27,28 Two particular TIDieR items were most often omitted, intervention fidelity and intervention modifications. First, intervention fidelity—the continual appraisal and enhancement of reliability and internal validity of an experiment29—assesses the extent to which an intervention was delivered as intended.30 Omitting this information can have widespread negative consequences throughout biomedical research and clinical practice. For example, when fidelity is not evaluated, outcomes may be due to either an effective treatment or an unknown factor added to the intervention or overlooked during its delivery.31 A resultant type I error (the belief that a nonsignificant outcome is significant) or type II error (the belief that a significant outcome is nonsignificant) may influence future research and patient care. Second, modifications made to the intervention should be clearly documented such that results can be accurately interpreted and reproduced. For example, Koh et al.32 studied the effect of single-row versus double-row suture anchor repair in rotator cuff tears. However, after the study began, the investigators switched from metal to bioabsorbable sutures. Without this information, future attempts to replicate this study could produce conflicting results that are difficult to explain. Omitting these essential intervention details is concerning for researchers and clinicians because it prohibits early detection of protocol deviations and ultimately may result in unforeseen costs and the waste of research resources.31 Thus, we assert that authors should consider implementing strategies to assess intervention fidelity to increase the transparency, reliability, and validity of study designs and outcomes.
Since the TIDieR checklist’s publication in 2014, intervention reporting in orthopaedic trials has not experienced significant improvement. Our time-series analysis showed a slight yet nonsignificant increase in adherence to TIDieR items in trials published after the TIDieR checklist’s release. These findings suggest that the TIDieR checklist has fallen short of its desired effect. Previous studies have offered several explanations for why reporting guidelines are not used appropriately. For example, Hirst and Altman33 have suggested that reporting guidelines are not used appropriately because authors lack awareness of the newest guidelines and may not understand their usefulness. As an attainable solution to this problem, we recommend the incorporation of the TIDieR checklist directly into the more well-known, -used, and -cited CONSORT checklist for RCT reporting.34 This integration would be seemingly effortless because the TIDieR checklist is a direct extension of item 5 of the CONSORT guidelines, which only addresses the how and when regarding trial intervention. In our sample, items 6 (“How?”) and 8 (“When?”) of the TIDieR checklist were adequately reported in 96% of orthopaedic RCTs (168 of 175). These findings indicate that authors provided sufficient detail to comply with item 5 of the widely used CONSORT guidelines but failed to report the necessary level of detail needed to fulfill all aspects of the TIDieR checklist. Considering that 7 of the 9 journals from which our sample of orthopaedic RCTs was collected currently endorse or require author adherence to the CONSORT guidelines, we contend that merging the TIDieR checklist with the CONSORT guidelines might increase authors’ awareness—and subsequent implementation—of the TIDieR checklist in orthopaedic RCTs.
If integrating the TIDieR checklist into the current CONSORT guidelines is not plausible, we recommend that journals at least consider endorsing the TIDieR checklist alongside other reporting guidelines. Previous studies have shown the influence of journal endorsement on the quality of RCT reporting. For example, a 2012 systematic review of 53 meta-analyses reporting results from over 16,000 RCTs found increased RCT reporting quality for trials published in journals endorsing the CONSORT guidelines compared with non-endorsing journals.35 Currently, some journals require that authors adhere to both the CONSORT and TIDieR guidelines when submitting the results of an RCT for review. For instance, the Journal of Orthopaedic & Sports Physical Therapy currently requires authors to submit a completed CONSORT and TIDieR checklist at the time of submission; otherwise, the manuscript “will be returned to the authors.”36 Enforcing similar requirements by journals in orthopaedic surgery might be the first step in increasing the quality of orthopaedic RCT intervention reporting.
Several influential parties in the medical community have promoted the use of reporting guidelines to increase the transparency and reliability of study outcomes. For instance, the International Committee of Medical Journal Editors37 and the National Library of Medicine38 both encourage journals to promote and monitor reporting standards of submitted manuscripts. In rehabilitation medicine, an editorial published by 28 journal editors has emphasized their journals’ decisions to adopt a more aggressive stance on reporting guidelines.39 A similar alliance in the orthopaedic literature might help raise awareness and increase the standards for adequate intervention reporting. Ultimately, questions remain about educating researchers on how to use and adopt reporting guidelines. Some investigators offered the use of software programs to facilitate more complete RCT reporting. One such program uses an online writing tool that applies the CONSORT Statement during the initial drafting of the trial.40 This program outlines all essential CONSORT items in a bulleted fashion, followed by examples of proper reporting practices. The use of this software has been associated with more complete reporting of CONSORT items40; thus, a similar Web-based program using the TIDieR checklist might be worth exploring.
Limitations
Regarding limitations, we used a random sample of RCTs from the published orthopaedic literature; therefore, our results may not be generalizable across the breadth of the entire field. In addition, we evaluated only orthopaedic RCTs. The same study performed in a different area of medicine might yield different outcomes. Finally, with the TIDieR checklist’s recent publication in 2014, it is possible that a similar study conducted at a later date might yield differing outcomes.
Conclusions
Our results suggest suboptimal reporting of orthopaedic RCT interventions. In addition, the TIDieR checklist’s intended effect—to better the quality of RCT intervention reporting—appears to have fallen short of its goal.
Footnotes
The authors report the following potential conflicts of interest or sources of funding: This study was funded through the 2019 Presidential Research Fellowship Mentor–Mentee Program at Oklahoma State University Center for Health Sciences. M.V. receives grant support from National Institute on Drug Abuse– National Institute on Alcohol Abuse and Alcoholism and U.S. Office of Research Integrity, Oklahoma Center for Advancement of Science and Technology and receives internal grants from Oklahoma State University Center for Health Sciences, outside the submitted work. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Supplementary Data
References
- 1.Wright J.G., Swiontkowski M.F., Heckman J.D. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85:1–3. [PubMed] [Google Scholar]
- 2.Chen X., Zhai X., Wang X., Su J., Li M. Methodological reporting quality of randomized controlled trials in three spine journals from 2010 to 2012. Eur Spine J. 2014;23:1606–1611. doi: 10.1007/s00586-014-3283-1. [DOI] [PubMed] [Google Scholar]
- 3.Harris J.D., Erickson B.J., Abrams G.D., et al. Methodologic quality of knee articular cartilage studies. Arthroscopy. 2013;29:1243–1252.e5. doi: 10.1016/j.arthro.2013.02.023. [DOI] [PubMed] [Google Scholar]
- 4.Montané E., Vallano A., Vidal X., Aguilera C., Laporte J.-R. Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: A systematic review. BMC Clin Pharmacol. 2010;10:2. doi: 10.1186/1472-6904-10-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chan S., Bhandari M. The quality of reporting of orthopaedic randomized trials with use of a checklist for nonpharmacological therapies. J Bone Joint Surg Am. 2007;89:1970–1978. doi: 10.2106/JBJS.F.01591. [DOI] [PubMed] [Google Scholar]
- 6.Schulz K.F., Altman D.G., Moher D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2011;154:291. doi: 10.7326/0003-4819-154-4-201102150-00016. [DOI] [PubMed] [Google Scholar]
- 7.Chess L.E., Gagnier J. Risk of bias of randomized controlled trials published in orthopaedic journals. BMC Med Res Methodol. 2013;13:76. doi: 10.1186/1471-2288-13-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hoffmann T.C., Glasziou P.P., Boutron I., et al. Better reporting of interventions: Template for Intervention Description and Replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. doi: 10.1136/bmj.g1687. [DOI] [PubMed] [Google Scholar]
- 9.Murad M.H., Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22:139–142. doi: 10.1136/ebmed-2017-110713. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jellison S. TIDieR orthopedic surgery. 10.17605/OSF.IO/EYH5F. Published 2019. Accessed July, 1, 2019. [DOI]
- 11.Lai R., Chu R., Fraumeni M., Thabane L. Quality of randomized controlled trials reporting in the primary treatment of brain tumors. J Clin Oncol. 2006;24:1136–1144. doi: 10.1200/JCO.2005.03.1179. [DOI] [PubMed] [Google Scholar]
- 12.Thabane L., Chu R., Cuddy K., Douketis J. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Int J Obes. 2007;31:1554–1559. doi: 10.1038/sj.ijo.0803640. [DOI] [PubMed] [Google Scholar]
- 13.Ghimire S., Kyung E., Kang W., Kim E. Assessment of adherence to the CONSORT Statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials. 2012;13:77. doi: 10.1186/1745-6215-13-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Checketts J.X., Sims M.T., Detweiler B., Middlemist K., Jones J., Vassar M. An evaluation of reporting guidelines and clinical trial registry requirements among orthopaedic surgery journals. J Bone Joint Surg Am. 2018;100:e15. doi: 10.2106/JBJS.17.00529. [DOI] [PubMed] [Google Scholar]
- 15.Samaan Z., Mbuagbaw L., Kosa D., et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–188. doi: 10.2147/JMDH.S43952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Turner L., Shamseer L., Altman D.G., et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11 doi: 10.1002/14651858.MR000030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hopewell S., Ravaud P., Baron G., Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: Interrupted time series analysis. BMJ. 2012;344:e4178. doi: 10.1136/bmj.e4178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Linden A. Conducting interrupted time-series analysis for single- and multiple-group comparisons. Stata J. 2015;15:480–500. [Google Scholar]
- 19.Hanley J.A. Statistical analysis of correlated data using generalized estimating equations: An orientation. Am J Epidemiol. 2003;157:364–375. doi: 10.1093/aje/kwf215. [DOI] [PubMed] [Google Scholar]
- 20.Hughes BT. TIDieR data analysis. https://doi.org/https://doi.org/10.17605/OSF.IO/3A46J. Published 2019. Accessed July 25, 2019
- 21.Duff J.M., Leather H., Walden E.O., LaPlant K.D., George T.J. Adequacy of published oncology randomized controlled trials to provide therapeutic details needed for clinical application. J Natl Cancer Inst. 2010;102:702–705. doi: 10.1093/jnci/djq117. [DOI] [PubMed] [Google Scholar]
- 22.Hoffmann T.C., Erueti C., Glasziou P.P. Poor description of non-pharmacological interventions: Analysis of consecutive sample of randomised trials. BMJ. 2013;347:f3755. doi: 10.1136/bmj.f3755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Glasziou P., Meats E., Heneghan C., Shepperd S. What is missing from descriptions of treatment in trials and reviews? BMJ. 2008;336:1472–1474. doi: 10.1136/bmj.39590.732037.47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Schroter S., Glasziou P., Heneghan C. Quality of descriptions of treatments: A review of published randomised controlled trials. BMJ Open. 2012;2 doi: 10.1136/bmjopen-2012-001978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Chantebel R., Chesneau A., Tavernier E., El-Hage W., Caille A. Completeness of descriptions of repetitive transcranial magnetic stimulation intervention: A systematic review of randomized controlled trials of rTMS in depression. J ECT. 2019;35:7–13. doi: 10.1097/YCT.0000000000000546. [DOI] [PubMed] [Google Scholar]
- 26.Yamato T., Maher C., Saragiotto B., et al. The TIDieR checklist will benefit the physiotherapy profession. Physiother Theory Pract. 2017;33:267–268. doi: 10.1080/09593985.2017.1304065. [DOI] [PubMed] [Google Scholar]
- 27.Jacquier I., Boutron I., Moher D., Roy C., Ravaud P. The reporting of randomized clinical trials using a surgical intervention is in need of immediate improvement: A systematic review. Ann Surg. 2006;244:677–683. doi: 10.1097/01.sla.0000242707.44007.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Yamato T.P., Maher C.G., Saragiotto B.T., Hoffmann T.C., Moseley A.M. How completely are physiotherapy interventions described in reports of randomised trials? Physiotherapy. 2016;102:121–126. doi: 10.1016/j.physio.2016.03.001. [DOI] [PubMed] [Google Scholar]
- 29.Borrelli B., Sepinwall D., Ernst D., et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73:852–860. doi: 10.1037/0022-006X.73.5.852. [DOI] [PubMed] [Google Scholar]
- 30.Moncher F.J., Prinz R.J. Treatment fidelity in outcome studies. Clin Psychol Rev. 1991;11:247–266. [Google Scholar]
- 31.Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(suppl 1):S52–S63. doi: 10.1111/j.1752-7325.2011.00233.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Koh K.H., Kang K.C., Lim T.K., Shon M.S., Yoo J.C. Prospective randomized clinical trial of single- versus double-row suture anchor repair in 2- to 4-cm rotator cuff tears: Clinical and magnetic resonance imaging results. Arthroscopy. 2011;27:453–462. doi: 10.1016/j.arthro.2010.11.059. [DOI] [PubMed] [Google Scholar]
- 33.Hirst A., Altman D.G. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One. 2012;7 doi: 10.1371/journal.pone.0035621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Moher D., Schulz K.F., Altman D., CONSORT Group The CONSORT Statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials 2001. Explore. 2005;1:40–45. doi: 10.1016/j.explore.2004.11.001. [DOI] [PubMed] [Google Scholar]
- 35.Turner L., Shamseer L., Altman D.G., Schulz K.F., Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev. 2012;1:60. doi: 10.1186/2046-4053-1-60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Journal of Orthopaedic & Sports Physical Therapy. Author center. https://www.jospt.org/page/authors
- 37.International Committee of Medical Journal Editors Recommendations. Preparing for submission. http://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html
- 38.National Library of Medicine, National Institutes of Health. https://www.nlm.nih.gov/ Published October 1993. Accessed August 28, 2019.
- 39.Chan L., Heinemann A.W., Roberts J. Elevating the quality of disability and rehabilitation research: Mandatory use of the reporting guidelines. Ann Phys Rehabil Med. 2014;57:558–560. doi: 10.1016/j.rehab.2014.09.011. [DOI] [PubMed] [Google Scholar]
- 40.Barnes C., Boutron I., Giraudeau B., Porcher R., Altman D.G., Ravaud P. Impact of an online writing aid tool for writing a randomized trial report: The COBWEB (CONSORT-based WEB tool) randomized controlled trial. BMC Med. 2015;13:221. doi: 10.1186/s12916-015-0460-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.