Is research quality in orthopedic manual therapy trials stagnating? Reflections and pathways for improving research quality and advance our profession

It has been recognized that inadequate reporting of essential elements in research studies hinders the systematic review process, often leaving reviewers unable to make definitive conclusions.[1] Two very basic elements of any research article should be that it contains enough information so that the findings can be reproduced and that it has all of the information needed by the authors of a meta-analysis and systematic review to draw appropriate conclusions.[2] Most meta-analyses arrive at an almost universal conclusion: there is insufficient high quality evidence to make definitive conclusions on anything. Is it because we as researchers are putting forth bad research? Is it the current peer review process? Is it that the important aspects of reproducibility of the study are left out in the editorial process? Incomplete reporting of the basic aspects of the research design may be killing the systematic review process and be the reason for the universal conclusions of meta-analyses.[1]

The current process of how research is reported is problematic and ultimately wastes valuable time and resources.[3] This problem is widespread and can affect all disciplines of health care research.[4–6] The factors recognized to influence the use of reporting guidelines include: individual factors; professional cultural factors; journal policy and procedural factors; and practical factors such as the cost of printing.[7] Others have suggested a less savory side to reporting issues which include: questionable conduct, inaccurate reporting, and lack of transparency.[8] Can the conclusions of a study be justified if you cannot identify the hypothesis, the research question, the research design, the randomization and blinding processes, who received the intervention, and what type of analysis was used?[9]

Greater use of structure within research manuscripts should improve reporting by authors, reviewers, and editors, which should assist in the creation of meta-analyses and systematic reviews.[10] There is no evidence that the use of reporting guidelines improves reporting.[7] Although hundreds of journals endorse the use of reporting tools such as the CONSORT, a number of reviews suggest that this has not improved reporting in clinical trials.[11–14] Numerous studies have found discrepancies between published research trials and what was reported on clinicaltrials.gov.[15–17] Potential reasons for these discrepancies include errors in reporting, limited space for publication in the journal, and the tendency to only report positive endpoints and results.[14–17]

We applaud the recent position of the International Society of Physiotherapy Journal Editors (ISPJE) to endorse the Template for Intervention Description and Replication (TIDieR) checklist in addition to the CONSORT guidelines for reporting of randomized clinical trials (RCTs).[18] Our study [19] published in this issue of JMMT suggests that part of the problem may be that reporting guidelines such as the CONSORT are unreliable and subjective in nature. Will the addition of the TIDieR checklist to the CONSORT help improve the reliability and completeness of reporting in RCTs? Identifying the reliability of the CONSORT with the addition of the TIDieR checklist may help identify the strengths and weaknesses of this new instrument, help in the evolution of this new tool, and ultimately improve the reporting of clinical trials.

We urge reviewers and journal editorial board members to be attentive to the completeness of the reporting of trials when critically appraising manuscripts. We urge clinicians and researchers planning their study to (1) register their observational study, clinical trial to clinicaltrials.gov before the start of data collection. This allows public viewing and optimal trial scrutiny; (2) include CONSORT, STROBE, STARD, and other reporting tools upon manuscript submission. Those should be submitted as Appendices that JMMT publishes electronically; (3) use Appendices to provide data required in the reporting tools when authors experience difficulties meeting the maximum number of words allowed for manuscript length. These data will be published electronically. Taking these few steps may help raise research quality and transparency of published trials and hopefully will lead to systematic reviews and meta-analyses able to conclude about the effectiveness or no effectiveness of tested interventions. This will improve research scientific rigor and help advance our profession.

Sean P. Riley
Doctor of Physical Therapy Program, Sacred Heart University, CT, USA
Brian T. Swanson
Department of Physical Therapy, University of New England, ME, USA
Steven F. Sawyer
Department of Rehabilitation Sciences and Center for Rehabilitation Research, Texas Tech University Health Sciences Center, TX, USA
Jean-Michel Brismée
Editor-in-Chief – Journal of Manual and Manipulative Therapy
Texas Tech University Health Sciences Center, Lubbock, TX, USA
jm.brismee@ttuhsc.edu