Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond

Reports of randomized controlled trials (RCTs) are the only tangible evidence of their conduct. Yet historians may well view the first 50 years of reporting of RCTs with some surprise. They will encounter what might be described as a cognitive dissonance: a disconnection between the increasing sophistication of the design (and the swelling cost) of these studies and the apparent lack of care — disastrous in some cases — with which they have been reported.

Improvements in the quality of reporting of RCTs must be based on strong evidence. A series of studies beginning in 1995 found empirical evidence that results may be biased when trials use inferior methods or are reported without adequate description of the methods; notably, failure to conceal the allocation process is associated with an exaggeration of the effectiveness of an intervention of 30% or more.^1,2 The cause for concern is obvious: if the conduct or reporting of RCTs is poor, treatments may be introduced that are less effective than was thought or that might even be ineffective. This concern led the Consolidated Standards of Reporting Trials (CONSORT) group to formulate a series of recommendations in 1996³ and 2001.^4,5 This commentary summarizes developments within CONSORT since 2001.

In the 18 months since their publication, the revised CONSORT Statement⁴ and an explanatory document⁵were downloaded approximately 25 000 times from the Web sites of the 3 journals initially publishing them (the Annals of Internal Medicine, the Journal of the American Medical Association and The Lancet) and were cited nearly 500 times. There are about 20 000 hits weekly to the CONSORT Web site (www.consort-statement.org). International editorial groups, such as the World Association of Medical Editors and the Council of Science Editors, have endorsed the statement, as have several hundred biomedical journals, including this one.

Three studies have suggested that the quality of RCT reporting in journals that have endorsed the CONSORT statement is superior to that in journals that have not endorsed the statement.^6,7,8 Even so, all journals continue to publish suboptimal reports of RCTs, and the global effort to improve the quality of reporting clinical research is far from over.

When formulating and revising the CONSORT recommendations for reporting the results of RCTs, the CONSORT group focused initially on trials that used a parallel group design with 2 treatment groups or “arms,” as this design is the most common in clinical research. However, a substantial minority of trials use alternative designs. Although most elements of the CONSORT statement apply equally to these other designs, certain elements need to be adapted, and in some cases additional elements need to be added for adequate reporting. Thus, our group is now developing CONSORT “extension papers” to fill in the gaps. A CONSORT extension paper for reporting randomized cluster (group) designs was published recently.⁹ Other extension papers in development consider equivalence and noninferiority, multiarmed parallel design, factorial (a special case of multiarmed) design, crossover design, and concurrent within individual trial designs. The 6 articles will have a standard structure, mirroring the features of previous publications.^4,5

The poor quality of reporting of harm (in terms of safety and side effects) in RCTs has recently received considerable attention. Of 60 RCTs on antiretroviral treatment involving at least 100 patients, only a minority provided reasons for and numbers per arm of withdrawals resulting from toxic effects and numbers of participants with severe or life-threatening clinical adverse events.¹⁰These observations have been validated in a substantially larger study of 192 trials covering antiretroviral therapy and 6 other clinical content areas.¹¹ To help address these problems the CONSORT Group developed a paper similar in format to the other extension papers. Ten recommendations that clarify harms-related issues are each accompanied by an explanation and examples to highlight specific aspects of proper reporting. For example, fever in vaccine trials may be defined with different cut-offs and may be measured at various body sites and at different times after vaccination.¹² The results of such assessments are obviously problematic. The fourth recommendation asks authors to report whether the measurement instruments used to assess adverse events were standardized and validated. The CONSORT paper for reporting harms is forthcoming.

Several related initiatives have followed a similar model, aimed at improving the quality of reporting of other types of research: QUOROM for systematic reviews of randomized trials,¹³ MOOSE for systematic reviews of observational studies,¹⁴ STARD for diagnostic tests^15,16 and (in the near future) guidelines for reports of controlled trials of botanical medicines and STROBE for observational studies. The groups developing these standards are seeking ways to work together under a unified “umbrella” so that they can capitalize on important synergies.

Continual review and updating of CONSORT are essential. Maintaining the activities of the CONSORT Group requires considerable effort, and a mechanism has been developed to monitor the evolving literature and help keep the CONSORT statement evidence-based. For some items on the CONSORT checklist, there is already solid evidence of an effect on the validity of the trial being reported. Methodological research validating other items is reported in a diverse set of journals, books and proceedings. To bring this body of evidence together, several CONSORT members have formed the ESCORT working party. They are starting to track down, appraise and annotate reports that provide Evidence Supporting (or refuting) the CONSORT Standards On Reporting Trials. The ESCORT group would appreciate receiving citations of reports readers consider relevant to any items on our checklist (via the CONSORT Web site).

These efforts would all benefit substantially if innovative funding mechanisms existed to provide practical support to build on and strengthen existing working relationships and extend this endeavour into new areas. However, with the exception of 5 years of a small amount of regular funding from the US National Library of Medicine, funding has been sporadic. Inconsistent funding does not provide the stability to enable long-term planning and execution of global efforts. Funding agencies spend millions of dollars to conduct RCTs, yet without sufficient attention to how those trials are conducted and reported, biases and misrepresentations will increasingly creep into their results and interpretations. Another paradox is that our requests for financial support to allow these groups to work together seem to fall on deaf ears, even though funding organizations are pushing for more international, interdisciplinary collaborations across multiple institutions, the exact makeup of CONSORT and the other groups.

More than 40 000 RCTs are now actively recruiting participants.¹⁷ If the results of these studies are to become the foundation of evidence-based health care, considerable energies and resources must be spent on improving their reporting.