STROBE, CONSORT, PRISMA, MOOSE, STARD, SPIRIT, and other guidelines – Overview and application

Abstract

The purpose of research is to seek answers and new knowledge. When conducted properly and systematically, research adds to humanity's corpus of knowledge and hence to our general advancement. However, this is only possible if reported research is accurate and transparent. Guidelines for all the major types of studies (STROBE, CONSORT, PRISMA, MOOSE, STARD, and SPIRIT) have been developed and refined over the years, and their inception, development, and application are briefly discussed in this paper. Indeed, there are currently over 250 of these guidelines for various types of medical research, and these are published by the EQUATOR network. This paper will also briefly review progress in acceptance and adoption of these guidelines.

Introduction

The Emperor Marcus Aurelius famously stated that “Nothing has such power to broaden the mind as the ability to investigate systematically and truly all that comes under thy observation in life.”[1] The purpose of research is to seek answers to questions or problems and to understand phenomena, and behavior or to test theories or hypotheses. When conducted properly and systematically, research adds to humanity's corpus of knowledge and hence to our general advancement. However, this is only possible if the quality of the reported research is accurate and transparent, and this was noted as far back as in 1938 with the observation that “… incompleteness of evidence is not merely a failure to satisfy a few highly critical readers. It not infrequently makes the data that are presented of little or no value.”[2] The first tangible step to accomplish this was a set of guidelines published by the International Committee of Medical Journal Editors (ICMJE) in 1988, which urged the need for total transparency, urging authors to “describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results.”[3] This was the basis of the CONSORT (Consolidated Standards of Reporting Trials) statement, ten years later in 1996, which averred that:

• The randomized controlled trial (RCT), more than any other methodology, can have a powerful and immediate impact on patient care … needs to convey … relevant information concerning the design, conduct, analysis, and generalizability of the trial … provide the reader with the ability to make informed judgments regarding the internal and external validity of the trial. Accurate and complete reporting also benefits editors and reviewers in their deliberations regarding submitted manuscripts. For RCTs to ultimately benefit patients, the published report should be of the highest possible standard.[4]

Guidelines for all of the major types of studies have been developed and refined over the years, and these will be discussed in this paper, but it must be noted that there are currently over 250 of these guidelines for various types of medical research and these are published by the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network, an international initiative that was set up in 2006 to promote high-quality reporting of health research studies through the wider usage of reporting guidelines, along with free online resources that facilitate these aspirations.[5]

GRADE: Grading of Recommendations Assessment, Development, and Evaluation

The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group was formed in 2000 to assess and address the limitations of grading systems in health care, and the GRADE approach is now considered the standard to follow in the development of guidelines.[6] A grading system is vital as clinical judgements about validity of evidence and wider recommendations in health care are complex decisions. This type of approach may help to prevent error/s, accelerate the critical appraisal of these judgments, and improve the communication of such information.[6] According to GRADE, systematic reviews and meta-analyses of RCTs comprise the highest level of evidence, followed by individual RCTs, nonrandomized trials, observational designs (e.g. cohort studies and case-control studies), and, lastly, case studies and expert opinions (also known as anecdotal evidence). GRADE also classifies quality of evidence as high, moderate, low, and very low, with RCTs as high quality and observational studies as low-quality evidence. However, a particular study's level may be demoted if there are issues with study design/implementation, imprecision leading to excessive confidence intervals, etc., Quality may however be promoted if the study investigates a large magnitude effect and/or the study fulfils the converse of the previously mentioned limitations. A guideline's formulation should also include a clear question with clear patient outcomes.[6] The Renaissance artist Michelangelo (1475-1564) famously emphasized the importance of “disegno” design.[7] However, well-designed studies do not necessarily lead to transparency—design is equally crucial for clarity and transparency of the presentation of methods used and obtained results.[8] Indeed, readers should bear in mind two sets of “tripods” when reading research. One tripod comprises the conflicting forces of researchers/authors striving to publish because of the publish or perish mantra, readers wishing to read less due to information overload, and journal editors' drives to increase their journals' impact factors.[9] The second tripod is that researchers use previously published papers as direction on how to perform research and to gauge whether obtained results have significance, clinicians use papers as guides to best treat patients, and public health uses research to devise cost-effective prevention and treatment policies.[8] Clearly, guidelines are crucial for all these reasons. This paper reviews the inception, development, and use of the six commonest guidelines: PRISMA, CONSORT, STROBE, MOOSE, STARD, and SPIRIT.

Guidelines

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses)

A study in 1987 of the methodologies of a sample of 50 review articles (from four leading journals in 1985 and 1986) found that none met a set of eight explicit scientific criteria:

1. Was the specific purpose of the review stated?

2. Were sources and methods of the citation search identified?

3. Were explicit guidelines provided that determined the material included in and excluded from the review?

4. Was a methodologic validity assessment of material in the review performed?

5. Was the information systematically integrated with explication of data limitations and inconsistencies?

6. Was the information integrated and weighted or pooled metrically?

7. Was a summary of pertinent findings provided?

8. Were specific directives for new research initiatives proposed?[10]

A more exhaustive analysis was carried out in 1987 by evaluating 83 meta-analyses with a 23-item scoring system and reached the same conclusions.[11] This led an multinational team of 30 clinical epidemiologists, researchers, statisticians, clinicians, and editors to create the Quality of Reporting of Meta-analyses (QUOROM) checklist and flow diagram for the meta-analyses of RCTs in 1996. The flow diagram was deemed to increase the transparency behind the decisions to include/exclude studies, decisions which might introduce bias.[12] QUOROM was modernized in 2009 and renamed PRISMA,[13] updated in 2020,[14] and further enhanced the following year with PRISMA-S, a “PRISMA for Searching” guide.[15] The 27-item PRISMA checklist also includes a four-phase flow diagram, in an effort to provide “helpful resources to improve reporting of systematic reviews and meta-analyses.”[13] PRISMA is effective not only when recommended by journals in instructions to authors but also enforced. For example, in 2011, a sample of 146 journals showed that PRISMA was incorporated in their author guidelines in only 27%, and more often in general/internal medicine journals than in specialty medicine journals (50 vs. 25%).[16] Furthermore, a study in 2013 showed that only circa a third of medical journals recommended PRISMA. That same study showed that for the previous year, a study of systematic reviews in journals that endorsed PRISMA included 90% of the PRISMA checklist, and 5% fewer items were found in papers from journals that did not endorse PRISMA. Adherence was particularly high for PRISMA item 17 (study selection) (100.0% vs. 63.3%).[17] A very recent study of systematic reviews and meta-analyses in the top five emergency medicine-related journals (based on their 5-year impact factor) similarly showed that PRISMA was not uniformly applied, and sometimes applied albeit with lacunae.[18]

CONSORT (Consolidated Standards of Reporting Trials)

“The whole of medicine depends on the transparent reporting of clinical trials.”[19] Evidence that researchers reported trials inadequately due to bias led to experts including medical journal editors, researchers, epidemiologists, and methodologists to meet in 1993 and create the SORT statement.[20,21] This was a 32-item checklist and flow diagram used to describe how a clinical trial was conducted. Independently and concurrently, a second group of experts also produced a similar set of guidelines.[22] In 1995, members of both groups met and merged the two into the CONSORT statement,[4] with a revision in 2001,[23] and an update in 2010.[24]

Consort is currently a 25-item checklist and a flow diagram. A comparison of RCTs before 1994 and after 1998 showed that there was an improvement in the reporting of important checklist items, albeit not to the desired, complete levels.[25,26]

STROBE (STrengthening the Reporting of OBservational studies in Epidemiology)

The STROBE Statement was created by the STROBE Initiative in 2007, an international collaboration of researchers, epidemiologists, methodologists, statisticians, and journal editors to try to rectify known lacunae in observational studies.[27] This initiative was inspired by the CONSORT Statement. STROBE's purpose is not only to aid authors to correctly report observational studies (cohort, case-controlled, and cross-sectional research) but also to assist reviewers and journal as well as readers to critically appraise such studies.[28] The statement consists of a 22-item checklist with extensions that cover subspecialities in medicine.[8] One of the better known extensions is STREGA (STrengthening the REporting of Genetic Association studies), which is utilized in genetic association studies.[29]

MOOSE (Meta-Analysis of Observational Studies in Epidemiology)

MOOSE is a 35-item checklist for epidemiological meta-analyses, of observational studies, that was created in 1997.[30] Although the abovementioned PRISMA is used for systematic reviews and meta-analyses, not all evidence can be synthesized from such studies. Furthermore, it may not be feasible or possible to conduct RCTs in certain topics. Moreover, such studies may simply not be available.[31] For these reason/s, a synthesis of observational studies may be suitable and complementary.[32,33] Additional advantages of observational studies are that they are able to identify/summarize rare events as the number of subjects may be larger than those that can be recruited in RCTs, and may also permit longer long-term follow-ups.[33] The most significant limitation however is the potential inability to avoid or balance bias, particularly selection bias.[32,33]

STARD (STAndards for the Reporting of Diagnostic accuracy studies)

In studies that deal with diagnostic accuracy, outcomes are compared with a standard, and it has been shown that such studies may be biased, with overoptimistic estimates of diagnostic accuracy.[34] In 2000, inspired by CONSORT, researchers, editors, and methodologists created a 25-item STARD checklist and flow diagram.[35] A study in 2008 showed that this guideline had had limited effect,[36] with some improvement by 2013.[37]

SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials)

Many RCTs used to lack protocol information detailing essential trial components, including vital information such as primary outcome/s, treatment allocation methods, and the utilization of blinding/masking methods.[38] For this reason, in 2007, a group of researchers, trial coordinators, methodologists, ethicists, statisticians, and journal editors created SPIRIT, a 33-item list and diagram inspired by CONSORT.[39] This naturally enables a SPIRIT-driven protocol to easily transition to a CONSORT-formatted paper.[8] SPIRIT also mandates the registration of a trial with requisite domains (e.g. https://clinicaltrials.gov/) to ensure transparency in execution and reporting.[8]

Discussion

CONSORT has led to a positive wave of transformations for medical research reporting, improving quality and transparency, and facilitating journal workflows and the peer-review process. Partially for this reason, COPE (Committee on Publication Ethics) was founded in 1997 by a group of UK medical editors to deliberate on examples of possible research/publication misconduct, and the deliberations of cases are published regularly in an anonymized format as a guide to appropriate action in similar situations for other editors. COPE is endorsed by most journals and lays out essential standards for the peer-review process. However, not all authors and journals adhere to the respective guidelines, inadvertently aided, and abetted by journal editors and peer reviewers who have neither the time nor the financial wherewithal nor perhaps the inclination to enforce guidelines and checklists. The obligation of conforming to the appropriate publication guidelines therefore devolves solely on author/s. Despite these guidelines, various reasons have been proposed for nonadherence and these include:[40]

• Authors may feel excessively constrained.

• Word count limitations may preclude the inclusion of all details.

• Guidelines may encourage fabrication of spurious information to fulfil the statement obligations.

Although there has been an overall improvement in the uptake and therefore the quality of published papers, much remains to be done by authors, reviewers, and editors. Ongoing reviews of this topic continue to reveal slow improvements.[41,42,43]

In conclusion, reporting guidelines in medical research should result in accurate and transparent reporting, allowing facile appraisal of findings. This goal is slowly being achieved, and the next hurdle-facing editors and reviewers will be artificially generated research by artificial intelligence programs.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.