Abstract
BACKGROUND:
In the busy world of cardiovascular medicine, abstracts may be the only part of a publication that clinicians read. Therefore, it is critical for abstracts to accurately reflect article content. The extended CONSORT (Consolidated Standards of Reporting Trials) Statement for Abstracts was developed to ensure high abstract quality. However, it is unknown how often adherence to CONSORT guidelines occurs among cardiovascular journals.
METHODS AND RESULTS:
We searched MEDLINE for randomized controlled trials published in 3 major cardiovascular journals (Circulation, Journal of the American College of Cardiology, and European Heart Journal) from 2011 to 2017. Post hoc, interim, and cost-effective analyses of randomized controlled trials were excluded. Two independent investigators extracted the data using a prespecified data collection form and a third investigator adjudicated the data. The primary outcome was frequency of subcategory adherence to CONSORT guidelines. A total of 478 abstracts were included in the analysis. Approximately half of the abstracts (53%; 255/478; 95% CI, 49%–57%) identified the article as randomized in the title. All abstracts detailed the interventions for both study groups (100%) and 81% (95% CI, 78%–85%) reported trial registration. Methodological quality reporting was relatively low: 9% (45/478; 95% CI, 6%–12%) described participant eligibility criteria with settings for data collection, 43% (204/478; 95% CI, 39%–47%) reported details of blinding, and <1% (4/478; 95% CI, 0%–2%) reported allocation concealment. Approximately 60% (301/478; 95% CI, 59%–67%) of the included abstracts provided primary outcome results while 55% (262/478; 95% CI, 51%–60%) reported harms or adverse effects.
CONCLUSIONS:
There is a high prevalence of nonadherence to CONSORT guidelines among leading cardiovascular journals. Efforts by editors, authors, and reviewers should be made to increase adherence and promote transparent and unbiased presentation of study results.
Keywords: cardiology, medicine, prevalence, publications, randomized controlled trials
Randomized controlled trials (RCTs) have the highest impact in the hierarchy of research designs.1 Considered the gold standard in assessing patient care interventions, RCTs play a critical role in developing new treatment regimens in medicine by the elimination of selection and confounding biases.2,3 Therefore, the reporting of data must be clear, transparent, and complete about the design, conduct, analysis, and interpretation of the trial.4 However, inadequate reporting of trials may lead to incorrect conclusions by preventing the reliable assessment of trial methods and biases.5 Hence, accurate reporting is essential for readers, especially medical professionals, to assess the quality and validity of the trial to make well-informed judgments when applying study results to patients.6 When comparing full-text articles to their abstracts, over 10% of published articles have considerable differences in their conclusions.7 Because the abstract may be the only section clinicians read, inappropriate and incomplete representation of data may lead to the improper application of results and, therefore, poorer patient outcomes.8,9
Consequently, to establish a standard for accurate reporting of data, the CONSORT (Consolidated Standards of Reporting Trials) statement was developed in 19966 and updated in 2001, 2007, and 2010.10–12 In 2008, the original CONSORT statement developed CONSORT for Abstracts13 as an extension to ensure high standards in journal and conference abstract quality. Prior studies have determined adherence to the CONSORT abstract checklist for RCTs in various fields of medicine.3,4,14–17 However, no study has been conducted focusing only on cardiovascular journals. To fill this knowledge gap, we conducted a study to assess adherence of abstracts to CONSORT checklist in 3 top-tier cardiovascular journals.
METHODS
Data Sources
In June 2018, we conducted a descriptive, cross-sectional study of RCT abstracts in 3 top-tier Cardiovascular journals, namely: Journal of the American College of Cardiology (JACC), European Heart Journal (EHJ), and Circulation. JACC and Circulation endorse CONSORT according to the endorsers’ section on the CONSORT statement website.18 Circulation recommends following International Committee of Medical Journal Editors’ Uniform Requirements for Manuscripts Submitted to Biomedical Journals, where the composition of the abstract is required to be in accordance with the CONSORT for Abstracts guidelines; whereas, the EHJ recommends following CONSORT guidelines for reporting of clinical trials. JACC recommends preparing structured abstracts according to a study entitled, “more informative abstracts revisited.”19–22
We conducted a MEDLINE search to identify all RCTs published between January 2011 and December 2017 in these 3 journals using the following search specifications: (“Journal of the American College of Cardiology”[Journal] OR “European Heart Journal”[Journal]) OR “Circulation”[Journal]) AND (Randomized Control Trial [type] AND has abstract[text] AND (“2011/01/01”[PDAT]: “2017/12/31”[PDAT])). No search restriction was applied.
Study Selection
Abstracts of primary RCTs published from 2011 to 2017 were selected. We included abstracts which used the terms “random,” “randomized,” and “randomly allocated” when describing the title or allocation of participants to interventions. Abstracts of studies using other designs including letters, editorials, observational studies, economic/cost effective analyses of RCTs, cohort studies, quasi-randomized trials, and post hoc/secondary analyses of previously reported RCTs were excluded.
Data Extraction, Checklist Development, and Inter-Rater Agreement
Two independent reviewers (Drs Shaikh and Ochani) assessed each abstract’s compliance with every aspect of the CONSORT statement for Abstracts checklist in a duplicative manner. The discrepancies were resolved by referring back to the published explanations for the CONSORT statement3 and by a third-party review (Dr Khan). Inter-rater agreement for each checklist item was evaluated by the chance-corrected measure of agreement, Cohen’s κ.23 The κ value obtained was 0.894.
When scoring for checklist items, all checklist items were given equal weightage of zero for not being reported or one for being reported. This distinction was made based on multiple previous studies14–17 and the fact that the CONSORT statement itself does not give varying weight to different items.3
The data extraction included the following information: random or randomized mentioned in the title, author’s contact information, trial design (eg, parallel, cluster, factorial, non-inferiority, superiority, and crossover), trial registration number, funding sources, information related to study methodology, that is, randomization (specific method of random sequence generation used), blinding (specifically who was blinded, ie, caregivers, investigators, patients, outcome assessors, or all), and allocation concealment (conducted by computer-generated sequences, telephone, or sealed envelope), numbers randomized and analyzed in study groups, participant information including eligibility criteria (specific condition(s) patients had to be selected), with or without setting of data collection (the type of health care center), interventions assigned including denomination, usage, course, and type (pharmacological, surgical, or both), study objectives (specific objectives of the study or a brief background), clearly defined primary outcome (prespecified primary outcome), primary outcome results in each group (raw numbers, P values, and effect size for each group), the presence or absence of any adverse effects and a definite conclusion.
Data for additional items beside the checklist that were extracted included the following information: Journal name, year of publication, impact factor of journal, number of authors (<4, 4–7, and >7), region where RCT was conducted (Europe, North America, Asia, and other), the major subspecialty of the study (heart failure, electrophysiology, cardiac imaging, preventive cardiology, or interventional cardiology), type of center (single or multiple), abstract length (<250 or >250), and abstract type (structured or unstructured). These additional items were collected to identify possible predictors for quality reporting.
This study was conducted in accordance with the guidelines for reporting meta-epidemiological methodology research.24 Methodological reporting quality of journals was compared with previous studies based on 3 domains: allocation concealment, whether randomization was explained and if blinding/masking was mentioned.4, 25–30
Data Analysis
Data were analyzed using Statistical Product and Service Solutions software (ver. 23.0 IBM SPSS). We determined adherence of abstracts to CONSORT for Abstracts Guidelines by calculating overall proportion (%) of RCT abstracts that included each of the individual items included in the checklist followed by a combined mean and binomial proportion CI.
RESULTS
The Figure highlights the detailed literature search process. The combined search strategy yielded 1192 abstracts initially identified in the 3 top-tier cardiovascular journals (JACC, EHJ, and Circulation). We excluded 18 abstracts that were not RCTs. Additionally, we further excluded 696 abstracts (583 post hoc/secondary analysis, 87 Insights, 12 economic/cost-effective analyses, and 14 observational studies). Our final cohort for analysis included 478 primary RCT abstracts.
Study Characteristics
Table 1 shows the number of RCTs published in the top 3 cardiovascular journals. Of the total identified, 478 RCTs, 41.4% were published in JACC (n=198/478), 28.5% in EHJ (n=136/478), and 30.1% in Circulation (n=144/478). Almost all abstracts (99.4%, n=475/478) were structured.
Table 1.
Journal Characteristics
| Journal Name | Abstract Structure | Impact Factor* | Number of RCTs Identified, n (%) (n=1174) | Number of Included RCTs, n (%) (n=478) | Use of CONSORT Endorsed |
|---|---|---|---|---|---|
| JACC | IMRAD | 16.8 | 501 (42.7) | 198 (41.4) | No |
| Circulation | IMRAD | 18.8 | 369 (31.4) | 144 (30.1) | Yes |
| EHJ | IMRAD | 23.4 | 304 (25.9) | 136 (28.5) | No† |
CONSORT indicates Consolidated Standards of Reporting Trials; EHJ, European Heart Journal; IMRAD, Introduction, Methods, Results, and Discussion; and JACC, Journal of the American College of Cardiology.
JCR 2017 impact factor.
Recommends under instructions to authors to follow CONSORT checklist but is not available on the CONSORT endorsers list.
Reporting of General Items
Table 2 shows the assessment of the CONSORT checklist in the included RCTs. Among the 478 trials, 53.3% (n=255) mentioned the term random/randomization in the title. Around three-fourths (75.7%, n=362/478) of the included RCTs gave complete details of the authors (postal and email address). Only 26.6% (n=127/478) of the abstracts mentioned the trial design (parallel, crossover, superiority, cluster, non-inferiority, or factorial).
Table 2.
CONSORT Checklist Items for Assessment From Abstracts of Included RCT
| Items | Assessment Criteria | Assessment of Individual Journals, n (%) | Overall n (%) (n=478) | ||
|---|---|---|---|---|---|
| JACC (n=198) | EHJ (n=136) | Circulation (n=144) | |||
| Title | Mentioned random in title | 97 (49) | 77 (56.6) | 81 (56.3) | 255 (53.3) |
| Authors | Addresses including postal and email | 157 (79.3) | 95 (69.9) | 110 (76.4) | 362 (75.7) |
| Trial design | Descriptions provided (parallel, factorial, crossover, etc) | 50 (25.3) | 36 (26.5) | 41 (28.5) | 127 (26.6) |
| Methods | |||||
| Participants | Eligibility criteria with settings of data collection | 18 (9.1) | 11 (8.1) | 16 (11.1) | 45 (9.4) |
| Only eligibility criteria provided | 185 (93.4) | 133 (97.8) | 138 (95.8) | 456 (95.4) | |
| Interventions | Details including denomination, usage, course of treatment for both groups | 198 (100) | 136 (100) | 144 (100) | 478 (100) |
| Objective | Specific objective/hypothesis | 197 (99.5) | 125 (91.9) | 84 (58.3) | 406 (84.9) |
| Outcome | Clearly defined primary outcome | 126 (63.6) | 95 (69.9) | 99 (68.8) | 320 (66.9) |
| Randomization | Reported the method of random sequence generation | 3 (1.5) | 6 (4.4) | 4 (2.8) | 13 (2.7) |
| Allocation concealment | … | 3 (2.2) | 1 (0.7) | 4 (0.8) | |
| Blinding | Mentioned blinding and who was blinded | 68 (34.3) | 75 (55.1) | 61 (29.9) | 204 (42.7) |
| Results | |||||
| Numbers randomized | Number of participants randomized in each group | 167 (84.3) | 122 (89.7) | 126 (87.5) | 415 (86.8) |
| Recruitment | Trial status | 42 (21.2) | 29 (21.3) | 27 (18.8) | 98 (20.5) |
| Numbers analyzed | Number of participants analyzed of each group | 155 (78.3) | 91 (66.9) | 83 (57.6) | 329 (68.8) |
| Outcomes | Primary outcome result for each group | 122 (61.6) | 88 (64.7) | 91 (63.2) | 301 (63) |
| Harms | Adverse event or side effect reported | 101 (51) | 82 (60.3) | 79 (54.9) | 262 (54.8) |
| Conclusions | Clear interpretation of trial | 198 (100) | 136 (100) | 144 (100) | 478 (100) |
| Trial registration | Reported registration number and name of trial register | 170 (85.9) | 77 (56.6) | 139 (96.5) | 386 (80.8) |
| Funding | Reported source of funding | 1 (0.5) | … | 1 (0.7) | 2 (0.4) |
EHJ indicates European Heart Journal; JACC, Journal of the American College of Cardiology; and RCT, randomized controlled trial.
Reporting of Trial Methodology
Among 478 abstracts, 95.4% stated the eligibility criteria, and only 9.4% of the abstracts mentioned the settings of data collection along with the eligibility criteria. All abstracts reported details of the intervention, including denomination, usage, and course of treatment for both groups. Only 13 (2.7%) abstracts mentioned the method of random sequence generation and only 4 (0.8%) provided information about allocation concealment. With regards to blinding, 42.7% (n=204/478) of the abstracts reported blinding and the groups who were blinded. Comparison of this study’s trial methodology with previous studies’ assessing trial methodologies is shown in Table 3.
Table 3.
Comparison of Methodological Quality Domains Between Studies
| Studies | Discipline of the Study | Allocation Concealment, n (%) | Randomization Explained, n (%) | Blinding/Masking, n (%) |
|---|---|---|---|---|
| Current study (n=478) | Cardiovascular | 4 (0.8) | 13 (2.7) | 204 (42.7) |
| Kuriyama et al42 (n=166) | Critical care | … | 3 (1.8) | 7 (4.2) |
| Hays et al25 (n=463) | General medicine | 37 (8.0) | 88 (19.0) | 137 (60.0) |
| Peters et al16 (n=18) | Otorhinolaryngology | … | 0 (0.0) | 12 (60.0) |
| Cui et al27 (n=328) | Clinical pathway | 0 (0.0) | 328 (100.0) | 29 (8.8) |
| Ghimire et al4 (n=129) | General medicine | 32 (11.8) | 84 (31.0) | 102 (37.6) |
| Mann et al28 (n=129) | Gerontology and geriatrics | … | 0 (0.0) | 21 (16.3) |
| Wang et al26 (n=345) | Traditional Chinese medicine | 0 (0.0) | 17 (4.9) | 39 (11.3) |
| Berwanger et al29 (n=227) | General medicine | 1 (0.4) | … | 92 (40.5) |
| Hopewell et al30 (n=37) | Oncology | 0(0.0) | 26(70.3) | 6 (16.2) |
Reporting of Results
Of 478 included abstracts, 86.8% (n=415) reported the number of participants randomized in each group and 68.8% (n=329) reported the number of participants analyzed in each group. Around 60% (301/478) of the included abstracts provided primary outcome results while 54.8% (262/478) reported harms or adverse effects.
Reporting of Additional Items
Table 4 shows all additional items sorted by journal. Almost all (475/478) abstracts were structured, which is a function of journal specification. The majority (89.3%, n=427/478) had >7 authors. Of the 478 abstracts, 55% (n=263) were European in origin, and 31% (n=146) focused on preventive cardiology. Around half of the abstracts (258/478) did not report whether the study was single- or multi-centered. A majority of the abstracts (383/478) exceeded the 250 words.
Table 4.
Additional Items Besides the CONSORT Checklist From the Included RCT
| Additional Items | Assessment Criteria | Assessment of Individual Journals n (%) | Overall n (%) (n=478) | ||
|---|---|---|---|---|---|
| JACC (n=198) | EHJ (n=136) | Circulation (n=144) | |||
| Abstract structure | Structured | 198 (100) | 133 (97.8) | 144 (100) | 475 (99.4) |
| Unstructured | 0 (0) | 3 (2.2) | 0 (0) | 3 (0.6) | |
| Number of authors | <4 | 2 (1) | 1 (0.7) | 1 (0.7) | 4 (0.8) |
| 4–7 | 25 (12.6) | 10 (7.4) | 12 (8.3) | 47 (9.8) | |
| >7 | 171 (86.4) | 125 (96.3) | 131 (91) | 427 (89.3) | |
| Region of publication | Europe | 96 (48.5) | 103 (75.7) | 64 (44.4) | 263 (55) |
| North America | 70 (35.4) | 16 (11.8) | 63 (43.8) | 149 (31.2) | |
| Asia | 23 (11.6) | 12 (8.8) | 12 (8.3) | 47 (9.8) | |
| Others | 9 (4.5) | 5 (3.7) | 5 (3.5) | 19 (4) | |
| Specialty | Heart failure | 25 (12.6) | 23 (16.9) | 16 (11.1) | 64 (13.4) |
| Electrophysiology | 4 (2) | 4 (2.9) | 3 (2.1) | 11 (2.3) | |
| Interventional cardiology | 58 (29.3) | 25 (18.4) | 38 (26.4) | 121 (25.3) | |
| Cardiac imaging | 3 (1.5) | 1 (0.5) | 2 (1.4) | 6(1.3) | |
| Preventive cardiology | 72 (36.4) | 45 (33.1) | 29 (20.1) | 146 (30.5) | |
| Other | 36 (18.2) | 38 (27.9) | 56 (38.9) | 130 (27.2) | |
| Centers | Single | 10 (5.1) | 5 (3.7) | 5 (3.5) | 20 (4.2) |
| Multicenter | 81 (40.9) | 53 (38.9) | 66 (45.8) | 200 (41.8) | |
| Not reported | 107 (54) | 78 (57.4) | 73 (50.7) | 258 (54) | |
| Word limit* | <250 | 16 (8.1) | 42 (30.9) | 37 (25.7) | 95 (19.9) |
| >250 | 182 (91.9) | 94 (69.1) | 107 (74.3) | 383 (80.1) | |
Word limits: JACC: <250; EHJ: <250; Circulation: <350 EHJ indicates European Heart Journal; JACC, Journal of the American College of Cardiology; and RCT, randomized controlled trial.
Assessment of Reporting Quality of the CONSORT for Abstract Checklist Items
Adherence by individual checklist item is shown in Table 5. Adherence was lowest for sources of funding (0.4%; 95% CI, 0%–2%), allocation concealment (0.8%; 95% CI, 0%–2%), and randomization (2.7%; 95% CI, 1%–5%); while it was highest for details of interventions (100%), conclusions (100%), objectives (84.9%; 95% CI, 82%–88%), numbers randomized (86.8%; 95% CI, 84%–90%), eligibility criteria designed for the patients to be included in a RCT (95.4%; 95% CI, 93%–97%), and trial registration (80.8%; 95% CI, 78%–85%).
Table 5.
Adherence by Checklist Item
| Variable | Observations | Mean | 95% CI |
|---|---|---|---|
| Title | 478 | 0.53 | 0.49–0.57 |
| Authors | 478 | 0.76 | 0.72–0.80 |
| Trial design | 478 | 0.27 | 0.23–0.31 |
| Eligibility criteria | 478 | 0.95 | 0.93–0.97 |
| Study setting | 478 | 0.09 | 0.06–0.12 |
| Interventions | 478 | 1.0 | … |
| Objective | 478 | 0.85 | 0.82–0.88 |
| Outcome | 478 | 0.67 | 0.63–0.71 |
| Randomization | 478 | 0.03 | 0.01–0.05 |
| Blinding | 478 | 0.43 | 0.39–0.47 |
| Allocation concealment | 478 | 0.01 | 0.00–0.02 |
| Numbers randomized per group | 478 | 0.87 | 0.84–0.90 |
| Recruitment | 478 | 0.21 | 0.17–0.25 |
| Numbers analyzed per group | 478 | 0.69 | 0.65–0.73 |
| Outcomes | 478 | 0.63 | 0.59–0.67 |
| Harms or adverse effects | 478 | 0.55 | 0.51–0.60 |
| Conclusions | 478 | 1.0 | … |
| Trial registration | 478 | 0.81 | 0.78–0.85 |
| Source of funding | 478 | 0.01 | 0.00–0.02 |
Reporting Quality of the CONSORT for Abstract Checklist Items Specific to Journals
In all 3 journals, <30% abstracts reported trial designs while only about 10% reported eligibility criteria with settings of data collection. Reporting of interventions and conclusions was, however, 100% across all 3 journals. Elements in the methodological domain including methods of randomization and allocation concealment were among the most poorly reported in all 3 journals. Primary outcome results were reported by <65% of abstracts across all journals: JACC (61.6%, n=122/198), EHJ (64.7%, n=88/136), and Circulation (63.2%, n=91/144).
DISCUSSION
We found that, like previous studies,4,17,25–30 abstract adherence to multiple individual CONSORT checklist items was significantly low. These included participant information, methods of random sequence generation, allocation concealment, and funding. On the contrary, author information, eligibility criteria, details about interventions, specific objectives, number of subjects randomized, conclusions, and trial registration were adequately reported (>75%). This observation is congruous with a previous study,4 which showed similar adequate levels of reporting of these items. This variation in compliance, with certain items being reported more adequately, such as eligibility criteria, as compared to others, such as methods of randomization, shows that authors and journals do not consider certain items valuable. Updating the CONSORT guidelines to define which items hold greater value could prove to be beneficial. Furthermore, the fact that the World Health Organization established an international clinical trial registry31 and that the International Committee of Medical Journal Editors follows an austere policy of only publishing registered32 clinical trials, explains the high reporting of clinical trial registration numbers.
This discrepancy in the reporting of individual items makes the case for authors and journals deeming particular items more important than others to include in abstracts. While it is undeniable that some items carry more weight than others33 the fact that the 3 most important items in the methodological domain (method of random sequence generation, allocation concealment, and details of blinding) are being overlooked is particularly concerning, as these are perhaps the most important aspects of RCTs that ensure the authenticity of the results.4,26–29 This is a shortcoming on the part of authors and journals and needs to be addressed as this affects the reliability and validity of almost all RCTs.30,34,35 However, the almost nonexistent reporting of funding sources in the abstract and the overwhelming reporting of funding sources within full-texts of RCTs36 perhaps makes the case for updating this particular aspect of the CONSORT guidelines as it is redundant. It also provides less use to practicing clinicians whose main concern is the result of the RCT. Researchers looking to analyze funding sources and conflicts of interest reporting, and their implications are more likely to read full-texts than rely solely on abstracts. Furthermore, the very low compliance rate of <1% observed in the reporting of methods of randomization is a failure on the part of journals and authors as studies37,38 have shown that improper reporting of this item tends to exaggerate the magnitude of effect sizes.
We also found that almost 40% of RCTs did not report primary outcome results for each group. This observation is consistent with those of previous studies29,39–42 which found a similar or larger proportion of RCTs that failed to report primary outcome results for specific groups. This is important as the results of the primary outcome of an RCT are arguably the most significant pieces of information for clinicians43 in deciding the overall effect of an RCT44 and whether the findings hold clinical value.45 Such low reporting can be explained by a few reasons. First, our study required an abstract to report raw numbers, P values, and effect sizes for each group as suggested by previous studies.3,4 If any item was omitted, the abstract was scored negatively. Second, outcome reporting bias has become common across the medical literature,46 and there is empirical evidence to suggest that statistically significant findings have a higher chance of being selectively reported.47,48
Regions of publication differed between journals with the EHJ having the highest number of RCT abstracts from Europe, Circulation from North America, and JACC from Asia. These differences in regional origins may account for some of the differences between adherence for individual items, representing a difference in the culture of writing and reporting studies,49,50 which further reinforces the need for standardizing RCT abstract reporting. Moreover, all the 3 journals have strict abstract word count limit. JACC and the EHJ allow 250 words, whereas Circulation allows 350 words. With low word count limits, it may be impossible for the authors to adhere to all the reporting guidelines.
An overwhelming number of journals including top-tier journals such as The Lancet and the Journal of the American Medical Association have endorsed CONSORT for Abstracts,18 which shows that journals recognize the importance of having a standardized method of reporting RCTs, even though it remains an evolving guideline.12,51 It also shows that journals understand the significance of clearly reporting each item so that health care providers can definitively appraise the applicability and implications of an RCTs results.52 Failure in reporting important items adequately can lead to exaggeration of effect sizes51 and a lack of proper understanding of adverse effects.53
However, nonadherence to these guidelines is still prevalent and reporting of clinically significant items, such as the primary outcome result, is poor. This supports the conclusion that there is a need for more aggressive enforcement of CONSORT for Abstracts by journals by strengthening or altering the peer-review process, and a need for authors to realize the full potential and importance of adhering to these guidelines to improve the overall quality of RCT abstract reporting. Potential ways in which we might be able to improve adherence is allowing greater word limit (perhaps 300–350 words), as allotted by Circulation, and hardwiring abstract reporting elements hence forcing authors to conform to reporting mandatory elements in RCT abstracts, such as the reporting of primary outcome results.
This study has limitations. First, we appointed equal weighting to each checklist item, which some experts disagree with but have not defined.35 Second, we assessed general cardiology journals and did not focus on subspecialized journals. The results of our study may not be applicable to journals focusing on other specialties or subspecialties. Our study also did not take into account any existing trends within journals; hence, it could be unfair to compare improvement trends between journals if these rates are different.25 Our study, however, has numerous strengths. We included RCTs published within a broad time span (2011–2017), which means these journals had ample time and opportunity to recommend and enforce implementation of CONSORT for Abstracts since its publication in 2008.13 Our methodology and data extraction are reproducible, as we used publicly available data, followed the checklist for items by selecting yes or no for individual items and achieved an agreement on items beforehand. Finally, our inter-rater agreement was high, demonstrated by the κ value of 0.894.
Figure. Flow diagram of the study.
EHJ indicates European Heart Journal; JACC, Journal of the American College of Cardiology; and RCT, randomized controlled trial.
WHAT IS KNOWN
This is the first study to assess the adherence of abstracts to CONSORT (Consolidated Standards of Reporting Trials) in cardiovascular medicine.
Adherence to the CONSORT in the abstracts of the 3 top-tier cardiovascular journals is suboptimal.
The reporting quality of primary outcome results, method of randomization, and allocation concealment in abstracts was particularly low.
WHAT THE STUDY ADDS
Nonadherence to CONSORT in abstracts of randomized controlled trials is likely to hinder health care providers in adequately appraising trials applicability and implications.
Inadequate reporting of randomized controlled trial abstracts may lead to incorrect conclusions as abstracts may be the only part of a publication that clinicians might read.
The improper application of trial results can affect patient outcomes.
Acknowledgments
We take responsibility for all aspects of the reliability and freedom from bias of the data presented and their discussed interpretation.
Footnotes
Disclosures
None.
REFERENCES
- 1.Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887–1892. doi: 10.1056/NEJM200006223422507 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Yusuf S Randomised controlled trials in cardiovascular medicine: past achievements, future challenges. BMJ. 1999;319:564–568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Agha R, Cooper D, Muir G. The reporting quality of randomised controlled trials in surgery: a systematic review. Int J Surg. 2007;5:413–422. doi: 10.1016/j.ijsu.2007.06.002 [DOI] [PubMed] [Google Scholar]
- 4.Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials. 2012;13:77. doi: 10.1186/1745-6215-13-77 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–639. [DOI] [PubMed] [Google Scholar]
- 6.Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365:1159–1162. doi: 10.1016/S0140-6736(05)71879-1 [DOI] [PubMed] [Google Scholar]
- 7.Altwairgi AK, Booth CM, Hopman WM, Baetz TD. Discordance between conclusions stated in the abstract and conclusions in the article: analysis of published randomized controlled trials of systemic therapy in lung cancer. J Clin Oncol. 2012;30:3552–3557. doi: 10.1200/JCO.2012.41.8319 [DOI] [PubMed] [Google Scholar]
- 8.Hopewell S, Eisinga A, Clarke M. Better reporting of randomized trials in biomedical journal and conference abstracts. J Info Sci. 2007;34:162–173. doi: 10.1177/0165551507080415 [DOI] [Google Scholar]
- 9.Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, Koepsell TD. Journal reading habits of internists. J Gen Intern Med. 2000;15: 881–884. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moher D, Schulz KF, Altman DG; CONSORT GROUP (Consolidated Standards of Reporting Trials). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–662. [DOI] [PubMed] [Google Scholar]
- 11.Kane RL, Wang J, Garrard J. Reporting in randomized clinical trials improved after adoption of the CONSORT statement. J Clin Epidemiol. 2007;60:241–249. doi: 10.1016/j.jclinepi.2006.06.016 [DOI] [PubMed] [Google Scholar]
- 12.Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG; Consolidated Standards of Reporting Trials Group. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63:e1–37. doi: 10.1016/j.jclinepi.2010.03.004 [DOI] [PubMed] [Google Scholar]
- 13.Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5:e20. doi: 10.1371/journal.pmed.0050020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW. Standards of reporting of randomized controlled trials in general surgery: can we do better? Ann Surg. 2006;244:663–667. doi: 10.1097/01.sla.0000217640.11224.05 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tiruvoipati R, Balasubramanian SP, Atturu G, Peek GJ, Elbourne D. Improving the quality of reporting randomized controlled trials in cardiothoracic surgery: the way forward. J Thorac Cardiovasc Surg. 2006;132:233–240. doi: 10.1016/j.jtcvs.2005.10.056 [DOI] [PubMed] [Google Scholar]
- 16.Peters JPM, Hooft L, Grolman W, Stegeman I. Assessment of the quality of reporting of randomised controlled trials in otorhinolaryngologic literature – adherence to the CONSORT statement. PLoS One. 2015;10:e0122328. doi: 10.1371/journal.pone.0122328 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Xu L, Li J, Zhang M, Ai C, Wang L. Chinese authors do need CONSORT: reporting quality assessment for five leading Chinese medical journals. Contemp Clin Trials. 2008; 29:727–731. doi: 10.1016/j.cct.2008.05.003 [DOI] [PubMed] [Google Scholar]
- 18.Endorsers: Journals and Organizations. CONSORT: Transparent Reporting of Trials, The CONSORT Group, The Ottawa Hospital Research Institute. http://www.consort-statement.org/about-consort/endorsers. Accessed July 20, 2018.
- 19.American Heart Association. Circulation. https://www.ahajournals.org/circ/manuscript-preparation. Accessed July 29, 2018.
- 20.Oxford Academic. European Heart Journal. https://academic.oup.com/eurheartj/pages/General_Instructions#2. Accessed July 29, 2018.
- 21.Elsevier. Journal of the American College of Cardiology. https://www.elsevier.com/journals/jacc-journal-of-the-american-college-of-cardiology/0735-1097/guide-for-authors. Accessed July 29, 2018.
- 22.Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited. Ann Intern Med. 1990;113:69–76. doi: 10.7326/0003-4819-113-1-69 [DOI] [PubMed] [Google Scholar]
- 23.McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22:276–282. [PMC free article] [PubMed] [Google Scholar]
- 24.Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22:139–142. doi: 10.1136/ebmed-2017-110713 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hays M, Andrews M, Wilson R, Callender D, O’Malley PG, Douglas K. Reporting quality of randomised controlled trial abstracts among high-impact general medical journals: a review and analysis. BMJ Open. 2016;6:e011082. doi: 10.1136/bmjopen-2016-011082 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wang L, Li Y, Li J, Zhang M, Xu L, Yuan W, Wang G, Hopewell S. Quality of reporting of trial abstracts needs to be improved: using the CONSORT for abstracts to assess the four leading Chinese medical journals of traditional Chinese medicine. Trials. 2010;11:75. doi: 10.1186/1745-6215-11-75 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Cui Q, Tian J, Song X, Yang K. Does the CONSORT checklist for abstracts improve the quality of reports of randomized controlled trials on clinical pathways? J Eval Clin Pract. 2014;20:827–833. doi: 10.1111/jep.12200 [DOI] [PubMed] [Google Scholar]
- 28.Mann E, Meyer G. Reporting quality of conference abstracts on randomised controlled trials in gerontology and geriatrics: a cross-sectional investigation. Z Evid Fortbild Qual Gesundhwes. 2011;105:459–462. doi: 10.1016/j.zefq.2010.07.011. [DOI] [PubMed] [Google Scholar]
- 29.Berwanger O, Ribeiro RA, Finkelsztejn A, Watanabe M, Suzumura EA, Duncan BB, Devereaux PJ, Cook D. The quality of reporting of trial abstracts is suboptimal: survey of major general medical journals. J Clin Epidemiol. 2009;62:387–392. doi: 10.1016/j.jclinepi.2008.05.013 [DOI] [PubMed] [Google Scholar]
- 30.Hopewell S, Clarke M, Askie L. Reporting of trials presented in conference abstracts needs to be improved. J Clin Epidemiol. 2006;59:681–684. doi: 10.1016/j.jclinepi.2005.09.016 [DOI] [PubMed] [Google Scholar]
- 31.Gülmezoglu AM, Pang T, Horton R, Dickersin K. WHO facilitates international collaboration in setting standards for clinical trial registration. Lancet. 2005;365:1829–1831. doi: 10.1016/S0140-6736(05)66589-0 [DOI] [PubMed] [Google Scholar]
- 32.De Angelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden MB; International Committee of Medical Journal Editors. Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. Lancet. 2005;365:1827–1829. doi: 10.1016/S0140-6736(05)66588-9 [DOI] [PubMed] [Google Scholar]
- 33.Sarveravan P, Astaneh B, Shokrpour N. Adherence to the CONSORT Statement in the reporting of randomized controlled trials on pharmacological interventions published in iranian medical journals. Iran J Med Sci. 2017;42:532–543. [PMC free article] [PubMed] [Google Scholar]
- 34.Nelson H Systematic Reviews to Answer Health Care Questions. Philadelphia, PA: Lippincott, Williams & Wilkins; 2014. [Google Scholar]
- 35.Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178. doi: 10.1136/bmj.e4178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hakoum MB, Jouni N, Abou-Jaoude EA, Hasbani DJ, Abou-Jaoude EA, Lopes LC, Khaldieh M, Hammoud MZ, Al-Gibbawi M, Anouti S, Guyatt G, Akl EA. Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance. BMJ Open. 2017;7:e015997. doi: 10.1136/bmjopen-2017-015997 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135:982–989. [DOI] [PubMed] [Google Scholar]
- 38.Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:1–76. [PubMed] [Google Scholar]
- 39.Song SY, Kim B, Kim I, Kim S, Kwon M, Han C, Kim E. Assessing reporting quality of randomized controlled trial abstracts in psychiatry: adherence to CONSORT for abstracts: a systematic review. PLoS One. 2017;12:e0187807. doi: 10.1371/journal.pone.0187807 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Khanpour Ardestani S, Karkhaneh M, Yu HC, Hydrie MZI, Vohra S. Primary outcomes reporting in trials of paediatric type 1 diabetes mellitus: a systematic review. BMJ Open. 2017;7:e014610. doi: 10.1136/bmjopen-2016-014610 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Huang YQ, Traore K, Ibrahim B, Sewitch MJ, Nguyen LHP. Reporting quality of randomized controlled trials in otolaryngology: review of adherence to the CONSORT statement. J Otolaryngol Head Neck Surg. 2018;47:34. doi: 10.1186/s40463-018-0277-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Kuriyama A, Takahashi N, Nakayama T. Reporting of critical care trial abstracts: a comparison before and after the announcement of CONSORT guideline for abstracts. Trials. 2017;18:32. doi: 10.1186/s13063-017-1786-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Andrade C The primary outcome measure and its importance in clinical trials. J Clin Psychiatry. 2015;76:e1320–e1323. doi: 10.4088/JCP.15f10377 [DOI] [PubMed] [Google Scholar]
- 44.Choudhary D, Garg PK. Primary outcome in a randomized controlled trial: a critical issue. Saudi J Gastroenterol. 2011;17:369. doi: 10.4103/1319-3767.84504 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Man-Son-Hing M, Laupacis A, O’Rourke K, Molnar FJ, Mahon J, Chan KB, Wells G. Determination of the clinical importance of study results. J Gen Intern Med. 2002;17:469–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Lancee M, Lemmens CMC, Kahn RS, Vinkers CH, Luykx JJ. Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs. Transl Psychiatry. 2017;7:e1232. doi: 10.1038/tp.2017.203 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e3081. doi: 10.1371/journal.pone.0003081 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Dwan K, Gamble C, Williamson PR, Kirkham JJ; Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8:e66844. doi: 10.1371/journal.pone.0066844 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Mauranen A Cultural differences in academic discourse-problems of a linguistic and cultural minority. AFinLAn Vuosikirja. 1993;51:157–174. [Google Scholar]
- 50.Cultural Yakhontova T. and disciplinary variation in academic discourse: the issue of influencing factors. Journal of English for Academic Purposes. 2006;5:153–167. doi: 10.1016/j.jeap.2006.03.002 [DOI] [Google Scholar]
- 51.Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Trials. 2010;11:32. doi: 10.1186/1745-6215-11-32 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Borg Debono V, Zhang S, Ye C, Paul J, Arya A, Hurlburt L, Murthy Y, Thabane L. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol. 2012;12:13. doi: 10.1186/1471-2253-12-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Hodkinson A, Kirkham JJ, Tudur-Smith C, Gamble C. Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open. 2013;3:e003436. doi: 10.1136/bmjopen-2013-003436 [DOI] [PMC free article] [PubMed] [Google Scholar]