Table 2.
Description of the studies’ findings
| Guideline | First author | Measure of quality | Results mean (SD)/% count | Authors’ conclusion |
|---|---|---|---|---|
| CONSORT | Al-Namankany54 | Modified 34-item CONSORT checklist | Compliance varied across items and articles. Good compliance of articles to CONSORT for introduction sections (96%–98%), discussion sections (96%–98%). Poor reporting in randomization methods (5%–9%), description of sample size calculation (4%), intention-to-treat analysis (1%) | Quality of reporting of RCTs in pediatric dental journals was generally poor, with negligible improvement after the publication of CONSORT Statement |
| Areia21 | Application of CONSORT/STARD | 15.7 (2.2) | Level of adherence is medium for quality of reporting in diagnostic endoscopy | |
| Augestad44 | CONSORT adherence, Jadad | 30.75 (4), 40% of the trials had a Jadad score of ≥3 points | Level of adherence is low for quality of reporting for RCTs of disease specific clinical decision support | |
| Balasubramanian45 | Modified CONSORT score, allocation concealment as assessed by Schulz et al, Jadad score | Medians of the modified CONSORT score were 85.45 (interquartile range 81.09–86.13) and interquartile range 68.97 (62.89–73.1 1) for RCTs from medical and surgical journals, respectively; 13% clearly explained allocation concealment; 37.7% of RCTs had a Jadad score of ≥ 3 | Quality of reporting of surgical RCTs was suboptimal, and reporting in surgical journals was inferior to surgical trials in medical journals “We found that the quality of reporting of general surgical RCTs leaves considerable room for improvement” |
|
| Bath55 | In all, 33 criteria of the CONSORT Statement and 53 additional factors relevant to acute stroke or trials in general. Trial quality was also assessed with a seven-point scale | Median total report quality was 40/86 (range 15–61). Median CONSORT criterion was 19/33 (9–29) | Poor quality for acute stroke RCTs “We believe that authors should follow the CONSORT guidelines and that referees and editors should ensure this happens” |
|
| Bian43 | 63-item revised CONSORT checklist designed for Chinese Herbal Medicine clinical studies | Median score of overall reporting quality was 32% (8%) | Overall quality of reporting of CHM RCTs was poor. Need to improve reporting in clinical trials in this area. “To improve the quality of reporting of RCTs of CHM, we recommend adopting a revised CONSORT checklist that includes items specific to CHM. We also recommend that editors of CHM journals require authors to use a structured approach to presenting their trials as a condition of publication” | |
| Bousquet57 | Reporting of procedure, randomization, dropouts, strict conduct of intention-to-treat analysis, sample size calculation, which was assessed with eight items of the CONSORT Statement | Four of the 94 studies met the eight items of the CONSORT Statement criteria | RCTs in subcutaneous immunotherapy and sublingual immunotherapy had poor reporting quality. Encourage more use of CONSORT Statement | |
| Capili58 | Adherence to CONSORT/presence of harms guidelines in RCTs | I7(range: 14–21) for CONSORT; 7/22 = 77.3% for harm | Level of adherence is bad for quality of reporting, for RCTs on acupuncture, for pain reduction | |
| Cavadas59 | A 25-item 2010 revised CONSORT Statement | No combined data: only a few items were reported in less than 50% of the studies; some items were reported in more than 90% of the studies | “RCTs in [pelvic organ prolapsed] are scarce. The quality of reporting is suboptimal in many aspects and has not improved in recent years” | |
| Lu73 | Percentage of articles that reported each applicable item of the CONSORT checklist | Sample size: only one (2.2%) of the papers mentioned sample size calculation. Randomization: 12 studies (26.1 %) were deemed to have authentic randomization. Blinding: 36 papers (78.3%) provided no information about blinding of either participants or investigators. Reporting of baseline characteristics: 39 papers (84.8%) reported the details of the baseline characteristics of participants. Length of follow-up: 22 papers (47.8%). There was no information provided on the length of time for which participants were followed. Loss-to-follow-up: a total of 36 studies (78.3%) failed to report dropout rates. Statistical reporting: only one paper (2.2%) did not report what statistical methods they had used | Findings indicate that the reporting quality of RCTs needs improvement for RCTs on the treatment of cancer pain in People’s Republic of China | |
| Chowers60 | CONSORT guidelines for adverse events, adjusted to the design of the HAART trial | No combined score: harms were reported in only 24% of trials, 1/49 reported on adverse events collection method | Large variability and a lack of standard reporting of adverse events between trials; many trials did not adhere to CONSORT recommendations | |
| de Vries62 | Adequate reporting of adverse drug reactions | Mean of 3, and 18% of articles scored 6 or higher | Insufficient reporting quality in adverse event reporting in RCTs of children | |
| Ethgen63 | CLEAR NPT – a checklist to evaluate RCTs of nonpharmacological treatments | Most studies failed to report 8/12 quality indexes in the checklist. Reporting of generation of allocation sequence was adequate in 38.8% of studies, treatment allocation in 26.3%, intention-to-treat analysis in 70.0% | Inadequate reporting amongst trials involving stents. “The current reporting of results of RCTs testing stents needs to be improved to allow readers to appraise the risk of bias and the applicability of the results” | |
| Eyawo64 | Revised CONSORT checklist to assess reporting of each items on the checklist in counts (percentage) | 14/16 items ranged from 2%–7%; the other two items, sample size determination, and reporting of masking were reported in 72% and 75% of the articles | Deficiencies in the design, planning, and reporting of noninferiority and equivalence trials in ophthalmology literature | |
| Farrokhyar64 | Modified CONSORT Statement and added factor relevant to surgical trials and CABG surgery | 51.7 out of 105 (11.5) | The total reporting quality of trials in this review varied substantially between publications (35–96 out of a possible max score of 105). The results showed that there is a need for improvement in quality of reporting | |
| Froud48 | Number and percentage of studies satisfying the revised 1l-item consort checklist | Most items were reported in an adequate percentage of studies; 5/11 reported in 78%–100% of the studies | Their results suggest that cluster randomized trials in oral health are of reasonable quality with respect to the key criteria of accounting for clustering in the design and analysis | |
| Fung49 (CONSORT and STROBE) Gagnier66 | Presence or absence of CONSORT (maximum score of 37 points) statement indicators Mean CONSORT score based on 42 items and the percentage of items reported | CONSORT: median and mean values of 89% and 83%, respectively 18.92 out of 42 (5.54), and 45% of items were reported across all trials |
Overall level of reporting is acceptable and has improved since the creation of CONSORT and STROBE “We found that reports of RCTs of herbal medicine interventions reported less than half of the necessary information in their published results” Overall adherence is low |
|
| Halpern67 | Percentage of articles that reported each applicable item of the modified CONSORT checklist and count of articles complying with modified CONSORT items | In the 23 articles in Anesthesia and Analgesia, the median percentage of correct CONSORT items was 63% | Poor – total number of items that are inadequately reported is high in the current RCT literature with obstetric anesthesia | |
| Herdan69 | 22-item CONSORT checklist expressed | On average 87.4% of the CONSORT items were reported | The reporting quality has improved significantly in the period after dissemination of the CONSORT Statement; however, reporting of adverse events needs attention | |
| Kiehna46 | Quality of reporting score using CONSORT (max score of 44) and Jadad score out of 5 points | 26.4 out of 44 (range: 17–38)/67% of studies had no description or the prestudy sample size calculation, 63% did not describe whether subjects, treatment providers or assessors/analysts were blinded | The quality of reporting of RCTs in neurosurgical journals remains suboptimal | |
| Kober71 | CONSORT criteria based on a 14-item questionnaire | 75% of studies reported only six of the 13 items; only 14% reported randomization process; only 13% provided details about concealment of allocation; only 13% provided a statement on study power; only 12% used intention-to-treat analysis |
Articles of Hodgkin’s lymphoma published after 1996 do not conform to the CONSORT recommendations | |
| Ladd25 | Assessment of 36 of the items from the CONSORT Statement based on a score out of 36 | 24.43 out of 36 (3.27) | The overall level of adherence to CONSORT has improved since 1994, and continues to remain highest among studies that have been published within journals that have adopted the CONSORT guidelines | |
| Li72 | Score out of 40 based on a 40-item modified checklist based on the CONSORT Statement | 42% of the studies included explained how sample size was determined; 14% of studies described whether or not outcome assessors were blinded | The reporting quality of these trials is suboptimal and substantial improvement is required to meet the CONSORT guidelines. Almost 50% of the trials we reviewed did not satisfy more than half of the criteria in the modified CONSORT checklist, and only 23% of RCTs provided adequate details of Tai Chi intervention used in the trials | |
| Marshman75 | 56 criteria based on the CONSORT Statement | 27/56, with variation between journals (23.2 to 27.7) | Poor adherence to the CONSORT checklist in RCTs in dental health | |
| Moberg-Mogren76 | Average NMECI score (0–201 subitems scale) | 104.2(32.9) | Less than half of the articles met criteria of these subitems in selected RCTs relevant to occupational therapy | |
| Moher47 | CONSORT checklist, frequency of unclear allocation concealment, and a five-point quality assessment instrument (Jadad) | 12.7/32 of the CONSORT checklist included; 81.3% unclear allocation concealment; 1.9/5 for the Jadad assessment scale | Overall, there was no difference in the PedCAM RCTs and conventional medicine quality, with both types achieving 43% of their maximum possible outcome | |
| Montané77 | Revised CONSORT checklist, 22 items | 10.5(2.7) | Quality was good in 23 (25%) of the articles and poor in 69 (75%) of the reports for RCTs on the efficacy of analgesic drugs in postoperative pain after TOS | |
| Montgomery50 | Qualitative look | N/A | Varying level of reporting quality factorial trials of complex interventions in community settings | |
| Norton-Mabus78 | NMNECI (2l2subitems) | 119.5(25.48) | Article consistency with CONSORT Statement was less than 60%. Occupational therapy RCT had higher consistency with the instrument, scoring higher than articles in speech therapy | |
| Parsons79 (combined CONSORT and STROBE guideline) | Overall compliance calculated as the weighted mean of the compliance rates for the seven selected journals, using a previously made questionnaire | 59% (CONSORT) | Very few papers fulfilling all criteria; general lack of statistical rigor | |
| Piggott80 | Compared RCTs of three different time period cohorts, with the CONSORT (condensed, 13-item) checklist | Quality of reporting variable; 30% of trials or less used true randomization, allocation concealment, intention-to-treat analysis, and power calculations | Quality of reporting over time cohorts was variable, no consistent improvement over time. Quality of reporting remains poor for RCTs in specialized palliative care literature | |
| Plint51 | 22-item checklist from the CONSORT Statement | Standardized mean difference between CONSORT-adopting journals and nonadopters was 0.83 (95% CI, 0.46–1.19) | Journal adoption of CONSORT is associated with improved reporting of RCTs | |
| Rios81 | Overall quality score, which is a 15 point overall reporting quality score made from CONSORT checklist | 10(2.03) | Suboptimal reporting quality in an endocrine journal | |
| Strech83 | A checklist based on the CONSORT Statement | There are 72 items on the checklist; 42% were reported adequately and 25% were reported inadequately | While some trial-related information is well reported, a good part of the reporting quality of RCTs in bipolar disorder falls well below the required and practically feasible level for many aspects essential for the adequate interpretation of methodological quality and clinical relevance. Authors should be further encouraged to follow the CONSORT criteria. No consistent trend could be shown for improvement in the quality of reporting over time, or for reporting essential methodological items differently. There is a consistent trend toward better reporting in journals that endorse the URM | |
| Thabane84 | Percentage of studies satisfying each of the 44 CONSORT criteria | 26.25 (4.51 ) and 60% adherence for reporting criteria: 90% satisfied criteria for the introduction; 19% for the methods; 75% the study protocol, 70% for the results | Overall, the quality of reporting is suboptimal in RCTs of weight loss intervention. Key reporting criteria that may impact the validity and generalizability of the results were adequately reported | |
| Walleser86 | CONSORT-CRT | 34% inadequately reporting on more than half of the CONSORT-CRT criteria | The quality of reporting in CRTs needs improvement. This will hopefully improve implementation and planning | |
| Wangge52 | Extension of the CONSORT Statement for noninferiority and equivalence trial | No blinding in 34.0%, noninferiority margin in 97.8%, with only 45.7% reporting the method of determining the margin | Adherence improved slightly after CONSORT for noninferiority trials | |
| Zhong89 | Number of studies describing each of the 38 modified consort items | Of the 38 CONSORT items, only five items were described in more than 80% of the 153 included | Adherence was suboptimal for two-group parallel randomized controlled clinical trials of multiherb formulae | |
| Zintzaras53 | 17-item CONSORT checklist | 17 CONSORT checklist items were reported in 7/18 studies, and 9/17 CONSORT checklist items were reported in all 18/18 studies | Proper assessment of the credibility and generalizability of the results can be ensured by reporting quality | |
| Ziogas50 | 24-item questionnaire based on the CONSORT checklist | 75% of the studies addressed 13 out of the 24 items of the CONSORT Statement | Reporting on myeloid malignancies remains unsatisfactory and requires further improvement to properly assess the validity of clinical research | |
| PRISMA | Ma74 | Adherence to PRISMA checklist items (27 items) | Title, introduction, limitations, and conclusions were reported well in 90% or more of the studies. Most other items varied from 30%–70% of the studies | Compliance with PRISMA reporting guidelines is low for systematic reviews on TCM published in Chinese journals |
| Weir87 (combined PRISMA and QUOROM guideline) | An integrated score consisting of the number of items completed over the total numbers of items on both the PRISMA and QUOROM criteria, resulting in cored ranking from 0% to 100% (excluding the items focused on in the abstract) | Mean = 63% (range 45%–81%) on a scale of 0%–100% | Systematic reviews of empirical computerized provider order-entry research had moderate quality | |
| Willis88 | Adherence to the 27-item PRISMA checklist | Of the 236 meta-analyses included following selection: 1% reported the study protocol; 25% reported the searches used; 32% reported the results of a risk of bias assessment; and 35% reported the abstract as a structured summary | Compliance with the PRISMA statement was generally poor; none of the review completely adhered to all 27 checklist items for the published meta-analyses of diagnostic tests | |
| QUOROM | Bereza56 | 18-item QUOROM checklist, ten-item checklist OQAQ used for scientific quality | 61% ± 19% (median 60%, range 39%–94%) for the QUOROM checklist. 58% ± 28% for OQAQ | “Reporting/scientific quality was considered less than fair-to-good. Stakeholders should strive for higher scientific quality of meta-analyses” |
| Hemels68 | 18-item QUOROM checklist | On average 50.2% of the CONSORT items were reported | The overall quality of reporting in the meta-analysis of RCTs in major depressive disorder was marginally acceptable | |
| Junhua70 | 18-item QUOROM checklist, ten-item checklist OQAQ used for scientific quality | No combined score; methodological and reporting flaws in more than half of the review articles. Flaws were mainly in the literature search, characteristics of included and excluded studies, quality assessment of primary trials, and data merging | Methodology and reporting quality are poor in both systematic reviews and meta-analysis of TCM published in journals in the People’s Republic of China | |
| Shea82 | 18-item QUOROM checklist, ten-item checklist OQAQ used for scientific quality | All systematic reviews were found to have good overall quality. OQAQ mean score was 5.02 (95% CI 3.71–6.32) | Reporting quality of Cochrane musculoskeletal systematic reviews was generally good, with room for improvement | |
| Vigna-Taglianti85 | QUOROM-based checklist (score out of 50) | 29.9/50 | “Oncologists should be aware that they could be relying on poor underlying documents. Writing groups should be aware of methodological problems, and should consult the existing manuals for the preparation of guidelines” | |
| Weir87 (combined PRISMA and QUOROM guideline) | An integrated score consisting of the number of items completed over the total numbers of items on both the PRISMA and QUOROM criteria, resulting in cored ranking from 0% to 100% (excluding the items focused on in the abstract) | 63% (range 45%–81%) on a scale of 0%–100% | Systematic reviews of empirical computerized provider order-entry research were of moderate quality | |
| STROBE | Cook61 | Quality of reporting, methodological quality, and the association between methodological quality and effect size | 253 (90) | Reporting the quality of experimental research on health profession education was found to be generally suboptimal |
| Fung45 (combined CONSORT and STROBE guideline) | STROBE (maximum score of 37 points) statement indicators | STROBE mean and median: 70% and 71%, respectively | Overall level of reporting is acceptable and has improved since the creation of CONSORT and STROBE | |
| Parsons79 (combined CONSORT and STROBE guideline) | Weighted mean of the compliance rates for the seven selected journals, using a previously made questionnaire | 58% (strobe) | Very few papers fulfilling all criteria, general lack of statistical rigor |
Abbreviations: SD, standard deviation; CONSORT, Consolidated Standards of Reporting Trials; RCT, randomized controlled trial; CHM, Chinese herbal medicine; HAART, highly active antiretroviral treatment; CLEAR NPT, checklist to evaluate a report of a nonpharmacological trial; CABG, coronary artery bypass surgery; STROBE, Strengthening the Reporting of Observational Studies in Epidemiology; NMECI, Nelson-Moberg Expanded CONSORT Consolidated Standards of Reporting Trials Instrument; PedCAM, Pediatric Complementary and Alternative Medicine Research and Education Network; TOS, thoracic outlet syndrome; N/A, not applicable; NMENCI, Nelson-Moberg, Norton Expanded Consolidated Standards of Reporting Trials Instrument; CI, confidence interval; URM, uniform requirement for manuscript; CRT, consolidated standard of reporting trial; NI, noninferiority; PRISMA, preferred reporting items for systematic reviews and meta-analyses; QUOROM, Quality of Reporting of Meta-analysis; OQAQ, Overview Quality Assessment Questionnaire; TCM, traditional Chinese medicine.