Abstract
Background
A considerable amount of randomized controlled trials (RCTs) have been published on statins and/or fibrates for diabetic retinopathy, a clinical condition associated with high social and economic burden. Adherence to the CONSORT statement items is imperative to ensure transparency and reproducibility in clinical research. The aim of this study is to assess the reporting quality and the adherence to CONSORT of RCTs assessing statins and/or fibrates for diabetic retinopathy.
Methods
We conducted a critical appraisal study at Discipline of Evidence-based Medicine, Escola Paulista de Medicina, Universidade Federal de São Paulo (Unifesp). A sensitive literature search was performed to identify all relevant RCTs, with no time or language limits. Two authors independently evaluated the reporting quality of the selected RCTs using the CONSORT statement as a standard.
Results
Thirteen reports of RCTs were included in this study. The adherence of the reports to CONSORT items ranged from 24% to 68%. The median score was 11 (interquartile range (IQR) 8 to 13). When analyzed separately, the methods sections of the reports had a median of three items (IQR 2 to 4) judged adherent to the methods items of CONSORT (items 3 to 12). The most underreported items were those related to trial design, title and abstract, allocation concealment, implementation of the randomization sequence, and blinding. Other important items, such as the one related to the description of the inclusion criteria, also had low adherence.
Conclusions
The overall adherence to the CONSORT checklist items was poor, especially in the items related to the methods section. RCT reports on statins and/or fibrates for diabetic retinopathy must be optimized to avoid reporting biases and to improve transparency and reproducibility.
Keywords: Clinical trials as topic, Publication bias, Hydroxymethylglutaryl-CoA reductase inhibitors, Hypolipidemic agents, Diabetic retinopathy
Background
When assessing the effects of an intervention for a specific clinical condition, randomized controlled trials (RCTs) are considered the preferable source of evidence to support its use [1]. Well-designed, well-conducted, and well-reported RCTs provide the most unbiased data for reducing the uncertainties around effects of an outcome of interest and for improving the reliability of findings [1].
A considerable amount of RCTs have been published on statins and/or fibrates for diabetic retinopathy, a clinical condition associated with high social and economic burden [2]. To ensure their transparency and reproducibility, the International Committee of Medical Journal Editors (ICMJE) recommends the use of reporting guidelines that aim to improve the quality of the reports from studies on healthcare [3, 4].
The CONSORT (CONsolidated Standards of Reporting Trials) [4] statement is used worldwide as a reporting guideline focused on RCTs. The statement was published in 1996 [5], updated in 2010, and consists of a checklist of 25 items that guide the reporting of essential items of a RCT [4]. The CONSORT checklist is divided into six sections: title and abstract (one item), introduction (one item), methods (ten items), results (seven items), discussion (three items), and other information (three items).
Assessing published trials for their completeness—i.e., the adherence to CONSORT checklists—is important for directing further publication policies and for minimizing the risk of selective and/or publication bias [6]. To our knowledge, there has been no such quality evaluation of RCTs on statins and/or fibrates for diabetic retinopathy.
The objective of this study was to evaluate the quality of reporting of RCTs about statin and/or fibrates for diabetic retinopathy by assessing its adherence to the CONSORT checklist.
Methods
Design and setting
This is a critical appraisal study performed in the Discipline of Evidence-based Medicine of Escola Paulista de Medicina, Universidade Federal de São Paulo (EPM-Unifesp).
Criteria for including studies
Types of participants
We searched for studies including type 1 or 2 diabetic patients, regardless of age and sex. Participants with or without diabetic retinopathy were considered, depending on the therapeutic or preventive purpose of the related RCT. We considered the diagnosis of diabetic retinopathy by any criteria previously validated.
Types of studies
We included RCTs with parallel, cross-over or cluster designs. We only included complete and published studies. Protocols were not included.
Types of interventions
We considered statin and/or fibrate, at any dosage, scheme, duration, and route of delivery. As comparators we considered other interventions, no intervention, or placebo.
Search for studies
We performed a sensitive search strategy without language, date, or publication status restrictions, using relevant descriptors and indexed terms in all databases. An electronic search was performed on January 17, 2018 in the following databases: Medical Literature Analysis and Retrieval System (MEDLINE, via PubMed), Embase (via Elsevier), Latin American and Caribbean Center on Health Sciences Information (LILACS, via Virtual Health Library), and Cochrane Central Register of Controlled Trials (CENTRAL, via Wiley). The full search strategies for each database are presented in Additional file 1.
We conducted additional searches in the clinical trial registries ClinicalTrials.gov and International Clinical Trials Registry Platform—World Health Organization (ICTRP-WHO), and in the grey literature source OpenGrey. We also performed a manual search from reference lists of all included studies and review articles for additional studies. We contacted field specialists about unpublished or ongoing studies that could fulfill our inclusion criteria.
Study selection
The selection of studies was performed by two authors (VM and RP) independently. The first step of the selection was the reading of titles and abstracts. All potentially relevant studies were taken to a second step that consisted of a full text reading. All studies that fulfilled our inclusion criteria were included for critical appraisal. A third author (RR) was consulted if disagreement occurred in any step of the selection process. Selection was performed using Rayyan software [7].
Data extraction and results presentation
Two authors (VM, RR, or LL) independently extracted data on results of the included studies and transfered data to an a priori developed sheet.
Assessment of reporting quality
Two authors (TG, LL, or VM) independently assessed the quality of the included studies. A third author (RR or RLP) was consulted in case of disagreement.
For assessing the reporting quality, we used the CONsolidated Standards Of Reporting Trials (CONSORT) statement, a validated tool comprising a checklist of 25 items [8]. We confirmed the adherence to the 25 items and scored each item as: 0 (no adherence) or 1 (full adherence). The final CONSORT-based score achieved by each RCT was determined as a percentage of the maximum possible score. After the exclusion of Not applicable items, if there were any, we present the CONSORT-based score in a 0 to 1 scale.
Results
Search results
Search strategies retrieved 1408 references. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram is depicted in Fig. 1. After the selection process, 13 reports of RCTs fulfilled the eligibility criteria and were included [9–21].
Fig. 1.
Flow diagram of the selection process
Reporting quality of RCTs
The adherence of RCTs to CONSORT items ranged from 24% to 68%. The median score was 11 (interquartile range (IQR) 8 to 13). When analyzed separately, the methods section items of CONSORT (items 3 to 12) had a median of three items judged adequate (IQR 2 to 4). The results section items (items 13 to 19) had a median of four items judged adequate (IQR 4 to 5) Fig. 2 presents the final score for each RCT report. Table 1 presents the number of times that each item was judged adequate. Additional file 2 presents the reporting quality of the included RCTs based on the 25 CONSORT items.
Fig. 2.
CONSORT-based score for each included randomized controlled trial
Table 1.
Number of times each CONSORT item was judged adequate among the 13 included randomized controlled trials
| CONSORT item | Number of times judged as adequate (percentage) | 95% Confidence interval |
|---|---|---|
| 1 | 1 (8%) | (0% to 35%) |
| 2 | 9 (69%) | (42% to 88%) |
| 3 | 0 (0%) | (0% to 27%) |
| 4 | 2 (15%) | (3% to 43%) |
| 5 | 9 (69%) | (42% to 88%) |
| 6 | 5 (38%) | (18% to 65%) |
| 7 | 2 (15%) | (3% to 43%) |
| 8 | 6 (46%) | (23% to 71%) |
| 9 | 1 (8%) | (0% to 35%) |
| 10 | 1 (8%) | (0% to 35%) |
| 11 | 1 (8%) | (0% to 35%) |
| 12 | 9 (69%) | (42% to 88%) |
| 13 | 2 (15%) | (3% to 43%) |
| 14 | 2 (15%) | (3% to 43%) |
| 15 | 9 (69%) | (42% to 88%) |
| 16 | 11 (84%) | (57% to 97%) |
| 17 | 11 (84%) | (57% to 97%) |
| 18 | 12 (92%) | (65% to 100%) |
| 19 | 8 (62%) | (35% to 82%) |
| 20 | 5 (38%) | (18% to 65%) |
| 21 | 11 (84%) | (57% to 97%) |
| 22 | 12 (92%) | (65% to 100%) |
| 23 | 4 (31%) | (12% to 58%) |
| 24 | 3 (23%) | (8% to 51%) |
| 25 | 9 (69%) | (42% to 88%) |
Item 3 (“trial design”) was judged adequately reported in none of the RCTs, mainly because they did not specify the allocation ratio. Items 1 (“title and abstract”), 9 (“allocation concealment”), 10 (“implementation” of the randomization sequence), and 11 (“blinding”) were reported adequately in only one RCT. Other important items such as 4 (“participants”), which refers to eligibility criteria and information regarding settings and locations from the RCTs, also had a very low adherence.
Discussion
This study included 13 reports of randomized clinical trials that assessed statins and/or fibrates for diabetic retinopathy. The adherence to the CONSORT items was poor, and thus the overall reporting quality was judged poor.
Reporting standards for RCTs have been broadly discussed since the first publication of the CONSORT statement and its proposed checklist with 25 items [22]. Inadequate reporting by a scientific study, especially involving experimentation in humans, is scientific misconduct and is associated with resource waste. Adequate reporting means transparency and improves reproducibility [23, 24].
The most underreported items in our study are related to the methods section. Overall, the methods sections were poorly reported and the results and discussion sections were better reported. The RCT with the highest score based on CONSORT adhered to 17 items (68% adherence).
Our findings are similar to other studies that have assessed the overall reporting quality and CONSORT statement checklist adherence in RCTs in a variety of fields [6, 25, 26]. As in other areas of health research, the reproducibility of clinical studies on diabetic retinopathy seems to be impaired by poor reporting. The low quality of reporting affects the synthesis of evidence in systematic reviews as well.
Our study has some limitations. The CONSORT checklist was proposed as a reporting guideline tool for writing, rather than a tool for the evaluation of already published reports. However, since there is no reliable validated tool to assess the reporting quality of RCTs, we believe that a score based on CONSORT is the most reliable option so far. We also planned to assess cluster trials using the original CONSORT items, but there is a CONSORT extension to cluster trials [27]. Nevertheless, the effect of this limitation on our results was probably null since no cluster trials were retrieved by the search strategy. Additionally, most of the included reports (8 out of 13) were published before the last CONSORT update (2010). However, this does not mean that previous RCTs should not adhere to reporting recommendations, because the checklist was already available in 1996 and reviewed in 2001. We did not expect that RCTs published before the updated CONSORT would adhere completely to CONSORT, but we expected that they would report everything that is important for study reproducibility and clinical practice application. Other studies and systematic reviews [6] have looked at the comparison between RCTs published before and after each version of CONSORT, which although important to confirm if the checklist indeed increased the reporting quality of the published studies, was not the objective of our study.
The results from our study should guide future research into diabetic retinopathy. The reporting of future studies must be optimized and their adherence to the CONSORT items is imperative to avoid reporting biases.
Conclusions
This study included 13 reports of RCTs that assessed statins and/or fibrates for diabetic retinopathy. The overall adherence to the CONSORT checklist items was poor, especially the items related to the methods section. Reporting from future studies must be optimized to avoid reporting biases and to improve transparency and reproducibility.
Supplementary information
Additional file 1: Search strategies for each electronic database.
Additional file 2: Adherence to the CONSORT items.
Authors’ contributions
Concept development: VM, LL, RLP, RR. Study selection and data extraction: RLP, LL, VM, TG, COCL. Drafting results and conclusions: VM, LL, RLP, RR, TG, COCL, PJL. Content review: PJL, RR. Approved the final version: VM, LL, RLP, RR, TG, COCL, PJL.
Funding
None.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Ethics approval
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Vânia Mozetic, Email: vaniamozetic@yahoo.com.br.
Letícia Leonel, Email: leticia.f.leonel@gmail.com.
Rafael Leite Pacheco, Email: rleitepacheco@hotmail.com.
Carolina de Oliveira Cruz Latorraca, Email: caru.pepm@yahoo.com.br.
Taís Guimarães, Email: guima.tais@gmail.com.
Patricia Logullo, Email: plogullo@uol.com.br.
Rachel Riera, Email: rachelriera@hotmail.com.
Supplementary information
Supplementary information accompanies this paper at 10.1186/s13063-019-3868-4.
References
- 1.Altman DG. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313(7057):570–571. doi: 10.1136/bmj.313.7057.570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lee R, Wong TY, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vision. 2015;2:17. doi: 10.1186/s40662-015-0026-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.International Committee of Medical Journal Editors. http://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html. Accessed 11 Oct 2018.
- 4.Altman DG, Simera I. A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network. J R Soc Med. 2016;109(2):67–77. doi: 10.1177/0141076815625599. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA. 1996;276(8):637–639. doi: 10.1001/jama.1996.03540080059030. [DOI] [PubMed] [Google Scholar]
- 6.Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030. doi: 10.1002/14651858.MR000030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan — a web and mobile app for systematic reviews. Syst Rev. 2016;5:210. doi: 10.1186/s13643-016-0384-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi: 10.1136/bmj.c869. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.ACCORD Study Group. ACCORD Eye Study Group. Chew EY, et al. Effects of medical therapies on retinopathy progression in type 2 diabetes. N Engl J Med. 2010;363(3):233–244. doi: 10.1056/NEJMoa1001288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chew EY, Davis MD, Danis RP, et al. The effects of medical management on the progression of diabetic retinopathy in persons with type 2 diabetes: the Action to Control Cardiovascular Risk in Diabetes (ACCORD) eye study. Ophthalmology. 2014;121(12):2443–2451. doi: 10.1016/j.ophtha.2014.07.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Cullen JF, Town SM, Campbell CJ. Double-blind trial of Atromid-S in exudative diabetic retinopathy. Trans Ophthalmol Soc U K. 1974;94(2):554–562. [PubMed] [Google Scholar]
- 12.Duncan LJP, Cullen JF, Ireland JT, et al. A three-year trial of atromid therapy in exudative diabetic retinopathy. Diabetes. 1968;17(7):458–467. doi: 10.2337/diab.17.7.458. [DOI] [PubMed] [Google Scholar]
- 13.Emmerich KH, Poritis N, Stelmane I, et al. Efficacy and safety of etofibrate in patients with non-proliferative diabetic retinopathy. Klin Monatsbl Augenheilkd. 2009;226(7):561–567. doi: 10.1055/s-0028-1109516. [DOI] [PubMed] [Google Scholar]
- 14.Keech AC, Mitchell P, Summanen PA, et al. Effect of fenofibrate on the need for laser treatment for diabetic retinopathy (FIELD study): a randomised controlled trial. Lancet. 2007;370(9600):1687–1697. doi: 10.1016/S0140-6736(07)61607-9. [DOI] [PubMed] [Google Scholar]
- 15.Fried LF, Forrest KY, Ellis D, et al. Lipid modulation in insulin-dependent diabetes mellitus: effect on microvascular outcomes. J Diabetes Complicat. 2001;15(3):113–119. doi: 10.1016/S1056-8727(01)00140-4. [DOI] [PubMed] [Google Scholar]
- 16.Gupta A, Gupta V, Thapar S, et al. Lipid-lowering drug atorvastatin as an adjunct in the management of diabetic macular edema. Am J Ophthalmol. 2004;137(4):675–682. doi: 10.1016/j.ajo.2003.11.017. [DOI] [PubMed] [Google Scholar]
- 17.Harrold BP, Marmion VJ, Gough KR. A double-blind controlled trial of clofibrate in the treatment of diabetic retinopathy. Diabetes. 1969;18(5):285–291. doi: 10.2337/diab.18.5.285. [DOI] [PubMed] [Google Scholar]
- 18.Ilyina Y, BezditkoAjaj P, Mohamed S, et al. Statins and fibrates as the treatment of nonproliferative diabetic retinopathy in type 2 diabetes mellitus. Spektrum Augenheilkunde. 2016;30(3):111–116. doi: 10.1007/s00717-016-0300-2. [DOI] [Google Scholar]
- 19.Massin P, Peto T, Ansquer JC, et al. Effects of fenofibric acid on diabetic macular edema: the MacuFen study. Ophthalmic Epidemiol. 2014;21(5):307–317. doi: 10.3109/09286586.2014.949783. [DOI] [PubMed] [Google Scholar]
- 20.Narang S, Sood S, Kaur B, et al. Atorvastatin in clinically-significant macular edema in diabetics with a normal lipid profile. Nepal J Ophthalmol. 2012;4(1):23–28. doi: 10.3126/nepjoph.v4i1.5846. [DOI] [PubMed] [Google Scholar]
- 21.Sen K, Misra A, Kumar A, et al. Simvastatin retards progression of retinopathy in diabetic patients with hypercholesterolemia. Diabetes Res Clin Pract. 2002;56(1):1–11. doi: 10.1016/S0168-8227(01)00341-2. [DOI] [PubMed] [Google Scholar]
- 22.Moher D, Schulz KF, Altman DG, et al. The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Med Res Methodol. 2001;1:2. doi: 10.1186/1471-2288-1-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–276. doi: 10.1016/S0140-6736(13)62228-X. [DOI] [PubMed] [Google Scholar]
- 24.Moher D, Altman DG. Four proposals to help improve the medical research literature. PLoS Med. 2015;12(9):e1001864. doi: 10.1371/journal.pmed.1001864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Baulig C, Krummenauer F, Geis B, et al. Reporting quality of randomised controlled trial abstracts on age-related macular degeneration health care: a cross-sectional quantification of the adherence to CONSORT abstract reporting recommendations. BMJ Open. 2018;8(5):e021912. doi: 10.1136/bmjopen-2018-021912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Huang YQ, Traore K, Ibrahim B, et al. Reporting quality of randomized controlled trials in otolaryngology: review of adherence to the CONSORT statement. J Otolaryngol Head Neck Surg. 2018;47(1):34. doi: 10.1186/s40463-018-0277-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Campbell MK, Piaggio G, Elbourne DR, Altman DG. CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661. doi: 10.1136/bmj.e5661. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1: Search strategies for each electronic database.
Additional file 2: Adherence to the CONSORT items.
Data Availability Statement
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.