Reporting quality of randomised controlled trial abstracts on age-related macular degeneration health care: a cross-sectional quantification of the adherence to CONSORT abstract reporting recommendations

Abstract

Objective

To assess the reporting quality of randomised controlled trial (RCT) abstracts on age-related macular degeneration (AMD) healthcare, to evaluate the adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement’s recommendations on minimum abstract information and to identify journal characteristics associated with abstract reporting quality.

Design

Cross-sectional evaluation of RCT abstracts on AMD healthcare.

Methods

A PubMed search was implemented to identify RCT abstracts on AMD healthcare published in the English language between January 2004 and December 2013. Data extraction was performed by two parallel readers independently by means of a documentation format in accordance with the 16 items of the CONSORT checklist for abstracts. The total number of criteria fulfilled by an abstract was derived as primary endpoint of the investigation; incidence rate ratios (IRRs) with unadjusted 95% CI were estimated by means of multiple Poisson regression to identify journal and article characteristics (publication year, multicentre design, structured abstract recommendations, effective sample size, effective abstract word counts and journal impact factor) possibly associated with the total number of fulfilled items.

Study characteristics

136 of 673 identified abstracts (published in 36 different journals) fulfilled all eligibility criteria.

Results

The median number of fulfilled items was 7 (95% CI 7 to 8). No abstract reported all 16 recommended items; the maximum total number was 14, the minimum 3 of 16 items. Multivariate analysis only demonstrated the abstracts’ word counts as being significantly associated with a better reporting of abstracts (Poisson regression-based IRR 1.002, 95% CI 1.001 to 1.003).

Conclusions

Reporting quality of RCT abstracts on AMD investigations showed a considerable potential for improvement to meet the CONSORT abstract reporting recommendations. Furthermore, word counts of abstracts were identified as significantly associated with the overall abstract reporting quality.

Strengths and limitations of this study.

• Reporting quality of randomised controlled trial (RCT) abstracts on age-related macular degeneration (AMD) therapy has not been assessed so far.

• The cross-sectional inclusion of all published RCT abstracts on AMD healthcare without selection of journals ensures maximum possible representativeness.

• Data extraction and evaluation were performed by two independent readers with long-term experience in clinical trial methodology and reporting bias evaluation.

• The readers were not blinded to the journal and publication period, so that the possibility of reader bias cannot be ruled out entirely.

• Journal characteristics could only be considered as far as published by the journals; the individual reviewing processes underlying the 136 abstracts might have taken additional influence on the actual abstract presentations.

Introduction

Randomised controlled trials (RCTs) are considered as optimum origin of evidence and have the highest grade of research designs.^{1 2} Frequently, readers of articles reporting on RCT start by screening the content of the abstract first, which subsequently guides the decision of whether or not to obtain and read the entire article.³ In addition to an overwhelming day-to-day workload, a steadily increasing number of publications and limited access to many full-text articles enforce healthcare providers to build their healthcare decisions on information in abstracts.⁴ Thus, abstracts of RCTs should contain accurate and clear information about implementation, evaluation, findings and synopsis of the clinical trial.⁵

Regarding the methodological details, there was no standardised reporting requirement before the Consolidated Standards of Reporting Trials (CONSORT) were established in 1996⁶ and revised in 2001⁷ and 2010,^{8 9} respectively. In order to prevent discrepancies between full-text articles and abstracts, as well as to improve the reporting quality of abstracts, an extension to the CONSORT statement was published in 2008.⁵ This statement contains 17 items comprising eight sections: title, author details (specific to conference abstracts), trial design, methods (including participants, interventions, objectives, defined primary outcome, randomisation, blinding), results (numbers randomised, recruitment, numbers analysed, outcomes, harms), conclusions, trial registration and declaration of funding.⁵ This enumeration ‘provides a minimum list of essential items, that authors should consider when reporting the main results of a randomised trial in any journal or conference abstract’.⁵

Several studies have been conducted to evaluate the quality of abstract reporting in general medical journals^10–14 or specific fields of medicine such as healthcare,¹⁵ anaesthesia,¹⁶ traditional Chinese medicine,¹⁷ HIV/AIDS,³ paediatrics,¹⁸ dentistry^19–22 and oncology.²³ These studies thoroughly demonstrated that there is still a need for optimisation in the reporting of RCT abstracts.

To our knowledge, the reporting quality of RCT abstracts in the field of ophthalmology and especially regarding the treatment of age-related macular degeneration (AMD) has not been assessed yet. AMD is a common eye disease and the leading cause for vision loss among people 50 years of age and older,²⁴ having led to increasing research need on therapeutic concepts and thereby to an increasing number of RCTs on AMD during the past decade. Therefore, the objectives of the present cross-sectional complete census were to assess the reporting quality of RCT abstracts in AMD healthcare, to evaluate the adherence to the CONSORT statement’s recommendations on minimum abstract information and to identify journal characteristics associated with this parameterisation of abstract reporting quality.

Methods

Study design

A cross-sectional evaluation of RCT abstracts on AMD healthcare published between 2004 and 2013 was implemented; to identify eligible RCT publication abstracts, a systematic review of abstracts was performed without any restrictions (implying a complete census of all published abstracts), according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidance (see online supplementary material for PRISMA checklist). Human subjects were not involved in this study.

Data sources and search strategy

A MEDLINE/PubMed search for all RCTs published between 2004 and 2013 was performed. The search strategy used the MeSH terms ‘randomised controlled trial’ as publication type and ‘macular degeneration’ as term in title/abstract and was limited to the publication date (1 January 2004 to 31 December 2013) as well as to English as publication language. The publication period was limited in order to reflect the period of 5 years before and after the publication of the extension of CONSORT in 2008. The search was carried out on 24 September 2015 as can be reproduced by online audit trail documentation (see online supplementary material).

Study selection

Only abstracts reporting on AMD healthcare were considered and then investigated for indications of an underlying RCT design: we included abstracts in which the allocation of participants to interventions was described by the words ‘random’, ‘randomly’, ‘randomised’, ‘randomisation’ or any other terminology suggesting that the participants were randomly distributed to treatment arms. Reports not associated with AMD therapy, without reference to randomisation or an obviously retrospective study design, economic analyses, diagnostic or screening tests, questionnaire reporting, study protocols, observational studies, editorials or letters were excluded as well as reviews, meta-analyses and non-human studies. Two parallel reviewers (CB, SK) independently selected the abstracts; disagreement about which abstracts had to be included was resolved by discussion and consensus, reasons for article exclusion were documented.

Patient and public involvement

The present work does not include original patient data but is based on the evaluation of published abstracts of RCTs in ophthalmological journals. Therefore, no patients or public were involved in this study.

Data extraction

A pilot study was performed with randomly selected abstracts concerning cataract surgery to identify problems and solve discrepancies in data collection and analysis.²⁵ The data extraction was performed in the same way (CB, SK) using the previously created and pretested data extraction and documentation form in accordance to the items of the CONSORT checklist for abstracts.⁵ Each item had a ‘yes’/‘no’ rating indicating whether the authors had reported this item or not. Only the ‘authors’ details’ item concerning contact details of the corresponding authors was excluded from the assessment, since this item is specific to conference abstracts. For the ‘outcomes’ item, two subitems were evaluated—as required by the CONSORT statement: (1) primary outcome result presented for each group/arm, (2) for primary outcome, effect size reported for trials with binary outcome resp. with continuous outcome and the precision of this effect size (CI). Only if both criteria were met, a correct reporting of the outcome results items was ascertained. For the ‘conclusions’ item, also two subitems had to be reported: (1) consistency with the reported results and (2) discussion of benefits of and harms from the intervention. Again, only if both subitems were reported, the correct reporting of ‘conclusions’ was ascertained.

Furthermore, the following information on journal and article characteristics were collected: name of the journal, name of first author, year of publication, multicentre design [yes/no], abstract format prespecified as structured (yes/no), effective sample size, impact factor (IF) of each journal according to the respective publication year. Due to the fact that word count limits in instructions for authors were found to show severe deviations from the number of words actually used in the abstract, both counts were recorded (word count limitation (status 2017) as well as actual word count in the abstract). In addition, since only the current word count specifications of each journal could be found (status 2017), the actual word counts for each abstract were used for exploratory evaluations (see below).

Primary endpoint

The abstract-wise primary endpoint of this investigation was the total absolute number of items (among all 16 considered) reported in the respective abstract. To provide maximum scientific evidence, we decided to evaluate all existing abstracts (n=136) and did not draw a random sample, thereby providing a complete census of all available abstracts.

Exploratory analysis

Descriptive analysis of the primary endpoint was based on medians and quartiles, graphical presentation on box whisker plots, accordingly. For each CONSORT abstract item, Cohen’s kappa (point estimate and one-sided 95% CI) was derived to assess the parallel readers’ agreement, respectively. A kappa point estimate of 0.60 or higher was considered as an indication for substantial interobserver agreement in the evaluation of the respective item.

Multivariate analysis

Incidence rate ratios (IRRs) with 95% CI (not adjusted for multiplicity with regard to the exploratory character of the multivariate analysis) were estimated by means of multiple Poisson regression to identify journal and article characteristics (publication year, multicentre design, structured abstract recommendations, effective sample size, effective abstract word counts and journal IF) possibly associated with the total number of fulfilled criteria. Poisson regression modelling was performed by backward variable selection via the Akaike information criterion, considering likelihood ratio test p values <0.05 as indicators of local statistical significance (ie, model exploration results were not formally adjusted for multiplicity). The multivariate analysis was conducted using ‘R’ V.3.4.0 (R Core Team, 2017).

Results

Out of 673 identified study publications, 537 had to be excluded (figure 1). In summary, a total of 136 abstracts fulfilling the eligibility criteria were included in the investigation.

Figure 1.

Flowchart of the literature search and identification of randomised controlled clinical trial abstracts; the exclusion of studies took place sequentially in the following order: no abstracts available, not affiliated to AMD, inappropriate study design, no indication for randomisation. AMD, age-related macular degeneration; RCTs, randomised controlled trials.

Characteristics of included abstracts and underlying journals

The search yielded 136 abstracts of which 55 (40%) were published between 2004 and 2008 (pre-CONSORT) and 81 (n=60%) between 2009 and 2013 (post-CONSORT). The abstracts were published in 36 different journals, of which only two (publishing four articles) had no Thomson Reuters IF. The median IF of all journals—according to the respective publication year—was 3.125 (IQR Q1 2.367; Q3 5.127). Only seven studies (5%) were published in journals with an IF of 10 or higher. Most of the publications (32%) were found in ‘Ophthalmology’ (43/136) followed by ‘British Journal of Ophthalmology’ 10% (13/136) and ‘Retina’ 9% (12/136) (table 1).

Table 1.

Listings of journals with at least one age-related macular degeneration RCT abstract considered in the evaluation with total number of publications (N) with abstracts evaluation from the respective journal as well as relative frequency (%) of these abstracts among all abstracts considered in this investigation, 5 years impact factor (IF) (Thomson Reuters, 2016) and information on word count limits according to the published instructions for authors of the respective journals (status 2017)

Journal (journal title abbreviation)	N	Proportion (%)	5 years IF (2016)	Word count limits (words)
Am J Ophthalmol	9	7	4.797	250
Ophthalmology	43	32	7.788	350
Graefes Arch Clin Exp Ophthalmol	4	3	2.274	250
Br J Nutr	1	1	3.784	250
Clin Hemorheol Microcirc	1	1	1.647	200
Atheroscler Suppl	1	1	3.310	250
N Engl J Med	4	3	64.201	250
Retina	12	9	3.779	200
Arch Ophthalmol	5	4	4.372	350
Br J Ophthalmol	13	10	3.466	250
Scand J Occup Ther	1	1	1.561	200
Acta Ophthalmologica	7	5	2.812	250
Eye (Lond)	5	4	2.547	250
Nutrients	1	1	4.187	200
PLoS One	1	1	3.394	300
Nutrition	2	2	3.312	250
Curr Med Res Opin	1	1	2.605	250
Invest Ophthalmol Vis Sci	4	3	3.786	250
Clin Rehabil	1	1	3.026	250
Trans Am Ophthalmol Soc	1	1	0.000	250
Optometry	3	2	0.000	250
Biomedical Papers	1	1	1.160	250
J Clin Neurosci	1	1	1.545	250
BMJ	1	1	19.355
Proc Natl Acad Sci U S A	1	1	10.414	250
JAMA Ophthalmol	1	1	5.425	350
Eur J Ophthalmol	1	1	1.161	250
Complementary and Alternative Medicines	1	1	2.644	300
BMC Ophthalmology	1	1	1.579	350
Experimental Eye Research	2	2	3.235	500
Current Eye Research	2	2	1.947	300
Annals of Nutrition and Metabolism	1	1	2.883	250
Lancet	1	1	48.082	300
Ophthalmologica	1	1	1.918	250
J Ocul Pharmacol Ther	1	1	1.726	250
Total	136

Only one journal did not state word count limits for abstracts in the instructions for authors (status 2017). All other 35 journals limited the words in abstracts to a total number between 200 and 500 words (see table 1). The actual number of words in abstracts varied between 141 (minimum) and 457 (maximum) with a median of 273 words. Seventy (51%) studies were single-centre trials with a minimum sample size of 7 participants up to 300 participants (median 46, Q1 28; Q3 100), and 66 (49%) studies were multicentre trials with a minimum sample size of 25 participants up to 2457 participants (median 223, Q1 117; Q3 494).

Overall reporting quality

None of the 136 abstracts reported all 16 items. The best abstract reported 14 of 16, the worst 3 of 16 items. The median number of reported items was 7 (95% CI 7 to 8). Comparing the pre-CONSORT and post-CONSORT periods, abstract reporting improved from median 7 (95% CI 6 to 7) to median 8 (95% CI 7 to 8) reported items. In total, 104 of 136 abstracts (77%) reported eight items or less, whereas 32 abstracts stated nine items or more (23%). Best reported items were ‘interventions’ (95% reported) and ‘objectives’ (98%). The worst reporting was on ‘outcomes’ with 0% (see online supplementary material table).

Reporting of ‘general items’

Only 42% (57/136) identified the presented investigation as randomised in the title. Seventy-one per cent (97/136) described the trial design as required (the information ‘randomised controlled trial’ or ‘multi-centre/single-centre’ were not sufficient).

Reporting of ‘trial methodology’

Eligibility criteria for participants and for interventions—to be documented for each treatment group—were reported in 90% (123/136) and 95% (129/136), respectively. Similarly, the specific objective or hypothesis was mentioned in 98% (133/136). Seventy per cent (95/136) reported a clearly defined primary outcome. Only 29% (39/136) of abstracts stated information on randomisation and 43% (59/136) on blinding. Details about trial status were given in 9% (12/136).

Reporting of ‘results’

Of the 136 abstracts included, 55% (75/136) reported the number of participants randomised to each group, but only 22% (30/136) reported the number of participants analysed for each group. Important adverse events were described in 48% (65/136). As CONSORT recommends three different subitems for reporting outcomes, this investigation found a total of 64% (87/136) abstracts reporting primary outcome results for each group, but none of the 136 abstracts reported on effect size and CI for the primary outcome in full detail. The proportion of abstracts describing effect size and CI for primary outcome was 7% (9/136), among which 7% (3 out of 41 trials) with binary outcome and 6% (6 out of 95 trials) with continuous outcome. In total, none of the 136 abstracts met both criteria, so the correct reporting frequency for the ‘results’ indicators was estimated at 0%.

Reporting of ‘conclusions’

Information on a trial registration number was found in 19% (26/136) of the abstract, and a possible funding source was referenced in 24% (33/136). Correct interpretations of the results in the ‘conclusions’ section as recommended by CONSORT were available in 17% (23/136): 96% (130/136) of the abstracts reported adequately and in compliance with the trial results, but only 18% (24/136) commented on both benefits and harms of the trial therapies in the ‘conclusions’ section.

Agreement among parallel readers

Table 2 shows the interobserver agreement for each item (Cohen’s kappa and one-sided 95% CI). Substantial agreement was found for almost all items under consideration. Notable less in agreement as indicated by kappa point estimates <0.6 were only found for the items ‘randomisation’, ‘numbers randomised’ and ‘recruitment’.

Table 2.

Item-wise interobserver agreement (Cohen’s kappa, asymptotic one-sided 95% CI) for 16 abstract reporting items as recommended by the Consolidated Standards of Reporting Trials statement: kappa point estimates >0.60 were considered as indicators of substantial parallel reader agreement (signed with *)

Item	Kappa point and one-sided 95% CI estimates for interobserver agreement	Kappa >0.60 (substantial agreement)
Title	0.91; ≥0.84	*
Authors*	–
Trial design	0.89; ≥0.81	*
Methods
Participants	0.76; ≥0.58	*
Interventions	0.79;≥0.56	*
Objectives	0.74; ≥0.40	*
Definition of primary outcome	0.63; ≥0.49	*
Randomisation	0.55; ≥0.42
Blinding (masking)	0.81; ≥0.71	*
Results
Numbers randomised	0.56; ≥0.42
Recruitment	0.51; ≥0.24
Numbers analysed	0.65; ≥0.51	*
Outcomes	1.00; ≥0.99	*
Harms	0.85; ≥0.76	*
Conclusions	0.69; ≥0.54	*
Trial registration	0.84; ≥0.72	*
Funding	0.94; ≥0.87	*

Journal characteristics associated with quality of reporting

The multivariate analysis (see table 3) only demonstrated the abstracts’ effective word counts as being significantly associated with a better reporting of abstracts (IRR 1.002, 95% CI 1.001 to 1.003). As could be expected, the latter indicates a proportional benefit of abstracts with increasing abstract text length. The multiple Poisson regression model fit was, however, estimated at only 11% (Nagelkerkés Pseudo R²) even after extensive exploratory sensitivity analyses (introducing interaction terms for the explanatory variables under consideration as well as introducing these variables as continuous and binary into the models, respectively).

Table 3.

Results of multivariate analysis (multiple Poisson regression) relating the total number of reported abstract items among 16 Consolidated Standards of Reporting Trials recommendations per considered randomised controlled trial publications (n=136) to general characteristics of the underlying journals; associations between violation counts and journal characteristics were quantified via incidence rate ratios (IRR) point estimates and the corresponding unadjusted two-sided 95% CI (CI lower and upper bound, respectively); unadjusted two-sided p values were derived from likelihood ratio tests and indicate locally significant associations in case of p≤0.05

	IRR	95% CI lower	95% CI upper	P values
(Intercept)	4.318	3.254	5.730	<0.001
Publication date (<2008/≥2008)	1.109	0.975	1.261	0.115
Abstract word count	1.002	1.001	1.003	0.001

AIC: 587.27

Discussion

In the present study, 136 RCT abstracts concerning the treatment of AMD were identified and assessed. The overall quality of reporting based on the CONSORT for abstracts checklist criteria was with median 7 reported items out of 16 items poor. Only two CONSORT items (‘interventions’ and ‘objectives’) were adequately reported in most abstracts (>90%). However, no abstract provided complete information on outcomes as required. The main problem was found in the reporting of effect size and CI (for each trial arm/group as well as in total). Less than 25% of abstracts included sufficiently reported ‘recruitment’, ‘numbers analysed’, ‘conclusions’, ‘trial registration’ and ‘funding’. Information on ‘randomisation’ was available in 29% which also implies that reporting was not transparent. In the conclusions section, most abstracts (96%) reported conclusions consistent with the results but just a few of them (18%) addressed potential limitations of the study or noted whether additional studies were required due to different reasons.

In particular, none of the 136 abstracts presented sufficient reporting of result outcomes, which demonstrates the crucial need for improvement in this field by, for example, provision of explicit instructions for authors in terms of standard reporting formats. The non-reporting of the effect size and its CI for primary outcome has to be identified as the main problem.

Two further studies of Hua (0% pre-CONSORT and 2.3% post-CONSORT)²¹ and Chen (1%)¹⁰ presented similar results. Other studies showed better but also improvable reporting of outcomes (Bigna 25.2% pre and 42.5% post³; Can 5.6%–18.4% pre and 20.4%–38.8% post, depending on the journal¹⁶; Berwanger 62.3%¹¹). From our point of view, the abstract is crucial, and therefore, the outcomes are essential in an abstract. Strict fulfilment of all criteria as indicated by the CONSORT statement for abstracts was therefore required by the authors: primary outcome result for each group and for primary outcome as well as effect size and CI reported appropriate for trials with binary outcome resp. with continuous outcome. In this respect, we have not allowed any scope for interpretation and the parallel reader evaluation followed a strict ‘no tolerance strategy’, which might explain the embarrassing result for this item in our investigation. In addition, in a randomised study with at least two different therapies, it should be possible to present an effect size between them. RCTs are expected to provide a high degree of evidence. However, 136 RCT abstracts did not contain sufficiently detailed data on the study outcome (effect size in combination with significance or confidence measure) in such explicit terms as required by the CONSORT statement. This lack of reporting quality certainly means a more crucial loss in transparency and reproducibility than, for example, suboptimal fulfilment of rather editorial recommendations. In summary, physicians conducting clinical research are strongly advised to involve a biometrician in the formulation of their results.

AMD is a common eye disease with an increasing need for research in the last years. However, the reporting quality of RCT abstracts concerning AMD therapy has not been assessed until we conducted our study. Our findings were disillusioning but consistent with previous studies assessing the quality of reporting of journal RCT abstracts concerning different diseases: Bigna,³ for example, found in 2016 in 312 abstracts concerning HIV median 6 reported items pre-CONSORT and median 7 items post-CONSORT, and stated that this suboptimal improvement was associated with the journal’s high IF, large number of authors and non-pharmacological/vaccine intervention in the trial. Also in 2016, Chhapola found in 891 abstracts in three leading paediatrics journals¹⁸ median 7 to 8 items (depending on the journal) pre-CONSORT and median 7 to 10 items post-CONSORT. The authors assumed that the reporting quality may be affected by constraints of space and word limit as well as structured versus unstructured abstracts. Ghimire²³ investigated 956 phase III oncology trial abstracts published pre-CONSORT and post-CONSORT for abstracts and found median 8.2 (95% CI 8.0 to 8.3) and 9.9 (95% CI 9.7 to 10.2) in the pre-CONSORT and post-CONSORT periods, respectively. A high IF and the journal of publication were independent factors significantly associated with higher reporting quality on their multivariate analysis. In our study, the multivariate analysis showed that only the effective number of words had an influence on the quality of abstracts. Hopewell⁵ showed that with a word limit of 250 to 300 words, the checklist items could be easily incorporated. This was also reflected in our investigation: Abstracts with >250 words showed better reporting than abstracts with 250 words or less. It is conceivable that with a word limit of maximum 250 words medical authors focus on medical aspects and less on methodological aspects such as randomisation, blindness, trial status or a correct outcome presentation as recommended by CONSORT. On the other hand, Berwanger¹¹ impressively demonstrated the possibility of expressing maximum information about methodological aspects in just a few words.

In this study, the IF of journals or structuring of abstracts was not found associated with the quality of reporting in AMD abstracts as was found in other studies. One reason for this could be the inclusion of 36 different journals, in which RCTs concerning AMD were published between 2004 and 2013. We had determined the IF for each journal depending on the year of publication: Therefore, we had a great variability in the IFs ranging from 0 to 52.414 and a median IF of 3.125. Most other studies relate to abstracts in preselected journals10–12 14 16–18 that specialised in the particular disease. These studies refer to a handful of preselected journals with only a few different IFs, which possibly explains this influence in contrast to our study. Furthermore, only eight journals with nine abstracts contained unstructured abstracts in this investigation. This could be the reason why an influence of abstract structure was not demonstrated here. The present investigation also failed to demonstrate an improvement of quality after publication of the CONSORT extension for abstracts in 2008. This result is similar to several previous surveys,^{18–20 26} whereas some other studies using a simple pre–post comparison showed a slight improvement.3 14 16 23 Due to the inclusion of all journals without preselection and over a long period of publication, the authors were not able to identify whether the respective journal made a reference to the CONSORT statement at the publication time of each RCT, and especially whether the journal contained a reference to the CONSORT extension for abstracts. This is certainly a limitation of this investigation which, however, was not preventable due to a lack of (online) information.

Our investigation shows high process validity from the methodological perspective, as the identification of RCTs, eligibility decision and data extraction were performed by two independent readers, both having several years of experience in the publication and review of clinical trials. In addition, all items were clearly parameterised before the investigation by means of a standard operating procedure on the CONSORT items’ evaluation, and then discussed and mutually validated by means of these procedures. For this purpose, a pilot study in a different indication (cataract surgery) was carried out²⁵ in order to identify possible weaknesses and interferences in advance. Nevertheless, kappa values for some items only showed moderate interobserver reliability. This is partly explained by the fact that it was difficult—based only on the written information—to decide whether abstracts contained information on ‘numbers randomised’ or ‘numbers analysed’. Only in a few cases, both information were presented explicitly. Therefore, the ‘moderate’ interobserver reliability in these items was not only based on different ratings, but also on non-transparent and unclear reporting in the respective abstract.

We did not take a random sample of RCTs, but rather used systematic review methodology to identify eligible RCT publication abstracts for our purpose, and then decided to perform a full cross-sectional census evaluation on these abstracts. The use of a sensitive PubMed research strategy and the data collection from 2004 to 2013 led to the inclusion of all published RCTs in AMD with no selection of special journals. This represents the whole body of scientific evidence in the field of AMD until the end of the publishing period under consideration—and actually, as we presume—until today. Of course, we are aware that the limitation of publication period (2004–2013) must be viewed critically: We sought to depict a 10-year period in our study, which covered both the period before and after the publication date of the CONSORT guidelines for abstracts (‘CONSORT for abstracts’ was published in January 2008⁵). Since we assumed that these recommendations did not find immediate uptake in the year of publication, we defined 2004–2008 as a 5 years pre-CONSORT and 2009–2013 as a sufficiently long-term post-CONSORT 5-year period. It must be admitted that even after a sufficient uptake time after the ‘CONSORT for abstracts’ being published, the publication date is only a proxy for the actual uptake of the recommendation. For example, a publication first submitted in 2007—thereby not underlying the recommendations’ content—might have been published in 2009 after a 2-year review and editing process. As a consequence, a 2009 ‘post-CONSORT’ publication would have been misclassified as actually being written in the pre-CONSORT period. However, during the literature research on this topic, we found that various authors had already examined the pre–post comparison.3 14 18 21 For this reason, we included the 2008 cut-point via the binary variable ‘publication date before/after 2008’ in the Poisson regression model. Consequently, the pre–post comparison was thought to be a possible factor influencing the quality of the abstracts. This, however, was not confirmed, which could possibly be due to the selected time periods. A further investigation covering the next 5-year period (2014–2018) could possibly show a trend towards CONSORT uptake.

An undeniable limitation is that both readers were not blinded to the journal and publication period so the possibility of assessor bias cannot be ruled out entirely. Furthermore, as in other studies,3 10 11 14 16–18 all 16 CONSORT items were equally weighted and only the presence of an item was rated as ‘yes/no’. However, it can be noted that certainly not all items have the same impact on the transparency of the reporting process, and some items, for example, on explicit result presentation, are certainly more important than others such as details on funding. Nevertheless, all items of the CONSORT checklist should be reported in an abstract and a median number of reported items of 7 (95% CI 7 to 8) must be considered as poor reporting quality.

A further limitation is that this study was carried out without preparing a study protocol in advance, as its evaluation was based on the previously tried and tested procedure of the pilot study.²⁵ Nevertheless, there remains a risk of selective reporting.

In summary, the reporting quality of RCT abstracts on AMD healthcare showed a considerable potential for improvement to meet the CONSORT abstract reporting recommendations. The reporting of the outcomes in particular was disillusioning, even though CONSORT provide detailed information in its checklist explanations on how these results should be presented. Since these further explanations are very detailed, it seems as if these notes are hardly being read, and therefore lack the degree of awareness they deserve. One possibility would be to provide elaborate examples/templates by means of commented ‘best-practice abstracts’ as supplementary material to a journal’s ‘instructions for authors’. These abstracts would be held in the authors’ terminology and clinical context, and thereby would become much more accessible than easier to imitate for the clinical author than the transfer of recommendation explanations.

Furthermore, word count limits for abstracts of 250 were identified as a significant determinant of the overall abstract reporting quality. These word count limits could possibly be neglected if annotated best-practice abstracts were made available, or else they could be modified according to the requirements of the respective journals in order to improve quality. To reduce the amount of research waste,²⁷ improvements in abstract quality are urgently needed. In particular, the correct statement of the exact effect size is of utmost importance in an abstract, in order to be able to transfer a statement into everyday clinical practice.