Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study

Abstract

Objectives

To evaluate the reporting quality of randomised controlled trial (RCT) abstracts presented at a leading international conference in sleep medicine (the SLEEP Annual Meeting), and to investigate the association between potential predictors and the reporting quality of trial abstracts in this field.

Design

Cross-sectional, research on research study.

Methods

A handsearch of the 2016–2018 SLEEP Annual Meeting abstract books was carried out to identify abstracts describing RCTs. Quality of reporting was assessed with the original 17-item CONSORT for Abstracts checklist. Univariable and multivariable linear regression analyses were performed to identify significant predictors of reporting quality. In addition, risk ratios were used to analyse the adequate reporting rate of each quality item by type of intervention and funding status.

Primary and secondary outcome measures

The overall quality score (OQS, range 0–17) in accordance with the CONSORT for Abstracts checklist (primary outcome), and the adequate reporting rate of each checklist item (secondary outcome).

Results

A total of 176 RCT abstracts were included and assessed. The mean OQS was 5.53 (95% CI 5.30 to 5.76). Only three quality items (objective, conclusions and funding) were adequately reported in most abstracts (>75%). None of the abstracts adequately reported authors, randomisation or outcome in the results section. According to the multivariable analysis, pharmacological interventions (p=0.018) and funding from the industry (p=0.025) were significantly associated with better reporting quality.

Conclusions

The reporting quality of RCT abstracts presented at SLEEP Annual Meetings was suboptimal. Pharmacological intervention and funding from industry were significant predictors of better reporting quality. Joint efforts by authors and conference committees are needed to enhance the reporting quality of RCT abstracts presented at sleep medicine conferences, and thereby reduce relevant research waste in this field.

Strengths and limitations of this study.

• Quality assessment was carried out using the original Consolidated Standards of Reporting Trials for Abstracts checklist, which facilitates the replication of this study and its inclusion in future meta-analyses.

• Assessors were blinded to author names and author affiliations of included abstracts during the quality assessment process.

• Only one major conference was included, thus, our findings may not be applicable for other conferences in sleep medicine.

Introduction

It is a shared obligation and responsibility for all researchers to ensure completeness, transparency and accuracy when reporting health research.¹ However, a previous article indicated that about half of research in biomedicine were poorly reported and therefore unusable, resulting in an avoidable waste of billions of dollars of funding every year.² Thus, research into the current reporting quality of medical research publications (articles, abstracts and protocols etc) and potential methods to reduce the relevant research waste are warranted.

Since the emergence of evidence-based medicine, high-quality randomised controlled trials (RCTs) have been considered as the highest level in the hierarchy of primary studies,³ and also the gold standard for the evaluation of healthcare interventions.⁴ The results of RCTs are often first presented at medical conferences in the form of abstracts, but only about one-third of these abstracts were subsequently published as full-length articles.^{5 6} Therefore, abstracts presented at conferences might be the only reports of the corresponding research that are available for potential readers. In addition, as systematic reviews often include such unpublished data,⁷ RCT abstracts presented at conferences can have significant impact on evidence-based clinical practice.

Considering the importance of RCT abstracts, an extension of the Consolidated Standards of Reporting Trials (CONSORT) statement was released in 2008. It summarised the essential items of RCTs that should be reported in journal and conference abstracts.⁸ However, in spite of the release of this statement (also known as the CONSORT for Abstracts guidelines), recent studies have suggested that the reporting quality of RCT abstracts presented at major dental, geriatric and urological conferences remained suboptimal.^9–11

The SLEEP Annual Meeting is an international forum for the communication of novel developments in clinical sleep medicine as well as sleep and circadian science, covering many important topics such as the treatments for insomnia and sleep-related breathing disorders (SBD). However, to our knowledge, there has been no assessment of the reporting quality of abstracts presented at this major conference. Therefore, we carried out a study to assess the reporting quality of RCT abstracts presented at the 2016–2018 SLEEP Annual Meetings, and to identify potential factors associated with the abstract reporting quality in this field.

Methods

Retrieval and selection of abstracts

The abstract books of the 2016–2018 SLEEP Annual Meetings were obtained from the official website (www.sleepmeeting.org). Thereafter, one author (QS) handsearched these books to identify all abstracts describing RCTs, with the eligibility of each identified abstract verified by two experts (FH and HH). Based on the Cochrane criteria for RCT selection, we aimed to include healthcare-related studies that were interventional, conducted in human participants and randomly allocated participants to different intervention groups.¹² Secondary analyses of RCTs and studies only included baseline data were excluded. All eligible abstracts were compiled into a Word document with all author names and affiliations removed to facilitate blinded quality assessment.¹³

Assessment of reporting quality

As determined a priori, the original CONSORT for Abstract guidelines and associated explanations, which included 17 quality items for conference abstracts⁸ was used to assess reporting quality. Before the assessment of all included abstracts, an internal pilot study was carried out to calibrate one author (QS) and an expert (FH) in iterative rounds of 20 randomly selected abstracts, until the inter-rater agreement was excellent or better (Cohen κ>0.75).¹⁴

Thereafter, one author (QS) assessed the reporting quality of the remained RCT abstracts, with all encountered issues and uncertainties resolved through a discussion with two experts (FH and HH).¹⁵ A score of ‘0’ (inadequate) or ‘1’ (adequate) was recorded according to the level of reporting for each checklist item. Thereafter, an overall quality score (OQS) was calculated by totalling the score of 17 items (score range: 0–17). Additionally, we recorded the reporting of 11 important subitems to provide as indicated in the explanatory document of CONSORT for Abstracts.⁸

Data extraction

After quality assessment was completed, two authors (QS and TZ) extracted the following information from each abstract, which could be potential predictors of abstract reporting quality: publication date (year), number of authors, geographical origin (first author), type of intervention (pharmacological vs non-pharmacological), number of centres (single centre vs multicentre), word count, research area (insomnia, SBD or others), sample size, as well as funding status (funded by industry vs not funded by industry). Research area was categorised in reference to the International Classification of Sleep Disorders, Third Edition.¹⁶ Another author (FH) verified the extracted data and all disagreements were resolved through discussions.

Statistical analyses

Cohen’s kappa coefficient was calculated to measure inter-rater agreement. Descriptive statistics were used to summarise the characteristics of all included abstracts, as well as the overall reporting quality and the adequate reporting rate of each checklist item/subitem. Additionally, linear regression analyses were performed to determine the association between potential predictors and abstract reporting quality (dependent variable, OQS). We conducted univariate analyses first, and then entered all significant predictors in the univariable analyses into multivariable modelling. Assessment of residuals did not indicate significant violation of normality. For the multivariable analysis, tolerance and variance inflation factor (VIF) were used to detect multicollinearity. We would have excluded any predictor from the final model, if it had a tolerance below 0.1 and/or a VIF above 10.¹⁷ Risk ratios (RRs) and their 95% CIs were calculated to illustrate differences in the adequate reporting rate of each item by types of intervention (pharmacological vs non-pharmacological) and funding status (funded by industry vs not funded by industry). For all analyses, statistical significance was defined as two-sided p<0.05.

Patient and public involvement

The present work was based on an assessment of presented RCT abstracts in the SLEEP Annual Meetings and did not include any participant, patient advisers or original data from patients. Patients were not involved in the design, recruitment and conduct of this study.

Results

Characteristics of included abstracts

A total of 176 RCT abstracts were identified and included (see online supplementary information for the flow chart). As demonstrated in table 1, most of the included abstracts were presented by authors from North America (76.7%), conducted in a single centre (90.9%), written in 301–350 words (71.0%) and describing non-pharmacological interventions (75.0%). Half of the included abstracts had 4–7 authors (50.0%). The majority of these abstracts were about insomnia (54.5%) and SBD (27.3%), and about one-fourth (26.1%) of them were funded by the industry.

Table 1.

Characteristics of included abstracts

Characteristic	Category	N (%)
Year	2016	53 (30.1)
	2017	64 (36.4)
	2018	59 (33.5)
No of authors	<4	21 (11.9)
	4–7	88 (50.0)
	>7	67 (38.1)
Continent	North America	135 (76.7)
	Europe	15 (8.5)
	Oceania	15 (8.5)
	Asia	8 (4.6)
	South America	3 (1.7)
Type of intervention	Pharmacological	44 (25.0)
	Non-pharmacological	132 (75.0)
No of centres	Single centre	160 (90.9)
	Multicentre	16 (9.1)
Word count	<250	3 (1.7)
	251–300	17 (9.7)
	301–350	125 (71.0)
	>350	31 (17.6)
Research area	Insomnia	96 (54.5)
	Sleep-related breathing disorders	48 (27.3)
	Others	32 (18.2)
Sample size	<25	46 (26.1)
	25–100	63 (35.8)
	>100	67 (38.1)
Funded by industry	Yes	46 (26.1)
	No	130 (73.9)
Overall		176 (100)

Pilot study

The internal pilot study was completed after round 2, in which excellent inter-rater agreement was reached (κ=0.841).

Reporting of general items

The adequate reporting rates of each CONSORT item and subitem among the 176 included abstracts are presented in table 2. Only 52 abstracts (29.5%) can be identified as randomised through their titles. Less than one-third (32.4%) of the abstracts described their trial design explicitly. Only six abstracts (3.4%) provided details about trial registration. Although authors’ names and their institutions were provided in the abstracts, no abstract reported contact details of the corresponding author. Nevertheless, a majority of the included abstracts (92.0%) reported their source of funding.

Table 2.

Reporting of each CONSORT checklist item and subitem in the included 176 abstracts

Items	Criteria and subitems	N (%)
1. Title	Identification of the study as randomised	52 (29.5)
2. Authors	Contact details for the corresponding author	0 (0)
3. Trial design	Description of the trial design (eg, parallel, crossover)	57 (32.4)
4. Participant	Eligibility criteria for participants and the settings where the data were collected	16 (9.1)
	4a. Eligibility criteria for participants	170 (96.6)
	4b. Settings of data collection	16 (9.1)
5. Interventions	Interventions intended for each group	130 (73.9)
6. Objective	Specific objective or hypothesis	170 (96.6)
7. Outcome*	Clearly defined primary outcome for this report	28 (15.9)
8. Randomisation	How participants were allocated to interventions	0 (0)
	8a. Random assignment	176 (100)
	8b. Sequence generation	0 (0)
	8 c. Allocation concealment	0 (0)
9. Blinding (masking)	Whether or not participants, caregivers and those assessing the outcomes were blinded	14 (8.0)
	9a. Generic description only (eg, single blind, double blind)	45 (25.6)
10. Numbers randomised	No of participants randomised to each group	79 (44.9)
11. Recruitment	Trial status (eg, ongoing, closed to recruitment, closed to follow-up)	41 (23.3)
12. Numbers analysed	No of participants analysed in each group	24 (13.6)
	12a. Intention-to-treat analysis or per-protocol analysis	12 (6.8)
13. Outcome†	For the primary outcome, a result for each group and the estimated effect size and its precision	0 (0)
	13a. Primary outcome result for each group	11 (6.3)
	13b. Estimated effect size	5 (2.8)
	13 c. Precision of the estimate (eg, 95% CI)	3 (1.7)
14. Harms	Important adverse events or side effects	22 (12.5)
15. Conclusions	General interpretation of the results	172 (97.7)
	15a. Benefits and harms balanced	10 (5.7)
16. Trial registration	Registration no and name of trial register	6 (3.4)
17. Funding	Source of funding	162 (92.0)

*Outcome reported in Methods section.

†Outcome reported in Results section.

CONSORT, Consolidated Standards of Reporting Trials.

Reporting of trial methodology

Most abstracts adequately reported the CONSORT items objective (96.6%) and interventions (73.9%). However, none of the included abstracts described the methods used for sequence generation and allocation concealment. Therefore, no abstract reported the item randomisation adequately. In terms of information regarding trial participants, although eligible criteria were reported in 96.6% of included abstracts, only 9.1% described the settings in which participants were studied. Besides, only 28 abstracts (15.9%) clearly defined the primary outcome of the trial. Forty-five abstracts (25.6%) provided information about blinding, but only 14 abstracts (8.0%) specified who (eg, participants, caregivers and outcome assessors) were blinded.

Reporting of trial results

Seventy-nine RCT abstracts (44.9%) reported the number of participants randomised to each group, but only 24 abstracts (13.6%) reported the number of participants analysed in each group. Only 6.8% of included abstracts stated the adoption of intention-to-treat or per-protocol analysis. Although 28 abstracts (15.9%) defined the primary outcome in their Methods sections, no abstract provided all the details required for the primary outcome in the Results section, including result for each group, the estimated effect size and its precision. Additionally, adverse events or side effects were reported by only 12.5% of the included abstracts. Forty-one abstracts (23.3%) stated the trial status (eg, ongoing, preliminary analysis and interim analysis).

Reporting of trial conclusions

Almost all included abstracts (97.7%) adequately reported conclusions that were consistent with the trial results. However, only 10 RCT abstracts (5.7%) balanced the benefits and harms of interventions in their conclusions.

OQS and associated factors

The mean OQS of the included 176 abstracts was 5.53 (SD 0.12; 95% CI 5.30 to 5.76). Table 3 demonstrates the results of linear regression analyses. In univariable analyses, four factors were significantly associated with better reporting quality of included abstracts: more authors (p=0.019), pharmacological interventions (p<0.001), research regarding SBD (p=0.004) and funding from the industry (p<0.001).

Table 3.

Linear regression-derived coefficients (B) and 95% CIs, with overall quality score as the dependent variable for included abstracts

Predictor	Category/unit	Univariable			Multivariable
		B	95% CI	P value	B	95% CI	Tolerance	VIF	P value
Year	1 year	0.07	(−0.22 to 0.36)	0.632
No of authors	1 person	0.10	(0.02 to 0.18)	0.019	0.05	(−0.03 to 0.13)	0.87	1.15	0.216
Continent	North America	Reference
	Europe	0.74	(−0.08 to1.56)	0.078
	Oceania	0.54	(−0.28 to 1.36)	0.197
	Asia	0.73	(−0.37 to 1.83)	0.191
	South America	−0.39	(−2.16 to 1.37)	0.661
Type of Intervention	Pharmacological	Reference			Reference
	Non-pharmacological	−1.02	(−1.53 to -0.51)	<0.001	−0.70	(−1.27 to to -0.12)	0.73	1.37	0.018
No of centres	Single centre	Reference
	Multicentre	0.45	(−0.35 to 1.25)	0.267
Word count	1 word	0.00	(−0.01 to 0.01)	0.923
Research area	Insomnia	Reference			Reference
	Sleep-related breathing disorders	0.78	(0.25 to 1.31)	0.004	0.42	(−0.13 to 0.97)	0.77	1.30	0.135
	Others	0.42	(−0.19 to 1.03)	0.179	−0.17	(−0.80 to 0.47)	0.77	1.30	0.599
Sample size	1 participant	−0.001	(−0.00 to 0.00)	0.139
Funded by industry	Yes	Reference			Reference
	No	−1.08	(−1.58 to -0.58)	<0.001	−0.66	(−1.24 to -0.08)	0.72	1.40	0.025

For multivariable analysis, constant=6.106, R²=0.153, adjusted R²=0.128, p<0.001.

VIF, variance inflation factor.

Bold values are those indicating statistical significance.

All these four predictors were entered into a multivariable model (p<0.001; R²=0.153, adjusted R²=0.128; constant=6.106). It was indicated that both pharmacological intervention (p=0.018) and funding from the industry (p=0.025) remained significant predictors of greater OQS. However, the low R² (15.3%) and adjusted R² (12.8%) values of the final model suggested that the influence of other factors not included in this model was likely to exist.

Reporting quality by type of intervention

According to the calculated RRs, five CONSORT items were reported significantly better in abstracts describing pharmacological interventions than those describing non-pharmacological interventions: title (RR 0.53; 95% CI 0.34 to 0.83), interventions (RR 0.78; 95% CI 0.67 to 0.91), objective (RR 0.95; 95% CI 0.92 to 0.99), outcome in methods section (RR 0.44; 95% CI 0.23 to 0.87) and harms (RR 0.10; 95% CI 0.04 to 0.25) (table 4).

Table 4.

Reporting quality of each CONSORT checklist item and subitem by type of intervention

Items	Criteria and subitems	Pharmacological (n=44), n (%)	Non-pharmacological (n=132), n (%)	Risk ratio (95% CI)
1. Title	Identification of the study as randomised	20 (45.5)	32 (24.2)	0.53 (0.34 to 0.83)
2. Authors	Contact details for the corresponding author	0 (0)	0 (0)	NE
3. Trial design	Description of the trial design (eg, parallel, cross-over)	19 (43.2)	38 (28.8)	0.66 (0.43 to 1.03)
4. Participant	Eligibility criteria for participants and the settings where the data were collected	2 (4.5)	14 (10.6)	2.33 (0.55 to 9.87)
	4a. Eligibility criteria for participants	40 (90.9)	130 (98.5)	1.08 (0.98 to 1.19)
	4b. Settings of data collection	2 (4.5)	14 (10.6)	2.33 (0.55 to 9.87)
5. Interventions	Interventions intended for each group	39 (88.6)	91 (68.9)	0.78 (0.67 to 0.91)
6. Objective	Specific objective or hypothesis	44 (100)	126 (95.5)	0.95 (0.92 to 0.99)
7. Outcome*	Clearly defined primary outcome for this report	12 (27.3)	16 (12.1)	0.44 (0.23 to 0.87)
8. Randomisation	How participants were allocated to interventions	0 (0)	0 (0)	NE
	8a. Random assignment	44 (100)	132 (100)	1.00 (1.00 to 1.00)
	8b. Sequence generation	0 (0)	0 (0)	NE
	8 c. Allocation concealment	0 (0)	0 (0)	NE
9. Blinding (masking)	Whether or not participants, caregivers, and those assessing the outcomes were blinded	5 (11.4)	9 (6.8)	0.60 (0.21 to 1.70)
	9a. Generic description only (eg, single blind, double blind)	27 (61.4)	18 (13.6)	0.22 (0.14 to 0.36)
10. Numbers randomised	No of participants randomised to each group	17 (38.6)	62 (47.0)	1.22 (0.80 to 1.84)
11. Recruitment	Trial status (eg, ongoing, closed to recruitment, closed to follow-up)	12 (27.3)	29 (22.0)	0.81 (0.45 to 1.44)
12. Numbers analysed	No of participants analysed in each group	8 (18.2)	16 (12.1)	0.67 (0.31 to 1.45)
	12a. Intention-to-treat analysis or per-protocol analysis	2 (4.5)	10 (7.6)	1.67 (0.38 to 7.32)
13. Outcome†	For the primary outcome, a result for each group and the estimated effect size and its precision	0 (0)	0 (0)	NE
	13a. Primary outcome result for each group	4 (9.1)	7 (5.3)	0.58 (0.18 to 1.90)
	13b. Estimated effect size	1 (2.3)	4 (3.0)	1.33 (0.15 to 11.61)
	13c. Precision of the estimate (eg, 95% CI)	1 (2.3)	2 (1.5)	0.67 (0.06 to 7.17)
14. Harms	Important adverse events or side effects	17 (38.6)	5 (3.8)	0.10 (0.04 to 0.25)
15. Conclusions	General interpretation of the results	40 (90.9)	132 (100)	1.10 (1.00 to 1.21)
	15a. Benefits and harms balanced	8 (18.2)	2 (1.5)	0.08 (0.02 to 0.38)
16. Trial registration	Registration no and name of trial register	2 (4.5)	4 (3.0)	0.67 (0.13 to 3.52)
17. Funding	Source of funding	40 (90.9)	122 (92.4)	1.02 (0.91 to 1.13)

*Outcome reported in Methods section.

†Outcome reported in Results section.

CONSORT, Consolidated Standards of Reporting Trials; NE, not estimable due to zero cell counts.

Bold values are those indicating statistical significance.

Reporting quality by funding status

The adequate reporting rates of six CONSORT items were significantly higher in abstracts funded by industry: title (RR 0.41; 95% CI 0.27 to 0.63), trial design (RR 0.56; 95% CI 0.37 to 0.85), outcome in methods (RR 0.47; 95% CI 0.24 to 0.92), blinding (RR 0.27; 95% CI 0.10 to 0.72), harms (RR 0.17; 95% CI 0.07 to 0.38) and funding (RR 0.89; 95% CI 0.84 to 0.95) (table 5).

Table 5.

Reporting quality of each CONSORT checklist item and subitem by funding status

Items	Criteria and subitems	Funded by industry (n=46), n (%)	Not funded by industry/unreported (n=130), n (%)	Risk ratio (95% CI)
1. Title	Identification of the study as randomised	24 (52.2)	28 (21.5)	0.41 (0.27 to 0.63)
2. Authors	Contact details for the corresponding author	0 (0)	0 (0)	NE
3. Trial design	Description of the trial design (eg, parallel, crossover)	22 (47.8)	35 (26.9)	0.56 (0.37 to 0.85)
4. Participant	Eligibility criteria for participants and the settings where the data were collected	3 (6.5)	13 (10.0)	1.53 (0.46 to 5.14)
	4a. Eligibility criteria for participants	43 (93.5)	127 (97.7)	1.05 (0.96 to 1.13)
	4b. Settings of data collection	3 (6.5)	13 (10.0)	1.53 (0.46 to 5.14)
5. Interventions	Interventions intended for each group	38 (82.6)	92 (70.8)	0.86 (0.72 to 1.02)
6. Objective	Specific objective or hypothesis	45 (97.8)	125 (96.2)	0.98 (0.93 to 1.04)
7. Outcome*	Clearly defined primary outcome for this report	12 (26.1)	16 (12.3)	0.47 (0.24 to 0.92)
8. Randomisation	How participants were allocated to interventions	0 (0)	0 (0)	NE
	8a. Random assignment	46 (100)	130 (100)	NE
	8b. Sequence generation	0 (0)	0 (0)	NE
	8 c. Allocation concealment	0 (0)	0 (0)	NE
9. Blinding (masking)	Whether or not participants, caregivers, and those assessing the outcomes were blinded	8 (17.4)	6 (4.6)	0.27 (0.10 to 0.72)
	9a. Generic description only (eg, single blind, double blind)	25 (54.3)	20 (15.4)	0.28 (0.17 to 0.46)
10. Numbers randomised	No of participants randomised to each group	16 (34.8)	63 (48.5)	1.39 (0.90 to 2.15)
11. Recruitment	Trial status (eg, ongoing, closed to recruitment, closed to follow-up)	11 (23.9)	30 (23.1)	0.97 (0.53 to 1.77)
12. Numbers analysed	No of participants analysed in each group	6 (13.0)	18 (13.8)	1.06 (0.45 to 2.51)
	12a. Intention-to-treat analysis or per-protocol analysis	2 (4.3)	10 (7.7)	1.77 (0.40 to 7.78)
13. Outcome†	For the primary outcome, a result for each group and the estimated effect size and its precision	0 (0)	0 (0)	NE
	13a. Primary outcome result for each group	4 (8.7)	7 (5.4)	0.62 (0.19 to 2.02)
	13b. Estimated effect size	1 (2.2)	4 (3.1)	1.42 (0.16 to 12.34)
	13c. Precision of the estimate (eg, 95% CI)	1 (2.2)	2 (1.5)	0.71 (0.07 to 7.62)
14. Harms	Important adverse events or side effects	15 (32.6)	7 (5.4)	0.17 (0.07 to 0.38)
15. Conclusions	General interpretation of the results	43 (93.5)	129 (99.2)	1.06 (0.98 to 1.15)
	15a. Benefits and harms balanced	8 (17.4)	2 (1.5)	0.09 (0.02 to 0.40)
16. Trial registration	Registration no and name of trial register	2 (4.3)	4 (3.1)	0.71 (0.13 to 3.74)
17. Funding	Source of funding	46 (100)	116 (89.2)	0.89 (0.84 to 0.95)

*Outcome reported in Methods section.

†Outcome reported in Results section.

CONSORT, Consolidated Standards of Reporting Trials; NE, Not estimable due to zero cell counts.

Bold values are those indicating statistical significance.

Discussion

Our study identified and evaluated 176 RCT abstracts presented at the 2016–2018 SLEEP Annual Meetings, whose overall reporting quality turned out to be suboptimal. Of the 17 CONSORT quality items, only three items (objective, conclusions and funding) were adequately reported in most abstracts (>75%). This pattern of reporting is slightly different from the findings of several previous studies regarding conference abstracts,^{9 18} in which intervention, objective and conclusions were well reported but the reporting of funding was poor. Such differences might result from the required structured format of SLEEP Meeting abstracts, which included the headings of ‘Introduction, Methods, Results, Conclusion and Support (if any)’. Almost all of the included SLEEP Meeting abstracts had a ‘Support’ section where authors reported the sources of financial support, which clearly demonstrated that instructions for authors can play an important role in abstracts reporting quality.

In the present study, none of the included abstracts provided sufficient details for corresponding author, randomisation and outcome in results section. Less than 10% of the included abstracts adequately presented participant, blinding and trial registration. For these items, the findings of two previous studies in dentistry⁹ and gerontology¹⁰ were generally in line with ours, which indicates that the inadequate reporting of corresponding author, participant, randomisation, outcome in results section and trial registration could be universal across medical specialties. However, compared with our results, similar studies in sports injury,¹⁹ burn²⁰ and endourology²¹ found greater adequate reporting rate in participant, randomisation and outcome in results section. Such discrepancies might be attributed to different levels of awareness and usage of the CONSORT and CONSORT for Abstracts guidelines in different medical fields.

The importance of titles in the identification of RCTs has been shown in the Cochrane Highly Sensitive Search Strategy. When carrying out a systematic review, researchers need to search controlled vocabulary (eg, Medical Subject Headings) and free-text terms related to the word ‘random’ and specific trial designs (eg, parallel, cross-over) to retrieve RCTs.²² Furthermore, considering that more than half of conference abstracts do not reach full publication,^{5 6} the corresponding studies can only be searched by title and abstracts. Therefore, it is important for conference abstracts to mention randomisation in an explicit way, to ensure correct indexing and identification through common search strategies. In the index section of abstract books of the 2016–2018 SLEEP Meetings, only nine abstracts were labelled as RCTs. However, as shown in our study, there were actually 176 RCT abstracts presented during these meetings. This may be related to the fact that only 30% of the included abstracts mentioned randomisation in their titles, and such lack of identifiability could hinder the dissemination of sleep medicine trials in conferences as well as their translation into clinical practice.

Conference abstracts are more likely to report preliminary or interim analyses, and the number of participants randomised and analysed is likely to change during the course of a trial.²³ Thus, the reporting of trial status is important for readers to critically appraise a trial. In addition, if a reader or systematic reviewer needs additional information about a trial, contacting the corresponding author or searching registration website could be the only viable methods to obtain the information. However, in our study, less than one-fourth provided details about recruitment, less than 4% of the abstracts reported authors and trial registration adequately.

It is impossible for a reviewer to appraise a study comprehensively if related information has not been provided. One previous study demonstrated that inadequate allocation concealment may lead to larger estimates of treatment effects.²⁴ In our study, no abstract provided details about sequence generation and allocation concealment. It is, therefore, impossible to determine for the included abstracts whether there were any systematic differences between study groups at baseline (selection bias).

According to the Cochrane handbook,²⁵ the main cause of performance bias was inadequate blinding of study participants and personnel, lack of sufficient blinding of outcome assessors may lead to detection bias. Thus, it is important to provide explicit details about blinding in the RCT. However, among the included abstracts of this study, only 8% of abstracts contained explicit description of blinding. Thirty-one abstracts used the terms ‘single blind’ or ‘double blind’, without specifying whether participants, caregivers or those assessing the outcomes were blinded. Such reporting deficiencies may be confusing for readers and systematic reviewers, and should therefore be avoided.

Large and unbalanced discrepancies in the number of participants randomised and the number analysed may be a hint of attrition bias. If the prespecified primary outcome was not reported, the trial might have a high risk of selective reporting bias.²⁶ However, in the present study, only 14% of the included abstracts reported the number of participants analysed in each group, only 16% clearly defined their primary outcome measure. In other words, about 85% of the included abstracts did not provide enough information for a quick assessment of their attrition and selective reporting bias.

The applicability of findings of an abstract depends on explicit description of eligibility criteria and the settings of data collection. Although 97% of abstracts adequately reported eligible criteria for participants, less than 10% reported their study settings. This makes it difficult for readers to assess the external validity of a trial and to determine its applicability to their own setting.⁸ Besides, less than 3% of included abstracts reported estimated effect size and precision of the estimate (eg, 95% CI) for the specified primary outcome. Although p values can provide information about statistical significance, full evaluation of clinical interventions needs to take into account the effect size and precision.²⁷

In this study, we explored the association between abstract reporting quality and nine factors. Most of these factors, including date of publication,^{12 28} number of authors,²⁹ type of intervention,²⁹ number of centres,²⁸ word count,^{9 30} research area¹⁸ and funding status,³¹ had been previously identified as significant predictors of RCT abstract reporting quality. However, in our study, only two factors remained significant predictors in the multivariable analysis. First, abstracts of pharmacological trials were better reported than non-pharmacological trials. This is generally in keeping with the finding of several previous studied regarding the reporting of trial abstracts^{28 31 32} and full articles,²⁷ but was in contradiction with a recent study regarding abstracts of HIV/AIDS-related RCTs.²⁹ Second, funding from the industry was associated with better abstract reporting. A recent study on RCT abstracts in psychiatry observed the same association,³¹ while in some other studies source of funding was not a significant predictor of abstract reporting quality.³² To what extent can the above-mentioned differences be attributed to subject variations is still unknown. Further studies from the field of sleep medicine are needed to confirm our results and explore the relevant causes and implications.

The present study is not without limitations. First, our final model of multivariable regression can only explain 12.8%–15.3% of the variation of OQS. Other factors, such as involvement of statisticians, awareness of CONSORT for Abstracts among the reviewers might also influence abstract reporting quality.^{33 34} However, we were not able to obtain such information from SLEEP Meeting abstracts books. Second, the quality assessment was carried out by only one author. However, similar to previous studies in this area,¹⁵ before formal assessment a pilot study was carried out to calibrate the assessor (QS) and an expert (FH).^{9 12 35} In addition, all discrepancies and uncertainties in the assessment process were resolved through a discussion with two experts (FH and HH). Third, only one major conference was included in this study, therefore, our findings may not be applicable to other academic conferences in the field of sleep medicine. Nevertheless, to our knowledge, this was the first study to assess the reporting quality of RCT abstracts and to explore relevant predictors in the field of sleep medicine. Our findings should be helpful in reducing relevant avoidable research waste,² and facilitate further studies into the reporting of sleep medicine research.

The SLEEP Annual Meeting is a major platform for disseminating the latest research in sleep medicine. Authors of meeting abstracts should take the responsibility of reporting their research adequately. In addition, we recommend that the organising committees of conferences in this field endorse the CONSORT for Abstracts guidelines in their online instructions for authors. Active implementation of the CONSORT for Abstracts guidelines and the adoption of a highly structured format could lead to significant improvements in the reporting of RCT abstracts.^{35 36}

In summary, the reporting quality of RCT abstracts presented at SLEEP Annual Meetings was suboptimal. Pharmacological intervention and industry funding were found to be significantly associated with better reporting quality. There is a need for efforts by authors and conference committees to improve the reporting of RCT abstracts presented at conferences in sleep medicine.