Skip to main content
NCBI home page
Contemporary Clinical Trials Communications logoLink to Contemporary Clinical Trials Communications
. 2022 May 25;28:100924. doi: 10.1016/j.conctc.2022.100924

Outcome reporting bias in nephrology randomized clinical trials: Examining outcomes represented by graphical illustrations

Frank Ward a,∗, Frances Shiely b,c
PMCID: PMC9160318  PMID: 35664503

Abstract

Background

Outcome reporting bias (ORB) is widely reported in the medical literature, but the contribution from published graphical illustrations is unknown. The aim of this study was to investigate the occurrence of ORB in contemporary nephrology clinical trials relating to the choice of outcomes reported through graphical illustrations.

Methods

An observational study was conducted using nephrology clinical trials searched from five high-impact medical journals from 2015 to 2020. Eligible trials reported a phase 2, 3 or 4 trial, contained at least one published outcome graphical illustration and were registered on a clinical trial registry. The primary outcome was the occurrence of ORB based on the choice of graphical illustrations in published trial manuscripts, deemed to be present if a graphical illustration displayed a secondary or unregistered outcome ahead of a trial's primary outcome, or if any unregistered trial outcome was presented as a graphical illustration.

Results

In 75 eligible clinical trials, the primary outcome for ORB was present in 60% of the trials (n = 45). Occurrence of the primary outcome did not differ significantly based on trial sample size, funding model, trial phase, individual medical journal or publication year. An unregistered trial outcome was graphically illustrated in 93% (n = 42) of those clinical trials with ORB present.

Conclusion

Outcome reporting bias based on the choice of graphical illustration is common, driven primarily by graphical illustration of unregistered trial outcomes. More appropriate choice of outcomes for graphical illustrations by authors, coupled with both increased enforcement of CONSORT guidelines by medical journals and specific guidelines for graphical illustrations choice, are desirable to address these findings.

Keywords: Outcome reporting bias, Graphical illustrations, Randomized clinical trials, Clinical trial registration

Highlights

  • What is already known on this topic:
    • •
      Although outcome reporting bias in commonly reported in the medical literature, the contribution from published graphical illustrations has not been previously examined.
    • •
      This study will highlight the role of graphical illustrations in the occurrence of outcome reporting bias, which is particularly relevant in the modern era of increasing data visualization.
  • What this study adds:
    • •
      The choice of published graphical illustration in contemporary nephrology clinical trial manuscripts is a common source of outcome reporting bias, primarily through illustration of unregistered trial outcomes.

Abbreviations

ORB

outcome reporting bias

RCTs

randomized controlled trial

PO

primary outcome

SO

secondary outcome

UO

unspecified outcome

CONSORT

Consolidated Standards of Reporting Trials

SPIRIT

Standard Protocol Items: Recommendations for Interventional Trials

PRISM-P

Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols

IQR

inter-quartile range

InsPECT

Instrument for the reporting of Planned Endpoints in Clinical Trials

1. Introduction

Outcome reporting bias (ORB) is widely reported in the medical literature [[1], [2], [3]]. Potentially leading to bias in the interpretation of trial results, ORB can misinform clinical decision-making and ultimately affect patient outcomes. There are several forms of ORB, including selective outcome reporting, reporting of unregistered outcomes, altering the assessment period of outcomes or changing the hierarchy of primary and secondary outcomes [3,4]. Even in high-impact journals, only a small minority of clinical trials are truly faithful in reporting all registered outcomes, with a high prevalence of omitting outcomes or reporting de novo unregistered outcomes [5].

Several studies of ORB have focused on mapping the outcomes in the published manuscript text to those pre-specified at trial registration [[6], [7], [8]]. The extent to which the choice of graphical illustrations (figures and graphs) might represent ORB has not been previously reported. The CONSORT guidelines recommend that each clinical trial primary and secondary outcome be reported with an estimate effect size and its precision [9]. However, the manner of portrayal of outcomes, including the use of graphical illustrations, remains at the authors’ discretion, and there is a distinct paucity of trial methodological research to guide researchers. The limited evidence available suggests that, although graphical illustrations are very commonly utilized in medical journals, the choice, content and configuration of these visual aids are often sub-optimal [[10], [11], [12], [13]].

Historical research has demonstrated that visual data, compared to text alone, was associated with superior reader comprehension and persuasion [14]. Miller and Barnett reported that a combination of visual and text reporting was superior to text alone regarding reader comprehension [15]. Marketing research claims that visual data is highly efficacious at attracting reader attention, increasing reader understanding and retention, and increasing circulation of data on digital platforms, although rigorous research methodology is generally not evident to support these claims [16,17]. Overall, graphical illustrations represent a potentially impactful method of conveying research outcomes, and hence could be an important source of ORB.

The aim of this study was to investigate whether ORB was evident in the graphical illustrations of contemporary nephrology randomized control trials (RCTs), when compared to the registered outcomes on trial registries. This study will give clinical researchers in all disciplines an insight into the current role of graphical illustrations in portraying trial outcomes, and the degree to which they contribute to ORB.

2. Methodology

2.1. Study design

A retrospective review of RCTs published in the New England Journal of Medicine (NEJM), the Lancet and the three leading clinical nephrology journals by impact factor rating (the Journal of the American Society of Nephrology, Kidney International and the Clinical Journal of the American Society of Nephrology) was conducted. In order to represent the contemporary era of trial reporting, the study included trials published in a 6-year period from Jan 1st, 2015 through Dec 31st, 2020. The STROBE reporting guidelines for observational studies were adhered to. Ethical approval was not required for the study.

2.2. Search strategy and trial eligibility

Using PubMed, all RCTs published relating to nephrology practice from 2015 to 2020 were identified through the following search strategy: ((“randomized clinical trial"[All Fields] AND “hypertension"[All Fields]) OR “diabetic kidney disease"[All Fields] OR “renal replacement therapies"[All Fields] OR “kidney stone"[All Fields] OR “vasculitis"[All Fields] OR “glomerulonephritis"[All Fields] OR (“hemodialysis"[All Fields] OR “renal dialysis"[All Fields] OR (“renal"[All Fields] AND “dialysis"[All Fields]) OR “renal dialysis"[All Fields] OR “hemodialysis"[All Fields]) OR “peritoneal dialysis"[All Fields] OR “kidney transplantation"[All Fields]) AND ((fha[Filter]) AND (randomized controlled trial[Filter]) AND (2015:2020[pdat])).

Inclusion criteria were: manuscripts reporting a single phase 2, 3 or 4 trial; outcomes reported for chronic kidney disease, acute kidney injury, nephrolithiasis, glomerular disease, hypertension, dialysis therapies or transplantation; trials that contained at least one outcome illustration (graph or figure) in the published manuscript; and registration with a clinical trial registry. Exclusion criteria were: RCTs that presented secondary analyses of previously reported trials without a new trial registration; studies that presented post-hoc analyses of previously published trials; and studies that lacked a designation of trial phase in their registration.

2.3. Study outcomes

The primary outcome of this study was the presence of ORB, which was deemed to be present if either a graphical illustration was used in a clinical trial to present a secondary outcome (SO) or previously unregistered outcome (UO) ahead of the trial's primary outcome (PO), or if any UO was graphical illustrated in a trial manuscript.

The secondary outcomes of this study were concerning the presentation of clinical trial outcomes in the text of its published abstract. The outcomes assessed were: the number and proportion of a clinical trial registered POs published in the abstract; the number and proportion of a clinical trial registered SOs published in the abstract; and whether any UOs were published in a clinical trial's abstract.

2.4. Data extraction

Eligible trials were entered into the citation manager Zotero. Trial abstracts and clinical registration status was screened to ensure suitability for inclusion. Trials excluded were recorded, along with reason for exclusion, and presented in the study flow diagram (Fig. 1).

Fig. 1.

Fig. 1

Open in a new tab

Study inclusion flow diagram.

Full text review of suitable manuscripts was conducted for extraction of the publication year, randomized sample size, study phase, funding model of the trial (public vs industry) and the number and nature (primary, secondary or unregistered) of outcomes published in graphical illustrations and in the abstract. The clinical trial registration database pertaining to each trial was reviewed to identify the number and list of pre-specified primary outcomes and secondary outcomes. Outcomes labelled as “other” in registration were considered to be secondary outcomes for the purposes of this study.

2.5. Analysis plan

For each included trial, the outcome graphical illustrations were mapped to the registered study outcomes, and categorized as primary, secondary, unregistered or a combination of these outcomes. Graphical illustrations pertaining to study flow diagrams or study operational diagrams, such as graphs demonstrating separation between intervention and control groups regarding the intervention, were not included in the analysis. Adverse events reported by graphical illustrations were included in the analysis. Outcome reporting bias that favoured the reporting of statistically significant results was recorded, adjudged to have occurred if a statistically significant SO or UO was graphically illustrated ahead of a PO, or if any statistically significant UO was graphically illustrated.

Categorical data were presented as absolute numbers and percentage of total, and compared using the Chi-square test (χ2). Continuous data were presented as median values with interquartile range, and compared by student's t-test. Subgroup analyses were conducted regarding the prevalence of outcome reporting bias in relation to the source of funding of the trial, the year of the study, the phase of the RCT and the number of pre-registered outcomes. An alpha level of 0.05 was considered statistically significant for this study.

3. Results

The search strategy revealed 2,056 clinical trials, of which 75 were suitable for inclusion (Fig. 1). The median sample size was 162 participants (IQR 99–417), with industry funding reported in 55% of the clinical trials (n = 41) and 42% designated as phase 3 clinical trials (n = 31) (Table 1). The majority of trials had a single registered primary outcome (77%, n = 58). Fifty-two percent of the trials (n = 39) had >4 registered secondary outcomes. Publication of two outcome graphical illustrations in a trial manuscript was the most frequent practice, occurring in 39% (n = 29) of trials, followed by 23% of trials having either one (n = 17) or three (n = 17) published.

Table 1.

Characteristics of Included Clinical Trials, stratified by Journal. Reported number (n) represent the number of clinical trials in each category.

NEJM Lancet KI JASN CJASN Combined
No. of RCTs 12 10 11 28 14 75
Sample Size
(median, IQR)		 454
(158–1459)		 185
(112–1777)		 118
(106–265)		 198
(54–299)		 110.5
(59–312)		 162
(99–417)
Publication Year
2015
2016
2017
2018
2019
2020		 1
–
1
1
5
4		 1
1
1
4
2
1		 1
1
–
5
4
–		 5
3
8
4
5
3		 3
3
1
4
3
–		 11
8
11
18
19
8
Funding
Public
Industry		 6
6		 3
7		 5
6		 12
16		 8
6		 34
41
Trial Phase
2
3
4		 –
10
2		 4
4
2		 6
4
1		 9
10
9		 8
3
3		 27
31
17
Registered Primary Outcomes
1
2
≥3		 8
4
–		 9
–
1		 10
–
1		 23
4
1		 8
3
3		 58
11
6
Registered Secondary Outcomes
≤4
5–8
≥9		 3
4
5		 5
3
2		 6
1
4		 16
7
5		 6
5
3		 36
20
19
No. of Published Outcome Graphical Illustrations
1
2
3
4
5		 6
5
1
–
–		 2
5
2
1
–		 2
4
2
3
–		 5
10
8
3
2		 2
5
4
2
1		 17
29
17
9
3
Open in a new tab

Abbreviations: RCT = randomized clinical trials, NEJM = New England Journal of Medicine, KI = Kidney International, JASN = Journal of the American Society of Nephrology, CJASN = Clinical Journal of the American Society of Nephrology.

Mapping of the graphical illustrations to the registered trial outcomes revealed that the first outcome graphical illustration was used to present a clinical trial PO in 47% of cases (n = 35) and a combination of a PO/SO, a PO/UO and a PO/SO/UO in 21% (n = 16), 8% (n = 6) and 8% (n = 6), respectively (Table 2). The first outcome graphical illustration was used to portray results that did not include a PO in 16% of trials (n = 12), a finding which was not significantly associated with any one specific journal (χ2 = 2.2, p = 0.68), trial phase (χ2 = 2.4, p = 0.29), funding model (χ2 = 0.7, p = 0.39), sample size (t = −1.4, p = 0.16) or the number of registered secondary outcomes (t = −1.07, p = 0.28). A clinical trial PO was not displayed in any graphical illustration in 7% of the clinical trials (n = 5), and 40% of these unpresented POs (n = 2) were not statistically significant. In trials that included a second published graphical illustrations (n = 58), the outcomes portrayed were predominantly a SO (n = 18, 31%) or an UO (n = 16, 28%).

Table 2.

Distribution of reported trial outcomes by order of published graphical illustration.

Order of Published Graphical Illustration First
n (%)		 Second
n (%)		 Third
n (%)		 Fourth
n (%)		 Fifth
n (%)
Eligible RCTs 75 58 29 12 3
Trial Outcome Reported 35 (47) 11 (19) 3 (10) – –
PO 4 (5) 18 (31) 12 (41) 7 (58) 1 (33)
SO 6 (8) 16 (28) 13 (45) 4 (33) 1 (33)
UO 16 (21) 6 (10) – – –
Combination of PO/SO 6 (8) 5 (9) 1 (4) 1 (9) 1 (33)
Combination of PO/UO 2 (3) 2 (3) – – –
Combination of SO/UO 6 (8) – – – –
Combination of PO/SO/UO
Open in a new tab

Abbreviations: RCTs = randomized clinical trials; PO = primary outcome, SO = secondary outcome, UO = unregistered outcome.

n = number of RCTs.

3.1. Primary outcome

Outcome reporting bias based on the choice of graphical illustration was present in 60% of the clinical trials (n = 45) (Fig. 2). In these 45 clinical trials, a SO or UO had been graphically illustrated prior to the PO in 16% (n = 7) of the trials, while an UO had been graphically illustrated in 96% (n = 43). Both of these criteria were present in 11% (n = 5) of the trials.

Fig. 2.

Fig. 2

Open in a new tab

Primary outcome occurrence in eligible RCTs, by individual medical journal and the combined cohort of RCTs.

The occurrence of ORB did not differ significantly between the journals (χ2 = 6.1, p = 0.14), with year of publication (χ2 = 5.2, p = 0.39) or with the trial sample size (t = −0.59, p = 0.55). Outcome reporting bias occurred in 54% of industry-funded trials (n = 22) compared to 53% of public-funded trials (χ2 = 1.29, p = 0.25), and in 63% (n = 17), 58% (n = 18) and 58% (n = 10) of phase 2, 3 and 4 trials, respectively, which was not statistically significantly different (χ2 = 0.15, p = 0.92). In the 45 clinical trials with ORB bias present, the offending graphical illustrations represented a statistically significant or positive findings regarding the intervention in 82% of cases (n = 37). Publication of a statistically significant versus non-significant UO by graphical illustration was not associated with trial sample size (t = −0.9, p = 0.36), the number of registered secondary outcomes (t = −1.1, p = 0.23), the trial phase (χ2 = 0.19, p = 0.9) or the funding model (χ2 = 0.2, p = 0.64).

3.2. Secondary outcomes

All clinical trials reported at least one PO in their abstract. Ninety-two percent (n = 69) of trials reported all registered POs in their abstract. All registered SOs were reported in 32% (n = 24) of the trial abstracts, whereas 24% (n = 18) of trials did not report any SOs in their abstract. The median number of SOs reported in the abstracts was 2 (IQR 1–3). An UO was reported in 63% (n = 47) of trial abstracts. This did not differ significantly between industry or publicly funded clinical trials (χ2 = 0.65, p = 0.41). Of those trials which reported an UO in their abstract, the reported UO was a statistically significant or positive result towards the trial intervention in 64% of occurrences (n = 30).

4. Discussion

In contemporary nephrology RCTs there was a high prevalence of ORB based on the choice of outcome graphical illustrations. This primarily resulted from the publication of unregistered outcomes through graphical illustrations. The occurrence of ORB was ubiquitous throughout the trials, and was not found to significantly differ based on the trial funding model, the trial phase, the randomized sample size or between individual journals. In a large majority of cases, the graphical illustrations leading to ORB represented outcomes that were statistically significant or positive results relating to the trial intervention.

Previous research has shown a similar high prevalence of ORB in RCTs, but no previous study has focused on the contribution of graphical illustrations to ORB [[6], [7], [8],[18], [19], [20]]. This is very surprising, given the key role that data visualization has assumed in the medical literature since the first principles were historically presented by authors such as Tufte and Cleveland, and particularly with the advent and growth of social media and electronic publishing [21,22]. There are many reasons why graphical illustration represent an important source of ORB. The human brain is designed to rapidly process visual data and the majority of the population are visual learners [23,24]. Data visualization is more user-friendly, provides superior reader satisfaction compared to text and may be more effective in terms of imparting knowledge [25,26]. The evolution of data visualization relates not only to the diversity of graphical illustrations that modern statistical packages can produce, but also developments such as the growing use of visual abstract infographics [[27], [28], [29]]. Infographics have become ubiquitous with the aid of user-friendly software, and present data in a logical and concise sequence. These visual abstracts capture a reader's attention and can deliver results in a persuasive manner, although their outright superiority over plain text still remains to be proven and is an area of active research [25,30,31]. Infographics can lead to significantly increased dissemination of research on electronic platforms compared to traditional media [32]. The logical assertion is that graphical illustrations can effectively draw and focus a reader on selected outcomes, hence facilitating ORB if used inappropriately. However, there remains no empiric methodological research examining how medical journal readers approach a trial manuscript, or what relative weight of importance are attributed to graphical illustrations compared to the text or abstract sections, which certainly warrants future research.

Despite the pre-requisite for clinical trial registration, there was a high level of unregistered outcomes being graphically illustrated. Since 2005, the International Committee of Medical Journal Editors has required mandatory trial registration, in an attempt to achieve full transparency and high quality in clinical trial reporting [33]. This has been further supported by design guidelines such as SPIRIT and PRISMA-P [34,35]. However, these measures have not proven to be the intended panacea, with non-registration of published trials and ORB remaining commonplace [6,7,[36], [37], [38], [39], [40]]. This inertia has been attributed to a lack of researcher awareness, but stricter enforcement by editorial teams is required, particularly moving away from the publication of un-registered trials or trials registered after outcome collection [[41], [42], [43]]. Despite the CONSORT guidelines requiring full disclosure of outcome measures altered during a trial, there is clear evidence that non-disclosure of changes, as well as other major deviations from CONSORT, have not proven to be a barrier to publication [44,45]. In this study, it is particularly poignant as the included studies were from five high-impact journals in the field of nephrology. In the future, the onus will need to be on the key stakeholders to be more rigorous in their adoption of CONSORT, with one suggestion that using an abbreviated form of the CONSORT checklist at clinical trial submission, followed by a full checklist submission at the time of acceptance, might improve overall compliance with CONSORT [46]. For journals that have endorsed the CONSORT guidelines, appropriate quality control must be in place to prevent low quality publications [[47], [48], [49]]. While the proposed introduction of core outcome sets will define what outcomes should be reported, the upcoming InsPECT/CONSORT-outcome extension will define how those outcomes should be reported [50,51]. Whether there will be any specific recommendation regarding the use of graphical illustrations is unknown, but the group's working documents suggest that authors will have to specifically describe how data will be presented [52]. Overall, through reduced reporting of “unimportant” outcomes, it will be hoped to avoid the lamentable occurrence of clinical trials that fail to translate into any benefits for patients [53].

Outcome reporting bias in this study typically presented statistically significant or “positive” results, which is consistent with previous reports [6,[54], [55], [56]]. The trial abstract, similar to graphical illustrations, is a highly visible component of the manuscript, and hence could be an important source of ORB. In our secondary outcome analysis, we examined the trial abstracts and found that the majority contained un-registered outcomes, with most also representing statistically significant findings when reporting their outcomes. This is consistent with previous studies of abstract reporting bias which have shown poor adherence with the CONSORT abstract extension, although a temporal trend towards improved abstract quality correlating to the extension guideline has been reported [45,[57], [58], [59]]. Combining this propensity for abstract reporting bias with the afore-mentioned use of visual abstract infographics in many of the high impact medicals journals, the trial abstract has the potential to become an even more potent contributor to ORB, unless stakeholders are to become more rigorous in their application of CONSORT.

As a unique study examining the role of graphical illustrations in ORB, these findings highlight a novel consideration for future researchers. The study also begins to address the current paucity of clinical trial methodological evidence regarding the optimal use of graphical illustrations. Although the study is limited in its size and scope, by focusing on contemporary nephrology trials in high impact journals and including only trials which have been registered, the results give a pragmatic insight into the current role of graphical illustrations in outcome reporting bias. The findings were consistent with previous findings relating to ORB in plain text.

5. Conclusion

Our findings suggest that the choice of graphical illustrations is an important source of ORB in contemporary nephrology clinical trials. Despite the increasing importance of graphical illustrations and visual data in clinical trials, there remains an absence of evidence in clinical trial methodological research to guide optimal use. While more rigorous engagement with the CONSORT guidelines by all stakeholders will be critical in the future to eliminate ORB, clear recommendations to direct the choice of published graphical illustration are desirable.

Author competing interest statement

All authors declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work; no other relationships or activities that could appear to have influenced the submitted work.

Funding

This study was not supported by any funding or grant

Transparency declaration

The lead author (FW) affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned (and, if relevant, registered) have been explained.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.

References

  • 1.Chan A.W., Hróbjartsson A., Haahr M.T., Gøtzsche P.C., Altman D.G. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–2465. doi: 10.1001/jama.291.20.2457. [DOI] [PubMed] [Google Scholar]
  • 2.Dwan K., Altman D.G., Arnaiz J.A., Bloom J., Chan A., et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3 doi: 10.1371/journal.pone.0003081. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Chan A.W., Altman D.G. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. Br. Med. J. 2005;330(7494):753. doi: 10.1136/bmj.38356.424606.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Mathieu S., Boutron I., Moher D., Altman D.G., Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–984. doi: 10.1001/jama.2009.1242. [DOI] [PubMed] [Google Scholar]
  • 5.Goldacre B., Drysdale H., Dale A., Milosevic I., Slade E., Hartley P., et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):118. doi: 10.1186/s13063-019-3173-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Howard B., Scott J.T., Blubaugh M., Roepke B., Scheckel C., Vassar M. Systematic review: outcome reporting bias is a problem in high impact factor neurology journals. PLoS One. 2017;12(7) doi: 10.1371/journal.pone.0180986. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Lee S., Khan T., Grindlay D., Karantana A. Registration and outcome-reporting bias in randomized controlled trials of distal radial fracture treatment. JB JS Open Access. 2018 Jul 24;3(3) doi: 10.2106/JBJS.OA.17.00065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Fleming P.S., Koletsi D., Dwan K., Pandis N. Outcome discrepancies and selective reporting: impacting the leading journals? PLoS One. 2015;10(5) doi: 10.1371/journal.pone.0127495. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Schulz K.F., Altman D.G., Moher D., for the CONSORT Group CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010 Mar 23;340:c332. doi: 10.1136/bmj.c332. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Schriger D.L., Sinha R., Schroter S., Liu P., Altman D. From submission to publication: a retrospective review of the tables and figures in a cohort of randomized controlled trials submitted to the British medical journal. Ann. Emerg. Med. 2006;48(6):750–756. doi: 10.1016/j.annemergmed.2006.06.017. [DOI] [PubMed] [Google Scholar]
  • 11.Pocock S.J., Travison T.G. Wruck LM Figures in clinical trial reports: current practice & scope for improvement. Trials. 2007;8:36. doi: 10.1186/1745-6215-8-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Cooper R.J., Schriger D.L., Close R.J. Graphical literacy: the quality of graphs in a large-circulation journal. Ann. Emerg. Med. 2002;40(3):317–322. doi: 10.1067/mem.2002.127327. [DOI] [PubMed] [Google Scholar]
  • 13.Weissgerber T.L., Milic N.M., Winham S.J., Garovic V.D. Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biol. 2015;13(4) doi: 10.1371/journal.pbio.1002128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Vogel D., Dickson W., Lehman J.A. 1986. Persuasion and the Role of Visual Presentation Support: the UM/3M Study.https://www.semanticscholar.org/paper/Persuasion-and-the-Role-of-Visual-Presentation-%3A-UM-Vogel-Dickson/3464f54e0e03859ff1b3c258ca737f01804483c2 [Google Scholar]
  • 15.Miller B.M., Barnett B. Understanding of health risks aided by graphics with text. Newspaper Res. J. 2010;31(1):52–68. [Google Scholar]
  • 16.Text vs visuals – the advantages of each in content marketing. 17/4/2019. https://contentmarketing.com/2016/09/12/text-vs-visuals-the-advantages-of-each-in-content-marketing/
  • 17.12 reasons why you should implement a visual content marketing campaign. 17/4/2019. https://digitalmarketingphilippines.com/12-reasons-why-you-should-implement-a-visual-content-marketing-campaign
  • 18.Vera-Badillo F.E., Napoleone M., Krzyzanowska M.K., Alibhai S.M., Chan A.W., Ocana A., et al. Bias in reporting of randomised clinical trials in oncology. Eur. J. Cancer. 2016;61:29–35. doi: 10.1016/j.ejca.2016.03.066. [DOI] [PubMed] [Google Scholar]
  • 19.Jones P.M., Chow J.T.Y., Arango M.F., Fridfinnson J.A., Gai N., Lam K., et al. Comparison of registered and reported outcomes in randomized clinical trials published in anesthesiology journals. Anesth. Analg. 2017;125(4):1292–1300. doi: 10.1213/ANE.0000000000002272. [DOI] [PubMed] [Google Scholar]
  • 20.Sendyk D.I., Rovai E.S., Souza N.V., Deboni M.C.Z., Pannuti C.M. Selective outcome reporting in randomized clinical trials of dental implants. J. Clin. Periodontol. 2019;46(7):758–765. doi: 10.1111/jcpe.13128. [DOI] [PubMed] [Google Scholar]
  • 21.Tufte E.R. Graphics Press; Cheshire, CT: 1990. Envisioning Information. [Google Scholar]
  • 22.Cleveland W.S. Hobart Press; Summit, NJ: 1993. Visualizing Data. [Google Scholar]
  • 23.Trafton A. In the blink of an eye: MIT neuroscientists find the brain can identify images seen for as little as 13 milliseconds. January 16, 2014. http://news.mit.edu/2014/in-the-blink-of-an-eye-0116 Available:
  • 24.Bradford W.C. Reaching the visual learner: teaching property through art. 2004. https://ssrn.com/abstract=587201 Available.
  • 25.Buljan I., Malički M., Wager E., Puljak L., Hren D., Kellie F., et al. No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials. J. Clin. Epidemiol. 2018;97:86–94. doi: 10.1016/j.jclinepi.2017.12.003. [DOI] [PubMed] [Google Scholar]
  • 26.McCrorie A.D., Chen J.J., Weller R., McGlade K.J., Donnelly C. Trial of infographics in Northern Ireland (TINI): preliminary evaluation and results of a randomized controlled trial comparing infographics with text. Cogent Med. 2018;5(1) doi: 10.1080/2331205X.2018.1483591. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Ibrahim A.M., Lillemoe K.D., Klingensmith M.E., Dimick J.B. Visual abstracts to disseminate research on social media: a prospective, case-control crossover study. Ann. Surg. 2017;266(6):e46–e48. doi: 10.1097/SLA.0000000000002277. [DOI] [PubMed] [Google Scholar]
  • 28.Koo K., Aro T., Pierorazio P.M. Impact of social media visual abstracts on research engagement and dissemination in urology. J. Urol. 2019;202(5):875–877. doi: 10.1097/JU.0000000000000391. [DOI] [PubMed] [Google Scholar]
  • 29.Lindquist L.A., Ramirez-Zohfeld V. Visual abstracts to disseminate geriatrics research through social media. J. Am. Geriatr. Soc. 2019;67(6):1128–1131. doi: 10.1111/jgs.15853. [DOI] [PubMed] [Google Scholar]
  • 30.Martin L.J., Turnquist A., Groot B., et al. Exploring the role of infographics for summarizing medical literature. Health Prof. Educat. 2019;5:48–57. doi: 10.1016/j.hpe.2018.03.005. [DOI] [Google Scholar]
  • 31.Mc Sween-Cadieux E., Chabot C., Fillol A., Saha T., Dagenais C. Use of infographics as a health-related knowledge translation tool: protocol for a scoping review. BMJ Open. 2021 Jun 16;11(6) doi: 10.1136/bmjopen-2020-046117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Kunze K.N., Vadhera A., Purbey R., Singh H., Kazarian G.S., Chahla J. Infographics are more effective at increasing social media attention and academic impact in comparison to original research articles: an altmetrics-based analysis. Arthroscopy. 2021;S0749–8063(21) doi: 10.1016/j.arthro.2021.03.056. 00318-2. [DOI] [PubMed] [Google Scholar]
  • 33.De Angelis C., Drazen J.M., Frizelle F.A., Haug C., Hoey J., Horton R., et al. International committee of medical journal Editors. Clinical trial registration: a statement from the international committee of medical journal Editors. N. Engl. J. Med. 2004;351(12) doi: 10.1056/NEJMe048225. 1250-1251. [DOI] [PubMed] [Google Scholar]
  • 34.Chan A.-W., Tetzlaff J.M., Altman D.G., Laupacis A., Gøtzsche P.C., Krleža-Jerić K., et al. SPIRIT 2013 Statement: defining standard protocol items for clinical trials. Ann. Intern. Med. 2013;158:200–207. doi: 10.7326/0003-4819-158-3-201302050-00583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Moher D., Shamseer L., Clarke M., Ghersi D., Liberati A., Petticrew M., -P Group Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015 Jan 1;4(1):1. doi: 10.1186/2046-4053-4-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Scott A., Rucklidge J.J., Mulder R.T. Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS One. 2015;10(8) doi: 10.1371/journal.pone.0133718. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Bradley H.A., Rucklidge J.J., Mulder R.T. A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatr. Scand. 2017;135(1):65–77. doi: 10.1111/acps.12647. [DOI] [PubMed] [Google Scholar]
  • 38.Rayhill M.L., Sharon R., Burch R., Loder E. Registration status and outcome reporting of trials published in core headache medicine journals. Neurology. 2015;85(20):1789–1794. doi: 10.1212/WNL.0000000000002127. [DOI] [PubMed] [Google Scholar]
  • 39.De Oliveira G.S., Jr., Jung M.J., McCarthy R.J. Discrepancies between randomized controlled trial registry entries and content of corresponding manuscripts reported in anesthesiology journals. Anesth. Analg. 2015;121(4):1030–1033. doi: 10.1213/ANE.0000000000000824. [DOI] [PubMed] [Google Scholar]
  • 40.Huić M., Marušić M., Marušić A. Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One. 2011;6(9) doi: 10.1371/journal.pone.0025258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Smyth R.M., Kirkham J.J., Jacoby A., Altman D.G., Gamble C., Williamson P.R. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ. 2011;342:c7153. doi: 10.1136/bmj.c7153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Dal-Ré R., Marušić A. Prevention of selective outcome reporting: let us start from the beginning. Eur. J. Clin. Pharmacol. 2016;72(10):1283–1288. doi: 10.1007/s00228-016-2112-3. [DOI] [PubMed] [Google Scholar]
  • 43.Gopal A.D., Wallach J.D., Aminawung J.A., Gonsalves G., Dal-Ré R., Miller J.E., et al. Adherence to the International Committee of Medical Journal Editors' (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in high-impact specialty society journals. Trials. 2018;19(1):448. doi: 10.1186/s13063-018-2825-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Ewart R., Lausen H., Millian N. Undisclosed changes in outcomes in randomized controlled trials: an observational study. Ann. Fam. Med. 2009;7(6):542–546. doi: 10.1370/afm.1017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Adie S., Harris I.A., Naylor J.M., Mittal R. CONSORT compliance in surgical randomized trials: are we there yet? A systematic review. Ann. Surg. 2013;258(6):872–878. doi: 10.1097/SLA.0b013e31829664b9. [DOI] [PubMed] [Google Scholar]
  • 46.Clavien P.A., Lillemoe K.D. A new policy to implement CONSORT guidelines for surgical randomized controlled trials. Ann. Surg. 2014;260(6):947–948. doi: 10.1097/SLA.0000000000001057. [DOI] [PubMed] [Google Scholar]
  • 47.Blair D.A., Hughes P.J., Woolley T.W., Jr. Pharmacy journal abstracts published in PubMed that abide by the CONsolidated Standards of Reporting Trials (CONSORT) guidelines. J. Med. Libr. Assoc. 2014;102(2):110–114. doi: 10.3163/1536-5050.102.2.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Baulig C., Krummenauer F., Geis B., Tulka S., Knippschild S. Reporting quality of randomised controlled trial abstracts on age-related macular degeneration health care: a cross-sectional quantification of the adherence to CONSORT abstract reporting recommendations. BMJ Open. 2018;8(5) doi: 10.1136/bmjopen-2018-021912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Turner L., Shamseer L., Altman D.G., Weeks L., Peters J., Kober T., et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst. Rev. 2012;11 doi: 10.1002/14651858.MR000030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.COMET Initiative Website. 2019. http://www.comet-initiative.org/ 3rd May 2021. [Google Scholar]
  • 51.Butcher N.J., Monsour A., Mew E.J., Szatmari P., Pierro A., Kelly L.E., et al. Improving outcome reporting in clinical trial reports and protocols: study protocol for the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT) Trials. 2019;20:161. doi: 10.1186/s13063-019-3248-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Draft InsPECT Extension Candidate Items (Round 3) 2019. https://osf.io/zyr9k/ 3rd May 2021. [Google Scholar]
  • 53.Heneghan C., Goldacre B., Mahtani K.R. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18:122. doi: 10.1186/s13063-017-1870-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Dwan K., Altman D.G., Cresswell L., Blundell M., Gamble C.L., Williamson P.R. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst. Rev. 2011;2011(1) doi: 10.1002/14651858.MR000031.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.McGauran N., Wieseler B., Kreis J., Schüler Y., Kölsch H., Kaiser T. Reporting bias in medical research—a narrative review. Trials. 2010;11:37. doi: 10.1186/1745-6215-11-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Hopewell S., Loudon K., Clarke M.J., Oxman A.D., Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst. Rev. 2009;1 doi: 10.1002/14651858.MR000006.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Sivendran S., Newport K., Horst M., Albert A., Galsky M.D. Reporting quality of abstracts in phase 3 clinical trials of systemic therapy in metastatic solid malignancies. Trials. 2015;16:341. doi: 10.1186/s13063-015-0885-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Ghimire S., Kyung E., Lee H., Kim E. Oncology trial abstracts showed suboptimal improvement in reporting: a comparative before-and-after evaluation using CONSORT for Abstract guidelines. J. Clin. Epidemiol. 2014;67:658–666. doi: 10.1016/j.jclinepi.2013.10.012. [DOI] [PubMed] [Google Scholar]
  • 59.Kuriyama A., Takahashi N., Nakayama T. Reporting of critical care trial abstracts: a comparison before and after the announcement of CONSORT guideline for abstracts. Trials. 2017;18(1):32. doi: 10.1186/s13063-017-1786-x. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data will be made available on request.

Articles from Contemporary Clinical Trials Communications are provided here courtesy of Elsevier

RESOURCES