Optimizing the use of twitter for research dissemination: The “Three Facts and a Story” randomized-controlled trial

Abstract

Background & Aims:

Published research promoted on twitter reaches more readers. Tweets with graphics are more engaging than those without. However, data are limited regarding how to optimize multimedia tweets for engagement.

Methods:

The “Three facts and a Story” trial is a randomized-controlled trial comparing a tweet featuring a graphical abstract to paired tweets featuring the personal motivations behind the research and a summary of the findings. Fifty-four studies published by the Journal of Hepatology were randomized at the time of online publication. The primary endpoint was assessed at 28-days from online publication with a primary outcome of full-text downloads from the website. Secondary outcomes included page views and twitter engagement including impressions, likes, and retweets.

Results:

Overall, 31 studies received standard tweets and 23 received story tweets. Five studies were randomized to story tweets but crossed over to standard tweets for lack of author participation. Most papers tweeted were original articles (94% standard, 91% story) and clinical topics (55% standard, 61% story). Story tweets were associated with a significant increase in the number of full text downloads, 51 (34–71) vs. 25 (13-41), p = 0.002. There was also a non-significant increase in the number of page views. Story tweets generated an average of >1,000 more impressions than standard tweets (5,388 vs. 4,280, p = 0.002). Story tweets were associated with a similar number of retweets, and a non-significant increase in the number of likes.

Conclusion:

Tweets featuring the authors and their motivations may increase engagement with published research.

Introduction

The impact of science is proportional to its dissemination. Social media and specifically Twitter is the most powerful tool available today to increase the audience for published research. Twitter is a public platform where information is shared in 280-character messages that can be enhanced with media and article hyperlinks. Observational studies and randomized trials have established that articles that are shared on twitter are more likely to be cited.^1,2 Few studies, however, have examined how the content of tweets impacts reader engagement.

In 2010, Cell Press originated the graphical abstract – a stylized presentation of the study aims and results. A case-crossover study by Ibrahim et al. showed that compared to tweets with the citation alone, tweets containing a version of the graphical abstract branded as the ‘visual abstract’ attracted more twitter attention/impressions and article visits.³ This study was not randomized and compared visual media to plain-text. Recently, Oska conducted a case-control crossover study for article tweets that randomized the order of tweets featuring the citation, visual abstract, and a key figure from the study.⁴ The authors demonstrated substantially higher twitter engagement using the visual abstract. Arguably, however, the strongest level of engagement with science is measured in article downloads. Indeed, randomized trials comparing images of the text abstract to visual abstracts have failed to show any difference in article views or downloads.^1,5,6

We hypothesized that attracting attention to the people and motivations behind the research will garner more attention than descriptions of the research itself. Herein, we report the results of the “Three Facts and a Story” randomized-controlled trial comparing tweets featuring the reasons behind the research to graphical abstracts.

Materials and methods

We conducted a randomized controlled trial using newly published papers for the Journal of Hepatology (JHEP). The Journal of Hepatology is the official journal of the European Association for the Study of the Liver and is published by Elsevier. We included all original articles and reviews published from 7/7/20-6/11/20. Randomization was conducted by coin-flip. All tweets were released at 8.00 pm CET (or CEST) by the journal account. After 7 days, each tweet was retweeted. The trial team did not engage with any JHEP tweets during the trial period.

Endpoints and outcomes

The primary endpoint was 28 days from the time of online publication. The primary outcome was full-text downloads. Secondary outcomes included total page views, retweets, likes, and impressions. Website engagement and download metrics were provided by the publisher. Twitter engagement metrics were downloaded from the Twitter platform.

Experimental groups

Fig. 1 contrasts the format of the tweets for 2 papers from each experimental groups.^7,8

Fig. 1. Twitter strategies.

We compared 2 twitter strategies: story and standard tweets. Examples of each are provided in the figure.

Standard tweets: At JHEP, all accepted manuscripts must have a graphical abstract. Accordingly, tweets featuring the title, the article link with links to the authors twitter-pages, and a graphical abstract were considered the standard procedure.

Story tweets: The experimental procedure is termed “3 Facts and a Story”. After randomization, authors of papers randomized to the experimental arm were contacted to provide a picture of the authors and to fill out a standardized survey detailing the reasons why they did the study and what they found. These responses were then edited by the JHEP social media editor (EBT) for the twitter format, divided into 2 tweets – titled “why we did this study” and “what we found” – to be included with a link to the paper. The image attached to the first tweet was of the author(s) and the image attached to the second was the graphical abstract.

Study characteristics

For the purpose of cohort description, we collected several variables. These included the article type (original research, snapshot), article category (clinical, basic), clinical trial, content (e.g. viral hepatitis, cirrhosis), and country of origin.

Analysis

The analyses were conducted based on the type of tweet released as it was expected some authors would decline to participate. Twitter statistics were calculated for the story tweets based on the first tweet alone (as story tweets included a pair of tweets). All outcomes were expected to be non-parametric continuous values. As such the primary and secondary outcomes were evaluated using Wilcoxon-Rank Sums testing. Using prior publications as a guide for power, it was felt that at least 20 studies per arm were required to detect differences in tweet impressions when 2 twitter strategies are evaluated.^1,3,4 As prior studies have not examined full-text downloads or reads, a power calculation was not possible for this outcome. Significance was considered to be a 2-tailed p value <0.05.

Results

Study descriptions

Fifty-four studies were randomized. Five studies were randomized to story tweets but crossed over to standard tweets because the authors did not respond to requests for their story submission. Overall, 31 studies received standard tweets and 23 received story tweets. Most papers tweeted were original articles (94% standard, 91% story) and clinical topics (55% standard, 61% story). While some topics were balanced (viral hepatitis 16% vs. 17%, cirrhosis 16% vs. 13%, fatty liver disease 6% vs. 4%) others were less balanced (cancer 23% vs. 35%, transplantation 10% vs. 0%, trials 10% vs. 4%) (Table 1).

Table 1.

Baseline details of studies.

	Standard graphical abstract	Three facts and a story
Type
Original	29 (94%)	21 (91%)
Review	2 (6%)	3 (9%)
Clinical science	17 (55%)	14 (61%)
Clinical trial	3 (10%)	1 (4%)
Topic
Non-alcoholic fatty liver disease	2 (6%)	1 (4%)
Alcohol-related liver disease	0 (0%)	1 (4%)
Viral hepatitis	5 (16%)	5 (17%)
Autoimmune hepatitis	0 (0%)	1 (4%)
Cholestatic liver disease	2 (6%)	0 (0%)
Cirrhosis management	5 (16%)	3 (13%)
Liver cancer	7 (23%)	8 (35%)
Metabolism/signaling	6 (19%)	5 (22%)
Liver transplant	3 (10%)	0 (0%)

Outcomes

The outcomes are detailed in Table 2. Story tweets were associated with a significant increase in the number of full text downloads, 51 (34-71) vs. 25 (13-41), p = 0.002. There was also a non-significant increase in the number of page views. Story tweets generated an average of >1,000 more impressions than standard tweets (5,388 vs. 4,280, p = 0.002). Story tweets were associated with a similar number of retweets, and a non-significant increase in the number of likes. Of note, the single most engaging paper was randomized to the standard arm,⁸ garnering 23,492 twitter impressions and 388 full-text downloads.

Table 2.

Outcomes of the clinical trial.

		Standard tweets	Story tweets	p value
Website engagement	Downloads (median IQR)	25 (13–41)	51 (34–71)	0.002
	Page views (median IQR)	43 (18–59)	60 (36–81)	0.07
Twitter engagement	Retweets (median IQR)	7.7 (3.6–11.8)	7.4 (5.0–9.7)	0.3
	Likes (median IQR)	18.5 (9.2–27.8)	20.7 (14.6–26.8)	0.06
	Impressions (median IQR)	4,280 (2,732–5,829)	5,388 (4,591–6,185)	0.0002

Differences between the study arms were assessed using Wilcoxon Rank-Sums tests.

Discussion

The purpose of research publication is to disseminate knowledge. One of the most powerful tools we can leverage to increase the impact of our work is social media and specifically Twitter. Particularly in the past year, there has been a surge of engagement on twitter by the community of liver disease researchers and Hepatology clinicians known as #LiverTwitter.⁹ To optimize the spread of information in this space, mounting evidence suggests that the form of the tweet is crucial to its function. Whereas prior trials have established that tweeting graphical/visual abstracts attract more attention than tweets with key figures or without any figure,^3,4 we conducted the “three facts and a story” trial to determine whether exposure to the author’s personal presence and motivations was more engaging. Our randomized trial shows that the story behind the work may yield more full text downloads than graphical abstracts.

As a randomized trial of size similar to its predecessors that met its primary endpoint, these data should inform our approach to social media dissemination of research. We have 2 conclusions. First, a format that highlights the personal perspective behind the research increases engagement. Future efforts could clarify whether tweets from the author themselves may be more effective than tweets through the journal account. Second, paired tweets may be more effective than one. A popular method of Twitter-based education is the tweetorial that threads together multiple tweets for a progressive, didactic delivery of information. The optimal length of a thread for the purpose of engagement is unknown.

These data must be interpreted in the context of the study design. First, this journal has a wide readership and a highly active, widely followed twitter account. Second, the tweets featured a wide variety of original articles including basic science, clinical trials, and narrative reviews. It is unclear from these data whether the tweet format should be tailored to the nature of the article. Third, although this journal is European and many papers involved non-native English speakers, the tweets were all in English, potentially limiting engagement among some subsets of the readership. Future interventions should strive for maximal inclusivity. Fourth, only short-term outcomes were evaluated and the durability of these findings is therefore unknown. Similarly, it is unknown if these data will translate into a higher citation rate.

The “three facts and a story” trial demonstrates an effective method for increasing engagement with published literature. Journals and authors should be encouraged to use paired tweets featuring the research’s motivations in addition to their findings.