Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: Drug Alcohol Depend. 2020 Oct 18;218:108357. doi: 10.1016/j.drugalcdep.2020.108357

News and Social Media Coverage is Associated with More Downloads and Citations of Manuscripts that Focus on Substance Use

Joseph J Palamar a,b, Eric C Strain c
PMCID: PMC7750283  NIHMSID: NIHMS1638938  PMID: 33317951

Abstract

Background:

A variety of substance use-related topics are discussed in the public discourse; however, it is unknown how public discussion of published substance-related findings relates to manuscript downloads and citations. This manuscript examines how traditional and social media coverage of published findings about substance use affects downloads and scientific citations.

Methods:

Altmetric and bibliographic information was obtained for manuscripts published in Drug and Alcohol Dependence between 2018 and 2019 (n=943). Associations were examined between news and social media coverage (i.e., Twitter, Facebook) in relation to number of manuscript downloads and number of citations. This was done in a bivariable manner and in a multivariable manner examining correlates of being in the top 10th percentile of downloads and citations.

Results:

73.3% of articles were shared on Twitter, 23.6% were shared on Facebook, and 13.9% were covered in news sources (with 4.0% receiving major media coverage). Epidemiology papers were among the most covered in the news, and e-cigarette review papers were among the most downloaded. News and social media coverage were positively associated with number of downloads and citations in bivariable models and with achieving the top 10% of downloads and citations in multivariable models (ps<.001). Publishing a press release was associated with higher likelihood of receiving additional news coverage (aPR=7.85, 95% CI: 5.15-11.97).

Conclusions:

Traditional and social media coverage of manuscripts focusing on substance use are associated with more downloads and citations. Researchers should consider sharing findings not only to increase downloads and citations but also to educate the general public.

Keywords: altmetrics, social media, Twitter, Facebook, citations

1. Introduction

For decades, citations have been the leading metric to determine the impact of manuscripts published in the scientific literature; however, alternative metrics (“altmetrics”) have become important in assessing dissemination of manuscript findings in today’s digital world (Trueger et al., 2015, 2020). Many recent studies have examined the impact of altmetrics such as Twitter shares of manuscript findings in relation to number of scientific citations in the medical literature. Although most studies have detected weak to moderate associations between social media mentions and number of citations, some yield mixed results, and thus far, no studies have examined these associations focusing on manuscripts about substance use. This is important to examine because while many medical studies are not discussed by the public, substance use remains a popular topic in the public discourse. Popular topics include the opioid crisis, marijuana legalization, and the recent popularization of vaping. This article seeks to determine whether news and social media coverage of manuscripts published in the journal Drag and Alcohol Dependence (DAD) are related to a greater number of manuscript downloads and citations.

The popularity of social media has exploded over the past two decades. Currently, 72% of Americans use social media, and 90% of those ages 18-29 use such platforms (Pew Research Center, 2019). Despite increasing popularity of platforms like YouTube and Instagram, Facebook is still among the most used social media platforms with an estimated 69% of adults using this site (Pew Research Center, 2019). Twitter is only used by an estimated 22% of adults (Pew Research Center, 2019), but it appears to be a primary social media platform for sharing scientific knowledge, and it contains more posts and shares about scientific findings than other platforms (Chan et al., 2020; Warren et al., 2020a). Although nearly half (49%) of Americans prefer to access news stories via television, a third (33%) access stories directly through news websites, 26% access news from the radio, and 20% access news through social media (Geiger, 2019). Further, more than half (54%) of American adults are estimated to rely on general news outlets for science news, and while it may be assumed that most of the public does not read scientific journals, an estimated 25% obtain information from “science magazines” (in print or online) (Funk et al., 2017).

Given these national estimates, it is reasonable to deduce that a large portion of information resulting from recent studies is consumed and shared online via news sources and social media. News and social media coverage of findings not only alert the public to new findings in real time, but interest in such findings can create a ‘buzz’ (Trueger et al., 2015), which may lead people to share and discuss findings publicly. Assuming findings are interpreted correctly, this not only helps disseminate new information widely and quickly, but it is also beneficial to authors. The first way that this is beneficial is that it provides authors and their work with exposure. In fact, some researchers now add altmetric scores to their curriculum vitaes and report them in grant proposals as a form of alternative impact (Trueger et al., 2015). The second benefit is that findings from prior studies often suggest that news and social media coverage is associated with a higher number of scientific citations.

In general, it appears that higher altmetric scores, which are largely based on news and social media coverage, are usually weakly or moderately positively associated with number of scientific citations (Asaad et al., 2020; Bardus et al., 2020; Han et al., 2020; Jeong et al., 2019; Klar et al., 2020; Kunze et al., 2020; Luc et al., 2020; Warren et al., 2020b). While the association between social media posts and scientific citations has often been found to be small (Bornmann, 2015; Haustein et al., 2014; Knight, 2014), news media specifically has been found to be robustly associated with number of citations (Anderson et al., 2020), particularly in lower impact journals (Dumas-Mallet et al., 2020). While indeed some studies appear to have focused more generally on the relationship between news and social media coverage and number of citations, many of these studies have focused on specific subspecialties such as thoracic surgery, coloproctology, plastic surgery, oral surgery, rehabilitation medicine, implantology, and endodontology. Each discipline appears to have a varying presence in the media and on social media (Trueger et al., 2020), so results of these studies are not likely very generalizable to all scientific manuscripts or specifically to substance use manuscripts. Little is known regarding how news and social media coverage relate to downloads or citations on substance use manuscripts and this information is needed to guide researchers in the substance use field.

DAD is an international substance use journal devoted to publishing original research and scholarly reviews. Manuscripts published in DAD include studies of the chemistry of substance use, pharmacological and behavioral effects of substances, treatment and prevention, and there is a particularly extensive focus on substance use epidemiology. In this manuscript, we examine relationships between news and social media coverage with number of downloads and citations in manuscripts published in DAD. We also characterize the top manuscripts with the highest number of news and social media coverage and examine the extent to which published press releases affect news coverage.

2. Methods

2.1. Procedure

Altmetric and bibliographic information was obtained for manuscripts published in DAD between 2018 and 2019. Analyses were limited to DAD, as this journal has historically tended to publish more articles annually than other peer-reviewed substance use journals, and hence has the most downloads and citations. Access to proprietary information for other journals can also make obtaining data challenging, and we thus limited the present analyses to DAD. Data were extracted from PlumX Metrics on July 3, 2020 and provided by Elsevier. PlumX is a leading provider of alternative metrics and this platform obtains metrics based on secondary data sources such as social networks and publishing platforms (e.g., PubMed, Scopus) in order to determine a manuscript’s performance regarding dissemination (Ortega, 2018). Data for manuscripts published in 2020 were not included in order to provide adequate time for scientific citations to occur. Analyses focused on the 943 manuscripts identified as full reports, short reports, or review articles. Letters to the editor, news articles, editorial board listings, errata, and corrigenda were excluded from analyses. Altmetrics downloaded included number of: 1) downloads, 2) traditional news stories covering the manuscript findings, 3) shares of findings on Twitter, and 4) likes, comments, and shares of posts about findings on Facebook. It should be noted that detection of news and social media mentions appeared to be dependent on the story or post including a link to the manuscript. Bibliographic information included number of citations detected in the scientific literature. This analysis is not based on human subjects research so it was exempt from review from the authors’ institutional review boards.

2.2. Analysis

First, the percentages of manuscripts covered in the news, on Twitter, and on Facebook were calculated. Distributions and measures of central tendency were then examined for these measures and for number of manuscript citations and downloads. Number of citations and downloads were also dichotomized to indicate whether a manuscript fell within the top 10% of citations (≥10) or downloads (≥2,000). Manuscripts were then ranked in descending order to determine the top five manuscripts with respect to: 1) traditional news coverage, 2) downloads, 3) scientific journal citations, 4) Twitter shares, and 5) Facebook likes, comments, and shares. Scatterplots were then created to present the distributions for media coverage and Twitter shares in relation to number of citations and downloads.

Spearman correlations were used to estimate bivariable associations between variables of interest, particularly media coverage, Twitter shares, and Facebook likes, comments, and shares in relation to number of citations and downloads. We also examined whether major media coverage was associated with downloads and citations. Major media coverage was determined a posteriori after examining sources listed as media coverage. Sources were categorized as major sources if circulation appeared to be geared toward national readership and the source’s name was recognizable as a popular source by the authors. Spearman correlation was used to take into account the nonnormality of data. Although PlumX did not provide specific information on press releases, we coded news coverage from press release sites (e.g., Eurekalert) and from sites that republish press releases (e.g., Medical Xpress, ScienceDaily) into indicators that a press release was likely published. To examine how press releases related to news coverage, chi-square was used to determine whether there were bivariable differences and adjusted prevalence ratios (aPRs) were calculated using generalized linear model using Poisson and log link to determine whether press releases affected the likelihood of additional media coverage. This was done controlling for number of tweets, Facebook likes, shares and comments, article type (i.e., full report, short report, review paper), and whether the manuscript was published open access.

Finally, media and social media variables were fit into multivariable logistic regression models to determine how they related to achieving the top 10% of citations and downloads. Models included article type and whether the manuscript was published open access as covariates. Models first examined media and social media independent variables in a binary manner, and they were then repeated to examine associations with these variables in a continuous manner. All multivariable models also controlled for month of publication with each month and year entered as indicator variables ranging from February 2018 through December 2019 with January 2018 as the comparison. Sensitivity tests were also carried out in which we repeated correlations and models removing detected press releases (e.g., from Eurekalert) and coverage from sites that republish press releases (e.g., Medical Xpress) from news story counts.

3. Results

The majority (73.3%) of manuscripts were shared on Twitter at least once; 23.6% of manuscripts were liked, commented on, or shared on Facebook, and 13.9% were covered in traditional news sources (with 4.0% of manuscripts overall receiving major media coverage such as The New York Times). We estimate that 5.4% of manuscripts had an associated press release (based on detected press releases and sites that republish press releases). When excluding press releases from media coverage, 12.8% of manuscripts received independent media coverage. Distributions of news and social media mentions were severely positively skewed due to outliers. With respect to traditional media, the mean number of mentions was 0.8 (SD=5.4), the median was 0 mentions, and the range was 0 to 109 mentions. The mean was 0.4 (SD=1.3) when excluding outliers. The mean number of Twitter shares was 11.7 (SD=28.2), the median was 4 shares, and the range of shares was 0 to 408. The mean was 7.8 (SD=11.0) when excluding outliers. Facebook had a higher mean number of shares due to an extreme outlier (mean=62.2, SD=1,724.6), with a median of 0 and a range of 0 to 52,959. The mean was 6.0 (SD=24.8) when excluding the outlier.

Table 1 presents the five most cited DAD manuscripts in traditional media sources. Three of these manuscripts focused on substance use epidemiology. Ellis and colleagues’ (2018) manuscript about the surge in methamphetamine use among people who use opioids was covered by 109 news sources recorded by PlumX. The majority of detected coverage was on the radio, and findings were covered in two major sources—NPR and in US World News & Report. Han and Palamar’s (2018) manuscript estimating prevalence of marijuana use among middle-aged and older individuals was recorded as being covered in the media 88 times, and findings were covered in 8 major news sources including The New York Times, NBC, MSM, TIME, Yahoo, Fortune, NPR, and in US World News & Report. Gaither and colleagues’ (2018) manuscript about racial disparities in long-term opioid therapy was covered in 62 recorded media stories but received no major news coverage, and Kaltenbach and colleagues’ (2018) manuscript about prenatal exposure to methadone or buprenorphine in relation to childhood development was recorded as being covered in the media 27 times including coverage in two major sources—US World News & Report and NPR. Finally, Santaella-Tenorio and colleagues’ (2019) manuscript about cannabis use disorder trends in the US was covered in 27 recorded news sources and in no major sources.

Table 1 –

DAD articles most cited in traditional news stories

News Articles Major Media Articles Authors Paper Title Open Access Downloads Citations Twitter Shares Facebook Mentions
109 2 Ellis et al. Twin epidemics: The surging rise of methamphetamine use in chronic opioid users No 2,789 22 40 4
88 8 Han and Palamar Marijuana use by middle-aged and older adults in the United States, 2015-2016 No 1,895 18 9 11
62 0 Gaither et al. Racial disparities in discontinuation of long-term opioid therapy following illicit drag use among black and white patients No 1,304 7 128 2
27 2 Kaltenbach et al. Prenatal exposure to methadone or buprenorphine: Early childhood developmental outcomes Yes 4,325 36 1 0
27 0 Santaella-Tenorio et al. Cannabis use disorder among people using cannabis daily/almost daily in the United States, 2002-2016 No 771 2 146 118
Open in a new tab

Note. DAD – Drug and Alcohol Dependence. Facebook mentions includes posts, likes, and shares. Major media articles this this analysis were categorized as those covered in The New York Times, The Atlantic, Newsweek, The Guardian, NBC, MSM, Yahoo, Fortune, Forbes, NPR, TIME, US World News & Report, Daily Mail, CNN, ABC, CBS, CBC, AOL, Washington Post, Business Insider, New York Daily News, Rolling Stone, Buzzfeed, or Vice. All five manuscripts were full reports.

Table 2 presents the most downloaded manuscripts published in DAD. The top three most downloaded manuscripts were review articles, two of which were open access. Four of the five most downloaded manuscripts were about e-cigarettes. Breitbarth and colleagues’ (2018) review about e-cigarettes as a delivery system for various drugs was downloaded 16,717 times, and Fadus and colleagues’ (2019) manuscript about the rise in e-cigarette use received 9,938 downloads. Brett and colleagues’ (2019) manuscript regarding discussions about e-cigarettes on social media received 6,432 downloads, and Krishnan-Sarin and colleagues’ (2019) manuscript about e-cigarette use among high school students received 6,139 downloads. Carew and Comiskey’s (2018) review about treatment for opioid use among older adults also received 8,892 downloads.

Table 2 –

Most downloaded DAD articles

Downloads Authors Paper Title Report Type Open Access Citations Twitter Shares Facebook Mentions News Mentions
16,717 Breitbarth et al. E-cigarettes—An unintended illicit drag delivery system Review Yes 15 52 62 1
9,938 Fadus et al. The rise of e-cigarettes, pod mod devices, and JUUL among youth: Factors influencing use, health implications, and downstream effects Review No 15 4 0 1
8,892 Carew and Comiskey Treatment for opioid use and outcomes in older adults: a systematic literature review Review Yes 18 24 2 0
6,432 Brett et al. A content analysis of JUUL discussions on social media: Using Reddit to understand patterns and perceptions of JUUL use Full No 12 7 0 0
6,139 Krishnan-Sarin et al. E-cigarette devices used by high-school youth Full No 25 2 3 0
Open in a new tab

Note. DAD – Drug and Alcohol Dependence. Facebook mentions includes posts, likes, and shares.

Table 3 presents the DAD manuscripts most cited in the scientific literature. Tupper and colleagues’ (2018) short report about drug checking for fentanyl adulteration was most cited and discussed in 45 manuscripts. Kaltenbach and colleague’s (2018) manuscript was cited 36 times, and Allem and colleagues’ (2018) manuscript characterizing e-cigarette-related posts on Twitter was cited 34 times. Hunt and colleagues’ (2018) review about prevalence of substance use among people with schizophrenia received 31 citations, and Cerda and colleagues’ (2018) manuscript about marijuana laws and adolescent use of drugs received 29 citations.

Table 3 –

DAD articles most cited by scientific journals

Citations Authors Paper Title Report Type Open Access Downloads Twitter Shares Facebook Mentions News Mentions
45 Tupper et al. Initial results of a drag checking pilot program to detect fentanyl adulteration in a Canadian setting Short No 1,883 24 115 6
36 Kaltenbach et al. Prenatal exposure to methadone or buprenorphine: Early childhood developmental outcomes Full Yes 4,325 1 0 27
34 Allem et al. Characterizing JUUL-related posts on Twitter Full No 3,484 5 1 24
31 Hunt et al. Prevalence of comorbid substance use in schizophrenia spectrum disorders in community and clinical settings, 1990-2017: Systematic review and meta-analysis Review No 2,543 25 0 0
29 Cerdá et al. Medical marijuana laws and adolescent use of marijuana and other substances: Alcohol, cigarettes, prescription drags, and other illicit drags Full No 3,415 77 47 1
Open in a new tab

Note. DAD – Drug and Alcohol Dependence. Facebook mentions includes posts, likes, and shares.

Table 4 presents the most shared DAD manuscripts on Twitter. Meier and colleagues’ (2019) manuscript about associations between adolescent marijuana use and adult brain structure was mentioned 408 times on Twitter. The following three most tweeted manuscripts were about opioids. Specifically, Davis and colleagues’ (2019) manuscript about laws limiting prescribing and dispensing of opioids for acute pain was tweeted about 269 times, Morgan and colleagues’ (2019) paper about overdose following initiation of naltrexone and buprenorphine was tweeted about 269 times, and Cicero and colleagues’ (2018) manuscript about diverted buprenorphine was tweeted about 214 times. Rowe and colleagues’ (2019) manuscript about overdose mortality in single room occupancy buildings in San Francisco was also tweeted about 210 times.

Table 4 –

DAD articles most shared on Twitter

Twitter Shares Authors Paper Title Open Access Downloads Citations Facebook Mentions News Mentions
408 Meier et al. Associations between adolescent cannabis use frequency and adult brain structure: A prospective study of boys followed to adulthood No 2,807 3 52,959 6
269 Davis et al. Laws limiting the prescribing or dispensing of opioids for acute pain in the United States: A national systematic legal review No 5,807 21 82 2
269 Morgan et al. Overdose following initiation of naltrexone and buprenorphine medication treatment for opioid use disorder in a United States commercially insured cohort Yes 4,524 12 405 2
214 Cicero et al. Understanding the use of diverted buprenorphine Yes 5,018 22 23 7
210 Rowe et al. Drag overdose mortality among residents of single room occupancy buildings in San Francisco, California, 2010-2017 No 265 0 0 0
Open in a new tab

Note. DAD – Drug and Alcohol Dependence. All five articles were full reports. Facebook mentions includes posts, likes, and shares.

Table 5 presents the DAD manuscripts most liked, shared, or commented on Facebook. By far, Meier and colleagues’ (2019) manuscript about marijuana use and brain development received the most Facebook attention with 52,959 likes, shares, or comments. Morgan and colleagues’ (2019) manuscript was liked, commented on, or shared on Facebook 405 times, followed by Brands and colleagues’ (2019) manuscript about the effects of cannabis on driving (n=268), Ashford and colleagues’ (2018) manuscript about the impact of word choice and bias regarding substance use (n=206), and Kirtadze and colleagues’ (2018) manuscript about underestimation of prevalence of substance use (n=166).

Table 5 –

DAD articles most liked, commented on, or shared on Facebook

Facebook Mentions Authors Paper Title Open Access Downloads Citations Twitter Shares News Mentions
52,959 Meier et al. Associations between adolescent cannabis use frequency and adult brain structure: A prospective study of boys followed to adulthood No 2,807 3 408 6
405 Morgan et al. Overdose following initiation of naltrexone and buprenorphine medication treatment for opioid use disorder in a United States commercially insured cohort Yes 4,524 12 269 2
268 Brands et al. Acute and residual effects of smoked cannabis: Impact on driving speed and lateral control, heart rate, and self-reported drag effects No 1,916 4 111 3
206 Ashford et al. Substance use, recovery, and linguistics: The impact of word choice on explicit and implicit bias No 4,044 20 153 11
166 Kirtadze et al. Republic of Georgia estimates for prevalence of drag use: Randomized response techniques suggest under-estimation No 279 1 2 0
Open in a new tab

Note. DAD – Drug and Alcohol Dependence. All five articles were full reports. Facebook mentions includes posts, likes, and shares.

Examination of variables of interest using bivariable correlations suggests that number of downloads was strongly related to number of citations (rs=0.54, p<.001) and that number of tweets was moderately associated with Facebook posts, likes, and comments (rs=0.48, p<.001). Number of tweets was moderately associated with number of news stories (rs=0.30, p<.001), and number of Facebook posts, likes, and comments was moderately associated with number of news stories (rs=0.33, p<.001). Sensitivity tests conducted in which we removed press releases from number of news stories yielded almost identical bivariable results.

With respect to press releases, these were positively associated with number of news articles (rs=0.43, p<.001) and number of major news articles (rs=0.24, p<.001) covering findings. As is shown in Table 6, 80.4% of manuscripts associated with a press release received additional media coverage compared to 9.0% of manuscripts receiving any media coverage without a press release (p<.001). Almost a quarter (23.5%) of manuscripts associated with a press release received major news media coverage compared to 2.9% of manuscripts not associated with a press release receiving such coverage (p<.001). With all else being equal, publishing a press release was associated with much higher likelihood of receiving any additional news coverage (adjusted prevalence ratio; aPR=7.85, 95% CI: 5.15, 11.97) and any major media coverage (aPR=7.73, 95% CI: 3.46, 17.25) compared to manuscripts not associated with a press release.

Table 6 –

Press releases published in relation to traditional media coverage

Received Traditional Media Coverage After Press Release
No Additional Media, % Additional Media, % aPR (95% CI)
Press Release Published
 No 91.0 9.0 1.00
 Yes 19.6 80.4 7.85 (5.15, 11.97)
Received Major Media Coverage After Press Release
No Major Media, % Major Media, % aPR (95% CI)
Press Release Published
 No 97.1 2.9 1.00
 Yes 76.5 23.5 7.73 (3.46, 17.25)
Open in a new tab

Note. Press releases were those detected as being published by a press release wire site or by sites that republish press releases. Major media articles this analysis were categorized as those covered in The New York Times, The Atlantic, Newsweek, The Guardian, NBC, MSM, Yahoo, Fortune, Forbes, NPR, TIME, US World News & Report, Daily Mail, CNN, ABC, CBS, CBC, AOL, Washington Post, Business Insider, New York Daily News, Rolling Stone, Buzzfeed, or Vice. aPR = adjusted prevalence ratio, controlling for month and year of publication, report type, open access, and number of Twitter and Facebook mentions.

All ps <.001

Table 7 presents bivariable associations between number of citations and downloads and other variables of interest. News and social media mentions were weakly or moderately associated with number of news mentions (rs=0.20-.32, ps<.001) and tweets (rs=0.19-.34, ps<.001), and number of citations and downloads were weakly associated with number of major news mentions (rs=0.18-.28, ps<.001) and number of Facebook likes, comments, and shares (rs=0.18-.29, ps<.001). Figure 1 depicts scatterplots demonstrating associations between news and Twitter coverage and number of citations and downloads. In sum, there were manuscripts that were widely disseminated via traditional media or social media that received a moderate number of citations and/or downloads, and others that received few citations or downloads. Despite the positive associations detected, the most cited and downloaded articles (primarily Tupper et al., 2018 and Breitbarth et al., 2018) received very little media or social media coverage.

Table 7 –

Bivariable correlates of number of citations and downloads

Number of Citations Number of Downloads
Number of Citations Top 10% of Citations Number of Downloads Top 10% of Downloads
News Mentions .24 .20 .32 .28
Major Media Mentions .22 .28 .21 .18
Twitter Shares .21 .19 .34 .22
Facebook Likes, Shares, or Comments .18 .21 .29 .27
Open in a new tab

Note. All correlations were computed using Spearman to account for skewed distributions. Facebook mentions includes posts, likes, and shares. All ps <.001

Figure 1 –

Figure 1 –

Open in a new tab

Scatterplots depicting associations between news and Twitter coverage with citations and downloads

Finally, correlates of achieving the top 10 percentile of citations and downloads were examined in a multivariable manner (Table 8). With all else being equal, compared to full reports, review articles were associated with increased odds for being among the top cited, and any news (adjusted odds ratio; aOR=3.28, 95% CI: 1.87, 5.75), Twitter (aOR=1.98, 95% CI: 1.03, 3.83), or Facebook (aOR=3.03, 95% CI: 1.80, 5.08) likes, comments, and shares were associated with increased odds for achieving the top ten percentile of citations. When traditional and social media mentions were examined in a continuous manner, number of news sources covering findings (aOR=1.05, 95% CI: 1.00, 1.09) and number of Twitter shares (aOR=1.04, 95% CI: 1.03, 1.05) remained significant.

Table 8 –

Multivariable models delineating correlates of number of citations and downloads

Correlates of Number of Citations Correlates of Number of Downloads
aOR (95% CI) aOR (95% CI) aOR (95% CI) aOR (95% CI)
Open Access
 No 1.00 1.00 1.00 1.00
 Yes 1.01 (0.47, 2.15) 0.96 (0.44, 2.12) 14.64 (7.32, 29.27) c 14.79 (7.56, 28.94) c
Report Type
 Full report 1.00 1.00 1.00 1.00
 Short report 0.73 (0.36, 1.47) 0.98 (0.49, 1.99) 1.10 (0.48, 2.52) 1.45 (0.65, 3.23)
 Review 4.28 (1.80, 10.21) b 6.61 (2.82, 15.51) c 4.03 (1.55, 10.49) b 6.60 (2.58, 16.85) c
Any News Mentions 3.28 (1.87, 5.75) c 4.07 (2.16, 7.67) c
Any Twitter Shares 1.98 (1.03, 3.83) a 2.02 (0.78, 5.24)
Any Facebook Mentions 3.03 (1.80, 5.08) b 4.12 (2.21, 7.72) c
Number of News Mentions 1.05 (1.00, 1.09) a 1.04 (1.01, 1.07) b
Number of Twitter Shares 1.04 (1.03, 1.05) c 1.01 (1.00, 1.02) b
Number of Facebook Mentions 1.00 (1.00, 1.00) 1.01 (1.00, 1.02) b
Open in a new tab

Note. Facebook mentions includes posts, likes, and shares. All models controlled for month of publication. aOR = adjusted odds ratio, CI = confidence interval.

a

p < .05,

b

p < .01,

c

p < .001

With respect to downloads (Table 8 continued), making one’s manuscript available via open access was consistently associated with increased odds of being in the top ten percent of downloads by about 15 times. Compared to full reports, review papers were consistently at increased odds for a manuscript being highly cited, and any coverage in the news (aOR=4.07, 95% CI: 2.16, 7.67) or on Facebook (aOR=4.12, 95% CI: 2.21, 7.72) was associated with increased odds of being in the top ten percentile of downloads. When examining traditional and social media coverage in a continuous manner, small but significant increases were detected with number of news (aOR=1.04, 95% CI: 1.01, 1.07), Twitter (aOR=1.01, 95% CI: 1.00, 1.02), and Facebook (aOR=1.01, 95% CI: 1.00, 1.02) mentions associated with increased odds of being in the top ten percent of downloads. Models were repeat ed after removing press releases from news media counts and results were nearly identical across models.

4. Discussion

In recent years, various studies have been undertaken to examine whether social media posts about published manuscript findings are associated with increased number of scientific citations. However, most such articles thus far have focused on subspecialties within medicine such as thoracic surgery and endodontology. While indeed substance use is often considered a medical subspecialty, DAD covers a broad spectrum of topics. This appears to be the first analysis to examine these associations with regard to manuscripts about substance use, and this manuscript also examined how traditional media coverage relates to scientific citations. Between 2018 and 2019, the majority (73.3%) of manuscripts published in DAD were shared at least once on Twitter. Other studies have also found that the majority of such social media dissemination of scientific findings occurs on Twitter (Chan et al., 2020; Warren et al., 2020a). About a quarter (23.6%) of manuscripts were shared, commented on, or liked on Facebook, and 13.9% were covered in traditional media sources. Distributions were severely positively skewed due to outliers, but the median number of Twitter shares was 4, and the median number of Facebook and media shares was 0. This suggests that even when such a manuscript is shared, it is typically not shared widely.

Traditional media in particular appears to be an underutilized source by researchers, and this is reflected in the findings of these analyses. Results from PlumX data suggest that only 13.9% of manuscripts published in DAD have received any news coverage, and this calculation includes published press releases. Epidemiology manuscripts focusing on national datasets were more likely to acquire extensive media coverage and two of these papers focused on marijuana use, one of which estimated use among older individuals and the other which examined use in relation to legalization—both ‘hot’ topics in the media in recent years.

A major limitation to news coverage tracking is that PlumX typically does not detect manuscript mentions when a link to the manuscript is not included in the news article. For example, Ompad and colleagues’ (2019) manuscript about substance use among construction workers was featured in many major news sources including USA Today, Newsweek, Fox News, US World News & Report, New York Daily News, and New York Post, and none of these were detected by PlumX. This appears to have occurred because reporters did not include a link to the study, although Newsweek did include a link and the article was still not detected by PlumX. Although altmetrics such as PlumX appear particularly useful in detecting study mentions on radio broadcasts, altmetrics are often unable to detect television coverage of findings. For example, Han and Palamar (2018) appeared on NBC’s Today Show to discuss their findings, and their findings were also discussed throughout a skit by Stephen Colbert on his Late Show, but such coverage was not listed in PlumX. Since we detected substantial under-recording of major media coverage, we did not examine specific media sources in great detail. These results lead us to believe that altmetrics covering traditional media is informative but limited.

Another finding related to traditional media was that manuscripts associated with a press release were almost eight times more likely to receive additional news coverage or major news media coverage. These findings build upon previous evidence that press releases are associated with more extensive media coverage (Bartlett et al., 2002; Schwartz et al., 2012; Stryker, 2002). This suggest that authors who are seeking to more widely disseminate their findings through the media should consider publishing a press release. These are typically authored in collaboration with media specialists at one’s university and shared through news wire sites such as Eurekalert, Medical Xpress, or ScienceDaily. Of course, while some reporters do cover manuscript results not associated with a press release, press releases make results immediately available to reporters and/or the public and they also imply a sense of importance regarding results. Press releases also provide reporters with pre-written scientific information from trusted academic institutions which appears to increase the likelihood of a study being covered by media sources (Autzen, 2014).

Downloads is also an important indicator of dissemination and we found that number of downloads is strongly correlated with number of citations. Downloads may help serve as an indicator for manuscripts that will subsequently become highly cited given that citations typically take some time to accumulate. Four of the most downloaded DAD papers focused on e-cigarettes—a prominent topic in the media. Perhaps unsurprisingly, three of these top five most downloaded manuscripts were reviews and two were available via open access. In fact, another finding was that review papers were more likely to be both downloaded and cited more often. This adds to previous research which has found that review papers are associated with more citations (Jeong et al., 2019)—up to seven times more citations depending on the subspecialty (Miranda and Garcia-Carpinterob, 2018). Open access papers were also much more likely to be downloaded than subscription manuscripts; however, they were not more likely to be cited. These findings corroborate previous research (Davis, 2011; Daviset al., 2008; Eysenbach, 2006).

With respect to social media, manuscripts about opioid use were particularly popular on Twitter–especially manuscripts that focused on naltrexone and buprenorphine. However, the most liked, shared, and commented on coverage of a DAD manuscript on Facebook was an article reporting findings that adolescent marijuana use is not associated with structural brain differences in adulthood. These topics are common in the public discourse and these results suggest that there are strong advocacy bases that discuss or promote these findings on social media. Twitter and Facebook coverage was consistently related to manuscripts receiving both more downloads and citations.

Despite these associations, we found that some of the most cited manuscripts (e.g., Tupper et al., 2018 and Breitbarth et al., 2018) received little news or social media coverage, which perhaps adds to the complexity of findings. Such manuscripts thus appear influential to the field even when results are not shared widely via news sources or social media. We believe academics are more likely to locate studies of interest to cite through search engines such as PubMed rather than news sources and social media posts. More research is needed to determine, for example, whether manuscripts focusing on specific topics are more likely to be cited, independent of news and social media coverage. Likewise, while ‘hot’ topics appear most likely to acquire news and social media coverage, this does not necessarily mean that these articles are the highest quality or the most fruitful. Results suggest that despite reasonably consistent associations, at times there is a disconnect between news and social media coverage and number of citations. Temporality of associations could not be confirmed, although media and social media coverage of manuscripts typically is greatest shortly after publication, while accumulation of scientific citations is lagged. Therefore, it is likely that such coverage typically preceded citations.

Future studies should further examine whether synergies exist between modalities of coverage, and randomized trials examining how news and social media coverage affect downloads and citations would be most ideal. Investigating whether specific media sources or influential individuals on social media sharing findings influences downloads and citations would be another important future study. Finally, studies comparing media mentions on altmetric sites to media mentions in other search engines (e.g., Lexis Nexus) may help better determine the reliability of altmetrics media mentions.

4.1. Limitations

There can be a lag in citations as it often takes weeks to months for an accepted manuscript to be published (Ortega, 2018). Further, manuscripts published a year or even a month sooner than another may be more likely to be cited and this is a limitation. However, we controlled for year and month of publication in all multivariable models to help account for this. While Twitter analytics focused on shares, data were not available on likes or comments on related posts. With regard to Facebook, data available were based on aggregated number of comments, likes, and shares, so number of Twitter and Facebook shares cannot be compared using such data. Twitter and Facebook are indeed two major social media sources, but they are not the only media sources used to share coverage of scientific publications. For example, Instagram is among the most popular social media apps and Reddit is among the most popular message-board style social media sites. Data from these and from other social media sources were not available.

PlumX and other Altmetrics engines are powerful sources for tracking news and social media coverage, but they have limitations. As discussed above, PlumX was not able to detect numerous major media news articles covering findings so results should be considered with caution. Downloads based on authors’ personal or public archives were also not likely detected. We categorized major media sources a posteriori based on what we determined to be nationally recognized sources. Since our categorization was not based on readership or circulation statistics this method can possibly be seen as subjective; however, we listed all sources labeled as major sources below relevant tables. With regard to social media posts, we were able to examine the quantity but not the quality. Therefore, it is unknown which posts contained informative information or opinions and which were merely liked or shared (Robinson-Garcia et al., 2017; Zhang and Wang, 2018). Further, it is unknown whether any of such posts were shared by major social media influences who tend to have a wide audience, or whether posts were more likely to be diffused within smaller (e.g., more scientific) communities (Alperin et al., 2019). Finally, this analysis was limited to manuscripts published in DAD so results may not be completely generalizable to other drug journals.

4.2. Conclusions

This is the first analysis to examine associations between news and social media coverage of substance use manuscript findings and number of downloads and citations. Our results add to a wealth of previous findings from other disciplines and we hope results are informative for researchers in the substance use field. We believe researchers should be encouraged to share their published findings as this appears to be associated with increased downloads and citations. We also believe that regardless of reasons for authors sharing their findings, these forms of dissemination appear to be beneficial to educating and engaging a broad group of potential readers, including other scientists, policy makers, as well as the more general public.

HIGHLIGHTS.

  • We examined whether news and social media coverage relates to downloads and citations

  • 73.3% of articles were shared on Twitter and 23.6% were shared on Facebook

  • 13.9% of articles were covered in news sources and 4.0% in major media sources

  • News and social media coverage were associated with more downloads and citations

  • Press releases were associated with increased likelihood of news coverage (aPR=7.85)

Acknowledgments

We would like to thank Miss Agnieszka Freda at Elsevier for providing these data. J. Palamar is funded by the National Institutes of Health (NIH) (R01DA044207, PI: Palamar).

Role of funding source

Research reported in this publication was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number R01DA044207. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Declaration of Interest: The authors declare no conflict of interest.

Conflict of Interest

E. Strain has consulted, done work for, or served on advisory boards to the following organizations: Analgesic Solutions, Caron, Indivior, The Oak Group, Otsuka, Pinney Associates, and UpToDate. In addition, he receives an honorarium from Elsevier for his work as Editor in Chief of Drug and Alcohol Dependence.

References

  1. Allem JP, Dharmapuri L, Unger JB, Cruz TB, 2018. Characterizing JUUL-related posts on Twitter. Drug Alcohol Depend. 190, 1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alperin JP, Gomez CJ, Haustein S, 2019. Identifying diffusion patterns of research articles on Twitter: A case study of online engagement with open access articles. Public Underst Sci. 28, 2–18. [DOI] [PubMed] [Google Scholar]
  3. Anderson PS, Odom AR, Gray HM, Jones JB, Christensen WF, Hollingshead T, … Seeley MK (2020). A case study exploring associations between popular media attention of scientific research and scientific citations. PloS One 15, e0234912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Asaad M, Howell SM, Rajesh A, Meaike J, Tran NV 2020. altmetrics in plastic surgery journals: does it correlate with citation count? Aesthet Surg J [DOI] [PubMed] [Google Scholar]
  5. Ashford RD, Brown AM, Curtis B (2018). Substance use, recovery, and linguistics: The impact of word choice on explicit and implicit bias. Drug Alcohol Depend. 189, 131–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Autzen C, 2014. Press Releases — the new trend in science communication. J Sci Commun. 13. [Google Scholar]
  7. Bardus M, El Rassi R, Chahrour M, Akl EW, Raslan AS, Meho LI, Akl EA, 2020. The use of social media to increase the impact of health research: systematic review. J Med Internet Res. 22, e15607. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bartlett C, Sterne J, Egger M, 2002. What is newsworthy? Longitudinal study of the reporting of medical research in two British newspapers. BMJ 325, 81–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bornmann L, 2015. Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics, 103, 1123–1144. [Google Scholar]
  10. Brands B, Mann RE, Wickens CM, Sproule B, Stoduto G, Sayer GS, … Le Foll B, 2019. Acute and residual effects of smoked cannabis: Impact on driving speed and lateral control, heart rate, and self-reported drug effects. Drug Alcohol Depend 205, 107641. [DOI] [PubMed] [Google Scholar]
  11. Breitbarth AK, Morgan J, Jones AL, 2018. E-cigarettes-An unintended illicit drug delivery system. Drug Alcohol Depend. 192, 98–111. [DOI] [PubMed] [Google Scholar]
  12. Brett EI, Stevens EM, Wagener TL, Leavens ELS, Morgan TL, Cotton WD, Hebert ET, 2019. A content analysis of JUUL discussions on social media: Using Reddit to understand patterns and perceptions of JUUL use. Drug Alcohol Depend. 194, 358–362. [DOI] [PubMed] [Google Scholar]
  13. Carew AM, Comiskey C, 2018. Treatment for opioid use and outcomes in older adults: a systematic literature review. Drug Alcohol Depend. 182, 48–57. [DOI] [PubMed] [Google Scholar]
  14. Cerdá M, Sarvet AL, Wall M, Feng T, Keyes KM, Galea S, Hasin DS, 2018. Medical marijuana laws and adolescent use of marijuana and other substances: alcohol, cigarettes, prescription drugs, and other illicit drugs. Drug Alcohol Depend 183, 62–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Chan TM, Dzara K, Dimeo SP, Bhalerao A, Maggio LA, 2020. Social media in knowledge translation and education for physicians and trainees: a scoping review. Perspect Med Educ, 9, 20–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cicero TJ, Ellis MS, Chilcoat HD, 2018. Understanding the use of diverted buprenorphine. Drug Alcohol Depend. 193, 117–123. [DOI] [PubMed] [Google Scholar]
  17. Davis CS, Lieberman AJ, Hernandez-Delgado H, Suba C, 2019. Laws limiting the prescribing or dispensing of opioids for acute pain in the United States: a national systematic legal review. Drug Alcohol Depend. 194, 166–172. [DOI] [PubMed] [Google Scholar]
  18. Davis PM, 2011. Open access, readership, citations: a randomized controlled trial of scientific journal publishing. FASEB Journal, 25, 2129–2134. [DOI] [PubMed] [Google Scholar]
  19. Davis PM, Lewenstein BV, Simon DH, Booth JG, Connolly MJ, 2008. Open access publishing, article downloads, and citations: randomised controlled trial. BMJ, 337, a568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dumas-Mallet E, Garenne A, Boraud T, Gonon F, 2020. Does newspapers coverage influence the citations count of scientific publications? An analysis of biomedical studies. Scientometrics 123, 413–427. [Google Scholar]
  21. Ellis MS, Kasper ZA, Cicero TJ, 2018. Twin epidemics: the surging rise of methamphetamine use in chronic opioid users. Drug Alcohol Depend. 193, 14–20. [DOI] [PubMed] [Google Scholar]
  22. Eysenbach G, 2006. Citation advantage of open access articles. PLoS Biology, 4, e157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Fadus MC, Smith TT, Squeglia LM, 2019. The rise of e-cigarettes, pod mod devices, and JUUL among youth: Factors influencing use, health implications, and downstream effects. Drug Alcohol Depend. 201, 85–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Funk C, Gottfried J, Mitchell A, 2017. Science News and Information Today. Pew Reseach. Retrieved from https://www.journalism.org/2017/09/20/science-news-and-information-today/
  25. Gaither JR, Gordon K, Crystal S, Edelman EJ, Kerns RD, Justice AC,… Becker WC, 2018. Racial disparities in discontinuation of long-term opioid therapy following illicit drug use among black and white patients. Drug Alcohol Depend. 192, 371–376. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Geiger AW, 2019. Key Findings about the Online News Landscape in America. Pew Research. Retrieved from https://www.pewresearch.org/fact-tank/2019/09/11/key-findings-about-the-online-news-landscape-in-america/
  27. Han BH, Palamar JJ, 2018. Marijuana use by middle-aged and older adults in the United States, 2015-2016. Drug Alcohol Depend. 191, 374–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Han SC, Kang HJ, Lee WJ, Chung HS, Lee JH, 2020. A bibliometric analysis using alternative metrics for articles in the Annals of Rehabilitation Medicine. Ann Rehabil Med. 44, 158–164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hunt GE, Large MM, Cleary M, Lai HMX, Saunders JB, 2018. Prevalence of comorbid substance use in schizophrenia spectrum disorders in community and clinical settings, 1990-2017: Systematic review and meta-analysis. Drug Alcohol Depend. 191, 234–258. [DOI] [PubMed] [Google Scholar]
  30. Haustein S, Peters I, Sugimoto CR, Thelwall M, Larivière V 2014. Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. J Ass Inf Sci Technol. 65, 656–669. [Google Scholar]
  31. Jeong JW, Kim MJ, Oh HK, Jeong S, Kim MH, Cho JR, … Kang SB, 2019. The impact of social media on citation rates in coloproctology. Colorectal Disease 21, 1175–1182. [DOI] [PubMed] [Google Scholar]
  32. Kaltenbach K, O’Grady KE, Heil SH, Salisbury AL, Coyle MG, Fischer G, … Jones HE, 2018. Prenatal exposure to methadone or buprenorphine: Early childhood developmental outcomes. Drug Alcohol Depend. 185, 40–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kirtadze I, Otiashvili D, Tabatadze M, Vardanashvili I, Sturua L, Zabransky T, Anthony JC, 2018. Republic of Georgia estimates for prevalence of drug use: Randomized response techniques suggest under-estimation. Drug Alcohol Depend. 187, 300–304. [DOI] [PubMed] [Google Scholar]
  34. Klar S, Krupnikov Y, Ryan JB, Searles K, Shmargad Y, 2020. Using social media to promote academic research: identifying the benefits of twitter for sharing academic work. PloS One, 15(4), e0229446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Knight SR, 2014. Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation 98, 490–496. [DOI] [PubMed] [Google Scholar]
  36. Krishnan-Sarin S, Jackson A, Morean M, Kong G, Bold KW, Camenga DR, … Wu R,2019. E-cigarette devices used by high-school youth. Drug Alcohol Depend. 194, 395–400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kunze KN, Polce EM, Vadhera A, Williams BT, Nwachukwu BU, Nho SJ, Chahla J, 2019. What is the predictive ability and academic impact of the altmetrics score and social media attention? Am J Sport Med. 48, 1056–1062. [DOI] [PubMed] [Google Scholar]
  38. Luc JGY, Percy E, Hirji S, Vervoort D, Mann GK, Phan K, … Antonoff MB, 2020. Predictors of high impact articles in The Annals of Thoracic Surgery. Ann Thoracic Surg [DOI] [PubMed] [Google Scholar]
  39. Meier MH, Schriber RA, Beardslee J, Hanson J, Pardini D, 2019. Associations between adolescent cannabis use frequency and adult brain structure: A prospective study of boys followed to adulthood. Drug Alcohol Depend. 202, 191–199. [DOI] [PubMed] [Google Scholar]
  40. Miranda R, Garcia-Carpinterob E, 2018. Overcitation and overrepresentation of review papers in the most cited papers. J Informetrics, 12, 1015–1030. [Google Scholar]
  41. Morgan JR, Schackman BR, Weinstein ZM, Walley AY, Linas BP, 2019. Overdose following initiation of naltrexone and buprenorphine medication treatment for opioid use disorder in a United States commercially insured cohort. Drug Alcohol Depend. 200, 34–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ompad DC, Gershon RR, Sandh S, Acosta P, Palamar JJ, 2019. Construction trade and extraction workers: A population at high risk for drug use in the United States, 2005–2014. Drug Alcohol Depend. 205, 107640. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Ortega JL, 2018. The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX. J Informetrics 12, 579–589. [Google Scholar]
  44. Pew Research Center, 2019). Social Media Fact Sheet. Retrieved from https://www.pewresearch.org/internet/fact-sheet/social-media/
  45. Robinson-Garcia N, Costas R, Isett K, Melkers J, Hicks D, 2017. The unbearable emptiness of tweeting-about journal articles. PloS One, 12, e0183551. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Rowe CL, Riley ED, Eagen K, Zevin B, Coffin PO, 2019. Drug overdose mortality among residents of single room occupancy buildings in San Francisco, California, 2010-2017. Drug Alcohol Depend. 204, 107571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Santaella-Tenorio J, Levy NS, Segura LE, Mauro PM, Martins SS, 2019. Cannabis use disorder among people using cannabis daily/almost daily in the United States, 2002-2016. Drug Alcohol Depend. 205, 107621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Schwartz LM, Woloshin S, Andrews A, Stukel TA, 2012. Influence of medical journal press releases on the quality of associated newspaper coverage: retrospective cohort study. BMJ, 344, d8164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Stryker JE, 2002. Reporting medical information: effects of press releases and newsworthiness on medical journal articles’ visibility in the news media. Prev Med. 35, 519–530. [DOI] [PubMed] [Google Scholar]
  50. Trueger NS, Thoma B, Hsu CH, Sullivan D, Peters L, Lin M, 2015. The altmetric score: a new measure for article-level dissemination and impact. Ann Emerg Med. 66, 549–553. [DOI] [PubMed] [Google Scholar]
  51. Trueger NS, Yilmaz Y, Chan TM, 2020. Leveraging tweets, citations, and social networks to improve bibliometrics. JAMA Netw Open 3, e2010911. [DOI] [PubMed] [Google Scholar]
  52. Tupper W, McCrae K, Garber I, Lysyshyn M, Wood E, 2018. Initial results of a drug checking pilot program to detect fentanyl adulteration in a Canadian setting. Drug Alcohol Depend. 190, 242–245. [DOI] [PubMed] [Google Scholar]
  53. Warren VT, Patel B, Boyd CJ, 2020a. Analyzing the relationship between Altmetric score and literature citations in the Implantology literature. Clin Implant Dent Relat Res. 22, [DOI] [PubMed] [Google Scholar]
  54. Warren VT, Patel B, Boyd CJ, 2020b. Determining the relationship between altmetric score and literature citations in the oral and maxillofacial surgery literature. J Oral Maxillofac Surg. S0278–2391(20)30339–6. [DOI] [PubMed] [Google Scholar]
  55. Zhang L, Wang J, 2018. Why highly cited articles are not highly tweeted? A biology case. Scientometrics 117, 495–509. [Google Scholar]

RESOURCES