Abstract
Background
Social media has been used to disseminate the contents of scientific articles. To measure the impact of this, a new tool called Altmetric was created. Altmetric aims to quantify the impact of each article through online media. This systematic review aims to describe the associations between the publishing journal and published article variables and Altmetric scores.
Methods
Searches on MEDLINE, EMBASE, CINAHL, CENTRAL, and Cochrane Library were conducted. We extracted data related to both the publishing article and the publishing journal associated with Altmetric scores. The methodological quality of included articles was analyzed by the Appraisal Tool for Cross-sectional Studies.
Results
A total of 19 articles were considered eligible. These articles summarized a total of 573,842 studies. Citation counts, journal impact factor, access counts, papers published as open access, and press releases generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to strong.
Conclusion
Citation counts and journal impact factor are the most common variables associated with Altmetric scores. Other variables such as access counts, papers published in open access journals, and the use of press releases are also likely to be associated with online media attention.
Systematic review registration
This review does not contain health-related outcomes. Therefore, it is not eligible for registration.
Keywords: Altmetric, Altmetrics, Social impact, Social media, Methodological review
Background
The most common way to assess the impact of an article is based on the number of citations [1]. The mean number of citations for all articles published in a journal in the preceding 2 years is called the journal’s impact factor [1]. However, the number of citations and the journal’s impact factor do not precisely reflect whether the message of the article is reaching a wider audience [2]. Currently, social media is being used to disseminate the contents of scientific articles [3, 4]. However, until recently, the impact of scientific articles on social media was not quantified. To measure this type of impact, a new score (called Altmetric) was created [3, 4].
Altmetric measures the impact of each article through the attention attracted online [3]. Moreover, the Altmetric score reveals the instantaneous attention attracted online for articles in news outlets, comments on blogs, number of tweets, and mentions on social media. There are two types of Altmetric scores. The Altmetric-mentioned score includes data sources involving social media (e.g., Facebook, Twitter), newspapers, encyclopedias (e.g., Wikipedia), online platforms (e.g., Faculty1000 and publication peer reviews), videos on YouTube, question-and-answer sites (e.g., Q&A stack overflow), and policy documents in PDF form available over the internet. The Altmetric reader score includes data sources involving reference managers available online (e.g., Mendeley, CiteULike, and Connotea). The Altmetric score can be graphically represented by a “donut.” The different colors of the Altmetric donut represent the number of mentions on each specific online media source. For example, mentions on Twitter are represented in blue (Fig. 1).
Fig. 1.
Description of the Altmetric donut
Research about Altmetric has been increasing and becoming more popular in recent years [5]. However, most articles about Altmetric published to date are only introductory tutorials or editorials [1, 3, 4, 6, 7]. Patthi et al. [2] published a systematic review in the field of dentistry that aimed to analyze the correlations between journal citations and Altmetric scores. The review concluded that journal citations and Altmetric scores are positively correlated (with Pearson’s r ranging from 0.30 to 0.61).
Recent articles from several research fields [8, 9] showed that the number of article citations and Altmetric score are positively correlated. Finch et al. [10] showed that the number of tweets (i.e., an Altmetric component) could predict citations within the first 3 days of article publication. Araujo et al. [11] found that number of citations and journal’s impact factor were positively associated with Altmetric [11]. These authors also found that the number of years since publication and having a descriptive title (i.e., a title describing the aim of the study but not revealing the main conclusions) were negatively associated with Altmetric [11]. Therefore, it is assumed that the publishing journal and publishing article variables, such as citation counts, journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and press releases generated by the publishing journal, are likely to be associated with Altmetric [11]. This systematic review aims to summarize all available evidence on the associations between the publishing journal and publishing article variables and Altmetric scores.
Methods
Research question
What publishing journal and publishing article variables are associated with Altmetric scores?
Search strategy for identification of studies
Systematic searches were conducted on MEDLINE, EMBASE, CINAHL, CENTRAL, and Cochrane Library, as per the Cochrane Handbook [12], including publications from the inception of these databases until March 31, 2021, without language restrictions. As the topic is novel, we used only two search terms (Altmetric OR Altmetrics) in all databases to ensure a more sensitive search strategy.
Inclusion and exclusion criteria
We included any original research studies that measured any type of association between the publishing journal and/or the publishing article with Altmetric scores, such as citation counts (i.e., number of citations), journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and press releases generated by the publishing journal. Studies that did not have at least one of these variables were excluded. Letters to the editor, editorials, and conference abstracts were also excluded. Moreover, we excluded articles that included a subset of highly cited papers or with extremely high Altmetric scores.
Data collection
Two review authors (AA and AV) independently screened all studies for eligibility and data extraction. All discrepancies identified during the stages and throughout the review were resolved via discussion or through arbitration provided by another investigator (DN). The study selection process included (1) screening the titles and abstracts and (2) screening of full-text articles.
Data extraction
Two review authors independently extracted the following data: (1) authors, (2) year of publication, (3) research field, (4) sample size of studies analyzed, (5) study design of the included studies, (6) study aims, (7) study results, and (8) study conclusions. Variables about the publishing journal included (9) journal impact factor, (10) access counts (considered the sum of HTML views and PDF downloads), (11) papers published as open access, and (12) press releases generated by the publishing journal. Variables about the publishing articles included (13) citation counts (i.e., number of citations), and (14) time since publication. We also collected data related to (15) the Altmetric mentioned score and (16) the Altmetric reader score. We contacted authors by email to request additional information that was not reported in the original manuscripts.
Ethics and registration
No ethical approval was required for this study. As this review has no health-related outcomes, no registration was needed [13].
Data analysis and quality of studies
Due to a large data heterogeneity, meta-analysis was not possible. For this reason, our results are reported descriptively. The quality criteria of included articles were analyzed using an adapted version of the Appraisal Tool for Cross-sectional Studies (AXIS) [14]. This tool was developed to systematically assess the quality of cross-sectional studies by assessing 20 items. Each item is rated as “yes,” “no,” or “don’t know/no comment” [14]. The AXIS was the tool that best covered the included studies. We adapted the AXIS by excluding items 7 (Were measures undertaken to address and categorize non-responders?), 9 (Were the risk factor and outcome variables measured correctly using instruments/measurements that had been trialed, piloted or published previously?), 13 (Does the response rate raise concerns about non-response bias?), 14 (If appropriate, was information about non-responders described?), and 20 (Was ethical approval or consent of participants attained?), as these items are unrelated to the aims of our review.
Results
Search results
The initial search yielded 1109 potentially eligible studies. After screening by title and abstract and removing duplicates, we considered 42 potentially eligible studies for inclusion and retrieved full-text articles. Nineteen published studies [11, 15–32] met the inclusion criteria and were included in this review. The study flow diagram of the eligibility assessment is presented in Fig. 2.
Fig. 2.
Study flow diagram of the eligibility assessment
Quality of studies
We did not consider the total score based on the instructions of the AXIS [14]. However, we observed that the studies included in general did not have good methodological quality (Table 1). We observed that most studies did not select a representative/random sample of a population (item 5), as most studies sampled the articles from main journals in their fields.
Table 1.
Quality assessment of the included studies by the AXIS tool
| Articles | Introduction | Methods | Results | Discussion | Other | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 8* | 10 | 11 | 12 | 15 | 16 | 17 | 18 | 19 | |
| Amath et al. [15] | Y | Y | C1 | Y | N | C2 | Y | N | N | Y | Y | Y | Y | Y | Y |
| Araujo et al. [11] | Y | Y | C1 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Araujo et al. [16] | Y | Y | C1 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Asaad et al. [17] | Y | C3 | C1 | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Ayoub et al. [26] | Y | C3 | C1 | Y | N | C2 | Y | N | N | Y | Y | Y | Y | Y | Y |
| Barakat et al. [18] | N | Y | C1 | Y | N | C2 | Y | N | N | Y | Y | Y | N | N | Y |
| Barbic et al. [19] | Y | C3 | C1 | Y | N | Y | Y | N | N | Y | Y | Y | Y | Y | C4 |
| Bornmann et al. [20] | Y | Y | C1 | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | N | Y |
| Chen et al. [21] | Y | C3 | C1 | Y | N | C2 | Y | Y | N | Y | Y | Y | Y | Y | Y |
| Costas et al. [22] | Y | C3 | C1 | Y | Y | Y | C5 | N | N | Y | Y | Y | Y | Y | C4 |
| Dagar et al. [23] | Y | C3 | C1 | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Didegah et al. [25] | Y | C3 | C1 | Y | Y | Y | N | N | Y | Y | Y | Y | Y | Y | C4 |
| Haneef et al. [24] | Y | Y | C1 | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Knight [27] | Y | C3 | C1 | Y | Y | C2 | Y | Y | N | Y | Y | Y | Y | Y | Y |
| Kunze et al. [29] | Y | C3 | C1 | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | C4 |
| Lamb et al. [30] | Y | C3 | C1 | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | N | Y |
| Richardson et al. [28] | Y | C3 | C1 | Y | N | C2 | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Richardson et al. [31] | Y | C3 | C1 | Y | N | C2 | Y | Y | N | Y | Y | Y | Y | Y | Y |
| Rosenkrantz et al. [32] | Y | C3 | C1 | Y | N | C2 | Y | N | N | Y | Y | Y | Y | Y | C4 |
Items 1. Were the aims/objectives of the study clear? 2. Was the study design appropriate for the stated aim(s)? 3. Was the sample size justified? 4. Was the target/reference population clearly defined? (Is it clear who the research was about?). 5. Was the sample frame taken from an appropriate population base so that it closely represented the target/reference population under investigation? 6. Was the selection process likely to select subjects/participants that were representative of the target/reference population under investigation? 7. Were measures undertaken to address and categorize non-responders? 8. Were the risk factor and outcome variables measured appropriate to the aims of the study? 9. Were the risk factor and outcome variables measured correctly using instruments/measurements that had been trialed, piloted, or published previously? 10. Is it clear what was used to determined statistical significance and/or precision estimates? (e.g., p values, CIs). 11. Were the methods (including statistical methods) sufficiently described to enable them to be repeated? 12. Were the basic data adequately described? 13. Does the response rate raise concerns about non-response bias? 14. If appropriate, was information about non-responders described? 15. Were the results internally consistent? 16. Were the results for the analyses described in the methods, presented? 17. Were the authors’ discussions and conclusions justified by the results? 18. Were the limitations of the study discussed? 19. Were there any funding sources or conflicts of interest that may affect the authors’ interpretation of the results? 20. Was ethical approval or consent of participants attained?
The items 7, 9, 13, 14, and 20 were excluded because those are unrelated to the aims of our review.
Y=Yes
N=No
C=Do not know/comment
*The risk factor analysis is not applicable for the outcome of interest of this study. Therefore, in this item, we consider only the analysis of measurement of the outcome variables
C1The sample size is not determined a priori because authors took in account the publications in a specific period from some journals
C2Authors analyzed all articles from specific journal(s), there was no description of selection process
C3Authors did not specify the study design
C4No statement about conflict of interest
C5The outcome variables measurement was not specified in the “Methods” section of this article
Characteristics of included studies
The 19 eligible studies were published between 2014 and 2021 and summarized a total of 573,842 articles. The study designs of the included articles were mixed research designs [15–28, 30–32] and randomized controlled trials [11, 29]. The research fields of these articles included biomedicine [20], burn care [31], ecology and conservation [30], emergency medicine [19], engineering and technology, gastroenterology and hepatology [26], general medicine [18], joint arthroplasty [29], medical education [15], medical and natural sciences [25], multidisciplinary [22], oncology [24], physiotherapy [11, 16], plastic surgery [17], psychiatry [23], radiology [32], rheumatology [21], social sciences and humanities, solid organ transplantation [27], and spine [28]. The main objective of the included studies was to assess the association between Altmetric scores and variables such as citation counts (i.e., number of citations), journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and/or press releases generated by the publishing journal. A summary of the methods, data analysis, results, and conclusions is presented in Table 2.
Table 2.
Summary of the objectives and methods according to the variables of interest in the review and author’s conclusions
| No | Author and year of publication (research field) | Objectives | Methods | Author’s conclusions |
|---|---|---|---|---|
| 1 |
Amath et al., 2017 [15] (Medical Education Journals) |
To analyze the relationships among Altmetric score, access counts and citation counts. |
Year of search strategy: 2012 and 2013. Sample size: n = 482. Data extraction: citation counts, access counts and Altmetric score. |
Altmetric scores were weakly correlated with readership (access counts) and impact (citation counts). |
| 2 |
Araujo et al., 2018 [11] (Physiotherapy) |
To analyze factors related with citation counts, journal impact factor and time since publication with Altmetric score. |
Year of search strategy: between 2010 and 2015. Sample size: n = 200. Data extraction: citation counts, journal impact factor, time since publication and Altmetric score. |
Researchers should preferably select high impact factor journals for submission. |
| 3 |
Araujo et al., 2021 [16] (Physiotherapy) |
To analyze factors related with citation counts, journal impact factor, open access and time since publication with Altmetric score. |
Year of search strategy: between 2015 and 2017. Sample size: n = 66 systematic reviews. Data extraction: citation counts, journal impact factor, open access, time since publication and Altmetric score. |
Researchers should preferably publish their articles in journals with high impact factor (which is indirectly linked to citations). |
| 4 |
Assad et al., 2020 [17] (Plastic Surgery) |
To analyze the relationship between citation counts and Altmetric score. |
Year of search strategy: 2016. Sample size: n = 1420. Data extraction: citation counts and Altmetric score. |
Altmetric scores were weakly correlated with citation counts. |
| 5 |
Ayoub et al., 2021 [26] (Gastroenterology and Hepatology) |
To analyze the relationship between citation counts and Altmetric score. |
Year of search strategy: 2014. Sample size: n = 4026. Data extraction: citation counts and Altmetric score. |
Altmetric scores were strongly correlated with citation counts. |
| 6 |
Barakat et al., 2019 [18] (General Medicine Journals) |
To analyze the relationship between citation counts and Altmetric score for high-impact general medicine journals. |
Year of search strategy: 2014. Sample size: n = 551. Data extraction: citation counts and Altmetric score. |
Altmetric scores were poorly correlated with the number of citations in the subsequent 3 years. |
| 7 |
Barbic et al., 2016 [19] (Emergency Medicine) |
To analyze the citation counts, journal impact factor and Altmetric scores in emergency medicine journals. |
Year of search strategy: 2014. Sample size: n = 50. Data extraction: citation counts, journal impact factor and Altmetric score. |
Altmetric scores were weakly correlated with citation counts and journal impact factor. |
| 8 |
Bornmann et al., 2018 [20] (Biomedical Area) |
To analyze the dimensions of measurement for citation counts and Altmetric. |
Year of search strategy: between 2011 and 2013. Sample size: n = 33.683. Data extraction: citation counts and Altmetric score. |
Altmetric reader score are associated to citation counts. |
| 9 |
Chen et al., 2019 [21] (Rheumatology) |
To analyze the relationship between citation counts and Altmetric score. |
Year of search strategy: between 2010 and 2015. Sample size: n = 1460. Data extraction: citation counts and Altmetric score. |
Disease area did not correlate with Altmetric and citations counts. Altmetric identified different articles as high impact compared with citation metrics. |
| 10 |
Costas et al., 2015 [22] (Multidisciplinary) |
To analyze the relationship between citation counts, journal impact factor and Altmetric score. |
Year of search strategy: 2013. Sample size: n = 500.229. Data extraction: citation counts, journal impact factor and Altmetric score. |
Altmetric scores were weakly correlated with citations. This findings suggests that the potential of Altmetric to replace traditional citation analysis is not very strong. |
| 11 |
Dagar et al., 2021 [23] (Psychiatry) |
To analyze the relationship between citation counts and Altmetric score. |
Year of search strategy: 2016. Sample size: n = 360. Data extraction: citation counts and Altmetric score. |
Altmetric scores were weakly correlated with citation counts. Besides that, the authors found a very high degree of public engagement with psychiatry research. |
| 12 |
Didegah et al., 2018 [25] (Social Sciences & Humanities, Engineering & Technology, Medical & Natural Sciences) |
To analyze the differences between citation counts and Altmetric score. |
Year of search strategy: between 2012 and 2014. Sample size: n = 13.623. Data extraction: citation counts and Altmetric score. |
Altmetric reader score are associated to citation counts. |
| 13 |
Haneef et al., 2017 [24] (Oncology) |
To analyze the variables journal impact factor, press release and open access with Altmetric score of articles evaluating cancer treatments. |
Year of search strategy: first 6 months of 2014. Sample size: n = 792. Data extraction: journal impact factor, press release, open access and Altmetric score. |
The press release and the journal impact factor are the most important factors associated with high online media attention were the presence. |
| 14 |
Knight, 2014 [27] (Solid Organ Transplantation) |
To analyze the association between citation counts and Altmetric score in the field of solid organ transplantation. |
Year of search strategy: between August 2011 and July 2012. Sample size: n = 6.979. Data extraction: citation counts and Altmetric score. |
Altmetric scores were weakly correlated with citation counts. Blogging and expert recommendation, in particular, are associated with higher citation rates. |
| 15 |
Kunze et al., 2020 [29] (Joint Arthroplasty) |
To analyze the relationship between citation counts and the Altmetric score. |
Year of search strategy: 2016. Sample size: n = 42. Data extraction: citation counts and Altmetric score. |
High methodologic quality and limited study bias markedly contribute to the Altmetric of RCTs in the total joint arthroplasty literature. |
| 16 |
Lamb et al., 2018 [30] (Ecology and Conservation) |
To analyze the association between citation counts and the Altmetric score. |
Year of search strategy: between 2005 and 2015. Sample size: n = 8.322. Data extraction: citation counts and Altmetric score. |
There are strong association between science communication (measured by the Altmetric score) and citation counts. |
| 17 |
Richardson et al., 2020 [28] (Spine Journals) |
To analyze the relationship between citation counts and the Altmetric score. |
Year of search strategy: January, February and March 2017. Sample size: n = 380. Data extraction: citation counts and Altmetric score. |
Altmetric scores were weakly correlated with citation counts in seven spine journals. |
| 18 |
Richardson et al., 2021 [31] (Journal of Burn Care & Research and Burns) |
To analyze the relationship between citation counts and the Altmetric score. |
Year of search strategy: 2017. Sample size: n = 285. Data extraction: citation counts and Altmetric score. |
Altmetric scores were weakly correlated with citation counts. Besides that, the authors recommend the combined use of Altmetric and traditional metrics such as citation count and impact factor. |
| 19 |
Rosenkrantz et al., 2017 [32] (Radiology) |
To analyze citation counts and Altmetric score for articles in popular general radiology journals. |
Year of search strategy: 2013. Sample size: n = 892. Data extraction: citation counts and Altmetric score. |
Articles published in four popular radiology journals overall received relatively low attention on social media comparison with citations. |
Statistical analysis and associations of included studies
Different types of analyses were conducted in the included studies: correlation analysis [15, 17–19, 22, 23, 26–29, 31, 32], regression analysis [11, 16, 21], boosted regression trees analysis [30], principal component analysis, and factor analysis [20, 25]. The main results of the included studies demonstrated that the variables citation counts (i.e., number of citations), journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and press releases generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to strong (Table 3). Summary of the association between variables of the publishing
Table 3.
Summary of the association between variables of the publishing journal and the publishing articles with Altmetric scores
| Studies and analyses | Citation counts | Journal impact factor | Access counts | Open access | Time since publication | Press release |
|---|---|---|---|---|---|---|
| Correlation analysis | ||||||
| Amath et al. [15] | r = 0.25 | r = 0.30 | ||||
| Assad et al. [17] | r = 0.33 | |||||
| Ayoub et al. [26] | r = 0.62 | |||||
| Barakat et al. [18] | r = 0.33 | |||||
| Barbic et al. [19] | r = 0.22 | r = 0.35 | ||||
| Costas et al. [22] | r = 0.18 | r = 0.19 | ||||
| Dagar et al. [23] | r = 0.43 | |||||
| Knight [27] | r = 0.16* and r = 0.23** | |||||
| Kunze et al. [29] | r = 0.36 | |||||
| Richardson et al. [28] | r = 0.32 | |||||
| Richardson et al. [31] | r = 0.12 | |||||
| Rosenkrantz et al. [32] | r = 0.20 | |||||
| Linear regression analysis | ||||||
| Chen et al. [21] | R2 = 0.00 | |||||
| Multivariate regression analysis | ||||||
| Araujo et al. [11] | β = 5.2* and β = 10.1** | β = 3.4 | β = −4.9 | |||
| Araujo et al. [16] | β = 2.9* and β = 6.37** | β = 15.36* and β = −3.21** | β = 0.74* and β = 4.04** | β = −21.99* and β = 18.13** | ||
| Haneef et al. [24] | RoM = 1.10 | RoM = 1.48 | RoM = 10.14 | |||
| Boosted regression trees analysis | ||||||
| Lamb et al. [30] Altmetric scores were strongly associated with citation counts. | ||||||
| Principal component analysis and factor analysis | ||||||
| Bornmann et al. [20] Altmetric reader score are associated to citation counts. | ||||||
| Didegah et al. [25] Altmetric reader score are associated to citation counts. | ||||||
*These numbers represent Altmetric mentioned. **These numbers represent Altmetric reader. r = correlation estimates. RoM = regression coefficients represent the logarithm of ratio of mean. β = β coefficient
Discussion
We aimed to summarize all available evidence on the associations between the publishing journal and publishing article variables and Altmetric scores. We found that citation counts (i.e., number of citations), journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and press releases generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to strong. In addition, we observed that citation counts and journal impact factor were associated with Altmetric scores in all included studies [11, 15–32].
There is a previous systematic review about the correlation between citation counts and Altmetric in medical research [2]. Moreover, there are articles that have measured associations between citation counts and Altmetric scores [15, 32]. In accordance with the systematic review [2] and these articles [15, 32], we found a positive correlation between citation counts and Altmetric scores. Similarly, our overview indicated positive associations (ranging from weak to moderate) between citation counts and Altmetric scores [11, 15–32]. These results are similar to those related to a journal’s impact factor [11]. This is not surprising, because a journal’s impact factor is based on citation counts of scientific articles [1]. We also found that most included studies, with the exception of those by Araujo et al. [11, 16] and Knight [27], did not analyze the Altmetric reader score. Thus, our findings are largely based on the Altmetric-mentioned score. We strongly recommend that further investigations on Altmetric reader score be conducted.
Regarding the quality of studies, the main limitation we observed was the lack of reporting the methods in detail. Items related to sampling, selection criteria, and statistical analysis in general were poorly described. On the other hand, the articles were clear in terms of data analysis and results. Finally, most authors presented the limitations of the study in their discussion and disclosed their potential conflicts of interest.
No studies identified specific characteristics of articles, for example, analysis of studies that published popular/hot topics (e.g., studies on coronavirus, miraculous diets, cancer prevention, early life on earth, religious evidence). Moreover, there is no analysis of studies comparing whether the direction of the results (i.e., positive versus negative conclusions) influences Altmetric scores. These characteristics are likely to increase the number of people who access and share these articles on social media [11]. We recommend that future studies identify if these characteristics are associated with Altmetric scores.
Finally, we propose 4 suggestions to improve the social impact and visibility of scientific articles: (1) select high impact factor journals for submission of articles; (2) use provocative titles (titles expressing the results of the trial) or interrogative titles; (3) use social media (Twitter, Facebook, etc.), websites, and blogs to disseminate principal findings; and (4) post the article with its digital object identifier (DOI) or the journal’s link to the article to be captured by Altmetric. These simple strategies are likely to improve the visibility of articles to a larger readership [5, 33]. The major strength of this study is the inclusion of articles from all fields of the research (n = 565,352 articles analyzed). On the other hand, a possible limitation of this study is the large heterogeneity of the included studies. Because of this, the data were analyzed only descriptively. Another potential limitation of our review is related to the selection of the databases we chose. We decided to cover the most comprehensive databases, such as MEDLINE, CINAHL, EMBASE, Cochrane Library, and CENTRAL, and we might have missed some eligible articles published in smaller databases or gray literature.
Conclusion
Citation counts, journal impact factor, access counts (considered the sum of HTML views and PDF downloads), papers published as open access, time since publication, and press releases generated by the publishing journal were associated with Altmetric scores.
Acknowledgements
Not applicable.
Authors’ contributions
A.A and A.V collected the data and performed the analysis. L.C supervised the project. A.A, A.V, D.N, G.Z, and L.C wrote the manuscript. The authors read and approved the final manuscript.
Funding
Not applicable.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Dinsmore A, Allen L, Dolby K. Alternative perspectives on impact: the potential of ALMs and Altmetrics to inform funders about research impact. PLOS Biol 2014;12(11):1–4. PMID:25423184, e1002003, DOI: 10.1371/journal.pbio.1002003 [DOI] [PMC free article] [PubMed]
- 2.Patthi B, Prasad M, Gupta R, Singla A. Altmetrics – a collated adjunct beyond citations for scholarly impact: a systematic review. J Clin Diagnostic Res 2017;11(6):16–20. PMID:28764311 [DOI] [PMC free article] [PubMed]
- 3.Brigham T. An introduction to Altmetrics. Med Ref Serv Q 2014;33(4):438–447. PMID:25316077, DOI: 10.1080/02763869.2014.957093 [DOI] [PubMed]
- 4.Melero R. Altmetrics- a complement conventional metrics. Biochem Medica 2015;2(25):152–160. PMID:26110028 [DOI] [PMC free article] [PubMed]
- 5.Araujo AC, Nascimento DP, Gonzalez GZ, Oliveira L, Costa P. How to increase the visibility of scientific articles through social media? Braz J Phys Ther [Internet] 2018;22(6):435–436. doi: 10.1016/j.bjpt.2018.08.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rhee JS. High-impact articles — citations, downloads, and Altmetric score. JAMA Facial Plast Surg 2015;17(5):1–2. PMID:26226501 [DOI] [PubMed]
- 7.Fabry G. Beyond the impact factor – what do alternative metrics have to offer? GMS J Med Educ 2017;34(2):1–8. PMID:28584875 [DOI] [PMC free article] [PubMed]
- 8.Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res 2011;14(1):e123. PMID:22173204 [DOI] [PMC free article] [PubMed]
- 9.Rosenkrantz AB, Ayoola A, Singh K, Jr RD. Alternative metrics (“Altmetrics”) for assessing article impact in popular general radiology journals. Acad Radiol 2017;24(7):1–7. PMID:28256440 [DOI] [PubMed]
- 10.Finch T, O’Hanlon N, Dudley SP. Tweeting birds: online mentions predict future citations in ornithology. R Soc Open Sci. 2017;4(11):1–11. doi: 10.1098/rsos.171371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Araujo AC, Nascimento DP, Gonzalez GZ, Maher CG, Costa LOP. Impact of low back pain clinical trials measured by the altmetric score: cross-sectional study. J Med Internet Res 2018;20(4):1–9. PMID:29622526 [DOI] [PMC free article] [PubMed]
- 12.Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ WV. Cochrane handbook for systematic reviews of interventions version [Internet]. Cochrane B Ser. 2021. PMID:21050962
- 13.Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev 2018;7(32):1–9. PMID:29463298 [DOI] [PMC free article] [PubMed]
- 14.Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open 2016;6(12):1–7. PMID:27932337 [DOI] [PMC free article] [PubMed]
- 15.Amath A, Ambacher K, Leddy JJ, Wood TJ, Ramnanan CJ. Comparing alternative and traditional dissemination metrics in medical education. Med Educ. 2017;51(9):935–941. doi: 10.1111/medu.13359. [DOI] [PubMed] [Google Scholar]
- 16.Araujo AC, Gonzalez GZ, Nascimento DP, Costa LOP. The impact of low back pain systematic reviews and clinical practice guidelines measured by the Altmetric score: cross-sectional study. Brazilian J Phys Ther [Internet] 2021;25(1):48–55. PMID:32067898, DOI: 10.1016/j.bjpt.2020.01.002 [DOI] [PMC free article] [PubMed]
- 17.Assad MD, Howell SM, Rajesh A, Meaike J, Nho V, Tran M. Altmetrics in plastic surgery journals: does it correlate with citation count? Aesthetic Surg J. 2020;40(11):1–19. doi: 10.1093/asj/sjaa158. [DOI] [PubMed] [Google Scholar]
- 18.Barakat AF, Nimri N, Shokr M, Mahtta D, Mansoor H, Masri A, Elgendy IY. Correlation of altmetric attention score and citations for high-impact general medicine journals: a cross-sectional study. JGIM. 2019;34(6):825–827. doi: 10.1007/s11606-019-04838-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Barbic D, Tubman M, Lam H, Barbic S, Carpenter CR. An analysis of Altmetrics in emergency medicine. Acad Emerg Med [Internet] 2016;23(3):251–268. doi: 10.1111/acem.12898. [DOI] [PubMed] [Google Scholar]
- 20.Bornmann L, Haunschild R. Do Altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data. PLoS One [Internet] United States; 2018;13(5):e0197133. [doi: 10.1371/journal.pone.0197133] [DOI] [PMC free article] [PubMed]
- 21.Chen WMY, Bukhari M, Cockshull F, Galloway J. The relationship between citations, downloads and alternative metrics in rheumatology publications: a bibliometric study. Rheumatology. 2020;59(2):277–280. doi: 10.1093/rheumatology/kez163. [DOI] [PubMed] [Google Scholar]
- 22.Costas R, Zahedi Z, Wouters P. Do “Altmetrics” correlate with citations? Extensive comparison of Altmetric indicators with citations from a multidisciplinary perspective. J Assoc Inf Sci Technol 2015;66(10):2003–2019. [doi: 10.1002/asi]
- 23.Dagar A, Falcone T. Altmetric scores analysis reveals a high demand for psychiatry research on social media. Psychiatry Res [Internet] 2021;298(113782):1–5. doi: 10.1016/j.psychres.2021.113782. [DOI] [PubMed] [Google Scholar]
- 24.Haneef R, Ravaud P, Baron G, Ghosn L, Boutron I. Factors associated with online media attention to research: a cohort study of articles evaluating cancer treatments. Res Integr peer Rev [Internet] 2017;2(9):1–8. doi: 10.1186/s41073-017-0033-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Didegah F, Bowman TD, Holmberg K. On the differences between citations and Altmetrics: an investigation of factors driving Altmetrics versus citations for Finnish articles. J Assoc Sci Tecnol. 2018;69(6):832–843. doi: 10.1002/asi.23934. [DOI] [Google Scholar]
- 26.Ayoub F, Ouni A, Case R, Ladna M, Shah H, Rubin DT. Dissemination of gastroenterology and hepatology research on social media platforms is associated with increased citation count. Am J Gastroenterol [Internet] 2021;1–3. [10.14309/ajg.0000000000001240]. in press. [DOI] [PubMed]
- 27.Knight SR. Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation. 2014;98(5):490–496. doi: 10.1097/TP.0000000000000307. [DOI] [PubMed] [Google Scholar]
- 28.Richardson MA, Bernstein DN, Mesfin A. Manuscript characteristics associated with the altmetrics score and social media presence: an analysis of seven spine journals. Spine J. 2021;21(4):548–54. [DOI] [PubMed]
- 29.Kunze KN, Richardson M, Bernstein DN, Premkumar A, Piuzzi NS, McLawhorn AS. Altmetrics attention scores for randomized controlled trials in total joint arthroplasty are reflective of high scientific quality: an Altmetrics-based methodological quality and bias analysis. J Am Acad Orthop Surg Glob Res Rev [Internet] 2020;4(12):1–10. doi: 10.5435/JAAOSGlobal-D-20-00187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Lamb CT, Gilbert SL, Ford AT. Tweet success? Scientific communication correlates with increased citations in Ecology and Conservation. PeerJ Life Eviron. 2018;12(6):1–14. doi: 10.7717/peerj.4564. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Richardson MA, Park W, Echternacht SR, Bell DE. Altmetric attention score: evaluating the social media impact of burn research. J Burn Care Res. 2021:1–5. 10.1093/jbcr/irab026. in press. [DOI] [PubMed]
- 32.Rosenkrantz AB, Ayoola A, Singh K, JrDuszak R, Duszak RJ, A.B. R, A. A, K. S. Alternative metrics (“Altmetrics”) for assessing article impact in popular general radiology journals. Acad Radiol [Internet] 2017;24(7):891–897. [doi: 10.1016/j.acra.2016.11.019] [DOI] [PubMed]
- 33.Araujo AC, Gonzalez GZ, Nascimento DP, Costa LOP, A.C. A, G.Z. G, D.P. N. Deep impact: 4 tips for authors and journal editors to improve Altmetric scores. Phys Ther [Internet] 2020;100(11):2060–2062. [doi: 10.1093/ptj/pzaa145] [DOI] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.