The Altmetric Attention Score: What Does It Mean and Why Should I Care?

You may have noticed a brightly colored donut with a central number when browsing the most cited or most read articles on our Toxicologic Pathology journal website (http://journals.sagepub.com/home/tpx, last accessed December 5, 2017) or when browsing articles associated with other websites. This measure of research impact is the Altmetric, short for alternative metrics, donut and score. Briefly, it can provide information on web-driven scholarly interactions for the article you are interested in, including your own (Hirsch 2005; Melero 2015; Warren et al. 2017). This score is intended to be complementary to traditional, citation-based metrics with most of the data derived from web-based social media feeds. Because it is updated in daily or real-time feeds, you can keep track of where articles are being shared and discussed among broader audiences. Compared to other measures of research impact, two main advantages of the Altmetric are the immediate availability of information on the reach and influence of an article and the ability to track how the attention changes over time.

If you select the Altmetric icon located near the title of an article (see Elmore 2007 as example), you will see information on the title, journal, month/year of publication, DOI number, Pubmed ID, Author(s) and abstract. The donut is located at the top of the left column of the article metrics website for any specific journal article (see https://sage.altmetric.com/details/817961 as example, last accessed December 5, 2017). It may be in one or many colors, with a central number, which is the score. Below the donut is a blue question mark that you may select with this pop-up information about the score: “The Altmetric Attention Score for a research output provides an indicator of the amount of attention that it has received. The score is derived from an automated algorithm, and represents a weighted count of the amount of attention that is picked up for a research output.” There is also a link that will tell you how the scores are calculated (About the Altmetric and the Altmetric Attention Score). Briefly, the Altmetric score represents a weighted count of the amount of attention for a research output from a variety of sources (Table 1). The colors of the Altmetric donut each represent a different source of attention (medium blue for LinkedIn, yellow for blogs, red for mainstream media sources, purple for policy documents, etc.) and the amount of each color will change depending on which sources a research output has received attention from (Figure 1; The Donut and Altmetric Attention Score). There are three main factors used to calculate the Altmetric score: volume (how many times the article is mentioned), sources (where the mentions come from), and authors (of each mention) (Table 2). Information on how each factor might affect the score is provided on the Altmetric Support website (help.altmetric.com; outputs and sources) and reproduced in Table 2.

Table 1.

Examples of Data Collection Sources for Altmetric Scores^*

Source name	Collection method	Update frequency	Notes
Twitter	Third party data provider API	Real-time feed	An online news and social networking service where users post and interact with messages, called “tweets.”
Facebook	Facebook API	Daily	An American for-profit corporation and an online social media and social networking service.
Policy Documents	PDFs collected and scanned from policy sources and repositories	Daily	Scanning and text-mining international policy document PDFs for references, which are looked up in CrossRef/PubMed and resolved to DOIs.
News	RSS feeds and API	Real-time feed	Manually curated news sources, with data provided via a third-party provider and RSS feeds direct.
Blogs	RSS feeds	Daily	A discussion or informational website published on the World Wide Web consisting of discrete, often informal diary-style text entries.
Mendeley	Mendeley API	Daily	A desktop and web program produced by Elsevier for managing and sharing research papers, discovering research data and collaborating online.
Scopus	Scopus API	Real-time feed	Elsevier’s abstract and citation database.
Post-publication peer reviews	PubPeer and Publons APIs	Daily	Peer review comments collected from item records and associated by unique identifier.
Reddit	Reddit API	Daily	An American social news aggregation, web content rating, and discussion website. Registered members submit content to the site such as links, text posts, and images, which are then voted up or down by other members.
Wikipedia	Wikipedia API	Real-time feed	Mentions of scholarly outputs collected from References section. English Wikipedia only.
Stack Overflow Q&A	Stack Overflow API	Daily	A privately held website created in 2008 to be a more open alternative to earlier Q&A sites. A platform for users to ask and answer questions.
F1000 Reviews	F1000 API	Daily	Faculty of 1000 Research is an open research publishing platform that offers immediate publication of articles and other research outputs without “editorial bias.” Includes transparent peer review and inclusion of all source data.
Google+	Google+ API	Daily	An internet based social network. Public posts only.
YouTube	YouTube API	Daily	An American video-sharing website. Scans for links to scholarly outputs in video comments.
Open Syllabus	Static Import from Open Syllabus	Quarterly	Academic data mining project based at Columbia University that analyzes over 1 million college course syllabi.
Web of Science	Clarivate Analytics API	Real-time feed	Citation counts from peer-reviewed literature.

^*Data from Sina Weibo, CiteULike, Pinterest and LinkedIn is no longer collected. Coverage ended 7/24/15 (Sina Weibo), 12/14 (CiteULike), 6/20/13 (Pinterest) and 3/12/14 (LinkedIn). All data previously collected remains in the database and continues to appear on the Altmetric details pages.

API = application programming interface

RSS = rich site summary

PDF = portable document format

DOI = digital object identifier

Figure 1.

Colors of the Altmetric Score each represent a different source. The amount of each color will change depending on which sources a research output has received attention from. Because the attention from many of the sources are real-time or daily feeds, the colors of the donut and the score may change daily. Figure reproduced from https://www.altmetric.com/about-our-data/the-donut-and-score/ and last accessed 12/05/2017.

Table 2.

Three Main Factors Used to Calculate the Altmetric Score

Volume	Sources	Authors
The score for an article rises as more people mention it. Only 1 mention from each person per source is counted. If someone tweets about the same paper more than once, Altmetric will ignore all but the first.	Each category of mention contributes a different base amount to the final score. For example, a newspaper article contributes more than a blog post, which contributes more than a tweet.	Altmetric looks at how often the author of each mention talks about scholarly articles, at whether or not there’s any bias towards a particular journal or publisher and at who the audience is. For example, a researcher sharing a link with other researcher counts for more than a journal account pushing the same link out automatically.

Information adapted from https://www.altmetric.com/about-our-data/the-donut-and-score/ and last accessed December 5, 2017.

Also in the left column of the article metrics website you will see additional information if you select the “more…” link such as where it ranks (in percentile) among all research outputs scored by Altmetric, how it ranks among outputs from this source, how the score ranks compared to outputs of the same age, and percentile among outputs of the same age and source. Below this information you will see where the article has been mentioned (e.g., news outlets, blogs) and how many readers referenced the article on sites such as Mendeley and CiteULike.

At the top of the article metrics website you can see the summary information for the article as well as tabs for information on blogs, Twitter, Facebook, Wikipedia, Google+, etc., depending on which websites and social media outlets the article has been mentioned on. At the bottom of the page there are three selections: Twitter Demographics, Mendeley Readers, and Attention Score in Context. Below Twitter demographics and Mendeley Readers you will see additional information on readership statistics such as geographics and demographics. The map provides a visual for the geographic information. If you choose Attention Score in Context you will see number of all research outputs, number of outputs from Toxicology Pathology, number of outputs of similar age and number of outputs of similar age from Toxicologic Pathology. This may help you to understand if the level of attention is typical compared to similar articles. Below this information you will find how many research outputs Altmetric has tracked across all sources so far and information about its ranking.

In the upper right-hand corner of the article metrics website, there are blue boxes that can be selected to view the article on the publishers’ website where you will be able to download the pdf (if available) and where you may select to receive email reports in order to view new mentions of your research outputs in a given timeframe (https://sage.altmetric.com/details/817961; Last Accessed December 5, 2017). You may choose how often you would like to receive reports and monitor trending outputs and attention across sources.

Finally, there are options at the top right-hand corner of the article metrics website to share via Twitter, Facebook, Google+ or email. You may also embed the Altmetric badges in your website. The badge is simply the multicolored donut with the central Almetric score. The badges are free to use for academic repositories and individual researchers.

Who’s using the Altmetric tools and attention score? A 2015–2016 global survey on research tool usage with over 20,000 responses indicated that traditional metrics tools such as the impact factor and large citation bases are still the most often used by researchers (Zenodo 2018). However, librarians recommended the Altmetrics tool as much as the large citation based tools. This may be because librarians are aware of the weaknesses of traditional metrics tools and thus more likely to explore alternatives. Use among researchers at various career stages differed in that the use of the impact factor increased for researchers farther along in their career compared to the postdoc group that most often used the Altmetric tool. This trend in use by postdocs may reflect a more social media savvy age group, a willingness to explore new avenues and tools, and the need to demonstrate immediate impact of recent publications while acquiring a faculty or senior level position in their institution.

So how to interpret the Almetric score? Importantly, the score is helpful to rank research outputs based on attention from various sources, but it can’t tell you anything about the quality of the article itself. It simply tracks attention and attention can be good or bad. As an example, an article could be blogged about many times because of negative feedback. For research articles, some feel that if they get mentioned on social media it’s because they relate to popular topics, not because they are examples of good research. Tweets may also not be considered a useful measure for the value of a serious research paper. However, social media can provide a measurement of early reaction to research because the time it takes to discuss such work on social media can be much less than the time it takes to acquire citation information. Also, social media can provide a more complete picture of the use of research than citation counts alone.

Altmetric scores may not be available for all articles and some articles may have scores that decrease. Altmetric started collecting data in the last half of 2011 so scores will be based on data from that time forward. Most older articles do not have scores unless they have mentions in recent years. Scores will typically increase over time but may decline if a social media user deletes a post or their profile.

Another issue to consider is that the attention score for journals varies. A good score for one journal might be a low one for another. As an example, an article in Science or Nature will typically score higher than one in Toxicologic Pathology because they have a larger readership and more people will be likely to share the article. But one benefit of the Almetric score is that someone with an article that has a particularly high score might wish to indicate such information in their curriculum vitae, performance review, or grant application. As an example, one could provide information such as: “This item has received an Altmetric Attention Score of 7, putting it in the top 25% in terms of all research outputs scored by Altmetric and among the highest-scoring outputs from the Toxicologic Pathology journal (#18 of 178). A full summary of the attention score and record of all of the online mentions can be found here” (and include a link to the associated details page).

Finally, the Altmetric score provides non-traditional metrics that are considered an alternative to the more traditional h-index or Hirsch index, which is an author-level metric that measures both the productivity and citation impacts of the publications of a scientist or scholar (Hirsch, 2005). It is argued that an article needs little social media attention to jump to the upper quartile of ranked papers. This suggests that there are not enough sources of altmetrics currently available to give a balanced picture of impact for the majority of papers. Also, there is the concern that likes and mentions can be bought. And when comparing the score of one article to another, both should be published in the same year and during a similar time within the year because articles may be mentioned predominately during the time period that they are first published. So, use and interpret the Altmetric score with care. It should be used in conjunction with impact factor, H-index, number of downloads and citation counts to provide a more rounded picture of the article’s impact.