Publishing your research work: Updated concepts and nuances of few metrics used to assess journal quality

Abstract

Authors have a multitude of options for journals for publishing their research. However, their choice is mostly based on academic credits required for promotion, cost of publication, timeliness of process, etc., The purpose of this narrative review is to enlighten the authors about some other journal metrics used to assess journal ranking and quality in international scenario. The main concepts discussed in this paper are the impact factor and cite score. The paper includes an explanation of terms like web of science, journal citation reports, and how they are related to impact factor. This will help the authors to make the right decision about choosing the right journal for publishing their research. Along with the historic concepts we have included the latest updates about changes being made to the journal citation report and impact factor released in 2023 June. Hopefully with the review paper, we will be able to encourage the inclusion of such concepts and curriculum of post-graduation courses considering publishing a paper and choosing a journal are an integral aspect of a researcher’s work life.

INTRODUCTION

As soon as an academician starts working on scientific research, the parallel thoughts they have is where to publish their work. The most common approach taken by authors especially in Indian scenario is to select a journal that fits in the criteria set by governing councils, that is, the dental council of India/Medical Council of India.

These councils formulate certain guidelines to categorize the journals on the basis of which each journal is allotted certain points/academic credits. This category is allotted by taking into consideration where is the journal indexed, that is, Pubmed, Scopus, etc.[1] These academic credits are useful as a part of academic requirements which in turn helps in promotions. Hence, this naturally becomes the most important criterion for Indian authors when choosing a journal for publishing their work.

Along with the above criteria, there has been enough published data on how authors across the globe select an appropriate journal for publishing their work. They have been considering the aim and scope of the journal, checking whether the readership matches the area of research, timeliness of review, the reputation of the journal and publisher, publication fees, etc.[2]

However, the knowledge about other internationally accepted metrics of journal quality is still an area that needs improvement within the research community. A few of these include the impact factor (IF), Cite score (CS), Immediacy Index, Half-life, Eigen factor, Cites per item, Article Influence Score, H index, etc.[3]

Usually, the faculty who has exposure to editorial boards and the internal process of running a journal has deeper knowledge about these aspects. As a student, even the authors of this paper did not have enough exposure to these concepts, as they were nowhere included in the curriculum too. However, since conducting research and further publishing it is an integral aspect of post-graduation, we want the dental fraternity to be updated about other important indices, concepts, and terms related to the performance of a journal, which should be equally considered while choosing a journal. In this manuscript, we will highlight the pros and cons of the two most popular metrics of evaluation of a journal’s quality, that is, impact factor and CS.

IMPACT FACTOR

Impact factor is a common measure used by authors to gauze the reputation of a journal. The general understanding is, the higher the IF, the better the journal’s scientific reputation. It was developed by Drs. Eugene Garfield and Irving H. Sher in 1955.[4] However, the impact factor ranking was first published in the year 1969.[5] It is now a product of Clarivate Analytics (formerly part of Thomson Reuters.)

Before we move on to further aspects of the impact factor, let us understand what is the Clarivate and Web of Science (WoS).

Clarivate and WoS

Clarivate is a company, which provides insights and analytics services related to scientific and academic research and life sciences. It is a subscription-based service.

Previously known as Web of Knowledge, WoS is a product/service of Clarivate. It is a paid-access platform that provides online access to several databases that provide reference and citation data from academic journals, conference proceedings, and other documents in various academic disciplines like medicine, social sciences, arts and humanities, life sciences, etc., WoS now has indexing coverage from the year 1900 to the present but it does not just simply list all existing journals.[6,7]

To be listed in WoS, a journal needs to put an application to the WoS team. The journal is then evaluated on a total of 28 criteria (24 quality criteria and 4 impact criteria) using parameters like quality of content, quality of publication, compliance with publication guidelines, etc., Because the checks are extensive, it almost takes a time of 9–12 months to get a decision on a journal’s application to the WoS.

The journals with the top-most quality within the WoS are included in the “Web of Science Core Collection.” The core collection further has multiple indexes containing information gathered from thousands of scholarly journals, books, book series, and conferences of different disciplines, for example, Science Citation Index Expanded, Social Sciences Citation Index, Arts and Humanities Citation Index, Emerging Sources Citation Index, Conference Proceedings Citation Index, and Book Citation Index.[8]

How is WoS related to IF?

Clarivate releases a “Journal Citation Report” (JCR) which is a compilation of citation metrics of all journals indexed in WoS, and this JCR contains the IF.

JCR is nowadays being used as a very popular tool by different people associated with the field of research and publication. Librarians use it as a reference for building and managing their journal collections. Publishers gauge journal performance and researchers use it to identify appropriate journals for publication of their work as it contains the information about IF.

Till 2022, only journals in the Science and Social Sciences indexes were used to get an impact factor. However, in 2023 some changes have been made, which have been discussed further in the paper.

How is IF calculated?

In simple words, IF tells us about the average number of times articles published in the past 2 years in the journal have been cited in the next year.

We can define it as the number of citations received by a journal in the designated year to items published in the previous two years (numerator), divided by the total number of scholarly items, also known as citable items, published in the journal in the previous 2 years (denominator).[5]

The numerator includes citations drawn from all indexes in the WoS Core Collection. Citable items include original research or review papers. Items like editorials, letters, and news items are excluded from the denominator.[6]

When is IF released and where can I check it?

Every year from May to June, Clarivate releases the “Journal Citation Report” (JCR) which contains the information about IF. Subsequently, all journals use this information from the JCR and publish the IF on their journal home pages. The JCR can be easily accessed from the official website of Clarivate, that is, https://mjl.clarivate.com/home.

Is IF the gold standard?—Nuances associated

With time, the single-handed approach of considering IF as the gold standard is changing. Moreover, IF is attracting criticism as well. A diverse group of highly influential academicians, including the editor-in-chief of Science, Bruce Alberts, and the president of the Royal Society, Sir Paul Nurse, have collectively signed the San Francisco Declaration on Research Assessment, calling for misuse of the IF to be renounced.[9]

Unfortunately, IF calculation does provide a loophole for manipulation. It has been reported that some journals engage themselves in activities to affect the numerator in the formular of IF. For example, they strategically publish papers with a high citation potential such as meta-analysis and systematic reviews toward the first half of the year to enable higher opportunities for citations as these articles would be visible for a longer duration. While we feel that the above act is not completely unethical but indirectly it might lead to a delay in the publication of other good research which have lesser potential of citations.

Another technique being used by journals to manipulate IF is asking the authors to add references from the same journal (self-referencing/self-stacking).[4,10]

However, things like these do not remain un-noticed for a long time, and in 2022, the editorial integrity team at Clarivate formally defined the criteria for self-stacking suppression and issued a warning to six journals rather than suppress the journal’s IF. In the coming years continued journal self-stacking will result in suppression of IF.[11]

Another malpractice used to manipulate IF is by controlling the denominator in the IF calculation formula. This is done by reducing the publication of less citable articles, like case reports, literature reviews, and letters to the editor. This practice is particularly harmful to early career researchers because a case report is usually the easiest manuscript to write and hence is the first choice of a new author to begin writing. However, in the rat race of improving the IF, sometimes the journals reject case reports and other such papers, which can be demotivating.

Other nuances linked to IF are that it only looks at citations in the current year to articles in the previous 2 years. Hence, it only works well for disciplines in which rapid citation is the standard.

It also does not take into account disciplinary differences in the expected number of citations. Also, there is no JCR for arts and humanities; therefore, no impact factor for those journals.

Changes in Impact Factor Calculation—Updates from 2023

One major development in the JCR release for 2023 is that all journals listed in the Emerging Sources Citation Index (ESCI) have also been given an impact factor. Otherwise, so far only the journals listed in the Science Citation Index Expanded (also a part of WoS Core collection) used to get an IF.

This is the first year where the IF will only be reported by Clarivate to one decimal place (as opposed to three). This is a welcome and long-called for change, as the use of three decimal places is misleading. For example, with three decimal places, IF of 3.145 might be interpreted as better than one with 3.055. This year though, both would be reported as having IF of 3.1, and effectively equal to each other in quality. This is a better representation of the meaningful level of precision of the journal impact factor.[12]

CITE SCORE

CS was launched in December 2016 by Elsevier and has become a competitor of IF as a metric.[13]

How is it calculated?

The CS for a journal is calculated by counting the citations received in one year to the articles published in the previous 3 years divided by the total number of published articles in those 3 years. Both the number of citations and published articles are counted from Scopus, that is, the journal should be indexed in Scopus.

How is it different from IF?

Highlighting the differences from IF, firstly it counts citations of the last 3 years instead of 2 years as in the case of IF. Secondly, the denominator includes all published documents like research articles, review articles, conference proceedings, book chapters, and data papers)

Another interesting difference is that the CS is calculated from a much larger database than the IF which is based on the WoS. It was reported by Clarivate in 2021 that there were 27,200 titles in the Scopus list at that time which amounts be around 15,300 more than the WoS list impact factor.

When is it released and where can I check it?

You can check the CS on the official Scopus website (scopus.com). The value of CS is released annually from May to June of the subsequent year of calculation. While the CS is updated on an annual basis, there is a CS Tracker, which gives you an indication of the CS monthly. Tracker is based on the same calculation formula as the CS just that it is meant for the current year rather than the previous.

CONCLUSION

We have tried to shed some light on two popular journal-level metrics, which can help the authors in selecting the best journal for their research papers. However, it is not possible to comment on which of these metrics is better than the other. The reason being the numerous nuances associated with each of these. As highlighted, there are loopholes in the calculation, which are being used to deliberately manipulate the values. Also, another angle to the indecisiveness about the better one is the difference of how fast or slow and in what quantity citations appear in different subject areas. For example, what might be a good IF for one field or subject area might not be good for the other? This could be because the research might not be happening at the same pace in both fields and 2 years period might not be enough for the citations to appear. Though cites core has tried to address this shorter time period of IF by taking into account the last 4 years, it comes with its own lacunae.

Each author needs to evaluate which metric is more relatable to their situation and accordingly select the best journal for their work. However, this selection is only possible if the authors are well-versed in these metrics—As they say, “Eyes see what the mind knows.” Thus, through this paper, we have tried to provide a concise insight on IF and CS.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.