Comparison Between Impact Factor, Eigenfactor Metrics, and SCimago Journal Rank Indicator of Pediatric Neurology Journals

Abstract

Background:

Impact Factor (IF) as a major journal quality indicator has a series of shortcomings including effect of self-citation, review articles, total number of articles, etc. In this study, we compared 4 journals quality indices ((IF), Eigenfactor Score (ES), Article Influence Score (AIS) and SCImago Journal Rank indicator (SJR)) in the specific Pediatric Neurology journals.

Methods:

All ISI and Scopus indexed specific Pediatric Neurology journals were compared regarding their 2011 IF, ES, AIS and SJR.

Results:

Fourteen pediatric Neurology journals were identified, 3 of which were only Scopus indexed and the others were both ISI and Scopus indexed. High correlation was found between IF and AIS (0.850). Correlations between IF and other indices were not that high. Self-citation, total article number and review articles were related to the IF and other indices as well as their ranks. English language and citation to non citable item didn’t have any effect on pediatric neurology journals ranks.

Conclusion:

Although all the above mentioned indicators can be used interchangeably, using all considered indices is a more appropriate way than using only IF for quality assessment of pediatric neurology journals.

1. INTRODUCTION

Bibliometrics methods such as Impact Factor (IF) are known as a citation rate measures and are major criteria for quality ranking of scientific journals (1, 2). Calculating the journal’s impact factor annually is based on the number of citations within a certain year to the items published in the journal during the two previous years divided by the number of the citable items in the same interval (3, 4).

Several drawbacks are associated with the use of impact factor as a sole method of journal quality assessment (5-7). Factors affecting the IF calculation, have been criticized recently, including: self citation, impact of the review articles, citation to non-citable items, total number of citable items and English language bias (8-10). The possible misuses of IF in the evaluation of journal quality and individual researchers are controversial issues which draw much attention to use other alternative metrics in addition to IF (11).

In 2007, new bibliometric indicators were designed: Eigenfactor score (ES) and Article Influence Score (AIS). They are freely available and both rank journals by a similar algorithm as Google’s Page Rank does, which reflects not only the number of citations but also the prestige of citation source. They have different time window based on 5 years of citations. ES doesn’t have denominator and is influenced only by total citations not the citable items of a journal. AIS calculate the mean influence of citations and is conceptually the same as SCImago Journal Rank Indicator (SJR) (12, 14, 15).

Other addition to the scientometrics is SCImago Journal Rank Indicator (SJR) proposed by SCImago research laboratory in Spain in 2007 (16). SJR is a journal quality indicator that uses Scopus indexed journals for quality assessment. It is computed using similar method as the ES and AIS, but considers citations in Scopus database in a 3 years period (17).

In the present study, we sought to compare the four mentioned quality metrics in the pediatric neurology specific journals. We showed the importance of major journal quality indicators besides IF, which are better to be considered while deciding for publication of scientific materials.

2. METHODS

Specific pediatric neurology journals were selected from pediatric category of the SCImago journal and country ranking website and ISI web of knowledge. 2011 IFs, ESs and AISs were obtained from Journal Citation Report (JCR) through ISI. Relevant information was extracted from their source databases including: influence of self citations, citations to non-citable items, citations to review articles and their influence on 2011 IFs were assessed. 2011 SJRs were retrieved from its official website: The SCImago Journal & Country Rank. Journal ranking according to IF, AIS, SJR and ES were compared. Correlations between IFs and other indices were estimated by Spearman correlation coefficient. Potential impact factors (IF of the journal if it was ISI indexed in the 3 previous years) for identified pediatric neurology journals which were only Scopus indexed were computed using citation overview of the Scopus and compared with other ISI indexed journals’ IF. All analyses were performed by using SPSS version 11.5.

3. RESULTS

We identified 11 pediatric neurology specific journals indexed by ISI and Scopus and 3 others which were Scopus indexed only. Table 1 shows the obtained Spearman correlation coefficients between studied indices (IF, ES, AIS and SJR). All the relevant information about journal quality measures in 2011 (Total citation, total citable items, citation to non-citable items, citation to review articles and self-citation) is summarized in table 2.

Table 1.

Comparison of Spearman correlation coefficients of 2011 IF with 2011(ES, SJR and AIS) for specific pediatric neurology journals.

Table 2.

Rankings of the pediatric neurology journals in 2011 according to SCImago, Impact Factor, Eigenfactor score and Article Influence score. *: Computed potential impact factor for Scopus indexed pediatric neurology journals. ×: These two journals haven’t been ISI indexed for 5 years, so they don’t have any AIS.

4. DISCUSSION

This is the first study which compared pediatric neurology journals regarding their quality assessment indicators and we tried to show the importance and scientific visibility of pediatric neurology journals which are only Scopus-indexed by quality standards in addition to IF. We used a new metrics (potential IF) to compare the journal regarding their quality. Potential IF is calculated assuming a journal to be ISI indexed in the previous 3 years. In other words, what would be the IF of a journal, if it was ISI indexed?

Generally high correlation has been reported between said journal quality indices (17). In our study (as shown in table 1), the highest correlation was between IF and AIS (0.850) and other correlations were not that high which means there are factors that cause differences between journal quality metrics. Despite moderate correlation between other metrics (IF with SJR and ES), their journal ranks were similar to each other with small differences and these measures can be used for evaluation of ISI indexed Pediatric Neurology journals interchangeably. However the researchers should be cautious about factors that affect the ranking order. Pediatric researchers should consider other journal quality metrics in addition to IF for publication of their researches.

Scopus as an extensive database indexes many journals from different countries and languages. SJR measure has shown some differences with IF, it if less affected by self-citation and include all type of articles in its denominator not only ISI citable items namely original and review articles (20, 21). Three of 14 Pediatric Neurology journals were Scopus indexed only (Journal of Pediatric Neurosciences, Journal of Pediatric Neurology and Iranian Journal of Child Neurology). Their potential IFs were 0.25, 0.12 and 0.102 respectively. These potential impact factors are fairly respectable and show that ISI indexed journals are not the only visible journals in the pediatric neurology field.

Total number of citable items constitutes the IF denominator and is one of the issues that affects IF annually (20). Journals can increase their IF by decreasing the total number of their citable items. Among the four journal quality metrics mentioned in the current study, ES does not have any denominator and is sensitive to total number of citable items. In other words journals with low number of articles are likely to have lower ES (22). Low number of citable items for Journal of Neurodevelopmental Disorders is the possible reason of its 1st place in IF ranking despite ranking 10^th by ES.

ISI consider 2 types of articles (original and review articles) as the only citable items which are counted in the denominator of IF fraction. Publishing high amount of non-citable items which can receive considerable number of citations will boost the IF because they won’t be counted in the denominator. SJR considers all type of articles in its denominator and is less influenced by non-citable items. It is suggested to use SJR indicator, in addition to IF for quality evaluation of journals which publish high amount of noncitable items (17). In our study of Pediatric Neurology journals no considerable citation to non-citable items was observed.

In general, number of citation to an article is related to the type of cited article. Usually review articles have the higher probability of getting cited than original articles (5, 21). Our study also supported this concept. One of our considered journals, Seminar in Pediatric Neurology, only publishes review articles (98.2% of total 2011 citations to 2009 and 2010 items were citations to review articles). None of the evaluated journal quality metrics in our study are taking into account the type of the cited articles. However journals publishing only review articles usually have low amount of citable items as was the case for Seminars in Pediatric Neurology (69 articles in 2009 and 2010). Among the compared indices, only ES can indirectly shows the effect of the type of the cited article. Because it doesn’t have any denominator, journals with lower number of articles or journals that only publishes review articles will acquire lower ES rank (rank of Seminar in Pediatric Neurology dropped from 5^th for IF ranking to 9^th for ES ranking) (21).

Another strategy for some journals to boost their IF is self-citation. In calculation of IF, there is no correction for self citation therefore editors tends to encourage citing their previous works to improve their journal IF (5, 23). Other journal quality indicators (ES, AIS, SJR) take into account the self citation since by self citation there would be no prestige transfer and no increase in these indicators. Highest self citation of Pediatric Neurology Journals was found in Developmental Neurorehabilitation (26% of its total citations). This can be a reason for the lower ranking for ES and SJR (11^th place) despite higher rank of IF (6^th place) of this journal.

For the interpretation of IF parameter, 2 years will be used as a time window, however for Eigenfactor scores (ES and AIS) 5 years period of time will be considered. Journals which are ISI indexed for less than 5 years won’t have any AIS rank and their ES will be low. Time duration that a journal has been ISI indexed should be taken into account while using these two metrics (14). Two of our considered journals (Seminars in Pediatric Neurology and Developmental Neurorehabilitation) didn’t get any AIS rank because they haven’t been ISI index for 5 years. This may be the other reason of the lower ES ranks for two mentioned Pediatric Neurology journals in addition to publishing review articles or using self citation.

Another factor that has an effect in increasing the IF of a journal is English language bias. Articles which are in English language get more citation compared to other languages (20). All the studied Pediatric Neurology journals were in English language, and no journal language effect in Pediatric Neurology journals’ ranking was observed.

5. CONCLUSION

To conclude, although IF is believed as the most common journal quality indicator, there are some drawbacks associated with its using as a sole criterion that draw attention to consider other journal quality indices. Some pediatric neurology journals are only Scopus indexed and can be evaluated particularly by SJR. In our opinion, journal editors should get familiar with the factors influencing IF (such as self citation, non-citable items, etc), in order to avoid using them liberally as new journal quality metrics are available which could account for these factors. Researchers and journal editors should be aware of these new journal quality metrics as they are more likely to be used in the future due to shortcomings of the IF.