Hard Impact: Journal Impact Factor and JMT

In the fall of 2008, the University of Pennsylvania Press (the publisher of JMT at the time) submitted to the Institute of Scientific Information (ISI) a request that JMT be evaluated for inclusion into the Web of ScienceSM. Among other things, acceptance would have resulted in JMT being included among the greater than 5,000 science journals for which ISI reports a yearly journal impact factor (JIF). In January of this year, JMT received word that its request for inclusion was rejected. The only reason given for this rejection was low citation activity. We can reapply for inclusion in 2013. So how should JMT, and by extension its readers, feel about this rejection? Is this akin to staying home on prom night, or should we consider this an expected response to a young narrowly focused journal; after all, few freshmen go to the prom. To better understand this, a brief a review of the JIF is warranted.

Prior to the 1960s, scientists had no easy way to know who was referencing their work. As such, researchers in narrow fields often missed the chance to contact collaborators and to gain the insight that the review of an individual's work by others often brings. To address this problem, Eugene Garfield suggested creating a citation index for Science [5]. This idea would ultimately result in the Science Citation Index which was first published in 1963 [6]. To create such an index, Garfield and his colleagues would have to either include every scientific journal published, a daunting if not impossible task, or they would have to decide which journals would be included. Since it was known then, as it is today, that the majority of citations come from a relatively small number of journals, they chose the latter. To determine which journals they should include, they created the journal impact factor in 1975 [6]. One advantage that the JIF had over simply counting the number of citations to articles published in a journal is that it would avoid eliminating small but yet important journals with few issues and articles. The formula they used to calculate JIF (Fig. 1) would give them the average number of times an article published in a journal in the last 2 years was referenced in the current year. The 2-year period was chosen as it was thought to reflect peak citation activity for high-impact articles.

Fig. 1.

The formula used to calculate JIF

In creating the JIF, Garfield and his colleagues unwittingly created a metric by which journals would compare themselves and similarly a metric by which researchers and their institutions would gauge the quality of their work (i.e., publications in higher impact journals equal higher quality research) [1]. While it is true that the number of times a researcher is cited correlates with the quality of their work (quality defined by a Nobel prize), the same may not be true for journals [4].

Since the yearly JIF is based only on a journal's citations occurring in the previous 2 years, those journals that represent rapidly changing immediate impact fields have a distinct advantage. Compared to areas such as molecular biology, where new discoveries are made almost every week, medical toxicology is a slower advancing science: When was the last time you read an article that made an immediate impact in your practice?

The journal subject material and audience also influence a journal's JIF. Since scientists/clinicians tend to cite journals within their own fields, journals whose material and audience are broader get a larger number of citations. This can result in a wide discrepancy between the JIF of journals in different areas. It is possible, for instance, for the top journal within one field to have a lower impact than the bottom journal in a different field [2]. This may be one reason why journals like Nature, Science, JAMA, and the New England Journal of Medicine, journals whose readers and content entail the entire scope of medicine and science, are always among the journals with the highest impact factors (Table 1). Since toxicology is a very narrow field with a very small and specialized readership, it should not be surprising that toxicology journals have small JIFs (Table 2); this has been confirmed by two well done reviews on JIF in toxicology journals [3, 7].

Table 1.

2009 JIFs of some high-impact journals

Rank	Journal	JIF
1	CA—Cancer Journal for Clinicians	87.925
4	New England Journal	47.050
5	Annual Review of Immunology	37.902
8	Nature	34.480
13	Lancet	30.758
15	Science	29.747
20	JAMA	28.899

Table 2.

2009 JIFs of JMT and comparable journals

Journal	JIF
Pediatrics	4.687
Annals of Emergency Medicine	4.232
Academic Emergency Medicine	2.478
Toxicon	2.128
Clinical Toxicology	1.460
Human and Experimental Toxicology	1.307
Journal of Emergency Medicine	1.265
Journal of Medical Toxicology	1.433

Because review articles are among the most cited works (think about the papers cited mostly commonly in an article's introduction) and because reviews often cite previously published articles from the same journal (commonly referred to as self-citation), journals that publish a high number of reviews tend to have higher JIFs. By publishing fewer original papers (and more review papers), a journal can increase its JIF. If one looks at the top 20 journals ranked by JIF, one half are dedicated solely to reviews (Table 1). By comparison, for the eight editions of JMT published between 2007 and 2008 (the years responsible for the most current JIFs reported by ISI), there were only seven review articles. In line with the previous argument, these articles were amongst the most cited for those 2 years.

Since the numerator of the JIF includes citations from any source while the denominator only counts manuscripts and reviews, those journals with many “non-substantive” articles (e.g., editorials, letters to the editor, news reports) have an advantage over journals with less of these articles. While JMT does include some of this content (e.g., Poison Pen, The Art of Toxicology), we have much less “non-substantive” content than many of the bigger name journals such as BMJ or Nature.

While case reports are often the most read content of journals, they are typically poorly cited [8]. For example, of the top ten most downloaded manuscripts published by JMT, four are case reports (including the first and second most downloaded papers). Among the top ten most cited papers, however, only one is a case report. Since the field of medical toxicology relies on case reports to inform clinicians of problems associated with new drugs and unique clinical presentations from common drugs, our journals are necessarily replete with case reports and consequently will receive lower numbers of citations.

With all this stacked against JMT, we as readers and editors should perhaps not feel bad nor surprised about not being included, at this time, in the Web of Science. It is difficult, however, to not wonder what the JMT's JIF would have been had it been calculated. We therefore sought to estimate what the 2009 JMT journal impact factor would have been and to compare it to other journals in which medical toxicologists publish. Using the program Scopus (Elsevier Publishing), we went through the 95 articles published in JMT between 2007 and 2008 (60 substantive and 35 non-substantive) and counted the number of times each was cited in 2009. Using the ISI formula for JIF, we calculated a 2009 JIF for JMT of 1.433, a rather respectable number compared to other journals in our area (Table 2). It should be noted however that this calculated JIF might be overinflated as Scopus contains more journals than ISI's Web of Science.

So while it may be nice to be included amongst the journals that ISI reports JIFs for, it is not an impact factor that will define the success of our journal. Rather, it is the journal's dedication to its readership (medical toxicologists). As long as we continue to provide the content that helps shape their practice, JMT will make a positive impact, and this is a factor that cannot be tracked by a number.