Abstract
Background and Aim:
This study was performed to evaluate the impact factors (IFs) and total citations of ISI-indexed nutrition journals in a 10-year period from 1999 to 2008 in order to assess the quality of nutrition journals.
Materials and Methods:
For this retrospective study, the IF and total citation data from 1998 to 2008 were collected through Journal Citation Reports of Thomson Scientific Corporation Web of Knowledge. We selected five highly cited journals in the “nutrition and dietetics” category for our analysis. These journals include Annual Reviews in Nutrition (ANNU REV NUTR), American Journal of Clinical Nutrition (AJCN), Progress in Lipid Research (PROG LIPID RES), Journal of Nutrition (J NUTR), and International Journal of Obesity (INT J OBESITY)
Results:
All five selected journals were ranked as one of the top ten “nutrition and dietetics” journals between 1999 and 2008 in ISI database. Most of selected journals’ IF had an upward trend during the 10-year period with fluctuation in some cases. AJCN consistently received the greatest number of total citations during the study period, although its IF was not the highest among the five journals studied.
Conclusion:
The IF illustrated changes in relative rankings of five highly cited journals included in the “nutrition and dietetics” category of the Web of Knowledge. Rank according to the absolute number of citations received, however, did not correlate with rank according to IF.
Keywords: Impact factor, nutrition and dietetics, total citation
INTRODUCTION
In 1995, Dr Eugene Garfield introduced journal impact factor (IF) in order to clarify the significance of absolute citation frequencies.[1,2] This bibliometric parameter relies on journal citation reports (JCR).[2–4] The JCR provides quantitative tools for ranking, categorizing, and comparing journals.[5] The annual IF is a measure of the ratio of recent citation to the recent citable items published.[5–8] Citation metrics and other statistic methods like IF are used to evaluate the significance and impact of individual journals and help librarians select which journals to subscribe to. These bibliometrics are also used by authors to decide where to submit their manuscripts.[3,7,8]
In order to calculate IF, Institute of Scientific Information (ISI) uses the average citation of articles over the past 2 years as numerator and articles published in the subsequent year as denominator.[6,7,9] For instance, the IF for 2008 would be calculated as the total citations in 2008 to articles published in 2006 and 2007 divided by number of citable articles published in 2006 and 2007. In other words, a journal's IF of 1.5 for the year 2008 means that on average articles published during 2006 and 2007 in that journal were cited 1.5 times by articles published in 2008 in all indexed journals.[10]
So far many journals, including nutrition journals, have been indexed by ISI. It is well known that review journals in different fields usually have the highest IFs in their own category.[3,6,11] For instance, Annual Review of Nutrition is one of the review journals in “Nutrition and Dietetics” category of ISI with the highest IF in its field. This journal publishes review articles from well-known nutrition researchers in the world. Peer-reviewed journals publishing original research communications mostly have lower IFs than that of review journals. One of highly cited journals in ISI's “Nutrition and Dietetics” category is American Journal of Clinical Nutrition (AJCN), which mostly publishes the original papers. Other examples of journals with high IFs in this field are Journal of Nutrition, International Journal of Obesity, etc.
Investigators from several fields have assessed the trend of journal IF over time. Such bibliometric studies give us a bird's eye view of biomedical research and also document performance of scientific journals. In some fields like occupational therapy [12] and biomedical engineering, [8] such evaluations have been performed and published. To our knowledge, no data are available assessing the time-trend of journals’ IF in the “Nutrition and Dietetics” category. This study aimed to evaluate a 10-year (1999-2008) time- trend IF of five highly cited journals from the “Nutrition and Dietetics” category of ISI-Web of Knowledge as a yardstick to assess the quality of nutrition journals during this period. We also considered whether the IF in our sample was affected by the types of articles published (i.e., original research vs review) independently of the absolute numbers of citations received.
MATERIALS AND METHODS
In this retrospective study, required information about journal IFs and total citation times of “Nutrition and Dietetics” category of ISI journals were collected for the last 10-year period (1999-2008) through JCR of Thomson Reuter Web of Knowledge (http://apps.isiknowledge.com). In 1999, a total of 43 journals were included in the Thomson Reuter ISI's “Nutrition and Dietetic” category. This rose to 53 in 2005 and 59 in 2008.
To focus on the top-ranked journals in this category, we selected five highly cited journals from among top ten in their category (between 1999 and 2008) in nutrition and dietetics category for our analysis: Annual Reviews in Nutrition (ANNU REV NUTR), American Journal of Clinical Nutrition (AJCN), Progress in Lipid Research (PROG LIPID RES), Journal of Nutrition (J NUTR), and International Journal of Obesity (INT J OBESITY). The rational for selecting these journals were their popularity among researches in the field. We also tried to select journals that are publishing just review papers along with those that are publishing original articles. Such selection would help us to compare the effect of review journals on the IF fluctuations.
RESULTS
All selected journals were ranked as one of top ten nutrition and dietetics journals between 1999 and 2008 in ISI category [Table 1]. Over the 10-year period, trends in Annu Rev Nutr, Am J Clin Nutr, and Prog Lipid Res journals’ percentile ranking were high and relatively stable. In the same timeline, rankings for J Nutr and Int J Obesity fluctuated marginally. Prog Lipid Res ranked first in 7 years of the 10- year timeline.
Table 1.
Metrics of impact factor for the five specific nutrition and dietetics journals, 1999 through 2008
Over the 10-year period, there was noticeable increasing trend in the IF of some journals like Am J Clin Nutr (70.29%) [Table 2]. The most IF percentile changes occurred in Annu Rev Nutr for 2004 (100.53%). In some cases like Int J Obesity, there was negative IF growth during the early years.
Table 2.
Percentage annual changes in impact factor during the 10-year study period referred to the 1999 impact factor
Five selected journals had progressive trajectory for total citation changes with a slight decrease over the 10-year period [Table 3]. Considering the year 1999 as a base timeline, Int J Obesity had a negative IF growth, but its total citation change was the highest (187.94%).
Table 3.
Percentage annual changes in the numbers of citations received during the 10-year study period referred to citations in 1999
Total citations for all selected journals increased from 1999 through 2008 [Figure 1]. This increment was mostly prominent in the case of Am J Clin Nutr, followed by J Nutr and Int J Obesity. Total citations for Annu Rev Nutr and Prog Lipid Res slightly increased between 1999 and 2008. Comparing all five selected journals, Am J Clin Nutr had consistently the highest total citations in all 10 years assessed although not the highest IF; Prog Lipid Res and Annu Rev Nutr had the lowest total citations in this timeline.
Figure 1.
Trends in total citation for five specific nutrition journals, 1999 - 2008. Total citations for all selected journals increased from 1999 through 2008
The IF of all five selected journals generally increased between 1999 and 2008 [Figure 2]. The IF of all five selected journals generally increased between 1999 and 2008. Although the Prog Lipid Res had upward and downward trajectory over the 10-year period, it had the highest IF both in 1999 and 2008. In some cases, these changes were linear and gradual like Int J Obesity. As expected, journals publishing review articles had the highest IFs almost all the times. J Nutr and Int J Obesity had limited growth on their IF over the 10-year period.
Figure 2.
Trends in impact factor for five specific nutrition journals, 1999 - 2008. The JIF of all five selected journals generally increased between 1999 and 2008
DISCUSSION
Although published papers play an essential role in different aspects of science, using the IF can be a reasonable indicator of quality for medical journals.[13] This study was the first to explore the bibliographical trajectory of nutrition and dietetics journals indexed in ISI JCR. We found the upward trend of five selected journals in terms of both IF and total citations through the last 10-year period from 1999 to 2008.
Despite the strength of the IF, several limitations of this measure must be taken into account. Several factors, not related to journal quality, can change the IF of journals. Of important ones is the scientific field of each journal. The highest journal IF for basic science is higher than the highest IF of specialty fields like genetic journals. So, comparison of journals’ IF should be limited to one field. Other factors that could change the IFs are the number of articles in each volume of the journal and the number of volumes published in each year. The type of articles in each journal could play an important role in changing the IF of a journal. Review articles could improve the IF; mostly they attract a greater number of references and more citations than other kinds of articles like original papers.[11,14,15,20] This is why the IFs of Annu Rev Nutr and Prog Lipid Res, which publish review papers, are the highest as compared with other selected journals in this study. As mentioned in the methods section, we used JCR of Thomson Reuter Web of Knowledge for obtaining data. Unfortunately, JCR does not provide any information about type of articles published in the journals. Our statements about the effect of review articles on the IF are based on the knowledge we got from the website of the journals. For example, ANNU REV NUTR is a journal that just publishes review articles, not originals papers. Other journals assessed in our investigation are mostly publishing original papers with one or two review articles in each issue. Another limitation is that the IF can be manipulated by editors and publishers by several ways; top ten of those have been discussed in detail by Falagas and Alexiou.[16] The lack of methodological transparency of ISI-Thompson-Reuters in defining the criteria used to calculate the IF for different journals is another major limitation of IF.[17,18] It must also be kept in mind that the numerator in the IF formula includes all citations (original articles, letters, commentaries, and editorials); while the denominator includes only the “citable” articles, not letters, commentaries, or editorials. Publication of non-original research which is a main characteristic of top journals could affect the IF of the journal. For example, when Golubic et al.[19] considered original articles and citations to them for IF calculation, the IF was decreased for both top-ranked and middle-ranked journals. This is of particular importance when a published letter, commentary, or editorial in a journal contains original research data. Because such publications will be significantly cited, increasing the numerator of the IF formula.
The IF formula reflects the variation of number of citations of journals.[4] One way to increase the IF is self citation. Self citation is an important indicator in the analysis of citations; corresponding to the number of times that journal cites bibliographical references from its own articles.[15–17,21] Although all these valuable criticisms of IF are true, this measure is not a useless bibliometric parameter for evaluation of journals’ quality.
With a bird's eye view of journals’ IF trend during the years, there is an upward way in the most, but not all, stages of science. This is why the selected journals in “Nutrition and Dietetics” category have an increased trajectory. For some journals like Annu Rev Nutr and Prog Lipid Res, there is a fluctuated trend in the last 10-year period; so, one reason of this up and down trend is the time limitation in formula. Expanding the size of time measurement in IF formula could change these ways much smoother.[11] A 2-year temporal window for citation is too short and classic articles are cited frequently even after several decades. Increasing this to a 5-year, for example, would result in a higher number of citations and higher IF.
According to the failure of journals’ IF as a part of citation analysis in scientometric field, scientists try to explore new parameters in order to fill these gaps and measure the impact of each journal in detail. The h-index was proposed in 2005 by Hirsch in his article titled “An index to quantify an individual's scientific research output.”
In this article, h-index was defined as “A scientist has index h if h of his/her number of papers (Np) have at least h citations each, and the other (Np - h) papers have no more than h citation each”.[18,22]
Egghe recommended g-index in order to improve the h-index of Hirsch to measure the citation performance of each article. In this article, he mentioned “If set of articles ranked in decreasing order of number of citations that they received, the g-index is the largest number such that the top g articles received at least g2 citations”.[19,23]
Up to now, no measure of quantifying journal's quality is perfect and each one has its own limitations. In spite of limitations, currently IF is frequently used as a proxy for the relative importance of a journal within its field. However, it must be kept in mind that IF cannot be used as a measure of performance of the journals because journals’ performance includes its review, editorial, and publication processes and IF measures how much the output of these processes is used by the scientific community.
Our study has some limitations too. The five selected journals were purposive and they were not a representative sample of all nutrition and dietetic journals. On the other hand, using IF and total citation count cannot cover all aspects of the journal's quality. Therefore, using some other bibliometric factors in the study might help us to judge better. Furthermore, database of our research was restricted to ISI-indexed journals and we did not include journals indexed in other resources.
In summary, these bibliometric studies are essential tools in consideration and analysis of research performance in each stage of science. Considering the importance of IF as a useful yardstick to compare quality of medical journals, especially in nutrition and dietetics category, we conclude that journals in this category of ISI are of high quality now than before.
ACKNOWLEDGMENT
The authors would like to thank the research council of Food Security and Nutrition Research Center for their support.
Footnotes
Source of Support: Food Security Research Center, Isfahan University of Medical Sciences
Conflict of Interest: None declared.
REFERENCES
- 1.Garfield E. Citation indexes to science: A new dimension in documentation through association of ideas. Science. 1955;122:108–11. doi: 10.1126/science.122.3159.108. [DOI] [PubMed] [Google Scholar]
- 2.Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295:90–3. doi: 10.1001/jama.295.1.90. [DOI] [PubMed] [Google Scholar]
- 3.Ogden TL, Bartley DL. The ups and downs of journal impact factors. Ann Occup Hyg. 2008;52:73–82. doi: 10.1093/annhyg/men002. [DOI] [PubMed] [Google Scholar]
- 4.Kamaruzaman J. In search of best impact factor and citation indexed journals towards achieving the goals of niversities. J Biochem Tech. 2008;1:23–9. [Google Scholar]
- 5.Impact factor. [Last accessed on 2008 Oct 23]. Available from: http://www.thomsonreuters.comproducts_services/science/academic/impact_factor/
- 6.Currie GM, Wheat JM. Impact factor in nuclear medicine journals. J Nucl Med. 2007;48:1397–400. doi: 10.2967/jnumed.107.041491. [DOI] [PubMed] [Google Scholar]
- 7.Saha S. Impact factor: A valid measure of journal quality? J Med Libr Assoc. 2003;91:42–6. [PMC free article] [PubMed] [Google Scholar]
- 8.Foo JY. The retrospective analysis of bibliographical trends for nine biomedical engineering journals from 1999 to 2007. Ann Biomed Eng. 2009 doi: 10.1007/s10439-009-9700-7. [In press] [DOI] [PubMed] [Google Scholar]
- 9.Ha CT, Tan SB, Chee SK. The journal impact factor: Too much of an impact? Ann Acad Med Singapore. 2006;35:911–6. [PubMed] [Google Scholar]
- 10.Ewing J. Measuring Journals. Notices of the American Mathematical Society. 2006;53:1049–53. [Google Scholar]
- 11.Amin M, Mabe MA. Impact factors: Use and abuse. Medicina. 2003;63:347–54. [PubMed] [Google Scholar]
- 12.Holguin JA. Occupational therapy and the journal citation reports: 10-year performance trajectories. Am J Occup Ther. 2009;63:105–12. [PubMed] [Google Scholar]
- 13.Dasit AM, Bonmat’ L, Sanfeliu P. Bibliometric analysis of the Spanish MR radiological production (2001–2007) Eur J Radiol. 2008;67:384–91. doi: 10.1016/j.ejrad.2008.02.042. [DOI] [PubMed] [Google Scholar]
- 14.Van Driel ML, Magin PJ, Del Mar CB, Furler J, De Maeseneer J. Journal impact factor and its importance for AFP. Aust Fam Physician. 2008;37:770–3. [PubMed] [Google Scholar]
- 15.Adler R, Ewing J, Taylor P. Reported of citation statistics. [Last accessed on 2010.03.05]. Available from: http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf .
- 16.Falagas ME, Alexiou VG. The top ten in journal impact factor manipulation. Arch Immunol Ther Exp (Warz) 2008;56:223–6. doi: 10.1007/s00005-008-0024-5. [DOI] [PubMed] [Google Scholar]
- 17.Rossner M, HIll E, Van Epps H. Show me the data. J Exp Med. 2007;204:3052–3. doi: 10.1084/jem.20072544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Rossner M, Van Epps H, Hill E. Irreproducible results: A response to Thomson Scientific. J Exp Med. 2008;1312:183–4. doi: 10.1084/jem.20080053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Golubic R, Rudes M, Kovacic N, Marusic M, Marusic A. Calculating impact factor: How bibliographical classification of journal items affects the impact factor of large and small journals. Sci Eng Ethics. 2008;14:41–9. doi: 10.1007/s11948-007-9044-3. [DOI] [PubMed] [Google Scholar]
- 20.Archambault E, Lariviere V. History of the journal impact factor: Contingencies and consequences. Scientometrics. 2009;79:1–15. [Google Scholar]
- 21.Benavent RA, Alcaide G, Dasit AM, Dios JG, Orive G, Zurian CV. Analysis of citations and national and international impact factor of Farmacia Hospitalaria (2001-2005) Farm Hosp. 2007;31:150–5. doi: 10.1016/s1130-6343(07)75362-x. [DOI] [PubMed] [Google Scholar]
- 22.Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102:16569–72. doi: 10.1073/pnas.0507655102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Egghe L. Theory and practise of the g-index. Scientometrics. 2006;69:131–52. [Google Scholar]