Citation Analysis of the Croatian Medical Journal: the First 15 Years

Abstract

The Croatian Medical Journal (CMJ) is a bimonthly scientific journal, publishing mostly original articles. It is indexed in the Index Medicus/MEDLINE, Current Contents/Clinical Medicine, and the Science Citation Index – expanded.

Since the CMJ’s 15th anniversary in 2007, our aim has been to assess the importance of the journal through its impact factor (IF) and immediacy index, with a particular focus placed on the proportion of self-citations.

According the Web of Knowledge database, the current official IF for CMJ is 0.825, ranking it at the 62nd place out of 103 journals within the Thomson Scientific category “Medicine – General and Internal.” The exclusion of self-citations resulted in a small decrease in the journal’s rank – to 66th place.

According to the Web of Science database, the predicted CMJ IF in 2007 is between 1.024 and 1.125, showing a clear increase. The immediacy index of the CMJ is continuously low, with a high contribution of self-citations, implying that articles published in the CMJ require more time to be cited, and that their topics are of particular interest to the journal's readers and contributors.

Self-citations contributed significantly to the IF in the first few years after the journal was established. The proportion of independent citations progressively increased, and of all the citations included in the IF in 2007, almost 70% were fully independent. Some of these citations were from articles published in journals with IF higher than 5.

Taken together, our data suggest that CMJ has significantly improved its citation data during the last 15 years, confirming that a quality-oriented editorial policy in a small peripheral journal may result in a truly increased international visibility.

The Croatian Medical Journal (CMJ) is a bimonthly peer-reviewed scientific journal, which publishes various types of articles in its ten sections – Cover Page, Editorial, Basic Science, Clinical Science, Public Health, Student CMJ, Medical Education, Columns, Book Reviews, and News and Comments. The full content of the journal is freely available at www.cmj.hr. During the last 5 years, the CMJ has published between 88 and 115 mostly original articles per year.

The journal was established in 1992, included in the Index Medicus/MEDLINE in 1998, and in the Current Contents/Clinical Medicine and Science Citation Index (SCI) – Expanded, under the category “Medicine – General and Internal,” in 1999. Since 2001, the official bibliometric data on the journal have been available in the Journal Citation Reports (JCR), published by Thomson Scientific (formerly the Institute of Scientific Information, ISI), which provides journal citation data, including the impact factor (IF) and immediacy index.

The CMJ’s editorial office continually evaluates the journal quality, which includes assessing the number and structure of citations of published articles. In 2007, the CMJ celebrated its 15th anniversary, which presented an opportunity to assess the past and current importance of the journal, analyzing the changes of its IF and immediacy index. IF is a measure how frequently an “average article” in a journal is cited in a particular year (1). It is calculated by dividing the number of article citations in a year published in the two previous years. Immediacy index measures the frequency of citation of an “average article” in a journal in the same year, and is calculated by dividing the number of current citations of articles published in the same year with the total number of articles in the same year. Since the official JCR data for 2007 are still not published, we calculated the IF and immediacy index manually and found a clearly positive trend in the journal's IF. We separately analyzed the proportion of self-citations and found a continuous decrease in self-citation proportion.

Methods

We accessed the current official data for the CMJ's IF, immediacy index, and journal ranking in the 2006 JCR edition.

We calculated the 2007 IF and immediacy index from the number of citations retrieved by the manual search of the Thomson Scientific databases Science Citation Index (SCI), Social Science Citation Index (SSCI), and Arts & Humanities Citation Index (A&HCI), available at the Web of Science (WoS), as described previously (2). Briefly, to calculate the IF we searched the ISI databases for citations in the 2007 to articles published in the CMJ during the previous two years (2005-2006), and divided the total number of citations with the number of articles published in 2005-2006. To calculate the immediacy index, we searched the ISI databases for citations in 2007 and divided this number with the number of articles published in 2007. We included only articles defined in WoS as “Article” type.

The citations were then analyzed to estimate the proportion of self-citations and independent citations. Citations by the CMJ itself to the articles published in the CMJ were considered as journal self-citations, whereas citations where one of the authors was the author of both the citing and cited articles were considered as author self-citations (2).

To estimate the relevance of citing articles, each citation was then matched with the IF of the journal in which it appeared, and the average IF of the citing journals was calculated by dividing the sum of IFs with the number of citations.

Results and discussion

According to the current official data from the 2006 JCR edition, the CMJ's IF was 0.825, ranking it at the 62nd place out of 103 journals within the ISI category “Medicine – General and Internal” (Table 1). The CMJ's ranking changed from 62nd to 66th place when journal self-citations were excluded from the IF calculation.

Table 1.

Data from the 2006 Science edition of the Journal Citation Reports™ from Thomson Scientific for the journals with impact factor (IF)>10 and journals ranked around the CMJ, sorted by IF and corrected IF (excluding self-citations) for 2006

Journal rank*	Journal rank corrected	Change in rank†	Journal	IF	IF corrected	Immediacy index	Current articles
1	1	0	The New England Journal of Medicine	51.296	50.449	12.743	303
2	2	0	Lancet	25.800	24.899	7.419	301
3	3	0	JAMA	23.175	22.426	7.869	267
4	4	0	Annals of Internal Medicine	14.780	14.261	3.720	164
5	6	-1	PLoS Medicine	13.750	12.519	3.419	148
6	5	+1	Annual Review of Medicine	13.237	13.203	4.343	35
58	54	+4	The National Medical Journal of India	1.000	0.894	0.278	36
59	52	+7	The Southern Medical Journal	0.987	0.936	0.471	155
60	57	+3	Yonsei Medical Journal	0.850	0.823	0.054	130
61	72	-11	Aviation, Space, and Environmental Medicine	0.830	0.549	0.098	163
62	66	-4	Croatian Medical Journal	0.825	0.607	0.435	88
63	74	-11	Wiener Klinische Wochenschrift	0.804	0.534	0.239	117
64	58	+6	Internal Medicine	0.797	0.797	‡	‡
65	62	+3	Annals of the Academy of Medicine, Singapore	0.772	0.716	0.318	132
66	71	-5	Military Medicine	0.747	0.550	0.037	272

*Out of 103 journals in the subject category “Medicine – General and Internal.”

†Negative value means decrease in rank.

‡Not reported in the Journal Citation Reports.

Our analysis showed a positive trend of the CMJ’s IF (Figure 1A). Looking back, the most prominent increase in the CMJ's IF occurred between 2000 and 2003, when it reached the highest official value of 0.943. After this, the IF varied between 0.690 and 0.825. According to the WoS database search, articles published in 2005 and 2006 acquired a total of 216 citations in 2007. Based on this data, the IF for 2007 should be 1.075. In our earlier IF calculations, the data obtained through the WoS search frequently differed from data in the JCR. This difference was always less then 5%, so that we can expect the official IF values in 2007 to lie between 1.024 and 1.125. Such data confirm that the increasing trend of the journal's IF has continued.

Figure 1.

The impact factor (A) and immediacy index (B) of the Croatian Medical Journal.

The immediacy index in general shows greater variability than the IF and it varied from 0.000 to 0.725 over the last 15 years (Figure 1B). The immediacy index, as a parameter showing how often articles published in a journal are cited within the same year, indicates the speed at which citations of a specific journal appear in the literature, and helps to compare the journals in emerging areas of research (3). Since the CMJ frequently publishes thematic issues covering specific topics, variations in the immediacy index may result from the differences in the research fields covered by the theme issues. The immediacy index of the CMJ corresponds to the immediacy index values of the journals grouped around the CMJ in the category “Medicine-General and Internal” (Table 1). The structure of the citations included in the immediacy index calculation shows that the majority of them are journal self-citations. These findings mean that the articles published in the CMJ require more time to be cited, and that topics covered by them are of particular interest to readers and contributors of the CMJ. However, one should keep in mind that the immediacy index does not reflect the visibility of the journal. The more relevant parameter is the journal's IF, because it is calculated for the period when it is expected that the journal will acquire the highest proportion of citations.

The question that often arises when discussing the increase in the CMJ's IF is whether this increase results from independent citations or mainly from journal self-citations. Such self citations reflect the poor international visibility of the journal (4). A relatively high self-citation rate may be influenced by several factors, such as: journal’s having a novel or highly specific focus; few relevant citations available from other sources; sociological factors in the practice of citation (researchers tend to cite and submit their own papers to the journals which they are most aware of); and the editorial practice of the journal (5). Another parameter that we have taken into account were author self-citations. Such self-citations do not necessarily have a negative influence on the journal's international visibility. On the contrary, author self-citations may even increase the journal visibility if they appear in leading high-quality journals. JCR does not exclude author self-citations in other sources when calculating the corrected IF. Such procedure is reasonable because author self-citations reflect citation behavior of the CMJ's authors and cannot be influenced by the editorial policy. In order to estimate the contribution of fully independent citations to the IF of the CMJ, we assessed both author and journal self-citations used for the IF calculation during the past 15 years (Figure 2). It is clearly visible that in the first few years, self-citations contributed significantly (more than 50% of total citations) to the journal's IF. The proportion of fully independent citations has been slowly but progressively increasing since 2001, and reached the stable proportion above 50% after 2004. Of all the citations included in the CMJ's IF in 2007, almost 70% were fully independent. It is important to emphasize that some of these citations were from the articles published in the journals with the IF higher than 5 (Table 2). This illustrates the place of the CMJ as an internationally recognized journal. Although its expected IF for 2007 is about 1, the average IF of the citing journals for each acquired citation is 2.045.

Figure 2.

Proportion of independent citations (closed bars), author self-citations in other journals (gray bars), and journal self-citations (open bars).

Table 2.

Citations to articles published in the Croatian Medical Journal in 2006 and 2007, in journals with impact factor (IF) higher than 5

	Cited article	Authors of the citing article*	Journal	IF
	Web of Science 2007: 21 (9.72%) citations in journals with IF>5; 4 author self-citations
1	Wokaunn M, Juric I, Vrbica Z. Croat Med J. 2006;47:759-66.	Sharp D.	Lancet	25.8
2	Rakic M, Popovic D, Rakic M, Druzijanic N, Lojpur M, Hall BA, Williams BA, Sprung J. Croat Med J. 2005;46:957-63.	van Ruler O, Mahler CW, Boer KR, Reuland EA, Gooszen HG, Opmeer BC, De Graaf PW, Lamme B, Gerhards MF, Steller EP, van Till JWO, de Borgie CJAM, Gouma DJ, Reitsma JB, Boermeester MA.	Journal of the American Medical Association	23.175
3	Hranjec T, Kovac A, Kos J, Mao W, Chen JJ, Grollman AP, Jelakovic B. Croat Med J. 2005;46:116-25.	Grollman AP, Shibutani S, Moriya M.	Proceedings of the National Academy of Sciences of the United States of America	9.643
4	Rudan I, Biloglav Z, Carothers AD, Wright AF, Campbell H. Croat Med J. 2006;47:532-42.	Campbell H, Carothers AD, Rudan I, Hayward, Biloglav Z, Barac L, Pericic M, Janicijevic B, Smolej-Narancic N, Polasek O, Kolcic I, Weber JL, Hastie ND, Rudan P, Wright.	Human Molecular Genetics	8.099
5	Saftic V, Rudan D, Zgaga L. Croat Med J. 2006;47:543-52.		Human Molecular Genetics	8.099
6	Bobinac D, Maric I, Zoricic S, Spanjol J, Dordevic G, Mustac E, Fuckar Z. Croat Med J. 2005;46;389-96.	Buijs JT, Henriquez NV, Van Overveld PGM, Van der Horst G, Que I, Schwaninger R, Rentsch C, Ten Dijke, Cleton-Jansen AM, Driouch K, Lidereau R, Bachelier R, Vukicevic S, Clezardin P, Papapoulos SE, Cecchini MG, Lowik CWGM,Van der Pluijm G .	Cancer Research	7.656
7	Hranjec T, Kovac A, Kos J, Mao W, Chen JJ, Grollman AP, Jelakovic B. Croat Med J. 2005;46:116-25.	Grollman AP, Jelakovic B.	Journal of the American Society of Nephrology	7.371
8	Marusic M, Sambunjak D, Marusic A. Croat Med J. 2006;47:372-5.	Ilakovac V, Fister K, Marusic M.	Canadian Medical Association Journal	6.862
9	Degoricija V, Sharma M, Legac A, Gradiser M, Sefer S, Vucicevic Z. Croat Med J. 2006;47:385-97.	Kiank C, Koerner P, Kessler W, Traeger T, Maier S, Heidecke CD, Schuett C.	Critical Care Medicine	6.599
10	Gatalica Z, Bing Z. Croat Med J. 2005;46:372-6.	Gururajan M, Dasu T, Shahidain S, Jennings CD, Robertson DA, Rangnekar VM, Bondada S.	Journal of Immunology	6.293
11	Parpura V, Chapman ER. Croat Med J. 2005;46:491-7.	Chen S, Kim JJP, Barbieri JTA.	The Journal of Biological Chemistry	5.808
12-18	Alonso A, Martin P, Albarran C, Garcia P, Fernandez de Simon L, Jesus Iturralde M, Fernandez-Rodriguez A, Atienza I, Capilla J, Garcia-Hirschfeld J, Martinez P, Vallejo G, Garcia O, Garcia E, Real P, Alvarez D, Leon A, Sancho M. Croat Med J. 2005;46:540-8.	Brettell TA, Butler JM, Almirall JR.	Analytical chemistry	5.646
	Andelinovic S, Sutlovic D, Erceg Ivkosic I, Skaro V, Ivkosic A, Paic F, Rezic B, Definis-Gojanovic M, Primorac D. Croat Med J. 2005;46:530-9.
	Budimlija ZM, Lechpammer M, Popiolek D, Fogt F, Prinz M, Bieber FR. Croat Med J. 2005;46:549-55.
	Crouse CA, Yeung S, Greenspoon S, McGuckian A, Sikorsky J, Ban J, Mathies R. Croat Med J. 2005;46:563-77.
	Halverson JL, Basten C. Croat Med J. 2005;46:598-605.
	Henke J, Henke L. Croat Med J. 2005;46:593-7.
	Schiffner LA, Bajda EJ, Prinz M, Sebestyen J, Shaler R, Caragine TA. Croat Med J. 2005;46:578-86.
19	Hranjec T, Kovac A, Kos J, Mao W, Chen JJ, Grollman AP, Jelakovic B. Croat Med J. 2005;46:116-25.	Arlt VM, Stiborova M, vom Brocke J, Simoes ML, Lord GM, Nortier JL, Hollstein M, Phillips DH, Schmeiser HH.	Carcinogenesis	5.366
20	Punda A, Polic S, Rumboldt Z, Bagatin J, Markovic V, Lukin A. Croat Med J. 2005;46:219-24.	Karagiannis A, Mikhailidis DP, Kakafika AI, Tziomalos, Athyros VG.	Current Pharmaceutical Design	5.270
21	Muula AS. Croat Med J. 2005;46:21-9.	Abegunde DO, Shengelia B, Luyten A, Cameron A, Celletti F, Nishtar S, Pandurangi V, Mendis S.	Bulletin of the World Health Organization	5.029
	Web of Science 2006: 12 (7.27%) citations in journals with IF>5; 3 author self-citations
1	Fenderson BA, Fenderson DA. Croat Med J. 2004;45:259-63.	Busch R, Byrne B, Gandrud L, Sears D, Meyer E, Kattah M, Kurihara C, Haertel E, Parnes JR, Mellins ED.	Nature Immunology	27.596
2	Marusic A. Croat Med J. 2005;46:1-3.	Clark J, Tugwell P	PLoS Medicine	13.750
3	Bosilkovski M, Krteva L, Caparoska S, Dimzova M. Croat Med J. 2004;45:727-33.	Ruiz CG, Miranda JJ, Pappas G	PLoS Medicine	13.750
4	Calkovska A, Sevecova-Mokra D, Javorka K, Petraskova M, Adamicova K. Croat Med J. 2005;46:209-17.	Westphal M, Traber DL	Critical Care Medicine	6.599
5	Lucin K, Matusan K, Dordevic G, Stipic D. Croat Med J. 2004;45:703-8.	Mangs AH, Speirs HJ, Goy C, Adams DJ, Markus MA, Morris BJ.	Nucleic Acids Research	6.317
6	Samija I, Lukac J, Maric-Brozic J, Kusic Z. Croat Med J. 2004;45:142-8.	Koyanagi K, O'Day SJ, Gonzalez R, Lewis K, Robinson WA, Amatruda TT, Kuo C, Wang HJ, Milford R, Morton DL, Hoon DS.	Clinical Cancer Research	6.177
7	Grdisa M. Croat Med J. 2004;45:136-41.	Mone AP, Cheney C, Banks AL, Tridandapani S, Mehter N, Guster S, Lin T, Eisenbeis CF, Young DC, Byrd JC.	Leukemia	6.146
8	Eminovic I, Komel R, Prezelj J, Karamehic J, Gavrankapetanovic F, Heljic B. Croat Med J. 2005;46:664-9.	Goodarzi MO, Shah NA, Antoine HJ, Pall M, Guo X, Azziz R.	The Journal of Clinical Endocrinology And Metabolism	5,799
9	Husedzinovic I, Barisin S, Bradic N, Barisin A, Sonicki Z, Milanovic R. Croat Med J. 2005;46:950-6.	Baldus S, Müllerleile K, Chumley P, Steven D, Rudolph V, Lund GK, Staude HJ, Stork A, Köster R, Kähler J, Weiss C, Münzel T, Meinertz T, Freeman BA, Heitzer T.	Free Radical Biology & Medicine	5.440
10	Gratwohl A, Baldomero H, Labar B, Apperley J, Urbano-Ispizua A; Accreditation Committee of the European Group for Blood and Marrow Transplantation (EBMT). Croat Med J. 2004;45:689-94.	Gratwohl A, Brand R, Apperley J, Crawley C, Ruutu T, Corradini P, Carreras E, Devergie A, Guglielmi C, Kolb HJ, Niederwieser D; Chronic Leukemia Working Party of the European Group for Blood and Marrow Transplantation.	Haematologica	5.032
11	Muula AS. Croat Med J. 2005;46:21-9.	Muula AS.	Bulletin of the World Health Organization	5.029
12	Parpura V, Chapman ER. Croat Med J. 2005;46:491-7.	Montana V, Malarkey EB, Verderio C, Matteoli M, Parpura V.	Glia	5.013

*Underlined authors are the authors of the cited article.

Despite the fact that the CMJ is a small journal from the so-called scientific periphery, with a relatively narrow readership, it has significantly improved its international visibility during the last 15 years, as is shown by the increase in proportion of international submissions (6), the increase in the number of hits to and downloads from the journal's web page (7), the progressive increase in the journal's IF, and the proportion of citations in high impact journals.

The international visibility and importance is also confirmed by its 34th rank among the top 100 journals indexed in Index Copernicus, a web based directory which performs the analysis of individual scientists, research institutions, and journals (8). The evaluation criteria for journal excellence by this directory include scientific quality, editorial quality, international availability, frequency-regularity-stability, and technical quality. Such evaluation is followed by an expert peer-review, estimating the scientific significance of the published material, up-to-date content, and educational value (9).

The position of a journal from the so-called scientific periphery (10) in the global scientific community is different from that of a prestigious, high impact journal. This is particularly visible in the CMJ's immediacy index, which varies greatly from year to year, depending mainly on the contribution of specific thematic issues, and consisting mostly of self-citations. However, the same variation and similar, or even lower immediacy index is characteristic of other journals with IFs similar to that of the CMJ (Table 1). This clearly shows that an IF-oriented editorial policy in any small journal, including the CMJ, should not override the quality standards in article selection and continuous education of potential contributors, which results in truly increased international visibility in the long run (10).