Abstract
Introduction: The number of publications and how often these have been cited play a role in academic promotion. Bibliometrics that attempt to quantify the relative impact of scholarly work have been proposed. The h-index is defined as the number (h) of publications for an individual that have been cited at least h times. We calculated the h-index and number of publications for academic emergency physicians at the rank of professor.
Methods: We accessed the Society for Academic Emergency Medicine professor list in January of 2012. We calculated the number of publications through Web of Science and PubMed and the h-index using Google scholar and Web of Science.
Results: We identified 299 professors of emergency medicine. The number of professors per institution ranged from 1 to 13. Median h-index in Web of Science was 11 (interquartile range [IQR] 6–17, range 0–51), in Google Scholar median h-index was 14 (IQR 9–22, range 0–63) The median number of publications reported in Web of Science was 36 (IQR 18–73, range 0–359. Total number of publications had a high correlation with the h-index (r=0.884).
Conclusion: The h-index is only a partial measure of academic productivity. As a measure of the impact of an individual's publications it can provide a simple way to compare and measure academic progress and provide a metric that can be used when evaluating a person for academic promotion. Calculation of the h-index can provide a way to track academic progress and impact. [West J Emerg Med. 2014;15(3):290–292.]
INTRODUCTION
The number of publications and how often these have been cited play a role in academic promotion. However, looking only at the number of publications may not provide an accurate measure of the impact or quality of a researcher's work. Bibliometrics that attempt to quantify the relative impact of scholarly work have been proposed. Of these alternative metrics, the h-index is the most widely used and studied.1 The h-index is defined as the number (h) of publications for an individual that have been cited at least h times.1 This attempts to take into account not only the publication output for an individual but also the impact of the publications as measured by the times they have been cited. For example, an individual with an h-index of 10 has ten publications that have each been cited at least 10 times. The h-index for academic physicians in several different medical subspecialties has been published and may start being incorporated as a metric for academic promotion.2–7 The h-index calculation includes all publications regardless of the author position on a particular paper.
METHODS
We accessed the Society for Academic Emergency Medicine (SAEM) professor list (http://stage.saem.org/full-professor-list) in January of 2012. SAEM is the main society for academic emergency physicians in the United States. SAEM keeps a list of emergency physicians at the rank of professor in the United States and Canada and the institution to which they belong. The list contains 312 names from 120 institutions. Three persons were listed twice. Six individuals had names that prevented reliable filtering to ensure accurate publication and h-index calculations and three individuals were deceased. One individual was listed as an assistant professor. For the remaining 299 individuals we calculated the number of publications through Web of Science (http://wokinfo.com) and the h-index using Google scholar (http://scholar.google.com) and Web of Science.
We utilized the author's last name, and first and middle initial as the initial search strategy. This was sometimes combined with a search strategy that did not include a middle initial, as a number of authors did not consistently use their middle initial on their publications. We utilized the Web of Science (WOS) filter functions to restrict the author search to life sciences research and to particular institutions when necessary to refine the search. We used the citation report for WOS, which calculates the h-index and reports the number of publications ascribed to the author and used for the calculation. For google scholar we utilized the same author name strategy. This returned a list of publications with citations by publication. We manually counted publications until reaching the h-index threshold (when publication number equaled citation number).
Data was entered and stored into a Microsoft Excel (Redmond, WA) file. Descriptive statistics were calculated using JMP (SAS Institute Inc., Cary, NC). No institutional review board approval was obtained as this is not a human research study.
RESULTS
We identified 299 professors of emergency medicine. The number of professors per institution ranged from 1 (54 institutions) to 13 (one institution). Median h-index in Web of Science (WOS) was 11 (interquartile range [IQR] 6–17, range 0–51), in Google Scholar median h-index was 14 (IQR 9–22, range 0–63) The median number of publications reported in Web of Science was 36 (IQR 18–73, range 0–359). Total number of publications had a high correlation with the h-index (r=0.884).
The table shows the h-index and number of citations reported for professors of other specialties. A number of the manuscripts reviewed reported mean and not median values. As the values are not normally distributed we think median values and interquartile ranges are a more accurate representation of these values. For comparison with some of the values in the table, the mean h-index for our list of professors was 16.2 in Google Scholar and 12.8 in WOS. The mean number of citations was 57.5 in WOS.
Table.
Publications and h-index reported for a number of specialties
DISCUSSION
Articles from other specialties have looked at the number of publications and h-index for different academic ranks. Although different authors have used different databases and report their numbers in different ways (means v medians) the general conclusion from all these articles is that there is an association between h-index and academic rank. The use of different databases may return different numbers of citations and calculate different h indexes for individuals.8 Different citation counts are returned with Scopus, Google Scholar and Web of Science.8 Both Scopus and Web of Science require a paid subscription. Scopus only includes citations since 1996. Google Scholar is free. We utilized the Web of Science database as it is the database available and licensed for use at our institution and is the database that was utilized for the original h-index calculations by Hirsch.1 We cross referenced this database with Google Scholar for the h-index calculation and PubMed for number of publications to verify that the datasets for a particular author appeared generally concordant and to see if there was a significant difference in the counts returned.
Comparing h-index across specialties may not be reliable as there are factors such as the number of investigators and citations within a field that will influence the number of times a particular article is cited.9 Svider et al found differences in the h-index of a sample of chairpersons of different specialties which they partially ascribe to the size of specialties and the resultant number of specialty journals and size of the audience for the publications.7 The h-index may be more useful to compare individuals in the same field than across fields.2 Hirsch, who proposed the h-index as a measure of scientific output, found it to be a better predictor of future achievement than total citation count and total number of publications.10 When used in this way it can give individuals and institutions an idea of how influential a person's publications are relative to others in the field. The h-index will be affected by how long articles have been published, as time will allow for the accumulation of a greater number of citations.1,2,10
LIMITATIONS
The use of databases to search for an individual's publications may miss articles that should be credited to a specific person. This will in turn affect h-index calculations. Any missed citations will tend to underestimate both the h-index and the total number of publications for an individual. We do not have access to the methodology used by various services to acquire publications and citations or to determine their accuracy. We only looked at the professor rank. We do not know how the h-index affects promotion and tenure and did not correlate these metrics with funding, tenure, age or geography.
We used the SAEM professor list. Academic emergency physicians that are professors but were not included in the list did not form part of our calculations. We do not know how many professors the list misses and how inclusion of these individuals would affect overall calculations.
CONCLUSIONS
The h-index is only a partial measure of academic productivity. It does not take into account other elements that play a role in academic promotion such as education, administration, lecturing and service to the institution. As a measure of the impact of an individual's publications it can provide a simple way to compare and measure academic progress and provide a metric that can be used when evaluating a person for academic promotion. Calculation of the h-index can give both individuals and institutions a way to track academic progress and impact. For academic emergency physicians at the rank of associate professor thinking about promotion this may provide them with a way to compare their academic productivity with those already at the rank of professor.
Footnotes
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
Supervising Section Editor: Mark I. Langdorf, MD, MHPE
Full text available through open access at http://escholarship.org/uc/uciem_westjem
REFERENCES
- 1.Hirsch JE. An index to quantify an individual's scientific research output. Proceedings Natl Acad Sci USA. 2005;102:16569–16572. doi: 10.1073/pnas.0507655102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Spearman CM, Quigley MJ, Quigley MR et al. Survey of the h index for all of academic neurosurgery:another power-law phenomenon? J Neurosurg. 2010;113:929–933. doi: 10.3171/2010.4.JNS091842. [DOI] [PubMed] [Google Scholar]
- 3.Pagel PS, Hudetz JA. An analysis of scholarly productivity in United States academic anaesthesiologists by citation bibliometrics. Anaesthesia. 2011;66:873–8. doi: 10.1111/j.1365-2044.2011.06860.x. [DOI] [PubMed] [Google Scholar]
- 4.Benway BM, Kalidas P, Cabello JM et al. Does citation analysis reveal association between h-Index and academic rank in urology? Urology. 2009;74:30–33. doi: 10.1016/j.urology.2008.10.045. [DOI] [PubMed] [Google Scholar]
- 5.Pagel PS, Hudetz JA. Scholarly productivity of United States academic cardiothoracic anesthesiologists: influence of fellowship accreditation and transesophageal echocardiographic credentials on h-Index and other citation bibliometrics. J Cardiothor Vasc Anesth. 2011;25:761–765. doi: 10.1053/j.jvca.2011.03.003. [DOI] [PubMed] [Google Scholar]
- 6.Rad AE, Brinjiki W, Cloft HJ et al. The H-Index in academic radiology. Acad Radiol. 2010;17:817–821. doi: 10.1016/j.acra.2010.03.011. [DOI] [PubMed] [Google Scholar]
- 7.Svider PF, Choudhry ZA, Baredes S et al. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123:103–106. doi: 10.1002/lary.23569. [DOI] [PubMed] [Google Scholar]
- 8.Kulkarni AV, Aziz B, Sham I et al. Comparisons of citations in Web of Science, Scopus and Google Scholar for articles published in general medical journals. JAMA. 2009;302:1092–1096. doi: 10.1001/jama.2009.1307. [DOI] [PubMed] [Google Scholar]
- 9.Radicchi F, Fortunato S, Castellano C. Universality of citation distributions: toward an objective measure of scientific impact. Proc Natl Acad Sci USA. 2008;105:17268–17272. doi: 10.1073/pnas.0806977105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hirsh JE. Does the h index have predictive power? Proceedings. Natl Acad Sci USA. 2007;104:19193–19198. doi: 10.1073/pnas.0707962104. [DOI] [PMC free article] [PubMed] [Google Scholar]