Abstract
Background
Scholarly productivity is an assessment metric for dermatology residents and faculty. How the bibliometric h-index, a publicly available metric that incorporates the quantity and quality of publications, relates to early career choices of dermatologists has not been investigated.
Objective
We determined the h-indices of the 2017 diplomates of the American Board of Dermatology to ascertain its association with career choice.
Methods
A cross-sectional analysis was performed using the published list of the 2017 diplomates. Gender and PhD status were compiled. The Scopus database was queried for publications and h-indices. The primary outcome was the pursuit of an academic position, nonacademic position, or fellowship after board certification.
Results
Among 475 (96%) diplomates, the median (range) h-index was 2 (0–14). Those with MD and PhD degrees had greater h-indices (6.4 ± 3.1 vs. 2.3 ± 2.3, P < .05). There was a difference (P < .05) in h-index between diplomates pursuing an academic position (3.6 ± 3.1), non-procedural fellowship (3.3 ± 3.1), procedural fellowship (2.5 ± 2.0), and non-academic position (2.1 ± 2.1).
Conclusions
The h-index quantifies academic productivity and may predict early career choices in dermatology.
What was known and gap
The h-index, a metric that incorporates the quantity and quality of an author's scientific contributions, has been evaluated as a predictor of career choices in radiation oncology, but could be useful in dermatology and other specialties that use scholarly productivity as an assessment for residents and faculty.
What is new
An assessment of h-index data among a cohort of recently board-certified dermatologists and its association with the choice to pursue specific positions or fellowships.
Limitations
The h-index may undervalue more recent publications and does not account for each author's relative contribution.
Bottom line
The h-index may have predictive value in the early career choices of dermatologists.
Introduction
In the assessment of residency and fellowship applicants, program directors often consider academic productivity as a measure of applicant curiosity and the potential for future contributions to the specialty. The most recent Program Director Survey of the National Resident Matching Program showed that “demonstrated involvement and interest in research” is considered an important factor in developing the rank list.1 However, there are few studies that examine how scholarly productivity is best measured and how it translates into early career choices.
The h-index, developed by Jorge E. Hirsch in 2005, is determined by counting the number of publications for which an author has been cited at least that same number of times.2 As the metric incorporates both the quantity and quality of an author's scientific contributions, its utility as a benchmark for academic promotion3,4 and comparative productivity in various academic disciplines and medical specialties5–9 have been explored at the individual, faculty,10,11 and national levels.12 At least one study demonstrated the superiority of the h-index over total publication count and citations per paper in predicting future scientific achievement.13
Although the h-index has been evaluated specifically as a predictor of career choices only in radiation oncology,14 previous studies have shown a relationship between scholarly productivity and an academic career in other specialties such as urology,15 neurological surgery,16 and emergency medicine.17 Understanding these associations in dermatology may prompt other specialties to consider the h-index as a component of their program selection metrics. The objective of this study is to assess h-index data among a cohort of recently board-certified dermatologists and its association with the choice to pursue either an academic position, non-academic position, or fellowship.
Methods
Using the list of the 2017 diplomates published by the American Board of Dermatology (ABD), an Internet search and telephone calls to residency coordinators were performed to ascertain career choices immediately after board certification and demographic data, including gender and PhD degree status. The Scopus citation database was used to collect the publication count and calculate the h-index for all subjects.18 Board certification examinations are administered shortly after graduation from a 3-year dermatology residency; upon successful completion, diplomate status is awarded by the ABD. In most cases, fellowship is started immediately after residency. Publication count, which is likely the most common metric of assessing scholarly activity on a residency or fellowship application, was collected for comparison to the h-index. Cross-referencing and confirmation of publications was performed using the PubMed database and public academic profiles. Subjects with incomplete or unverifiable data were excluded.
We defined 4 career choice groups: academic position, non-academic position, procedural fellow, and non-procedural fellow. An academic position was defined as an attending faculty position at a program certified by either the Accreditation Council for Graduate Medical Education (ACGME) or the Royal College of Physicians and Surgeons of Canada. Positions in private practice or integrated managed care were considered non-academic. Procedural fellowships included micrographic surgery and dermatologic oncology (Mohs) fellowships accredited by the American College of Mohs Surgery, cosmetic dermatologic surgery fellowships accredited by the American Society for Dermatologic Surgery (ASDS), procedural dermatology fellowships accredited by the ACGME, and independent fellowships identified by the ASDS. In our cohort, non-procedural fellowships included pediatric dermatology, dermatopathology, complex medical dermatology, and rheumatology/dermatology.
The study protocol was granted exemption status by the Institutional Review Board of the Albert Einstein College of Medicine.
Statistical analysis was performed using unpaired t tests and ANOVA tests for continuous variables and chi-square tests for categorical variables when applicable with statistical significance considered at the P < .05 level (GraphPad Prism 7, San Diego, CA).
Results
Of the 494 diplomates listed by the ABD in 2017, 475 (96%) had complete and verifiable demographic, professional, and bibliometric records that were available for analysis; 19 (44%) were excluded due to incomplete records. A total of 298 diplomates (63%) were female and 177 (37%) were male. A PhD degree had been completed by 31 (77%) diplomates. Career choices included non-academic position (248, 52%), academic position (104, 22%), procedural fellowship (80, 17%), and non-procedural fellowship (43, 9%).
The median (range) h-index for the included diplomates was 2 (0–14; Figure 1). The median (range) publication count for the included diplomates was 5 (0–73; Figure 2). Two hundred seventy-five diplomates (56%) had an h-index ≥ 2, while 27 (6%) had an h-index ≥ 8; 85 (18%) had an h-index of 0. Diplomates with a PhD had higher h-indices (6.4 ± 3.1 vs. 2.3 ± 2.3, P < .05). No statistically significant difference between male and female diplomates was demonstrated (2.9 ± 2.8 vs. 2.4 ± 2.3, P = .06).
Figure 1.
Overall Distribution of H-Index Among the 2017 ABD Diplomates
Figure 2.
Overall Distribution of Publication Count Among the 2017 ABD Diplomates
There was a statistically significant difference in average h-index between diplomates pursuing an academic position (3.6 ± 3.1), non-procedural fellowship (3.3 ± 3.1), procedural fellowship (2.5 ± 2.0), or non-academic position (2.1 ± 2.1) immediately after board certification (P < .05). Among the 85 diplomates with an h-index of 0, 53 (62%) pursued a non-academic position; among the 79 diplomates with an h-index of ≥ 4, 47 (39%) pursued a non-academic position. The relative proportion of diplomates pursuing each of the career choices, stratified by h-index, is depicted in Figure 3.
Figure 3.
Relative Proportion of Diplomates Pursuing Academic Versus Non-Academic Positions, Stratified by H-Index
There was a statistically significant difference in publication count between diplomates pursuing an academic position (8.8 ± 10.0), non-procedural fellowship (8.2 ± 8.0), procedural fellowship (6.3 ± 4.4), or non-academic position (4.4 ± 4.0) immediately after board certification (P < .05). Among diplomates with a publication count of 0, 55 (73%) pursued a non-academic position; among the 196 diplomates with a publication count of ≥ 6, 28 (40%) pursued a non-academic position. The relative proportion of diplomates pursuing each of the career choices, stratified by publication count, is depicted in Figure 4.
Figure 4.
Relative Proportion of Diplomates Pursuing Academic Versus Non-Academic Positions, Stratified by Publication Count
Discussion
Program directors cite academic productivity as a contributing factor in the assessment of a candidate for a residency or fellowship position,1,19,20 as there is evidence that it predicts future academic performance.21 We demonstrate that higher h-indices are associated with the pursuit of an academic career immediately after board certification. In contrast, diplomates with lower h-indices may be more likely to pursue non-academic positions in their early careers. These findings corroborate prior studies that have demonstrated a relationship between scholarly productivity and an academic career in other specialties.15–17
In dermatology, the relationship between the h-index and the choice to pursue fellowship appears to be more complex. Graduates of non-procedural dermatology fellowships, such as pediatric dermatology, often pursue academic positions.22 This career intention and interest may be related to the relatively higher h-indices in this cohort. However, the relationship between the h-index and the pursuit of procedural fellowships is less clear. A recent study showed that less than 30% of surveyed Mohs surgeons were employed in full-time academic positions.23 As such, perhaps the desire to pursue a non-academic Mohs career explains the relatively lower h-indices.
The h-index of a residency or fellowship program candidate, unlike a submitted curriculum vitae (CV), is standardized and more readily verifiable in many online bibliometric databases; therefore, the use of this metric may expedite the review process. Furthermore, in contrast to publication count, the h-index may also mitigate the issue of “phantom” publications24—research that is listed on an applicant's CV but never published—which has previously been shown in up to 24% of applicants who cited publications at a single residency program.25 A similar single institution study showed that 22% of radiation oncology residency applicants misrepresented publications.26 Additional studies are needed to further clarify the extent of this problem in other specialties.
We also found that the publication count demonstrated predictive value in the early career choices of recent dermatology residency graduates. Most program directors, however, will not take the time to verify each item on every CV, especially in the review of numerous applications during the interview process. Moreover, a cursory review of the volume of publications may be misleading, with regards to the impact of the work. A recent study showed that an increasing number of case reports was negatively associated with academic institution employment, whereas first author and high-impact publications predicted not only an early academic career, but also post-residency productivity.27 The use of the h-index incorporates publication impact; therefore, it may serve as an additional tool to help interpret a candidate's likelihood for future scholarly pursuits.
This study involves limitations. Despite our meticulous efforts to reconcile discrepancies in the Scopus database, it is possible that some inaccuracies persist. The current study design cannot distinguish research primarily performed during medical school—which may be performed to enhance one's chances of acceptance to a dermatology residency—from that performed during residency. In addition, the opportunities to teach, pursue more specialized niches in dermatology, and various lifestyle considerations are also important motivating factors in choosing an academic career. Furthermore, the h-index itself encompasses several limitations. Foremost, the metric ignores author placement; as the order of authorship is dictated by the relative contributions to the project, it is possible that an author has a high h-index while making small contributions to many frequently cited publications. Lastly, as time is required for a publication to accumulate citations, subjects with longer careers in academics prior to board certification, including those with PhD degrees, are potentially more likely to have higher h-indices.28
As there is an association between increasing scholarly activity and the pursuit of an academic career, it is conceivable that programs that promote mentorship and cultivate curiosity for research may lead to a greater number of trainees inspired to remain in an academic environment. Dedicated research time has been described as the sole predictor of scholarly productivity among a cohort of residents.7 Future studies should aim to investigate the longitudinal role scholarly activity plays in the continued evolution of medical careers.
Conclusions
The average 2017 diplomate of the ABD has published a minimum of 2 manuscripts cited at least 2 times. This study demonstrates an additional tool for evaluating dermatology resident academic productivity and indicates that the h-index may have predictive value in the early career choices of dermatologists.
References
- 1.National Resident Match Program. Results of the 2018 NRMP Program Director Survey. 2020 https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf Accessed June 24.
- 2.Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–16572. doi: 10.1073/pnas.0507655102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lopez J, Susarla SM, Swanson EW, Calotta N, Lifchez SD. The association of the h-index and academic rank among full-time academic hand surgeons affiliated with fellowship programs. J Hand Surg Am. 2015;40(7):1434–1441. doi: 10.1016/j.jhsa.2015.03.026. [DOI] [PubMed] [Google Scholar]
- 4.Gast KM, Kuzon WM, Jr, Waljee JF. Bibliometric indices and academic promotion within plastic surgery. Plast Reconstr Surg. 2014;134(5):e838–e844. doi: 10.1097/PRS.0000000000000594. [DOI] [PubMed] [Google Scholar]
- 5.Benway BM, Kalidas P, Cabello JM, Bhayani SB. Does citation analysis reveal association between h-index and academic rank in urology? Urology. 2009;74(1):30–33. doi: 10.1016/j.urology.2008.10.045. [DOI] [PubMed] [Google Scholar]
- 6.Choi M, Holliday EB, Jagsi R, Wilson LD, Fuller CD, Thomas CR., Jr Citation-based estimation of scholarly activity among domestic academic radiation oncologists: five-year update. J Radiat Oncol. 2014;3(1):115–122. doi: 10.1007/s13566-013-0103-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gutovich JM, Den RB, Werner-Wasik M, Dicker AP, Lawrence YR. Predictors of radiation oncology resident research productivity. J Am Coll Radiol. 2013;10(3):185–189. doi: 10.1016/j.jacr.2012.06.036. [DOI] [PubMed] [Google Scholar]
- 8.Rana S, Holliday EB, Jagsi R, Wilson LD, Choi M, Thomas CR, Jr, et al. Scholastic activity among radiation oncology residents at US academic institutions: a benchmark analysis. J Cancer Educ. 2013;28(3):541–546. doi: 10.1007/s13187-013-0500-2. [DOI] [PubMed] [Google Scholar]
- 9.Pagel PS, Hudetz JA. Scholarly productivity of United States academic cardiothoracic anesthesiologists: influence of fellowship accreditation and transesophageal echocardiographic credentials on h-index and other citation bibliometrics. J Cardiothorac Vasc Anesth. 2011;25(5):761–765. doi: 10.1053/j.jvca.2011.03.003. [DOI] [PubMed] [Google Scholar]
- 10.Holliday EB, Jagsi R, Wilson LD, Choi M, Thomas CR, Jr, Fuller CD. Gender differences in publication productivity, academic position, career duration, and funding among U.S. academic radiation oncology faculty. Acad Med. 2014;89(5):767–773. doi: 10.1097/ACM.0000000000000229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Zhang C, Murata S, Murata M, Fuller CD, Thomas CR, Jr, Choi M, et al. Factors associated with increased academic productivity among US academic radiation oncology faculty. Pract Radiat Oncol. 2017;7(1):e59–e64. doi: 10.1016/j.prro.2016.06.012. [DOI] [PubMed] [Google Scholar]
- 12.Tasli L, Kacar N, Aydemir EH. Scientific productivity of OECD countries in dermatology journals within the last 10-year period. Int J Dermatol. 2012;51(6):665–671. doi: 10.1111/j.1365-4632.2011.05112.x. [DOI] [PubMed] [Google Scholar]
- 13.Hirsch JE. Does the H index have predictive power? Proc Natl Acad Sci U S A. 2007;104(49):19193–19198. doi: 10.1073/pnas.0707962104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.McClelland S, 3rd, Jaboin JJ. The relationship between preresidency peer-reviewed publications and subsequent citation-based scholarly activity of United States radiation oncology residents. Int J Radiat Oncol Biol Phys. 2018;102(3):666–668. doi: 10.1016/j.ijrobp.2018.06.026. [DOI] [PubMed] [Google Scholar]
- 15.Thompson RH, Lohse CM, Husmann DA, Leibovich BC, Gettman MT. Predictors of scholarly productivity, pursuit of fellowship, and academic practice among urology residents using medical student application materials. Urology. 2018;120:49–55. doi: 10.1016/j.urology.2018.07.016. [DOI] [PubMed] [Google Scholar]
- 16.McClelland S., 3rd Pre-residency peer-reviewed publications are associated with neurosurgery resident choice of academic compared to private practice careers. J Clin Neurosci. 2010;17(3):287–289. doi: 10.1016/j.jocn.2009.07.098. [DOI] [PubMed] [Google Scholar]
- 17.Stern SA, Kim HM, Neacy K, Dronen SC, Mertz M. The impact of environmental factors on emergency medicine resident career choice. Acad Emerg Med. 1999;6(4):262–270. doi: 10.1111/j.1553-2712.1999.tb00387.x. [DOI] [PubMed] [Google Scholar]
- 18.Scopus. Search for an author profile. 2020 https://www.scopus.com/freelookup/form/author.uri Accessed May 18.
- 19.Weissbart SJ, Stock JA, Wein AJ. Program directors' criteria for selection into urology residency. Urology. 2015;85(4):731–736. doi: 10.1016/j.urology.2014.12.041. [DOI] [PubMed] [Google Scholar]
- 20.Al Khalili K, Chalouhi N, Tjoumakaris S, Gonzalez LF, Starke RM, Rosenwasser R, et al. Programs selection criteria for neurological surgery applicants in the United States: a national survey for neurological surgery program directors. World Neurosurg. 2014;81(3–4):473–477.e2. doi: 10.1016/j.wneu.2013.08.019. [DOI] [PubMed] [Google Scholar]
- 21.Rezek I, McDonald RJ, Kallmes DF. Pre-residency publication rate strongly predicts future academic radiology potential. Acad Radiol. 2012;19(5):632–634. doi: 10.1016/j.acra.2011.11.017. [DOI] [PubMed] [Google Scholar]
- 22.Craiglow BG, Resneck JS, Jr, Lucky AW, Sidbury R, Yan AC, Resnick SD, et al. Pediatric dermatology workforce shortage: perspectives from academia. J Am Acad Dermatol. 2008;59(6):986–989. doi: 10.1016/j.jaad.2008.08.036. [DOI] [PubMed] [Google Scholar]
- 23.Zhang S, Mina MA, Brown MD, Zwald FO. Retention of Mohs surgeons in academic dermatology. Dermatol Surg. 2015;41(8):903–912. doi: 10.1097/DSS.0000000000000431. [DOI] [PubMed] [Google Scholar]
- 24.Chung CK, Hernandez-Boussard T, Lee GK. “Phantom” publications among plastic surgery residency applicants. Ann Plast Surg. 2012;68(4):391–395. doi: 10.1097/SAP.0b013e31823d2c4e. [DOI] [PubMed] [Google Scholar]
- 25.Nasseri Y, Kohanzadeh S, Murrell Z, Berel D, Melmed G, Fleshner P. Phantom publications among applicants to a colorectal surgery residency program. Dis Colon Rectum. 2011;54(2):220–225. doi: 10.1007/DCR.0b013e3181fb0e7a. [DOI] [PubMed] [Google Scholar]
- 26.Yang GY, Schoenwetter MF, Wagner TD, Donohue KA, Kuettel MR. Misrepresentation of publications among radiation oncology residency applicants. J Am Coll Radiol. 2006;3(4):259–264. doi: 10.1016/j.jacr.2005.12.001. [DOI] [PubMed] [Google Scholar]
- 27.Stephens MR, Barbieri J, Lipoff JB. Predicting future dermatology academic productivity from medical school publications. 2019] doi: 10.1016/j.jaad.2019.08.076. [published online ahead of print September 6. J Am Acad Dermatol. doi: [DOI] [PubMed]
- 28.Yuan JT, Aires DJ, DaCunha M, Funk K, Habashi-Daniel A, Moore SA, et al. The h-index for associate and full professors of dermatology in the United States: an epidemiologic study of scholastic production. Cutis. 2017;100(6):395–398. [PubMed] [Google Scholar]