Rapid increase in departmental scholarly activity independent of residents demonstrates reproducibility and success of intensive research initiative in neurosurgery department

Abstract

Background: Neurological surgery remains one of the most competitive specialties with a match rate of <70%. Historically, medical student performance was gauged through the USMLE Step 1. However, with the recent exam score change, metrics such as recommendation letters, research, and clerkship grades carry increased importance. Research experiences vary greatly between institutions and medical students depend on faculty/resident mentorship in order to facilitate scholarly activity. We previously reported our 2-year intensive research initiative (IRI) in a neurosurgery program. Here we report successful implementation of the IRI in a disparate setting, a department devoid of residents, and demonstrate the IRI's reproducibility with non-resident learners.

Materials & Methods: We compared retrospective data from 2007 to 2020 with the IRI's results during the 2-year study period (July 2020–July 2022).

Results: The IRI resulted in a rapid exponential increase in publications, with medical student led peer-reviewed publications (PRPs) increasing 1000% and pre-residency fellow (PRF) PRPs increasing by 4900%. Learner involvement on PRPs pre-IRI was 31%, increasing to 72% post-IRI implementation.

Conclusions: We present the IRI's success increasing academic productivity despite utilizing only non-resident learners. Students underrepresented in medicine and those at non-tier 1 institutions receive unequal research and clinical opportunities, therefore, prioritizing and providing sufficient opportunities/mentorship is crucial in their success in matching into competitive specialties. Our IRI allows for early faculty/resident student mentorship and gives students more flexibility as it allows medical students at varying stages to participate in research with no set time frame.

1. Introduction

Neurological surgery continues to be one of the most highly competitive surgical subspecialties represented in the National Resident Matching Program (NRMP) with an overall yearly Match rate of only 65%.¹ Historically, stringent standardized measures are used to gauge a medical students’ performance, with one of the most significant metrics being the United States Medical Licensing Examination (USMLE) Step 1 score. However, as Step 1 has become an unscored pass/fail examination, other candidate metrics have increased importance in candidate evaluation, including, letters of recommendation, clerkship grades, class rank (AOA status), and research experiences. Research experience has become one of the more vital ways to select candidates for the most competitive residency programs, including neurological surgery, where programs increasingly factor research experience in interview selection and subsequent applicant ranking. NRMP data from 2018 to 2021 shows that ∼70% of programs use research experiences when considering which applications to interview.2, 3, 4 Furthermore, the research gap between successful Match applicants in selective surgical subspecialties and unmatched applicants has widened dramatically over the past 15 years. For example, in Neurological Surgery, the gap between successful applicants compared to unmatched applicants has increased from 8.0 to 5.5 research experiences, respectively, to 25.5 and 11.7 research experiences, respectively.¹

Unfortunately, medical student research opportunities have significant institutional variance, with students from top NIH-funded institutions and top 20-ranked medical schools matching applicants with higher mean publication volumes, largely due to increased research opportunities.⁵ Similarly, geographical medical school location has been shown to affect publication volume, with the highest publication volumes concentrated in the West Coast and Mid-Atlantic regions.⁵ To maintain a diverse neurosurgical workforce, neurosurgical programs will need to identify qualified students at smaller institutions, necessitating that sufficient research opportunities are made available for these students in order for them to be given the opportunity to excel and compete in the increasingly selective match process for neurological surgery. However, the same disparity exists in plastic surgery, dermatology, and other highly selective surgical subspecialties. For example, the pressure for research experiences in plastic surgery has already caused nearly 25% of applicants to extend their medical education by incorporating a dedicated research year during medical school.⁶

We previously reported our 2-year intensive research initiative (IRI) experience at New York Medical College (NYMC), which aimed to rapidly increase departmental scholarly activity within a neurological surgery department with a residency training program by focusing on resident and medical student mentorship.⁷ That study demonstrated how rapidly programs could exponentially increase their departmental scholarly activity to fulfill the Accreditation Council for Graduate Medical Education (ACGME) core requirement of providing an environment of scholarly activity and research. There were no previous specific guidelines or recommendations for rapidly increasing departmental scholarly activity in order to comply with this ACGME core requirement, so we summarized the limited literature and shared our experience with the IRI as a possible solution for programs that needed rapid academic turnaround. Previous reported interventions consisted predominantly of protected research time, assigning mentors, adding research requirements, and offering incentives, but there was also a plethora of barriers reported, described primarily as limitations in financial resources, dedicated resident training time, mentorship, and faculty interest.

In this study, we share the results of a second implementation of the IRI model, this time at the University of New Mexico (UNM) Neurosurgery Department, which had recently lost their neurological surgery residency training program. We share these results to show how the IRI could even be successfully implemented in a department without residents, by focusing on medical students and pre-resident learners. Importantly, given the high discrepancy in research opportunities between medical schools across the nation, we implemented our IRI in order to help under-represented students and pre-residency fellows match into competitive specialties. This study should provide encouragement and a blueprint, for departments that need to rapidly increase scholarly activity in their department, with or without residents. Through this 2-year report of the IRI at UNM, we hope to provide a template for other neurosurgery groups seeking to increase their departmental scholarly activity by sharing another example of rapid academic success after implementing the IRI.

2. Methods

We previously reported our 2-year intensive research initiative (IRI) experience that rapidly increased departmental scholarly activity within a neurological surgery residency training program by focusing on resident and medical student mentorship.⁷ That study demonstrated how departments could rapidly obtain exponential increases in their departmental scholarly activity, satisfying the residency requirement of providing an environment of scholarly activity and research. There were no previous guidelines, so we summarized the limited literature from other specialties, and shared our IRI experience to offer a possible solution for departments that needed rapid increases in academic productivity.

This second IRI implementation in a different, novel department, occurred when the same senior leadership team, including the senior author of the original IRI paper, were hired to rebuild the Neurological Surgery department at UNM, as Chair and Program Director, and regain accreditation for a neurological surgery residency training program. The goal of this second IRI implementation was to verify if the original IRI success could be reproduced in a new department despite the lack of residents, by focusing on medical students and pre-resident research mentorship. Tracked metrics for learner engagement with the original IRI included peer-reviewed publications, poster presentations, and oral presentations. Our IRI intervention at UNM included the appointment of clinical research faculty, regular research meetings with learners, the use of a departmental project management program, creating a summer research program for medical students, providing an MPH/biostatistician support for learners, and regular research meetings intended to establish regular follow-up on our monthly research goals.

We collected retrospective data from our neurosurgical department between 2007 and 2022. The IRI was implemented from July 1, 2020 to July 31, 2022. This retrospective study was conducted in early 2023. Academic productivity for this time period consisted of peer-reviewed publications (PRP), as the data for oral presentations and poster presentations were missing prior to implementation of the IRI. The accuracy of our analysis was verified with a PubMed query for every department member from 2007 to 2022. Only Pubmed indexed publications were used in our analysis.

Our primary outcome measure was departmental scholarly activity, measured by peer-reviewed publications (PRP). Pre- and post-IRI data was used to evaluate IRI's impact on departmental scholarly activity. Learner involvement on departmental scholarly activity was also measured pre-and post-IRI implementation, with a newly termed “mentorship index”, defined as the overall percentage of PRP that included learners, with learners defined as medical students, pre-residency fellows, and residents (residents only prior to 2020). The mentorship index was calculated for the entire department (Fig. 3A) and also for the most prolific author or authorship groups, both pre- and post-IRI (Fig. 3B). Publications with multiple authors from within the department were only counted once.

Fig. 3.

Mentorship Index, defined as the percentage of overall publications that include Learners, with learners consisting of medical students and pre-resisdency fellows (and residents pre-IRI), calculated as, (Learners on a publication / Total # of publications) x 100, for: a) the entire neurosurgery department's faculty b) the IRI authors (MS & CB) score for their publications compared to the most productive researcher's publications prior to the IRI implementation.

3. Results

A total of 6 PRF's were recruited during this timeframe and all 6 participated in departmental scholarly activity to varying degrees. Similarly, medical student involvement ranged between 10 and 20 active students. The cumulative annual departmental scholarly activity, represented as PRP, can be seen in Fig. 1, with the red line denoting the implementation of the IRI in July 2020. PRPs varied greatly before IRI implementation, ranging from 2 to 13 PRPs per year. Following full IRI implementation, PRPs increased over 350% (Fig. 1), from 2019 (13 PRPs) to 2022 (59 PRPs). PRPs for learners also dramatically increased, prior to IRI implementation (2007–July 2020), PRPs for learners were on average 3.8/year with an average of 8.6 total departmental PRPs/year (Fig. 2).

Fig. 1.

Annual departmental research products from the University of New Mexico between 2007 and 2022, with intensive research initiative (IRI) implementation July 2020.

Fig. 2.

Annual departmental research products for medical students and pre-residency fellows from the University of New Mexico between 2007 and 2022 with intensive research initiative (IRI) implementation July 2020.

Following IRI implementation, learner led publications dramatically increased to 42 publications/year, resulting in a 1000% increase in PRP/year, with the majority of these PRPs being led by medical students. Among pre-residency fellows (PRFs), prior to IRI implementation (2007–July 2020), PRPs for PRFs were <1, with 4 total PRPs among PRFs for those years (Fig. 2). Following IRI implementation, PRPs/year for PRFs increased to 15 PRPs/year, resulting in a 4900% increase in PRP/year. Learner involvement in PRPs increased 41% over the course of the IRI; with 72% of PRPs including learners over the 2-year IRI, while only 31% of PRPs had learner involvement in the two years prior to IRI implementation (Fig. 3a). Similarly, after adjusting for clinical research faculty who were actively engaged with learners, the mentorship index, or the overall percentage of departmental PRP that included learners, increased by 62%, with 94% of publications in this group having learner involvement, compared to 32% 2-years prior to IRI implementation (Fig. 3b). Thus far, 75% (12/16) of non-resident learners matched into their desired specialties. All medical students within this group (10) matched into their desired specialty, including, Neurological Surgery, Plastic surgery, Diagnostic Radiology, Orthopaedic Surgery, and Obstetrics and Gynecology. Our three medical students within this group who are underrepresented in medicine all matched into their desired specialties (Neurological Surgery, Plastic Surgery, and Obstetrics and Gynecology).

4. Discussion

Implementation of the previously described Intensive Research Initiative (IRI) into a different, novel, neurological surgery department was highly successful despite not having residents. The rapid exponential increase in departmental scholarly activity in two years was unprecedented. We had a 350% increase in medical student participation in departmental scholarly activity, and a 4900% increase in PRF involvement in PRP/year. These current results validate our prior study, which showed that IRI implementation can rapidly produce an exponential increase in departmental scholarly activity at an astonishing rate not previously reported in the literature.⁷ This rapid exponential growth in departmental scholarly activity at UNM was foundational to our successful neurological surgery residency application.

With increasing emphasis being placed on research productivity when applying for competitive specialties, including neurological surgery, many medical students eagerly pursue research opportunities. However, no established systemic structure exists for medical student involvement in research and medical student participation in scholarly activity varies greatly between institutions, as research availability and medical student involvement on publications are unequally distributed.⁸ Prior reports have shown students who participate in research years, extending their medical education, have higher success in matching into various competitive surgical subspecialties, including otolaryngology, plastic surgery, and orthopedic surgery.^6,9,10 Medical students at top-tier and highly funded NIH institutions are more likely to have publications and are more likely to participate in research fellowships, likely secondary to increased opportunities.^5,11,12 Similarly, students with higher first-author publications, total publications, and higher Hirsch indices (H-index; or the measure of productivity and citation impact of publications) are more likely to match at higher ranking institutions.^13,14

A study by Kashkoush et al evaluating neurosurgery applicants, demonstrated that applicants’ H-index serves as the most powerful research predictor in successfully matching into Neurosurgery¹³ They concluded that students should begin research projects early in their medical school careers to increase their odds in the neurosurgical Match.¹³ However, students of underrepresented backgrounds or students at non-tier 1 programs may suffer due to lack of available opportunities. Students underrepresented in medicine frequently cite lack of early faculty mentorship, resources, and both clinical/research opportunities as reasons for not pursuing competitive surgical specialties.^15,16 In order to increase diversity among surgical specialties, prioritizing and providing sufficient opportunities and mentorship for students underrepresented in medicine and those at non-tier 1 institutions, are crucial in their success in matching into these competitive specialties. Implementation of our IRI allows for early faculty and resident mentorship and provides more flexibility than summer or research fellowship programs, as it allows medical students at varying stages to participate in research with no set time frame.⁷ Similarly, our IRI allows continuous medical student involvement in departmental scholarly activity, thereby providing consistent research projects for medical students while integrating them into the department. We hope this IRI can provide a blueprint for any academic department that is looking to increase departmental scholarly activity and provide more opportunities for medical students to perform research, in particular with the emphasis on diversity, equity, and inclusion, all academic departments should take note of the rapid exponential departmental scholarly activity programs can achieve with medical students alone serving as primary authors.

5. Limitations

Although the results of our IRI are rapid, produce exponential growth, and have now been demonstrated to be reproducible, we are unable to report the long-term or post-IRI follow-up at this institution at this time. We will continue to track our IRI results at the 5 and 10-year marks to assess for the IRI's durability and will plan on reporting our results at that time. Research type also affects IRI success as all publications during the IRI consisted of clinical research and did not include basic science research. Lastly, departmental expectations and incentives may need to change for departments to be successful in achieving rapid exponential departmental scholarly activity growth with IRI implementation, as providing faculty involvement and mentorship is crucial for learners to enjoy success.

6. Conclusions

Our neurosurgery department implemented a rapid 2-year intensive research initiative (IRI) aimed at increasing departmental scholarly activity through pre-residency fellow and medical student faculty mentorship. Our IRI resulted in a 350% increase in medical student and 4900% increase in PRF scholarly activity and this is the most rapid report of successful exponential departmental scholarly activity growth reported in the literature to date.⁷ Though long-term IRI follow-up is warranted, we show our research initiative has now been effective and reproducible in two different institutions, both with, and without, a neurological surgery residency training program. Institutions committed to successful medical student mentorship must find ways to get students opportunities for rich academic productivity and the IRI can provide a framework for motivated faculty and learners to rapidly increase departmental scholarly activity.

FUNDING

The authors did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Ethical approval

Institutional Review Board approval was not sought or obtained for this study.

CRediT authorship contribution statement

Samantha Varela: Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Formal analysis, Data curation. Meic H. Schmidt: Writing – review & editing, Supervision, Resources, Project administration. Christian A. Bowers: Writing – review & editing, Visualization, Validation, Supervision, Resources, Project administration, Methodology, Formal analysis, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Abbreviation

ACGME

Accreditation Council for Graduate Medical Education

AOA

alpha omega alpha

DSA

departmental scholarly activity

H-index

Hirsch index

IRI

intensive research initiative

NRMP

national resident match program

NYMC

New York Medical College

PRF

pre-residency fellow

PRP

peer-reviewed publications

UNM

University of New Mexico

USMLE

united states medical licensing exam