Abstract
Context
This study explores the long‐term impact of the Professional Student Mentored Research Fellowship (PSMRF) program at the University of Kentucky College of Medicine (UKCOM) on medical students’ research productivity and career paths.
Methods
Demographic characteristics, academic profiles, number of publications and residency placements from 2007 to 2012 were used to assess 119 PSMRF graduates against a comparison cohort of 898 UKCOM (non‐PSMRF) students.
Results
PSMRF students had higher MCAT scores at admission (31.5 ± 0.6 vs. 30.6 ± 0.2, p = 0.007) and achieved higher USMLE Step 1 scores (228 ± 4.2 vs. 223 ± 1.5, p = 0.03) than comparison group. PSMRF students were more likely to publish PubMed‐indexed papers (36.7% vs. 17.9%, p < 0.0001), achieve AOA status (19.3% vs. 8.5%, p = 0.0002) and match to top 25 US News and World Report residency programs (23.4% vs. 12.1%, p = 0.008). A greater proportion of PSMRF fellows matched to top tier competitive specialties (23% vs. 14.2%, p = 0.07), however this difference was not statistically significant.
Conclusions
The PSMRF program shows a significant increase in enrollment, as well as positive associations with indicators of success in medical school and subsequent quality of residency program.
Keywords: outcomes research, student research, mentoring
Introduction
The advancement of scientific knowledge has been the foundation of the academic discipline of medicine and one of the core tenets of medical practice. From the development of new surgical techniques to the creation of novel pharmaceuticals, physicians have long been drivers of the innovative science that has shaped modern healthcare. Due to the extent of their patient interaction, physicians are uniquely positioned to pinpoint specific needs in clinical practice and to use their scientific training to develop targeted solutions. However, despite the rich legacy left by pioneering physicians, the same dedication to research appears less common among today's clinicians. In a National Institutes of Health (NIH) progress report on clinical research, Nathan and Varmus described several of the contributing factors to the decline in physician‐scientists over the last decade—including: (1) length and cost of clinical training; (2) increased complexity of clinical and basic science; (3) burdensome regulations of patient‐oriented research; (4) long work hours associated with combined administrative/clinical responsibilities; (5) intense competition for research grants; and (6) inadequate enthusiasm.1 These factors, coupled with limited institutional research‐oriented resources and guidance during medical school, pose major barriers for the development of physician‐scientists.
Realizing this deficit in training, many institutions have designed cocurricular research programs to bolster medical students’ research interests and skills. Such Medical Student Research Fellowship (MSRF) programs are targeted to provide students with concrete incentives to develop research skills (e.g., guided skill‐building, faculty mentorship, formal presentation opportunities), and provide stipends to offset some of the financial burden of graduate training. Many of these programs are able to provide the additional research experience without lengthening the overall time of training—alleviating fears of delayed graduation or incumbent debt.2 In fact, based on documented increases in students’ interests in academic medicine and research careers, Solomon et al. conclude that MSRFs should be a primary strategy to reverse the decline in the number of physician‐scientists.3
Among existing MSRFs, those that incorporate both didactic teaching and mentored research projects boast high level of satisfaction and positive impact on career interests in clinical research. One such example, the Doris Duke Clinical Research Fellowship Program (DDCRF), was initiated in 2000 and is currently based at 10 US medical schools. A study showed 99% of their fellows stated that they had a positive research experience and 87% reported an increased commitment to clinical research careers.4 However, MSRFs similar to DDCRF require the medical students to take a year off their medical studies to enroll in the program. Even highly motivated and interested students are often hesitant to delay their graduation from medical school to pursue such training. Furthermore, these fellowships are highly competitive and not available in many universities.
The Professional Student Mentored Research Fellowship (PSMRF) at the University of Kentucky College of Medicine (UKCOM) was initiated in 2003 as a pilot program for students in various health professions who are seeking exposure to the research process. Its main goal is to enable students to engage in a meaningful way in research while not extending their professional schooling years. Its only prerequisites are completion of an innovative 12‐session introductory course titled “Introduction to Clinical Research” (ICR, Table 1) and good academic standing. Following completion of the ICR course, students, with program assistance, identify a mentor, develop a hypothesis‐driven proposal, and apply for enrollment in the PSMRF program. Once accepted as fellows, students are required to attend core seminars and the College‐wide Dean's Research Lecture Series, as well as participate in an annual Center for Clinical and Translational Science (CCTS) research conference at the University of Kentucky (Figure 1). The program goals are for fellows to inculcate an understanding of directed research design and execution, provide connections to explore academic career paths in various specialties, develop comfort with all phases of the research process and, above all, provide fellows with “hands‐on” research experience through faculty mentorship. Repeatedly throughout the program, fellows engage in self‐reflection and, in turn, are evaluated by their mentors. Subject to satisfactory progress, a $3,000 stipend is awarded to fellows in three installments over the length of the program (12–18 months). The program is typically completed during the fellows’ M2 year of medical school, but a small number participate during M3. As per design, the PSMRF adds no additional time to medical training.
Table 1.
Example of topics discussed during Introduction to Clinical Research course
|
Figure 1.
Descriptive diagram of PSMRF program.
The aim of this study is to explore specific and longer term effects of the PSMRF program both during and after medical school. We also examine the demographic profiles of PSMRF participants as compared to their classmates, and track traditional indicators of academic success.
Methods
Analyses of ICR course enrollment and applications to the PSMRF program were conducted using data from all medical students who attended UKCOM between 2007 and 2014. With regard to analysis of the PSMRF program and its association with markers of academic achievement, the study population included 119 medical students who completed PSMRF from 2007 to 2012 and a comparison group consisting of their respective cohort of 898 UKCOM matriculates who did not participate in the fellowship. Data were collected to document the demographic characteristics, academic profiles, and residency placements of UKCOM medical students during this time period. Demographic variables included age, gender, undergraduate GPA (science and nonscience) and Medical College Admission Test (MCAT) score (total and subscales). Academic profile variables included publication record, Alpha Omega Alpha (AOA) academic honor society status, and Steps 1 and 2 (CK) of the United States Medical Licensing Exam (USMLE). Finally, residency placement variables included academic medical centers (AMC) versus community hospital status, residency program rank (as defined by U.S. News and World Report, USNWR), and specialty “tier.”5 “Tiers” were based on 2014 results from the National Residency Match Program (NRMP), and calculated as the ratio of total positions offered in a specialty to the number of US seniors for whom that specialty was the first or only choice. Tier 1 was limited to specialties offering 0.1–0.9 spots per US senior and included Dermatology, Internal Medicine/Emergency Medicine (Dual), Neurosurgery, Orthopedic Surgery, Otolaryngology, Plastic Surgery, Radiation Oncology, and Thoracic Surgery. Though they participate in a separate match, Urology and Ophthalmology were also included in this tier. Tier 2 was limited to specialties offering 1.0–1.9 spots per US senior and included Anesthesiology, Child Neurology, Neurology, Emergency Medicine, Medicine/Pediatrics, Obstetrics and Gynecology, Pediatrics, Physical Medicine & Rehab, Radiology—Diagnostic, General Surgery, and Vascular Surgery. Tier 3 was limited to specialties offering 2.0–2.9 spots per US senior and included Family Medicine, Internal Medicine, Pathology, and Psychiatry.
All findings were derived via a secondary analysis of preexisting data. Per the exempted, IRB‐approved protocol, all data were deidentified prior to analysis. Parametric and nonparametric statistical tests were used, as appropriate. Statistical significance was defined as p ≤ 0.05. All standard errors are reported as ± values.
Results
The number of students enrolled in the elective ICR course increased steadily from 2007 (n = 22) to 2014 (n = 91). The number of PSMRF applications also increased accordingly—from 15 in 2007 to 46 in 2014. Awarded stipends for PSMRF have risen from 15 in 2007 to 32 in 2014 (Figure 2). PSMRF student projects spanned over 50 different medical fields including primary care, internal medicine, and surgical specialties and subspecialties. Of interest, 30% of PSMRF students subsequently matched into residency specialty directly related to the research focus of their PSMRF project.
Figure 2.
Trends in medical students involvement in ICR and PSMRF.
Student demographic profiles including age, gender, proportion of University of Kentucky undergraduates and science versus nonscience GPA showed no statistically significant differences between PSMRF students and their classmates. However, on average, PSMRF students entered medical school with higher scores on the MCAT subscales of physical sciences (10.7 ± 0.1 vs. 10.2 ± 0.1; p = 0.002), biological sciences (10.9 ± 0.1 vs. 10.6 ± 0.08; p = 0.006)—as well as total scores (31.5 ± 0.6 vs. 30.6 ± 0.2, p = 0.007) (Table 2).
Table 2.
Demographics and academic success predictors
| Non‐PSMRF | PSMRF | |
|---|---|---|
| Demographics | ||
| Total | 898 | 119 |
| Age | 23.8 ± 0.2 | 23.9 ± 0.4 |
| Gender (M:F) | 58:42 | 63:37 |
| Undergraduate predictors | ||
| MCAT score | 30.6 ± 0.2 | 31.5 ± 0.6** |
| Physical | 10.2 ± 0.1 | 10.7 ± 0.3** |
| Biological | 10.6 ± 0.1 | 10.9 ± 0.3** |
| Verbal | 9.9 ± 0.1 | 9.9 ± 0.3 |
| Science GPA | 3.6 ± 0.02 | 3.6 ± 0.08 |
| Nonscience | 3.8 ± 0.02 | 3.7 ± 0.06 |
| Undergrad GPA | 3.7 ± 0.02 | 3.6 ± 0.06 |
| PubMed publications | 0.1 ± 0.04 | 0.1 ± 0.08 |
| Medical school predictors | ||
| Step 1 score | 223 (19) ± 1.5 | 228 ± 4.2* |
| Step 2 score | 235 (20) ± 1.7 | 235 ± 5.1 |
| AOA+ status | 76 (8.5%) | 23 (19.3%)*** |
| PubMed publications | 0.3 ± 0.06 | 0.8 ± 0.3*** |
*p < 0.05; **p < 0.01; ***p < 0.001.
+AOA, Alpha Omega Alpha Honor Medical Society.
With regard to overall number of PubMed‐indexed publications, PSMRF students authored 0.8 ± 0.3 papers, compared to 0.3 ± 0.06 (p < 0.0001) (Table 2). PSMRF students were more than twice as likely than their classmates to author or coauthor a PubMed‐indexed paper (36.7% vs. 17.9%, p < 0.0001). Of those medical students who published PubMed‐indexed papers, PSMRF authors published 2.1 ± 0.51 publications compared to 1.4 ± 0.15 for non‐PSMRF authors, p < 0.001.
On average, PSMRF students scored significantly higher (228 ± 4.2) on the USMLE Step 1 compared to their classmates (223 ± 1.5), p = 0.03, but showed no significant difference in USMLE Step 2 scores. More than twice the percentage of PSMRF students were awarded AOA status compared to their non‐PSMRF classmates (19.3% vs. 8.5%, respectively; p = 0.0002) (Table 2).
Finally, a greater percentage of PSMRF fellows compared to non‐PSMRF students matched to top 25 research residency programs as ranked by USNWR (23.4% vs. 12.1%, p = 0.008) (Figure 3). There was no difference between the groups as far as matching in residency programs affiliated with AMCs—(95.3% vs. 88.6%, p = 0.1). There was a trend showing that larger proportions of PSMRF students successfully matched into competitive tier 1 specialties (23% vs. 14.2%, p = 0.07), respectively.
Figure 3.
PSMRF versus non‐PSMRF match to USNWR rankings.
Discussion
Although the elective PSMRF program is early in its development, it has seen a steady increase in student enrollment—as has participation in the prerequisite ICR course. From the upward trend in annual applications since 2007, it may be concluded that the PSMRF program has been well‐received by UKCOM students. This may be due to the fact that in‐depth exposure to a field of interest allows students to better select future careers based on mentor guided professional development.
The variety of clinical research topics currently funded by the PSMRF program is also a measure of program success, given that some academic departments are not adequately resourced to compensate medical students for their time. The PSMRF stipend provides students the financial flexibility to pursue any valid research interest regardless of departmental funding. In addition to incentivizing students, the availability of research funding has also been shown to increase research productivity and collaboration among scientists.6, 7, 8 Thus, the funding provided by the PSMRF program has the potential to impact both research output and rigor in traditionally underfunded fields at this institution, with the long‐term goal of reviving student interest across many specialties.
A key goal of the PSMRF program has been to spark interest in translational research and provide opportunities for experienced and inexperienced students alike to become involved in the scientific process. The challenge of such programs is to ensure the recruitment of students who are representative of the medical student body but also maintain a serious interest in learning about the research process. Demographic data indicate that PSMRF students are, in fact, representative of their classes in terms of age, gender, undergraduate degree and entering GPA.
In terms of the traditional markers of academic and research success during medical school, we examined standardized test scores, AOA status and publication record while at UKCOM. We found that PSMRF students perform significantly better than non‐PSMRF students on USMLE Step 1, which for most PSMRF students occurs at the end of the PSMRF program. However, these differences are not sustained on USMLE Step 2, which occurs following students’ clerkship training. This, taken with the higher PSMRF MCAT scores, suggests that the fellowship may attract students who differ in their innate aptitude or interest in the preclinical sciences. However, given that undergraduate science GPA was not different between the groups, it is also possible that participation contributes to improved USMLE Step 1 performance. Overall, these data confirm that PSMRF participation has no detrimental impact on medical school academic performance.
PSMRF students, compared to their counterparts, are more likely to have a PubMed‐indexed paper published from their work at UKCOM, and they are also 1.5 times more prolific. As yet another marker of research achievement, PSMRF fellows are twice as likely to attain AOA status. Per AOA standards for membership, “The top 25% of a medical school class is eligible for nomination to the society, and up to 16% may be elected based on leadership, character, community service, and professionalism. Members may also be elected by chapters after demonstrating scholarly achievement and professional contributions and values during their careers in medicine.”9 While the exact selection process varies slightly by chapter, induction into AOA indicates a high level of achievement. These results further suggest that rather than being a detriment to time management, the PSMRF experience is associated with increased academic and research productivity.
Finally, we examined the residency program matches of former PSMRF fellows. Though relatively more PSMRF students matched to the most competitive specialties, this difference was not statistically significant. We did, however, find that a significantly higher percentage of PSMRF fellows matched to top‐ranked residency programs, based on USNWR rankings. While these rankings have been the subject of much criticism in the academic world,10 they remain an undeniably important influence on student and public perception of academic prestige.
The present findings are subject to several limitations. First, USNWR rankings do not account for institutions that are not directly listed as teaching hospitals of US medical schools. Second, medical training is a dynamic, ongoing process—making it difficult to unequivocally attribute any observed effect due to a specific cause. Similarly, many of the outcomes we have examined were likely influenced by factors shown to differ among PSMRF students prior to their completion of the elective. Third, while publications are arguably a valid measure of research productivity, they tell us little about students’ levels of involvement, collaboration, or creativity. Thus, we cannot assume that student authorship conveys a standard level of participation.
Conclusion
The PSMRF program, as conducted at the UKCOM, offers a popular, structured opportunity for mentor‐guided research among undergraduate medical students. Although early in its development, the program has shown a robust increase in enrollment. Moreover, we found no negative impacts associated with the added academic demands of the PSMRF elective, and modest, positive associations with selected indicators of success in medical school and residency.
Future studies are needed to further validate these findings and expand our inquiry to include longer term program impact.
Acknowledgments
This study is supported by NCATS, UL1TR000117. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
References
- 1. Nathan DG, Varmus HE. The National Institutes of Health and clinical research: a progress report. Nat Med. 2000; 6(11): 1201–1204. [DOI] [PubMed] [Google Scholar]
- 2. Ley TJ, Rosenberg LE. The physician‐scientist career pipeline in 2005: build it, and they will come. JAMA. 2005; 294(11): 1343–1351. [DOI] [PubMed] [Google Scholar]
- 3. Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical student research in the development of physician‐scientists. J Invest Med. 2003; 51(3): 149– 156. [DOI] [PubMed] [Google Scholar]
- 4. Gallin EK, Le Blancq SM. Clinical Research Fellowship Program L. Launching a new fellowship for medical students: the first years of the Doris Duke Clinical Research Fellowship Program. J Invest Med. 2005; 53(2): 73–81. [DOI] [PubMed] [Google Scholar]
- 5. AAMC . National resident matching program, results and data: 2014 main residency match®. Washington, DC: 2014. [Google Scholar]
- 6. El‐Sawi NI, Sharp GF, Gruppen LD. A small grants program improves medical education research productivity. Acad Med. 2009; 84(10 Suppl): S105–S108. [DOI] [PubMed] [Google Scholar]
- 7. Godin B. Funding research: a bibliometric evaluation of the NSERC Research Grants Program. Montreal: Quebec Observatoire des sciences and des technologies, 2002. [Google Scholar]
- 8. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007; 298(9): 1002–1009. [DOI] [PubMed] [Google Scholar]
- 9. Alpha Omega Alpha Honor Medical Society. 2014; Available at: http://www.alphaomegaalpha.org/index.html. Accessed September 14, 2014.
- 10. McGaghie WC, Thompson JA. America's best medical schools: a critique of the U.S. News & World Report rankings. Acad Med. 2001; 76(10): 985–992. [DOI] [PubMed] [Google Scholar]