Impact of elective versus required medical school research experiences on career outcomes

Abstract

Many US medical schools have added a scholarly or research requirement as a potential intervention to increase the number of medical students choosing to become academic physicians and physician scientists. We designed a retrospective qualitative survey study to evaluate the impact of medical school research at the University of Alabama at Birmingham (UAB) on career choices. A survey tool was developed consisting of 74 possible questions with built-in skip patterns to customize the survey to each participant. The survey was administered using the web-based program Qualtrics to UAB School of Medicine alumni graduating between 2000 and 2014. Alumni were contacted 3 times at 2-week intervals during the year 2015, resulting in 168 completed surveys (11.5% response rate). MD/PhD graduates were excluded from the study. Most respondents completed elective research, typically for reasons relating to career advancement. 24 per cent said medical school research increased their desire for research involvement in the future, a response that positively correlated with mentorship level and publication success. Although completion of medical school research was positively correlated with current research involvement, the strongest predictor for a physician scientist career was pre-existing passion for research (p=0.008). In contrast, students motivated primarily by curricular requirement were less likely to pursue additional research opportunities. Positive medical school research experiences were associated with increased postgraduate research in our study. However, we also identified a strong relationship between current research activity and passion for research, which may predate medical school.

INTRODUCTION

Physician scientists—those ‘who devote a substantive percent of their professional effort to research anywhere along the entire spectrum of biomedical inquiry’¹— bridge the gap between the research and medical arenas by offering a unique perspective on scientific questions and translating newfound knowledge into clinical practice. Furthermore, the contributions of physician scientists often include dedication to trainee education and service in leadership positions. It is disconcerting, then, that the decline in the physician scientist workforce first described in 1979² has largely persisted.³ Specifically, attrition is seen on both ends as ‘medical students’ intentions to pursue a research career (have) declined’ while the current physician scientist workforce is aging.^3,4 Considering the ever-increasing complexity of modern medicine, which compels critical reasoning and data literacy through emphasis on evidence-based medicine,^5–7 disinterest among medical trainees towards research presents a considerable problem.⁸ Factors responsible for such an attitude are unclear, but early exposure to research in medical education has become a common approach to addressing it.

The University of Alabama at Birmingham School of Medicine (UAB SOM) is a research-intensive public medical school that matriculates ~185 students each year. It offers several opportunities for dedicated, full-time research including 8–10 weeks of summer research after the MS1 year; year-out programmes such as a Masters of Public Health, Masters in Basic Medical Science, or Howard Hughes Medical Institute Fellowship; a dedicated 2-month time period to complete a required scholarly activity; and elective research of variable duration for MS4 students. Scholarly activity is typically completed during the MS3 year and is intended to foster students’ analytical thinking and rational decision-making capacity. On completion of the project, students prepare a final written report as well as a poster to be presented at an internal Medical Student Research Day. While the others are voluntary avenues for additional experience, scholarly activity became a mandatory part of the UAB SOM curriculum in 2007, reflecting a national trend toward a compulsory undergraduate medical research experience.^9–14 However, there remains a general paucity of studies investigating the long-term outcomes and utility of mandatory medical research experiences.¹⁵ In the current study, we characterize the long-term effects of medical school research experiences through a retrospective, self-reported survey addressing past research experiences and current participation in research of recent UAB SOM graduates. We then use the Social Cognitive Career Theory (SCCT) as a comprehensive framework to discuss optimization of medical school research experiences to improve long-term retention of physician scientists.

MATERIALS AND METHODS

Ethical approval was granted for studies involving human subjects by the University of Alabama at Birmingham Institutional Review Board for Human Use, February 10, 2015, protocol number E150126003. The survey used in this study was designed with assistance from the UAB Evaluation and Assessment Unit. It consisted of 74 total questions incorporating skip patterns and was conducted online using Qualtrics (V.07844920.62s, 2016, Qualtrics). In order to gauge medical school research experience, questions regarding type of research, motivation, level of mentorship, impact on future career goals, and productivity were asked for each possible research experience. These questions were presented as skip pattern items in blocks specific to the research experiences completed by each respondent. Questions concerning the residency match and current work environment were included as markers of career choice. Current research involvement was assessed by four questions including type of research, role in research, importance of research in selecting a job, and discussion of research with patients. The final three questions pertained to career satisfaction, including contributions to healthcare and consideration of retraining in a new specialty. Gender and graduation year were the only required demographics. Survey face validity was established by experts in medical student research (RGL) and survey design (LH). Subsequently, a group of five medical students and faculty reviewed questions for clarity.

A pilot test of 37 fourth year medical students at the UAB SOM just prior to their graduation in 2015 was used to confirm appropriateness of answer scales and anchors, resulting in rewording of two questions. The study was then advertised in the UAB SOM Alumni Newsletter, followed by an email distribution containing a link to the survey to UAB SOM alumni graduating between 2000 and 2014. Emails were provided by the UAB SOM Development and Community Relations Office. The population was contacted via email two additional times, three times total, at 2-week intervals. Accessing the survey was implied as consent to participate in accordance with the approving regulatory board.

We received 168 completed surveys for a response rate of 11.5%. MD/PhD respondents (n=25) and surveys containing responses inconsistent with year of graduation (n=14) were excluded from analysis, resulting in a final data set of 129 responses. For respondents who completed multiple research experiences, descriptive characteristics for each were aggregated. Table 1 provides an example of selected questions; the full survey tool is included in the online supplementary file 1.

Table 1.

Selected survey questions

Question	Answer scale
What motivated you to pursue this research? (Select all that apply)	To fulfil a research requirement To strengthen my residency application To pursue a passion for research To address a specific research or medical interest To learn more about a medical specialty Other (please describe)
What type of mentorship did you EXPECT to receive in this research? What type of mentorship did you ACTUALLY receive in this research?	No mentorship Little mentorship Some mentorship Substantial mentorship 5. Too much mentorship/overbearing
To what extent did this research influence your career goals?	This experience ELIMINATED my desire for research involvement in the future This experience DECREAESD my desire for research involvement in the future This experience DID NOT POSITIVELY OR NEGATIVELY CHANGE my desire for research involvement in the future This experience INCREASED my desire for research involvement in the future This experience GREATLY INCREASED my desire for research involvement in for future
At which stages of training after medical school did you complete additional research experiences? (Select all that apply)	Internship Residency Fellowship First faculty/attending position Subsequent faculty/attending positions
What is your role in the research?	Primary investigator or project leader—in charge of research project Co-investigator—make significant contributions towards design and execution of research Collaborator—contribute material and/or intellectual resources to project but not directly involved Research staff—consent patients and/or collect data for project Other—please describe

Factor analysis was performed in RStudio (V.0.99.473, copyright 2009–2015, RStudio) using the PCA function from the FactoMineR package with scaled data and the ω function from the Psych package. Regression analysis was also performed in RStudio using the glm (logistic regression) and lm (linear regression) functions with backward elimination.

Internal consistency scores for the total data set were α coefficient=0.84, λ coefficient=0.91, ω hierarchical coefficient=0.48, and ω total coefficient=0.88. Four questions were excluded from analysis due to poor factor loading and low consistency. These questions addressed the topics of participation in a year-out or Master of Public Health programme during or prior to medical school; completion of multiple research experiences within the same laboratory or research group; residency programme matched into; and rank list position matched into. Differences between cohorts identified by factor analysis were described by ORs and χ² analysis using GraphPad Prism (V.4.02, May 2004, GraphPad Software).

RESULTS

Defining medical school research involvement of survey respondents

Of the 129 UAB SOM alumni included in our study, 71 (55%) were male and 58 (45%) were female which is consistent with gender ratios reported in Association of American Medical Colleges graduation questionnaire data from 2000 to 2014. Respondents per graduating class varied from 3 (2003) to 14 (2012), with a slight skew towards more recent graduates (figure 1). One hundred (76%) completed research before starting medical school, and there was a positive association between graduation year and completing undergraduate research. Inclusive of all four UAB SOM research opportunities (scholarly activity, summer research, year-out programmes, and elective MS4 research), 92 UAB SOM alumni (71%) completed at least one research experience, as detailed in table 2. We asked respondents to select the factors that motivated them to pursue the experience. The most popular response was ‘to strengthen my residency application’ (n=64; 70%), followed by ‘to fulfil a research requirement’ (n=58; 63%), ‘to learn more about a medical specialty’ (n=39; 42%), ‘to address a specific medical or research interest’ (n=36; 39%), and finally ‘to pursue a passion for research’ (n=23; 25%).

Figure 1.

Number of respondents by graduation year. Scholarly activity was added as a requirement for all incoming students starting in 2007 (graduating class of 2011).

Table 2.

Characteristics of medical school research for UAB SOM alumni

Characteristics of medical school research	N=92 (%)
Medical school research experiences
Summer research between MS1/2	72 (78%)
Year-out or Masters programme	19 (21%)
Scholarly activity rotation	44 (48%)
Elective research in MS4	17 (18%)
Multiple experiences	47 (51%)
Type of research
Clinical	35 (38%)
Basic science	19 (21%)
Public health or education	14 (15%)
Multidimensional	24 (26%)
Authored a publication	41 (45%)
Authored an extramural presentation	43 (47%)

UAB SOM, University of Alabama at Birmingham School of Medicine.

Next, we asked respondents who completed research to rate the level of mentorship they received during each research experience. The majority reported either some (n=31; 34%) or substantial (n=47; 51%) mentorship. Fourteen respondents (15%) reported no or little mentor-ship. Finally, we asked how medical school research influenced respondent career goals. Answers were split somewhat evenly between decreased desire for research involvement in the future (n=30; 33%), no change in desire for research involvement in the future (n=37; 40%), and increased desire for research involvement in the future (n=25; 27%). Further analysis of responses to this question revealed that motivation ‘to pursue a passion for research’, higher scores on the mentorship scale, and authoring a publication were significant predictors for increased desire for future research involvement, suggesting these factors tended to have a polarizing effect.

The majority of UAB SOM alumni respondents were engaged in elective research during their medical training, with a strong tendency towards clinical research. Research participation appeared to be driven primarily by an interest in career progression rather than an inherent enthusiasm for research. Although the large majority of respondents received at least a moderate level of mentorship, less than half were credited with a publication as a result of their experience(s). For the majority, involvement in medical school research did not have a positive effect on interest in future careers involving research, which may be mediated by motivation, mentorship, and publication.

Identifying the cohort of respondents currently involved in research

With respect to respondents’ current careers, 61 (47%) of respondents were involved in research at the time of the survey. We identified a principal component for current research involvement which accounted for almost 30% of variance in response data and split respondents into distinct positive and negative cohorts (figure 2A). Three factors showed a strong correlation with current research involvement score. First was the type of research being conducted, which most respondents reported to be clinical (figure 2B). Second was the respondent role in the research, and these roles were divided evenly between primary investigator and co-investigator (figure 2C). Third was the effect of research opportunities on job selection, which was perceptible in only a small proportion of this population (figure 2D). We also identified a principal component for career satisfaction which accounted for just over 15% of variance in response data and was characterized by questions regarding eagerness to work, personal contributions to healthcare, and consideration of retraining in a new specialty (figure 2A). Notably, respondents who positively scored current research involvement registered significantly higher career satisfaction scores (p=0.0358; figure 2A).

Figure 2.

Current research activities of respondents. (A) Multiple factor analysis identified a principal component for current research involvement, which was scored as positive or negative and accounted for 30% of variance in our data set (x-axis). A second principal component for career satisfaction was also observed, which accounted for 15% of variance in the data set (y-axis). (B) Types of research respondents reported being involved in currently. (C) Respondents role in their current research project. PI or project leader—in charge of research project. Co-I—made significant contributions towards design and execution of research. Collaborator—contributed material and/ or intellectual resources to project but was not directly involved. Research staff—consented patients and/or collected data for project. (D) The influence of research opportunities on job selection as rated on a five-point scale from 0 (not at all) to 5 (critical). Co-I, co-investigator; PI, primary investigator.

We then determined what aspects of medical school research, if any, differed significantly based on current research status. Respondents who positively scored current research involvement were more likely to have been motivated ‘to pursue a passion for research’ (OR=5.71, 95% CI 1.97 to 16.56), to have completed clinical research (OR=3.09, 95% CI 1.37 to 6.96), to have been motivated ‘to learn more about a medical specialty’ (OR=2.94, 95% CI 1.35 to 6.42), and to have reported an increase in desire for research involvement in the future as compared with respondents who were negative for current research involvement (table 3). Unexpectedly, mentorship and publication did not differ significantly between cohorts although, as described above, both contributed to desire for future research involvement.

Table 3.

Medical school research characteristics that differ between UAB SOM alumni who are currently involved in research (CRI+) and not currently involved in research (CRI−)

Characteristic	CRI+ N=59 (%)	CRI− N=70 (%)	OR (95% CI)	p Value
Average graduation year	2009	2007		0.09
Completed summer research	38 (64%)	34 (49%)		0.08
Type of research—clinical	23 (39%)	12 (17%)	3.09 (1.37 to 6.96)	0.009
Motivated to pursue research by a passion for research	18 (31%)	5 (7%)	5.71 (1.97 to 16.56)	0.001
Motivated to pursue research to address a specific research or medical interest	21 (36%)	15 (21%)		0.08
Motivated to pursue research to learn more about a medical specialty	25 (42%)	14 (20%)	2.94 (1.35 to 6.42)	0.007
Average desire for future research involvement	Increased	Decreased		0.001
Authored an extramural presentation	25 (42%)	19 (27%)		0.09

UAB SOM, University of Alabama at Birmingham School of Medicine.

Medical school research characteristics and future career choices

Finally, we used regression analysis to determine the relationship between involvement in medical school research and continued participation in research. Completion of any medical school research was a significant positive predictor for continuing research during residency training (p=0.005, OR=27.42, 95% CI 2.77 to 284.32). Voluntary research, including summer research and MS4 elective research, was significantly correlated with employment in an academic medical center (p=0.003, OR=6.86, 95% CI 1.89 to 24.96), a principal investigator role in current research (p<0.05, OR=6.58, 95% CI 1.02 to 42.50), and strong consideration of research opportunities during job selection (p=0.002, OR=2.90, 95% CI 1.52 to 5.53). The required scholarly activity was also positively associated with employment in an academic medical center (p=0.009, OR=4.82, 95% CI 1.49 to 15.65), as well as with current research involvement (p=0.02, OR=3.27, 95% CI 1.19 to 9.01). However, those who completed the scholarly activity requirement were recent graduates and, thus, the majority were still in residency or fellowship training programmes, which typically include research commitments. With respect to motivation, passion to pursue research was a significant positive predictor for current research involvement (p=0.008, OR=6.00, 95% CI 1.60 to 22.50), principal investigator role in current research (p=0.008, OR=4.63, 95% CI 1.49 to 14.46), and strong consideration of research opportunities during job selection (p=0.003, OR=2.46, 95% CI 1.38 to 4.39).

DISCUSSION

To the best of our knowledge, this is the first study including long-term outcomes associated with mandatory research experiences at a US medical school. We identified a positive relationship between medical school research, including the compulsory 8-week scholarly activity, and current research activity. This relationship was primarily driven by respondents who were motivated by a pre-existing passion to voluntarily complete medical school research as compared with those who were motivated by a requirement. Mentorship and productivity during undergraduate medical training emerged as additional factors that mediate interest in postgraduate research involvement. Overall, these data suggest that elective research is more indicative of future career plans than compulsory research, and medical school activities may be insufficient to inspire interest in research in students who are not already predisposed to a research-inclusive career. As a single institution study, it is unclear whether our results are generalizable. However, our emphasis on aligning programme goals with students’ passions and the importance of positive research experiences recapitulate the findings of similar studies at Mount Sinai, Stanford, University of California San Francisco, Vanderbilt, and other medical schools as described below.

Across institutions, scholarly projects tend to subscribe to the same ideals of increasing students’ critical reasoning capacities and encouraging continued involvement in academic medicine. Unfortunately, specific research projects and mentors able to satisfy these requirements are often difficult to identify, contributing to a general lack of rigor.¹⁴ From the student perspective, mandatory research experiences are difficult to navigate, requiring additional time and effort which compound the inherent stresses of medical school.¹⁶ Considering these environmental factors, the debatable impact of these programmes on students’ sustained pursuit of research should not be surprising.

SCCT emphasizes a central role for perceived self-efficacy and expected outcomes in shaping individual career interests, goals, and eventual choices.¹⁷ Notably, personal successes, positive mentorship experiences, verbal affirmation, and positive mentality are suggested to determine an individual’s self-efficacy and expected outcomes.¹⁸ Within the context of scholarly projects, SCCT suggests that effective implementation will necessitate aligning individual student interests with those of the scholarly programme. Indeed, this was found to be the case by Prober and Khan¹⁹ who highlighted the value of providing opportunities for students to explore materials germane to their individual objectives, aptitudes, and passions. Understanding patient care to be a passion of most medical students may help explain the tendency we observed toward clinical research projects, as these afford an unequivocal link to potential improvements in patient care. Additionally, students’ relative familiarity in the area likely contributes to a greater perceived ability to successfully complete these types of projects.

Medical school research experiences are often viewed as an opportunity to add a line to a student’s CV through presenting or publishing his/her work, herein reflected by the 70% of respondents who selected ‘to strengthen my residency application’ as a source of motivation. Furthermore, students seem to assume that having a tangible product at the completion of their research experience, regardless of duration, is expected. Publication does appear to have a positive impact on perceived self-efficacy, postulated to encourage continued research involvement,¹³ and our results echoed this idea. The converse also appears to hold true, as failure to publish likely incurs additional stresses that yield a negative perception of research and decreased desire for continued involvement.²⁰ Placing such emphasis on productivity generates largely unrealistic expectations that may be a significant obstacle to the original intent of increasing student interest in research. Stipends, awards at internal conferences, and other means of recognition are therefore commonly employed as alternative means of positive reinforcement.²¹ Zier et al²² noted that the success of undergraduate medical research programmes at Mount Sinai ‘depends on students believing that they have benefited from their research’. We would expand this thought to be contingent on students expecting favorable outcomes from pursuing research.

Although publications and other forms of reinforcement do appear to correlate with future research involvement, our results suggest that a minority (25%) of students intending to ‘pursue a passion for research’ are the most likely to include research throughout their future careers. In general, though, it seems that undergraduate medical research experiences tend to decrease or have no effect on a student’s long-term interest in research. Consequently, because pursuing a passion seems to be a predictor for rather than outcome of undergraduate medical research experiences, the question becomes: When does this passion arise and at what point would positive interventions best prime students for future research involvement? And, to address the minority cohort of students who reported increased desire for future research involvement, what are the ideal conditions for conducting student research? Addressing these questions will allow for design and implementation of research experiences and programmes that are better able to incite a desire for long-term research involvement and thereby impact career decisions.

Since mentorship was a predictor of future research involvement, it follows that individual mentors could perhaps profoundly impact the career trajectory of some students. To understand the approach of such influential mentors, we identified and interviewed two UAB physician scientists with established records of mentoring, as judged by the number of medical students choosing to work with each and the reported productivity of those experiences. Striking similarities in mentoring styles quickly emerged. Both mentors stressed the importance of aligning the research project with the student’s interests while ensuring that the student’s participation would further his/her career aspirations, research-related or otherwise (eg, matching into a competitive residency). Each mentor described some system of guiding students through the project with a series of discrete deadlines, culminating in abstract submission and/or manuscript preparation, corroborating the importance of productivity we and others have identified. Interestingly, a common mentoring strategy was enabling students to provide meaningful contribution, irrespective of project outcome. Within the scope of our findings, the genuine interest these mentors place in each student fosters productivity and likely imparts a positive perception of the research experience, factors critical for shepherding medical students toward academic careers.

One weakness of this study is the limited number of responses. Only 1585 emails were available for UAB SOM graduates from 2000 to 2014. Of these, another 293 were invalid, and the response rate was only 11.5% in the remaining 1292. We did not have data on non-responders and, thus, were unable to characterize this population. Owing to the nature of our study, we expect respondents had a more favorable attitude towards research than non-respondents. However, it is unclear how much effect response bias had on our results. The logistical difficulties we encountered demonstrate a need for long-term follow-up of physicians to more accurately track career trajectory. Lack of training outcomes is a well-established problem within medical education, particularly with respect to physician scientists. A recent article commented on the progress made by MD/PhD programmes in measuring outcomes of their graduates and the need for additional data collection at the residency and fellowship levels to better characterize the physician scientist pipeline.²³ It is important that we expand these efforts to include MD programme graduates, who comprise half of the 8200 physician principal investigators receiving National Institutes of Health (NIH) grants.³ Moreover, some of our most interesting findings came from reflective questions pertaining to satisfaction with research experiences. This construct is difficult to measure, but widespread use of a standard research experience evaluation tool would help determine the most effective format for incorporating research into medical school.

In conclusion, this study addresses the deficiency of long-term outcomes data regarding elective and required medical school research experiences and identifies key characteristics associated with continued research activity. Our results support the idea that early positive research experiences, high quality mentorship, and productivity may prime medical students for later research involvement.^24–26 Moreover, these elements are both more prevalent and more influential among students with underlying passion for research, suggesting an upward spiral effect. In contrast, students disinterested in research may be less likely to have a favorable medical school research experience, which may further diminish their likelihood of pursuing research. These findings challenge the benefit of compulsory medical school research with respect to encouraging academic and research-oriented careers in the absence of student interest. However, realigning the objectives of required research to better match student passions and expectations could potentially foster the positive associations needed to develop a strong perceived efficacy in future research experiences.

Significance of this study.

What is already known about this subject?

• The physician scientist workforce is aging while medical students’ intentions to pursue research careers have declined.

• Many US medical schools, including the University of Alabama at Birmingham School of Medicine (UAB SOM), have recently introduced required research experiences into their curricula.

• The objectives of these programmes include increased critical reasoning and retention in academic medicine. Previous studies suggest success with regards to the former, but little data exist regarding long-term outcomes associated with required research.

What are the new findings?

• Medical school research experiences were positively correlated with current research activity in a subset of UAB SOM alumni.

• The strongest effects were in respondents who voluntarily completed medical school research as compared with those who were motivated by a requirement.

• Mentorship and publications during medical school were additional factors that mediated interest in postgraduate research involvement.

How might these results change the focus of research or clinical practice?

• Our data suggest that voluntary research is more indicative of future career plans than compulsory research, and medical school activities may be insufficient to inspire interest in research in students who are not already predisposed to a research-inclusive career.