Abstract
There is a well‐documented shortage of physician researchers, and numerous training programs have been launched to facilitate development of new physician scientists. Short‐term research training programs are the most practical form of research exposure for most medical students, and the summer between their first and second years of medical school is generally the longest period they can devote solely to research. The goal of short‐term training programs is to whet the students’ appetite for research and spark their interest in the field. Relatively little research has been done to test the effectiveness of short‐term research training programs. In an effort to examine short‐term effects of three different NIH‐funded summer research training programs for medical students, we assessed the trainees’ (N = 75) research self‐efficacy prior to and after the programs using an 11‐item scale. These hands‐on training programs combined experiential, didactic, and mentoring elements. The students demonstrated a significant increase in their self‐efficacy for research. Trainees’ gender, ranking of their school, type of research, and specific content of research project did not predict improvement. Effect sizes for different types of items on the scale varied, with the largest gain seen in research methodology and communication of study findings.
Keywords: aging, medical students, training
Introduction
Data show the importance of starting research training early in career development,1, 2 e.g., medical school. The best way to examine effectiveness of programs is to follow trainees and assess academic success and career choices. However, some short‐term outcome measures are needed. One such outcome is improvement in research self‐efficacy.
Self‐efficacy is a person's belief in her or his own capability to achieve a specific goal.3 Individuals with high self‐efficacy are likely to view difficult tasks as areas to be mastered rather than avoided. Prior research has demonstrated that individuals’ career decisions are largely based on their perceived ability to succeed in the career.4 Research self‐efficacy and outcome expectations mediate research career success.4, 5 Therefore, increasing research self‐efficacy early in the training pipeline may increase the number of physician scientists. Bakken et al. reported that women physicians rated their ability to apply clinical research skills and knowledge lower than men did.6
To our knowledge, no studies on medical student short‐term research training programs have examined effects of these programs on research self‐efficacy. The present study evaluated effects on research self‐efficacy of three separate NIH‐funded medical student summer research training programs—Medical Students’ Training in Aging Research (MSTAR), Medical Students’ Sustained Training and Research Experience in Aging and Mental Health (M‐STREAM),7, 8 and UCSD‐CTSA's short‐term TL1 program. These programs shared several common characteristics. All students conducted an 8‐ to 12‐week research project the summer after the first year of medical school. Program staff coordinated didactic sessions covering topics of experimental design, research methodology, and interpretation of experimental data, and met with students to ensure proper matching with appropriate mentors. All TL1 and MSTAR students as well as 44% of M‐STREAM participants spent their summer at UCSD and participated in ethics training as well as online human subjects training. Additionally, the M‐STREAM students who spent their summer at different institutions (including UCSD) attended a 2‐day weekend workshop at UCSD, where didactic sessions covering research and other opportunities after medical school, applying to and selecting residency training programs, how to achieve a successful work/life balance, statistics and methodology, and successful aging were presented. An important goal of this weekend event was also networking and sharing among the trainees from across the country. Thus, all students spent at least some time at UCSD and a majority of the participants spent the entire summer at UCSD. These programs combined a hands‐on, mentored research experience with research‐related didactics.
There were, however, some differences among these programs. The MSTAR program focused on research in geriatrics, M‐STREAM concentrated on geriatric psychiatry or neuroscience, and the TL1 program focused on clinical research in any branch of medicine. Trainees for MSTAR and M‐STREAM were selected from medical schools in the United States, and could choose clinical, basic, or translational research. All CTSA TL1 students came from UCSD, and conducted clinical research. M‐STREAM trainees could choose any medical school for the summer whereas MSTAR and TL1 students worked at UCSD. To reduce differences in training across sites, we held conference calls with mentors, and provided them with brochures on mentoring. Each conference call included a structured agenda as well as time for mentors to contribute ad hoc discussion. The agenda for the calls included the following: welcome and introductions, overview of the program including funding source, duration, general expectations from mentors/mentees which included the need for evaluations and ratings such as the research self‐efficacy scale. We emphasized the importance of collecting data from both trainees and mentors to conduct research on training.
We hypothesized there would be significant improvement on the research self‐efficacy scale.9 We conducted a factor analysis and examined whether these factors improved over time as well. We also explored the potential relationship between students’ self‐efficacy and variables such as gender, type of college they attended, type of research, and program type, as well as differences among scale items in terms of effect size of improvement between pre‐ and postparticipation.
Methods
Participants were 75 medical students who had completed 1 year of medical school at the first assessment. Data from 2010 and 2011 were used for analysis, except for CTSA TL1, for which only 2011 data were available.
To assess research self‐efficacy, we used a modification of a scale developed by Schwarzer and Jerusalem,9 which used core competencies and an adapted scoring system from general self‐efficacy scales. Scholz et al.10 examined the psychometric properties of Schwarzer's scale and their baseline sample consisted of 19,120 participants (mean age 25 years, 56% female, 35% students) from 25 countries. The original self‐efficacy scale was created to assess a general sense of perceived self‐efficacy, and was not designed to target a specific behavior or skill. As noted by Schwarzer and Jerusalem9 the general self‐efficacy scale does not measure specific behavior and it is necessary to add a few items to cover the particular content of interest. Therefore, to assess medical students’ research self‐efficacy, we modified the original questions to specifically target abilities related to research in these trainees. Students ranked the accuracy of 11 statements related to research skills on a Likert‐type scale from 1 (not at all true) to 4 (exactly true). Data were collected within 2 weeks of the beginning and end of the program (with no access to original responses).
Results
We examined internal consistency on the revised measure which resulted in a Cronbach's Alpha of 0.906 for the pretest and 0.858 on the posttest, which were comparable to those reported with the original general self‐efficacy scale.9 The summary scores were normally distributed at baseline. Research self‐efficacy increased significantly after completing the summer research training. There was no significant interaction between year of data collection and the three program types, or for the main effect of year or program for the overall self‐efficacy at the baseline. Therefore, data for both years were combined to increase statistical power. The main effect of time (pre‐, post‐) was highly significant (f = 46.2, df = 1, 72, p < 0.001) indicating improvement in the overall research self‐efficacy. There was no significant association of improvement in trainees’ self‐efficacy with gender, type of college attended, type of research, or program type.
Factor analysis of the items on the scale produced three factors explaining 73% of variance (Table 1). Factor 1 consisted of six items related to research methodology and communication, and explained 52% of the variance. Factor 2 consisted of three items relevant to regulatory and organizational aspects, and explained 11% of the variance. Factor 3 consisted of two items related to interpersonal experience, and explained 9% of the variance. Total score for each factor was computed by summing scores on all factor items. Changes in these sums from pre‐ to postprogram were compared across the three factors. There was a significant improvement in Factors 1 and 2, but not Factor 3. Partial eta square was largest for the Factor 1 (0.427), intermediate for Factor 2 (0.195), and smallest for factor 3 (0.085).
Table 1.
Premean (SD) | Postmean (SD) | df | T | p | Partial eta squared | |
---|---|---|---|---|---|---|
Factor I: Research methodology and communication | 17.74 (3.6) | 20.46 (2.5) | 75 | −7.5 | <0.001 | 0.427 |
Item #1 I understand primary research methodology principles | ||||||
Item #2 I understand how to critically review the scientific literature | ||||||
Item #3 I am able to generate a hypothesis‐driven research proposal | ||||||
Item #7 I am able to articulate research findings in a concise scientific presentation | ||||||
Item #8 I am able to present research findings in written format, including an abstract and a manuscript | ||||||
Item #9 I am able to recognize gaps in knowledge or skills, and develop learning plans to correct deficiencies | ||||||
Factor 2: Regulatory and organization‐level aspects | 9.59 (1.7) | 10.51 (1.5) | 75 | –4.5 | <0.001 | 0.195 |
Item #4 I understand ethical issues involved in human and/or animal research | ||||||
Item #5 I understand informed consent and IRB functions | ||||||
Item #6 I understand regulatory and research issues related to inclusion of women and minorities | ||||||
Factor 3: Interpersonal aspects | 6.97 (1.0) | 7.34 (.84) | 75 | −2.6 | 0.010 | 0.085 |
Item #10 I am able to work well as a valued member of a research team | ||||||
Item #11 I understand mentor/trainee roles, responsibilities, and potential issues | ||||||
Total | 34.30 (5.5) | 38.31 (3.9) | 75 | −6.8 | <0.001 | 0.384 |
Discussion
A hands‐on, mentored research training experience combined with didactics resulted in a significant increase in research self‐efficacy among medical students. Unlike Bakken et al.6, we found no gender differences in reported self‐efficacy. Students’ undergraduate or medical schools or training location seemed to make no significant difference. Trainees’ self‐efficacy improved most on research methodology and communication, followed by regulatory and organizational aspects. However, there is a possibility of a ceiling effect as these students may be highly motivated and start with a high research self‐efficacy score.
This study has several limitations. The sample was not randomly selected, but consisted of students interested in research experience. As there was no control group, we cannot rule out a possibility that observed improvements in self‐efficacy could have occurred as a result of maturation or activities outside of our training program. As such, future studies should examine change in research self‐efficacy among students who do not participate in structured research training programs. Such a comparison may require adjustment for baseline differences in self‐efficacy, as students who choose to participate in research training programs might be more motivated and confident in research skills relative to nonparticipants prior to program entry. There were no objective measures of improvement. Lastly, it will be years before we can determine whether the training had an impact on career choice.
More research on the effectiveness of research training is needed. The research self‐efficacy scale can assess programs’ short‐term success as it asks about one's self‐perceived ability to do research. Other ways of measuring success over a longer time period would include number of publications, grants, and choice of a research career.
Acknowledgments
This work was supported by National Institutes of Health grants NCRS ULRR031980, NCRS UL1TR000100, TL1 TL1TR00098, T35 AG26757, R25 MH71544, and P30MH066248.
References
- 1. Kupfer DJ, Hyman SE, Schatzberg AF, Pincus HA, Reynolds CF. Recruiting and retaining future generations of physician scientists in mental health. Arch Gen Psychiatry. 2002; 59: 657–660. [DOI] [PubMed] [Google Scholar]
- 2. Rosenberg LE. Physician scientists: endangered and essential. Science. 1999; 283: 331–332. [DOI] [PubMed] [Google Scholar]
- 3. Bandura A. Self‐efficacy In Ramachaudran VS, ed. Encyclopedia of Human Behavior. New York: Academic Press; 1994: 71–81. [Google Scholar]
- 4. Hollingsworth MA, Fassinger RE. The role of faculty mentors in the research training of counseling psychology doctoral students. J Couns Psychol. 2002; 49: 324–330. [Google Scholar]
- 5. Kahn JH. Research training environment changes: impacts on research self‐efficacy and interest In JH Kahn Chair: Research Training in Counseling Psychology: New Advances and Directions. Symposium conducted at the Annual Convention of the American Psychological Association, Washington, DC; 2000. [Google Scholar]
- 6. Bakken LL, Sheridan J, Carnes M. Gender differences among physician‐scientists in self‐assessed abilities to perform clinical research. Acad Med. 2003; 78: 1281–1286. [DOI] [PubMed] [Google Scholar]
- 7. Halpain MC, Trinidad GI, Wetherell JL, Lebowitz BD, Jeste DV. Intensive short‐term research training for undergraduate, graduate, and medical students: early experience with a new national‐level approach in geriatric mental health. Acad Psychiatry 2005; 29: 58–65. [DOI] [PubMed] [Google Scholar]
- 8. Jeste DV, Halpain M, Trinidad G, Reichstadt J, Lebowitz BD. UCSD's short‐term research training programs for trainees at different levels of career development. Acad Psychiatry 2007; 31: 160–167. [DOI] [PubMed] [Google Scholar]
- 9. Schwarzer R, Jerusalem M. Generalized Self‐Efficacy Scale In Weinman J, Wright S, Johnston M, eds. Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs. Windsor, UK: NFER‐NELSON; 1995: 35–37. [Google Scholar]
- 10. Scholz U, Gutiérrez Doña B, Sud S, Schwarzer R. Is general self‐efficacy a universal construct? Psychometric findings from 25 countries. Eur J Psychol Assess. 2002; 18: 242–251. [Google Scholar]