Processes and Outcomes from a Medical Student Research Training Program in Integrative, Complementary and Alternative Medicine

Abstract

In response to the growing need to train a new generation of clinician scientists, a research program was developed to train medical students in integrative, complementary and alternative medicine (ICAM) research early in their careers. A total of 25 students (100%) successfully completed a 10 week program. Students reported significantly increased levels of knowledge in all 7 ICAM topics at the conclusion of the program. All students presented their work at one or more local research symposia. In addition, the average number of quality research outputs, which included manuscripts, awards, and abstracts presented at national and international meetings, was 1.5 per student, which exceeded benchmarks based on prior program outcomes. Results from this program may be useful when planning larger or longer term projects aimed at attracting physicians who will become our next generation of academicians, researchers, and healers.

Introduction

One of the major challenges to academic research is a critical shortage in the number of qualified physician investigators.¹ A steady decline in the number of physician-scientists has been identified for at least two decades,² and a shortage of clinicians applying for federal research grants exists.³ A report by the National Advisory Board for Medical Rehabilitation Research recommended addressing these shortages by increasing the number of training programs, especially those at the pre-doctoral level, that will bear the next generation of scientists.⁴ Novack, et al⁵ described how the next generation of scientists must assume the role of “clinician healers.”

A field with a growing need for more academic “clinician healers” is that of Integrative, Complementary, and Alternative Medicine (ICAM). According to a 2002 National Health Interview Survey,⁶ over 19% of adults (38.3 million) did not use conventional medical care, and of that group almost 1/4 (9.5 million) used alternative medicine. These individuals had more health needs and were more likely to delay getting conventional medical care because of cost and other factors. A 2015 National Health Statistic Report⁷ showed that dietary supplements, deep breathing exercises, and yoga were the most common ICAM approaches used by Americans at rates of 17.7%, 10.9%, and 9.5% of the population, respectively. Glick et al highlighted that many people use these ICAM approaches for chronic pain.⁸ In fact, ICAM interventions are now recommended by the Institute of Medicine (IOM) as a routine part of care for chronic pain.⁹ As a result of the growing popularity of ICAM and to fill gaps in scientific evidence, The National Institutes of Health (NIH) have increased efforts to conduct large, multicenter clinical trials and a variety of smaller scale basic and clinical research projects. Considerable focus has been placed on research priorities to understand the mechanisms of ICAM on chronic pain, the efficacy in treating pain, and symptom management in chronic disease, to name a few.¹⁰

While the NIH, the Veterans Administration (VA), and the IOM have made major progress in addressing barriers to training in ICAM research by providing mentored programs in fellowships, opportunities for women and minorities, career development awards, and special initiatives for loan repayment, we still remain at a crossroads where our initiatives to improve training have not produced the number and quality of trainees needed.¹ One problem is that many of the mechanisms to address these issues have concentrated on MD/PhD programs, research in sub-specialty fellowships, and other programs focused on post-doctorate level trainees, with few emphasizing early medical student research experiences.² A second problem is that too few medical schools provide optimal training in research.¹¹ The IOM has demonstrated that the number of medical students who receive research training is insufficient and that their training is too narrow to be capable of sustaining the level of medical research that is necessary to understand and treat complex conditions. Proponents of evidence-based practice have argued that research training should be part of all curricula of pre-doctoral students, particularly because without such training, practitioners may ignore scientific research and engage in practices that are demonstrably useless or harmful.¹²

Early exposure to research often determines whether a trainee will ultimately choose a research career.¹³ Some medical educators promote early exposure to role models with careers that successfully combine medical research and patient care.¹⁵ Surveys of medical students have indicated that the majority endorse research and are eager to participate but that many attend schools that do not offer integrated research opportunities.¹⁴ Even if schools do offer research programs, ICAM practices are not always integrated into the curriculum. Therefore students who go on to do research as a career may not conduct ICAM research because of lack of exposure or knowledge of the field.

Therefore, a training program for medical students interested in ICAM research was developed and deployed with recruitment efforts focused on students who were early in their careers. The purpose of this project was to understand elements of a research training program, gather participant feedback, and evaluate the short-term success of the training program.

Hypotheses

• Students will report significantly increased levels of knowledge in key ICAM topics at the conclusion of the program, compared to baseline.

• The training program will meet or exceed the benchmark of an average of 1 high quality output per student, defined as presentations at national conferences, peer-reviewed manuscript publications, and awards. A prior study² reported that 46-49% of students in NIH- funded training programs produce a manuscript, presentation or award from their work. The goal for this project was for all students to produce quality outputs.

Methods

Recruitment of Trainees

The program was conducted within the Department of Physical Medicine and Rehabilitation (PM&R) at the University of Pittsburgh School of Medicine (UPSOM) as part of an NIH-funded training award (5T35AT005933-02) from 2010-2015. Students were recruited through advertisements which were placed on the PM&R department website, in brochures and flyers, at Integrative Health and PM&R Medical Student Interest Group meetings and other medical student events, and through word of mouth advertising from the Dean’s Office, Office of Student Affairs, and Office of Minority Affairs.

Eligibility requirements were: 1) enrolled at UPSOM, 2) agreed to commit to 10 contiguous 40-hour weeks of research, 3) have successfully completed the first year of medical school, and 4) proposed project must have relevance to ICAM field.

Each eligible student was required to make contact with an approved mentor and devise a 3 page proposal. Approved mentors were those faculty members within the School of Medicine from a variety of departments who were known leaders in the field of ICAM research and clinical care. Students were required to submit a scientific project proposal to the Dean's Summer Research Program (DSRP) within the School of Medicine. Applications that had relevance to ICAM were identified. These identified Applications were peer reviewed by two reviewers. Selection of applications for funding was made on the basis of scientific merit. Top candidates were offered scholarships which included a stipend of approximately $4000 and additional funds for supplies or travel to a scientific meeting. Five student applications were selected per year for five years.

Students were given training in research integrity through the Office of Research Integrity Guidelines for Ethical Practices in Research and completing the web-based modules on research practice fundamentals. Trainees were required to submit completion certificates of online training modules as well as approval letters from the Institutional Review Board or the Institutional Animal Care and Use Committee if human or animal subjects were being used in research. Mentors were encouraged to engage students in projects that had already been approved by these committees rather than devising a new protocol. The analysis of outcomes from this training program was conducted under a separate, approved Institutional Review Board protocol using a waiver of informed consent.

The summer began with trainees attending an introductory session. A survey (see Appendix 1) was administered at the beginning of the summer, and was based on the work by Findley et al¹⁶ to determine which fields students envisioned as a career path and their interest in pursuing research as part of their career. The survey asked students to rank their self-reported knowledge level in 7 ICAM topic areas using the following scale: 0=none, 1=a little, 2=some, 3=a lot. Students were given an introductory lecture on research fundamentals applicable to the field of ICAM. Appendix 2 provides a sample summer curriculum.

Students then began working with their mentor on their research project, which was carried out during the summer between their first and second year of medical school, when students would otherwise have been on break. Since the intention of the training program was to support students entering into clinical residency programs, some clinical exposure to patients was allowed. Students were permitted to shadow their small group or research mentors during inpatient or outpatient clinical care (e.g. a student doing research on prolotherapy or acupuncture could shadow a clinician who performs these interventions). However, clinical work was not allowed to exceed 20% of the trainee's effort or one day per week.

Students then participated in a DSRP Early Summer Retreat held at the medical school for all medical students participating in various research programs over the summer between the first and second years of medical school. This retreat featured various topics including one by the Senior Vice Chancellor on important themes in biomedical research and one by the Associate Dean for Medical Student Research on “making the most out of the relationship with your mentor.” This was followed by round-table discussions with research faculty so students could get individual feedback on their projects. Trainees also attended the Rehabilitation Research Day Symposium which consisted of research presentations by invited guest speakers followed by awards for scientific paper and poster presentations by faculty, post-docs and other students who had completed projects in the previous year.

Over the summer, Small Group Workshops were held approximately bi-weekly. Early sessions included interactive discussions about statistics and to critically analyze each other's proposals. Subsequent workshops were geared toward topics that are fundamental to new researchers.¹⁶ Facilitators of the workshops were ICAM practitioners and/or School of Medicine faculty. Workshops covered topics such as defining research questions and formulating hypotheses, critical literature review, specifying subjects and sampling, precision and accuracy of measurement, use of questionnaires, secondary outcome measures, study designs, evaluating diagnostic tests, ethics, and sample size. Each issue was reviewed in the context of a specific ICAM topic and using published, peer-reviewed articles from the field. For example, when acupuncture was introduced, students got hands-on experience in placing needles, discussed difficulty with blinding in randomized trials, and reviewed papers that presented a credible control or sham intervention.

Half way through the summer, students gathered for a DSRP Mid-Summer Retreat, open to all medical students participating in various research programs. This retreat was comprised of lectures on scientific writing and presentation skills and a guest speaker who discussed his or her own personal journal in an academic research career.

Trainees submitted a one-page progress report at week five. The trainees were asked to outline progress made over the summer, list potential obstacles encountered and ways they would be overcome, discuss their relationship with their mentors, and chart the plans for completing their project. The Program Director reviewed and discussed progress with each trainee individually.

At approximately week 8, students were encouraged to submit a travel proposal outlining their plan to present their work at a national or international conference of their choosing. At week 10, students were asked to submit a scientific abstract which would be the submission to the professional conference. The survey (Appendix 1) was re-administered at the conclusion of the program. Additional questions were asked about strengths and weaknesses of the program and what factors, including this training program, may have influenced the student's decision about which field of medicine they would like to pursue.

The trainees then presented posters at the DSRP Symposium held at the medical school. This symposium was comprised of guest lecturers and awards. In the following Spring, Trainees were expected to present their work at the same Rehabilitation Research Day Symposium that they had previously attended as an observer. This allowed the students to prepare for their upcoming national presentation.

Follow up surveys were administered over the 5 year duration of the program to collect data on publications of peer reviewed manuscripts, abstracts accepted at conferences, and awards. Scores from the self-assessment of ICAM knowledge at the two time points were compared using the Wilcoxon test. An alpha level of 0.05 was set a priori as the significance level.

Results

A total of 5 students in each of 5 years (n=25) were recruited, and all completed the program successfully. The average number of students funded each summer through all summer research funding mechanisms at the School of Medicine was 98. The mean age of trainees was 24.1 (2.9 SD) years. The demographics of the students in the ICAM summer research program are compared to those of the student bodies of the School of Medicine in Table 1. Response rate was 100% (n=25) on the entrance survey and 80% (n=20) on the exit survey because the exit survey was not collected until year 2.

Table 1.

Demographics of Students

	ICAM summer program n=25		2010-2014 School of Medicine Classes n=741
	n	%	n	%
Gender
Male	16	64.0	395	53.3
Female	9	36.0	346	46.7
Ethnicity
Hispanic or Latino	0	0.0	36	4.9
Not Hispanic or Latino	24	96.0	623	84.1
Did not answer	1	4.0	82	11.1
Race
White/Caucasian	10	40.0	444	59.9
Black/African American	2	8.0	59	8.0
Asian	10	40.0	185	25.0
Mixed Race	2	8.0	0	0.0
Did not answer	1	4.0	53	7.2
Disability
Any Disability	0	0.0	1	0.0

On the entrance survey, interest in medical fields varied considerably, spanning 23 different specialty areas. The fields of Orthopedics (n=8), PM&R (n=7), Internal Medicine (n=5), Family Medicine (n=5), and Emergency Medicine (n=5) were the most frequent choices.

Students who had graduated at the time this manuscript was written had matched to the following medical specialties: Orthopedics, Internal Medicine, Pediatrics, General Surgery, Family Medicine, Radiology, and Anesthesia. On the exit survey, the most common fields of interest for residency for those who did not yet graduate were PM&R, Orthopedics , and Internal Medicine.

The faculty participating as mentors and in didactics were from multiple disciplines including the School of Nursing, School of Health and Rehabilitation Sciences, and the School of Medicine, including the Depts. of PM&R, Psychiatry, OBGYN, Internal Medicine, and Orthopedics. Ten students chose a mentor from the field of PM&R.

Students’ self-reported knowledge level in all 7 ICAM topic areas improved significantly from baseline to post-program (See Table 2).

Table 2.

Likert scores of self-reported knowledge level in each of several ICAM topics (0=none, 1=a little, 2=some, 3=a lot)

	Before ICAM Summer Program	After ICAM Summer Program
	Median (range)	Median (range)	p value
Integrative, complementary, or alternative medicine in general	1 (0-2)	2 (1-3)	0.003*
Whole medical systems (e.g. homeopathy, naturopathy, Chinese medicine)	1 (0-3)	2 (1-3)	0.003*
Mind Body Medicine (e.g. Mindfulness based stress reduction)	1 (0-3)	2 (0-3)	0.006*
Energy Medicine (e.g. biofield and bioelectromagnetic therapy)	0 (0-3)	1 (0-3)	0.002*
Manipulative based Practices (e.g. massage, manual therapy)	1 (0-2)	2 (0-3)	0.001*
Therapy Modalities (e.g. biofeedback, electrical stimulation)	1 (0-2)	2 (0-2)	0.032*
Eastern Exercise programs (e.g. yoga, Tai Chi)	1 (0-2)	2 (1-3)	<.001*

^*Significant at p<0.05 level

All students presented abstracts at one or more local research symposia as oral presentations or posters. Also, at the time of drafting of this manuscript, the 25 students had produced in total 38 additional high quality scientific outputs. These included 3 published scientific manuscripts in scientific medical and rehabilitation journals, 1 published review article in a scientific medical journal, 24 abstracts accepted for presentation at national scientific medical and rehabilitation meetings in the U.S. and Puerto Rico, 5 abstracts accepted at international scientific medical and rehabilitation meetings (1 in Canada, 1 in Spain, 1 in the Netherlands, and 2 in Sweden), 1 national research award, and 4 local research awards.

Table 3 displays qualitative data regarding trainee interest in a research career and influence of the program on career decisions. Explanations for each answer were condensed and paraphrased in this Table.

Table 3.

Trainee Interest in a Research Career and Influence of the Program on Career Decisions

	Entrance Survey	Exit Survey	Reasons for Answer
	n=25	n=20
Will you conduct research as part of your ultimate career?
Yes	84%	80%	-enjoyed doing summer research -enjoy discovering new things -allows physician to better help patients -allows continuous learning -promotes open-mindedness and creativity -wish to practice in academic setting -wish to be involved in medical education -interest in population health - to supplement a clinical practice -yes, but only if patients remain primary focus -wish to take active role in scientific community and improve knowledge -want to answer important clinical questions and translate research back to bedside -investigate efficacy of ICAM interventions, or patient outcomes
No	4%	0%
Unsure	12%	20%	-unsure about specialty choice -more interest in clinical than basic science research -more aware of difficult funding climate
Do you anticipate a research, clinical, or combined career?
Research	0%	0%
Clinical	8%	0%
Combined	76%	85%	-more interest in clinical than basic science research
Unsure	16%	15%	-unsure about feasibility of incorporating research into a mostly clinical career
Did participation in ICAM research influence your interest in a research career?
Yes		55%	-positive experience leading own project or working with mentor -confirmed interest in doing research in general or in topic area -was not interested in research before but now interested -exposure to all research in the field was motivating -research was fun and engaging -improved ability to understand and use medical literature -improved understanding of variety breadth of research opportunities -demonstrated how one can incorporate research into a clinical career
No		35%	-already committed to a research career -do not want to do research as part of a future career -possibly, but would need more experience to make a decision
Unsure		10%	-still unsure about a research career -need more clinical exposure
Did participation in this research program influence your interest in a specialty?
Yes		60%	-confirmed interest in field that was already of interest -recognized practical application of research to field of interest -helped to narrow down specialty choices -immersion in topic area helped to confirm a general area of interest -piqued interest in a new specialty area -reinforced commitment to field that focuses on function and quality of life -showed how ICAM can be used in field of interest
No		30%	-already committed to a specialty area -need more clinical experience to make decision
Unsure		10%	-still unsure about a specialty -narrowed down choices but still deciding about a specialty

Trainees also provided several suggestions for improvement, which emerged as themes. Specific quotes are listed for each theme below. First, some students wished for more longitudinal didactic experiences geared specifically toward helping them with their own research projects or overcome obstacles.

• “It might have been helpful if more of the didactic time was spent specifically focusing on issues with our individual projects and how they played out as the summer went by.”

• “I think I would have liked to meet with [the Program Director] and the group around the mid and end of summer points in order to briefly share experiences, frustrations, and to clarify expectations.”

Second, some students asked for a separate, longitudinal journal club that ran in parallel to the didactic program, and some students would have preferred discussing review articles rather than a few select experimental studies within each ICAM topic.

• “I prefer a broader overview of the therapy (strengths/weaknesses of the therapy, what research has been done and how?) than discussing a few select journal articles.”

• “Journal club that meets maybe three times over the summer to go through papers that cover topics in alternative medicine.”

A few miscellaneous comments included recommendations that the program be longer, include more exposure to certain specific ICAM topics, limit travel distance for workshops, or use interactive technology to poll students during didactic sessions.

The trainees also reported strengths of the program, which emerged as several themes. A few illustrative quotes for each theme are provided below. First, some students enjoyed their own research project the most out of all components of the program.

• “I liked that the program kept me on my toes in terms of getting things done in terms of my research goals.”

• “Getting to understand the science behind the protocols and performing them.”

• “Overall, I liked the blend of classes and independent research.”

Second, students also remarked on their specific mentors or course facilitators.

• “Dr. ____'s mentorship was incredible, and the lab as a whole was a great place to be as a novice researcher.”

• “I thought the statistics workshop Dr. ____ gave us was very helpful and helped me to make better sense of things.”

The majority of positive comments, however, surrounded either the immersive nature of the didactic courses, or the exposure to ICAM practices.

• “I enjoyed learning about alternative therapies and their application to wellness. It is encouraging to see allopathic medicine beginning to respect the wealth of experience available in other healing ideologies.”

• “What I liked best was the schedule of the different didactic sessions was very reasonable. During the didactic sessions, I learned so much about alternative and complementary medicine that I had not known before. It was my first time learning about myofascial anatomy, tai chi and massage therapy.”

• “I enjoyed the hands-on and interactive sessions of the programs very much. The Tai Chi, acupuncture, myofascial anatomy, and yoga sessions were especially memorable since I was able to reinforce learning through practice.”

• “I very much appreciated the breadth of exposure that this T-35 program enabled. I cannot imagine myself getting the same level of meaningful or intellectual exposure to the practitioners and their fields of expertise without significant personal effort, otherwise.”

• “The myofascial anatomy session was very engaging and really surprised all of us in terms of how interactive it was. I would highly recommend continuing this session in future years.”

• “It was great to explore other areas of medicine that I doubt I would have had exposure to in medical school.”

Discussion

The future of academics in ICAM depends on training new investigators who will be our next generation of clinician scientists. Although the final class of students in this program will not graduate until 2017, the early results of this program appear promising. All 25 students successfully completed the program. They reported significant increases in their knowledge about ICAM topics and produced more quality scientific outputs than expected, based on prior literature.²

A high level of interest in PM&R was shown by students enrolling in this program, and even more in students exiting the program. This is not surprising given that the Dept. of PM&R was designated as the home for this research program. The field of PM&R is a leader in making innovative therapies available to patients and embraces the notion of giving patients multiple options for treatments. The field of PM&R has a longstanding history of integrating nonconventional therapies into protocols that already include allopathic treatments, and which has led to the interdisciplinary team model that we use today.¹⁷

Several lessons were learned in deploying this program, which may be helpful to others in developing a new program or scaling a small program to larger groups. First, hands-on immersive experiences rather than standard lectures seem to motivate students and enhance the learning experience, particularly with respect to ICAM modalities. Second, while it may not be possible to anticipate all topics of student projects, linking topics of workshops to the topics of students’ projects as best as possible may improve the learning experience. However, it is also important to provide a diverse and broad range of topics, including training in scientific methods, especially statistics. Third, early research exposure seemed to positively influence many students’ interests in research as a career or in a specific medical field. We surmise this is due to participants gaining exposure to a clinical field indirectly via working with research mentors in that field. Robust research programs may therefore be critical to the future growth of specialties. However, early direct clinical exposure may be equally as important as early research exposure in influencing specialty choice. Finally, after the program, a slightly higher percentage of students were interested in integrating research into clinical careers. Mentorship and career counseling from faculty who successfully balance their clinical and research careers may help those students who are not sure how to combine clinical and research careers and are concerned about funding climates. Mentors should emphasize how critically analyzing the literature and understanding levels of evidence are examples of skills that excellent clinicians should possess.

Four limitations of the program deserve discussion. First, although the percentage of racial minorities recruited for this research program was higher than that of recent medical student classes, a large proportion of students were of Asian descent, suggesting that racial diversity could be increased even further. Second, the representation of females and ethnic minorities fell below that of the student bodies. Only one student in the School of Medicine reported a disability during the years of this program, and this student did not participate in summer research. Additional targeted recruitment of under-represented groups would be needed in future programs. Third, only short term outcomes were collected; a follow up study would be needed to delineate long term outcomes once all trainees complete residency. Fourth, gain in knowledge about ICAM topics was collected as a self-report measure, rather than as a quantitative examination.

It is hoped that training medical students in ICAM research will not only increase the number of practitioners utilizing ICAM therapies in their practice, but also improve the research capacity in this field, which would be expected to have long lasting effects.

Conclusions

A training program was designed and deployed to enhance opportunities for medical student training in clinical and basic research in ICAM topics. Twenty five students were successfully trained, reported significant increases in knowledge of ICAM topics, and produced quality research outputs that exceeded expected benchmarks. Experiences from deploying the program may be useful when planning larger or longer term projects aimed at attracting physicians who will become our next generation of academicians, researchers, and healers.