Disparities Elimination Summer Research Experience (DESRE): An Intensive Summer Research Training Program to Promote Diversity in the Biomedical Research Workforce

Abstract

The Disparities Elimination Summer Research Experience (DESRE) was created to provide hands-on health equity research training opportunities to undergraduate and graduate students, particularly those from backgrounds underrepresented in biomedical research. Funded by NIH’s National Institute on Minority Health and Health Disparities, a total of 22 students participated in 4 annual cycles of an intensive, 6-week, full-time, residential research training program consisting of didactics, community immersion experiences, peer mentoring, ethics training, and hands-on health disparities research. Demand for the program was high; by the 4th year of implementation, more than 500 applications were received for the cohort’s six slots. More than half of DESRE participants came from minority-serving institutions and/or identified as a member of a minority group. Students reported a significant increase in self-reported competency across all of the program’s 26 learning objectives from pre- to post-assessment. Further, the program had a 77% success rate in promoting a career in biomedical research and/or health disparities elimination, including 100% of minority participants either entering a graduate program and/or entering careers focused on health equity. Key success factors and lessons learned are discussed.

Introduction

There is a well-documented and persistent lack of diversity within the biomedical research workforce. Fewer than 5% of doctoral degrees in science and engineering are awarded to members of underrepresented minority groups.1 Even for minority researchers who earn doctorates, barriers remain – African American investigators are 10% less likely to receive NIH R01 funding in comparison to their Caucasian counterparts (even when controlling for educational background, country of origin, training, previous funding, publication record, and employer characteristics).2 This lack of diversity throughout the biomedical research pipeline has led to calls from the Institute of Medicine and National Research Council to “replenish and diversify the clinical research workforce” in order to bring “new perspectives and opportunities for scientific advancement and an intensified focus on understanding and eradicating health disparities.”3 This diversification is critical to the advancement of all realms of health research, but has particular relevance for work focused on minority health and health equity.

A critical component of increasing diversity in research careers is the availability of formal, funded training opportunities. Unfortunately, there are limited opportunities for individuals across stages of educational attainment and career level to gain hands-on, mentored training experiences in health equity research. This lack of available training opportunities creates a major obstacle for individuals to enter biomedical research careers focused on health equity. While training grants are available across a variety of funding organizations, these funds nearly always take a siloed approach to training (eg, supporting only undergraduate students) and typically in a specifically defined, narrow biomedical approach (eg, cancer genetics). To be optimally impactful, health equity training programs should provide a full range of training necessary for exploring a career focused on health equity that is also integrated into a cohesive program across the educational spectrum (ie, undergraduate students to doctoral students). Further, such programs should be responsive to the growing literature that indicates groups underrepresented in biomedical research careers may benefit more from emerging models of mentorship and training rather than the traditional, one-on-one didactic mentoring that has been in place for centuries. Models incorporating multiple simultaneous mentors (“multiple mentoring”) and the use of peer mentoring have been widely supported in the literature, but remain relatively uncommon.4-10 Programs based on these emerging models have the capability to address the stark lack of representation of diverse groups in biomedical research careers, but require an intentional approach to diversity that may differ from the ways in which program faculty were themselves trained.

To that end, the Disparities Elimination Summer Research Experience (DESRE) was created with support from the National Institutes of Health’s National Institute on Minority Health and Health Disparities (NIMHD) through Center of Excellence funding. DESRE is an intensive, 6-week, full-time, residential research training program consisting of didactics, community immersion experiences, peer mentoring, ethics training, and hands-on health equity research. The founding goal of DESRE was to address the lack of health equity researchers by enacting an innovative training and mentoring program to increase the number of students entering health equity-focused biomedical research careers.

Methods

Theoretical Underpinnings and Design Process

To develop DESRE, we followed a formal instructional design process known as ADDIE,11 a model consisting of five steps: Analyze, Design, Develop, Implement, Evaluate. Utilizing an ADDIE-based approach to instructional design creates a cyclical process that ensures programs are goal-oriented, feasible, appropriately evaluated, and systematically improved as needed. In addition, the conceptualization of training programs as a focused, intensive educational experience is consistent with calls in the literature for more formalized, systems-focused training to promote diversity in the biomedical research workforce.4 During the Analyze step, we focused on the goals of the program (to increase diversity in the health equity workforce), its intended audience (groups underrepresented in biomedical research careers), and the resources necessary to successfully implement the program (financial, instructional space, housing, personnel). Next, to Design the program, we developed the specific objectives and strategies that would be implemented. This included establishing the program’s 26 learning objectives across four domains (health disparities, research methods, statistics, and research ethics), deciding upon the structure of the program (including its length, residency requirement, weekly calendar, etc.), and creating the recruitment plan. We then Developed the program’s materials, including its syllabi, operational manual, program policies, recruitment materials, and evaluation measures. The program then held its pilot Implementation, which involved final preparations for the first cycle, engagement of trainees in the cycle, and comprehensive measurement of process and outcome measures. Following the pilot and subsequent cycles, a comprehensive Evaluation of the program was conducted and improvements to the program made after each cycle.

In addition to its intentional curricular design, the mentoring components of DESRE were specifically designed as an integrative, multi-approach model to mentoring in order to address well-established limitations to traditional mentoring models in meeting the needs of groups underrepresented in biomedical research careers. Specifically, our approach integrated traditional, multiple, and peer mentoring approaches to create an interconnected mentoring team for each program participant. Traditional mentoring takes a one-on-one, top-down approach in which a recognized expert in the field or training area meets routinely with a mentee to share their knowledge and experience (eg, a senior mentor working individually with a mentee). Such approaches contain effective elements (eg, fostering a strong connection with a specific individual), but do not have some of the specific advantages that newer approaches bring. Multiple mentoring (or mentoring circles),5,6 in which an individual has a mentoring team rather than a single, identified “mentor” allows for diverse backgrounds, skills sets, and viewpoints to be brought into the mentoring process. Multiple mentoring helps to reduce the often competitive and hierarchical nature of individual mentoring, which can serve as a barrier for members of racial/ethnic minority groups and women to receive effective mentoring.7-10 Further, by creating a “coaching team,” multiple mentoring is responsive to calls in the literature for a more systematic coaching approach to the mentoring of under-represented minority scholars.4 Finally, peer mentoring, in which mentees receive advice and input from individuals at or close to their current level of training/career, often allows for more salient, timely feedback regarding strategies for success and removes much of the pressure associated with a more traditional mentoring relationship.12

In DESRE, a multi-level mentoring model was achieved by each trainee having a traditional mentor through their research project, multiple mentors through the other program leadership and didactic instructors, and peer mentoring through formal program activities (eg, weekly peer mentoring sessions and programmatic staff from a diverse range of educational backgrounds) and informal interactions with other trainees and staff. This approach also allows each participant to serve not only as a trainee, but also as a mentor through the experience. For example, a master’s student in the DESRE program would have a traditional mentor through the research project PI, multiple mentors through the didactic course instructors, would receive peer mentoring from other DESRE student participants (undergraduate, master’s, and doctoral), and would also serve as a peer mentor to fellow DESRE master’s and undergraduate students. Additional peer mentoring was received through interaction with postdoctoral researchers within DESRE’s parent Center of Excellence grant.

Structure

DESRE consisted of 6 full-time weeks of didactics, immersion experiences, peer mentoring, professional development, and hands-on research experiences (Figure 1). Students received a stipend and on-campus housing throughout the program.

Figure 1. DESRE curriculum model.

Originally, didactics consisted of four mini-courses in: research ethics, health disparities, research methods, and statistics. After the pilot year, the need for an additional didactic course on health literacy became evident and was incorporated into years 2-4. A health disparities seminar was also held weekly, in which health equity researchers came to discuss with DESRE participants both their research and their overall career path. Students registered for and received three hours of course credit for the didactic component. The didactic portion’s learning objectives can be seen in Table 3 in the results section, which also summarizes the pre-post changes in self-rated competency across the program’s learning objectives. While homework was intentionally discouraged in didactic classes due to the intensive nature of the remainder of DESRE’s activities, students did collaborate each year to create two infographics as a part of the health literacy component.

A weekly immersion experience was a critical component of ensuring that trainees were directly exposed to, and worked with, communities affected by health disparities, ranging from migrant farmworkers to children receiving services at federally qualified health centers (FQHCs). These day-long trips involved visiting local communities to conduct windshield tours, to meet with various local stakeholders (eg, farm owner), and to deliver health promotion programming to a variety of audiences (ranging from children in Early Head Start to patrons of local senior centers). At the conclusion of each immersion experience, an hour-long seminar was held to discuss ways to work with communities to achieve health equity, whenever possible inspired by the communities visited that day.

Peer mentoring and professional development were delivered as weekly seminars and were designed to prepare undergraduate students to apply and be accepted to graduate programs, to prepare master’s level students to apply and be accepted to doctoral programs, and to prepare all students to enter careers in health disparities elimination. The peer mentoring seminar was delivered by either a master’s or doctorate-level research associate/postdoc working within the parent grant. The professional development seminar was delivered by a faculty mentor other than the students’ research mentors and addressed topics including mock interviews, managing online presence, creating a career plan, and writing personal statements.

Hands-on research experiences comprised the remainder of activities. Students were paired with a research mentor and worked an average of 20 hours per week to support the faculty member’s ongoing health equity-focused research. Students were divided into two groups of three students each, with a graduate student serving as team lead and two undergraduate students supporting. This arrangement further enhanced DESRE’s overall team/peer mentoring approach. In addition to assisting with ongoing project efforts, teams worked together to prepare two manuscripts and corresponding presentations based upon existing data within the same projects (eg, assisted with Phase II of a project and analyzed data that had been collected in that project’s Phase I). This approach allowed students to rapidly experience more of the overall process of data collection, analysis, and dissemination.

An example of the weekly schedule is presented in Table 1. Key revisions to the schedule over the years included: relocation of the community immersion experience to Wednesday from its original Friday position in order to “break up” the week; the addition of an independent work slot that allowed participants flexible time to meet with additional mentors or catch up on research work; and the addition of biweekly social activities (eg, movie night, bowling) to help build cohort cohesion.

Table 1. Example weekly schedule.

	Monday	Tuesday	Wednesday	Thursday	Friday
8:00 AM	Ethics	Research Methods	Community Immersion	Health Literacy	Health Literacy
9:00 AM				Research Projects	Research Projects
10:00 AM	Research Projects	Research Projects
11:00 AM
12:00 PM	Lunch	Lunch	Lunch	Independent Work and Lunch	Lunch
1:00 PM	Research Projects	Research Projects	Community Immersion		Research Projects
2:00 PM
3:00 PM	Health Disparities			Statistics	Peer Mentoring/Professional Development
4:00 PM		Disparities Seminar	Working with Communities

Recruitment and Selection

For the pilot year (2013) all four program participants came from the home department of one of the project’s principal investigators. For subsequent cycles (2014-2016), a national search was conducted for the six slots – two graduate students and four undergraduate students each cycle. Print and digital media were created and circulated to a continually updated list of programs throughout the country. The core of this list consisted of academic departments and career services for all HBCUs and HSIs in the nation, as well as all McNair programs. Each year, this list was expanded to additional universities throughout the country.

Interested students submitted an application packet consisting of a cover letter, an application form, a resume/CV, GRE scores, transcripts, and two letters of recommendation. After an initial screening of applications for completeness and responsiveness, applications were reviewed by a team of four interdisciplinary faculty members who created individual rank-order lists to guide a consensus-building discussion to select the six finalists. Alternates were also selected, but rarely needed (one year’s final enrolled cohort was precisely the six initial finalists selected).

Evaluation

In addition to a comprehensive process evaluation that helped shape several revisions (eg, the move of the community immersion experience to the middle of the week), DESRE’s outcome evaluation consisted of pre- and post-assessments examining students’ self-rated comfort with each of the program’s 26 learning objectives on a 6-point Likert-type scale in response to the stem “How comfortable are you with your ability to do the following?” Students were also contacted periodically after the program to monitor their progress toward a career focused on health equity. Outcome evaluation data (ie, change in self-reported comfort with the program’s learning objectives) were analyzed using paired t-tests.

Results

Four cycles of DESRE were conducted in the summers of 2013-2016. The 2013 cycle was an initial pilot year, with all participants coming from a single institution. Once the program’s recruitment shifted to its national focus, there was overwhelming demand for the program. In 2014, the first year of full implementation, approximately 100 applications were received for the program’s six slots. By 2016, the number of applications reached more than 500.

Across the four summers, DESRE enrolled 22 students: 3 predoctoral, 4 masters, and 15 undergraduates. More than half of participants (12 of 22) came from minority-serving institutions (ie, HBCU or HSI) and/or identified as a member of a minority group, and more than 80% of program participants (18 of 22) were female. Students came from a total of 17 universities across 12 states, representing 18 different majors ranging from dietetics to anthropology. A summary of program participant characteristics and outcomes can be found in Table 2.

Table 2. Participant characteristics.

Demographics, n=22	%
Female	81.8
Male	18.2
African American	18.2
Asian	9.1
Hispanic	13.6
Non-Hispanic White	59.1
Major (undergraduates), n=15
Psychology	33.3
Biochemistry	6.7
Nursing	6.7
Biology/Microbiology	26.7
Dietetics	6.7
Public Health	6.7
Public Policy	6.7
Economics	6.7
Degree program (graduate), n=7
Psychology	14.3
Public Health	42.9
Anthropology	14.3
Sociology	14.3
Social Work	14.3
% Entered graduate program (undergraduates only)	46.7
% Entered health equity career (undergraduates only; non-duplicated)	33.3
% Entered health equity career (graduate)	71.4

When examining the results of the outcome evaluation, students reported a significant increase in self-reported competency across all of the program’s 26 learning objectives from pre- to post-assessment, as shown in Table 3. In addition to demonstrated improvement in health disparities career skills, DESRE demonstrated a strong ability to promote careers in biomedical research, particularly among members of underrepresented racial/ethnic minority groups. Of the seven graduate/predoctoral students who completed DESRE, five are now in positions consistent with a research career: one is a faculty member with a research program focused on health disparities; one is in industry as a health researcher; one is enrolled in a doctoral program; one works in community engagement/organizing; and one is a mental health practitioner focused on health disparity populations. Of the 15 undergraduate students, 12 are now in positions consistent with a research career: six are currently in doctoral programs; one is in a master’s program; two work in research teams at academic institutions; one works at a non-profit as a health researcher; one works as a health educator; and one is employed in the health care industry. We therefore achieved a 77% success rate in promoting a career in biomedical research and/or health disparities elimination through DESRE, including substantial success in promoting further graduate studies among alumni. Most impressively, 100% of minority participants subsequently: a) entered a master’s or doctoral program; and/or b) are currently employed in careers focused on health equity.

Table 3. Pre- and Post-Test Results of DESRE Learning Objectives.

Learning Objective		Prea	Posta	Pb
Health Disparities
	1. Discuss the origins of health disparities and ways in which they can be eliminated	4.5	5.3	<.001
	2. List and quantify the predominant health disparities faced in rural settings	4.1	5.6	<.001
	3. Describe disparities associated with race and ethnicity	4.5	5.6	<.001
	4. Explain how gender and sexual orientation are associated with health disparities	4.5	5.6	<.001
	5. Translate knowledge of the source of rural health disparities into targets for action	3.7	5.3	<.001
	6. Describe the role of health behaviors in counteracting health disparities	4.3	5.5	<.001
	7. Discuss how external factors such as stigma and access to care impact elimination of health disparities	4.8	5.7	<.001
Research Methods
	8. Discuss the importance of and differences between internal and external validity	4.1	5.3	<.001
	9. Describe the various levels of measurement in research studies	4.0	5.3	<.001
	10. Compare the relative strengths and weaknesses of different types of research designs frequently used in health research studies	4.0	5.0	.001
	11. Select an appropriate sampling strategy for a given research question	3.9	4.6	.002
	12. Design a survey appropriate for addressing a health-related research question	4.0	4.7	.006
	13. Apply the fundamentals of data management	3.7	4.6	<.001
	14. Describe the parts of a typical research funding proposal	3.3	4.6	<.001
Biostatistics
	15. Describe the different levels of data and how their central tendency is measured	3.5	5.1	<.001
	16. Discuss various distributions of data	3.7	4.7	<.001
	17. Explain the role of error and bias in data analysis	4.0	4.8	.01
	18. Describe how interaction and confounding complicate data analysis and interpretation	3.9	4.8	.007
	19. Perform statistical tests including t-tests, ANOVAs, and regressions	3.6	5.1	<.001
	20. Interpret and explain results of analyses to a variety of potential audiences	3.5	4.8	<.001
Research Ethics
	21. Describe the history of research ethics and how it shapes ethical practice today	4.2	5.4	<.001
	22. Discuss how vulnerable populations are protected in human subjects research	4.7	5.5	.003
	23. Explain the responsibilities of researchers in ensuring ethical conduct of human subjects research	4.9	5.7	<.001
	24. Discuss the importance of privacy and confidentiality in research	5.1	5.7	.002
	25. Identify the role and purview of IRBs, IBCs, and IACUCs	3.7	5.3	<.001
	26. Describe the identification of and procedures for handling conflicts of interest in human subjects research	4.0	5.3	<.001

a. Pre- and post-values were rated on a 6-point Likert-type response scale ranging from 1 = very uncomfortable doing to 6 = very comfortable doing.

b. P calculated from paired t-tests.

Because of DESRE’s success in achieving its objectives, the program has received repeated national attention. In 2014, we were invited to present the initial results of DESRE at the convention of the American Public Health Association in a special, inter-Section Council spotlight session highlighting impactful collaborative work, and in 2016 DESRE was named the National Rural Health Association’s Outstanding Program of the Year.

Discussion

The Disparities Elimination Summer Research Experience (DESRE) proved to be a feasible, impactful summer program for promoting diversity in the biomedical research workforce and encouraging students to pursue careers focused on health equity. Students not only demonstrated significant improvements in self-rated competency across all of the program’s 26 learning objectives, the program had a 77% success rate in advancing health equity careers among all participants and a 100% success rate among minority participants. There are several components we feel contributed to this success.

First, DESRE was intentionally designed to engage learners from across the educational spectrum in a shared learning experience. This included undergraduate students, graduate students, postdocs, and faculty members working together to enhance the learning experience. This allowed trainees to work with and learn from individuals at each of the levels of training the program sought to promote. It also enhanced the learning that occurred at each individual level – for instance, the inclusion of graduate students in the didactic portion allowed undergraduates to experience a higher level of learning. Our goal was partially to demystify each educational level and to make it seem more salient and achievable.

Second, DESRE had an intentional focus on direct exposure to and experience working with communities impacted by health disparities. The full-day community immersion experiences each week ensured that students saw in real life the principles being learned about in the classroom and researched within the projects. These immersion experiences were often rated as the highlight of the overall program, with comments such as “the most valuable aspect has been going out and doing community-based interventions.” We further had an intentional focus on community-engaged research, which is critical to achieving health equity, resulting in comments from students reflecting this learning such as “it’s about what the community says they need and not about [what] we think they need.” Nearly every process evaluation mentioned the powerful role the community experiences had in shaping both their understanding of health equity issues and their own personal dedication to careers focused on achieving health equity.

The third critical component of DESRE was its use of multiple simultaneous mentoring programs designed to maximize each participant’s ability to receive mentoring in the way most beneficial to them. As described earlier, this included traditional “expert-learner” dyads, multiple/team mentoring approaches, and comprehensive peer mentoring. Consistent with the rationale of providing diverse mentoring approaches, student comments varied in which type of mentoring was most valuable to them, supporting the need for students to have access to different types of mentoring within a training program. This was reflected in comments such as the most valuable component of the summer being “getting advice from the faculty about future career” from one trainee, but “the most valuable aspect was learning from my peers” from another trainee. Although not an intentional part of the program’s design, we also learned that the interactions trainees had with the staff and especially the leadership of the community-based organizations visited during the community immersion experiences provided valuable mentoring and the opportunity to interact with role models, with comments such as the most valuable part of the summer being “the opportunity to meet and network with professionals in the real world” and another trainee commenting about the impact of “meeting powerful women…who are all making a big impact on…health disparities.” Once these comments were received, we were more intentional with providing opportunities to interact directly with the leadership of community-based organizations working on health equity issues.

Finally, our use of a residential cohort model promoted group cohesion and generated long-term relationships that have been maintained many years after the program. The residential approach further encouraged peer mentoring among trainees from very diverse backgrounds – for instance, in one cohort, we had students majoring in anthropology, epidemiology, biology, nursing, biochemistry, and integrative biology at universities from Florida to California all working and living together. This ability to share broader experiences directly enhanced learning and promoted respect for diversity of background and thought. To help encourage cohort bonding, we launched each year with a team-building retreat and in later years instituted pre-planned informal social gatherings. We found that students were much more likely to discuss challenging topics with program faculty – such as work/life balance and charting a career path – during these social gatherings than during more formal mentoring sessions.

Conclusion

In conclusion, DESRE is an innovative research training program designed from the ground up to meet the needs of trainees from backgrounds underrepresented in biomedical research careers. It demonstrated substantial success in recruiting health equity-focused students into an intensive training program that subsequently supported their pursuit of additional graduate studies and/or a career focused on health equity. The use of similar training programs that incorporate intentional instructional design and modern models of mentoring hold great potential for increasing the diversity of the biomedical research workforce.