Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Aug 1.
Published in final edited form as: J Cancer Educ. 2022 Feb;37(1):37–45. doi: 10.1007/s13187-020-01779-1

Improving Representation of Underrepresented Minority (URM) Students in Oncology Biomedical Research Workforce: Outcome Evaluation from the ReTOOL Program

Folakemi T Odedina 1,2,3, Linda S Behar-Horenstein 4, Parisa Fathi 1,3, Ernest Kaninjing 3,5, Jennifer Nguyen 3,6, Nissa Askins 1,3, R Renee Reams 3,7, Bereket Mochona 7, Adaora Ezeani 3,8, Debra Lyon 9, Kayanna Jacobs 2, Michael Maduka 2
PMCID: PMC9342587  NIHMSID: NIHMS1720920  PMID: 32533539

Abstract

The Research Training Opportunities for Outstanding Leaders (ReTOOL) program was implemented in 2012 to increase the representation of racial and ethnic minorities in the biomedical workforce. Specifically, the ReTOOL program aims to foster the capacity for scientific research among underserved populations as well as address the cultural appropriateness of research projects. This paper describes the impact of the ReTOOL program in enhancing the research training of underrepresented minority (URM) students. Forty URM students who completed the ReTOOL program between 2012 and 2019 were invited to participate in the program evaluation. The response rate was 73% with 29 participants. Of the 29 participants, 26 trainees self-identified as Black or African-American. A structured survey developed for the program was employed for data collection, using a Likert Scale ranging from 1 to 5, with 5 being the best. The item ratings ranged from 4.45 to 4.80. Responses to open-ended questions show that ReTOOL has been instrumental in socializing and acculturating participants into the habits of scientific thinking. The combined use of quantitative and qualitative inquiry depicts that ReTOOL has been highly successful in fostering participant enrollment in advanced health-related or professional degree programs.

Keywords: Minority undergraduate research, Cancer research training, Summer research training, ReTOOL program, Biomedical research workforce, Program evaluation, Program assessment

Introduction

Overview of the ReTOOL Program

Increasing the representation of racial and ethnic minorities in the biomedical workforce is critically important. It is one of the top priorities for the National Institutes of Health (NIH). For example, the Center to Reduce Cancer Health Disparities (CRCHD) at the National Cancer Institute (NCI) is leading efforts aimed at increasing the number of underrepresented minority (URM) students in the area of oncology research. In line with the NCI CRCHD’s efforts and based on a systems analysis approach, the Research Training Opportunities for Outstanding Leaders (ReTOOL) program [1] was implemented in 2012 (see Fig. 1). The ReTOOL program was established to address the lack of well-trained minority cancer scientists in Florida and has been continuously funded by the Department of Defense (DOD) Prostate Cancer Research Program (PCRP) of the office of the Congressionally Directed Medical Research Programs (awards W81XWH-12-1-0083 and W81XWH-14-1-0243) and the NCI (awards P20CA192992 and R25CA214225) since 2012. Initiated as a partnership program between University of Florida (UF) and Florida A&M University (FAMU), the ReTOOL program has been expanded to include all the historically black colleges and universities (HBCUs) in Florida, namely Bethune Cookman University (BCU), Edward Waters College (EWC), and Florida Memorial University (FMU), as well as two Hispanic Serving Institutions in Florida: the University of Central Florida (UCF) and Florida International University (FIU). Variations of the ReTOOL program have been implemented, including Basic, Advanced, and Comprehensive ReTOOL programs.

Fig. 1.

Fig. 1.

Using a logic systems analysis approach, the essential components of the ReTOOL program includes inputs, processes, outputs, outcomes, tracking and evaluation.

The Basic ReTOOL program was established in 2012 as a 10-week summer research experience at UF for URM students who have not had any research experience. Eligible students were college students at either the freshman or sophomore level at the time of program application. Funded by the DOD PCRP (W81XWH-12-1-0083, W81XWH-14-1-0243), the Basic ReTOOL program focused only on prostate cancer research training between 2012 and 2016.

The Advanced ReTOOL program provided additional 10-week summer research training experience for URM students who were at the sophomore or junior level at the time of program application. The Advanced program was funded by the NIH/National Cancer Institute (P20CA192992). Research training under the Advanced program was broader, covering all cancers. While the Basic ReTOOL program was a bridge program for the Advanced ReTOOL program, qualified students who had not participated in the ReTOOL program were also eligible for the Advanced program.

In 2017, both the Basic and Advanced ReTOOL programs were integrated as a component of the Comprehensive ReTOOL program. The Comprehensive ReTOOL program builds on the Basic and Advanced programs to train undergraduate students in the areas of basic, clinical, and behavioral cancer research. The Comprehensive program is a 15-week summer program funded by the NIH/National Cancer Institute (R25CA214225) and expands the ReTOOL program as follows: (1) augments the recruitment of Black male students, who are significantly underrepresented in biomedical, behavioral, and clinical sciences in Florida; (2) focuses the research training on a better understanding of the unique contributions of and embedded relationships among social, behavioral, and genetics factors; (3) focuses the research training on a better understanding of the “gaps” among ethnically diverse Blacks of African ancestry, including native-born US Blacks, Black immigrants, and indigenous Blacks in Africa, the Caribbean, and Europe; and (4) provides continuous mentoring for trainees by including faculty mentors at trainees’ home institution as part of the mentoring team. College students at all levels are eligible for the Comprehensive ReTOOL program.

Comprehensive ReTOOL Program Core Requirements

The ReTOOL program framework is based on themes that emerged from the literature on best practices for increasing underrepresented minorities in science. These themes include the following: (1) the provision of hands-on research experiences; (2) interactions between aspiring and established scientists; (3) networking; (4) mentoring; (5) collaboration; (6) commitment; (7) program structure and consistency; (8) peer linkages; and (9) information about career opportunities [28]. The ReTOOL program provides 15 weeks of research training that includes lectures, community engagement, networking, peer linkages, presentations, and hands-on research experience under the mentorship of a UF cancer researcher. Program opportunities are designed to enhance participant professional development and increase their competitiveness in graduate or professional school applications.

Although the research experience of each trainee is unique, all ReTOOL trainees’ research training meet the following core requirements:

  1. Acquire a basic understanding of the prevention, detection, diagnosis, and treatment of cancer.

  2. Be familiar with major studies and findings in the area of cancer disparities.

  3. Gain expertise in methodologies needed to conduct cancer disparities research.

  4. Critically review and evaluate research.

  5. Gain an understanding of fundamental issues regarding the ethical conduct of research.

  6. Formulate a research question and design a study to answer the question.

  7. Disseminate research findings through published abstracts and peer-reviewed publications.

After successfully completing the summer activities, the program culminates with trainee dissemination of their research findings during a showcase. The research showcase includes a poster presentation by trainees followed by 10-min oral presentations, a question/answer session, and concludes with an award ceremony. Based on the scientific rigor of their project, presentation delivery, and presentation style, the top two trainees are recognized in the Basic Science category and the top two trainees are recognized in the Behavioral/Clinical Science category during the award ceremony. The research showcase provides trainees realistic experience on how to communicate their science to other scientists.

A critical component of the ReTOOL program is the program evaluation for continuous quality improvement. In this paper, we report the outcomes of the program evaluation and experiences of participants.

Methods

Participants

Participants were undergraduate URM students who completed the ReTOOL program between 2012 and 2019. Participants were from multiple institutions in Florida, including University of Florida, Florida A&M University, Bethune-Cookman University, Edward Waters College, Florida Memorial University, and University of Central Florida. Once students are admitted to the program, they complete a baseline survey. Follow-up surveys are implemented annually using the contact email addresses provided by trainees at baseline. The 2019 posttest survey, which was implemented in the summer of 2019, is reported in this paper.

Program Evaluation Survey

The ReTOOL program evaluation survey was developed in 2010 and includes both close-ended and open-ended questions. Close-ended items were used to collect demographic information and descriptions of participant’s current status, scholarly activities, and social media presence. Using Likert Scale, which ranged from 1 to 5 (where 5 was the highest rating), participants were asked to rate 10 items which were designed to assess beliefs about their programmatic experiences (i.e., courses offered, training experiences, mentoring, community outreach experiences, preparation for a cancer research career, program infrastructure such as communications with mentor, director and staff, and the stipend provided). Additionally, the ReTOOL trainees were asked open-ended questions designed to explore the specific benefits of program participation and mentoring and its impact on their research, career development, and funding proposal development, as well as what they liked least and most about the program.

Data Collection and Analyses

Since 2012, 55 students have participated in the ReTOOL summer experience program. Current and valid emails were available for 40 of the 55 ReTOOL trainees at the time of the survey. They were sent an email with a link to the Qualtrics ReTOOL Survey, with reminders email sent 2 weeks later. Of the 40 trainees, 29 (73%) unique responses were received. These numbers are fairly consistent with other studies of long-term tracking [9]. Follow-up with participants is scheduled on an annual basis to ensure trackable emails. Although we attempted to follow-up with the 15 (27%) unreachable participants via LinkedIn and consultation with their university mentors, their emails were unattainable. For the quantitative data, the Qualtrics software was used to report the descriptive statistics for the evaluation, including frequency reports and the mean analyses. The qualitative data was exported from the Qualtrics software for ethnographical analyses conducted by the program evaluator. The program evaluator read each line of data and coded text segments that were relevant for evaluation purpose. Reading line-by-line and coding segments assisted in making supporting quotations more accessible. Data were analyzed inductively, a process that involves a search for emergent themes considered to be important to the description of the phenomenon. It involved several steps including (a) coding and seeing particular text as an important prior to the interpretation, (b) organizing the data to identify and develop themes, and (c) thematic identification, a process that emerges from pattern recognition within the data. The method is driven by what the researchers want to know. In the context of this program evaluation, the analysis was guided by the participants’ subjective descriptions of their experiences [10].

Credibility, or confidence in truth of the findings, was established through triangulation. Triangulation was accomplished using the following: (a) different types of data, (b) multiple data sources, and (c) qualitative analytical tools including line-by-line coding. Confirmability was achieved by engaging more than one person in analyzing the data [11].

Results

Quantitative Assessment of ReTOOL Program

Of the 29 participants, 26 trainees self-identified as Black or African-American, two as Other, and one as White. Majority of the participants were females (20) and non-Hispanic (27). The age range was between 19 and 35 years with a mean of 24.34 years. Participants were asked to rate their overall evaluation of the components provided by the ReTOOL program. Each question was preceded by a stem that read: “On a scale of 1–5, with 5 being the best, please state your overall evaluation of the following components:...” The ReTOOL program components that they were asked to rate were the following: preparatory courses; research training; interactions with your mentor; quality of the mentoring; satisfaction with the training program; quality of the training program; community outreach experience; ability of the program to prepare me for career in the area of cancer; communications with ReTOOL program mentor, director, or staff; and stipend provided for the ReTOOL program (see Table 1). Ratings across all program experiences were high, ranging from 4.45 to 4.80. Items that received the highest ratings included satisfaction with the training program, quality of the training program, and quality of the mentoring experiences.

Table 1.

Participants’ mean ratings on program activities (n= 20)

Program Activities
** “On a scale of 1 – 5, with 5 being the best, please state your overall evaluation of the following components of the ReTOOL program.
Mean *Range
The preparatory courses provided during the program. 4.50 1 – 5
The research training experiences provided by the program 4.60 1 – 5
Your interactions with your mentor. 4.70 1 – 5
The quality of the mentoring experiences. 4.65 1 – 5
Overall satisfaction with the training program. 4.80 1 – 5
The quality of the training program. 4.65 1 – 5
My community outreach experiences. 4.55 1 – 5
The ability of the program to prepare me for career in the area of prostate cancer. 4.60 1 – 5
Communications with ReTOOL program mentor, director or staff. 4.45 1 – 5
The stipend provided for the ReTOOL program. 4.60 1 – 5

Admission to Graduate Program/Health Professional Schools

The primary goal of the ReTOOL program is to foster admission to graduate or health professional programs by strengthening trainees’ application and making them highly competitive. Twenty (69%) reported graduate or professional school enrollment including: two in biomedical science, four in medicine/medical science, four in pharmacy, two in biology, two in psychology, one in biotechnology science and engineering, one in health informatics and information management, one in molecular and cellular biology, one in nursing, one in immunology, and one unspecified.

Career Focus on Cancer

Increasing URM engagement in cancer research is another program goal. Although the program is less than 10 years and still early to validly assess cancer career focus, one trainee is a postdoctoral research fellow in Psycho-Oncology at Memorial Sloan Kettering. Another trainee is a 3rd-year PhD candidate at University of Southern California studying molecular pharmacology/toxicology with focus on male reproductive pathologies, including cancer. Furthermore, seven trainees completed post-baccalaureate training program focused on oncology at University of Florida. Five trainees reported working in non-cancer health entities such as non-cancer research, teaching labs, and dental services.

Scholarship Activity

All of the participants experienced scholarship activities during their summer research training program. Twenty-five (86%) of the trainees reported engagement in varied scholarly activities post-ReTOOL training, including university teaching, conference presentations, graduating with honors, earning fellowships, prestigious internships (i.e., Eli Lilly and Company, the FDA), and serving as leaders in professional organization at their university. Since 2012, ReTOOL trainees have authored or co-authored a total of 8 publications [1220].

Qualitative Assessment of ReTOOL Program

Twenty-three (79%) trainees responded to at least one or more of the open-ended questions (see Table 2).

Table 2.

ReTOOL Program Open-ended Survey

Open-ended questions # of responses
Please describe the benefits of participation. 24
How has the C-ReTOOL research training program informed your research? 23
How has your C-ReTOOL research training program mentor assisted you in the development of your research planning? Name one to three instances or skills from your interaction with mentors that were pivotal in your development as a biomedical scientist. 24
Has the C-ReTOOL research training program helped you obtain grant funding? If so, give an example. 24
Has the C-ReTOOL research training program helped improve your proposal writing? If so, give an example. 24
Has the C-ReTOOL research training program helped you avert scientific misconduct? Explain why. 24
How has your understanding of the role of collaboration in biomedical science changed as the result of the C-ReTOOL research training program? 24
What did you like least about the ReTOOL program and why? 23
What did you like best about the ReTOOL program and why? 24

Program Benefits

Participants explained that exposure to research activities honed their scientific interest and cultivated skills. One participant described how experiences “foster[ed] my interests in research and science.” For others, benefits included opportunities to conduct cancer research, observe patient experiences, and view “the results in real time.” Another participant explained that the program introduced him to “new work that I never considered” and to “a field in research and academia.” Others asserted that program experiences facilitated their acceptance into graduate school.

Participants reported a wide range of opportunities, such as conducting research, acquiring an understanding of pertinent literature, and learning to write a scientific journal manuscript. They described how introductions to clinical, wet lab, and translational researches fostered work proficiency in self-directed research projects. Participants expressed appreciation for networking opportunities such as occasions to present their research findings at conferences, being able to attend professional workshops, and for the mentoring they received.

Informed Research

Participants described how experiential activities helped them “think differently about experimental design” and that using “the literature” was essential to considering how to design study protocol and to support and refute findings. One participant explained being stretched professionally in ways that “forced me to acquire new research techniques and skills” [after] designing research experiments [and] writing her own “research protocol.” Others acquired an increased “knowledge of cancer related research and race disparities in health research” including palliative and hospice care. A few participants explained how their emergent expertise was shaped by conducting different cancer-related experiments. One participant reported that her “interest in drug development and pharmaceutical innovation” sprang from this program while another individual’s initial interest in qualitative research expanded.

Mentoring

Participants cited myriad ways that mentors supported the habits of thinking and behaving like a scientist via personal and professional supports. They described learning the “importance of collaboration and communication,”“how to write like a scientist,” and the steps involved in planning and executing a research study. One participant described the impact of undertaking thoughtful project management as follows.

In order to expand our operations and increase productivity to drive the production of more usable data, I had to research, analyze, and present different protocols to my mentor. After getting the go-ahead, I had to execute the protocols efficiently and effectively to obtain essential data. Ultimately, this led to a significant contribution in the overall results, so I was included in the publication.

Another participant shared that networking with both a mentor and others in the laboratory was a pivotal factor “in motivating [me] to continue a career in research.”

Mentor interactions were perceived as instrumental in facilitating career planning, motivation to attain professional goals, and nurturing the thought processes of a scientist [21, 22]. One participant explained how immersion into his mentor’s group, “in all lab activities and [being] treated me like I was a permanent member in the lab prepared me for life as a fulltime graduate student.” Persistent interactions that required students to explain research planning and defend their thinking were central to ensuring that participants learned how to present defensible research plans. One participant explained how weekly mentor meetings “ensure[d] I was on the correct path” and being paired with another scientist aided “in developing my research plan.” Participants learned the importance of questioning, seeking for data that justifies findings, and to avert complacency by simply following another researcher’s suggestion. One mentor assisted a participant in research planning by guiding her to think like a researcher. He “taught me to critically think and analyze to fully understand the application of my research project. He forced me to write my own research protocol and understand the biochemistry and importance of each step.” Rather than simply give her the tools, this mentor “pushed me to look for research articles, kits and [to locate what was] available in the lab.”

Several participants discussed developing familiarity and experience with fundamental laboratory techniques such as cell culture, Western Blot, PCR, molecular cloning, and “sterile technique which are important in RNA and DNA isolation experiments.” Others reported that “documenting findings in a journal” [and] learning “to use certain lab equipment” were essential to their acculturation as emergent scientists.

Communicating and understanding scientific results were one of the central foci of training experiences. Participants identified relevant activities such as “learning how to write and read abstract”, how to present scientific results in a poster, and the development of presentation skills.

Proposal Development

Six (21%) of the ReTOOL trainees believed that the ReTOOL program was instrumental in raising awareness about how to write a grant proposal. This finding supports the lower tier rating given to the programs’ preparatory courses. A few participants reported that if not for ReTOOL program, they would have remained unaware of the availability of grant funding (such as the NIH mechanism). While most did not have an opportunity to write a proposal during the program, several opined that they were now prepared to do so. One participant suggested that her proposal writing improved because she had been “exposed to literature reviews, [had] to write my own proposal, and [was] surrounded by peers” who wrote proposals. Another claimed that program experiences helped her complete a fellowship application while one other participant felt that she had learned about both “the process and professionalism needed to construct an actual proposal.”

Averting Scientific Misconduct

Eleven (38%) of the participants remarked that they learned how to avert scientific misconduct while seven indicated that the program did not impact their ability to do so. Participants commented that having an understanding of HIPPA and IACUC laws and ensuring participant confidentiality were essential to conducting ethical research. Others reported attaining a better understanding of why “conduct[ing] experiments in a reproducible manner” is essential. They surmised the importance of “proper documentation of results,” and reporting all findings, rather than only the most desirable outcomes because “excluding data could lead to misinterpretation.” One participant reported how examples provided during didactic training helped her realize “what to look out for when reviewing” and how to “avoid plagiarism” when writing scientific papers.

Role of Collaboration in Scientific Research

Participants provided rich insight regarding how their perspectives about collaboration changed as the result of ReTOOL program. One individual shared that prior to this program, he “did not know that many disciplines came together [to conduct] research.” He discovered that interdisciplinary biomedical research is critical to allowing “a wide range of interpretations and presentation of ideas.” Another participant pointed out that “collaboration with other disciplines” is critical to providing the best treatments to patients. One individual reported that “working in a lab alone is impossible.” She now understands “that biomedical science is all about a big picture.”

Participants stressed the nature of teamwork. Although researchers tend to be independent, it is “important to realize that [research inquiry] it is a team effort.” Similarly, learning “how to communicate with thought leaders to work towards a common goal” is an essential activity. Biomedical research relies on interacting with scientists across disciplines. This requires an ability to work together because “you never know who might have the answer to your question.” The program fostered interactions among team members from different disciplines. These experiences effectively prepared participants to engage with other scientists, clinicians, and researchers so that they could adequately assume roles in projects. Collegial interactions promoted talking about research and offered opportunities for networking and intellectual debate about the potential promise and limitations of ideas.

Application of Scientific Knowledge

Participants explained the value of learning how to read scientific literature. Becoming acquainted with background knowledge was important to conducting their own studies and “experimental set ups.” Developing familiarity with the body of scientific studies enhanced their ability “to interpret conclusions and apply them to the real world.” Becoming familiar with scientific studies helped participants learn how to identify variables and understand research study purposes. Some participants explained that reading the literature influenced their thinking. One participant described that it expanded her notion of “all [of] the possibilities and different ways people can think at any given moment.”

Program Preferences

Participants were uniformly appreciative for experiential learning opportunities within a “collaborative environment” that permitted growth “as scientist[s] and lifelong learner[s].” Working with others in a team science format “impact[ed] their maturation as future researchers” [2325]. For example, working with graduate students provided viewpoints about graduate school, how to select the most appropriate program that fit their research interest, and helped raise awareness about importance of seeking funding to pay for school. Networking with peers, other scientists, and faculty provided opportunities to discuss their projects and acquire new perspectives. These interactions promoted fellowship and forged relationships with fellow trainees. Participants expressed appreciation for the weekly cancer seminars, “hands on learning given in labs with well-experienced mentors” and “opportunities to do presentations.” One participant pointed out that, “Being able to choose what I wanted to research for the summer had a huge impact on how I saw myself as a researcher in the future.” Overall, they were grateful for the opportunity to learn about health disparities and the role of research in addressing the needs of the minority community.

For the program evaluation, we also asked the ReTOOL trainees to confirm aspects of the program that they liked least. Three remarked that research experiences and mentoring were unsatisfying. Mentoring, like other relationships, may not be without problems. Researchers have reported that mentoring can be effective or detrimental. Negative impact may result from poor communication, conflicts of interest, and a lack of commitment [2629]. For example, one participant explained that “research experiences are limited to oncology.” He wanted opportunities in ambulatory settings. Two others reported difficulties with their mentor, including infrequent contact or negative interactions. “My lab mentor was not very involved and [was] discouraging about my research. He gave me old research...instead of something he was currently working on. If it weren’t for the post-doc, I would not have done much research during the program.” Along the same lines, one participant pointed out that “some ReTOOL scholars have no say in what they study with their mentor for the summer.” Two reported that the program was too long, while others felt that weekly seminars were unnecessary. Two participants requested that information about how to write a grant proposal be added to the curriculum. Two others wished that “more competitive funding or housing assistance” had been provided. These findings provide program information for continuous quality improvement. For example, we have implemented a mentor information session prior to the program implementation each summer and mentors who are not helpful for trainees are dropped from the program. In addition, we have included a ReTOOL program information session for potential trainees to provide clear expectations about the program focus on oncology, length of the program (which cannot be modified), and importance of weekly research seminar (which cannot be modified).

Discussion

The ratings for the ReTOOL program by participants were highly positive, a clear indication that the program was achieving its goals, especially its primary goal of bridging trainees to graduate or health science professional programs (79%). Taken together, the quantitative and qualitative findings suggest that the ReTOOL program has been highly successful in fostering participant enrollment in advanced health-related or professional degree programs. Participants explained how program experiences heightened their belief that science is a viable and positive career choice, an important component to pipeline development. The degree to which they internalized and aligned with values of the scientific community was remarkably enhanced. These findings suggest that the program has been instrumental in socializing and acculturating participants into the habits of scientific thinking.

As shown via the qualitative evaluative data, the ReTOOL program helped participants: (1) identify and shape research interests, (2) expand their knowledge of cancer related research and race disparities; (3) think critically, question their own, and others’ research; (4) better communicate and understand scientific results; (5) become aware of grant funding and the technical skills needed to write proposals; (6) appreciate participant confidentiality, laboratory reproducibility, and proper documentation of results; and (7) value of teamwork. Beyond the outstanding research training experiences offered within the ReTOOL program, trainees learned the importance of bridging research gap in the area of cancer disparities.

It is worth noting that the program evaluation focused on a 15-week programmatic experience at a single location. It is acknowledged that only a segment of the sample responded to either quantitative or qualitative questions. Thus, the perceptions of non-respondents is unknown. Also, it is acknowledged that the further participants are from the year of completion, the likelihood of responses being dependent on retrospective memory increases. Thus, the findings may be limited with respect to the accuracy of participant responses due to recall bias. A 27% loss to follow-up due to lack of trackable emails is acknowledged as another limitation. In spite of these limitations, the program assessment report provides a key insight on the impact of summer research training programs for URM students. The topic is significant and has relevance for cancer education. Especially noteworthy is the fact that the ReTOOL program reached a large number of Black trainees. We have learned several lessons regarding the importance of follow-up to ensure trackable emails. As a result, we plan to contact program participants every 6 months to ensure that we have correct email addresses and to account for potential name changes. We plan to contact participants through email, texting, and by establishing a program Facebook account.

We recommend that developers of cancer education training program implement rigorous evaluation. The combined use of quantitative and qualitative evaluative approaches is likely to provide a comprehensive overview for continuous quality improvement. The depth of insight available from the use of both approaches will benefit cancer education training program developers and trainees as well as those who they will serve.

Although long-term outcomes among trainees in these types of programs take time to accrue, we have seen highly positive outcomes within 7 years. Overall, the findings demonstrate promise for pipeline development for the field of biomedical research or clinical practice. Future studies should focus on trainee outcomes across similar programs to identify similarities and differences and to offer formative guidance about potential program restructuring.

Conclusion

The ReTOOL program provides trainees summer research training opportunities that includes didactic classes, seminars, grand rounds, career exploration through myIDP website, connection with academic resources to bridge to graduate, and professional health care programs and mentoring. Faculty mentors help trainees acquire insight into careers and programs of potential interest. The ReTOOL program broadens trainees’ career interests by increasing their opportunities to engage with those in varied career fields. Over time, it is likely to increase the representation of racially and ethnically diverse, well-trained biomedical scientists who contribute to the science of cancer disparities.

Acknowledgments

The authors wish to thank Ms. Shannon Pressey and Mr. Kenneth Stokes for their role in supporting the ReTOOL program. Dr. Paul Okunieff is acknowledged for his support in developing and implementing the ReTOOL program. In addition, we would like to acknowledge the RETOOL faculty advisors and mentors for their commitment to training underrepresented minority students.

Funding Information

The program was funded by the Department of Defense Prostate Cancer Research Program of the office of the Congressionally Directed Medical Research Programs (Awards W81XWH-12-1-0083 and W81XWH-14-1-0243) and NIH/National Cancer Institute (Awards P20CA192992 and R25CA21422).

Footnotes

Compliance with Ethical Standards

Conflict of Interest The authors declare that they have no conflict of interest.

Ethics Approval Not applicable; this was an evaluation program.

Consent to Participate Not applicable; this was an evaluation program.

Consent for Publication Not applicable; this was an evaluation program.

Code Availability Not applicable.

Data Availability

The program data are not available publicly because they include student identifiers, which are protected.

References

  • 1.Odedina FT, Reams RR, Kaninjing E, Nguyen J, Mochona B, Lyon DE, Askins N, Behar-Horenstein LS (2018) Increasing the representation of minority students in the biomedical workforce: the ReTOOL program. J Cancer Educ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Hedin JT (1988) Minority teens in research. Gift Child Today 11(3):19–20 [Google Scholar]
  • 3.Valdez JR, Duran RP (1991) Mentoring in higher education. Paper presentation at the Annual Meeting of the American Educational Research Association (Chicago, IL April)
  • 4.Frierson HT (1996) Perceptions of faculty preceptors in a summer research program targeted at minority undergraduate students. Paper presentation at the Annual Meeting of the American Educational Research Association, NY: (April 8–13, 1996) [Google Scholar]
  • 5.Walter NB (1997) Retaining aspiring scholars: recruitment and retention of students of color in graduate and professional science degree programs. Paper presentation at the Annual Meeting of the Association for the Study of Higher Education (ASHE) (22nd). Albuquerque NM (November 6–9, 1997) [Google Scholar]
  • 6.Thurmond VB, Cregler LL (1999) Why students drop out of the pipeline to health professions careers: a follow-up of gifted minority high school students. Acad Med 74(4):448–451 [DOI] [PubMed] [Google Scholar]
  • 7.Johnson JC, Williams B, Jayadevappa R (1999) Mentoring program for minority faculty at the University of Pennsylvania, School of Medicine. Acad Med 74(4):376–379 [DOI] [PubMed] [Google Scholar]
  • 8.Lewis V, Martina CA, McDermott MP, Trief PM, Goodman SR, Morse GD, LaGuardia JG, Sharp D, Ryan RM (2016) A randomized controlled trial of mentoring interventions for underrepresented minorities. Acad Med 91:994–1001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Desmond RA, Venkatesh R, Padilla LA, Daniel CL, Litton AG, Heimburger DC, Michael Brooks C, Waterbor JW (2019) Twenty-five-year follow-up of short-term Cancer research trainees at the University of Alabama at Birmingham: a brief report. J Canc Educ 34:1010–1013. 10.1007/s13187-018-1404-y [DOI] [PubMed] [Google Scholar]
  • 10.Saldaña J (2013) The coding manual for qualitative researchers. Sage, Los Angeles [Google Scholar]
  • 11.Creswell JW (2015) Educational research: planning, conducting, and evaluating quantitative and qualitative research, 5th edn. Pearson, Upper Saddle [Google Scholar]
  • 12.Chakraborty PK, Zhang Y, Coomes AS, Kim WJ, Stupay R, Lynch LD, Atkinson T, Kim JI, Nie Z, Daaka YG (2014) Protein-coupled receptor kinase GRK5 phosphorylates Moesin and regulates metastasis in prostate Cancer. Cancer Res 74:3489–3500 [DOI] [PubMed] [Google Scholar]
  • 13.Elsamadicy AA, Adogwa O, Sergesketter A, Behrens S, Hobbs C, Bridwell KH, Karikari IO (2017) Impact of age on change in self-image 5 years after complex spinal fusion (≥5 levels). World Neurosurg 97:112–116 [DOI] [PubMed] [Google Scholar]
  • 14.Elsamadicy AA, Adogwa O, Sergesketter A, Hobbs C, Behrens S, Mehta AI, Vasquez RA, Cheng J, Bagley CA, Karikari IO (2017) Impact of race on 30-day complication rates after elective complex spinal fusion (≥5 levels): a single institutional study of 446 patients. World Neurosurg 99:418–423 [DOI] [PubMed] [Google Scholar]
  • 15.Lee J, Dallery J, Laracuente A, Ibe I, Joseph S, Huo J, Salloum RG (2019) A content analysis of free smoking cessation mobile applications in the USA. Cambridge Core 14(4):195–2020. 10.1017/jsc.2019.6 [DOI] [Google Scholar]
  • 16.Mochona B, Jackson T, McCauley D, Mazzio E, Redda KK (2016) Synthesis and cytotoxic evaluation of pyrrole hetarylazoles containing benzimidazole/pyrazolone/1,3,4-oxadiazole motifs. J Heterocycl Chem 53:1871–1877 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Morozov VM, Li Y, Clowers MM, Ishov AM (2017) Inhibitor of H3K27 demethylase JMJD3/UTX GSK-J4 is a potential therapeutic option for castration resistant prostate cancer. Oncotarget 8(37): 62131–62142. 10.18632/oncotarget.19100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Odedina F, Oluwayemisi AO, Pressey S, Gaddy S, Egensteiner E, Ojewale EO, Moline OM, Martin CM (2014) Development and assessment of an evidence-based prostate cancer intervention programme for black men: the W.O.R.D. on prostate cancer video. Ecancermedicalscience 8:460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Roach T, Alcendor DJ (2017) Zika virus infection of cellular components of the blood-retinal barriers: implications for viral associated congenital ocular disease. J Neuroinflammation 14:43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Taylor TR, Barrow J, Makambi K, Sheppard V, Wallington SF, Martin C, Greene D, Yeruva SLH, Horton S (2017) A restorative yoga intervention for African-American breast Cancer survivors: a pilot study. J Racial Ethn Heal Disparities 5:62–72. 10.1007/s40615-017-0342-4 [DOI] [PubMed] [Google Scholar]
  • 21.Hacker BM, Subramanian L, Schnapp LM (2013) Strategies for mentor matching: lessons learned. Clin Transl Sci 6(5):414–416. 10.1111/cts.12050 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Eller LS, Lev EL, Feurer A (2014) Key components of an effective mentoring relationship: a qualitative study. Nurse Educ Today 34: 815–820. 10.1016/j.nedt.2013.07.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Bennett LM, Gadlin H (2012) Collaboration and team science: from theory to practice. J Investig Med 60(5):768–775. 10.2310/JIM.0b013e318250871d [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Croyle RT (2008) The National Cancer Institute’s transdisciplinary centers initiative and the need for building a science of team science. Am J Prev Med 35(2):S90–S93 [DOI] [PubMed] [Google Scholar]
  • 25.Croyle RT (2012) Confessions of a team science funder. Transl Behav Med 2(4):531–534 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Bickel J, & Rosenthal SL (2011). Difficult issues in mentoring: recommendations on making the ‘undiscussable’ discussable. Acad Med, 86, 1229–1234. doi: 10.1097/ACM.0b013e31822c0df7 [DOI] [PubMed] [Google Scholar]
  • 27.Blazer DG, Hernandez LM (2006) Genes, behavior, and the social environment: moving beyond the nature/nurture debate. National Academies Press, Washington, D.C. [PubMed] [Google Scholar]
  • 28.Eby L, Allen T (2002) Further investigation of protégés’ negative mentoring experiences. Group Organ Manag 27:456–479. 10.1177/1059601102238357 [DOI] [Google Scholar]
  • 29.Eby L, McManus S, Simon S, Russell J (2000) The protégé’s perspective regarding negative mentoring experiences: the development of a taxonomy. J Educ Behav 57:1–21. 10.1006/jvbe.1999.1726 [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The program data are not available publicly because they include student identifiers, which are protected.

RESOURCES