A Translational Approach to Cancer Research Education and Training

Abstract

The demand for biomedical researchers and health science professionals has increased over the past several decades. This need is particularly acute in the fields of cancer research and oncology in which technological advances have fueled an unprecedented pace of laboratory discoveries and their applications in novel diagnostic and therapeutic strategies. Internships that expose undergraduate students to cancer research and patient care serve an important function in meeting this need by educating trainees about careers in this field and inspiring them to pursue these professional paths. Moreover, the translational impetus of cancer research incorporates research, regulatory, business and clinical components, providing students with even more cancer-focused career options. With the goal of providing hands-on experiences in cancer research and oncology to undergraduate students who comprise the next generation of cancer physician-scientists and will fill this demand in our professional workforce, the Nathan Schnaper Intern Program (NSIP) has grown from a small laboratory-based local summer internship to a competitive national program. In this study, we evaluate three new modules of the NSIP research, education and clinical components that have been implemented in the first two years of National Cancer Institute Cancer Research Education Grants Program funding. The impact of these modules on intern satisfaction, learning and near-term career trajectory are assessed to identify the most effective approaches and key measures of program outcomes.

BACKGROUND.

Significant progress in the detection and treatment of human cancers has reduced mortality rates from this disease. However, cancer remains a major health concern, with over 1.7 million new cases and more than 600,000 deaths due to cancer projected for 2019 [1]. Recent advances in sequencing technologies, molecular analysis of cancer cells and understanding of the critical role of the immune system in disease progression have fueled the translation of scientific discoveries into novel diagnostic and therapeutic approaches. Integral to these translational medicine success stories is the bidirectional collaboration of laboratory scientists and oncologists. To continue the productive bench to bedside and back paradigm that is required to fight the diverse malignancies that comprise this complex disease, it is essential to educate the next generation research scientists and clinical oncologists on the need for, and importance of, this collaborative translational pipeline. However, despite this unprecedented opportunity to impact cancer detection and treatment, there is a shortage of young scientists and clinicians with strong training in the field of cancer [2–4] and the need for oncologists is predicted to outpace clinicians trained in this field by 2025 [5].

Undergraduate internships provide authentic workplace experiences at an early stage in their education that help students make informed decisions about their future paths [6–10]. Providing an accurate picture of the different career paths that comprise highly collaborative fields such as translational medicine is particularly important. This ‘real life’ exposure introduces interns to a spectrum of related careers while illustrating the interdisciplinary interactions that are central to this work and dispelling any one-dimensional preconceptions of specific professions. From the institution and mentor’s perspective, internships provide an important mechanism to recruit students into specific fields while interns gain training and credentials that will enhance their success on this professional trajectory [11]. Incorporating the internship model, the National Cancer Institute created the Cancer Research Education Grants Program (CREGP) R25 funding opportunity announcement that includes a Short-term Research Experiences program “to provide hands-on exposure to cancer research in a laboratory or a field setting for a full-time (40 hours per week) period of 10 to 15 weeks to stimulate the interest and advance the knowledge base of participants to consider further education and training for future careers as cancer researchers” [12,13]. Recent CREGP funding for the Nathan Schnaper Intern Program (NSIP) at the University of Maryland Marlene and Stewart Greenebaum Comprehensive Cancer Center (UMGCCC) has supported the development of innovative new components to achieve CREGP goals.

The NSIP evolved from a small local program initiated over 50 years ago created to provide undergraduate laboratory assistants to researchers at the Baltimore Cancer Research Program (predecessor to the current UMGCCC), to a nationally competitive internship with the overarching goal of inspiring and educating the next generation of cancer researchers and oncologists. Among many notable early alumni of this program was the late Dr. Merrill Egorin, who served as an intern in 1969 when he was an undergraduate student at the Johns Hopkins University by the Program’s first director, Dr. Nicholas Bachur. Dr. Egorin, who pioneered research in the pharmacology of cancer therapeutics and is internationally recognized for his contributions to that field [14], joined the UM Cancer Center as a founding member in 1981. In addition to leading cutting-edge research and clinical studies, Dr. Egorin fittingly assumed the role as director of the internship that originally gave him his start in cancer research. With the growth of the UM Cancer Center and corresponding increase in summer interns- that numbered between 8 and 12 during that period- Dr. Egorin introduced structured educational components into the program that included weekly seminars and end-of-summer presentations. Further growth of the internship from a local program advertised via ‘word of mouth’ to one that offered research experience to undergrads across the US came in 1997 when Dr. Doug Ross took over as program Director. As the then president of the American Association for Cancer Education (AACE), Dr. Ross recognized how a national presence would enhance the quality and diversity of the applicant pool. Towards this goal, he created the first internship website and online application that had the anticipated positive impact on the program (https://www.umm.edu/nsip). The combination of out-of-state and local undergraduates in the internship cohorts significantly enhanced the cohesiveness of the group and visibility of the program nationwide. Feedback on the program revealed a strong desire for a clinical component to provide interns, many of whom were aspiring physicians, with a broader perspective of their research applications. Accordingly, shadowing of UMGCCC oncologists was incorporated into the curriculum, providing the seed for the current focus on translational research. The growth in program components and reputation led to a dramatic increase in demand for internship positions- the Program typically receives 4–5-times more applications from highly qualified students than it can support; however, stable funding to match this demand was lacking. The annual budget, and thus number of interns admitted, depended on philanthropic donations and cancer center contributions. Recognizing this need, friends and patients of the late Dr. Nathan Schnaper, an iconic figure in the UM Cancer Center and ardent supporter of the program who met personally with the interns each year [15], created an endowment to provide a foundation for sustained support of program components. In honor of Dr. Schnaper’s lifelong commitment to the program, the current program bears his name. With this important institutional commitment in place, NSIP leadership sought and received NCI CREGP funding to provide the enhanced level of support required to expand program scope and content and create a top tier internship that integrates research, educational and clinical components (Fig 1). Here we evaluate three new modules of NSIP programming that were introduced in the first two years of CREGP funding. The impact of these modules on intern satisfaction, learning and near-term career trajectory are assessed to identify the most effective approaches and key measures of program outcomes.

Fig 1.

The NSIP integrates cancer-focused research, clinical and educational components

METHODS.

Study Participants.

1. Interns.

Recruitment and selection.

NSIP interns are full-time undergraduate students who are US citizens or permanent residents recruited from colleges and universities across the US through an interactive website and program representation at national meetings. Specifically, the NSIP Director, Coordinator and faculty mentors distribute program literature and give talks at cancer-focused meetings such as AACE and the American Association of Cancer Research. In addition, the NSIP is represented at meetings featuring students from minority groups such as the Annual Biomedical Research Conference for Minority Students and the Society for the Advancement of Chicanos/Hispanics and Native Americans in Science, with the goal of enhancing diversity in the applicant pool. To supplement this national exposure, the program Director visits local universities each year to give presentations on the NSIP and meet with students. Among these, NSIP maintains strong relationships with several smaller institutions and Minority Serving Institutions that have played foundational roles in the program’s development to ensure their students have access to this robust internship experience.

Internship applications are reviewed by a selection committee comprised of past and current NSIP mentors. Each application is reviewed by two committee members and scored on a 1–9 scale (1 = best) based on short answer questions, a personal statement, two letters of recommendation and transcript. Importantly, prior research experience is not a criterion for selection, as training in laboratory techniques is included as a new component of the NSIP curriculum (see below). This policy ensures that talented students from diverse backgrounds, who have not had the opportunity or resources to conduct research at their undergraduate institutions, have access to the internship. The committee discussion focuses on the applicant’s interest in cancer research or oncology and how the internship experience will advance their educational and career goals as driving criteria for selection.

Composition of cohorts.

The NSIP applicant review format has historically resulted in the enrollment of robust cohorts of interns and is evident from the academic and demographic data in Table 1. Outcomes evaluated in this study focus on the first two years of program implementation following CREGP funding (2016, 2017). Information on the 2018 cohort is also provided in Table 1 since the results section includes data from a new smart phone survey application collected from this group. The prevalence of female NSIP interns in all years reflects a corresponding distribution in the applicant pool and parallels a national trend for the biomedical sciences at all levels of post-secondary education [16]. All procedures performed in studies involving human participants were in accordance with the ethical standards of our institution and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Intern participation in the NSIP does not require formal consent and is not subject to approval by our Institutional Review Board. No personal identifying information is presented in this report; only summary data are analyzed.

Table 1.

NSIP Intern Statistics.

Year	2016	2017	2018	All years
Undergraduate institutions	16	14	16	34
% Fr/So/Jr/Sr	10/47/33/10	10/50/30/10	0/33/57/10	5/32/53/10
Mean GPA	3.8	3.6	3.7	3.7
States of Residence	7	7	9	13
Female	85% (18)	70% (14)	76% (16)	77% (48)
URM	24% (5)	35% (7)	33% (7)	31% (19)
# interns	21	20	21	62

2. Mentors.

Mentors are UMGCCC Program in Oncology (PIO) members from UMB professional schools and partner institutions; the NSIP mentor pool is comprised primarily of faculty from the School of Medicine. Mentors sign a formal agreement articulating their responsibilities that include guiding the research project, meeting regularly to review progress, assisting in the preparation of intern presentations, attending NSIP events, and evaluation of program components.

Evaluation.

Evaluation of intern satisfaction and learning was performed by pre- and post-activity surveys using a 5-point Likert scale (1= Excellent; 2 = Very Good; 3 = Good; 4 = Fair; 5 = Poor) or as defined in the figure legends. Surveys were administered via electronic forms in 2016 and 2017. A smart phone application was used to collect the subset of data reported for the 2018 cohort. Data shown in tables are the means of all intern responses; the standard deviation of the means was <0.1 for all datasets and is not shown. The significance of differences in intern responses to pre- and post- module surveys in Fig 2 was determined using a two-tailed Student’s t-test with p-values indicated in the text and figure legends. One-year follow-up surveys were administered remotely via Survey Monkey (www.surveymonkey.com).

Fig 2.

Interns’ knowledge of breast cancer topics from pre- and post AI module surveys. Percentage of interns from the 2018 cohort at each level is depicted in shaded bars (left axis); weighted mean values of survey responses are shown on each line (right axis). Statistical significance in the difference between pre- and post-module values is indicated for each topic (*p<0.05; **P<0.01).

RESULTS.

A Laboratory Practice and Techniques (LPT) class enhances intern research skills independent of their prior level of training.

Research experience is not a criterion in the evaluation of NSIP applicants; accordingly, interns enter the program with a wide range of laboratory skills. With the goal of providing training in fundamental laboratory techniques that are broadly applicable in cancer research and will foster the interns’ understanding of, and productivity in, their independent UMGCCC research projects, we partnered with Loyola University Maryland to introduce an intensive hands-on class at the beginning of the program. Led by Loyola faculty and held at state-of-the-art teaching laboratories on the Loyola campus that is located just six miles from the UMGCCC, the LPT class integrates laboratory etiquette, mathematical calculations (e.g. dilutions) and recordkeeping concepts into sections on cell culture, confocal microscopy and gene expression (RNA/protein) analysis. In the first year of implementation, the LPT class comprised a two-week module that 76% of interns rated as ‘too long’ in exit surveys; anecdotal information indicated that this sentiment was widely shared among NSIP mentors. Accordingly, the class was condensed to one week in subsequent years; this new format was favorably received with 78% of interns rating the length as ‘just right’. Importantly, the streamlined curriculum did not adversely impact intern evaluations of class content as survey responses were not markedly different in the first and second years. Specifically, intern surveys revealed that the LPT class both increased their practical laboratory skills and was directly relevant to their projects (Table 2). All ratings fell into the excellent to very good range and this high satisfaction was reflected in intern comments from their surveys (excerpts below). A trend toward lower ratings was observed for the confocal microscopy section but did not reach statistical significance (p> 0.05). Intern feedback suggested that this difference reflected the specialized nature and more limited use of this technique in their UMGCCC projects (excerpts below).

Table 2.

Intern evaluations of LPT laboratory components.

LPT component	NSIP year	Cell culture		Confocal microscopy		Protein/RNA Analysis
Research exp. (mos)		0–12	12+	0–12	12+	0–12	12+
Topic is relevant to NSIP project	2016	1.1	1.1	2.2	1.6	1.0	1.5
	2017	1.5	1.4	1.8	2.0	1.0	2.2
	mean	1.3	1.25	2.0	1.8	1.0	1.85
Increased intern research skills	2016	1.1	1.3	1.7	1.4	1.0	1.0
	2017	1.0	1.0	1.0	1.2	1.0	1.4
	mean	1.05	1.15	1.35	1.3	1.0	1.2

A 5-point Likert scale in which 1 is the top score was used. The numbers of interns with 0–12 mos/12+ mos research experience are: 2016: 10/11; 2017: 9/11.

To assess the potential impact of prior research experience and exposure to techniques in the LPT course on the value that interns ascribed to this training, class evaluations among interns with less than, and greater than, one-year research experience were compared. This analysis revealed high intern evaluations in both years surveyed, independent of their existing level of laboratory training (Table 2). Intern rating of the RNA/protein analysis section, however, identified a trend toward lower scores from interns with one year or more of research experience. Although the difference in mean scores from interns with more or less research experience, all of which fell into the very good-good range, did not reach statistical significance, we queried interns to explore potential explanations for this trend. Intern responses revealed that for those interns who had done prior research, experience with RNA/protein analysis techniques, as compared to other methods in the LPT class, was the most prevalent component of their previous training. Thus, prior experience with gene expression analysis correlated with the lower value this group attributed to this LPT section. In contrast, RNA/protein measurement techniques were novel, and of greater value for interns who lacked this prior training. For the other laboratory skills, both interns with little or no prior laboratory experience, and those who had conducted one or more years of research, indicated that the LPT training helped them to better understand the scientific concepts underlying the techniques and learn important variables for optimizing experimental design. Other aspects of the LPT course, beyond the hands-on techniques learning, also received strong mean evaluation scores as shown in Table 3.

Table 3.

Intern evaluations of the LPT module.

LPT evaluation category	2016	2017
Impact on cohort cohesiveness	1.5*	1.4
Engagement of Loyola instructors	1.0	1.0
Quality of teaching facilities	1.0	1.2
Mentor’s feedback on utility of LPT training	1.9	2.1

^*mean evaluation scores using a 5-point Likert scale in which 1 is the top score are shown.

2016:

• “The [LPT] course really instilled a certain comfort level with working in the lab.”

• “I enjoyed the hands-on experience that we got—it served as a good refresher or taught us something new. The professors at Loyola were very passionate and it was great to learn from them.”

• “I very much appreciated being introduced to the common research techniques so that I could transition more smoothly into the lab. One of the best parts was having a week to get to know the other interns before we went to our separate labs.”

• “[confocal] was not very beneficial to learn in terms of my lab. However, I still think confocal microscopy should be included, because I found that to be one of the most interesting components”

• “creation of confocal slides was not as important of a skill to learn in terms of my lab work. [however,] the use of confocal microscope was an awesome experience that I would definitely keep.”

• “Confocal was super cool and exciting but could have spent less time with it because wasn’t relevant to my research.”

2017:

• “The LTP course was integral to having success in the lab this summer. It allowed me to work more independently from the start of the summer.”

• “The faculty that ran the LPT course at Loyola were amazing. They truly cared about us and cared that we learned something from our week with them. They treated us as their own students and were sincere in their desire to make us better scientists and students.”

• “The confocal microscopy lesson was a bit lengthy… I would’ve rather spent more time on the machine then learning history of microscopy that was hard to follow – although, Dr. Thompson and Dr. Brogan are always great to listen to.”

• “even though it was not useful for my research project, the opportunity to actually use a confocal microscope was amazing!”

Small Group Deconstruction of a UMGCCC Translational Research Success Story.

An established educational component of the NSIP is the ‘Bench to Bedside and Back’ series that features interactive seminars by UMGCCC investigators on spectrum of translational research topics. To augment these talks we sought to provide interns with a more cohesive picture of all the steps involved in translational research by following the development of a specific therapeutic agent. To achieve this objective, we introduced a new module focused on the pioneering research of the late UMGCCC researcher Dr. Angela Brodie, whose discovery of aromatase inhibitors (AI) led to the current front-line therapy for hormone responsive breast cancer [17,18]. This intern-driven activity begins with an overview of the AI story by a faculty trainee of Dr. Brodie. Interns are then divided into five groups of 4–5 interns each of which is assigned different steps in the AI translational research story: discovery, in vitro studies, animal models, clinical trials, and current applications and advances. Preceptors for each group provide reading material to guide the interns’ research and meet with them twice over the course of two weeks. In the first meeting, groups review their readings, discuss questions and plan their presentations; in the second meeting they polish and practice their presentations. The small groups’ research culminates in presentations to the entire cohort as part of a half-day mini-symposium on this landmark work. A unique feature of this module is that the small group preceptors are alumni from the Brodie lab who authored the studies that the interns research. This direct connection to the research not only provides an engaging back story that goes well beyond the published studies but gives interns first-hand insights into the challenges and rewards of this groundbreaking work. In evaluations of this new module, interns praised all aspects of its implementation, content and effectiveness in achieving the primary goal of increasing their understanding of the steps involved in translational research (Table 4). In particular, intern comments revealed that the preceptors’ ‘insiders’ input’ on the studies was a strength of the module. This aspect was further enhanced by Dr. Brodie’s attendance and active participation in the mini-symposium in the first year it was introduced.

Table 4.

Intern evaluations of AI Small Groups module.

AI Small Groups evaluation category	2016	2017
Mentors were helpful in understanding the content and organizing the talks	1.2	1.3
Group size allowed participation and input from all members	1.0	1.0
Readings were appropriate in content and number	1.5	1.4
Presentation format effectively explained key events in the translational research pipeline	1.5	1.1

2016:

• “Getting to talk to the people who actually conducted this research, especially Dr. Brodie (I think everyone secretly wanted to get a picture and an autograph), was one of the best parts of the internship!”

• “I really like that I got to connect with the other interns and talk to people from the Brodie lab” “[the module] gave us insight to the major hallmarks in breast cancer treatment… I really got a sense of the importance of bench work and scientists and how their work is truly translational and serves as the foundation for medicine today.”

• “I really enjoyed this new aspect of the summer. I only suggest that maybe when discussing the individual talks with their groups, mentors could elaborate more on the decision-making process of each transition between stages in the process. Not just what they did but how and why Dr. Brodie designed each study the way she did.”

2017:

• “I like how this connected the interns to research conducted right at UMB. It was helpful in the sense that we got more exposure to reading scientific papers as well as following a true translational cancer research story from beginning to end.”

• “I really enjoyed deconstructing the development of Aromatase Inhibitors in our small group modules. Over the past year I actually had the opportunity to work with Aromatase inhibitors on a research project, so it was very interesting to see how much actually went into the process of transitioning from bench to bedside.”

To assess the interns’ grasp of specific concepts in this module, a new smart phone application was introduced in year three of the program to administer pre- and post-module surveys, facilitate recording of intern responses and enhance their compliance. Results from the first year in which this evaluation tool was implemented revealed a significant increase in the interns’ understanding of material presented by each of the small groups (Fig 2). Specifically, a comparison of pre- and post-module responses demonstrated improvement in the mean subject matter understanding with the percentage of interns reporting the highest level of knowledge more than doubling after completing the module (increases in the bright red-shaded bars in Fig 2). This increase was observed across all topics providing an indication of the effectiveness of this learning format. The interns’ evaluations of the AI small groups module were largely positive; however, some interns indicated that this component should be coordinated with a scientific communication workshop that is typically scheduled later in the summer. Furthermore, several interns suggested that more than two preceptor meetings would help them to better prepare for the presentation. This feedback has been incorporated into the development of a revised NSIP schedule.

Interns gain a lens toward the clinical side of translational research through participation in leukemia boards.

NSIP interns gain first-hand experience in clinical medicine and exposure to the human side of cancer by shadowing UMGCCC oncologists. With the goal of providing a clinical perspective of translational cancer research at the UMGCCC and an appreciation for the innerworkings of this process, interns attend UMGCCC Leukemia Boards. In this new component of the program, Oncology Fellows meet with groups of interns immediately prior to the Leukemia Boards to review different cases that will be presented, introduce therapeutic approaches and treatment regimens and explain ongoing clinical trials. After listening to the cases and data presented in the Leukemia Boards, interns again convene with the Oncology Fellows in a follow-up session to ask questions and discuss interpretations. This first-hand exposure to oncologists working in a team setting to evaluate novel agents in the context of patient data provides profoundly meaningful outcomes that link directly to the research topics interns have studied and conducted throughout the summer. Indeed, Leukemia Board presentations include data from patient samples generated in by laboratory scientists illustrating the critical importance of the bi-directional flow of information between the laboratory and the clinic in translational research.

The effectiveness of the sessions with Oncology Fellows to prepare interns for the material in the Leukemia Boards, and to help them digest this information after the conference, received high ratings from interns and fell in the ‘excellent’ to ‘very good’ categories (Likert scores between 1 and 2; Table 5). Interns also gave high ratings to the Leukemia Boards session per se, despite the fact that they were not familiar with much of the material presented. Indeed, while it was not realistic for the interns to review all cases prior to the meeting, the experience of understanding just a subset of cases discussed effectively ‘whet their appetites’ for more learning on this topic as reflected in their evaluations below. Specifically, comments on this module provided an important additional source of feedback and fell primarily into the areas of: i) insights into the roles of a clinician, ii) the practical interface between laboratory and clinic iii) the desire to understand more about the detection and treatment of this malignancy as exemplified in the excerpts below.

Table 5.

Intern evaluations of Leukemia Boards Module.

Leukemia Boards evaluation category	2016	2017
Pre-conference discussion with Oncology fellows provided a good preview of topics	1.3	1.0
Conference provided insights into factors dictating therapeutic regimens and strategies	1.7	1.5
Post-conference discussion clarified subjects and enhanced understanding	1.5	1.5

2016:

• “This was an incredible experience to gain insight into the lives of doctors. I did not realize that doctors from many specialties meet regularly to discuss particularly challenging cases… The introduction about leukemia prior to the conference was extremely helpful and I felt like I understood some (more than I expected) of the doctors’ conversation. The de-briefing afterwards to ask questions was also very valuable because many new words/ideas/concepts were introduced during the conference.”

• “the conference was difficult to follow sometimes when the physicians used acronyms I was not familiar with”

• “It was useful to see how the doctors collaborate with different doctors, pathologists, and genetics.”

• “The pre- and post-conference discussions were so well run… it [the conference] provided a great opportunity to witness what physicians do on a day to day basis.”

2017:

• “The pre- and post-conference discussions helped me to prepare to enter the conference with some background knowledge and help me collect my thoughts and understandings after the conference. I also like the conference itself because it gives a glimpse into not only the daily lives of doctors, but also how they discuss different treatment options or potential clinical trials, which are all stemming from lab research, so it ties many interesting components together.”

• “I was unaware of the fact physicians had those types of meetings, so it was very eye opening to see one.”

• “I personally got a little lost in the fast-paced leukemia conference, but it was interesting to experience the environment of those types of conferences.”

• “It was interesting to see the patient cases presented like a case study. Even though I do not know that much about leukemia, the pre-conference discussion made the actual conference much easier to follow.”

• “The leukemia conference provided a realistic look into the discussion and debate that goes into patient care.”

DISCUSSION.

The NSIP has a strong record of educating undergraduate students in cancer biology and oncology through hands-on research experiences. With the overarching goal of inspiring and educating the next generation of cancer researchers and oncologists, we introduced three new modules supported through a CREGP grant to enhance the NSIP research component and emphasize the translation of laboratory discoveries into clinical applications. Surveys and evaluations were collected to assess the effectiveness of module implementation and the interns’ understanding of content presented. All evaluations were collected in a self-report format that is inherently subject to biases due a number of potential factors including the interns’ preferred teaching formats or academic interests. In addition, interns may give program components high ratings in an attempt to please leadership or mentors from whom they plan to request future professional references. To counteract the potentially confounding influence of these factors on program evaluation, surveys are conducted anonymously and interns are assured that their standing in the program is independent of any outcomes. In the case of the knowledge-based assessments conducted for the AI module, pre-module surveys were given after introductory lectures so the interns could first gain an appreciation for the complexity of cancer research and better evaluate their true initial knowledge of this subject. This strategy was implemented to help avoid artificially high pre-module scores based on the interns’ inflated perception of their initial cancer knowledge; i.e. at the beginning of the program, interns may ‘not know what they don’t know’. The increase in their understanding across a range of different topics suggests that this strategy provided an accurate assessment of information gained.

Intern feedback and evaluations provided valuable information to improve these new modules. For example, shortening duration of the LPT component in response to intern suggestions provided an additional week for research in their UMGCCC labs without adversely impacting the LPT content. Furthermore, survey results indicated that all interns found this training to be valuable, dispelling initial concerns that it may not be a productive use of time for the subset of interns with significant prior research experience. The AI Small Groups module was widely praised as an engaging learning activity that provided important insights into translational research. However, feedback indicated that additional time with the preceptors, and holding the scientific presentation workshop in advance of the AI module, comprise ways in which this module can be improved. Implementation of a smart phone application to assess the interns’ knowledge acquisition in this module revealed that each small group presentation was effective in increasing the other interns’ understanding of their step in the translational research pipeline. The Leukemia Boards module exposed interns to the real-life interplay between clinicians and researchers and a level of professional teamwork that both surprised and excited most interns. Interns appreciated the Oncology Fellows’ guidance in navigating an overwhelming array of new information and several commented that even more background material would have helped to understand additional cases that were presented. While a full review of cases to be discussed is neither practical nor appropriate for interns at the undergraduate level, such comments indicate that the module was successful in achieving our goal of stimulating their interest in the clinical aspects of translational research.

Evaluation and assessment of the new NSIP modules supported their effectiveness in increasing the interns’ understanding of translational cancer research and provided valuable feedback that will be incorporated into program implementation in future years. However, the impact of these components on the interns’ educational plans and professional paths will require short- and long-term follow-up surveys and tracking of their career trajectories [19–24]. In this regard, a collateral benefit of NSIP group modules is the cohort cohesiveness that develops over the course of the summer and, in turn, provides a foundation to continue these connections as NSIP alumni. Such an alumni network greatly facilitates the maintenance of an up to date database of contact information that is critical for long-term tracking. Towards this goal of longitudinal tracking, one-year follow-up surveys are sent to assess the impact of the NSIP experience on their subsequent undergraduate education. Responses from the 2016 and 2017 cohorts indicate that NSIP research and educational components enhanced the intern’s understanding of, and engagement in, their upper level courses in cancer specifically and biological sciences in general (Table 6). As success in specific undergraduate disciplines often correlates with the students’ ultimate career paths, the NSIP experience may positively impact the students’ decisions to enter biomedical research or clinical science fields. Consistent with this prediction, over 85% of alumni have matriculated to professional schools since the program’s inception and 100% in the three years of CREGP funding. During this period, 47% have entered medical school, 33% have matriculated to PhD or MD/PhD programs and 20% have taken other professional paths (e.g. dentistry, industry or government research). The continued success of the NSIP and programs like it nationwide is important to fill the need for talented cancer researchers and oncologists [2–5]. To accelerate progress towards this goal and expand upon the outcomes of individual programs, the development of a national network of programs connected through professional societies and Cancer Center Consortia provides a potential mechanism to both enhance program resource capacity and increase training opportunities for the next generation of cancer researchers. Such programmatic collaborations represent a logical area for future growth.

Table 6.

One-Year NSIP follow-up survey.

Question	Strongly agree	agree	neutral*	Survey Excerpts
My experience in the NSIP has had a positive impact on my subsequent coursework.	93%	7%	0%	“The weekly [NSIP] lectures provided me with knowledge of topics I had not yet encountered, so when they come up in classes now, they feel more familiar and are subsequently easier to grasp.”
				“Many of the mechanisms and pathways I learned about for my research were a major part of my curriculum for biochemistry”
The experiences gained in the NSIP have allowed or inspired me to pursue new opportunities.	71%	29%	0%	“Before NSIP I had never been able to conduct translational research… NSIP opened the doors to a new field of research that helped me discover my career goals.”
Participating in the NSIP has influenced my career path.	57%	36%	7%	“I was on the fence about pursuing and MD/PhD before NSIP. I feel that my research experience helped me finalize my decision to become a physician scientist.”
				“After participating in the NSIP, I definitely want research to comprise at least some part of my career in medicine.”
After my participation in the NSIP, cancer research and/or medicine are more likely to be part of my career.	72%	14%	14%	“My passion for cancer research has become even stronger. I am performing new research in a lab this semester as a consequence of gaining a greater interest in cancer research and techniques last summer.”
				“I feel very passionate about translational cancer research and hope to pursue it in throughout my career.”

^*note 0% of survey questions had responses of ‘disagree’ or ‘strongly disagree’ so these are not shown.