The origin and implementation of the Broadening Experiences in Scientific Training programs: an NIH common fund initiative

Abstract

Recent national reports and commentaries on the current status and needs of the U.S. biomedical research workforce have highlighted the limited career development opportunities for predoctoral and postdoctoral trainees in academia, yet little attention is paid to preparation for career pathways outside of the traditional faculty path. Recognizing this issue, in 2013, the U.S. National Institutes of Health (NIH) Common Fund issued a request for application titled “NIH Director’s Biomedical Research Workforce Innovation Award: Broadening Experiences in Scientific Training (BEST).” These 5-yr 1-time grants, awarded to 17 single or partnering institutions, were designed to develop sustainable approaches to broaden graduate and postgraduate training, aimed at creating training programs that reflect the range of career options that trainees may ultimately pursue. These institutions have formed a consortium in order to work together to develop, evaluate, share, and disseminate best practices and challenges. This is a first report on the early experiences of the consortium and the scope of participating BEST programs. In this report, we describe the state of the U.S. biomedical workforce and development of the BEST award, variations of programmatic approaches to assist with program design without BEST funding, and novel approaches to engage faculty in career development programs. To test the effectiveness of these BEST programs, external evaluators will assess their outcomes not only over the 5 yr grant period but also for an additional 10 yr beyond award completion.—Meyers, F. J., Mathur, A., Fuhrmann, C. N., O’Brien, T. C., Wefes, I., Labosky, P. A., Duncan, D. S., August, A., Feig, A., Gould, K. L., Friedlander, M. J., Schaffer, C. B., Van Wart, A., Chalkley, R. The origin and implementation of the Broadening Experiences in Scientific Training programs: an NIH common fund initiative.

Over the past decade, several national reports and commentaries reviewing the numbers, composition, career outcomes, and trajectories of the U.S. biomedical workforce have been published that have garnered the attention of the popular media (1–7). Uniformly, these reports point to a large number of predoctoral and postdoctoral trainees, the unusually long training period of this combined traineeship, and the dependence of biomedical research programs upon the contributions of these trainees. According to the 2012 U.S. National Institutes of Health (NIH) Advisory Committee to the Director report, ∼23% of the biomedical workforce is currently in a tenure-track faculty position (1). Furthermore, a vast majority of the graduates of biomedical training programs were shown to be in careers other than tenure-track faculty positions. These diverse career pathways include careers in government, regulatory science and academic administration, industry/biotechnology, science writing and communication, public policy, and teaching at primarily undergraduate institutions as well as nontenure-track research faculty positions at academic institutions (4, 8, 9). This trend has also been recognized in the United Kingdom (10).

Despite these realities, faculty who mentor predoctoral and postdoctoral scholars tend to focus almost exclusively on preparing trainees for tenure-track faculty careers. The paradigm is a straight and narrow pathway where trainees complete predoctoral training followed by postdoctoral training and then enter academic careers (Fig. 1A). Those who pursue other nonacademic biomedical careers do so in an inefficient manner, with little career-specific mentoring or resources. Scant attention has been paid during training to the preparation of trainees for diverse careers. Faculty mentors have neither been prepared to provide broad career guidance nor are they informed about how to direct their trainees toward jobs outside academia, even if they are enthusiastic about doing so. Moreover, few institutional resources have been allocated to support the faculty and trainees to balance competing demands. These trends have been reinforced by a culture that values academic careers above other options (e.g., NIH training grant criteria had historically emphasized trainee success only as obtaining academic positions).

Figure 1.

A) The career paths of biomedical trainees under the traditional model of Biomedical Graduate Education. In this linear process of scientific research and training with the direct mentorship from faculty advisors, Ph.D. and postdoctoral trainees perform scientific research and are encouraged toward tenure-track research, research or teaching positions in academia, or to research in industry (solid black arrows). Without formal preparation for other diverse careers, trainees obtained positions in other biomedical careers in an indirect manner with little career-specific training or support (dashed black arrow). B) The approach of BEST programs, where career development initiatives enhance graduate and postdoctoral preparation for a range of careers. Common BEST programmatic themes, such as career development [e.g., Individual Development Plan (IDP)], experiential learning (e.g., internships and externships), professional development (e.g., team building and management), and mentorship (e.g., from faculty and alumni), will directly guide trainees to many career possibilities, such as administration, law, industry/biotechnology, science policy/communication, government, and other science careers, which are viewed as equal, successful, and well-suited career outcomes.

Historically, the perception has been that trainees provide service by driving scientific research in the laboratory and that their preparation for diverse careers should occur after their Ph.D. or postdoctoral training. However, the broader workforce development needs of the country could be harmed by the dichotomy between completion of grant-funded tasks and career development. Today, there is a growing consensus that the full range of career paths should be included and defined such that tenure-track academia is only one possibility among many other options—all being viewed as successful outcomes (Fig. 1B). In fact, the NIH now recognizes this shift and has amended the parent funding opportunity announcement for T32 training grants to expand trainee evaluation beyond academic careers (11).

In March 2013, the NIH Common Fund issued a request for application (RFA) titled “NIH Director’s Biomedical Research Workforce Innovation Award: Broadening Experiences in Scientific Training (BEST) (DP7)” (12). Institutions involved in biomedical research training were invited to submit proposals designed to develop sustainable approaches to broaden graduate and postgraduate training, such that the training programs reflect the range of career options that trainees may ultimately pursue. Importantly, the intent of the RFA was that career development for trainees pursuing an academic faculty career cannot be the focus of the program but also should not be discouraged or negatively impacted by these new approaches. It was important to broaden approaches, not restrict them. The BEST awards were not designed to be training grants but, rather (as research grants), were envisioned to be a series of experiments aimed at identifying a range of the most effective practices to support the career development of all trainees. The expectations included broad institutional support and faculty engagement, as well as involvement of external partners representing careers in the private and nonprofit sectors. Institutions were expected to address the dissonance between trainees performing research under the mentorship of their faculty advisor while simultaneously trying to develop the independent competencies needed for entry into a wide range of science careers. The rigorous evaluation of these programs is intended to advance our understanding of how to create didactic and experiential opportunities to support all career outcomes. The results of the rigorous evaluation will be shared with the broader scientific training community, where other programs may benefit from program approaches and incorporate successful practices into new or existing career development training programs.

Consistent with the philosophies of the NIH Common Fund, these awards were explicitly launched as a 1-time 5 yr award, not eligible for renewal. The BEST institutions are expected to develop institutional structures that can be sustained beyond the award period.

RESPONSE OF THE BIOMEDICAL RESEARCH TRAINING COMMUNITY TO THE RFA

The response from the biomedical research training community was robust and the competition fierce, in line with success rates of other NIH applications. In round 1, 10 awards were made, followed by 7 in round 2 (Table 1). It was evident from the submissions that a large number of institutions had already recognized the issues raised in the biomedical workforce reports and were well underway to supporting a new design for doctoral training consistent with NIH goals by investing in and acting upon some of the necessary components laid out in the reports.

TABLE 1.

List of BEST awards and a link to each program

Institution	Title of award	Principal investigators	Link to program
Awarded in 2013
Cornell University	Cornell University BEST Program	Avery August	http://www.best.cornell.edu
Emory University and Georgia Tech	Beyond the Professoriate: Transforming Pathways for Biomedical Research Careers	Nael A. McCarty, Mary DeLong, Wendy Newstetter, Lisa A. Tedesco, Keith D. Wilkinson	http://www.gs.emory.edu/best
New York University School of Medicine	NYU STEP	Keith J. Micoli, Carol S. Reiss	http://www.med.nyu.edu/research/nyustep
University of California, Davis	Frontiers of University Training to Unlock the Research Enterprise (FUTURE)	Frederick J. Meyers, Lars F. Berglund, Andrew B. Hargadon	http://www.future.ucdavis.edu
University of California, San Francisco	Motivating INformed Decisions (MIND): Careers for the Future Biomedical Workforce	Theresa O’Brien, Jennie B. Dorman, William Lindstaedt, Keith R. Yamamoto	http://www.mind.ucsf.edu/
University of Colorado, Denver/Anschutz Medical Campus	Innovative Biomedical Graduate Training for Workforce Readiness	Inge Wefes	http://www.gs.ucdenver.edu/best
University of Massachusetts Medical School (Worcester)	An Integrated Curriculum and Community-Based Approach to Career Development	Cynthia N. Fuhrmann, Phillip D. Zamore	http://www.BEST.umassmed.edu
Vanderbilt University Medical Center	Vanderbilt ASPIRE Program	G. Roger Chalkley, Kathleen L. Gould, Kimberly A. Petrie	http://www.medschool.vanderbilt.edu/aspire/
Virginia Polytechnic Institute and State University	Mentorship and Development Program for Biomedical Trainees	Audra Van Wart, Michael J. Friedlander	http://www.info.vtc.vt.edu/best/
Wayne State University	Wayne State University-BEST	Ambika Mathur	http://www.wayne.edu/gradschool/best/
Awarded in 2014
Boston University Medical Campus	BU’s BEST	Linda E. Hyman, William W. Cruikshank, Barbara Schreiber	http://www.bu.edu/best/
Michigan State University	MSU BEST: Integrated Biomedical Training for Multiple Career Options	Stephanie Watts	http://www.best.msu.edu
Rutgers, the State University of New Jersey	Interdisciplinary Job Opportunities for Biomedical Sciences-iJOBS	Martin Yarmush, James Millonig	http://www.ijobs.rutgers.edu
University of California, Irvine	UCI-GPS: UC Irvine Graduate Professional Success	David A. Fruman	http://www.gps.bio.uci.edu
University of Chicago	myCHOICE	Erin J. Adams, Victoria E. Prince, Julian Solway, Alan Thomas	http://www.mychoice.uchicago.edu
University of North Carolina, Chapel Hill	UNC ImPACT Grant (Immersion Program to Advance Career Training)	Patrick J. Brennwald, Patrick D. Brandt, Jeannette G. Cook, Erin Hopper	http://www.tibbs.unc.edu
University of Rochester Medical Center	URBEST: The University of Rochester BEST Program	Stephen Dewhurst	http://www.urbest.urmc.edu

The BEST institutions, principal investigators, and program website addresses are listed. All of these awards are nonrenewable and are for 5 yr total.

DEFINING ASPECTS OF THE BEST-FUNDED PROGRAMS

The NIH plan for implementation of the BEST program included formation of a consortium made up of the 17 BEST awardees, anticipating that there would be much in common across the individual programs as well as significant differences. The BEST consortium and associated partners and stakeholders are to learn from the results of this series of experiments, and what has been learned will be disseminated to the scientific community. Representatives from all BEST award institutions attended a kickoff meeting in October 2013 (first 10) and 2014 (first 10 together with the second 7), and all will continue to attend annual meetings throughout the award period. These initial meetings offered an opportunity for the programs to share their ideas and concepts, identify common themes, roadblocks, and strategies, and develop working groups to continue sharing information, challenges, and successes on an ongoing basis. A report of the second annual BEST consortium meeting held in October 2014 summarizes some of these activities (13). Monthly awardee webinars and the formation of theme-specific working groups further support communication and dissemination.

An administrative core for the consortium has been established at Vanderbilt University School of Medicine. The responsibilities of the center are to arrange and schedule the annual conference, organize the monthly awardee webinars and consortium committees, and build and maintain an overarching website for all the BEST activities. The website will serve 2 constituencies: the BEST programs themselves, and the general public. Together, the website and active meetings will facilitate interactions between the programs. Recently, the BEST consortium website was launched (visit http://www.nihbest.org to learn more about consortium and individual program activities).

Here, we provide a first report on the early experiences of the consortium, identifying common themes among BEST programs of the member institutions and noting their individual variations. This information should prove useful for those institutions outside the BEST consortium who are contemplating or in the process of implementing broad-based career development activities for their biomedical trainees.

VARIATION IN APPROACHES TO DESIGNING THE BEST PROGRAMS

Variation in scope and trainee requirements

Although the BEST programs share many common programmatic elements, as described in Table 2, the individual programs differ in the scope of trainees targeted, as well as activities or services provided. Some programs focus upon a subset of their pre- and/or postdoc population (e.g., a pilot of 30–50 trainees), with a subset of activities available to the general population of trainees, whereas others target the entire trainee pool either at an academic medical school and/or university graduate school. An important area of variation among programs is in the requirement for credit-based courses vs. simply providing voluntary opportunities driven by trainees’ specific interests. Several programs will award certificates to enrolled trainees who complete certain milestone requirements over the time of training.

TABLE 2.

Variation in approaches to designing the BEST programs

I. Scope: approaches to defining the scope of the program
Emphasis for learning objectives			Career areas
Career exposure			Narrow: defined career tracksa
Career exploration skills			Broad: all careers
Career decision-making skills
Professional skills
Career-specific skills
II. Trainees: approaches to defining the target trainee population
Recruitment models		Participation			Other requirements
Open to all trainees		All elements are required			Prequal vs. postqual Ph.D. students
Cohort (trainees apply)		Some elements required; others optional			Postdoctoral fellows >1 yr at institution
		All elements are optional			PI approval is required
III. Programmatic elements: approaches to curricular offerings
Career developmentb	Professional developmentb			Experiential learning		Mentorship
Self-efficacy	Writing and presentation			Site visits		Peer and small group
Career identity	Networking			Job shadow		Career coaching
Career exploration	Teamwork and leadership			Job simulation; externships		Mentoring by faculty
Career decision making	Wellness			Internships (part or full time)		External career mentor
IDP	Workplace readiness, job search skills
IV. Faculty: approaches to engaging faculty in the program
Engagement
Assess faculty needs, provide information and/or training, codevelop solutions, serve as instructors or panelists, participation by trainees’ research mentors
V. Partners: approaches to engaging internal and external partners
Types of partners			Engagement of Ph.D. alums and other professionals
Ph.D. alums and other professionals			Networking events
Employers/companies			Informational interviews and job shadows
Peer institution (create a dual institutional program)			Contribute to developing curriculum
Other schools, colleges, departments, or programs			Contribute to resources
Regional industry/employer advocacy organizationc
Career/job-oriented company or consultant

In designing their BEST programs, each institution took different approaches, resulting in 17 unique experiments. Here, we have organized the different approaches taken by the BEST institutions into 5 key areas: scope, trainees, programmatic elements, faculty, and partners. The combination of approaches taken by institutions within each of these areas constitutes a complete training program. We suggest that this framework could be used to help guide the development of new BEST-like or BEST-inspired training programs at other institutions, taking into account—just as the BEST programs did—institutional culture, environment, and local expertise. IDP, Individual Development Plan; PI, principal investigator.

^aEntrepreneurship/business/innovation, science communication/writing, government and nonprofit research, intellectual property/tech transfer/legal, policy/public affairs, regulatory, biotech/pharma, education/outreach, and academe.

^bDelivered through different methods: panels, seminars, workshops, courses, and resources (e.g., print and web materials).

^cFor example, Massachusetts Biotechnology Council (http://www.massbio.org) and California Life Sciences Association (http://www.califesciences.org).

Common programmatic elements

The commitment that all trainees should have the opportunity for career development support is an important shared goal of all programs. The programs also share initial domains of programmatic emphasis (listed with examples from individual institutions):

• Career development skills: understanding career options, self-reflection, making use of Individual Development Plans, networking, and job search skills.

• Professional development skills: team building, time management, oral and written communication, self-reflection, and cognitive assessment of leadership, conflict, and negotiation skills.

• Experiential learning: brief intensive, hands-on experiences with partners (e.g., internships and externships) outside of the university (e.g., biotechnology, science writing), or within the university (e.g., intellectual property, grants administration). The inference, in many cases, is that these partners may be potential employers of at least some of the trainees.

• Mentorship: includes their primary research advisor as well as peer mentoring and/or connecting to alumni and professionals in their career of interest.

Each of these domains requires staff with specialized skills in programmatic development as well as forming effective alliances with external partners.

An emphasis on faculty engagement

The consortium recognizes faculty engagement and support as critical determinants for the success of the BEST programs. The successful applicants commonly developed programs that support and complement individual faculty mentoring. Anecdotal information shared among the consortium membership indicates that faculty concerns (including trainee time away from the laboratory) can lead to resistance. Many institutions are renewing efforts to communicate the underlying need for and importance of their BEST program. As a result, we have learned from discussions within our consortium that many faculty 1) understand issues concerning employment after training, 2) recognize their own responsibility and express a desire to help their trainees, and 3) recognize the need for institutional support and the need for “training of the trainers” to adequately address these needs.

In the initial funding period, the programs addressed these issues directly, in accordance with the respective culture at their home institutions. They have developed approaches to inform their faculty and enlist their support. This has ranged from informational meetings, to focus groups, to asking for faculty sign-off for a trainee to participate in BEST activities. Some institutions have arranged for BEST activities to be limited to relatively small amounts of time and delivered at times when lab activities would not conflict (i.e., weekends or evenings). Some BEST institutions have noticed that faculty buy-in has improved following adoption of some of these practices (surveys within the consortium).

An important hypothesis for the consortium to test is whether students who receive career development training will concomitantly demonstrate equal or even increased lab research productivity. Such successes have already been reported by the NIH-funded Institutional Research and Academic Career Development Award program in which postdoctoral trainees of diverse backgrounds spend 25% effort in developing teaching skills at undergraduate institutions with a historical mission of training students from groups underrepresented in the biomedical research workforce. The research productivity of these postdocs, as measured by first-author research publications and duration of the postdoctoral training period, has improved in comparison to their non-Institutional Research and Academic Career Development Award counterparts (14). Similar research productivity improvements have been reported for biomedical postdocs active in K-12 outreach programs (15).

EMERGING STRUCTURES WITHIN THE BEST PROGRAMS

Table 2 summarizes the 5 key areas (scope, trainees, programmatic elements, faculty, and partners) that influence individual institutions’ approaches to program design. Defining specific learning objectives and career areas has influenced the scope of individual programs (e.g., broad training vs. more narrowly defined career tracks). Thus, some programs have elected to make their activities available to all Ph.D. and postdoctoral trainees, some focus their program on Ph.D. students alone, and several focus upon a subset of trainees that has elected (or has been selected) to join. Here, we see individual programs literally experimenting with a wide range of ideas. Individual approaches are highlighted on the BEST consortium website (http://www.nihbest.org), which includes links to individual institutional program websites and summaries, to further explore the different approaches and results in detail and supplement new or existing programs without BEST funding.

Additional examples of intraprogram variability are displayed in Table 2 under curricula available at different sites, as well as strategies for engaging faculty (viewed by all participants as immensely important). Finally, all 17 programs have taken substantial steps to engage external partners, although in vastly different ways outlined in Table 2.

EVALUATION AND DISSEMINATION

One aspect that makes BEST distinct from training grants is that the BEST program is viewed by the NIH as a multi-institutional experiment. The dearth of long-term reliable national data on trainee outcomes and how training programs impact these outcomes was one of the motivations to develop BEST. The programs will include evaluation 1) at each individual institution, and 2) across the consortium. Each program has dedicated staff for the purpose of continuous quality analysis and improvement on the programmatic activities and overall program results including trainee outcomes. Qualitative and quantitative outcomes are discussed during the monthly webinar meetings. At the same time, the NIH Common Fund has contracted with an external evaluation firm (Windrose Vision, Fairfax, VA, USA), and each BEST institution is collaborating to conduct a higher-level comparative evaluation across all 17 programs. The outcomes to be tested and measured in this cross-consortium effort were developed by NIH program staff, as well as staff from the evaluation firm, and the awardees. The desired impacts are to 1) enhance trainees’ ability to make informed career decisions, 2) reduce time to their desired career position (reduce time in postdoctoral training positions), and 3) develop institutional infrastructure to continue BEST activities after the award period ends. The institutions will be gathering specific metrics such as trainee’s initial career goals, understanding of career opportunities, participation in specific programmatic activities, productivity (abstracts, papers, presentations, etc.), time to degree (or time in postdoctoral position), and demographic information, among others. The results of these evaluations will be disseminated to the broader scientific community, in order to encourage and facilitate the collection of similar outcome measures nationwide.

Dissemination is expected to be a key long-term goal of both the individual awardees and the consortium. Because of the importance of this goal, the consortium is expected to share information with the training community including barriers to implementation together with solutions that could be adopted by other programs and institutions. We expect to identify barriers internal to the training organization, between the training institution and their external partners, and institutional structures necessary to sustain these training changes. The consortium will also share the results of the individual program evaluations and the larger NIH cross-site evaluation of BEST activities with the rest of the training community as they become available (http://www.nihbest.org).

EARLY RESULTS

Internal surveys indicate that the BEST institutions have implemented a range of key innovative training paradigms. Most institutions now have career development offices or personnel specifically dedicated to biomedical research graduate students and postdocs. In addition, most programs make substantial use of external partners (often alumni) to establish off-campus interactions at a high level, have events such as annual career symposia, have outcome records and track graduates of their programs, and have or are developing internships for individuals who want to obtain specific exposure to different careers. Finally, a commitment has evolved to inform graduate student recruits of career outcomes for alumni of their institution, emphasizing the realities of the current environment.

COMPLEMENTARY INITIATIVES

The BEST initiative is part of a larger context with active conversations and new policies accompanying the consortium efforts. Until recently, very few institutions provided nonacademic career training and resources designed to meet the evolving needs of the biomedical research workforce (8). Over the past few years, such training is rapidly becoming the accepted norm, with increased recognition of the breadth of careers pursued by our trainees and the need for interventions and resources to support their pursuit of career success. BEST programs and others are well positioned to flourish because of this new environment, heightened faculty awareness, institutional commitment, support from the NIH and other funding/policy groups, and external partnerships. In addition to BEST, other funding opportunities have arisen.

Of course, the BEST program is not the only mechanism for training transformation and innovation. The Burroughs Wellcome Fund reinstated its Career Guidance Grant in 2015, which had first launched in 2012 prior to the NIH BEST RFAs (16). There are several other new initiatives that have become available in the last year or so. These include a new R25 (education) program from the NIH National Institute of General Medical Sciences called “Innovative Programs to Enhance Research Training” (17), a career development supplement for T32 awards through the NIH National Institute of General Medical Sciences (18), and the National Science Foundation Research Training Program (19) encouraging bold new potentially transformative models for science, technology, engineering, mathematics graduate training to ensure that trainees develop skills to be competitive for a wide range of careers in science, technology, engineering, and mathematics. Additionally, the NIH has taken steps to more broadly define successful career outcomes for trainees, including an amendment to the parent funding opportunity announcement for T32 training grants. The text now reads “The career outcomes of individuals supported by NRSA training programs include both research-intensive careers in academia and industry and research-related careers in various sectors, e.g., academic institutions, government agencies, for-profit businesses, and private foundations” (11).

CONCLUSIONS

In future years, BEST programs will have to tackle the challenge of converting successful BEST initiatives into sustainable enterprises, likely in the face of shrinking budgets. For the United States to sustain its leadership role in biomedical research, it is crucial to invest in a robust workforce development plan, one that prepares trainees to enter many of the diverse careers available after training. The NIH Common Fund (and other public and private funding initiatives mentioned above) strongly and enthusiastically supports this effort. The transformative paradigm is that career development can be simultaneously pursued during the training period rather than following the previous sequentially delayed training model. As depicted in Fig. 1, the goal of the BEST program is for biomedical trainees to be career ready immediately following their training period, which will likely result in greater satisfaction with their training experience.

The NIH BEST initiatives are designed to establish new models for training, with resources and opportunities for practical experience that will act as a springboard to successful careers across our scientific enterprise. These experiments, set across the 17 single or partnering institutions within the BEST consortium, will test hypotheses and identify both best practices and unintended consequences. Although challenges will arise, we expect that new models should emerge to influence and transform our fundamental approach to graduate and postdoctoral education.

The BEST consortium institutions, individually or as a whole, will disseminate approaches that will work in different contexts/institutional cultures and will catalyze new initiatives in career development. Strong institutional infrastructure combined with programs designed to support the trainees, the faculty, and their community partners will be evaluated over the next 5 yr of the award period. Follow-up evaluation will continue for an additional 10 yr beyond the grant period, thus testing long-term effects and the sustainability of these programs long after the initial NIH funding is completed.

Glossary

BEST

Broadening Experiences in Scientific Training

NIH

U.S. National Institutes of Health

RFA

request for application