Graduate Education for the Future: New Models and Methods for the Clinical and Translational Workforce

Abstract

This paper is the third in a five-part series on the clinical and translation science educational pipeline, and it focuses on strategies for enhancing graduate research education to improve skills for interdisciplinary team science. Although some of the most cutting edge science takes place at the borders between disciplines, it is widely perceived that advancements in clinical and translational science are hindered by the “siloed” efforts of researchers who are comfortable working in their separate domains, and reluctant to stray from their own discipline when conducting research. Without appropriate preparation for career success as members and leaders of interdisciplinary teams, talented scientists may choose to remain siloed or to leave careers in clinical and translational science all together, weakening the pipeline and depleting the future biomedical research workforce. To address this threat, it is critical to begin at what is perhaps the most formative moment for academics: graduate training. This paper focuses on designs for graduate education, and contrasts the methods and outcomes from traditional educational approaches with those skills perceived as essential for the workforce of the future, including the capacity for research collaboration that crosses disciplinary boundaries.

Introduction

Future progress in the biomedical sciences depends on the ability of researchers to engage more effectively in translational and interdisciplinary initiatives, involving teams of scientists from diverse disciplines and backgrounds.^1,2 If the most cutting-edge biomedical research of the future depends on scientists’ ability to engage in translational and interdisciplinary team-based research, then we need to take care that the next generation of clinical and translational investigators can work effectively as members of cross-disciplinary teams. Without appropriate preparation for career success, we risk the loss of talented scientists in clinical and translational research.³ To ensure adequate preparation and strengthen the pipeline, it is important to carefully consider graduate education in the sciences, a logical point of intervention for the biomedical workforce.

Clinical and translational science covers a broad range of disciplines, ranging from basic science, to patient oriented research, to population health sciences. For the purposes of this paper, we apply the following definitions:

• “Translational research fosters the multidirectional and multidisciplinary integration of basic research, patient-oriented research, and population-based research, with the long-term aim of improving the health of the public.”⁴

• “Interdisciplinary research is any study or group of studies undertaken by scholars from two or more distinct scientific disciplines. The research is based upon a conceptual model that links or integrates theoretical frameworks from those disciplines, uses study design and methodology that is not limited to any one field, and requires the use of perspectives and skills of the involved disciplines throughout multiple phases of the research process.”⁵

Throughout the rest of this paper, we will use the term “interdisciplinary” research as the umbrella term incorporating translational research. In addition, we will use interdisciplinary science to encompass team science, since the interdisciplinary work in biomedical research is most often conducted by a team with members from varied disciplines. Finally, in referring to “graduate education” in the biomedical sciences, we will focus on training obtained via a PhD program in the basic, clinical, or population sciences; and training obtained via pursuit of research Master's degree by clinician-scientists after completing a clinical doctorate (e.g., the MD, DO, PsyD, etc.).

This paper is the third in a five-part series on the clinical and translational science educational pipeline.³ The overall goal of this series is to describe how institutions may develop an effective educational pipeline along the entire academic and career development continuum. Here, we focus on the crucial step of graduate research education as we pursue the objective of crafting training programs that will best prepare young scientists for careers in clinical and translational research.

Traditional Graduate Research Education

Graduate education in the sciences focuses on producing experts in a given field. Whether the training is received in a research doctoral program or in a Master's program after the clinical doctorate, the emphasis in most programs is on depth rather than breadth of knowledge. A graduate research project (i.e., a doctoral dissertation or a Master's thesis) is usually conceived as an exercise in which the student scholar learns all he or she can about a precisely-defined, well-posed, and carefully circumscribed problem. By its very structure, then, most graduate training programs encourage students to drill deeply into a very fine slice of a scientific problem.

There is a tension here. We want future scholars to take on hard questions and complex challenges that compromise health; yet during training, student scholars are advised to maintain a narrowly-defined focus (limiting the scope of their work) to ensure that the problem they tackle is “manageable” and their research plan is “feasible.” In this milieu, how can we expect young investigators to embrace interdisciplinary pursuits? For the future of graduate education, we must seek ways that continue to offer students deep expertise in their chosen fields, while also furnishing students with exposure to the promise and practice of interdisciplinary science, and strategies they can employ to ensure that their careers can achieve both ends.

Graduate Research Education for the Future: Establishing the Competencies for Interdisciplinary Research

If our goal is to modify graduate education such that it prepares scholars for roles in interdisciplinary and translational team science, then the implementation of this goal depends upon articulating the knowledge, skills, and competencies that are required for success in this realm.⁶ We present here a brief summary of existing recommendations.

Gebbie and colleagues conducted a Delphi survey of recognized interdisciplinary practitioners and thought leaders in an effort to derive a set of competencies.⁷ The 27 panelists included directors of funded interdisciplinary centers, scholars engaging in interdisciplinary research in the allied health professions, and authors of publications demonstrating experience in interdisciplinary research. The final set of competencies included 17 competencies falling under 3 domains (see Table 1).

TABLE 1.

Required competencies for interdisciplinary research as defined by Gebbie et al. (2008) The scholar who has completed doctoral work with an emphasis on interdisciplinary research is able to:

Domain	Competency
Conduct research	• Use theories and methods of multiple disciplines in developing integrated theoretical and research frameworks • Integrate concepts and methods from multiple disciplines in designing interdisciplinary research protocols • Investigate hypotheses through interdisciplinary research • Draft funding proposals for interdisciplinary research programs in partnership with scholars from other disciplines • Disseminate interdisciplinary research results both within and outside his or her discipline
Communicate	• Advocate interdisciplinary research in developing initiatives within a substantive area of study • Express respect for the perspectives of other disciplines • Read journals outside his or her discipline • Communicate regularly with scholars from multiple disciplines • Share research from his or her discipline in language meaningful to an interdisciplinary team • Modify his or her own work or research agenda as a result of interactions with colleagues from fields other than his or her own • Present interdisciplinary research at venues representing more than one discipline
Interact with others	• Engage colleagues from other disciplines to gain their perspectives on research problems • Interact in training exercises with scholars of other disciplines • Attend scholarly presentation by members of other disciplines • Collaborate respectfully and equitably with scholars from other disciplines to develop interdisciplinary research frameworks • Author publications with scholars from other disciplines

Another set of competencies comes from the Clinical and Translational Science Award (CTSA) consortium, funded by the National Institutes of Health (NIH). In 2009, the Education and Career Development Key Function Committee of the CTSA, comprised of education and career development program directors of approximately 60 funded CTSA hubs, developed a set of core competencies for clinical and translational research at the Master's level (https://ctsacentral.org/consortium/best-practices/335-2/). They proposed 14 general thematic areas, and within each area, 4-10 specific competencies. Four of the thematic areas specifically address interdisciplinary team science, communication, and leadership ability (see Table 2).

TABLE 2.

Core Competencies in Thematic Areas IX, XI, XI, and XIII developed by the CTSA Educational and Career Development Key Function Committee (2009)

Core Thematic Area	Competencies
Scientific Communication (IX)	1. Communicate clinical and translational research findings to different groups of individuals, including colleagues, students, the lay public, and the media. 2. Translate the implications of clinical and translational research findings for clinical practice, advocacy, and governmental groups. 3. Write summaries of scientific information for use in the development of clinical health care policy. 4. Translate clinical and translational research findings into national health strategies or guidelines for use by the general public. 5. Explain the utility and mechanism of commercialization for clinical and translational research findings, the patent process, and technology transfer.
Translational Teamwork (XI)	1. Build an interdisciplinary/ intradisciplinary/ multidisciplinary team that matches the objectives of the research problem. 2. Manage an interdisciplinary team of scientists. 3. Advocate for multiple points of view. 4. Clarify language differences across disciplines. 5. Demonstrate group decision-making techniques. 6. Manage conflict. 7. Manage a clinical and/or translational research study.
Leadership (XII)	1. Work as a leader of a multidisciplinary research team. 2. Manage a multidisciplinary team across its fiscal, personnel, regulatory compliance and problem solving requirements. 3. Maintain skills as mentor and mentee. 4. Validate others as a mentor. 5. Foster innovation and creativity.
Cross Disciplinary Training (XIII)	1. Apply principles of adult learning and competency-based instruction to educational activities. 2. Provide clinical and translational science instruction to beginning scientists. 3. Incorporate adult learning principles and mentoring strategies into interactions with beginning scientists and scholars in order to engage them in clinical and translational research. 4. Develop strategies for overcoming the unique curricular challenges associated with merging scholars from diverse backgrounds.

Rubio and colleagues reviewed the literature, linking the definition of translational research to the evaluation of translational research training programs, and, therefore, to the competencies that would be expected of graduates of such programs.⁴ Their analysis yielded 5 competencies (see Table 3).

TABLE 3.

Five competencies for translational research (Rubio et al., 2010)

1. Critically examine the research process.

2. Think “out of the box” to develop ways to impact health care by transferring knowledge from and to the bench, bedside, and community.

3. Engage in multidisciplinary collaboration.

4. Understand successful approaches to community engagement.

5. Develop appropriate techniques to manage multidisciplinary research teams in the future.

Through a novel examination of the Integrative Graduate Education and Research Training (IGERT) programs funded by the National Science Foundation (NSF), Borrego and Newswander derived competencies for interdisciplinary science.⁸ In analyzing the text of proposals that succeeded in garnering IGERT funding from 1999 through 2006, four distinct areas of competency emerged (see Table 4). The authors noted the importance placed by the training programs on strong disciplinary skills, coupled with the ability to engage with researchers across disciplines. In a subsequent report on the IGERT program, Abt Associates defined 6 core competencies for conducting interdisciplinary research; these are presented in Table 5.⁹

TABLE 4.

Four areas of competency in IGERT program proposals (Borrego and Newswander, 2010)

1. Grounding in multiple disciplines

2. Integration and broad perspective

3. Teamwork

4. Interdisciplinary communication

TABLE 5.

Six core competencies essential for conducting interdisciplinary research (Gamse et al., 2013)

1. Ability to develop depth of knowledge in one discipline or field of study

2. Ability to recognize the strengths and weaknesses of multiple disciplines

3. Ability to apply the approaches and tools from multiple disciplines to address a research problem

4. Ability to work in a team with individuals trained in different disciplines

5. Ability to communicate research based on one discipline or field of study to academic researchers trained in different disciplines

6. Ability to communicate about interdisciplinary research to non-academic audiences (laypersons)

Acquiring the Competencies for Interdisciplinary Research

If interdisciplinary skills are important for the next generation of clinical and translational investigators, how can we best instill them in our graduate and post-doctoral trainees? Let us examine what approaches have been implemented, and to what degree their effectiveness has been evaluated.

A recent study of the IGERT program surveyed IGERT trainees and training program directors on a host of issues relating to capacity for interdisciplinary science, including the mechanisms by which the programs develop competencies.⁹ The study's authors noted a striking consistency between the responses of students and program leaders, most of whom cited the following activities as important in promoting interdisciplinary skills: courses and seminars in multiple disciplines, interdisciplinary courses, laboratory and field experiences, and interdisciplinary team projects.

To assess current approaches to teaching interdisciplinary research skills, the Education and Career Development Key Function Committee of the CTSA consortium designed and administered a web-based survey of education leaders from all (about 60) CTSA-funded institutions nationwide.¹⁰ The survey results showed that the vast majority of CTSA education leaders felt it was important to provide training opportunities in interdisciplinary team science to young investigators, yet only a little over half of these same institutions reported that they offered such training. Respondents also expressed a desire for greater guidance in evaluating the effectiveness of their efforts.

Starting from the premise that interdisciplinary team science skills can be taught, Larson et al.¹¹ provide a sketch of a didactic course, entitled “Building Interdisciplinary Research Models” organized around the competencies proposed by Gebbie.⁷ The course provides an overview of the theory and methods for interdisciplinary research, interspersed with speakers who present on the interdisciplinary work they have engaged in, both successfully and less so. Students in the course are required to complete two written assignments designed to foster collaborative skills. Feedback from anonymous course evaluations has been reasonably positive, but students felt that they needed still greater opportunity to apply the content of the course, with more exposure to and familiarity with the logistics of launching and maintaining an interdisciplinary research project.

Another example of interdisciplinary training at the graduate level is the University of Colorado TL1 (T32) Clinical Translational Certificate Program. Funded through the University's CTSA, each year a cohort of eight pre-doctoral fellows is selected. Applicants must develop a translational training plan and identify a clinical mentor with whom they will meet on a monthly basis throughout their graduate years. Through this program, basic science students are given the unique opportunity to observe firsthand, through shadowing in the mentor's clinic, for example, how their research impacts patient health care. Other requirements include enrollment in translational coursework such as biostatistics and clinical trial design, with the overarching goal of giving the student an awareness and added value of clinical translational implications of their laboratory bench work in the laboratory.

An excellent resource for students and scholars seeking greater practical support for implementing an interdisciplinary research program is “Collaboration and Team Science: A Field Guide”.¹² This field guide presents advice and a set of best practices for anyone who contemplates joining, leading, or building a research team with participants from multiple disciplines. Its concrete, step-by-step approach (from starting to thinking about an interdisciplinary project, through resolving conflicts and improving teamwork) is a welcome addition to the training armamentarium, and can serve as an essential tool for young investigators who need substantive, application-oriented guidance on how to approach collaborative research.

As this brief summary shows, there are multiple sets of competencies suggested for graduate student training. It is important for educators in clinical and translational science to review these proposed competencies and come to consensus around a core set of competencies that will guide the design and content of training programs for the future.

Assessing Training Programs for Interdisciplinary Research

Once we agree upon a core set of competencies for interdisciplinary team science, it is crucial that appropriate curricular content is designed to achieve them, and that these educational efforts be rigorously evaluated. Clinical and translational investigators typically receive strong training in research methods; yet those of us involved in training are often slow to apply our research skills to evaluate our educational programs, leaving a disconnect between our investigative orientation and our classroom-based approaches.

Fortunately, some researchers have begun to apply “research-oriented” thinking to educational programming. Lee and colleagues¹³ propose metrics and approaches for assessing key factors thought to impact research career success, both extrinsic (e.g., grants and published papers) and intrinsic (e.g., job satisfaction and life satisfaction), based on the career-success model developed by Rubio¹⁴. Specifically, they suggest the use of two established and validated instruments, the Cross-Disciplinary Collaborative Activities Scale¹⁵ and the Research Collaboration Scale¹⁶ as measures of participation in, satisfaction with, and perceived impact of interdisciplinary collaboration.

Perhaps the most comprehensive and ambitious efforts to evaluate education for interdisciplinary team science have been aimed at NSF's IGERT program. Hackett and Rhoten devised a creative study to directly compare the research abilities of IGERT students and non-IGERT students.¹⁷ Teams of IGERT and non-IGERT students competed against one another to respond to an “RFA-like” research challenge. Scientific experts rated the students’ research proposals at the end of the exercise. With small samples (8 teams, 6 members per team), it is difficult to draw definitive conclusions. However, the authors were surprised to see that the IGERT students who were more senior (those with 3 or more years in training) did fairly poorly compared both to more junior IGERT students (those with 1-2 years of training) as well as to the non-IGERT students, both junior and senior. In particular, the senior IGERT student teams did the worst with respect to their “interdisciplinary quality” ratings from the expert panel of evaluators. As the authors noted, there could be many explanations for this unexpected finding, including the small sample, the “artificial” nature of the research challenge, and the timing of the evaluation – but these results were troubling.

Subsequent to the Hackett and Rhoten analysis, Abt Associates conducted further, more detailed analyses of IGERT trainees, their learning outcomes, and early career trajectories.^18,9 In a comparative study of career outcomes in IGERT graduates versus non-IGERT graduates, dissertations of IGERT graduates were more interdisciplinary, drawing on an average of 3 distinct disciplines (versus only 2 for non-IGERT graduates). Furthermore, IGERT graduates were somewhat more likely to be “integrating multiple disciplines” as part of their current work (84% versus 73%), and also more likely to be teaching courses that required them to integrate two or more disciplines (63% versus 50%). Finally, IGERT graduates also had a higher probability of working and networking with scientists or technologists in other disciplines (92% versus 84%). These numbers attest not only to the potential effectiveness of IGERT training, but also to the near-ubiquity of interdisciplinarity in the sciences.

Mentorship for Building Interdisciplinary Competencies

Mentorship is a critical component of efforts to train and sustain nascent interdisciplinary researchers, and mentor training programs have gained traction.¹⁹ In a study that perhaps provides the most persuasive evidence that training is beneficial, Pfund et al. reported a randomized trial of a case-based, 8-hour mentor training program among 283 mentors of clinical and translational research scholars.²⁰ Mentors were invited to assess their own skills before and after training, and their mentees were interviewed separately, blinded as to their mentors’ intervention status. The primary outcome was change in the mentors’ self-reported composite scores on the Mentor Competency Assessment (MCA).²¹ Results showed a significantly greater change in MCA composite scores from baseline in the intervention group versus the control. In addition, the mentees of mentors in the intervention group, compared with the control, reported a greater number of positive behavioral changes by their mentors.

In periods of diminishing grant funding, faculty may be reluctant to take on mentoring or venture outside their silos. Institution-sponsored mentorship training programs can be a vehicle to strategically enhance interdisciplinary research, expand mentorship impact, and bring together faculty from a variety of fields to help build an interdisciplinary community.

Future Directions

The Institute of Medicine's assessment of the CTSA programs begins by reiterating the broad goal of the consortium, “to facilitate and accelerate the translation of laboratory discoveries into new and better preventive and treatment solutions to improve human health,” and goes on to emphasize that meeting this challenge “requires approaches to training and education that are outside of traditional scientific fields”.² The IOM report also posits that the “transformation of training and education is possible only through the dissemination of successful approaches and practices.” It is the aim of this paper to discuss what these recommendations imply for graduate education in the clinical and translational sciences, with an emphasis on curricular content, evaluation, and the ability to measure learning outcomes and career impact.

With respect to training for interdisciplinary research careers, large-scale evaluation using accepted metrics could truly be transformative to the field of interdisciplinary science graduate education. A number of important steps need to occur before this transformation can be realized:

1. Define interdisciplinary research competencies. Educators must carefully review the existing competencies, identify where they overlap and where they diverge, and promote a finalized set of competencies to inform a wide range of current and future training programs. To achieve credibility and buy-in, this activity would likely need to be undertaken by a national body with the reputation and resources to attract the attention and respect of a large and diverse set of biomedical translational researchers and stakeholders from academia, industry, community-based organizations, and government.²² To facilitate interactions with stakeholders outside of academia, it may also be beneficial to include enhanced training in regulatory science, inferences from “big” data, biomedical informatics, and health economics, in addition to interdisciplinary team science skills.

2. Extend curricular approaches. It is necessary to extend traditional educational activities and create novel approaches for teaching interdisciplinary skill development. Experts in adult learning should work side-by-side with clinical and translational educators and scientists to share and discuss teaching and mentoring strategies used to foster interdisciplinary research skills.

3. Build more opportunities to “train the trainers.” With increased dialogue and sharing of approaches, it will be important to explore enhanced training opportunities for the educators themselves, to ensure that they will have the knowledge and skills to employ new methods of instruction, if it is believed that these new methods will be better suited to delivering the desired competencies.

4. Develop and pilot new metrics for interdisciplinary skills and career outcomes. To evaluate program effectiveness, we need to enumerate the goals of training that are aligned with the accepted interdisciplinary team science competencies. There is a need to develop and validate instruments and metrics that can be used across programs. Many educational programs remain solely focused on more “traditional” measures of career success, like success in obtaining peer-reviewed funding, promotions, and number of published manuscripts. While these are important indicators of career and scientific impact, they sculpt a narrow vision of career success. More recently, attention has been given to less traditional career outcomes, such as comfort with interdisciplinary team approaches for designing and conducting research¹³, a broader and more flexible view of academic promotion criteria^1,23, and a greater range of career pathways for translational scientists^22,24. Part of this effort might include the exploration of a career pathway for scientists whose primary role is not only to function in their original discipline, but to formally facilitate successful interdisciplinary teams.

5. Conduct carefully-designed, rigorous studies of educational programs. Once we have established a set of interdisciplinary research competencies, developed innovative programs to foster these skills, and generated and piloted metrics for assessing program success, we will finally be ready to apply our research skills to evaluate and compare a variety of educational methods and strategies. We can build on the ground-breaking evaluative work conducted in interdisciplinary science education to coordinate large-scale studies to determine the most effective methods to enable our young scholars to become confident, successful interdisciplinary scientists.^9,17,18

6. Promote interdisciplinary mentorship. There is ample evidence demonstrating the benefits of strong mentorship to the careers of young scholars.^25,26 It is important to build on this evidence base and promote interdisciplinary mentorship (mentoring provided by teams of mentors) as the norm, and not the exception. This seems to be the trend, with an increasing number of clinical and translational research programs encouraging (or even requiring) mentorship by a team, with representatives from disparate disciplines.

In contemplating these recommendations, we face a number of challenges to their implementation. While many are convinced of the benefits of interdisciplinary team approaches, their acceptance is not universal. Furthermore, changing the structure and content of long-established graduate programs will undoubtedly be met with some resistance. It is always difficult to effect change; especially in academia, where tradition and respect for the past are lauded (and rightly so). But we cannot allow this resistance to stop inevitable educational advancement. So how can we proceed?

Oddly enough, pushing past some initial educational inertia may in and of itself advance the change process. In their experience, the IGERT directors claimed that the program increased the “prominence of interdisciplinary research” at their institutions, and served as a “springboard for longer term institutional change”.⁹ It may be that setting up a single new program, therefore, can become catalytic, with repercussions extending far beyond the initial effort. To be sure, the IGERT program directors did not claim that the path was easy. Instructional faculty in existing programs will no doubt be wary of new requirements that may be seen as foreign and burdensome (and quite different from the traditional training they likely received themselves); and they will also question the impact on time-to-completion for graduate students in their programs. We have to be cognizant of over-burdening students who seek to complete their degree in an acceptable time frame in an effort to minimize debt and to join the workforce as soon as possible. There may also be skepticism around training aims that may be viewed as “soft” outcomes, such as interdisciplinary readiness. However, the best way to respond to concerns about “soft” outcomes may be to rely on good research with “hard” data. For example, the IGERT analyses from 2011 indicated that the median time to completion for students in IGERT PhD programs was approximately 6 months shorter than time-to-completion for students in the non-IGERT programs. In addition, IGERT students reported a slightly lower level of difficulty in securing jobs after graduation. When presented with hard data on actual student outcomes, faculty may well become more open to educational innovation and the realities of working in interdisciplinary team science; and students may gravitate to the programs with more innovative curricula and higher job placement rates and program satisfaction, providing further impetus for change among the faculty.

Finally, we conclude by noting that the effectiveness of training interventions may be environment-dependent. Any effort to provide training to enhance interdisciplinary collaborative skills will occur within a system or organization in which interdisciplinary work may or may not be valued. It is difficult to envision successful interdisciplinary training in any environment where it is not modeled, not rewarded, or not aligned with the culture of the institution. We have not touched in this paper on the important topic of academic reward systems. Though this issue is critically important, it diverts us from our primary focus (graduate education); instead, we direct the reader to the careful analyses of other authors.^1,23,27 It may be that both efforts (to enhance training and to create a more welcoming environment for interdisciplinary work) must be undertaken in tandem if we are to more fully realize the benefits and impact of interdisciplinary team science.