A National Survey of Mentoring Programs for KL2 Scholars

Abstract

There is limited information on how academic institutions support effective mentoring practices for new investigators. A national semistructured telephone interview was conducted to assess current “state of the art” mentoring practices for KL2 scholars among the 46 institutions participating in the Clinical Translational Science Awards (CTSA) Consortium. Mentoring practices examined included: mentor selection, articulating and aligning expectations, assessing the mentoring relationship, and mentor training. Telephone interviews were conducted in winter/fall 2009, with 100% of the CTSAs funded (n= 46) through 2009, participating in the survey. Primary findings include: five programs selected mentors for K scholars, 14 programs used mentor contracts to define expectations, 16 programs reported formal mentor evaluation, 10 offered financial incentives to mentors, and 13 offered formal mentoring training. The interviews found considerable variation in mentoring practices for training new investigators among the 46 CTSAs. There was also limited consensus on “what works” and what are the core elements of “effective mentoring practices. Empirical research is needed to help research leaders decide on where and how to place resources related to mentoring. Clin Trans Sci 2010; Volume 3: 299–304

Introduction

Improving the health of the US population is a major goal of a number of public agencies such as the National Institutes of Health (NIH). One of the challenges in meeting this national goal is to cultivate a workforce capable of developing and testing new technologies, medications, procedures, and community‐based interventions. One strategy that has been implemented by the NIH to nurture new researchers is the Clinical and Translational Science Awards (CTSA). This NIH initiative began in 2006 with initial awards to 12 universities with the ultimate goal of funding 60 CTSAs.

One of the key functions of the CTSA program is to provide research education, training, and career development for the next generation of scientists who work in the area of clinical and translational science. One centerpiece of the CTSA is the KL2 program, a mentored career development initiative “intended to produce new clinical research leaders who can cross the boundaries of their disciplines and draw upon the strengths of other fields.” ¹ The National Center For Research Resources (NCRR) KL2 funding mechanism is similar to other K‐series career development awards and provides 50%–75% protected time and research support for 2–5 years. In the spring of 2010, there were over 375 KL2 scholars in training among the 46 CTSAs. The KL2 program is similar to the K12 series of institutional K awards.

In addition to providing training, research infrastructure, and salary support for junior faculty and new investigators, the CTSA program, consistent with NIH’s K12 Roadmap and Clinical Research Curriculum Award (K30) programs; has identified mentoring as a critical element in assisting new researchers, as they become independent investigators. The Requests for Applications for the CTSA award explicitly states that CTSA programs need to demonstrate how mentors of scholars and trainees will be trained and evaluated (RFA‐RM‐10‐020, p. 20). The NCRR executive leadership acknowledged the importance of mentoring for its scholars and trainees when in November of 2008, the CTSA Education Key Function Committee convened a “mentor working group” to identify the types of initiatives developed by CTSA institutions to support the mentoring of KL2 scholars in their Mentored Career Development Training Programs.

The mentor working group membership consisted of Education and Career Development Directors from 18 CTSA institutions. This group was led by Dr. Michael Fleming and staff at the Institute for Clinical and Translational Research (ICTR) at the University of Wisconsin‐Madison. The working group was charged to complete the following activities: (1) conduct a literature review on what is known about effective mentoring practices; (2) collect new data as appropriate using interviews or surveys of KL2 program leaders, mentors and scholars; (3) post this information on the CTSA WIKI; and (4) develop a consensus statement of “effective mentoring practices.” This is the first in a series of papers focused on the various elements of mentoring written by members of the CTSA mentor working group. The ultimate goal of this effort is to develop an evidence‐based consensus statement on “effective” mentor practices.

Research based on mentoring has been steadily growing since Kathy Kram’s seminal work in the 1980s ² that laid the framework for thinking about mentoring as a relationship with four different phases, initiation, cultivation, separation, and redefinition, operating within two primary domains, psychosocial and career development. Since then, the fields of education, psychology, business, and, increasingly, academic medicine, have focused on identifying the “desired characteristics and actions” ³ of the mentor and mentee across personal, relational, and professional domains (e.g., active listener, accessible, and experienced), as well as how effective mentors interact with mentees across cultural, personality, and gender differences. ⁴

A subset of the mentoring literature specific to academic medicine ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ examines how administrators, program directors, and others can build program initiatives that maximize a shared understanding of these characteristics with the hope of increasing the number of junior investigators whose careers are being fostered by effective mentoring practices. The literature indicates that mentor selection, negotiating clear expectations, mentor training, and rewards for mentoring are all areas where department chairs and medical school deans can help mentors and mentees engage in processes that build effective mentoring practices and encourage communication and reflection. ⁴ , ⁵ , ⁶

Methods

A semi‐structured telephone interview was conducted to collect baseline information on KL2 Programs and their efforts to support mentoring across the 46 CTSA institutions. While other data collection methods, such as mailed questionnaires and web surveys were considered, an interview provided both quantitative and qualitative data as well as increasing the response rate. The University of Wisconsin (UW) at Madison Health Sciences Institutional Review Board determined the interviews met federal criteria for exempt status. The interviews with the director of the KL2 program at each of the 46 CTSAs were conducted by UW ICTR research staff.

Acknowledging that mentoring is “explicitly situated in a social context and shaped by the institutional culture and climate,” ³ the interview schedule was designed to both capture institutional and program characteristics and to understand current mentoring practices for KL2 scholars. Initially, background and descriptive information was collected that included existence of a K30 or K12 Roadmap program prior to receiving the CTSA, the number of KL2 scholars in the program, length of the training period, a description of the KL2 program supports and training components, and KL2 program expectations for the scholars.

The interview then proceeded to more systematically collect detailed information about the programmatic structures in place that explicitly support communication of expectations for the mentor and scholar (e.g. contracts, formalized meetings, etc.), formal evaluation of the mentoring relationship, and efforts to train and support mentors. We were interested in discovering what types of programmatic interventions and formalized mechanisms existed to foster and monitor the effectiveness of the mentoring relationship. The schedule consisted of 26 items (see Figure 1 ) that addressed these three key areas of interest. Specifically, items 1–10 explored mechanisms for articulating the expectations, both programmatic expectations as well as those the mentors and mentees hold of each other. Items 11–18 inquired into how programs are finding ways to monitor the health and effectiveness of the mentoring relationship and items 19–26 asked about mentor development training programs available to scholars and mentors alike.

Figure 1.

Interview protocol used to collect the information contained in this report.

The 30‐minute telephone interviews were conducted with 32 of the then 46 CTSAs from January to March of 2009. The remaining 14 were completed in September and October of 2009 (see list of all participating institutions in the Acknowledgements section). The response rate was 100%. The phone interviews were not audio recorded, but notes were taken by the interviewer and one additional staff person. The interview data were summarized into a one‐page Institutional Mentoring Profile (see Figure 2 for example) for each CTSA. The profile was then reviewed and revised by each interviewee before distribution in a formal report circulated to members of the mentor working group, CTSA Education Key Function Committee, and attendees of the 2009 ACRT/SCTS Joint Annual Meeting. In addition, the profiles were posted on the CTSA WIKI.

Figure 2.

Template of mentoring profile contained on the CTSA WIKI for each of the institutions who participated in this study.

Analysis

Descriptive data representing the number of scholars, previous NIH K30, or K12 awards held, use of a mentoring contract, formal evaluation of the mentoring relationship, availability of a mentor training program, and fiscal incentives for mentors were compiled into a database and organized by year funded (see Table 1 ). In addition, the data were summarized into information about the mentoring infrastructure that include the process of mentor selection, negotiating expectations between mentors and mentees, mentor evaluation, and mentor training and support. This information is summarized below.

Table 1.

Summary of a national survey of CTSAs (n= 46).

	2006 grantees n= 12	2007 grantees n= 11	2008 grantees n= 14	2009 grantees n= 8
Roadmap K12 prior to award	3 (25%)	4 (36%)	1 (7%)	0
NCRR K12 prior to award	4 (36%)	6 (54%)	3 (22%)	0
NCRR K30 prior to award	10 (83%)	9 (82%)	11 (80%)	6 (75%)
Average number of KL2 scholars	14 scholars	10 scholars	5 scholars	5 scholars
Average duration of KL2 award	3 years	3 years	3 years	3 years
MS required	9 (75%)	7 (63%)	8 (57%)	2 (25%)
MS encouraged	1 (8%)	1 (8%)	0	1 (12%)
Mentor contract used	5 (42%)	5 (45%)	4 (30%)	0
Mentor training programs	4 (36%)	4 (36%)	3 (22%)	2 (25%)
Formal evaluation of mentor	6 (50%)	6 (54%)	3 (22%)	1 (12%)
Fiscal incentives to mentors	3 (25%)	4 (36%)	1 (7%)	2 (25%)

Results

Mentor selection

The mentoring relationship begins with the selection of the mentor. While the vast majority of KL2 programs (n= 37) expect scholars to select a mentor when they apply, many programs expressed a willingness to offer assistance when approached prior to submission. Five programs actually make the match themselves, while four programs have developed a collaborative process. Some programs have pools of experienced mentors in place to guide scholars (e.g., Mayo Clinic). At the University of Florida, the applicant proposes a mentor and the program recommends additional mentors, including one who must be from an internal Mentor Advisory Committee. Selection of mentors is critical to the KL2 application review process. In fact at the University of Alabama‐Birmingham, they also interview the mentor during the selection process. If a poor match is made initially, valuable protected time is lost while trying to add an additional mentor to the team.

Communicating expectations

Programmatic expectations of the mentoring relationship include frequency of interaction, schedule of mentor team meetings, and a shared expectation that the research goals of the dyad are in line with programmatic expectations. Program leaders reported that these expectations are typically communicated through the KL2 application materials, in formal orientation sessions and frequently, in less formal meetings between the KL2 Program Director and the mentor or in conversation between the mentor and scholar.

Beyond the expectations of intended KL2 program outcomes for the scholar, each scholar also has individualized needs of their mentors that are important to communicate. To help facilitate these conversations between scholar and mentor, 28% of the programs utilize some form of written agreement or contract signed by the mentor and mentee. Designed to guide postdoctoral training, the Association of American Medical Colleges (AAMC) compact ¹⁴ has served as a common starting point for KL2 programs interested in using a formal written agreement between scholar and mentor. This document lays out high‐level career expectations such as “respecting all ethical standards when conducting research” and leaving behind “all original notebooks, computerized files, and tangible research materials” at the end of their appointment. From the mentor’s perspective, the compact articulates commitments such as providing “sufficient opportunities to acquire the skills necessary to become an expert in an agreed upon area of investigation” or encouraging “the interaction of the postdoctoral appointee with fellow scientists both intra‐and extramurally.” ⁷

However, the AAMC compact does not attend to the more interpersonal relationship expectations that are also important to address for effective communication to occur. The University of Alabama‐Birmingham has implemented as part of the KL2 application process, an explicit forum for these issues. KL2 candidates and their mentors are required to complete a series of questions together, covering not only the expectations the mentee and mentor have of each other, but also relationship concerns such as, “What will be the ground rules for discussions?” (e.g., confidentiality, openness, candor, truthfulness) and “If problems arise, how will they be resolved?” This exercise creates a forum for the mentor and the scholar to negotiate working and communication styles early on while also aiding the KL2 selection committee in its assessment of whether a candidate has an appropriate mentor match.

Given the multidisciplinary nature of the clinical and translational research being conducted by KL2 scholars, multidisciplinary mentors are required or encouraged by two thirds of the programs. Therefore, expectations need to be negotiated not just between the mentor and mentee but also among members of the mentor team. Some programs address this need by requiring quarterly meetings of the mentor team. Others ensure this alignment is in place even before the scholars are selected. For example, the University of California‐Davis asks that the primary and secondary mentors each write a letter indicating their terms of support for the scholar (e.g., time, research resources). The mentors then meet and merge their intentions into one letter of support called a “codified promissory note” that demonstrates how they will operate as a team to meet the needs of the scholar. This letter is then incorporated into the KL2 application and considered integral to determining selection of awardees.

Assessing the mentoring relationship

The traditional means of assessing the success of the mentoring relationship is the ultimate career success of the mentee. Typical metrics of career progress include successful grant submissions and research publications. By achieving these benchmarks, the scholar and by extension the mentor, are deemed successful. The majority of the KL2 programs evaluate the scholar’s progress on an annual basis with some sites collecting this information every 3–6 months. The limitation of this approach, as identified by program directors is two‐fold: first, it is unclear whether the scholar is achieving these results because of good mentoring or in spite of poor mentoring; and second, an annual assessment does not allow for timely intervention if there are concerns about progress.

Seventeen of the 46 KL2 programs reported conducting formal evaluations of the mentoring relationship, while an additional five programs are developing an instrument. Annual or semi‐annual surveys are the most common means of evaluating the relationship, and typically it is the mentee who evaluates the mentor’s effectiveness in providing guidance on intellectual growth and development, professional career development, academic guidance, personal communication, and mentor as role model. While the most commonly used published tool is the Berk Mentoring Effectiveness Scale, ¹⁵ some institutions such as the Mayo Clinic, the University of Illinois at Chicago, University of California at Davis, and Columbia University have developed their own evaluation measures. ¹⁶ These instruments can be accessed by CTSAs on the CTSA mentor WIKI.

Program directors indicated an interest in creating a standardized process for assessing scholar progress and the mentoring relationship on a regular basis so as to more readily identify behavioral antecedents to problematic relationships. However, scholars are understandably often reluctant to disclose information that may reflect poorly on their mentors because of the important role they play in their future success. A second challenge for program directors is how to proceed if unsatisfactory mentor performance is disclosed. Case Western Reserve has formed a Mentoring Committee whose role is to intervene in such cases but most KL2 program directors expressed that the responsibility of communicating performance issues with the mentor resides with them.

Mentor support and training

One of the challenges for programs is identifying appropriate rewards and incentives to recruit a limited pool of clinical research faculty to mentor KL2 scholars. While faculty members’ productivity benefits directly from the contributions of the graduate students and postdoctoral fellows they mentor, this is less the case for mentors of KL2 scholars who are expected to demonstrate an independent track record of scholarship and carve out a research agenda unique from their mentors. Therefore, other means for supporting mentors who do take on the responsibility of mentoring junior clinical and translational researchers becomes of interest.

Ten CTSAs reported incentive initiatives for mentoring of KL2 scholars using a variety of methods, including providing salary support that ranges from a set amount to a percent time, travel dollars for professional meetings, access to core laboratories, and biostatistical support. Two programs reserve fiscal rewards to a senior group of mentors who serve the KL2 program in an active advisory capacity. Creating a culture that publicly expresses gratitude for mentoring is also seen as a means of providing mentor support. For example, Johns Hopkins now has mentoring as a promotion criteria to full professor while other programs present annual awards to outstanding mentors (some, such as Duke University, carry financial incentives up to $10,000) and/or send letters of appreciation to the mentor’s Deans and Department Chairs. These are visible examples of a culture that values the contributions of mentors.

Mentor training also reflects an institution’s commitment to provide resources for mentor support. Overall, nine training programs currently exist with 13 more in development indicating a growing interest and attention to improving mentoring. The format, delivery, and intensity of the training programs in place varies greatly, ranging from half‐day, case‐based workshops to full‐day conferences with invited guest speakers to fuller explorations of personal experiences through case studies and the shared experiences. The curriculum of the more developed programs tends to spread over the course of six to eight 2‐hour sessions. ¹⁷ Methodologies for training include role play, case studies, and personal reflection. Extensive programs exist at the University of California, San Francisco and Oregon Health and Science University and share similar curricular themes, such as communication, diversity, career development, and fostering independence.

Discussion

The findings presented are a result of the first comprehensive survey of mentoring programs among 46 universities awarded NIH’s Clinical Science Translational Awards. The paper provides unique information on programmatic mentoring supports designed to assist KL2 funded investigators become independent scientists working in the area of clinical and translational research. As noted in Table 1 , CTSA‐funded institutions with prior K12 roadmap grants had the largest number of KL2 trainees. The study found that less than 20% of the 46 CTSAs offered comprehensive mentor programs.

Two barriers to developing mentoring structures were identified in the interviews. First, many CTSAs experienced budget cuts to their programs and program directors reported that mentor training programs were a common casualty of the cutbacks. A second obstacle is presented by the experiences of senior faculty who succeeded in spite of not having mentors who were formally trained. They hold earned skepticism as to the impact on scholars’ career success of creating additional mentor supports. These perceptions are justified by a lack of empirical studies investigating the effectiveness of mentor training.

This survey provides valuable information about the types and range of mentoring initiatives in place or in development for KL2 scholars and their mentors. The findings of this study go beyond the KL2 programs. Mentoring is a core element of any research‐training program whether for Ph.D. students, postdoctoral research fellows, junior faculty, or senior faculty shifting to clinical translation science. The development of a comprehensive mentor program is essential if we are to successfully train the next generation of scientists.

Future inquiries that explore how these program supports are actually utilized by scholars and mentors and to what extent they are impacting the quality of the mentoring relationship are warranted. The late Carol Bland ³ states that in order for mentoring structures to be successful, they need to be embedded in a culture that values mentoring and tailored to meet the specific need of subspecialties. Therefore, further investigation into the unique mentoring needs of early career clinical and translational researchers that extend beyond the more general needs of junior faculty or differ from the needs of other groups such as basic scientists is also worthwhile.

Conclusion

This study provides useful baseline data on the current state of mentoring supports for junior clinical and translational researchers among 46 major research institutions. The national CTSA mentoring working group continues to meet and is working on producing a series of papers to assess effective mentor practices to establish a national consensus statement of where KL2 programs could effectively invest their energy and resources.