Policies, Activities and Structures Supporting Research Mentoring: A National Survey of Academic Health Centers with Clinical and Translational Science Awards

Abstract

Purpose

To document the frequency of policies and activities in support of mentoring practices at institutions receiving a U.S. National Institutes of Health’s Clinical and Translational Science Award (CTSA).

Method

The study consisted of a 69-item survey with questions about the inclusion (formal or informal) of policies, programs/activities and structures supporting mentoring within CTSA-sponsored research (i.e., KL2 programs) and, more broadly, in the CTSA’s home institution. The survey, conducted from November 2010 through January 2011, was sent to the 55 institutions awarded a CTSA at the time of the survey. Follow-up phone interviews were conducted to clarify responses as needed.

Results

Fifty-one of 55 (92%) institutions completed the survey for institutional programs and 53 of 55 (96%) for KL2 programs. Responses regarding policies and activities involving mentor criteria, mentor–mentee relationship, incentives, and evaluative mechanisms revealed considerable variability between KL2 and institutional programs in some areas, such as having mentor qualification criteria and processes to evaluate mentors The survey also identified areas, such as training and women and minority mentoring programs, where there was frequent sharing of activities between the institutional and KL2 programs.

Conclusions

KL2 programs and institutional programs tend to have different preferences for policies versus activities to optimize qualification of mentors, the mentor–mentee relationship, incentives and evaluation mechanisms. Frequently, these elements are informal. Individuals in charge of implementing and maintaining mentoring initiatives can use the results of the study to consider their current mentoring policies, structures, and activities by comparing them to national patterns within CTSA institutions.

A great deal of attention has been paid to mentoring in academic medicine as a way to promote research productivity and career success. ^1–3 While a definition of mentoring varies according to the context in which it occurs, mentoring broadly encompasses career-related and psychosocial functions.⁴ For the survey reported here, we defined research mentorship as having the following functions: developing mentee research independence, teaching and providing guidance on acquisition of research skills, including coursework and other training opportunities; counseling; facilitating socialization; and role modeling professional behavior, including the responsible conduct of research.⁵ Effective mentoring has the potential to enhance mentee professional identity and personal competence ^6–9 and, within the context of academic medicine, may contribute to greater research productivity and faculty advancement. ^1,10,11

Research mentoring, and mentoring in general, can occur either in informal or formal environments within the context of a variety of relationship structures, which include one-on-one mentoring, team mentoring, group mentoring, and peer mentoring.¹¹ Traditionally, research mentoring has been conducted as a “cottage industry,” involving a one-to-one mentor–mentee relationship between a more senior researcher and a relatively more junior researcher. More recently, academic health centers and their departments have begun to play a larger role in encouraging research mentorship.^12–14 Many of these efforts have coincided with national reports addressing the need to stem the leakage from the pipeline of junior researchers in the current environment. ^15–18 In response to these concerns, many institutions have developed mentoring programs and initiatives where the institution provides organizational structure around policies and activities that support mentoring. ^19–24 In addition, the National Institutes of Health (NIH) funds individual and institutional career awards (K awards) that provide funding and protected time to pursue a mentored research experience to achieve scientific independence. ²⁵

A significant initiative that addresses the enhancement of research mentorship is the creation of the Clinical and Translational Science Awards (CTSA) Consortium by the NIH. The CTSA funding mechanism has implemented innovative clinical and translational research efforts at over 60 academic health centers since 2006. ²⁶ A goal of the CTSA award is that it will improve existing clinical and translational research career development. To pursue this goal, CTSA awards support a funding mechanism, called KL2, that is similar to other mentored research career award mechanisms at the NIH. The KL2 funding mechanism supports KL2 programs that train physicians and other health practitioners in clinical and translational research for 2–5 years under the guidance of one or more mentors. ²⁷ The size and scope of the KL2 program provide a rich environment to address the impact of various mentoring practices and develop more effective mentoring environments. In fact, the CTSA Consortium Education Key Function Committee formed a mentoring workgroup to facilitate the gathering and sharing of information on mentoring practices.

A number of reports have documented the role of institutions to develop policies and/or activities to support research mentoring.^8,28–34 A survey by Silet et al in 2010 looked at mentoring in KL2 programs in light of what is present in the literature regarding best mentoring practices. ³⁵ In the present report, we share results of a survey that documents the policies and activities within KL2 programs funded by the NIH’s CTSA mechanism and within the broader institution mentoring initiatives that were present from November 2010 through January 2011. The survey is grounded in a conceptual model presented by Keyser and colleagues in 2008 describing how institutions can integrate and assess key domains important in research mentoring. ⁵ As such, this model can provide insight into the policies and activities that KL2 programs and institutions use to support research mentoring and the interplay between these initiatives.

Method

Survey development

Through an iterative process, we used the framework from Keyser et al., ⁵ input from experts in the mentoring field, other surveys, and the literature to guide item development. The 69-item survey focused on questions associated with mentoring activities and policies within KL2 and institutional mentoring structures. The survey used an open-ended matrix format comprising five domains, on three levels: The domains included Mentor Criteria, Mentor Incentives, Mentor–Mentee Relationships, Mentor–Mentee Research, and Mentee/Mentor Professional Development. The three levels include what policies existed about these domains, what activities existed to help achieve these domains, and, lastly, where the structural responsibility for each domain lies. We converted these domains outlined in the Keyser et al. framework into close-ended questions with a discrete set of responses, and an “other” response option for potential responses we did not foresee.

The integrative nature of CTSAs makes it complicated to view CTSA activities without also looking at activities that are happening at the institutional level, and so we chose to ask similar questions about mentoring activities at both these levels. Items were further refined by piloting the survey. The Columbia University Institutional Review Board approved the study.

Survey administration

We administered the web-based survey from November 2010 through January 2011 to potential respondents from the 55 CTSA institutions that were funded during the survey period. Prior to administering our survey, we contacted KL2 educational leaders at each CTSA awarded institution to determine who was appropriate to answer the KL2 and institutional survey questions. Depending on the responses received to our initial queries, we sent out one of three versions of the survey to individuals representing a total of 55 institutions: a survey addressing mentoring at the KL2 level (to 9 institutions), one looking at mentoring at the broader institution (to 9 institutions), and lastly, a survey that combined both sets of questions (to 46 institutions).

As surveys were completed, we conducted 30-minute follow-up phone interview to clarify inconsistent, unclear or unique responses when possible. The interviews also provided additional details about other KL2 or institutional mentoring initiatives and resources.

Results

Descriptive data on respondents, KL2 scholars, and mentors

We received responses from 53 of 55 KL2 programs (96%) and from 51 of 55 institutions (92%). We received responses from 53 of 55 KL2 programs (96%) and from 51 of 55 institutions (92%).

Regarding the size of KL2 programs, 15 (28%) reported having fewer than 5 KL2 scholars, - 23 (44%) had between 5–10 KL2 scholars, and 15 (28%) had more than 10 KL2 scholars. These differences reflect the fact that more recent CTSA-awarded institutions are smaller in overall size and that the average number of mentors in each year’s awarded KL2 programs has decreased from 30.67 mentors in 2006 to 11 in 2010. The average number of years that a KL2 scholar is funded across all programs is 2.6 years.

Implementation of policies and activities at the KL2 level

Table 1 provides results from the 51 institutional responses and the 53 KL2 program responses on the presence of various mentoring policies and activities and structures on both the KL2 and institutional levels at the CTSA-awarded academic health centers

Table 1.

Numbers and Percentages of Responses to Questions About Policies and Activities Related to Mentoring Within Institutional and KL2 Mentored Career Development Programs at Academic Health Centers With Clinical Translational Science Awards (CTSAs), November January 2011 ^*

Policies/Activities, with type of program	Yes: Number (%) [% formal]†	No: Number (%)	Don’t know: Number (%)	Other: Number (%)	No response given: Number (%)
Criteria to qualify as a mentor
KL2 programs	48 (91) [51]	5 (9)	-	-	-
Institutional programs	14 (27) [10]	34 (67)	2 (4)	-	1 (2)
Policies on mentor responsibilities
KL2 programs	42 (79) [51]	10 (19)	1 (2)	-	-
Institutional programs	22 (43) [18]	21 (41)	7 (14)		1 (2)
Mentor/mentee written agreement
KL2 programs	20 (38)	33 (62)	-	-	-
Institutional programs	7 (14)	38 (74)		-	6 (12)
Policies to manage conflicts
KL2 programs	30 (57) [21]	21 (39)	2 (4)	-	-
Institutional programs	25 (49) [29]	19 (37)	7 (14)	-	-
Incentives
KL2 programs	28 (53)	25 (47) 47	-	-	-
Institutional programs	35 (69)	12 (23)	4 (8)
Formal face to face training for mentors
KL2 programs	34 (64)	19 (36)	-	-	-
Institutional programs	23 (45)	26 (51)	2 (4)		-
Processes to evaluate mentors
KL2 programs	42 (79)	11 (21)	-	-	-
Institutional programs	20 (39)	20 (39)	9 (18)	1 (2)	1 (2)
Women/minority mentoring programs
KL2 programs	10 (19)	40 (75)	1 (2)	2 (4)	-
Institutional programs	30 (59)	12 (23)	9 (18)	-	-

^*The table data are from responses to a survey of 55 U.S. Academic Health Centers with CTSAs (i.e., KL2 programs) to questions about the presence or absence of policies and activities related to mentoring within CTSA-specific training programs and within the institution more broadly. Fifty-one institutions (92%) and 53 KL2 programs (96%) completed the survey. The survey was conducted from November 2010 through January 2011. All percentages in the table are based on the number of KL2 programs responding.

^† Some questions included the option of “Yes, formal” and “Yes, informal,” referring to policies, programs/activities, and structures supporting mentoring within the CTSA-sponsored research and in the CTSA’s home institution. All “Yes” responses were combined into a single percentage, shown in parentheses. The percentages of schools that listed “formal” are shown within brackets. Where there are no brackets associated with the “Yes” responses, the question did not differentiate between formal and informal policies or activities.

The majority of responding KL2 programs (48; 91%) identified one or more criteria, formal or informal, for their faculty mentors at the KL2 level to meet to qualify as a mentor (see Figure 1, panel A). The three most frequently selected qualifications related to the mentor’s seniority and experience: independent research funding 45 (85%), previous mentoring experience 41 (77%) and senior faculty level 36 (68%). In our follow-up phone interviews, many of the respondents stated that it was fairly common that the primary mentor had at least two of the three criteria.

Figure 1.

Percentages of responses from 53 of 55 U.S. Academic Health Centers with a Clinical Translation Science Award (CTSA) to specific questions about the content of specific policies, activities, and structures supporting mentoring within their CTSA -funded KL2 programs. Data were obtained from November 2010 through January 2011 from the authors’ survey. Respondents were able to check all options that applied, including “Other” and “I don’t know.”

Addressing mentor responsibilities, we asked the 42 (79%) respondents who indicated they had some policy regarding expectations for their KL2 mentor (see Table 1) to identify these policies or expectations. The responses are summarized in Figure 1, panel B.

As the literature indicates that written guidelines outlining mutual expectations between mentors and mentees are important in the mentoring relationship, we asked about the presence of mentor–mentee written agreements. ^36–38 Twenty (38%) KL2 programs reported having written agreements (see Table 1). Looking at the presence of KL2 policies to manage conflicts (see Table 1), 30 (57%) KL2 programs reported having policies in place, with 11 (21%) identifying that these policies were “formal” (i.e., that policies were written to deal with conflicts).

Two activities that are prominent in discussions of institutional efforts to support research mentoring are providing incentives and training. ^{10,24,39–41} 28 (53%) KL2 programs reported having incentives (Table 1), with consideration in annual evaluation or promotion being the most common (Figure 1, panel C). Thirty-four (64%) KL2 programs reported that they offered face-to-face mentor training with follow-up interviews revealing considerable variability related to the expectations, scope and content of the training, ranging from inclusion of training within the orientation to multiple sessions covering mentoring skills and competencies. Only 19(approximately 56%) of those programs offered training that required mentors to attend. When asked about evaluation of the mentor and/or mentoring relationship in the KL2 program, 42 respondents (79%) reported using one or more evaluation process (Table 1) to evaluate the mentor and/or mentoring relationship in the KL2 program, As shown in Figure 1, panel D, the processes used most frequently to carry out these evaluations involved asking mentees for their perceptions by means of formal meetings or a written survey. Not shown in Table 1, of the 42 programs that evaluate mentors, 9 (21%) provided written feedback to mentors and 17 (40%) provided verbal feedback.

Implementation of policies and activities at the institutional level

Institutional data pertaining to responses to specific policies and activities are shown in Table 1 alongside KL2 program data. For most of the aspects that we surveyed, institutions were less likely than KL2 programs to have in place the policies, activities and structures in question. The differences were particularly pronounced with regards to having criteria for mentors and policies on mentoring responsibilities. . Of the 51 institutions responding, only 14 (27%) indicated having formal criteria for mentors, whereas 48 (91%) KL2 programs had such criteria. Regarding policies on mentor responsibilities, 22 (43%) of institutional respondents reported having such policies.

Evaluation of mentor performance was also less frequently reported at an institutional level, being reported in approximately 20 (39%) institutions. Of the 20 institutions providing evaluation on mentor performance, 4 (20%) reported that formal written feedback and was provided to the mentor and 10 (50%) reported providing informal feedback.

Offering formal face-to-face training, the availability of incentives, and the inclusion of policies to manage conflicts were more equally reported at both the KL2 and institutional levels. This finding reflects the sharing of these features between the KL2 and institutional levels, with KL2 programs often utilizing existing institutional policies. However, with regards to mentor training, survey responses indicated that KL2 program training activities tended to be extended to or adopted at institutional levels. In contrast, programs specifically addressing mentoring for women or minorities were much more likely to be reported at an institutional level.

Only 26 (51%) institutional respondents indicated that their institution had a program that matched mentors with mentees, with another 2 (4%) responding that they were unsure if mechanisms for matching existed. Of those institutions that matched mentors and mentees, responsibility for matching was at the departmental level for around 10 (40%) respondents, with another 8 (around 30%) indicating that responsibility was at the level of individual health sciences schools. Overall, respondents to questions at the institutional level had higher rates of responses of “I don’t know” to questions related to policies and activities such as the presence of mentor responsibilities, evaluation processes, and the use of mentor/mentee agreements.

Discussion

The results of our survey of CTSA-funded institutions provide a snapshot of the policies and activities used within these institutions to support mentoring of junior research investigators. With separate responses pertaining to elements at the KL2 and the institutional levels, the results further reveal the interplay between the policies and activities across these levels within the institutions. While the context of every institution is unique, we believe that the patterns of response frequencies to the different elements in our survey provide valuable insights to organizations striving to use mentoring programs to enhance the success of junior researchers.

To facilitate a discussion of these insights, we organized the findings using a conceptual framework adapted from Keyser et al, ⁵ whose framework also helped guide the development of our survey. This framework states that policies, activities and structures (i.e., accountable entities for developing or implementing policies and activities) fall under three broad domains (goals) of mentoring programs: optimizing qualifications of mentors, supporting mentor–mentee relationships, and guiding mentor engagement.

Optimizing qualifications of mentors

To optimize qualifications of mentors, our results indicate that the KL2 programs had a preference for policies over activities: 48 (91%) of our respondents had criteria to qualify to be a mentor, while only 34 (64%) provided face-to-face training for mentors. Furthermore, 27 (51%) of KL2 programs had formalized these criteria, but only 19 (56%)% had made their face-to-face training required. In contrast, at the institutional level, the preferred approach to optimizing qualifications of mentors was reversed. Institutions offered face-to-face training more frequently than they established selection criteria, 23 (45%) and 14 (27%), respectively). We speculate that these preferences stem from the fact it is more cost–effective for program leaders to promote the selection of mentors who already have desired qualifications than to train unqualified mentors and that, at the KL2 level, it is more likely that such criteria can be established, communicated, and enforced because they have a more homogeneous focus, which makes it easier to recruit, select, and fund only those trainees who have chosen qualified mentors.

Our results also reveal a preference for specific mentor qualifications among KL2 programs, including independent research funding, previous mentoring experience, and advanced rank. These preferences, based on the experience of a diversity of institutions, can guide organizations building and improving their own mentoring programs. Assuming that there is likely to be practical benefit from establishing a limited number of “high-yield” qualifications, these results help to identify which qualifications to consider first.

Guiding mentor engagement

To guide mentor engagement, our results indicate that KL2 programs relied more heavily on activities to evaluate and give feedback than to provide incentives: 42 (79%) and 28 (53%), respectively. Again, in contrast, institutions tended to rely more on incentives than activities for evaluation and feedback: 35 (69%) and 20 (39%), respectively. As in our discussion of results regarding approaches to optimize qualifications, we speculate that KL2 programs may prefer to guide mentor engagement through evaluation and feedback because these processes are more likely within the authority of the KL2 leadership and may aid in NIH reporting requirements and continuing program support. Soliciting mentees’ perceptions, either in a meeting or via a written survey, appears to be the mainstay of evaluation processes and can be achieved with minimum resources and as part on ongoing efforts to monitor and guide mentee progress. Since some of the incentives included in our survey depend upon resources beyond the control of the KL2 program, it is only logical that those incentives would be used more frequently at the institutional level. Those incentives include annual faculty evaluations and processes to ensure that quality mentoring contributes to promotion and tenure consideration.

We include programs for promoting women and minorities as mentors in this part of our discussion because such programs often prioritize recruitment and community building—two areas directly related to guiding mentor engagement. Not surprisingly, our results document that activities specific to these groups rarely occur at the KL2 level. In fact, we speculate that, as with other activities most logically offered at the institutional level (e.g., membership in a teaching academy), KL2 leadership may actually piggy-back on institutional initiatives and report them as aligned with their programs. Based on the results of our survey, we encourage leaders within training programs, such as the KL2, to maximize their use of institutional resources, especially in such areas as training and incentives.

Supporting mentor–mentee relationships

To support mentor–mentee relationships, our results indicate that KL2 programs have a preference for policies that establish mentor responsibilities (42; 79%) over policies for managing conflicts (preferred by 30; 57%) and a written agreement (preferred by -------; 38%). Institutions favored policies for managing conflicts (25; 49%) over policies about mentor responsibilities (22; 43%) or a written agreement (7;14%). Interestingly, policies for mentor responsibilities were more likely to be formalized at the KL2 level than at the institutional level (27; 51% versus 9; 18%, respectively), while policies to manage conflict were less likely to be formalized at the KL2 level than the institutional level (11; 21% versus 15; 29%, respectively). Perhaps this finding, as with previous findings discussed, stems from the availability of resources at the institutional level for actually implementing policies for managing conflict (e.g., greater awareness of, and access to, conflict resolution expertise). Of some surprise to us is the relatively small number of KL2 programs that were using written mentor–mentee agreements (21; 38%). We speculate that this number will increase as programs and institutions recognize the potential value of these documents to clarify mutual expectations to avoid misunderstanding or conflict.⁴² In training programs, such as the KL2, including such agreements as part of program requirements would require few resources and would be relatively easy to enforce as part of ongoing activities to monitor a mentee’s progress.

Limitations

While the survey focused on research mentoring, because mentoring policies and activities are shaped by the context within which mentoring occurs, a limitation in our survey is that institutional mentoring programs may reflect a broader research mentoring context, or even a non-research focus, than KL2 programs do, which focus exclusively on research mentoring within a relatively narrow context, Also, we speculate that some institutional respondents may have had limited knowledge about the subjects of particular survey questions, which is likely reflected in the increased frequency of “I don’t know ” in institutional responses. Potential reasons for this response may be the informal nature of policies and activities at the institutional level and the possibility that the institutional representative responding did not have sufficient knowledge to answer certain questions, despite being identified as the most appropriate responder. The relatively limited availability of programs, the relatively low specificity of policies and the higher rate of “I don’t know” responses at the institutional level may also reflect a lack structure at that level, i.e., the lack of an office or high-profile leader charged with the mission of developing and implementing policies and programs to encourage mentoring for faculty at the institution. In contrast, CTSA KL2 programs, by definition and through the NIH funding requirements and resources provided, have a clearly defined structure and mechanism of accountability.

Future Directions

While mentoring exists in a variety of formats in academic medicine, institutions have begun to develop policies and activities to encourage effective mentoring of junior investigators. These efforts are made with the expectation that research mentoring programs and other formal initiatives may more effectively assist junior researchers in academic health sciences to achieve their goals in an increasingly complex and competitive research environment.^{3, 11, 13} These programs may also provide institutional benefits such as improving research productivity and quality, increasing faculty retention and recruitment, and contributing to overall faculty vitality.

The results of this survey provide insights into the composition of various policies and activities present in KL2 and institutional mentoring programs within the context of the conceptual model proposed by Keyser and colleagues. ⁵ As a conceptual model, there is an assumption of an association between the proposed framework of mentoring programs and positive mentoring outcomes and individual, institutional and societal benefits. However, much more research is needed to assess the degree to which any of these strategies work to enhance the short, intermediate and long-term goals at each of these levels. Understanding the current implementation of various domains in the framework among existing programs can assist in targeting and designing these further investigations within the varying institutional environments and cultures where research mentoring occurs.