Long-term evidence that a pediatric oncology mentorship program for young investigators is feasible and beneficial in the cooperative group setting: A report from the Children’s Oncology Group

Abstract

Background:

Mentorship of junior faculty is an integral component of career development. The Children’s Oncology Group (COG) Young Investigator (YI) Committee designed a mentorship program in 2004 whose purpose was to pair YIs (faculty ≤10 years of first academic appointment) with a senior mentor to assist with career development and involvement in COG research activities. This study reports on the committee’s ability to achieve these goals.

Procedure:

An online survey was sent to YIs who were registered with the program from 2004 to2015, assessing three major domains: (1) overall experience with the mentor pairing, (2) satisfaction with the program, and (3) academic accomplishments of the mentees.

Results:

The response rate was 64% (110/171). Overall, YIs rated the success of their mentorship pairing as 7.2 out of 10 (median) (25th, 75th quartile 3.6, 9.6). The direct effects of the mentorship program included 70% YIs reporting a positive effect on their career, 40% reporting any grant or manuscript resulting from the pairing, 47% forming a new research collaboration, and 43% receiving appointment to a COG committee. Respondents reported success in COG with 38% authoring a manuscript on behalf of COG and 65% reporting a leadership position including seven current or past COG discipline chairs and 20 study chairs. Finally, 74% of respondents said they would consider serving as mentors in the program in the future.

Conclusion:

The COG YI mentorship program has been well received by the majority of the participants and has helped to identify and train many current leaders in COG.

1 |. INTRODUCTION

The completion of formal academic training does not conclude the need for further career development.¹ Mentorship as defined by Selwa is a long-term relationship between a senior practitioner and a junior practitioner with responsibility to provide the support, knowledge, and impetus that can facilitate professional success.^2,3 The American Academy of Pediatrics Committee on Pediatric Research issued a policy statement that is supportive of career development for junior faculty and encourages programs that help to establish the mentorship of junior faculty.^1,4 The mentoring relationship is often beneficial to both the mentor and the mentee but also has value to an organization as it helps to identify and nurture future leaders in the organization.⁵

The Children’s Oncology Group (COG) includes over 200 children’s hospitals in the United States, Canada, Australia, and New Zealand whose mission is to cure and prevent childhood and adolescent cancer through scientific discovery and compassionate care.⁶ COG is led by members of the participating institutions and is made of up of many different types of committees as shown in Supplementary Table S1. There are 10 pediatric disease committees and six pediatric cancer domains that create and run clinical trials for most pediatric cancers. Additionally, since there are many different types of providers who care for pediatric cancer patients, there are 13 discipline committees as well as 13 administrative committees. The Young Investigator (YI) Committee was established in 2000 with the initial goals of (1) facilitating interactions between YIs in basic science and clinical research fields, (2) identifying successful investigators to serve as mentors for YIs, (3) facilitating involvement of YIs in COG committees, and (4) providing support and guidance for these collaborations.⁷ The label of “young investigator” is broadly defined as individuals at the instructor or assistant professor level and within the first 10 years of their initial faculty or staff appointment.

In 2004, the COG YI Committee founded a mentorship program with the purpose of helping junior faculty (mentees) connect with more senior members (mentors) within the COG to promote their career development. Early findings from an assessment of the mentorship program were promising, showing that a formal mentorship program was associated with subjective and objective utility for the development of academic pediatric subspecialists and that mentor–mentee pairs that met at least quarterly demonstrated greater academic productivity than pairings that met less frequently.⁷ The present study was initiated to assess the long-term success of the program from the viewpoint of the mentees.

2 |. METHODS

2.1 |. Overview

The COG YI Committee membership consists of liaisons from each of the 10 major pediatric cancer disease committees and 11 of the domain and discipline groups. It is committed to promoting the interests of YIs and facilitating their involvement in the COG by organizing programs for YIs attending the annual COG meeting, offering a career development lunch where YIs can interact with senior investigators from each oncology discipline, and hosting a poster and oral abstract forum where YIs have the opportunity to showcase their research. The YI Committee also administers the mentorship program by soliciting applications, pairing mentees to mentors, and collecting evaluations through an online evaluation system. Mentorship pairings formally last for 3 years. Applicants interested in being paired with a mentor must have completed their terminal training program and be in a faculty or staff position at a COG-affiliated institution. The program is open to and has enrolled varied types of mentees including physicians, nurses, psychologists, and nutritionists. Enrollment is a noncompetitive process, and prospective mentees complete an application describing their clinical and academic interests (Supplementary document S1). YI Committee liaisons from relevant disease and discipline committees help to match the mentee to an appropriate mentor, selected from experienced COG members whose interests align with the mentee’s goals. Once notified of the pairing, mentees are instructed to contact the mentor within 3 months of the initial pairing to set initial goals and expectations for the relationship. Mentees complete formal evaluations at 3 months, 1 year, 2 years, and 3 years, while mentors complete formal evaluations at 1 and 3 years. The YI Committee monitors the mentee–mentor pairings closely and aims to assist in resolving challenges such as differing expectations or poor communication. Evaluations are completed through REDcap surveys that are stored in a password protected database (Supplementary document S2).

2.2 |. Procedure

There were 171 mentees who participated in the program between March 2004 and August 2015. A REDCap survey was created to assess the long-term outcomes and accomplishments of participants in the program. The survey was sent electronically to all past participants with reminder e-mails to nonresponders every 2 weeks up to four times. The Institutional Review Board at Vanderbilt University determined this study was exempt because it is a program evaluation.

2.3 |. Survey

The survey was composed for the purpose of this study. After initial conception, the survey was administered to three independent physicians, amended based on feedback and then used for distribution. There were three main parts to the survey: (1) Mentorship: participants were asked to rate, on a scale of 1–10, the success of the mentorship pairing (i.e., was the pairing productive throughout the program?); (2) Program: in the program assessment section, mentees rated their assessment of the value of the program on a Likert scale (strongly agree, agree, neutral, disagree, and strongly disagree); and (3) Comments: the free text responses provided in the comment section at the end of the survey were analyzed separately through content analysis, a qualitative approach for analyzing data from open-ended questions that involves identifying common categories and themes across content.⁸ A full copy of the survey is provided in Supplementary Document S3. In order to evaluate how mentors have benefited from the program, the 1-year evaluation mentors filled out for their mentees was examined and this survey is provided in Supplementary Document S2.

2.4 |. Analysis plan

Frequencies and distributions of mentee responses are reported. Differences in mentee ratings on demographic variables were examined using the Wilcoxon rank-sum test for continuous variables and Fisher’s exact test for nominal variables.

3 |. RESULTS

The response rate was 64% with 110 of 171 mentees responding to the survey. Mentees who responded were 46% male, while mentors were 61% male. This is in contrast to survey nonresponders who were 28% male. The 110 respondents were involved with 18 different committees, with pathology (N = 23), acute lymphoblastic leukemia (N = 15), acute myeloblastic leukemia (N = 11), and central nervous system tumors (N = 11) being the most prevalent (Fig. 1). There was no significant difference in committee affiliation between respondents and nonrespondents. Most respondents were assistant (41%, n = 45) or associate (33%, n = 36) professors and most (36%, n = 40) completed the program more than 6 years prior to completing the survey (Table 1). Nonphysicians were 5% of the survey respondents but made up 18% of the nonrespondents.

FIGURE 1.

Children’s Oncology Group disease/discipline committees into which the mentor/mentee pairings were made. N = 112 as there were two pairings where the pairing was a joint appointment between two committees. ALL, acute lymphoblastic leukemia; AML, acute myeloid leukemia; CNS, central nervous system tumors; NHL, nonHodgkin lymphoma; AYA, adolescent and young adult oncology; DVL, developmental therapeutics

TABLE 1.

Characteristics of the cohort

Characteristic	n (%)
Current academic rank
Instructor	5(5)
Assistant professor	45 (41)
Associate professor	36 (33)
Full professor	11 (10)
Academic center but not ranked	8(7)
Community program	5(5)
Year started first academic job
1992–2000	19(17)
2001–2005	28 (26)
2006–2010	30 (27)
2011–2014	33 (30)
When completed mentorship program
Still in the program	26 (24)
Within past 3 years	24 (22)
Between 3–6 years ago	20 (18)
Over 6 years ago	40 (36)

3.1 |. Mentee assessment of the program

Participants rated the mentorship pairing as productive or successful throughout the program (median score 7.15 [25th, 75th quartile 3.58, 9.63]). These scores were higher for those currently in the program (n = 26, median = 7.55) or within 3 years postcompletion (n = 24, median = 8.55) compared to those between 3 and 6 years postcompletion (n = 20, median = 5.65), or over 6 years postcompletion (n = 40, median = 6.0) (P = 0.118). Those mentees who dropped out of the mentorship program prior to completion (n = 10) were asked to give reasons why the pairing was unsuccessful. These reasons included mentor not a good fit (n = 3), mentor too busy (n = 9), mentee too busy (n = 2), and communication difficulties (n = 1). Female mentees rated their mentors more favorably (median 8.0) than males (median 6.1, P = 0.173). Female mentees when paired to female mentor (N = 26) reported the highest median success scores (8.7), followed by female mentee to male mentor (N = 33) (7.9), male mentee to female mentor (N = 17) (6.8), and lastly male mentee to male mentor (N = 34) (5.1). None of the gender comparisons reached statistical significance.

3.2 |. Mentee assessment of program effects on their career

Responses to the self-assessment are listed in Table 2. Overall, 70% (n = 77) of respondents reported that the mentorship program had a positive effect on their career. The scores were highest from the group currently in the program (81%) and lowest in the mentees who were 3–6 years from completion of the program (55%). The majority of respondents (58%) rated that the program helped them to get more involved in COG activities. This was highest in those within 3 years of completing the program (67%), and somewhat lower in those currently in the program (58%) and those who were more than 3 years postcompletion (55%). In reporting academic productivity that arose from the program, 40% of respondents agreed that the program led directly to a grant or manuscript publication, 47% formed a new research collaboration, and 43% obtained membership on a COG committee. “Being appointed to a COG committee” was lowest in those currently in the mentorship program (31%) and highest in those within 3 years of completing the program (54%).

TABLE 2.

Self-assessment responses

Question	Strongly agree n (%)	Agree n (%)	Neutral n (%)	Disagree n (%)	Strongly disagree n (%)
Overall the program had a positive effect on my career	41 (37)	36 (33)	17 (16)	4 (4)	12 (11)
The program helped my career development	44 (40)	28 (26)	17 (16)	9 (8)	12 (11)
The program got me more involved in COG	37 (34)	27 (25)	22 (20)	11 (10)	13 (12)
The program led directly to a grant or manuscript publication	29 (26)	15 (14)	24 (22)	25 (23)	17(16)
The program led directly to new research collaborations	31 (28)	21 (19)	25 (23)	17 (16)	16 (15)
The program directly helped me get on a COG committee	33 (30)	14 (13)	17 (16)	29 (26)	17 (16)

3.3 |. Metrics of productivity or leadership in COG

Of the 110 respondents, 70 (64%) reported to have served on a COG discipline or study committee. As might be expected, this is least frequent among those still in the program (35%) but more frequent among respondents who completed the program within 3 years (71%), 3–6 years (70%), and more than 6 years ago (75%). Many of the respondents have authored/coauthored a COG manuscript (38%), and the majority report some type of leadership position (65%) within the COG or another national oncologic organization. The leadership positions included seven COG discipline chairs, two discipline vice chairs, 20 study chairs, and three study vice-chairs, as well as members of steering committees and leaders of task forces. Authoring a COG manuscript and being appointed to a leadership position were infrequent in those currently in the program (15% and 8%, respectively) but were similar between those who completed the program at different times. For participants who obtained a COG leadership position, and excluding respondents currently in the mentorship program, those with leadership positions reported higher median success in their pairing (8.3) than those without a leadership position (6.2), P = 0.121. Finally, 10% of the former mentees have subsequently served as mentors in the program, and 74% of respondents said they would consider serving as a mentor in the program in the future if requested.

3.4 |. Qualitative analysis of free text feedback

Content analysis of the free text feedback in the evaluations revealed a number of themes: (1) program value for COG involvement, (2) program value for career development, (3) desire to serve as a mentor in the future, (4) time constraints impacting the mentor/mentee relationship, and (5) changing roles and responsibilities.

Mentees repeatedly mentioned that involvement in the COG YI mentorship program was successful in helping them to navigate within COG and increase their involvement in research, committees, and annual meetings. One mentee wrote, “This is a wonderful program enabling young investigators to get involved in COG”; another agreed, “a very helpful program to help navigate through the committees.” Additionally, mentees noted that involvement in the YI mentorship program not only facilitated their participation in COG but also benefited their careers as a whole, citing greater access to senior colleagues and professional contacts, increased academic productivity (e.g., publications and presentations at national meetings), and invaluable career advice. One mentee stated, “Coming from a smaller institution, the expertise of my mentor and the ability to make national connections I would not have otherwise made, has been vital to my early success as a PI.” Another noted, “(my mentor and I) continue to publish together. We are always working on 2–3 or more projects/manuscripts at any given time. There has never been a lapse when we weren’t working on a paper together.” Several mentees noted that their experience in the mentorship program has fostered their specific desire to become mentors in the future. One noted, “(I) would love to serve as a mentor someday to give back in return for all that was given to me.” Others indicated that while they were interested in becoming mentors in the future, they did not yet feel that they had the experience to provide effective mentorship to junior faculty.

Challenges were not uncommon in the mentor–mentee relationships. A common theme that emerged was related to time constraints. Both mentors and mentees reported having limited time to devote to the relationship and, more often, limited time to devote to COG responsibilities. One mentor noted, “The program is just fine. Time for academic work at most centers is now severely limited due to increasing clinical responsibilities.” Another challenge involved changing roles or responsibilities that resulted in mismatches between mentors and mentees. For example, one mentee explained, “My role became much more clinical and I was not able to focus on any research projects.” Another had a similar experience and shared, “Ultimately, I chose to go back to clinical medicine only and left for a community based hospital which is no reflection on this program or (the mentor).”

3.5 |. Mentor assessment of the program

Mentor evaluations at the completion of 1 year into the program were available in 51 of 171 pairings (29.8%). Mentors reported that the mentorship pairings were actively maintained after 1 year 82% of the time. They reported corresponding with the mentee at different frequencies: 8% weekly, 26% monthly, 33% quarterly, 26% semiannually, and 8% annually. The mentors reported that approximately 80% of the time, they established goals and a project for the pairing, and their expectations were matched to their mentees’ expectations in 76% of the cases. They reported the pairing was beneficial to them 83% of the time and 96% of the mentors were willing to be a mentor again in the future. They reported giving general career advice 69% of the time, specific advice related to COG 59%, helped to write papers with the mentee 28%, and serve as a connection to obtain a job in 2% of cases.

4 |. DISCUSSION

This long-term assessment of the COG YI mentorship program demonstrates that it is rated favorably by mentees and that the graduates of the program are able to secure leadership positions within COG and author COG-related manuscripts. The YI mentorship program is inclusive and open to participation from any COG member. With representation on the YI Committee by liaisons from all of the major committees, this program allows for the identification of experienced, well-matched mentors regardless of the mentees’ interest. The program’s online evaluation system has also been an effective tool to elicit feedback on the mentorship program so that issues with mentee–mentor pairing can be promptly addressed and allowing for future program improvements based on actual participant experience.

Although mentoring is a nonreimburseable activity, it is highly beneficial to the mentor and the mentee, and we believe the organization as well.⁹ Motifs of the most successful mentoring relationships include setting clear expectations to define the mentoring relationship, ensuring that authorship of the resulting work product reflects appropriate level of involvement, and providing appropriate and timely mentor feedback (praise and constructive criticism) designed to promote career development.⁹ The comments received from our assessment mirrored these positive themes, describing the optimal relationship as one in which the mentor was actively invested and assisted the mentee in obtaining success. In contrast, the negative comments focused on the mentor not prioritizing the mentee. The YI Committee attempts to give prospective mentees realistic expectations about the mentoring program. Although the program can aid in the mentee’s career development and lead to leadership roles in COG, it may take several years for such an opportunity to arise. Although the mentor can help to create opportunities, it is expected that the mentee demonstrate the ability and effort to complete projects when an opportunity arises. It is note-worthy that although over 42% of respondents reported that the program helped them to be appointed to a COG committee, over 70% of respondents ultimately got into a leadership role within a COG committee. This demonstrates that, even if the initial mentorship pairing from the program fails to advance a participant’s COG involvement, a motivated investigator actively seeking additional opportunities can still achieve success.

The success rating for mentee–mentor pairing has trended up over time. This improvement may be a result of changes in the program that included extending the program from 2 to 3 years to allow for more time for project development, and the introduction of a 3-month evaluation to facilitate goal setting, establishing appropriate expectations early in the process, and allowing intervention by the YI Committee when needed.⁷ Some earlier systematic reviews indicated that women have more difficulty in forming successful mentoring relationships; however, this was not seen in our study and, in fact, the female mentee pairings reported greater success than the male pairings.^10,11

There are several limitations to this study. Although the response rate was 63% with 110 responses and a diverse distribution of positive and negative evaluations of the program, there were some nonresponders who could have differed from the responding cohort. A reporting bias is possible as those who had stronger feelings about the program may have been more likely to respond and those who derived success from the program may have been more likely to have reported it. There were 14 mentees who did not respond to the long-term assessment but previously evaluated, of which 11 spoke positively about the program, one had a mixed response, and two were negative. Hopefully, this is indicative of a similar trend among the remaining nonresponders; however, it is unknown how the other nonresponders truly felt about the program. A recall bias is also possible as respondents who completed the program many years ago may not have fully recalled the details of the program or the specifics of their mentoring relationship/experience. Nevertheless, it is clear that a substantial number of respondents have had a positive experience from the program. Only 9% (16/171) of the mentorship program mentees have been nonphysicians and their response rate for the long-term assessment was 31% (N = 5), therefore making it difficult to draw any conclusions about the success of the program for this group.

There have been several other successful multiinstitutional mentorship programs piloted in pediatrics,^12–14 including a successful pilot by the American Society of Pediatric Hematology/Oncology (ASPHO), which provides earlier access to mentorship while still in academic training. Fourteen of the first 16 mentees in the ASPHO program agreed that the program was a rewarding experience.¹² Future directions for the YI mentorship program will be to develop a curriculum and guidelines for prospective mentors to assist them in their role of providing effective mentorship that is capable of leading to the mentee’s immediate and long-term success in COG. We also believe that these data will be informative to other organizations and help them establish and organize similar programs, enabling them to meet their mentorship needs and the needs of other pediatric oncologists.

Abbreviations:

ASPHO

American Society of Pediatric Hematology/Oncology

COG

Children’s Oncology Group

YI

Young Investigator