Efficacy of the AMIGO inter-institutional mentoring program within pediatric rheumatology

Abstract

OBJECTIVE:

The small size of many pediatric rheumatology programs translates into limited mentoring options for early career physicians. To address this problem, the American College of Rheumatology (ACR) and the Childhood Arthritis and Rheumatology Research Alliance (CARRA) developed a subspecialty-wide inter-institutional mentoring program, the ACR/CARRA Mentoring Interest Group (AMIGO). We sought to assess the impact of this program on mentoring within pediatric rheumatology.

METHODS:

In a longitudinal 3-year study, participant ratings from the AMIGO pilot program were compared with those after the program was opened to general enrollment. Access to mentoring as a function of career stage was assessed by surveys of the US and Canadian pediatric rheumatologists in 2011 and 2014, before and after implementation of AMIGO.

RESULTS:

Participants in the pilot phase (19 dyads) and the general implementation phase (112 dyads) reported comparable success in establishing mentor contact, suitability of mentor-mentee pairing, and benefit with respect to career development, scholarship, and work-life balance. Community surveys showed that AMIGO participation as mentee was high among fellows (86%) and modest among junior faculty (31%). Implementation correlated with significant gains in breadth of mentorship and in overall satisfaction with mentoring for fellows but not junior faculty.

CONCLUSION:

AMIGO is a career mentoring program that serves most fellows and many junior faculty in pediatric rheumatology across the US and Canada. Program evaluation data confirm that a subspecialty-wide inter-institutional mentoring program is feasible and can translate into concrete improvement in mentoring measurable at the level of the whole professional community.

Mentoring refers to an extended relationship through which a person of advanced professional standing guides a more junior colleague toward career success (1). Within medicine, mentoring usually targets academic advancement, but it may also foster success in non-academic aspects of professional life, such as work-life balance (2, 3). Recognizing the critical role of mentoring in academic medicine, academic and professional institutions increasingly seek to formalize this process to ensure that trainees and junior faculty receive appropriate guidance (4–6). However, documenting the success of mentoring programs has proven to be a major challenge (2, 6, 7).

Within rheumatology, lack of effective mentoring is increasingly recognized as a barrier to academic success (8–11). Pediatric rheumatology presents a particular challenge for early-career physicians seeking subspecialty-specific mentors, because most programs are small and therefore offer few local options (12, 13). Further, nearly all pediatric rheumatologists practice within teaching hospitals and therefore must negotiate an academic career. Surveys of fellows and junior faculty correspondingly identify mentoring as a major unmet career need in pediatric rheumatology (14, 15).

Recognizing this “mentorship gap,” the two major professional organizations serving North American pediatric rheumatology -- the American College of Rheumatology (ACR) and the Childhood Arthritis & Rheumatology Research Alliance (CARRA) -- developed the ACR/CARRA Mentoring Interest Group (AMIGO) (14). AMIGO includes two interventions. First, educational/networking sessions are held at each ACR and CARRA annual meeting to address common problems facing early career pediatric rheumatologists. Second, an inter-institutional mentoring program pairs interested fellows and junior faculty with volunteer career mentors at different centers. A pilot program consisting of 20 mentee-mentor dyads was launched in November 2011, and yearly cycles of enrollment have been open to all fellows and junior faculty since that time.

To assess the impact of AMIGO, a two-part program evaluation strategy was designed. First, participants were surveyed to assess measures of process adequacy and self-reported benefits. Second, US and Canadian pediatric rheumatologists were surveyed before and after implementation to identify associated changes at the level of the whole community. The implementation and evaluation of AMIGO are reported here.

MATERIALS AND METHODS

The AMIGO program.

AMIGO mentees and mentors were recruited by notices on the highly-subscribed McMaster University pediatric rheumatology list serve and the ACR’s Pediatric Rheumatology list serve; by announcements at meetings; and by direct email to US and Canadian division heads, program directors, and other community leaders. Fellows were encouraged to enroll during their initial year of subspecialty training, and junior faculty (defined as assistant professor and below) could participate as mentees until achievement of the rank of associate professor or R01-level funding. All interested mentees were accommodated during each annual cycle. Mentors were selected from faculty volunteers elicited as above.

Matching was performed using a computer algorithm based on responses of prospective mentees and mentors to an online questionnaire. The algorithm proposed 5 candidate mentors for each mentee, after evaluating all potential dyads for compliance to program rules (mentees and mentors could not be affiliated with the same institution; mentors had to be of higher academic rank) and fit to mentee requests for mentor characteristics, including general career track (clinician/educator, clinical researcher, basic researcher), mentor gender, and specific expertise areas, such as balancing home and family or working part-time. If mentors were not sufficiently differentiated by these criteria, overlap between mentee- and mentor-expressed research interests and disease area interests were employed. In general, fellows were matched with junior faculty mentors, and junior faculty mentees were matched with senior faculty (defined as associate and full professors). Junior faculty could participate simultaneously as mentee and mentor. More senior mentors could be assigned 2 mentees. Final matches were adjudicated by a volunteer AMIGO Steering Committee, which was guided but not bound by the algorithm results. Mentorship duration was set at 3 years. Mentors were not compensated for AMIGO participation.

Once assigned, mentees and mentors were introduced by email and encouraged to exchange curricula vitae. A “mentoring tool kit” of related guidance was provided (https://www.rheumatology.org/Education/Careers/AMIGO/). Dyads received periodic email reminders to maintain contact. The manner, frequency and content of mentor-mentee interactions were left to the discretion of participants.

AMIGO participant surveys.

AMIGO mentees and mentors provided email addresses that were used to direct participants to anonymous online surveys using the Qualtrics interface (Qualtrics, LLC, United Kingdom). The first survey, in March 2013, included only 2011 pilot dyads. The second survey, of all AMIGO participants, was conducted from December 2013 - January 2014. Participants were asked to report whether and how dyads had interacted to discuss the mentee’s career; whether the skill set of the mentor was appropriate to address needs of the mentee, as assessed by the participants (termed mentor-mentee fit); whether the mentee experienced benefit in specific domains related to professional life; and whether the interaction was expected to prove helpful to the mentee.

Surveys of the pediatric rheumatology community.

To evaluate the impact of AMIGO on the community as a whole, US and Canadian pediatric rheumatologists were directed from the McMaster and ACR list serves to anonymous online Qualtrics surveys. Participants were asked to respond only once. The first survey was administered from December 2011 to January 2012; participants who had just enrolled in the AMIGO pilot phase (November 2011) were asked to provide pre-AMIGO answers. The second survey was administered via the same channels between February and March 2014. Participants were asked to report access to a mentor in specific career domains at the home institution, at an outside institution, both or neither, and to rate overall satisfaction with the career mentoring.

Statistical analysis.

Significance of differences between proportions were calculated using Chi Square or Fisher’s exact test as appropriate. Data were prepared using SPSS (SPSS, Inc., Chicago) and analyzed using GraphPad Prism software (GraphPad Software, Inc., San Diego).

Institutional Review Board approval.

Surveys of AMIGO participants were considered program evaluation and therefore not subject to review per the guidelines of the Brigham and Women’s Hospital. Anonymous surveys of the general pediatric rheumatology community were considered exempt human subjects research by institutional review boards at the University of Michigan and the Robert Wood Johnson Medical School.

RESULTS

AMIGO Pilot Phase Implementation and Assessment.

The AMIGO pilot program was launched in November 2011 with 20 mentee-mentor dyads. Pilot mentee participants were selected randomly from among 60 interested fellow and junior faculty, and paired with one of 45 volunteer faculty mentors. One dyad dissolved due to the mentee’s departure from North America. The remaining 19 dyads were surveyed after 17 months to ascertain the dynamics and potential benefit of the interaction (response rate 37/38, 97%). Mentor-mentee fit was generally good (Figure 1.A.). All dyads made contact at least once to discuss the mentee’s career, and the majority reported 2 or more interactions, though total duration of contact was often modest (Figure 1.B–C). As previously reported, mentees reported benefit in domains including career development and scholarship, while mentors reported an improved sense of connection to the community and enhancement of their teaching profiles (14).

Figure 1. AMIGO process measures in pilot phase and general implementation.

A-C. Pilot phase 17 months after initial match (n=39 mentees + mentors). D-F. General implementation (mentor-mentee fit, n=140 mentees + mentors; most recent interaction with mentor, n=73 mentees; whether pairing was likely to prove beneficial to mentee, n=138 mentees + mentors).

AMIGO General Implementation and Assessment.

AMIGO was opened to general enrollment in 2012. 57 dyads were matched in November 2012 and 38 in November 2013, drawing mentors from a pool of 52 volunteers in 2012 and 36 in 2013. All AMIGO participants were surveyed by email between December 2013 and January 2014. Respondents included 77 of 112 mentees (69% response rate, including 7 who were also mentors). Given the highly variable duration since match (16 mentees in 2011, 28 mentees in 2012 and 33 mentees in 2013), the survey did not attempt to assess frequency and duration of mentor-mentee interactions. However, mentor-mentee fit was comparable to that reported in the AMIGO pilot (Figure 1.D.). Mentees reported that 95% of dyads (69 of 73 respondents) had engaged in at least one discussion regarding the mentee’s career, generally within the last 2–6 months, including approximately 90% who reported having met in person (Figure 1.E. and data not shown). Assessment of overall benefit to mentees was comparable in pilot and general implementation phases (Figure 1.F).

Mentees were asked to report perceived benefit from AMIGO in specific career domains. Gains were most frequently reported in career development, scholarship, work-life balance, and connectedness to the pediatric rheumatology community (Figure 2). These results are concordant with findings from the pilot phase (14).

Figure 2. Proportion of fellow and junior faculty mentees reporting benefit from AMIGO in specific professional domains.

Data reflect n=31 fellows and n=18–28 junior faculty mentees from the 2014 survey of AMIGO participants. All comparisons ns via Fisher’s exact test.

Surveys of the pediatric rheumatology community.

To assess the effect of AMIGO on mentoring for early career rheumatologists, we performed pre-AMIGO (2011) and post-AMIGO (2014) surveys of the US and Canadian pediatric rheumatology community. Characteristics of the respondents are enumerated in Table 1. Since surveys were administered by list serve, we were unable to calculate absolute response rates. However, the US and Canadian pediatric rheumatology community includes fewer than 350 faculty and fellows, such that response rates exceeded 50% for both surveys. Respondent populations were comparable across surveys (Table 1). In the 2014 survey, 86% of fellow respondents reported participating in AMIGO, whereas fewer junior faculty participated as mentees (31% participated as mentees and 29% as mentors, including 10% as both mentees and mentors).

Table 1.

Respondents to pediatric rheumatology community surveys before and after AMIGO implementation

Survey	pre-AMIGO*				post-AMIGO**
	Total	Fellows	Junior Faculty	Senior Faculty	Total	Fellows	Junior Faculty	Senior Faculty
n	227	56	74	64	177	36	58	61
Female	116 (51%)	40 (71%)	48 (67%)	28 (44%)	105 (59%)	27 (75%)	29 (67%)	32 (53%)
AMIGO participants	--	--	--	--	99 (58%)†	31 (86%)	42 (50%)‡	37 (60%)

Percentages reflect the proportion of respondents to each question.

^*Survey conducted 12/2011–1/2012. Number of respondents: academic rank n=194; gender n=174.

^**Survey conducted 2/2014–3/2014. Number of respondents: academic rank n=166; gender n=165; AMIGO participation n=170.

^†Includes respondents for whom no academic rank was provided.

^‡Of 58 junior faculty respondents, 18 (31%) participated as mentees and 17 (29%) as mentors, including 6 (10%) that participated both as mentees and as mentors.

Respondents at all career levels were asked to report access to mentoring in domains relevant to academic rheumatology: clinical practice, teaching, research, setting career goals, and identifying how to achieve career goals (Figure 3). In both 2011 and 2014 surveys, most fellows and junior faculty acknowledged having a mentor of some kind in each domain. In 2011, fellows were much less likely than senior faculty to have mentors outside their home institutions (p<0.05 in all categories except teaching and how to achieve career goals; Figure 3 top). By 2014, the proportion of fellows with outside mentors increased markedly in the domains of research, setting career goals, and achieving career goals (Figure 3 bottom). As a result, the difference in the proportion of fellows and senior faculty with outside mentoring was no longer observed (p=ns for all domains). No change was observed in clinical or teaching domains. No change was observed in the proportion of fellows and junior faculty who reported having no mentor in a domain.

Figure 3. Access to mentors in pediatric rheumatology.

2011 respondents were 54 fellows, 72 junior and 59 senior faculty. 2014 respondents were 34 fellows, 57 junior and 57 senior faculty. * <0.05, ** <0.01 via Chi squared.

Finally, we compared overall satisfaction with mentoring before and after AMIGO. Both fellows and junior faculty reported considerable satisfaction at baseline. An improvement nevertheless emerged for fellows; a similar but statistically non-significant trend was observed among junior faculty. No change was observed among senior faculty (Figure 4).

Figure 4. Satisfaction with mentoring before and after AMIGO.

Data compare the baseline survey in 2010–2011 (light gray bars, n=53 fellows, 60 junior faculty, 57 senior faculty) and the post-AMIGO survey in 2014 (dark gray bars, n=35 fellows, 57 junior faculty and 55 senior faculty). Fellows p=0.01, junior faculty p=ns, senior faculty p=ns via Chi squared.

DISCUSSION

Mentoring is a major contributor to career success in academic medicine.(6) Recognizing a particular need for improved access to mentoring in pediatric rheumatology, the ACR and CARRA developed the AMIGO mentoring program. The present study sought to assess whether AMIGO conferred measurable value.

AMIGO participants were able to establish mentee-mentor contact. The quality of mentee-mentor fit was largely preserved in the transition from pilot phase to general implementation, though a small decrement was apparent, potentially reflecting the lower ratio of available mentors to mentees in later matches. Nevertheless, perceived utility was preserved, suggesting no corresponding loss of program value. These results confirm the feasibility of a subspecialty-wide inter-institutional mentoring program.

A major challenge facing mentoring initiatives is how to evaluate overall impact (2, 6). Programs have sought to measure success by assessing self-reported benefits (4, 16, 17) or comparing participant outcomes to historical norms or to those of program non-participants (16, 18). These measures are limited by the subjective nature of the responses, by responder bias, and by selection bias arising through comparison of physicians who elect to enroll in such programs with those who choose otherwise (6).

To understand the utility of AMIGO, we collected participant self-reports, identifying specific gains in career development, scholarship, work-life balance and connectedness to the community. These findings support the overall design of the program, but do not provide an objective assessment of impact. Thus it is a unique strength of the AMIGO evaluation strategy that we also assessed the global state of mentoring within pediatric rheumatology before and after program implementation. Compared with pre-AMIGO data, fellows in the post-AMIGO era reported better access to mentors outside their home institutions (we term this breadth of mentoring) and in overall mentoring satisfaction. Modest trends in the same direction could be identified for junior faculty, potentially reflecting the much lower participation in this subgroup (31% as mentees vs. 86% among fellows). Interestingly, areas of improved breadth of mentoring corresponded closely with career domains in which AMIGO survey respondents also reported benefit (research and career development), lending credibility to participant self-reports. To our knowledge, these results represent the first evidence that a structured program can alter the landscape of mentoring within an entire medical discipline.

Implementation of AMIGO failed to reduce the proportion of fellows or junior faculty who reported having no mentor at all in specific domains. In part, this reflects a very high baseline of self-reported access to a mentor of some kind. For fellows, baseline access ranged from 70% to almost 100%, depending on domain (Figure 3). These percentages are within the very broad range of self-reported access to mentoring within medicine (2) and agree with a previous report of access of mentoring during rheumatology fellowship (19). Improvement above this very high background would have been challenging to achieve, in particular since participation in AMIGO was not ubiquitous. However, it is well recognized that having a network of mentors is advantageous, particularly where these mentors bring different perspectives (20). AMIGO mentors provide a view from outside of the mentee’s institution, a perspective that is particularly helpful since many pediatric rheumatology programs are highly enriched for faculty who trained locally. We are encouraged to observe that implementation of AMIGO correlated with a striking increase in breadth of mentoring for fellows, and we speculate that this enhancement contributed to the overall improvement in satisfaction with mentoring over this interval.

Participation in AMIGO was voluntary. Early career pediatric rheumatologists may have elected not to participate in AMIGO for many reasons. Some may have been too disconnected from the community to know about the program, or engaged in a career path (for example, full-time clinical practice) for which they perceived no need of external mentoring. Others may already have had satisfactory mentoring arrangements, or perceived themselves as too established to benefit from an AMIGO mentor. These factors likely explain why AMIGO participation as mentee was more penetrant among fellows than junior faculty, where the ranks of instructor and assistant professor encompass a great range of seniority. It is therefore not surprising that we could measure no concrete mentoring gains in the junior faculty subgroup. Further evaluation using qualitative methods may inform how to better serve this subgroup, especially in the critical early years after fellowship.

We recognize important methodological limitations to this study. AMIGO mentees who experienced less benefit from the program may have been less likely to complete the evaluation surveys. If so, then participant reports overestimate the benefits of AMIGO, though this caveat does not negate the advantages reported by those who did respond. The community surveys were cross-sectional assessments at two points in time, and implementation of AMIGO was only one factor varying over the intervening period. Response rates could not be measured accurately. Thus, it is difficult to assess the extent to which our data reflect the community as a whole. Despite these limitations, we are reassured that survey findings are comparable to those of a 2012 survey of pediatric rheumatology fellows and recent fellowship graduates that found 78% to have no mentor outside their home institutions (15). Further, participants in the 2011 and 2014 community surveys were demographically comparable, and measures not likely to be affected by AMIGO (such as availability of clinical mentors, or mentoring for senior faculty) remained largely stable. These internal controls suggest that, at a minimum, inter-survey comparisons remain valid.

A final limitation of our program evaluation strategy is that we made no attempt to measure concrete outcomes such as retention in research, success obtaining extramural funding, and rate of academic promotion. These important outcomes were considered impracticable to assess meaningfully, without a control group, in a cohort of the size available within North American pediatric rheumatology.

The extent to which the AMIGO model can be generalized to adult rheumatology, or to other medical disciplines, remains to be determined. The size of the pediatric rheumatology community was one of the reasons AMIGO was necessary, but it also helped to make AMIGO feasible. Strong support by ACR and CARRA leadership, within a collegial and well-integrated community, promoted rapid and enthusiastic embrace of the program. Early “buy in” by stakeholders such as division chiefs and fellowship directors facilitated mentee participation. The small size of the community allowed the AMIGO Steering Committee to tailor mentor-mentee matches knowing many of the participants, particularly at the mentor level. Similar personalization could be more difficult in a larger subspecialty, such as adult rheumatology.

A further important difference is that non-academic clinical practice is rare in pediatric rheumatology but common in adult rheumatology. One option may therefore been to target an AMIGO-like program to adult rheumatology fellows and junior faculty interested in academic careers. Such a program could contribute to fostering the pipeline of junior investigators required to secure a robust research base in rheumatology, as suggested by the 2012 ACR Blue Ribbon Panel on Academic Rheumatology and by the Early Career Investigator Subcommittee of the ACR Committee on Research (9, 21).

In summary, the AMIGO evaluation strategy confirms both the feasibility and the utility of a subspecialty-wide inter-institutional mentoring program in pediatric rheumatology. Process measures document successful expansion of AMIGO from the pilot phase through general enrollment. Surveys of the general community before and after implementation of AMIGO confirm objective community-wide gains that correlate closely to participant reports. Together, these data support the investment in time, effort and resources that went into developing AMIGO, and suggest that analogous programs could be developed in adult rheumatology and potentially in other medical subspecialties.

SIGNIFICANCE AND INNOVATION.

• The American College of Rheumatology/Childhood Arthritis Research and Rheumatology Alliance Mentoring Interest Group (AMIGO) is an inter-institutional mentoring program that includes most fellows and many junior faculty in US and Canadian pediatric rheumatology.

• AMIGO mentees report that participation benefits multiple aspects of career development.

• Surveys of the pediatric rheumatology community before and after AMIGO identify overall improvement in the mentoring environment for pediatric rheumatology fellows, representing the first objective outcomes data for any mentoring program in medicine.

Abbreviations:

ACR

American College of Rheumatology

AMIGO

ACR/CARRA Mentoring Interest Group

CARRA

Childhood Arthritis & Rheumatology Research Alliance