Implementing and Evaluating a Mentor Training to Improve Support for Early-Career Scholars in Tobacco Regulatory Science

Abstract

Introduction

To implement and evaluate a blended online and in-person training to help mentors of early-career researchers appreciate the complexities of Tobacco Regulatory Science (TRS), refine TRS mentoring skills, and become acquainted with resources for providing effective guidance to TRS mentees.

Methods

TRS mentors engaged in a two-part pilot test of the training program. Authors evaluated both the online and in-person training using retrospective pre-post evaluations, which measure learning at the conclusion of a training program, and post-program focus groups. Twenty learners completed the online training, and 16 learners attended the in-person training module. Nine participants completed evaluations for the online module, and 12 participants completed evaluations for the in-person module.

Results

Program assessments revealed that participants found that the training achieved its overall goals. The majority of respondents (87.5%) rated the online portion of the training as valuable. For the in-person training, participants reported statistically significant improvements regarding confidence in: helping mentees to identify skills and training to effectively pursue TRS, assisting mentees in weighing career trajectories, and guiding mentees in conducting research responsive to TRS regulatory priorities.

Conclusions

The novel mentoring program was well received by faculty seeking to strengthen skills for mentoring early-career TRS researchers to navigate the complex landscape of TRS, explore diverse funding opportunities, and discern potential career trajectories. It provided unique content to address issues outside the traditional tobacco research training curriculum and offered specific information on regulatory policies, priorities, and opportunities.

Implications

This research documents the deployment and evaluation of a blended online and in-person training program for investigators mentoring early-career researchers working in TRS. Our assessment discovered that participants found the training to be valuable to their overall mentoring objectives. The training comprises a novel curriculum for investigators engaged in mentoring early-career researchers in a unique field, thus filling a deficit in the published literature by presenting a curriculum that has been customized to the unique needs of TRS mentors.

Introduction

Tobacco Regulatory Science (TRS) is a nascent field, and recent years have witnessed an increasing need for training programs that specifically address the conduct of regulatory research as it pertains to both traditional and novel tobacco products,^1–3 thus echoing broader recognition regarding the necessity for educational curricula uniquely suited to regulatory science as a whole.⁴ In a 2017 study by Russo et al.,³ senior tobacco regulatory scientists serving as mentors were asked to comment on the challenges specific to TRS mentoring. Commonly cited themes included: ensuring that mentees understand the unique aspects of regulatory science and how to frame research questions to meet regulatory needs, equipping mentees with the broad training and skills that encompass TRS competencies, and helping mentees to identify their individual strengths.

The TRS mentor training described here embodies such a program and may be useful for mentors working across the second iteration of Tobacco Centers of Regulatory Science (TCORS) 2.0, which were awarded more than $151 million in federal grant funding between 2018 and 2022.^5,6 Like the previous TCORS 1.0, this network of nine federally funded TRS research centers maintains a strong emphasis on training and career enhancement.

The NIH Tobacco Regulatory Science Program and the FDA’s Centers of Excellence in Regulatory Science and Innovation (CERSIs) have made significant investments in training TRS scientists. Federal investment incorporates dedicated support for pre-doctoral, post-doctoral, and early-career faculty trainees, including issuance of K mechanisms in TRS. Mentoring also may play a significant role in equipping early-career scientists to conduct meaningful TRS. Indeed, evidence supports that structured, purposeful mentorship is critical for faculty success in academia.^7–9 Sambunjak et al.’s systematic review of mentoring in academic medicine identified that mentoring has an important effect on research productivity, including publication and grant success.¹⁰ Although existing educational program models,^11,12 curricula,^13–15 and other published resources¹⁶ share strategies and best practices for cultivating mentoring skills within academic health sciences as a whole, the relative newness of TRS translates to a dearth of TRS-specific mentoring curricula available in the published literature.

Whereas earlier work outlined core competencies in TRS,² many scientists working in the field come from basic or clinical research backgrounds and have received little formal training in TRS core competencies—not to mention the unique nuances of regulatory science. Thus, even well-established investigators may struggle to provide effective TRS-specific mentorship for early-career researchers (Figure 1).³ Following from the development of core competencies² and interviews with mentees and leaders in TRS,³ we designed and implemented a two-module mentoring program specifically tailored for TRS investigators serving as mentors to early-career researchers in the field. We evaluated the two-part mentor training pilot and presented the results here.

Figure 1.

Cultivating key realms of understanding for mentees in Tobacco Regulatory Science (TRS)

Methods

Study Participants

We offered the training to a pre-selected group of mid-career faculty investigators who fit the following criteria: (1) currently serving as mentors to early-career TRS researchers and (2) previously had committed to attending the annual TCORS 1.0 Grantees Meeting hosted at the NIH in October 2017. A total of 20 learners completed the online training. Sixteen individuals attended the in-person training module.

Study Design and Procedure

The TRS mentor training was a pilot program designed for faculty working with trainees and early-career scientists in the field. We chose a two-part educational model in order to provide time-effective training for faculty working at geographically disparate institutions, with the intention that the initial training component could be completed online and the second module deployed in-person at national TRS meetings.

As outlined in the Facilitator’s Guide (Supplementary File 1), we envisioned several educational objectives for our two-part program. We designed the first module as a refresher course to remind participants of the unique characteristics and research priorities of TRS. We distributed a link to participants by email for the <30 minutes online training the week before the in-person module was scheduled to occur. The online training included a video to re-acquaint mentors with the TRS curriculum domains and competencies,² a summary of common challenges in TRS mentoring, an introduction to effective TRS mentoring strategies, and an overview of TRS mentoring resources.

The second training module was a 1.5 hours in-person session. We describe the session agenda in the Facilitator’s Guide (Supplementary File 1). Through the interactive slide presentation (Supplementary File 2), small group case studies (Supplementary File 1), and large group debriefing session, we sought to equip learners to: foster mentee understanding of how regulatory science is unique, guide mentees in framing research questions that meet regulatory needs, prepare mentees with broad skills, and help mentees to identify their individual strengths.

Following the half-hour interactive presentation, participants were divided into three groups to discuss case studies presenting TRS-specific mentoring dilemmas. After a 30-minute discussion, participants reconvened as a large group for a debriefing discussion, during which representatives from each group reported on the conversations that had occurred at their tables. Finally, participants voiced their commitments to peers through SMART (Specific, Measurable, Achievable, Relevant, and Time-related) goals¹⁷ to practice in follow-up.

In advance of the training, we designed a multi-pronged evaluation strategy involving both quantitative assessments and qualitative evaluations and obtained the appropriate IRB approval from Boston University Medical Campus (Supplementary File 1). Following the in-person training module, we asked participants to complete retrospective pre-post assessments of knowledge and confidence gained through the online and in-person trainings. The retrospective pre-post assessments measured participants’ learning at the end of the training to gather information through self-assessment of competencies prior to and after the educational intervention.¹⁸

Immediately following the in-person training, we divided participants into two focus groups, which met in separate rooms for approximately 30 minutes. Members of the training team facilitated the focus groups. With the consent of all participants, we audio recorded the focus group sessions for accuracy.

Measures

The online training module evaluation included pre-assessment and post-assessment questionnaires. The pre-assessment questionnaire involved rating confidence on a scale of 1 (not at all confident) to 7 (extremely confidence) regarding participants’ ability to identify: TRS domains and competencies, how TRS is unique compared to other fields of academic research, and which research questions meet TRS regulatory needs and research priorities. The post-assessment questionnaire included the same three measures as were present on the pre-assessment questionnaire. However, the latter questionnaire also contained inquiries regarding participants’ thoughts on: the training time commitment; whether the training was valuable; and the extent to which the training adequately explained the knowledge, skills, and competencies it presented. Furthermore, the post-assessment questionnaire included an open-ended question asking for suggestions regarding changes or additions to the module.

The in-person training evaluation included measures related to competencies discussed at the in-person training. It asked participants to rate on a scale of 1 (not at all skilled) to 7 (extremely skilled) statements regarding how equipped participants felt to: help mentees identify and acquire specific TRS skills/training to effectively pursue TRS opportunities; assist mentees in weighing different career trajectories; and help mentees identify and acquire specific TRS skills/training to effectively pursue TRS opportunities. Additionally, the evaluation queried whether participants had completed the online module prior to the session, and two open-ended questions related to any changes in participants’ future mentoring practices and areas for improvement of the training.

The qualitative focus group guide focused on questions related to satisfaction and competencies gained through the in-person training. Specific questions pertained to various activities in training, including the online module, background presentation, case studies, and small group discussion.

Analysis

Descriptive statistics were used to summarize results. The paired t-test was used to assess the statistical significance of differences between the pretest and posttests. A p value of < .05 was used to reject the null hypothesis. The research team used an inductive approach to analyze focus group interview data and open-ended survey response data. Two team members read each transcript and coded for program strengths/weaknesses and areas for improvement using NVivo Software (QSR International Pty Ltd., Version 11, 2017).

Results

Quantitative Results

Nine out of 20 participants completed the quantitative survey on the online learning module. Most reported that the training was a “valuable use of their time” (87.5%) and “agreed” or “strongly agreed” that “the content of the online learning module adequately explained the knowledge, skills, and attitudes it presented” (66.7%). In a retrospective pre-post survey of knowledge and experience gained in the online training, respondents reported a statistically significant increase in confidence in three components: (1) helping mentees to identify and acquire specific TRS skills/training to effectively pursue TRS opportunities (1.0-point improvement); (2) assisting mentees in weighing different career trajectories (1.3-point improvement); and (3) guiding mentees to be responsive to TRS regulatory needs (0.6-point improvement) (Table 1).

Table 1.

Retrospective Pre–Post Survey of Knowledge and Experience Gained in TRS Mentor Training

Question	Prior to training	After training	Statistical significance
Online module (N = 9)
I am able to identify TRS domains and competencies	4.7	5.8	t = −3.2, p = .013
I am able to identify how TRS is unique compared to other fields of academic research	4.9	6.3	t = −3.8; p = .005
I am able to identify which research questions meet TRS regulatory needs and research priorities	5.3	6.1	t = −2.4; p = .043
In-person module (N = 12)
Help mentee identify and acquire specific TRS skills/training to effectively pursue TRS opportunities	4.5	5.5	t = −5.1, p < .001
Assisting mentees in weighing different career trajectories	4.2	5.5	t = −10.1; p < .001
Ability to guide mentees to be responsive to TRS regulatory needs	5.0	5.6	t = −3.9; p = .002

TRS = Tobacco Regulatory Science. Scale for online module questions is 1 = not at all confident; 4 = moderately confident; 7 = extremely confident. Scale for the in-person module questions is 1 = not at all skilled; 4 = moderately skilled; 7 = extremely skilled.

Twelve out of 16 participants completed the survey of the in-person case-based training. In a retrospective pre-post survey of knowledge and experience gained in the training, respondents reported a statistically significant improvement in confidence in being able to help mentees identify TRS skills (1.1-point improvement), assist mentees in weighing different career trajectories (1.3-point improvement), and guide mentees to be responsive to TRS regulatory needs and research priorities (0.8-point improvement) (Table 1).

Qualitative Results

In response to the open-ended survey question regarding what they liked best about the training, respondents reported enjoying the opportunity to share experiences, learn from one another, and think about topics they do not often consider. Respondents noted that they would focus on: not creating “mini-me’s,” considering the goals of the mentee, discussing multiple options, and increasing communication with mentees.

Focus group participants reported that the online module was a good overview of the basics of TRS, but some individuals expressed a desire for more content about general mentoring. When asked about the in-person training, respondents found the training helpful for thinking about mentees’ needs and trajectories. Facilitating open lines of communication and mapping mentees’ career trajectories and mentoring needs was especially helpful. Case studies also were described as being eye-opening in regards to industry-related dilemmas that TRS trainees might face.

Respondents discussed their desire to guide mentees in conceptualizing the diversity of funding and career paths available beyond academia and government agencies. They were eager to help mentees connect with senior researchers whose career trajectories aligned with mentees’ professional goals and who could share strategies and best practices for navigating the complexities of industry interactions.

Focus group participants mentioned that it was enjoyable to discuss mentoring issues with other mentors. They suggested the creation of additional resources for TRS mentors and mentees. A network could offer, according to one participant, “more information about certain areas so we can best serve our mentees, one of [these areas] being job opportunities that are not traditionally academic.”

Respondents expressed a desire for more variety and less focus on cases specifically related to the tobacco industry. They suggested taking time for mentors to reflect on each case, and whether they had experienced similar scenarios, in order to include these reflections in the discussion. Other respondents wanted more concrete tools as “takeaways” to inform future interactions with mentees.

Discussion

A specific understanding of regulatory policy and law and the ability to coach mentees in analytical regulatory thinking are essential to effective TRS mentorship, which necessitates unique education beyond traditional tobacco control research training. Effective mentoring in TRS requires broad skills, including the ability to facilitate opportunities for mentees’ professional growth. Providing mentees with training in core TRS competencies² is imperative to enabling their career growth.

After implementing our mentor training, we found that providing a brief refresher of online material prior to an in-person module was useful. Focus group discussions revealed that general resources focused on more generic mentoring skills were helpful to participants as well. We also learned that it may be more effective to distribute the evaluation for the online module immediately following implementation, since participants may have found it confusing to complete evaluations for both modules simultaneously.

Limitations

The pilot training was only offered to mid-career mentors by email and word of mouth. Due to the two-part nature of the curricula, some learners only participated in the online or in-person modules, thus diminishing their ability to benefit from the experience of completing the entire curriculum. Additionally, the relatively small size of the pilot training is noted as limiting analyses of the evaluation data.

The training was tailored for a specific TRS audience and therefore comprised a unique curriculum, which presents a limitation to broad applicability. However, the format and complexities specific to mentoring in TRS may be applied to other FDA-regulated industry sectors, including food, drugs, medical devices, and cosmetics. Finally, the training focused solely on United States research mentors and corresponding mentoring needs related to the conduct of TRS in federal, academic, and industry contexts. The curriculum, therefore, requires customization for successful implementation in international settings.

Future Directions

Currently, evidence is emerging related to the potential harm of new tobacco and nicotine products on the market. As the literature expands, the FDA’s priorities continue to undergo corresponding shifts. TRS mentors require educational resources that can provide factual and up-to-date information that might be used to provide sound guidance to their mentees. In order to meet the need for more resources related to TRS mentoring, we are interested in building a virtual, open-access “TRS Mentoring Toolkit” for both mentors and mentees in the field. Such a toolkit might track user analytics and satisfaction with the goal of employing these data to formulate recommendations for the FDA regarding further utilization of the mentoring resource. As a first step towards creating the toolkit, we have compiled a list of references for mentors in TRS (Supplementary File 1).

Conclusions

The TRS mentor training comprises a novel curriculum for investigators engaged in mentoring early-career researchers in a unique field. It fills a deficit in the published literature by presenting educational materials that have been customized to the specific needs of mentors supporting TRS researchers. Positive evaluations of the pilot deployment indicate success in building mentors’ skills and confidence. The training also shows promise for broader use, as it may be adapted to suit the needs of mentors working in FDA-regulated industry sectors beyond tobacco.

Funding

This research was supported by the National Heart, Lung, And Blood Institute of the NIH and the Food and Drug Administration (FDA) Center for Tobacco Products (CTP) under Award Numbers P50HL120163 and U54HL120163. Additionally, the project was supported, in part, by the Tobacco Centers for Regulatory Science (TCORS) Center for Evaluation and Coordination of Training and Research (CECTR), Award Number U54CA189222 from the National Cancer Institute (NCI) and the FDA. ACV reports support from NIH Award Numbers P50DA036114 and U54DA036114. CDD and EJB report funding from American Heart Association (AHA) Award Number 18SFRN34110082. EJB reports support from NIH/NHLBI Award Numbers R01 HL128914, R01 HL092577, and R01 HL126136, as well as from the Robert Wood Johnson Foundation (“Studying mHealth technologies to help people improve their health and share their health information in real time with health care providers”). The content is solely the responsibility of the authors and does not necessarily represent the official views of the FDA, NCI, or NIH.

Declaration of Interests

None declared.