Long-Term Evaluation of a Train-the-Trainer Workshop for Pharmacy Faculty Using the RE-AIM Framework

Abstract

Background.

Although two thirds of tobacco users express interest in quitting, few pharmacists address tobacco use as part of routine practice. Historically, pharmacy schools provided inadequate tobacco cessation training for students. To address this educational gap, train-the-trainer workshops were conducted between 2003 and 2005 to train pharmacy faculty (n=191) to teach a shared, national tobacco cessation curriculum at their academic institutions.

Objective.

To characterize faculty perceptions of the train-the-trainer workshops and estimate the long-term reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) of the shared curriculum at pharmacy schools.

Methods.

This study is the second phase of a sequential mixed methods study. Results from Phase 1, a qualitative study, informed the development of survey items for Phase 2. Applying the RE-AIM framework, a web-based survey was developed and administered to train-the-trainer participants.

Results.

Of 191 trainees, 137 were locatable; of these, 111 completed a survey (81.0%). Most (n=87; 78.4%) reported current employment in academia. The most highly rated reason for attending a workshop was to improve teaching of tobacco cessation content, and 98.1% reported moderate or high confidence for teaching tobacco cessation. Among those who practice in a clinical setting, 70.6% reported asking their patients about tobacco use all or almost all the time. Just over three fourths of faculty respondents who work in academia believe that shared curricula should be more broadly considered for use in pharmacy schools, and 79% agreed that shared curricula are a cost-effective approach to teaching.

Conclusion.

Evidence is provided for long-term reach, effectiveness, adoption, implementation, and maintenance of the Rx for Change program. Participants perceived that the workshop resulted in long-term, positive effects on their careers as well as their teaching and clinical practice.

INTRODUCTION

Although the prevalence of tobacco use has decreased significantly over the past 50 years, in 2018 the U.S. Centers for Disease Control and Prevention reported that 13.7% of U.S. adults currently smoked cigarettes either every day or some days.¹ While two thirds of smokers are interested in quitting, and a little over half report having attempted to quit in the past year, fewer than 10% are successful.² This is not surprising, given that proven methods for quitting are markedly underutilized—it is estimated that only 19.6% use a medication, 3.8% attend a cessation class or a program, 2.7% receive one-on-one counseling, and 2.6% call the telephone quitline.³ Furthermore, although advice from health professionals to quit smoking has increased since 2000, 42.8% of adult cigarette smokers who saw a health professional during the past year reported not receiving advice to quit.² This is unfortunate, because assistance from a health professional at least doubles the odds of successfully quitting, and quit rates are increased even further if a medication is used as part of the quitting plan.^4,5 As a result, it is important for health professionals to ask all patients about tobacco use and, at a minimum, strongly advise tobacco users to quit and use evidence-based strategies as part of their quitting plan.^4,5

To prepare health professionals for this important responsibility, a shared tobacco cessation curriculum, Rx for Change: Clinician-Assisted Tobacco Cessation (https://rxforchange.ucsf.edu),⁶ was developed in 1999⁷ and has been disseminated widely for more than two decades. Originally designed for pharmacy students, but over time adapted for other health professionals (students and licensed practitioners), the program adheres to the principles set forth in the U.S. Public Health Service Clinical Practice Guideline for Treating Tobacco Use and Dependence,⁴ and serves as a vehicle for nationwide dissemination of guideline principles. As of March 2019, 15,576 users representing all 50 US states and 94 countries globally were registered users on the Rx for Change web-site.⁸ Based on early success of the program within pharmacy schools in California,^7,9 and as a direct result of an identified need to expand the tobacco cessation expertise of pharmacists in general,¹⁰ a plan was set in motion to disseminate the Rx for Change program to schools of pharmacy across the US. To prepare pharmacy faculty members to successfully implement the curricular content at their respective institutions, funding from the National Cancer Institute supported five 2.5-day train-the-trainer workshops in 2003 (n=3), 2004 (n=1), and 2005 (n=1).¹¹ A total of 191 faculty members participated in a workshop, representing 89 of 91 existing schools of pharmacy at the time (98%). High levels of anticipated adoption of the curriculum were reported immediately following the training—68.3% reported a high likelihood of implementing Rx for Change in the upcoming year,¹¹ and in 2016 (more than a decade later) a national survey that was commissioned by the Centers for Disease Control and Prevention conducted by the American Association of Colleges of Pharmacy estimated that 73.5% of 135 pharmacy schools nationwide were integrating Rx for Change materials into their Doctor of Pharmacy curriculum.¹²

Assessing both the short- and long-term impact of a faculty development program after the implementation is important, yet very few program developers do so.¹³ Given the unique nature of the Rx for Change shared curriculum, its associated longevity of use, and the programmatic use of federal funds to support its dissemination, an evaluation of the long-term impact of the train-the-trainer approach to dissemination is of scientific interest. To complement data reported by the American Association of Colleges of Pharmacy¹² and to assess the long-term impact of the train-the-trainer program, in 2019-2020 a web-based survey was administered to all pharmacy faculty members who attended an Rx for Change train-the-trainer workshop. Applying the RE-AIM framework,¹⁴ the survey was designed to characterize the reach, effectiveness, adoption, implementation, and maintenance of use of the Rx for Change curriculum, as well as the impact of the program on the faculty attendees’ tobacco-related activities in the realms of teaching, practice, and research.

METHODS

Design and conceptual framework

This study is the second phase of a two-phase mixed methods, sequential exploratory research project.

In phase 1, a randomly-selected subset of 18 pharmacy faculty trainees were interviewed by telephone to explore their perceptions of the Rx for Change program, the train-the-trainer workshop that they attended, and their subsequent experiences with program implementation. These qualitative data,¹⁵ informed the development of this phase 2 quantitative survey for administration to the entire cohort of pharmacy faculty members who attended a train-the-trainer workshop.

The survey builds upon phase 1 findings and applied the RE-AIM framework¹⁴ to estimate the impact of the train-the-trainer workshops with respect to its: (a) Reach to pharmacy schools across the United States, (b) Effectiveness on faculty confidence, their students’ confidence, and tobacco cessation-related practices, (c) Adoption of the curriculum as a resource for teaching tobacco cessation in pharmacy schools, (d) Implementation of the curriculum in pharmacy schools and challenges faced during implementation, and (e) Maintenance of the adoption of the curriculum in the long-term.

Study participants

Pharmacy faculty members who attended an Rx for Change train-the-trainer workshop in 2003, 2004, or 2005 were targeted for completion of the study survey. Because 15 years had elapsed between the train-the-trainer programs and administration of the survey in 2019, extensive internet searches were required to locate individuals. This included searching web-pages of their initial academic institutions (at the time of the train-the-trainer workshops), use of broader internet search engines such as Google, and accessing the membership list of professional associations (i.e., American Association of Colleges of Pharmacy, American College of Clinical Pharmacy, American Pharmacists Association). When e-mails bounced back as undeliverable, and no further information was found on the internet, a contact attempt was made through LinkedIn. If no response was obtained, or if no LinkedIn account was identified, the individual was then classified as unreachable.

Of 191 original faculty members,¹¹ two participants were excluded because they were involved in the development of the Rx for Change program, and three were deceased (Figure 1). Of the remaining 186 faculty members, what was perceived by the team to be a valid e-mail address was identified for 137 (73.7%). For 49 potential participants (26.3%), an active e-mail address could not be identified (i.e., all contact attempts were unsuccessful).

Figure 1.

Study population and participation flow chart.

Study measures

Survey items were developed to represent important themes identified in the phase 1 qualitative research study,¹⁵ and these items were mapped to the RE-AIM elements.¹⁴ Some measures were selected from a survey that was used previously to evaluate the Rx for Change train-the-trainer workshops.¹¹ Items and response options are described below; because of the retrospective nature of the study, for most items a “not applicable” or “do not recall” option was included, and these responses were removed from the denominator, as appropriate. To provide evidence for content validity, an initial draft of the survey was created by the primary author and was iteratively revised and reviewed by all authors. The final web-based survey was then pilot tested with three faculty members, who completed the survey prior to administering to the remainder of the study population. Upon request, the complete survey is available from the corresponding author.

Demographics.

Participants were asked to indicate their current career status: employment in a pharmacy school, non-pharmacy school, practice site where care is provided to patients, retired (not providing care to patients), or other position (not providing care to patients). These selections were not mutually exclusive. Additionally, if in academia, respondents provided their academic rank and whether they currently held an administrative position.

Reach.

All participants were asked to rate the importance of eight reasons, derived from our prior qualitative study,¹⁵ that potentially influenced their decision to attend an Rx for Change train-the-trainer workshop (1=not at all important, 2=a little important, 3=moderately important, 4=very important, 5=extremely important). Participants working in academia also were asked whether they currently teach smoking cessation content at their institution and to indicate the extent to which the Rx for Change curriculum is used at their institution (all of it or almost all of it, most of it, some of it, or none).

Effectiveness.

Survey items assessed the perceived degree to which participation in the Rx for Change workshop impacted their career (not at all, a little, moderately, very, or extremely impactful). Participants’ current confidence (none, low, moderate, or high) was assessed for teaching tobacco cessation content, precepting Introductory/Advanced Pharmacy Practice Experience (IPPE/APPE) students for tobacco cessation activities, and providing tobacco cessation counseling to patients. Faculty were also asked to rate their perception of the extent to which receiving Rx for Change education (as part of coursework) impacted their students’ competency and confidence for providing tobacco cessation counseling and their students’ readiness to apply their knowledge in practice (not at all, a little, moderately, very, or extremely impactful). Those currently working in a clinical setting reported how often they ask their patients about tobacco use and which approaches are used when assisting patients with quitting.

Adoption.

Using a 4-point scale (1=none, 2=low, 3=moderate, 4=high), participants rated five characteristics of the Rx for Change program, which are described by Everett Rogers’ Diffusion of Innovations Theory to be associated with adoption of new programs:¹⁶ comprehensiveness of content, appropriateness of teaching methodologies used, simplicity of implementing Rx for Change, compatibility for integration into existing curriculum structures, and relative advantage over other tobacco cessation content that is available elsewhere or developed internally. Additionally, participants were asked to identify tobacco-related enhancement activities (e.g., new initiatives) with which they had been personally involved since their workshop attendance and whether they were interested in receiving information regarding access of newly-developed tobacco-specific virtual patients and standardized patient/Objective Structured Clinical Examination (OSCE) cases.

Usefulness of the Rx for Change website for supporting teaching of tobacco cessation was rated as not at all, a little, moderately, very, or extremely useful. Finally, participants were asked whether they perceive shared curricula, in general, to be a cost-effective approach to teaching, whether shared curricula should be more broadly considered for use in pharmacy schools, and whether they had advised other pharmacy faculty members and/or non-pharmacy faculty members to consider adopting Rx for Change at their institution.

Implementation.

Participants identified approaches that they have used for teaching tobacco cessation and rated challenging aspects associated with implementing (or attempting to implement) the Rx for Change program (not at all, a little, moderately, very, or extremely challenging). For one of the challenges, i.e., limited time in the curriculum, participants who selected a little, moderately, very, or extremely challenging were asked to indicate how they overcame this challenge. Participants also indicated the number of hours of tobacco cessation currently integrated into their institutions’ required Doctor of Pharmacy curriculum (1 to <4 hours, 4 to <6 hours, 6 to <8 hours, 8 to <10 hours, 10 or more hours).

Maintenance.

For participants currently working in academia, the likelihood that Rx for Change content would be used to teach tobacco cessation during the next academic year was assessed (not at all, a little, moderately, very likely, or extremely likely). As a proxy measure for ongoing maintenance of implementation of the Rx for Change curriculum, the survey assessed the frequency by which participants log into the Rx for change website (never, less than once a year, about once a year, about 2 to 10 times a year, more than 10 times a year, I have used the website in the past, but no longer do because I do not currently teach Rx for Change).

To enhance further dissemination of Rx for Change to schools of pharmacy, respondents were asked to rate their perceptions of the effectiveness of four strategies revealed by phase 1 interviewees: (a) provide enduring on-demand web-based train-the-trainer programs that can be accessed at any time, (b) conduct a 1-day session before an American Association of Colleges of Pharmacy (AACP) or other professional meeting, (c) conduct more live, on-site train-the-trainer workshops, similar to the San Francisco workshops, and (d) conduct “live” web-based trainings or webinars. Response options were 1=not at all, 2=a little, 3=moderately, 4=very, and 5=extremely effective.

Further advancement of the role of pharmacy in tobacco cessation.

Respondents rated their perceived importance of seven potential actions for advancing the role of pharmacy in tobacco cessation (1=not at all, 2=a little, 3=moderately, 4=very, 5=extremely). Specific actions assessed were: (a) include tobacco content in the core curriculum of all pharmacy schools, (b) include tobacco-related questions on state board licensing examinations, (c) have students apply tobacco cessation counseling skills during IPPE/APPE rotations, (d) provide a web-based “booster” training for students to complete, prior to APPE, (e) provide a train-the-trainer program for faculty with free CE (live or online), (f) partner with State Departments of Health, and (g) partner with tobacco quitlines.

Survey administration and analysis

Web-based surveys were administered using Qualtrics. An introductory e-mail described the purpose of the research and provided a consent document and link to the survey. Two reminder notices were sent to non-responders. Because university servers commonly filter surveys that are distributed via Qualtrics, a final contact was made from the investigators’ (KH or RC) e-mail address. The full survey required approximately 15 minutes to complete in its entirety, although skip patterns were embedded that rendered a briefer survey for participants who no longer work in academia. A $20 Amazon.com gift card was provided to study participants.

Data were analyzed using SPSS software, version 26.¹⁷ Descriptive statistics were computed to characterize the study population and their survey responses. Unless otherwise indicated, denominators for computed percentages included only those individuals for whom the question was displayed. Approval to conduct the study was obtained from the Purdue University Human Research Protection Program.

RESULTS

Study participants

Of 137 faculty members for whom a viable e-mail address was identified, 111 (81.0%) completed the survey (59.7% of the eligible cohort; Figure 1). These respondents represented 75 (84.3%) of the 89 schools or colleges of pharmacy that participated in a train-the-trainer workshop.¹¹ Of the 111 respondents, 87 (78.4%) reported a current employment position in academia (of whom 27 were clinical faculty), and 34 (30.6%) currently practiced in a clinical setting (responses not mutually exclusive). Others had either retired (n=7; 6.3%) or work in a non-academic, non-clinical setting (n=10; 9.0%). Within academia, 41 respondents were full professors, 25 were associate professors, and 5 were assistant professors. Fourteen were the Chair or Head of a department, 20 were an Assistant or Associate Dean, and 2 were a Dean.

Reach

Among all 111 respondents, the most important reason for attending an Rx for Change train-the-trainer workshop was to improve teaching of tobacco cessation content, which was rated as very or extremely important by 86.2% (Table 1), and 32.2% of respondents reported currently teaching tobacco cessation (n=28 of 87 in academia). Most respondents’ academic institutions were either utilizing some (n=32 of 65; 49.2%) or most/all (n=26; 40.0%) of the Rx for Change materials for tobacco cessation curricular content (22 reported not knowing this information).

Table 1.

Reasons influencing faculty members’ decision to attend an Rx for Change workshop between 2003 and 2005 (n=111).^a

Reasons (respondents)	Ratingb [n, (%)]
	Not at all important (1)	A little important (2)	Moderately important (3)	Very important (4)	Extremely important (5)	Average rating
To improve my teaching for tobacco cessation (n=109)	1 (0.9)	4 (3.7)	10 (9.2)	35 (32.1)	59 (54.1)	4.3
To improve my skills for treating tobacco use and dependence (n=109)	3 (2.8)	8 (7.3)	13 (11.9)	27 (24.8)	58 (53.2)	4.2
To improve the tobacco content in our curriculum (n=108)	2 (1.9)	4 (3.7)	17 (15.7)	31 (28.7)	54 (50.0)	4.2
To be a part of this national training initiative (n=110)	5 (4.5)	16 (14.5)	26 (23.6)	28 (25.5)	35 (31.8)	3.7
An opportunity to meet colleagues with similar interests (n=106)	10 (9.4)	18 (17.0)	22 (20.8)	34 (32.1)	22 (20.8)	3.4
It was encouraged by a mentor or colleague (n=99)	20 (20.2)	9 (9.1)	23 (23.2)	26 (26.3)	21 (21.2)	3.2
It was required or encouraged by my university administration (n=105)	32 (30.5)	16 (15.2)	13 (12.4)	25 (23.8)	19 (18.1)	2.8
An opportunity to travel to San Francisco at no cost (n=109)	45 (41.3)	30 (27.5)	15 (13.8)	11 (10.1)	8 (7.3)	2.1

^a“I do not recall” and missing responses were removed from the denominator; < 6% for all items, except “It was encouraged by a mentor or colleague,” which was 10.8%.

^bItem wording: “How important were each of the following in your decision to attend an Rx for Change train-the-trainer workshop between 2003 and 2005?”

Effectiveness

When asked to rate the extent to which attending the train-the-trainer workshop impacted participants’ career, 12.6% reported it was extremely impactful, 32.4% very impactful, 33.3% moderately impactful, 18.9% a little impactful, and 2.7% not at all impactful. Nearly all participants (98.1%) reported a moderate or high level of confidence for teaching tobacco cessation, and 96.8% reported moderate or high confidence for precepting IPPE/APPE students for tobacco cessation activities. Ninety-seven percent reported moderate or high confidence for providing tobacco cessation counseling to patients. Most respondents perceived the Rx for Change training to be either very or extremely impactful on their students’ competency (81.3%) and confidence (73.6%) for tobacco cessation counseling, and for their readiness to apply their knowledge in practice (78.1%).

Among participants who were currently providing patient care (n=34), most reported asking their patients about tobacco use all or almost all the time (70.6%) or at least half of the time (5.9%). Most indicated that they apply motivational interviewing techniques when discussing tobacco cessation with patients (76.5%), integrate brief counseling into practice (Ask-Advise-Refer; 70.6%), integrate comprehensive counseling using the 5 A’s (Ask, Advise, Assess, Assist, Arrange; 67.6%), and address the 5 R’s for those not ready to quit (Relevance, Risks, Rewards, Roadblocks, Repetition; 61.8%). Half of the respondents provide tobacco quitline cards/telephone number (50.0%), and 41.2% check for potential smoking-drug interactions when filling prescriptions.

Adoption

Respondents’ ratings for characteristics of the Rx for Change program are reported in Table 2 (n=111); all four characteristics were rated at least 3.6 on a 4-point scale (1 to 4), with comprehensiveness of the content rated highest. As shown in the upper half of Table 3, respondents have been engaged in a variety of tobacco-related enhancement activities since their participation in a train-the-trainer workshop. Among participants who work in academia, 67 (77.9%) said they were interested in learning more about newly-developed tobacco-specific virtual patients and standardized patient/OSCE cases.

Table 2.

Faculty members’ ratings of characteristics of the Rx for Change curriculum, derived from Rogers’ Diffusion of Innovations Theory¹⁶ (n=111).^a

Characteristics (respondents)	Ratingb [n, (%)]
	None (1)	Low (2)	Moderate (3)	High (4)	Average rating
Comprehensiveness of content (n=111)	0 (0.0)	0 (0.0)	11 (9.9)	100 (90.1)	3.9
Appropriateness of teaching methodologies used (n=109)	0 (0.0)	3 (2.8)	18 (16.5)	88 (80.7)	3.8
Simplicity of implementing Rx for Change (n=108)	0 (0.0)	3 (2.8)	25 (23.1)	80 (74.1)	3.7
Compatibility for integration into your existing curriculum structure (n=104)	0 (0.0)	5 (4.8)	25 (24.0)	74 (71.2)	3.7
Relative advantage over other tobacco cessation content that is available elsewhere or developed internally at your school of pharmacy (n=92)	3 (3.3)	3 (3.3)	21 (22.8)	65 (70.7)	3.6

^a“I do not recall” and missing responses were removed from the denominator; < 7%, with the exception of “relative advantage” item, which was 17.1%.

^bItem wording: “Please rate each of the following characteristics of the Rx for Change curriculum.”

Table 3.

Tobacco-related activities and teaching approaches (n=111).

Tobacco-related enhancement activitiesa	n (%)
Increased the number of hours of tobacco cessation content that pharmacy students receive in the core curriculum	71 (64.0)
Added a new tobacco-related skills/practice laboratory activity	65 (58.6)
Conducted a research project related to tobacco	27 (24.3)
Implemented standardized patients for students to counsel for practice (i.e., not high stakes)	22 (19.8)
Implemented a tobacco-specific objective structured clinical examination (OSCE) to formally evaluate students	13 (11.7)
Developed a tobacco cessation elective	11 (9.9)
Developed an inter-professional activity focused on tobacco	7 (6.3)
Tobacco cessation teaching approachesb (inside and outside of classroom)	n (%)
Taught tobacco lectures in the classroom	96 (86.5)
Taught pharmacy practice laboratories/workshops for students (e.g. role playing with case studies, hands on use of medications for cessation)	84 (75.7)
Served as an IPPE or APPE preceptor for students in a clinical setting where patients receive tobacco cessation counseling	73 (65.8)
Taught continuing education programs	40 (36.0)
Facilitated group tobacco cessation programs for patients	27 (24.3)
Created web-based lectures/podcasts for students to view prior to classroom instruction (e.g., flipped classroom technique)	9 (8.1)

^aItem wording: Since participation in an Rx for Change train-the-trainer workshop (in 2003-2005), with which of the following have you been involved? (Select all that apply)

^bItem wording: In your history of teaching tobacco cessation (in any institution where you have worked), which of the following approaches have you used? (Select all that apply)

Abbreviations: OSCE, Objective Structured Clinical Exam; IPPE, Introductory Pharmacy Practice Experiences; APPE, Advanced Pharmacy Practice Experiences.

The majority of respondents rated the usefulness of the Rx for Change website for supporting teaching of tobacco cessation as either very (40.5%) or extremely useful (49.4%). Most (79%) of those who work in academia believe that shared curricula are a cost-effective approach to teaching, 17.4% were neutral, and 3.5% disagreed; 77.9% agreed that shared curricula should be more broadly considered for use in pharmacy schools. Since participating in the workshop, 73.0% of 111 participants had advised other pharmacy faculty members to consider adopting Rx for Change at their institution, and 26.1% had advised non-pharmacy faculty at other health professional schools.

Implementation

When participants implemented the curricular materials, the most common approach for teaching was lecture format (86.5%), followed by pharmacy practice laboratories/workshops for students (75.7%) (bottom half of Table 3). Limited time in the curriculum was the most challenging aspect associated with implementation, with faculty rating it as extremely (14.0%), very (26.2%), moderately (28.0%), a little (22.4), or not at all challenging (9.3%). Of those who perceived limited time in the curriculum as being a challenge (n=97), 76.3% reported prioritizing specific content to fit the number of hours that were allowed. Other methods were also used to address the limited time in the curriculum: assigned content/materials for students to read or review outside of class (26.8%), gradually increased time dedicated to tobacco in the curriculum over the years (21.6%), asked the curriculum committee to allow for additional curricular time (12.4%), and developed an elective course for tobacco cessation to cover more material (12.4%); options were not mutually exclusive. At the time of survey, 24 of those who worked in academia were uncertain how many hours of tobacco cessation content were currently being taught; among others, 77.8% of respondents’ institutions taught less than six hours of tobacco cessation content in their Doctor of Pharmacy core curriculum; 22.2% taught the recommended minimum of six hours.¹⁸

Maintenance and further dissemination of the Rx for Change program

Among respondents with current employment in academia, 61.3% indicated it is very or extremely likely that the Rx for Change curriculum will be used to teach tobacco cessation during the next academic year at their institution (2019-2020; n=24 who reported “I do not know” were excluded from the analysis). Of all respondents, 32.4% reported accessing the website annually, and of those who teach tobacco cessation (n=28), 78.8% access the website at least annually. Respondents’ perceptions of the effectiveness of four strategies to further disseminate the Rx for Change program are shown in Table 4.

Table 4.

Faculty members’ perceptions of strategies to further disseminate the Rx for Change program to colleges/schools of pharmacy (n=111).^a

Strategy (respondents)	Ratingb [n, (%)]
	Not at all effective (1)	A little effective (2)	Moderately effective (3)	Very effective (4)	Extremely effective (5)	Average rating
Provide enduring on-demand web-based train-the-trainer programs that can be accessed at any time (n=110)	1 (0.9)	10 (9.1)	18 (16.4)	45 (40.9)	36 (32.7)	4.0
Conduct a 1-day session before an AACP or other professional meeting (n=110)	3 (2.7)	8 (7.3)	19 (17.3)	46 (41.8)	34 (30.9)	3.9
Conduct more live, on-site train-the-trainer workshops, similar to the San Francisco workshops (n=110)	2 (1.8)	8 (7.3)	26 (23.6)	47 (42.7)	27 (24.5)	3.8
Conduct “live” web-based trainings or webinars (n=109)	0 (0.0)	9 (8.3)	29 (26.6)	47 (43.1)	24 (22.0)	3.8

^a“No opinion” responses were removed from the denominator (<2%).

^bItem wording: “In your opinion, how effective would the following strategies be in further disseminating Rx for Change to the newer colleges/schools of pharmacy?”

Abbreviation: AACP, American Association of Colleges of Pharmacy.

Further advancement of the role of pharmacy in tobacco cessation.

Table 5 provides faculty perceptions of strategies for further advancing the role of pharmacy in tobacco cessation. The three most highly rated strategies were (1) including tobacco content in the core curriculum of all pharmacy schools, (2) having students apply tobacco cessation counseling skills as part of experiential education, and (3) including tobacco-related questions on the pharmacy licensure examination.

Table 5.

Faculty members’ perceptions of potential methods for further advancing the role of pharmacy in tobacco cessation (n=111).^a

Methods (respondents)	Ratingb [n, (%)]
	Not at all important (1)	A little important (2)	Moderately important (3)	Very important (4)	Extremely important (5)	Average rating
Include tobacco content in the core curriculum of all pharmacy schools (n=110)	0 (0.0)	2 (1.8)	2 (1.8)	25 (22.7)	81 (73.6)	4.7
Have students apply tobacco cessation counseling skills during IPPE/APPE rotations (n=111)	0 (0.0)	1 (0.9)	3 (2.7)	44 (39.6)	63 (56.8)	4.5
Include tobacco-related questions on NAPLEX (n=106)	2 (1.9)	2 (1.9)	13 (12.3)	31 (29.2)	58 (54.7)	4.3
Partner with State Departments of Health (n=107)	0 (0.0)	5 (4.7)	13 (12.1)	41 (38.3)	48 (44.9)	4.2
Partner with tobacco quitlines (n=103)	0 (0.0)	5 (4.9)	14 (13.6)	36 (35.0)	48 (46.6)	4.2
Provide a train-the-trainer program for faculty with free CE (live or online) (n=110)	1 (0.9)	5 (4.5)	19 (17.3)	45 (40.9)	40 (36.4)	4.1
Provide a web-based “booster” training for students to complete, prior to APPEs (n=111)	3 (2.7)	14 (12.6)	33 (29.7)	31 (27.9)	30 (27.0)	3.6

^a“No opinion” responses were removed from the denominator; < 5% for all items, except “partner with tobacco quitlines,” which was 7.2%.

^bItem wording: “Please rate how important the following are for advancing the role of pharmacy in tobacco cessation.”

Abbreviations: NAPLEX, North American Pharmacist Licensure Examination; IPPE, Introductory Pharmacy Practice Experiences; APPE, Advanced Pharmacy Practice Experiences.

DISCUSSION

The methods used in this study were consistent with recommendations^19,20 for evaluating faculty training workshops: (1) highlight application, (2) apply theory, (3) use mixed methods, and (4) conduct long-term evaluation. In regards to application, the Rx for Change workshops aimed to equip faculty participants with the necessary knowledge and skills to teach comprehensive tobacco cessation and empower them to champion the adoption of the Rx for Change curricular materials at their academic institutions. Additionally, the survey results provided evidence for enhanced faculty performance in teaching, clinical practice, and research. In regards to applying theory, a theoretical framework was applied¹⁴ in tandem with the phase 1 qualitative results. In regards to using mixed methods, during phase 1 of our research,¹⁵ Kirkpatrick’s four-level model²¹ was applied as a framework to assess faculty members’ reaction, learning, behavior, and outcomes that resulted from attending the train-the-trainer workshop. Results from the qualitative phase 1 informed development of the survey instrument to be administered in this phase 2 study. Through further quantification, this national survey capably characterized the impact of the workshops on implementation and sustainability. The mixed methods approach that was applied across the two phases is an appropriate methodology, given the relative lack of existing evidence to guide phase 2 survey development, and because it provided multiple sources of evidence.²² Finally, the study provided a long-term evaluation (15 years after the workshop training), which can be challenging, yet useful and essential to the enhancement of health professional education.^19,20 Our study adds valuable knowledge to the literature, particularly given that the pharmacy literature currently lacks long-term studies evaluating faculty development programs. Most studies evaluating the impact of workshops are conducted shortly after the training—as an example, a workshop training provided to pharmacy faculty aimed to enhance trainees’ ability to implement and teach cultural competency in their pharmacy schools was evaluated 9 months later.²³ Another program that trained pharmacy faculty to implement a pharmacogenomics curriculum was evaluated after the training,²⁴ however, a longer follow-up evaluation has not been reported. In a review of published articles describing teaching and learning programs within pharmacy education, only one²⁵ of 21 programs focused on faculty training.²⁶ The study by Stein and colleagues evaluated the short-term impact (via pre- and post-surveys) of a 1-day faculty development training on teaching competency.²⁵ Experts in the medical education field have emphasized the importance of a long-term follow-up of faculty training programs, which is more likely to provide a better understanding of its maintenance and long-lasting effects.²⁷ By following recommendations for evaluating faculty training programs,^19,20 our study exhibited methodological strengths in which a unique train-the-trainer workshop was evaluated.

Our findings add to the body of pharmacy literature and mirror results from research conducted with training programs for medical school faculty. We found evidence for a positive impact of attending a workshop that aimed to prepare faculty to implement a tobacco cessation curriculum, and those who attended the workshop reported a valuable impact on their careers. Similar to prior studies, training resulted in high confidence in teaching^19,25,28,29 and providing clinical services to patients.^30,31 Most (70.6%) of our participants who work in a clinical setting reported asking patients about tobacco use all or almost all the time. This value is higher than that which is reported in the literature—e.g., prior reports indicate that between 4% and 39% of pharmacists ask their patients about tobacco use.^10,32–35 Participants in this study also implemented changes to their teaching and clinical practices as a result of participating in the train-the-trainer program, which is similar to findings in the medical literature.³⁶ Enhanced teaching strategies, such as utilization of active learning methods, was an important benefit from the training.^19,37 Similar to findings from a systematic review, partial adoption was more common than adoption of the entire program.³⁸ Yelon and colleagues posit that a suitable measure of the long-term positive consequences of training is enhanced performance, as well as talking with or advising colleagues about an idea or behavior.²⁷ In our study, three quarters of the faculty trainees advised others to consider adopting Rx for Change at their institution, a positive and unanticipated consequence supporting long-term impact of the program. Experts emphasize communicating to faculty trainees the expectations of these programs, that they are to apply and implement what they have learned, and to “broadcast this message at every possible opportunity.”²⁰ With the Rx for Change program, most web-site users have indicated that they heard about the program from a faculty member or colleague;⁸ this is further evidenced in the current study and likely has contributed to the sustained use of the curriculum over time. There is much to learn, however, from the medical profession—the medical literature describing national programs to train medical educators^36,39 can provide guidance for other professions. Educators and researchers are encouraged to contribute to the pharmacy literature by developing and evaluating similar initiatives.

In this study, a needed improvement was identified for teaching tobacco cessation in health professional programs. More than three quarters of participants indicated that fewer than six hours of tobacco cessation content is currently being taught in their institutions. According to an AACP white paper,¹⁸ this is less than the minimum recommendation of six hours. Several steps have been initiated to achieve the needed improvements pertaining to preparing future pharmacists in the area of tobacco cessation. To enhance pharmacy students’ competency, confidence, and readiness to provide tobacco cessation counseling in clinical settings, tobacco-specific virtual patients and standardized patient/OSCE cases have recently been developed and are available to health professional educators through the Rx for Change website. More than three fourths of participants requested more information about these resources, suggesting an interest in further expanding tobacco education in schools of pharmacy. Given the large increase in the number of pharmacy schools since the last workshop in 2005, new dissemination efforts are needed and can include providing additional train-the-trainer programs, either as web-based enduring programs, in tandem with professional meetings, live on-site workshops, and live webinars.

A limitation of the study is the finite sample of potential participants, starting with a cohort of 191 faculty who attended a train-the-trainer workshop. Because of the long duration of time elapsed, more than a quarter of the cohort was not locatable; however, of those invited, 81.0% completed a survey and this is a respectable response rate that provided valuable information regarding training workshop sustainability. Another limitation is the lack of a control group; however, a one-group cohort is a common study design for educational programs—a recent systematic review of medical faculty training programs determined that the majority of studies do not include a control group.¹⁹ Other important limitations include the possibility of social desirability bias and the inherent weaknesses of self-reported opinions and behaviors (compared to objective measurements). For example, faculty members reported the impact of the training on their career, their confidence for teaching tobacco cessation and counseling patients, and perceived impact of the program on students. Use of more objective measures would be valuable in further estimating the effectiveness of the program beyond self-reported ratings and perceptions. Finally, there is a potential for respondents to have forgotten some of the details surrounding their participation in the train-the-trainer program and subsequent events.⁴⁰ Despite these limitations, this study provides a unique contribution to the literature.

CONCLUSION

The study provides evidence for long-term reach, effectiveness, adoption, implementation, and maintenance of the Rx for Change program as a result of train-the-trainer programs targeting pharmacy faculty members. The study is unique, in that the long-term impact of faculty development programs are rarely conducted or reported. Faculty who attended a train-the-trainer workshop 15 year ago valued the impact of the training on their careers, confidence for teaching, confidence for patient counseling, and perceived their implementation of the Rx for Change program to positively impact their students. Furthermore, most (70.6%) of those who practice in a clinical setting reported asking their patients about tobacco use all or almost all the time, and the majority utilize one or more of the established tobacco cessation counseling techniques. Faculty also reported positive changes in their teaching and practice as a result of workshop participation, and most reported advising others to consider the adoption and implementation of Rx for Change curriculum. The training workshop appears to have had a lasting, positive effect on faculty participants as well as implementation and sustainability of the Rx for Change curriculum 15 years later.