Evaluation of a Train-the-Trainer Program for Cultural Competence

Abstract

Objectives

To develop, implement, and evaluate the impact of a cultural competence train-the-trainer workshop for pharmacy educators.

Methods

A 2-day train-the-trainer workshop entitled Incorporating Cultural Competency in Pharmacy Education (1.65 CEUs) was provided to pharmacy faculty from schools across the United States. Baseline, posttraining, and 9-month follow-up surveys assessed participants' (n = 50) characteristics and self-efficacy in developing and teaching content.

Results

At baseline, 94% of faculty members reported no formal training in teaching cultural competence. After completing the workshop, participants' self-rated confidence for developing and teaching workshop content significantly increased. The number of participants who rated their ability to teach cultural competence as “very good” or “excellent” increased from 13% to 60% posttraining. Participants reported teaching 1 or more aspects of the workshop curriculum to nearly 3,000 students in the 9-months following training.

Conclusions

The workshop significantly increased faculty members' perceived and documented ability to teach cultural competence. The train-the-trainer model appears to be a viable and promising strategy for meeting the American Council for Pharmacy Education accreditation standards relating to the teaching of diversity, cultural issues, and health literacy.

INTRODUCTION

In 2006, influenced by changes in the health care environment and a changing vision of pharmacy practice, the American Council for Pharmacy Education (ACPE) revised its accreditation standards and guidelines for doctor of pharmacy (PharmD) degree programs. One of the important changes was a focus on graduates' ability to factor and account for diversity, cultural issues, and health literacy in the provision of patient-centered care.¹ These skills are also referred to as cultural competence, which the Office of Minority Health defines as the awareness, knowledge, behaviors, and skills that enable individuals to work effectively in cross-cultural situations.²

Cultural competence curriculum development, implementation, and assessment efforts in medical and nursing education have been well described.³ However, less is known about the similar efforts in pharmacy education. A survey of curriculum chairs and student organization leaders found that 18 of 49 (37%) colleges and schools of pharmacy surveyed had made curricular changes to include cultural competence-related content areas in the curriculum over the past 5 years.⁴ Another survey of online curricula and course descriptions found that of 97 colleges and schools, only 10% offered 1 or more focused classes on diversity or cultural competence.⁵ While standards and guidelines for cultural competence training are available for medical education, they have yet to be detailed for pharmacy and other health professions.⁶ Furthermore, resources for training pharmacy educators in cultural competence and strategies for integrating this content into the curriculum are scarce.

To meet this need, 2 of the authors (M.A., S.M.) adapted and implemented a 2-day train-the-trainer cultural competence program for pharmacy educators modeled after one developed by the Center for the Health Professions at the University of California-San Francisco. The train-the-trainer model is an effective method for disseminating new content knowledge to health professionals.^7-11 It is a useful strategy for supporting curricular change in that faculty members are provided ready access to comprehensive teaching materials, which can ease the amount of time required to develop curricular content. This model also provides faculty members an opportunity to experience and interact with new content guided by experts who can provide direction on effective approaches to incorporating content into existing curricula. This context allows workshop faculty members an opportunity to coach, observe, and provide participants with feedback on special facilitation skills for content areas that may be sensitive (eg, racism). The objective of this report is to describe the implementation and longitudinal evaluation of our train-the-trainer program for disseminating a cultural competence curriculum.

METHODS

The target audience for the workshop was pharmacy educators at colleges and schools of pharmacy in the United States and Canada. The course was advertised through electronic mail and web-based announcements posted on the Center for the Health Professions' web site. All prospective participants completed a registration form and a brief baseline questionnaire. The questionnaire asked applicants to describe their current teaching and clinical experience and roles and their plans for developing and implementing cultural competence training into their PharmD curriculum.

The train-the-trainer workshop, entitled Incorporating Cultural Competency in Pharmacy Education, was a 2-day, 16.5-hour (1.65 CEU) course offered once in 2004 and once in 2005 to a maximum of 30 participants per session. The aim of the workshop was to provide participants with an opportunity to learn relevant content and to apply this knowledge by initiating development of their curriculum plans. The training objectives are described in Appendix 1. The workshop outline, organization, and content were based on previously developed train-the-trainer workshops for health professionals at the University of California at San Francisco. The content and activities were modified for an audience of pharmacy educators. A detailed program agenda and section-specific objectives are available from the corresponding author. The workshop consisted of didactic and experiential activities (eg, role plays, case discussions) designed to allow participants to learn and experience relevant content areas and discuss curriculum development and facilitation challenges. On the second day, participants were asked to initiate and present their plans for content development and implementation.

We evaluated the impact of the workshop using a quasi-experimental, one-group design involving survey administration. The study was approved by the Committee on Human Research (CHR) in the Office of Research at the University of California, San Francisco. The main purpose of the study was to evaluate participants' self-efficacy in: (1) cultural competence, and (2) developing and implementing related course materials. To do so, a set of 3 survey instruments was designed around training objectives and underlying constructs. The survey instruments were completed immediately before the workshop (pretraining baseline survey), at the end of the workshop (posttraining survey), and 9-months after the workshop. Survey instruments collected the following information: (1) sociodemographic factors, (2) prior cultural competence training and teaching experience, (3) measures of self-efficacy in teaching relevant content, and (4) perceptions of the adoptability of the workshop content and materials. The final survey was intended to capture participants' reported behaviors during the academic year following training. These instruments were based on previous instruments used by Center for the Health Professions to evaluate cultural competence trainings for health professionals and assessment materials utilized in the Rx for Change: Clinician-Assisted Tobacco Cessation program (http://rxforchange.ucsf.edu), a national train-the-trainer's course for health professions students, educators, and practitioners.¹¹

Course participants were informed about the study by the lead author at the beginning of the workshop. Each subject was assigned a unique identifier to assure confidentiality and facilitate linking survey responses across the 3 assessments. To enhance response rates for the 9-month assessment, participants were contacted twice by e-mail: once to inform them of the follow-up survey and a second time to remind them of the survey deadline.

Sociodemographic factors were assessed in the pretraining survey. These included sex, age, race/ethnicity, years in current position, current academic level, and area of clinical expertise. Participants' experience in working and/or serving as a clinical preceptor in a patient-care setting serving a diverse patient population was also assessed.

Prior training and teaching experience were assessed in the pretraining survey by asking respondents to estimate the number of hours of cultural competence training they had completed. Teaching experience was assessed by asking participants to report the number of years they had taught cultural competence content and the the number of students they instructed in this content area annually. Participants were also asked to describe whether their teaching experiences included didactic lectures, conference sections, and/or continuing education programs on diversity, cultural sensitivity, and/or cultural competence.

The pretraining survey instrument and both posttraining survey instruments measured participants' confidence in developing and implementing cultural competence-related content. The survey conducted immediately after training also measured participants' perceptions of the workshop, training materials, and evaluation of course instructors and logistics. Participants were also asked whether they would recommend the course to other pharmacy and health professions educators.

Using Rogers' Diffusion of Innovations as an underlying theoretical framework for evaluating the dissemination process,¹² posttraining perceptions associated with adoptability of the workshop content and materials were assessed with respect to: (1) compatibility for integration into the existing curriculum structure, (2) relative advantage over other available cultural competency training materials, (3) relative advantage over other cultural competence content and materials currently taught and used in the curriculum, (4) comprehensiveness of content, (5) appropriateness of teaching methodologies discussed, (6) confidence in their skills for teaching cultural competence, and (7) likelihood of adoption of cultural competence at their college or school of pharmacy. Response options included 1 = none, 2 = low, 3 = moderate, and 4 = high.

On both the posttraining and 9-month survey instruments, the perceived importance was assessed (1 = not at all, 2 = a little, 3 = moderately, 4 = very, and 5 = extremely important) and likelihood of adoption in the coming academic year (1 = definitely not, 2 = probably not, 3 = not sure, 4 = probably yes, 5 = definitely yes) for the content areas addressed in the workshop. We also assessed whether participants personally had the ability to determine whether cultural competence-related content would be integrated into their school's curriculum.

Using a 5-point scale (1 = not at all, 2 = a little, 3 = moderately, 4 = very, and 5 = extremely important), a series of perceived barriers to use of the workshop content were assessed, including lack of curriculum time, financial resources, faculty content expertise, faculty interest, faculty's perceptions of importance of cultural competence, access to comprehensive, evidence-based resources for teaching cultural competence-related content, and available advanced pharmacy practice experience (APPE or clerkship) sites with access and exposure to diverse patient populations. These items were assessed posttraining.

In the 9-month survey instrument, the extent to which faculty participants had integrated cultural competency content into the pharmacy curriculum at their institution was assessed. Specifically, we assessed the amount of curricular time dedicated to cultural competency, methods by which the content had been implemented or delivered (eg, lectures, conference sections) and the number of students taught.

Statistical analyses involved computation of simple summary statistics to characterize the questionnaire responses. Group comparisons were made using t tests. Analyses were conducted using SPSS Version 10.1.3. (SPSS Inc, Chicago, Ill).

RESULTS

Fifty pharmacy educators participated in 1 of 2 workshops; of these, 49 completed a pretraining survey instrument, and 50 completed a posttraining survey instrument. Thirty participants (60%) completed the 9-month follow-up survey instrument. Participant demographics are described in Table 1. Most participants were female, white, and assistant professors with expertise in pharmacy practice. One third had previously received cultural competency training.

Table 1.

Characteristics of Participants in a Train-the-Trainer Program for Cultural Competence (N = 49^*)

^*Includes only participants who completed a pretraining survey (98%)

Denominators may vary due to missing data

At the conclusion of the workshop, participants reported that 55% of the course material was new, 36% was a necessary review of content that had been learned before, and 9% was an unnecessary review of content that had been learned before. Participants' perceptions of the cultural competency curriculum materials are summarized in Table 2. The majority of participants reported that the workshop materials were highly compatible with existing curricular structure (63%) and had a high degree of relative advantage over available cultural competence training materials (61%) and those currently used in their curriculum (64%). Three fourths of participants rated the workshop content “highly comprehensive,” and 90% rated the teaching methodologies “highly appropriate.” Table 3 summarizes the change in participants' reported confidence in developing and teaching cultural competence curricula. We observed a significant improvement in confidence from for all 8 items from baseline to the conclusion of the workshop (p values < 0.001).

Table 2.

Participants' Posttraining Perceptions of the Curriculum Materials Used and the Train-the-Trainer Program in Cultural Competence (N = 50)

^*Response scale: 1 = none, 2 = low, 3 = moderate, 4 = high

Table 3.

Comparison of Pretraining and Posttraining Ratings of Confidence in Developing and Teaching Curriculum (N = 50)

^*Response scale: 1 = not at all confident, 2 = a little confident, 3 = moderately confident, 4 = very confident, 5 = extremely confident

When asked to rate their overall ability to teach cultural competence to their students, the number of participants rating themselves as “very good” or “excellent” increased from 13% at baseline to 60% at the conclusion of the workshop. Overall, we observed a significant increase (2.5 ± 0.8 versus 3.7 ± 0.8; t ₄₆ = 8.1, p < 0.001) in self-reported abilities (Figure 1). Among respondents for whom posttraining and 9-month follow-up survey responses were available (n = 30), there was a nonsignificant overall decrease in self-rated abilities over time (3.6 ± 0.7 versus 3.5 ± 0.8; t ₂₉ = 0.8, p = 0.45).

Figure 1.

Faculty self-ratings of overall ability to teach cultural competency to pharmacy students (n = 47): pre- versus post-training assessments (p < 0.001).

All respondents (n = 47; 3 missing data points) stated that they would recommend the workshop to other pharmacy faculty members, and that they would recommend the workshop to faculty members from other health professional schools (eg, medical, nursing, dental) who might be interested in teaching this topic to their students.

At baseline, more than two thirds (69%) of participants reported that they personally had the ability to determine whether cultural competence content could be integrated into the curriculum; 19% were not sure, and 13% did not have the ability. Table 4 summarizes respondents' perceived importance posttraining of specific workshop content in the required curriculum at their institution and likelihood of implementing this content in the upcoming academic year. On average, all core content areas were rated to be very or extremely important (mean range, 4.2 to 4.8).

Table 4.

Posttraining Perceptions of Importance and Likelihood of Implementing Workshop Content as Required Content (N = 50)

^*Instructions on questionnaire: Please indicate the response that best describes your perception of how important it is for each of the following cultural competence-related issues to be covered in your school's required coursework

^†Mean (standard deviation); Response scale: 1 = not at all important, 2 = a little important, 3 = moderately important, 4 = very important; 5 = extremely important

^‡All pretraining vs. posttraining differences were significant (p < 0.05; pretraining scores not shown)

Def. = Definitely; Prob. = Probably

Among the 7 potential barriers to implementation (assessed posttraining), lack of available curriculum time was the most important; 37% and 22% of respondents rated this barrier as very or extremely important, respectively. Other potential barriers, and the corresponding percentage of respondents who perceived the barrier to be very or extremely important (combined), included lack of faculty content expertise (57%), lack of faculty's perceived importance of cultural competence as applicable to a pharmacist's job (43%), lack of faculty interest in cultural-related issues (41%), lack of financial resources (31%), lack of APPE/clerkship sites that provide diverse patient populations (33%), and lack of access to comprehensive, evidence-based resources for teaching cultural competence content (25%).

Upon conclusion of the workshop, 74% indicated a high likelihood of adoption of cultural competence coursework at their pharmacy school (27% indicated moderate likelihood). Participants anticipated teaching a median of 5.0 hours of cultural competence content in their school's curriculum in the coming academic year (inter-quartile range, 2-8.5 hours), with the distribution of responses as follows: 0 hours, 2%; 1 to 3 hours, 31%; 4 to 6 hours, 36%; more than 6 hours, 31%. Of the 7 potential barriers to implementation listed above, none were significantly associated with the anticipated number of hours to be implemented in the upcoming academic year.

The median actual number of hours taught by participants, as reported by 21 respondents at the 9-month follow-up assessment, was 6.0 hours (interquartile range, 3.5-12.5 hours) during the academic year following the train-the-trainer program. Excluding 2 respondents who reported teaching no students, participants taught content to a median of 96 students (interquartile range, 50-155 students). Table 5 delineates the extent to which each training component was implemented during the academic year after the workshop as reported on the 9-month follow-up survey instruments.

Table 5.

Extent to Which Each Cultural Competency Training Component Was Taught During the Academic Year Following the Train-The-Trainer Program (N = 30)

^*n (%)

DISCUSSION

In this longitudinal evaluation of cultural competence training for pharmacy faculty members, we found that 2-day train-the-trainer sessions were well-received and significantly increased participants' confidence in their ability to develop and teach cultural competency content. These data suggest that, as with prior reports, a train-the-trainer model is suitable for curricular dissemination. Participants' responses provide insight into the baseline knowledge, skills, and perceptions of a cross section of pharmacy faculty members from institutions actively seeking to develop and implement curriculum in this subject area. The results also shed light on content areas that pharmacy educators perceive to be less important (eg, models for understanding cultural competence, how to work effectively with interpreters) and were least likely to report teaching.

In the academic year following the training, participants taught more than 2900 students. This finding is notable, given that 94% of the workshop participants had no prior training in teaching cultural competence. The self-selection of participants, as well as the high degree of compatibility for integration of the materials and requiring participants to develop and discuss curricular plans during the workshop may have contributed to this outcome.

Finally, our evaluation has helped us identify some areas for future improvement for this train-the-trainer program. These include: (1) development of more didactic course materials for participants to use, (2) increased attention to how to use and facilitate activities that raise awareness of content topics, and (3) greater emphasis on evaluation of cultural competence curricula.

Strengths of this study include a longitudinal design that enabled us to follow participants from 2 workshops over a 9-month period. We achieved very high response rates for the baseline and posttraining response rates (98% and 100%, respectively). However, our assessment of longitudinal impact was hindered by a low response rate of 60% to the 9-month follow-up survey. It is likely that individuals who did not respond to the 9-month follow-up survey were less likely to have implemented content in the academic year following training. If so, our findings of the impact of this training course on teaching of cultural competence content would be diminished. A significant problem with many educational program evaluations, including this one, is their dependence on self-assessed competencies and impact. While we found significant increases in knowledge, self-confidence, and skills, objective evaluation of these changes was not feasible within the scope of this effort. An important question for future research is whether self-reported increases in knowledge and confidence translate into behavioral changes in participants' teaching and practice of culturally competent care.

We recognize that our educational initiative is not a randomized trial testing the train-the-trainer method of program dissemination (versus other methods), and as such our results likely are not generalizable to other methods of dissemination, such as online coursework or “webinars.” However, our pretest/posttest design is characteristic for this type of research. Although there was no control group in our study, our design does enable examination of change scores and the baseline measure adds statistical power, which is important for our small sample size.

Another limitation of our study is the scale we used for participants to classify how and to what degree various cultural competence content-related areas were taught in their curriculum over the course of the academic year following their participation in the workshop. This scale did not enable us to characterize fully which content was delivered and to what extent. In our ongoing research, these survey items have been revised to operationalize the teaching methods as well as quantify the minutes allocated toward teaching each content area. Finally, we conducted an evaluation of our own program, and this lends potential for bias. While it would have been ideal to contract with an external evaluation team, this was not possible because of funding constraints.

CONCLUSION

Accreditation standards now require colleges and schools of pharmacy to ensure that their graduates demonstrate an ability to account for the role and impact of cultural factors in patient care and health care communications. These skills are addressed by the field of cultural competence, which is relatively new to many health professional institutions. Integrating new content into pharmacy education poses a double challenge: faculty members must be effectively trained and possess strategies for rapidly turning around and teaching the content to their students. We believe that our train-the-trainer model, in combination with a shared national curriculum, is an effective and efficient approach for preparing pharmacy educators to teach cultural competence. As such, this appears to be a viable model for rapidly promoting widespread sharing of curricular materials and strategies given the reality of new standards and limited resources.

Appendix 1. Course Objectives