Global Health Learning Outcomes in Pharmacy Students Completing International Advanced Pharmacy Practice Experiences

Abstract

Objective. To examine the global health learning outcomes of Doctor of Pharmacy (PharmD) students from three US schools who participated in international advanced pharmacy practice experiences (APPEs).

Methods. A mixed-methods, prospective study was used to assess fourth-year PharmD students at three US pharmacy schools who participated in an international APPE during the 2017-2018 academic year and a matched cohort (control group) of PharmD students who did not participate in an international APPE. To evaluate students’ self-perceived growth in the Consortium of Universities for Global Health (CUGH) competencies, all students completed a 13-item retrospective pre-post instrument using a five-point Likert scale. The students who had completed an international APPE were invited to participate in a focus group (N=22). Paired and independent t tests and multiple linear regression were used to analyze data. Qualitative open-ended questions and focus group data were mapped to knowledge, skills, and attitudes themes.

Results. The students who completed an international APPE (N=81) showed significantly more growth in CUGH competencies than students who did not (mean improvement in total score of 10.3 [7.0] vs 2.4 [6.0]). International APPE participation was the only significant predictor of growth in CUGH competencies. The international APPE students reported improvements in cultural awareness and appreciation, communication skills, problem-solving skills, adaptability, self-awareness, personal and professional outlook, and global health perspective.

Conclusion. Pharmacy students’ participation in international APPEs led to significant improvement in all CUGH competencies. The CUGH competency framework appears to be a suitable instrument to assess pharmacy students’ global health learning outcomes.

INTRODUCTION

Global health education among health professions programs is expanding and evolving. Demand for global health experiences by health professions students is high, with nearly a third of dental students, more than a quarter of medical students, and 7% of pharmacy students participating in them during their professional program.^1-3 While global health education is historically associated with North American institutions, other regions of the world, including Australia, Germany, Sweden, and the United Kingdom, are incorporating global health into their health professions curricula.^4-7 Health professions programs are also incorporating global health into residency and fellowship training, with more than half of medical residency programs now offering a global experience.^8-11

As global health education offerings increase, health professions programs are focusing more on how to best deliver interprofessional experiences, which remains a significant gap in global health education.¹² Effective collaboration between health profession trainees will be essential not only to maximize learning, but also to achieve the Sustainable Development Goals by 2030 and the broader global health agenda.¹³ The Sustainable Development Goals were established and approved by all United Nations member states in 2015 as a framework for improving lives worldwide by 2030, with ambitious goals ranging from ending poverty to ensuring the health and wellbeing for all. While several health professions have created global health competency frameworks, it is unknown as to how these apply across multiple professions.^14-16 To address this matter, the Consortium of Universities for Global Health (CUGH) developed a global health competency framework that highlights “global citizen” level competencies that all health professional trainees should achieve while participating in a global health experience.¹⁷ The framework aims to help with the challenge of creating a common set of core learning objectives and competencies that are applicable across health profession programs to strengthen global health learning.

The CUGH framework reflects learning outcomes demonstrated in medicine, nursing, pharmacy, and social work among other health professions that send students abroad. Health professional students participating in global health experiences gain knowledge regarding disease states, cultural differences, and social determinants of health as well as skills in cross-cultural communication, leadership, and empathy. Positive attitudinal changes range from a broadened perspective of patient care to a stronger appreciation for resources and cultural beliefs.^18-21These learning outcomes may impact career decisions as those who participate in global health experiences may be more likely to work in a primary care setting or with underserved populations.²²

To date, no profession has used the CUGH global health competency framework specifically to assess students’ global health learning. Furthermore, there is only limited data suggesting that use of the CUGH framework as a self-assessment tool for global health experiences is plausible.²³ Further, the assessment of learning outcomes is predominantly in the context of low- to middle-income countries with little representation from developed nations. Pharmacy is an emerging and underrepresented profession in global health, and this research serves as an opportunity to determine the applicability of the CUGH competency framework for global health assessment. The primary objective of this study was to assess the global health learning outcomes of pharmacy students who participated in an international APPE using the CUGH competency framework. Second, this study sought to assess learning outcomes beyond global health, recognizing that not all learning outcomes for international experiences may fit within the CUGH competency framework.

METHODS

A convergent, parallel, mixed-methods design was used prospectively across several institutions to assess the global health and broader learning outcomes of students who participated in an international advanced pharmacy practice experience (APPE) during their final year of pharmacy school. The international APPEs were four to eight weeks in duration and predominantly in inpatient or ambulatory care settings. Students completing an international APPE from May 2017 to April 2018 at the University of North Carolina at Chapel Hill, Purdue University, or the University of Colorado were invited to participate in the study (international APPE group). Participants were matched with a control group of pharmacy students at their institution who were not participating in an international APPE. In addition to being at the same institution, control group students were identified by having a rotation during the same month and being the same age as the international APPE rotation student.

The authors translated the CUGH competency framework into an assessment instrument by taking the 13 CUGH global health competencies for global citizens and placing them verbatim into Qualtrics (Qualtrics, Provo, UT) for students to use in self-assessing their achievement of each of the competency statements and rating it on a five-point Likert scale (strongly disagree=1 to strongly agree=5). Students in both groups completed the pre-rotation and post-rotation self-assessment within one week after completing the APPE. A summation score for each student was derived by totaling the student’s self-assessed Likert-scale scores for each of the 13 competencies, resulting in a maximum score of 65.²⁴ Demographic information, which included prior travel history, grade point average (GPA), gender, and whether students completed a prior public or global health course, was collected for students in both groups.

The students completing the international APPE answered an additional four open-ended questions at the end of the survey instrument in Qualtrics regarding what knowledge, skills, attitudes, or other attributes were gained or enhanced through their experience. Students were also asked whether they would be willing to participate in a focus group interview. The open-ended questions allowed for both verification of any pre-post growth on the CUGH competency statements and for students to comment on any learning outcomes that may not have been captured by the CUGH competency statements. Interviews further explored any changes in knowledge, skills, and attitudes that students described in the open-ended questions to determine alignment with the CUGH competency statements and identify additional learning outcome themes. Focus group participants were grouped by institution and then by the World Bank country classification of their international APPE. The lead investigator at each institution conducted the interviews for their respective students via ZOOM 4.1 (Zoom Video Communications, San Jose, CA) using a common script. Interviews were recorded and transcribed verbatim.

All quantitative data analyses were conducted in SPSS for Windows, Version 23 (IBM, Armonk, NY). Only data from matched pairs, ie, an international APPE student matched with a control group student, were included in the quantitative analysis. Continuous data are presented as mean and standard deviation. Categorical data are presented as frequency and percent. Using recommendations from Carifio and Perla, parametric techniques were used to analyze CUGH scores.²⁴ Cronbach alpha was used to examine the reliability of the CUGH competency statement items. Independent t tests were used to examine differences between independent groups and paired t tests were used to examine differences for paired data. Chi-square tests were used to analyze categorical variables for independent groups. Multiple linear regression was used to model pre-post differences in student responses to the 13 CUGH competencies. Statistical significance was established at α=.05.

Qualitative data were analyzed using MAXQDA 2018 (VERBI Software GmbH, Berlin, Germany), including comments from the open-ended knowledge, skills, and attitudes questions, as well as from focus group transcripts. A two-cycle, open-coding process was used in which the principal investigator coded each response while other research team members independently coded an equal number of the responses. Coding discrepancies were resolved during face-to-face meetings to develop a codebook. Using the codebook, a second cycle of coding was conducted that followed a similar process where the principal investigator re-coded all of the responses using the final codebook while other research team members independently re-coded their respective sections. The principal investigator rectified discrepancies again before presenting the entire data set for review and verification by the research team. Investigators recorded and discussed analytic memos after each coding cycle. A conventional content analysis approach was used for the qualitative data.²⁵

Data from the open-ended questions and focus groups were evaluated for depth of reflection based on the adaptation of Mezirow’s hierarchy by Kember and colleagues.²⁶ Kember’s four-category scheme ranges from the lowest level, called habitual action (non-reflection), to the highest level, called critical reflection. The lower two levels of habitual action and understanding are characterized by limited levels of thinking, while the higher two levels of reflection and critical reflection are characterized by the application of theory and having a changed perspective.²⁶ Reflection assessment allowed for analysis of each learning outcome category (knowledge, skills, and attitudes) to help determine whether any category left a greater impression upon students while abroad. Two members of the research team coded each comment for reflective depth from 1 (habitual action) to 4 (critical reflection). The institutional review board of each participating institution reviewed this study and determined it to be exempt.

RESULTS

One hundred fifty-two students consented to participate in the study and completed the survey instrument, including 81 students who participated in an international APPE and 71 matched students in the control group who did not participate in an international APPE. Country locations for the international APPEs were as follows, with number of students who participated in parentheses: United Kingdom (18), Kenya (17), Malawi (14), Australia (7), Ireland (4), China (3), Guatemala (3), Japan (3), Zambia (3), Ethiopia (2), India (2), Moldova (2), Canada (1), New Zealand (1), and Tanzania (1). There were no significant differences found between students in the international group and those in the control group in terms of age, gender, prior international travel for vacation, or prior public or global health coursework (Table 1). However, there were significant differences between students in the international group and those in the control group in GPA (3.62 [SD=.3]) vs 3.51 [SD=.3], p=.03), number of countries visited (4.0 [SD=3.1] vs 2.2 [SD=2.4], p<.01), prior international travel for any purpose (87.3% (n=62) vs 69.0% (n=49), p=.01), and prior international, non-vacation travel (53.5% (n=38) vs. 29.6% (n=21), p<.01).

Table 2.

Students’ Responses Regarding Global Health Competency Attainment Prior to and Following Completion of an International Advanced Pharmacy Practice Experience

Table 1.

Demographics for Pharmacy Students Who Completed an International Advanced Pharmacy Practice Experience or Served in a Control Group in a Study That Examined Changes in Students’ Global Health Competencies

The mean CUGH score for students in the international group was 43.1 (7.3) prior to the experience and 53.3 (6.3) following the experience (p<.01), while the mean CUGH score for students in the control group was 42.1 (8.6) prior to and 44.6 (8.1) following the experience (p<.01). The growth difference, defined as the difference between the post- APPE CUGH score and pre-APPE CUGH score, for each competency statement and for the overall score are presented in Table 1. There was no difference in pre-APPE CUGH scores between students in the international and control groups: 43.1 (SD=7.26) vs 42.1 (SD=8.6), respectively; p=.46. However, a significant difference was found in post-APPE CUGH scores between students in the international group and students in the control group: 53.3 (SD=6.3) vs. 44.6 (SD=8.1), respectively; p<.01. Students in the international group also demonstrated greater growth from pre- to post-APPE than students in the control group: 10.3 (SD=7.0) vs. 2.4 (SD=6.0), respectively; p<.001. The Cronbach alpha was .90, indicating high internal consistency for the instrument. All assumptions of regression were met for the regression model, including lack of multicollinearity. The only significant predictor of CUGH growth was participating in an international APPE (p<.001), with a positive beta coefficient of 7.9 when controlling for all other variables in the model. All other variables were not significant as shown in Table 3.

Table 3.

Regression Model Predicting Doctor of Pharmacy Students’ Improvement in Their Attainment of Global Health Competencies After Completion of an International Advanced Pharmacy Practice Experience^a (N=142)

All 81 students who participated in an international APPE completed the open-ended questions, and 22 students participated in focus group discussions with 18 students coming from the University of North Carolina at Chapel Hill and two each from Purdue University and the University of Colorado. The major themes observed from code mapping across the knowledge, skills, and attitude domains for both the open-ended questions and focus group data, along with corresponding representative student quotes, are presented in Table 4. As students progressed from discussing knowledge, to skills, to attitudes, their reflection levels became more critical in depth, with mean values of 1.9, 2.1, and 2.4, respectively.

Table 4.

Learning Outcomes and Representative Student Quotes Obtained from Survey and During Focus Groups Conducted After Students’ Participation in an International Advanced Pharmacy Practice Experience

Students demonstrating cultural awareness described their understanding of a cultural value, belief, or norm, whether it was related to health care or other activities of daily living. Over half of the students who responded to the open-ended questions on the survey and three-fourths of students in the focus groups indicated having increased cultural awareness, with the majority of those students discussing how culture influences the delivery of care. Students explored how patients’ health behaviors and beliefs influence treatment approaches and decisions. Students also described increased knowledge of cultural norms outside of a health care context.

Understanding cultural differences coincided with students describing patient care differences, focusing on social determinants of health, barriers to care, and disease state management. Almost half of the students demonstrated an understanding of how social determinants of health influence disease burden and impact health care delivery. Students most frequently noted resource limitations as a barrier to care, which was often associated with supply chain management and workforce development issues.

Over three-fourths of students in the focus group mentioned adaptability as an attribute that was enhanced during their international APPE. Students most frequently described adaptability in the context of effectively using limited resources to manage patients or adjusting to cultural norms such as communication. Students’ adaptability also improved through learning a new health care system and navigating team dynamics. These experiences enabled students to acclimate to new surroundings as a result of being immersed in a high degree of cultural and patient care differences.

Many students noted that their problem-solving skills improved. Problem-solving was often associated with limited resources, ranging from limited formularies to the lack of electronic health records. Overcoming barriers to care and navigating unfamiliar health care systems also contributed to skill development. Some students noted how working through communication barriers, either language or cultural, enhanced their ability to problem solve. Students also described problem-solving outside of clinical environments as part of their activities of daily living in their new culture.

Over two-thirds of students reported having improved communication skills, which was usually the result of having to overcome language and cultural barriers to effectively communicate information to patients and other health care providers. A few students noted that they would have more patience or empathy for patients in the future because of experiencing communication barriers themselves. Some students reported adapting their verbal and nonverbal communication in consideration of cultural differences, both in non-English and English-speaking countries.

Self-awareness was a noticeable theme that emerged during the focus groups. During their international APPE, students reflected on their implicit bias, became more aware of their environment, and learned how to adapt to be respectful of the local culture. Self-awareness often occurred in the form of increased confidence when navigating new situations or working in a clinical environment. Students reflected on how their international experience taught them about themselves and their ability to adapt and succeed despite various barriers.

Among students who mentioned having improved cultural awareness, approximately a fourth progressed further into developing a sense of cultural appreciation. In other words, students were not only aware of cultural differences, but also appreciated how these differences impact patient care. Students often described their cultural appreciation as a newfound respect that they had for the interface of culture and health care. Student comments increased in terms of depth of reflection as they progressed from cultural awareness to cultural appreciation, moving upward on Kember’s categorical hierarchy from an average of 2.0 to 2.9.

Several students went beyond cultural awareness and cultural appreciation by describing a desire to incorporate and apply their cultural learning as part of who they are personally and professionally. Students described the importance of assessing patients holistically in order to better incorporate cultural values and beliefs when managing their health. Students developed a sense of ownership for patient outcomes as they advocated for the patient to be involved in their own care as a way to ensure cultural values and beliefs were upheld in treatment decisions. Beyond professional practice, students discussed how their experience challenged their own personal beliefs and values to the point of reconsidering how they view the world beyond patient care.

Students developed an open mindset with regard to cultural and patient care differences that shaped their career outlook. Almost half of students indicated a changed perspective regarding patient care that resulted primarily from their observation of cultural or resource limitations while on their APPE. Developing an appreciation for alternative approaches to patient care, including complementary and alternative medicine, resulted in a deeper appreciation for different health care systems and practices. Students discussed how their experience motivated and inspired them to be a better practitioner for their chosen career direction. This was most often due to witnessing the local health care providers’ dedication to patient care and determination to improve their profession.

Students in the focus group noted that they did not have a good understanding of what global health was before starting their international APPE. The knowledge gained through the experience led to a better understanding of global health issues and contributed to an enhanced global health perspective for the care of patients and populations. Students discussed being more aware of how their actions impact health globally and how global health applies to their local settings.

DISCUSSION

This is the first study to use the CUGH framework to assess learning outcomes of pharmacy students who pursued global health experiences across diverse international locations. While significant growth occurred across all 13 CUGH competencies, student comments and discussion focused primarily on two competencies: how culture influences health care (3a) and communication skills with patients and health care team members (5d). Secondary areas of focus included barriers to care in limited resource settings (7b) and how social determinants of health impact access to quality health services (3b). These competencies may have represented the highest degree of relative difference for students between their environment at home and their new environment in an international setting, and could be why they were the most impressionable experiences as students worked towards understanding what those differences meant. As these four areas (ie, 3a, 5d, 7b, and 3b) align with global health competency frameworks in medicine, nursing, and dentistry, they can serve as a starting point for addressing the challenge of interprofessional global health activities, and may help drive global collaborative practice.^14-16,27 Health profession programs can use these competencies as part of interprofessional pre-departure training to help students better prepare for and conceptualize their global health experience.

Although many of the students had previously taken a global or public health course, they struggled with understanding global health until they arrived at their international site, which reinforces the importance of experiential context for the application of global health content. One context that might be useful to strengthen global health education before and after an international experience is rural health. With the emerging recognition that global health is synonymous with the contextualization of public health to local communities, global health competencies may be translatable to local rural communities considering that most global health work occurs in rural settings that have numerous barriers to care and health inequalities.^28,29 Working with rural or underserved populations may help students connect global health learning to local public health practice by reinforcing that the global in global health refers to the scope of the health issue rather than the location.³⁰ This can further help health professions programs shift their global health training away from non-contextualized traditional models towards competency-based education that prioritizes local community needs.³¹ Rural health experiences appear to align with many of the global health principles assessed in the CUGH competency statements. Therefore, rural health experiences may be an alternative approach to learning the global health principles given the significant cost of international experiences.³²

Outside of specific CUGH global health competencies, students also acquired broader learning outcomes, from increased confidence and adaptability to improved empathy and patient care skills, which mirror those found in other health professions.^18-21 An underrepresented area in the literature that appeared from student comments was the development of patient advocacy associated with cultural sensitivity. Global health experiences highlight not only the importance of patient voice in decision making, but also how patients can be effective educators in shaping student learning.³³ Another underrepresented learning outcome was problem-solving ability. A substantial part of global health learning occurs outside of patient rounds or a clinic as students faced a myriad of differences to navigate in clinical and non-clinical environments. Student reflection on these differences can challenge implicit beliefs on how to solve problems, possibly leading to a paradigm shift in decision-making.³⁴

Student comments and discussion associated with the deepest level of critical reflection were often associated with positive changes in attitude, such as having an altered patient care perspective or a shift in professional outlook, rather than knowledge or skills, which are what most of the CUGH global health competencies for global citizens represent. Reflective inquiry and assessment should be used to strengthen attitudinal and mindset changes to further develop pharmacy students into global citizens. Structured student reflection on the cognitive, socio-emotional, and behavioral dimensions of global citizenship as outlined by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) can help frame global health learning in a more holistic context across professions.³⁵

Reflection also may be integral in facilitating the learning progression of students while they are abroad. While analyzing student responses and learning outcomes, there appeared to be a process of learning that students developed while on global health experiences. Students seemed to use the skills of communication, problem-solving, and adaptability to make meaning of the cultural and patient care differences they encountered as they navigated new cultural norms. As students went through this process, reflection may have helped some make the transition from acknowledging cultural differences to wanting to use their newfound cultural insight to improve patient care. This type of cultural learning progression of going from the cognitive to affective domain through reflection has been documented in nursing.³⁶ As indicated by Mezirow’s theory of transformative learning, critical reflection can lead to the reevaluation of cultural values and beliefs as part of a broader identity negotiation process, which has been seen in medical students who participate in global health experiences.^34,37 Reflective exercises and structured debriefing sessions after the international experience may further enable personal, professional, and cultural attitudinal changes that can translate back into a local context.

The results of this study highlight the need for additional scholarship related to global health education. More research is needed to determine the process of student learning to achieve learning outcomes on global health experiences and how such a process may best enable personal and professional transformation. Future research should also assess how global health education differs across country locations and whether certain APPE activities have a greater influence on learning outcomes. There remains a challenge with identifying best practices for assessing global health education abroad as well. With the emergence of entrustable professional activities (EPAs) as a means to operationalize competency assessment, future work should explore whether EPAs can be used to describe and assess global health experiences.

The study design allowed for content validation of the CUGH competency framework as an assessment instrument for global health learning that can be used across multiple country locations. The constructs presented in the instrument went through expert validation through the analysis of the open-ended questions and focus group discussions. Most learning outcome assessments of international experiences are author derived and not predicated on prior frameworks. Another strength of the design is that it addressed common limitations of prior health professions learning outcome studies in international settings in that this is a multi-institutional, longitudinal, mixed-methods design that included diverse international locations.^18-20 The instrument was administered retrospectively to mitigate response shift bias and used self-directed evaluation as this has been suggested as an appropriate mechanism for global health competency assessment despite the inherent limitations of self-assessment approaches.^38,39 Kember’s four-level categorical hierarchy of reflection is well studied and used in health profession education.⁴⁰

While this study addressed gaps in our current knowledge, there are several limitations to this work. Only one health profession was studied and most of the focus group data came from students at one institution. Demographic information as to the nationality and background of students was not collected. All of the schools that participated were part of public academic medical centers, which may also have influenced the results. Not all international APPEs were explicitly structured as global health experiences, and this could have affected the magnitude of students’ growth related to each competency. Finally, the survey instrument had limited validation and no pilot testing was conducted prior to implementation.

CONCLUSION

Results of this study indicate that international APPEs are associated with significant growth across all CUGH global health competencies for global citizens and that these competencies can be translated into a global health assessment instrument for pharmacy and potentially other health professions. These experiences help to develop students’ cultural progression and can be used to strengthen global health learning. Global health experiences can be transformative opportunities that equip and shape students with new perspectives that translate locally, helping students become better practitioners.