A Consensus Approach to Investigate Undergraduate Pharmacy Students’ Experience of Interprofessional Education

Abstract

Objective. To assess the development of knowledge, attitudes, and behaviors for collaborative practice among first-year pharmacy students following completion of interprofessional education.

Methods. A mixed-methods strategy was employed to detect student self-reported change in knowledge, attitudes, and behaviors. Validated survey tools were used to assess student perception and attitudes. The Nominal Group Technique (NGT) was used to capture student reflections and provide peer discussion on the individual IPE sessions.

Results. The validated survey tools did not detect any change in students’ attitudes and perceptions. The NGT succeeded in providing a milieu for participating students to reflect on their IPE experiences. The peer review process allowed students to compare their initial perceptions and reactions and renew their reflections on the learning experience.

Conclusion. The NGT process has provided the opportunity to assess the student experience through the reflective process that was enriched via peer discussion. Students have demonstrated more positive attitudes and behaviors toward interprofessional working through IPE.

INTRODUCTION

Enhanced coordination of health care practitioners through interdisciplinary collaboration benefits patients by preventing fragmentation of care.^1,2 Interprofessional teams improve the quality of patient care,^3,4 with lower costs^4,5 and decreased length of hospital stay.⁶ Interprofessional education (IPE), defined as “education expressly intended to promote the effective function of a health team involving the relevant health professions,”^7,8 has received much focus globally as a means to achieve this collaborative practice. The Centre for the Advancement of Interprofessional Education (CAIPE) describes IPE as “occasions when two or more professions learn with, from and about each other to improve collaboration and quality of care.”⁹

Interprofessional education can take many guises, some of which may not be as effective as others in cultivating collaborative practice.¹⁰ Certain fundamental conditions have been claimed to be crucial for the success of IPE in achieving positive attitude change at the undergraduate level. The “contact hypothesis” outlines prerequisites of a physically and emotionally comfortable learning environment, such as ensuring that the setting and participants are positive and cooperative; there is institutional support and collaboration; members of the group are representative (eg, of a profession or social group), and of equal status and there should be positive feedback to students.¹¹

In the development and evaluation of IPE initiatives, there are two learning theories that can be applied, namely the behaviorist and constructivist approaches. Hean and colleagues delineate how behaviorists focus more on the outcomes of learning expressed as behaviors.¹² This theory has been largely excluded from literature describing IPE curriculum design.¹² However, the Kirkpatrick model¹³ of evaluation of learning outcomes adapted by Barr and colleagues¹⁴ (Table 1), which is behaviorist in approach, has been used to measure the effectiveness of IPE programs.^15,16

Table 1.

Classification of Interprofessional Outcomes as Designed by Kirkpatrick/Barr^13,14

The measurement of change in student behavior within interprofessional working (level 3 in Table 1) is an example of a behaviorist approach to evaluation. This has traditionally been hard to identify and measure especially at the undergraduate stage¹² except through the method of self-reporting by the student.^15,17 More advanced levels of this outcome framework such as change in organizational practice (level 4a) and benefits to patients (level 4b) are problematic to use in measuring prequalification and require longitudinal evaluation.¹⁸ Constructivists focus on the process of learning or constructing knowledge and encompass a range of theories under two categories: cognitive and social constructivism. Hean and colleagues highlight that the use of various theories within currently published IPE literature has created an “un-navigable quagmire” and recommend that future researchers apply theories to soundly and robustly underpin practice, both in the design of IPE within curricula and in its subsequent evaluation.¹² Other researchers agree that the lack of appropriate research around the effectiveness of IPE should be addressed through the application of more rigorous evaluative methods to comprehend the potential impact of IPE on professional practice and health outcomes.^2,14 Walsh and colleagues recognize it is the methodological difficulties that have limited researchers’ ability to generate this evidence thus far.¹⁹

Members of CAIPE issued a report that extensively reviewed IPE evaluations, and suggested the need to use a range of methodologies in the investigation of interventions to strike a balance between evaluation of process and of outcome. Authors of this article also suggested qualitative techniques for the former and quantitative techniques for the latter. A scarcity of data exist to show how long changes in attitude or knowledge are sustained and how learning applies to practice post-IPE.¹⁰

We recognized the need both to evaluate the effectiveness of our IPE strategy and to contribute to the growing IPE evidence base. In this article, we describe our IPE initiatives by means of the learning theory that underpins them and categorize the expected learning outcomes using levels 1-3 of the Kirkpatrick/Barr model (Table 1).

METHODS

Interprofessional education is delivered as a strand throughout Durham University’s four-year master of pharmacy (MPharm) curriculum, which is described by Husband and colleagues.²⁰ For the first two years of both the medicine and pharmacy undergraduate programs, students are located on the same campus. Interprofessional education starts within the second week of both programs, at a time when serendipitous and informal interprofessional encounters have taken place between new and returning medical and pharmacy students. Our curriculum developers have followed a strategy, as has been reported elsewhere, ²¹ that IPE should occur at the earliest opportunities in preregistration education to avoid students developing negative stereotypes and a preference for uniprofessional working over multiprofessional practice. In the United Kingdom, the single route to registration with the professional body for pharmacy is a 4+1 model where four years of undergraduate study are followed by one year of statutory training in employment. Once this is completed, a student may apply to join the pharmaceutical register. This approach is contrary to that advocated by Areskog,^22,23 and Pirrie and colleagues²⁴ who believe that IPE should be introduced after students have a clear comprehension of their professional roles.

Four to five sessions each academic year existed within the structure, where interprofessional working was revisited with increasing levels of sophistication and complexity as students progressed, as aligned to the concept of the spiral curriculum.²⁰ In designing the IPE sessions, there is no one ideal location for IPE within the curriculum; rather, there are many opportunities for enhancing learning through IPE. Sessions in level 1 (year 1) were mainly biprofessional, with all sessions including students from the undergraduate medicine program, and one session also including nurse practitioner students. In subsequent levels, students from other programs including social care, education (both from the same institution), and nurses (from a neighbouring institution) joined the pharmacy students.

Year 1 of the pharmacy program hosted four IPE sessions, the descriptions of which, and associated aims and expected pedagogical outcomes categorized using the Kirkpatrick/Barr model are summarized in Table 2. Further to this, logic diagrams²⁵ have been constructed to show how each session works and link outcomes with the session activities and processes, and the theoretical assumptions that underpin them (Appendix 1a-d). Logic models have been found to facilitate thinking, planning, and communication about intervention objectives and accomplishments, and have been adopted here for the clear description of each IPE session as an educational intervention.²⁵

Table 2.

Learning Outcomes and Outcome Levels of Interprofessional Education Sessions

We used a mixed methods approach to explore the students’ learning experience and outcomes and the context in which learning occurs. We aimed to evaluate the IPE strategy within level 1 and better comprehend how implementation, causal mechanisms, and contextual factors shape learning and result in the outcomes experienced by students.

The Nominal Group Technique (NGT) is an evaluative methodology described as “semi quantitative and qualitative”³² in which responses from participants are based on a single topic. The NGT, which was initially developed for marketing research, has been employed in addressing potentially complex qualitative concepts and has become useful in examining education, policy, and research. The methodology requires direct participant involvement (in a small group setting) in a way that is nonhierarchical, and where all participants have an equal voice and all responses to the topic have equal validity.³³ The steps with NGT are depicted in Figure 1.

Figure 1.

An activity flow diagram for a nominal group discussion.

The NGT sessions were held after the second (IPE Game), third (Patient Safety) and fourth (SimMan^®) IPE sessions. Students were briefed by the facilitator as to the purpose of the discussion and then asked to reflect in writing on their most recent IPE experience, and in particular to list negative and positive reactions. Rich data obtained through this method allowed aspects of context, implementation of delivery, and causal mechanisms to be explored from the student perspective. There has been debate as to what constitutes the optimal size group for NGT, with suggestions generally ranging from five to nine.³³ At each NGT session, up to 10 students were invited to ensure this quotient was met.

Student experience of IPE was measured quantitatively after each IPE session throughout the year using two validated tools for exploring students’ self-assessment of their attitudes to collaborative learning and working. The Readiness for Interprofessional Learning Scale (RIPLS) ³⁴ and the 12-item adapted version of the 18-item Interdisciplinary Education Perception Scale (IEPS) ³⁵ were used to detect changes in attitudes over time. These tools have been used in various studies for graduate³⁶ and undergraduate students^37-39 as well as practicing professionals.⁴⁰

Despite numerous studies, it is still unclear which scale is superior for finding attitude differences among students in tested health professions. The RIPLS was designed to assess novice students’ attitudes toward interprofessional learning, while the IEPS assesses perceived attitudes about team collaboration within the students’ own profession. The IEPS may thus be appropriate for advanced or senior students once they have had greater exposure to members of their own profession.⁴¹ However, due to the lack of empirical evidence to support this, we employed both scales but used the RIPLS at the earliest point, which was in the second week of the students’ program, when they are still considered novices, and the IEPS was added in at the second data collection point, after students had time to integrate with members (classmates, more advanced students, staff) of the same profession.

A further questionnaire was constructed using the accumulated statements from each of the three NGT sessions. Statements were listed and accompanied with a five-point Likert scale to measure level of agreement. Students were asked to rate their response to each statement in relation to each of the four IPE sessions they had participated in.

Ethical approval for the study was granted by the School of Medicine, Pharmacy and Health Ethics Sub-Committee within Durham University to survey students through pre- and post-session questionnaires and via partaking in nominal group discussions. All students were provided with participant information leaflets and asked to provide written informed consent to participate in the study.

The cohort studied were the year 1 undergraduate pharmacy students (n=81). RIPLS and IEPS questionnaires were administered to and collected from the whole cohort at the beginning of each of the facilitated sessions. For the NGT, an academic mentor (AP) from the year 2 pharmacy cohort invited students to participate in the studies. This was carried out using a convenience sampling approach. Different students attended at each of the three data collection points. Again, the first NGT session took place after students had experienced both the Anatomy course lecture and the IPE game because the former provided an opportunity for the two cohorts of students, pharmacy and medicine, to learn with one another but did not include any form of interaction. Subsequent NGT sessions took place after the Patient Safety session and the SimMan^® session.

All data from questionnaires were put into Microsoft Excel worksheets and were checked for completeness. Not all 81 pharmacy students attended all four of the IPE sessions due to an approved or unapproved absence, and therefore were unable to complete the questions pertaining to that session, so only their completed answers where included in the analysis. Quantitative data from the two questionnaires (RIPLS and IEPS) were analyzed using basic descriptive statistics at baseline and post-intervention across all four IPE sessions, as was the final questionnaire. Responses to the statements over the various data collection time points were tested for significant difference using the chi-square test, where statistical significance was defined as p<0.05.

Data from students’ positive and negative reflections about their IPE experiences, which they ranked in order of importance, provided a descriptive evaluation of the IPE session. The focused reflection that followed was transcribed verbatim from audio recordings by one author (AP) and checked for accuracy by another (HN). They were then analyzed individually by two authors (HN and ZN) using framework analysis as described by Ritchie and Spencer.⁴² Resultant themes were discussed between the two authors (HN and ZN) for agreement and clarification, and a third author was consulted to mediate any discrepancies (IO).

RESULTS

The response rates for the RIPLS, IEPS (both of which had 3 data collection points), and final questionnaire were 81.4% ±3.4%) (n=66), 79.1% (±5.7%) (n=64), and 73.2% (n=59) respectively.

Responses for both RIPLS and IEPS were highly positive in all subsections (RIPLS consisting of: teamwork and collaboration; professional identity; and roles and responsibilities. IEPS consisting of: competency and autonomy; perceived need for cooperation; and perception of actual cooperation). Across all statements within both the RIPLS and IEPS, the chi-square analysis showed no statistical difference in students’ responses when compared to students’ baseline responses, or when compared longitudinally with students’ responses throughout the academic year.

A different set of five students participated in each of the three NGT discussions. These students were those from the original 10 who agreed to participate. All reflections from each NGT session were classified by level of outcome using the Kirkpatrick/Barr model and also by their positive or negative connotation as displayed in Figure 2.

Figure 2.

The number of statements of a positive (grey) and negative (black) connotation from each of the NGT sessions that related to the levels of outcomes categorized by Kirkpatrick/Barr (1=Learner’s reaction; 2a=Modification of attitudes/perceptions; 2b=Acquisition of knowledge and skills, and 3=Change in behavior).^13,14

The five most important reflections as ranked by individuals within the NGT sessions have been identified and level of agreement with these statements in relation to the IPE session has been investigated within the entire cohort from the final questionnaire that was administered (Table 3).

Table 3.

Items Ranked Most Important from the Nominal Group Technique Discussions and Interprofessional Student Agreement

The findings from the nominal group discussions following on from the ranking of ideas are presented as themes based on the students’ experiences in each of the IPE sessions. The themes identified inductively through the natural course of the discussion and featured in all three NGT sessions. One theme, which every student from each of the NGST discussions contributed to, related to the organization of the IPE session (which relates most closely to the behaviorist focus upon outcomes, namely Kirkpatrick/Barr model’s level 1 of outcome classification: leaner’s reaction).

Many of the following themes generated from claims of the pharmacy students in the three NGT discussions (NGT1-3 between pharmacy students P1-P15) are commonly reported in the evaluation of IPE delivery and are recognized as crucial factors for its success (Appendix 2):

• - Not achieving an appropriate group mix to allow a heterogeneous learning environment;⁴³

• - External buy-in^44,45 which is also one of the prerequisites stated in the contact hypothesis to frame an environment conducive for interprofessional working;¹¹

• - Poor relationship to real life work;^44,45

• - The level of skill of the facilitator.^44,46

Some students made some positive comments on good facilitation, learning material and briefings provided at the beginning of the sessions which enhanced their learning experience.

Four further principles were identified that relate to adult and experiential learning:

1. Perceived relevance of the learning opportunity. Eleven of the 15 students across the three NGT groups made comments that began to demonstrate reflection in how the information was relevant to their educational and professional progress:

The perceived relevance of an educational experience or opportunity is a powerful facilitator to engagement and learning.^47,48 Students related better to the sessions where they could envisage the applicability to their future role and profession.⁴⁹

2. Perceived demands of the learning opportunity. A majority of the students (10 of the 15 students across NGT1-3) also displayed how their perceptions of their learning environment and what was expected from them affected their learning experience. The first NGT discussion revealed that some negativity from students from other groups toward the board game affected student motivation to attend and also engage:

This demonstrates that the explicit message of attendance was clear to pharmacy students; however this was counteracted by the implicit messages from the disengagement and negativity of other students. Some of the students claimed that knowing the aims, objectives and learning outcomes of the sessions could have made them more ready to engage and enrich their learning experience.

3. The self-concept of the learner. Students (nine of the 15 across NGT1-3) reflected on the level of challenge that each IPE session posed and related that to aspects of their self-concept. Students wish to view themselves as competent, self-directed, appropriately self-evaluative and exercising choice;^48,50 any phenomenon that attacks this may produce resistance and rejection.

There was some feeling that where the level of challenge was perceived too high, the enjoyment of the session was detrimentally affected.

The learning gap between what students think they know and what they think they need to know can stimulate learning through revealing learning needs and motivating learners to close the gap. However, if that gap is too large the student’s self-concept can be negatively affected and demotivation and dejection can result, which counteracts productive and engaged learning.^51,52

Conversely, some students found where this disjuncture existed, particularly in the SimMan^® session, they gained an appreciation of the extent and depth of knowledge they would one day be expected to possess. They valued this stark realization in knowledge differential toward gaining a better understanding of the pharmacist’s role and also in recognizing the journey of development they were travelling to achieve.

4. Links to prior learning. Lastly, most students (eight out of the 15 across NGT1-3) identified and appreciated where IPE sessions related to earlier experiences or learning within the curriculum. The foundations provided by the previous iteration should serve to support new learning, but also improve learners’ approach to IPE where they feel more comfortable.^48,53

DISCUSSION

The baseline data of this year 1 cohort of health professions students demonstrated a high level of preparedness and positivity toward IPE. This self-reported attitude and perception as assessed by the RIPLS and IEPS did not change significantly from the IPE sessions throughout the academic year. Researchers have acknowledged the self-reported nature of these two tools, which necessitates caution in interpreting their results as they may not be representative of actual interprofessional learning attitudes within a health care setting.^54,55 Further studies have suggested that there may need to be a significant differential between levels of exposure to IPE for these tools to be sensitive enough to detect a change in perception and attitudes.^41,56 Lie and colleagues conclude that no single scale may adequately record attitude change, and multiple strategies including qualitative measures should be incorporated to best study attitudinal change. Nevertheless, the reported positive attitudes toward IPE here can be considered as the optimum foundation for student engagement and motivation for learning within the experience.⁴¹

The NGT discussions showed a shift from mixed responses to more positive comments through the progressive IPE sessions. Also, there were initial responses within the first session (NGT1) that related mostly to the organization of the IPE sessions and the learner’s reaction, as they saw the session facilitator and the classroom setting as being very influential in their learning experience. After subsequent IPE sessions, responses became more abstract as students made reference to the development of their knowledge and skill, higher levels of outcome on the Kirkpatrick/Barr model, rather than the tangible external elements. This internalization of knowledge, skills, attitudes, and behaviors characteristic of the profession develop through a process of socialization and gaining experience in the practice setting. Students learn to become a member and practice using the aforementioned characteristics by participating in communities of practice.⁵⁷ Interprofessional education sessions and clinical placements, and interaction with others in their own and related professions, offer opportunities for such learning. The pattern of more developed outcomes was also reflected in the ranked comments about the IPE sessions, where levels of outcomes reported became more sophisticated in nature relating more to the impact on attitudes, knowledge, skills and behaviors. The final questionnaire allowed the generalizability of these comments to be tested within the entire cohort (Table 3). The levels of agreement or strong agreement were generally high (>50%) across all the statements except for the statement pertaining to the patient safety session: “We did not have enough background knowledge to tackle the tasks.” Only 35% of the cohort agreed with this statement, 22% were undecided, and 44% disagreed. This finding likely reflects the differences in academic ability/self-efficacy among the students rather than the level of challenge of the tasks.

Students enter the pharmacy degree program, for the most part, from a college or equivalent where teaching is traditionally didactic and directed by the educator. Higher education requires a shift toward more self-directed and learner-motivated learning styles, which can be quite a difficult transition for some students to navigate and adjust to. The structure of the IPE sessions was designed to accommodate this transition and thus commenced with an interactive lecture (the anatomy session). This involved transfer of knowledge from educator to student where students could then only be expected to be able to recall knowledge (“knows”), the lowest form of competence based on Miller’s triangle.⁵⁸ Subsequent sessions were based around small-group work where students became more self-directed, initially with an exploratory IPE game, then a facilitated patient case scenario, both of which required students to use individual or collaborative knowledge (“knows how”) in solving the issue at hand. The final session of simulated practice was pitched at the next stage of the triangle, and students were expected to “show how” to apply their knowledge and skill.

The three nominal group discussions yielded four themes in particular which related to principles of adult learning theory. These reflect how the students related to the learning opportunities presented to them in the IPE sessions. Arriving at these themes demonstrates how the NGT procedure has given the opportunity for students to reflect collectively among peers and also documents how students demonstrate the three dimensions of reflection described by Jay and colleagues.⁵⁹ They initially describe the matter for reflection – an attitude, behavior, or action within or as a consequence of the IPE session (descriptive dimension), the group discussion allows comparison of alternate views and perspectives of that same matter (comparative dimension), and establishment of a renewed perspective (critical dimension).⁵⁹ The data collection process has provided “a place, a space, and a time for reflection” as recommended by Clark, that is essential for transformative learning where students can think about their perspectives as well as those of others.⁶⁰

CONCLUSION

The RIPLS and IEPS were ineffective in providing any information on development of students’ readiness and motivation toward interprofessional working. However, we found that the NGT is an effective way to capture student experience and record growth in learning and behaviors. The nominal group discussions and prioritized statements provided students an opportunity to reflect on their experience. Students explored meaning and began to understand how their experiences influenced their plans and motivation for future learning and development. Sharing the process with peers seemed to facilitate the reflective capability of the students as ideas and perceptions were bounced off one another, refined, and reconsidered. This outcome would support the global use of reflective portfolios by students undertaking an IPE program, but with an added dimension of peer review and potentially assessment. In this way students would have individual longitudinal records of their journey, as they navigate their own expectations and emotions and the reactions to others, and appreciate these in the context of subsequent behaviors and dynamics. The peer-review component would allow students to revisit their experiences, compare and critique with others, and potentially renew and progress their understanding toward achieving transformative learning. If each student were able to engage with this process effectively, it could be an invaluable way to demonstrate students’ growth in professional identity, attitudes, and behaviors toward preparation for collaborative practice.

Appendix 1a. Logic Model for the Anatomy Session

Appendix 1b. Logic Model for the IPE Game Session

Appendix 1c. Logic Model for the Patient Safety Session

Appendix 1d. Logic Model for the SimMan^® Session

Appendix 2. Student Quotes from Specific Nominal Group Discussions that Relate to the Generated Themes