Impact of Interprofessional Telehealth Case Activities on Students’ Perceptions of Their Collaborative Care Abilities

Abstract

Objective. To describe the development, implementation, and assessment of simulated interprofessional education (IPE) telehealth case activities.

Methods. Faculty from pharmacy and physician assistant schools developed interprofessional cases covering topics addressed in both curricula and designed for specific levels of learners. Using a telehealth format, pharmacy students were paired with physician assistant students and met at specified times in a virtual room. Faculty representing both professions assessed students as they discussed a patient case, determined a diagnosis, and collaborated to develop appropriate treatment options. Pre-experience and post-experience data and student reflections were collected.

Results. Pharmacy students’ responses to items on the Team Skills Scale (TSS) completed after participating in four IPE telehealth case activities indicated positive changes. Mean total TSS scores significantly improved from pre-experience 62.3 (SD 8.4) to post-experience 72.6 (SD 5.7). Quantitative evaluation of student teams’ participation in an interprofessional activity was assessed using the Creighton Interprofessional Collaborative Evaluation (C-ICE) instrument and the average score was 90%. Theme analysis was performed on student reflections and the most prominent themes identified were satisfaction from interacting with other health care professionals, increased confidence in clinical decision-making ability, and affirmation that IPE telehealth cases should be included in each year of the curriculum.

Conclusion. Implementation of interprofessional cases using telehealth technology is an effective way for pharmacy schools to incorporate IPE into their curriculum. Students reported improved self-perception of interprofessional competence and attitudes toward interprofessional collaboration after participating in IPE telehealth cases.

INTRODUCTION

Reports from the Institute of Medicine reflect a long-held belief that interprofessional education (IPE) is an essential component for improving communication between health care providers, increasing patient safety, and optimizing patient care.^1,2 IPE presupposes that when health professionals from different disciplines work collaboratively, patient outcomes will improve. Various organizations and accrediting bodies encourage incorporation of IPE into the health professions’ curricula. The American Association of Colleges of Pharmacy, the Interprofessional Education Collaborative (IPEC), and the World Health Organization all support the goal of interprofessional collaborative practice as a means to improve the quality of health care.^3-5 The Accreditation Council for Pharmacy Education (ACPE) Accreditation Standard 11 recommends student exposure to, participation in, and demonstration of competence in interprofessional team dynamics, education and practice.⁶ Embedding IPE instruction into the didactic and experiential portions of the curriculum creates a practice model for students. Newly graduated health care professionals are expected to enter the workforce well prepared to provide team-based care through interprofessional collaboration. Repeated educational exposures help advance collaboration among developing clinical teams and improve the quality of communication between future practitioners.

Requiring IPE experiences for all students presents challenges for universities. Common barriers to IPE include psychosocial, cultural, structural, curricular, financial, and logistical complexities. Faculty must contend with increased workload and demands on their time, as well as limited resources.

The purpose of this yearlong pilot study was to develop, implement, and assess simulated interprofessional telehealth case activities to incorporate them into the Dispensing and Patient Care (DPC) pharmacy skills laboratory course.

Creighton University is a private, coeducational, Jesuit university. The School of Pharmacy and Health Professions offers a four-year Doctor of Pharmacy degree through a unique program comprised of both campus and distance pathways. The two cohorts complete didactic coursework and examinations simultaneously during the school year. However, campus students attend weekly DPC laboratory sessions concurrently with their didactic classes, while distance students complete intensive DPC laboratory sessions on campus during the summer months following their didactic coursework. When faculty are contemplating any programmatic change, parity between pathways must always be considered.

The DPC sequence consists of three mandatory, two- or three-credit hour lecture/laboratory courses offered to pharmacy students during the first three didactic years. Students participate in a variety of active-learning exercises, including objective structured clinical examinations (OSCEs) and simulated rounding scenarios in which faculty portray prescribing health care providers, assess student performance, and provide feedback.

While these are effective methods for pharmacy students to gather information, analyze a patient case, and make therapeutic recommendations, student feedback revealed that some did not consider them very realistic exchanges. Ideally, students from other health care disciplines would interact with pharmacy students in simulated practice experiences. Simulation is an IPE strategy to engage students as members of a team, thereby promoting communication, role exploration, and team development. Faculty hypothesized that the curricular addition of IPE activities, pairing pharmacy students with health care students from professions with prescriptive authority, would provide increased opportunities for students to improve their clinical decision-making abilities and enhance interprofessional communication skills in a more real-life format. The IPE activities were intended to augment, not replace, OSCE and simulated rounding activities.

METHODS

The first significant challenges encountered in planning IPE activities were finding a block of time when all faculty and students were free, and available rooms with enough space to facilitate IPE encounters. Using telehealth technology, multiple “virtual rooms” were created for interprofessional teams of faculty and students to meet at designated times. This solution minimized many of the existing roadblocks and allowed faculty to proceed with the study. Other universities have implemented virtual IPE learning environments with positive results.^7-9

Identifying an interprofessional health care partner was the next imperative. Because very few shared, open times were found when comparing pharmacy students’ schedules to those of other health care students within the university, faculty contacted a physician assistant (PA) program approximately 60 miles away to inquire about their interest in participating in an interprofessional telehealth program. This PA college did not have a school of pharmacy on its campus. Faculty from both schools met to determine the practicality of collaborating in this educational venture. The initial discussion established times during the semester when both pharmacy and PA students could meet and interact in simulated case studies.

Faculty designed versatile interprofessional cases that could be integrated into each course in the DPC sequence, simulating diverse health care settings (eg, community pharmacy, hospital, ambulatory care clinic, etc.), and varying in complexity to correspond with the clinical abilities of first-, second-, or third-year pharmacy students. For example, obtaining a patient medication history and transcribing a prescription were activities reinforced multiple times in the first year DPC laboratory sessions. Similar scenarios were embedded in the interprofessional telehealth cases, and students were assessed on their ability to apply these skillsets. Second-year students obtained medication histories and provided clinical recommendations. Third- and fourth-year students obtained medication histories that included intricacies requiring the student to determine prescription and non-prescription treatments or suggest an appropriate therapeutic substitution if the patient had problems with the recommended treatment. All PA students were in their second didactic year (PA2) and had completed one year of foundational courses and one semester of clinical medicine courses. Their knowledge base was comparable to a second- to third-year pharmacy student. PA faculty participated in creating and reviewing the cases to ensure goals and learning objectives for their students were being met, that topics/diseases incorporated into the cases had been covered in the curriculum, and the level of difficulty was appropriate for their students.

After determining availability of participants in both programs, four pilot sessions with different levels of learners were scheduled throughout the year during the fall and spring semesters (Figure 1). A user-friendly web-deliverable format, Cisco WebEx (Milpitas, CA), was chosen to host the virtual rooms, allowing each student to log on with few technology issues. PA faculty set aside class time for students to practice logging onto a session, and to ensure each laptop’s video camera and microphone were working properly. Although the PA school was not affiliated with Creighton University, WebEx enabled shared use between both institutions. Clinical recommendations were assessed in real-time, while session recordings permitted faculty to re-watch captures and evaluate interprofessional team interactions.

Figure 1.

Timeline and Process for Yearlong IPE Telehealth Case Study Session

Six virtual rooms were used for the first iteration of the pilot (P1/PA2 session), each moderated by a designated pharmacy faculty member who was present to answer questions and keep the telehealth cases on task throughout the 75-minute time block. Thirty student teams rotated through six faculty rooms in 15-minute sessions. Two PA faculty participated and floated between the six virtual rooms. Each virtual room used a different case to discourage cross-talk among students (Room/Case 1: drug-induced cough, Room/Case 2: serotonin syndrome, Room/Case 3: asthma, Room/Case 4: hypertension, Room/Case 5: depression, and Room/Case 6: COPD). Pharmacy and PA student teams met in a designated virtual room during one of thirty 15-minute time slots to collaborate on a patient case. A unique session code, assigned by pharmacy faculty, was required to enter a virtual room. These codes were individually sent to students prior to the meeting. Paired pharmacy and PA students logged on with their session code and entered the virtual room at their appointed time. The pharmacy faculty moderator and PA faculty listened to the students discuss the case, establish a diagnosis, and work collaboratively to determine appropriate treatment and dosing. Cases followed a “vignette” format that allowed concise, straightforward presentation of critical information that students required to arrive at a diagnosis and to formulate a basic treatment plan within a very limited timeframe. The moderator supplied necessary case details and the interprofessional team analyzed the problem and discussed medication options. If students needed guidance, pharmacy faculty moderators provided additional medical information or prompts. Faculty assessed P1 students on their ability to obtain a medication history and accuracy of transcribing a prescription and PA students on diagnosis and treatment.

The P2, P3, and P4 sessions became progressively more complex. Eight virtual rooms, with one pharmacy faculty moderator in each, were created for the P2/PA2 session. Pharmacy and PA student pairs were scheduled in each room every 20 minutes, for a total of 30 team case discussions in the 80-minute time block (two rooms had three teams). Eight distinct cases involving weight-based dosing were developed for the P2/PA2 teams. All cases involved infants/children presenting with symptoms consistent with otitis media. Embedded imagery equipped the PA student to view the child’s ear with an otoscope and assess eyes, nasal passages, and throat. The team established a diagnosis, selected an appropriate antibiotic and pain reliever, then determined an accurate weight-based dose for each product.

In the P3/PA2 session, students encountered complex cases in which the patient had multiple issues requiring a prescription, non-prescription and non-drug therapy recommendations (eg hospitalized diabetic patient, emergency room patient in extreme pain, and a pediatric patient with a fever and rash). Several medication choices were appropriate for each case, but there were also inappropriate medication selections that could cause patient harm.

The P4/PA2 cases required the student teams to provide a diagnosis, treatment recommendation and a therapeutic alternative. Unique cases were developed for each virtual room (benign prostatic hyperplasia, urinary tract infection, hypercholesterolemia, and strep throat). In these scenarios, after they provided a diagnosis and treatment recommendation, the faculty moderator informed the student teams their patient had an issue requiring a therapeutic substitution (ie, an allergy to the recommended medication, no insurance coverage, therapy that would cause a major drug interaction with what the patient was currently taking).

In all cases (P1/PA2, P2/PA2, P3/PA2, P4/PA2), faculty used a rubric to evaluate real-time clinical decision-making and provided feedback to the teams. Each session was recorded, and faculty assessed team interactions using the Creighton Interprofessional Collaborative Evaluation (C-ICE) instrument.¹⁰

Assessment of IPE team performance is necessary to evaluate students’ ability to provide health care when IPE is an integral component of quality care. The C-ICE instrument (Creighton University, Omaha, NE) was developed so faculty evaluators could efficiently and effectively assess interprofessional performance during a team activity. The instrument is specifically designed to provide a quantitative evaluation of student teams’ participation in an interprofessional activity. It is comprehensive in nature, linked to the IPEC core competencies⁴ (Values/Ethics for Interprofessional Practice, Roles/Responsibilities, Interprofessional Communication, Teams and Teamwork). It is designed to evaluate group or team performance, rather than individual student performance. It may be used in a variety of interprofessional activities, live or simulated environments, and with students from beginning to advanced levels. The C-ICE is a 25-point dichotomous tool in which the evaluator awards one point if the interprofessional team demonstrates competency in a specific area, or no point for failure to do so. There is also a “not applicable” option. Faculty provide feedback to the student teams regarding team strengths, weaknesses, and areas for improvement.

Pharmacy and PA students were invited to participate in a pre- and post-Team Skills Scale (TSS) survey. To examine changes in attitudes over time, a paired sample t-test was used on TSS scores obtained for each of the four piloted interventions.¹¹ We used SPSS Inc. Version 22.0 (Armonk, NY) for all analyses. The TSS survey was completed one week prior to and one week after each of the telehealth case studies. The TSS measures perception of capabilities for effective team interactions and consists of 17 questions which use a 5-point Likert scale (1=poor to 5=excellent) to assess interprofessional team skills.¹² The TSS has been routinely used at our university for some time, is well documented in the literature, and is applicable across a variety of disciplines.¹³ Results of a study by Packard and colleagues demonstrated the TSS may be appropriate for one-time IPE experiences.¹⁴ Students who participated in one IPE telehealth session were asked to reflect on their experience after faculty received several unsolicited emails from students who wanted to share details of their interprofessional encounter. In case study sessions P2/PA2, P3/PA2, and P4/PA2, additional questions were asked pertinent to interprofessional team interactions. The Creighton University Institutional Review Board assigned exempt status to this study.

RESULTS

There were 172 students who participated in the four pilot sessions scheduled throughout the year (P1/PA2 Session 1, n=44; P2/PA2 Session 2, n=44; P3/PA2 Session 3, n=44; P4/PA2 Session 4, n=40). This study used a mixed-methods approach to assess both quantitative and qualitative data. All data were examined using descriptive statistics. Students were evaluated on their clinical knowledge and interprofessional team skills. Although the pilot sessions were not graded, instructors used answer keys and faculty-developed rubrics to assess student knowledge and to provide feedback to each team of students. Faculty reviewed recorded telehealth sessions to assess interprofessional interaction and communication with the C-ICE instrument. The TSS was utilized to measure students’ perceptions.

Because the P1 students only had foundational pharmacy courses and did not yet possess clinical knowledge, and the second-year PA students had never been involved in an IPE collaborative scenario, the first case was “graded” as a pass/fail assignment. Thirty teams completed the case studies. Fourteen P1 students provided a correct medication history and submitted an appropriate written prescription. Four of these pharmacy students participated in multiple cases. All of the pharmacy students passed the exercise. Twenty-three PA students provided a correct diagnosis/appropriate therapy and passed, while seven students earned a failing score after an incorrect diagnosis.

The otitis media case (P2/PA2) was scored using a 12-point rubric. The pharmacy section, based on drug knowledge and appropriate dosing, comprised three points, while the physician assistant portion, centering on the physical examination, was allotted seven points. One point was awarded for teamwork (active listening and incorporating information) and one point for communication (verbal and nonverbal). Thirty teams completed this second iteration of the pilot, including 14 pharmacy students (five completed multiple cases) and 30 PA students. The average total score for this exercise was 89%. In their respective sections, pharmacy students averaged 2.8/3 (92%) and physician assistant students averaged 5.9/7 (84%).

The (P3/PA2) cases were complex and required multiple recommendations. The 6-point rubric awarded one point for each correct diagnosis and one point for each appropriate medication and non-drug therapy recommendation. A score of zero was assigned to any team that recommended a dangerous medication combination that could result in patient harm. Thirty teams completed this series of case studies, with 14 pharmacy students (eight completed multiple cases) and 30 PA students. The average total score for this exercise was 88%. In their respective sections, pharmacy students averaged 2.7/3 (89%) and physician assistant students averaged 2.6/3 (88%).

Twelve P4 students and 28 PA students participated in the final session (P4/PA2). Six pharmacy students completed multiple cases. These complex cases required health care teams to determine a diagnosis and treatment, followed by selection of an appropriate therapeutic alternative. Points were deducted if the prescribed product could cause patient harm. The 4-point rubric awarded two points for an accurate diagnosis and treatment regimen and two points for a safe, correct therapeutic substitution. The average total score for this exercise was 90%. The average for the correct diagnosis and treatment was 1.8/2 (88%) and the average for appropriate therapeutic substitution was 1.9/2 (93%).

There were 162 students who voluntarily completed both the pre- and post-TSS survey (94% response rate) (Table 1). Mean (SD) pre- and post- scores were compared using paired students’ t-tests. Mean total TSS scores significantly improved, pre- 62.3 (8.4) and post- 72.6 (5.8), p<.001. There were statistically significant improvements for all independent TSS questions after the case intervention. Table 2 lists the average scores for all four sessions. The subsequent offerings all produced statistically significant improvements for all independent TSS questions after the case intervention and mean total TSS scores (Table 2).

Table 1.

Team Skills Scale Survey Mean Scores for All Sessions (n=162)

Table 2.

Interprofessional Groups Mean Total TSS Scores Pre- and Post-Case (n=162)

In the latter pilot cases (Sessions 2 through 4), students were asked additional survey questions. There were 122 students who responded (95% response rate). Students were asked the format of case delivery they preferred. Pre-survey, students indicated they favored live interactions (52%) over telehealth (33%) and telephonic (15%) exchanges. Post-survey, this changed to telehealth (53%) over live (41%) or telephonic (6%) sessions. Notably, the number of students who indicated a preference for the telehealth format increased by 20% from the pre- to post-survey. Over 90% of students (n=110) responded that they would be interested in participating in additional offerings of interprofessional telehealth cases. Ninety-five percent of respondents (n=116) believed that being provided opportunities to work with other health care providers will benefit them in their future professional careers.

Faculty assessed team interactions using the newly developed Creighton-Interprofessional Collaborative Evaluation (C-ICE) instrument which measures interprofessional student team performance.¹⁰ Over the 12-month period, 64 team interactions were recorded and were assessed with the C-ICE instrument. Twenty-two teams were not recorded because students chose not to sign the permission form (two teams), technical difficulties were present (eight teams), or faculty did not turn on the record feature (12 teams). The pass score was set at 75%. Two teams did not meet the 75% minimum. The average score was 90%.

Qualitative data were collected from Creighton student reflections and theme analysis was conducted by a group of pharmacy faculty members with expertise in interprofessional education. The written reflections were collated and analyzed using a modified method delineated by Braun and Clarke.¹⁵ Faculty initially coded data using broad groupings or categories. The relevant data extracts were sorted according to overarching themes. A deeper review was performed to determine whether to combine, refine, separate, or discard initial themes. The last step of the process involved analyzing the grouped data.

Twenty-four pharmacy students voluntarily submitted reflective feedback. The most prominent theme was satisfaction from interacting with other health care professionals. Pharmacy students also repeatedly stated that these case activities increased their confidence in their clinical decision making. A final recurring theme was the affirmation that these IPE telehealth cases should be included in each year of the curriculum. Sample reflections are included in Appendix 1.

DISCUSSION

This pilot study was intended to immerse students in a “real-life” experience that encouraged interprofessional teamwork and communication while providing safe patient care. Partnering with a prescribing school without a pharmacy program proved tremendously valuable. Faculty worked collaboratively to develop and schedule cases, assess student teams and provide feedback. The joint case development sessions sometimes unearthed interesting discoveries that led to mutually beneficial adjustments in the cases. For example, PA faculty noted their second-year students had not yet phoned a prescription into a pharmacy. Consequently, all cases for PA students and P1 students incorporated a simulated activity in which PA students phoned in a prescription order to be transcribed by the pharmacy student.

A significant advantage associated with the telehealth platform was the potential for Creighton distance-based students from across the United States to participate in real-time IPE case studies from their own homes. Very few technical difficulties arose because practice “log-on” sessions were used to confirm each student’s microphone and camera were working and that students could connect to the WebEx virtual rooms. Student feedback for each iteration was extremely favorable and most viewed this as a positive learning experience.

After the initial round of cases, faculty noted that a few students assigned to later time slots logged into a virtual room before the session in progress was completed. Moreover, it was determined that 15 minutes allowed insufficient time for faculty moderators to thoroughly assess student recommendations and provide detailed feedback. To remedy these problems, students were encouraged to join the virtual room no earlier than their scheduled time and five additional minutes were added to subsequent cases. Consequently, the number of virtual rooms was increased from six to eight to expeditiously accommodate 30 interprofessional teams.

During the first session, two students refused to sign the permission form allowing them to be recorded. Subsequently, faculty provided a more detailed explanation to students regarding the purpose of the recordings, noting they would only be viewed by faculty who were assessing team interactions. Since then, we have not had a student decline to be recorded. In the event of a future declination, the C-ICE may be used for real-time grading.

Creating multiple cases for each pilot session and for multiple levels of student learners was time consuming for faculty. Six to eight cases were created for each of the four sessions. To limit students’ ability to pass case details to other students scheduled in later time slots, multiple, unique scenarios were created. Adjusting case complexity allowed P1 through P4 students the opportunity to participate at a grade-appropriate level. This strategy required significant effort on the part of both pharmacy and PA faculty.

If students participate in the telehealth case studies for a course grade, security of the cases becomes a concern. In the pilot sessions, each room had its own distinct case. Students did not know which virtual room they would enter, nor which case they would be receiving. For future sessions, a different case will be assigned for each time slot to prevent talk or passing of answers among students. For example, all student teams from 8:00 am to 8:20 am will have the hypertension case. All students from 8:20 am to 8:40 am will have the diabetes case. Even if students in an early session speak with a group following them, there will be enough unique cases to prevent passing of answers.

Students recruited for the first session of the pilot project could participate in subsequent offerings, and TSS survey responses were noticeably higher with each iteration. This could be due to students becoming increasingly confident in their abilities and communication skills throughout the year; however, there should be a ceiling effect. Additionally, students who volunteered were interested in IPE and in performing live cases with other health care practitioners. They were motivated to participate and had a high level of engagement, so this experience may have been more positive for them because of their initial interest.

A challenge in using the C-ICE instrument is the focus on team versus individual performance. Although student teams are assigned a shared numerical score, the qualitative dialogue included on the tool provides valuable individual feedback. The qualitative discussion constitutes an assessor’s justification for the assigned score. Typical feedback might include, “Pharmacy student: you did not define the discipline specific term ‘AUC.’” “PA student: you did not consider patient-specific circumstances, such as no insurance.” The C-ICE evaluates learning outcomes related to Kirkpatrick Level 3 behavioral change, and is one of very few IPE assessment tools with demonstrated validity and reliability linked to the IPEC Core Competencies. While not perfect, it provides students with useful quantitative and qualitative feedback to develop their IPE skills. The goal is to better prepare students for interprofessional activities they will encounter in their advanced pharmacy practice experiences, at which time they will be assessed on an individual basis.

Twenty teams were not assessed with the C-ICE instrument because of “operator error.” The technical difficulties experienced with eight teams related to one faculty member’s personal computer malfunctioning, not problems with the WebEx platform. Twelve teams were not recorded because faculty forgot to engage the record function. A specific intent of the pilot sessions was to address problems and mistakes before integrating these cases into a course for a grade.

The cost for our school to design and implement this project was minimal. Faculty created cases during available free time and they can be reused next year since participants in the pilot will have advanced one grade level. New cases are being developed as time permits. Faculty from both universities volunteered as moderators, so no stipend was paid. Sessions were scheduled well in advance to ensure adequate faculty facilitators were present for the cases. WebEx is a videoconferencing platform already used by our university, so there was no additional cost to use this tool. If a school cannot afford a teleconferencing system, other free or low-cost video chatting/web conferencing applications are available.

The pilot sessions were very successful. Students were able to join the virtual rooms, review a patient case, make a diagnosis and recommendation, and receive feedback on their performance. Moving forward, these activities will be incorporated into a course for a grade. Partnerships are now developed with two colleges of nursing, allowing interprofessional experiences pairing pharmacy and nurse practitioner students. Cooperative endeavors among faculty from each health profession ensure case content is appropriate, and diagnosis and treatment recommendations are achievable for each level of learner. Future objectives include development of IPE experiences with medical students and the addition of standardized patients to the cases to enhance engagement opportunities.

This type of program could easily be implemented in other colleges and schools of pharmacy. For schools not associated with a medical center, this report demonstrates how positive collaborative partnerships can be formed to provide simulated team experiences. These interactive case study discussions fulfill many of the ACPE standards relating to IPE.

CONCLUSION

Implementing simulated interprofessional telehealth case studies was a practical way to address IPE requirements and positively affected pharmacy students’ self-perception of interprofessional competence. The design eliminates many barriers to interprofessional education. Other schools could easily adopt and implement such an experience as it meets ACPE accreditation standard guidelines.

Appendix 1. Select Student Reflections

I loved working with PA students and being there as a team member as they thought through a diagnosis and what treatment they would use for the patient. I was surprised at how much I knew. It was a great learning experience and a confidence booster. I am able to help with other cases, so please contact me if you need volunteers.

  All I can say is “why haven’t we done these every year?” I was worried about participating and found out it wasn’t as bad as I thought. I felt like we were there as a team to help each other out. I think this made the experience worth doing. If all health care practitioners worked together like this, what a benefit it would be to our patients.

  I loved participating in this! I think the idea of interdisciplinary activities like these is great preparation for all involved and gives good practice with real-life situations. Seeing how different disciplines think about the case was interesting. These activities were beneficial and should be in all years of the curriculum.