Abstract
Background:
In order to prepare current and future educators and clinicians to lead interprofessional education (IPE) and interprofessional collaborative practice (IPCP), faculty and staff need training in collaborative approaches to developing, implementing, assessing, and sustaining high quality IPE across the interprofessional learning continuum. The Train-the-Trainer Interprofessional Team Development Program (T3-ITDP) is a 3.5-day program designed to develop expert IPE teams through interactive workshops, coaching, and the development and implementation of an IPE or IPCP (IPECP) project for their home institutions.
Purpose:
The purpose of this research was to assess the impact of the T3-ITDP on the development and implementation of IPECP projects by participating teams.
Methods:
The T3-ITDP impact survey was created and administered to collect data on the scope and impact of participant teams’ projects, including learner and project outcomes, training methods, dissemination plans, assessment strategies, and teams’ intentions to continue working together beyond the initial project. With human subject’s approval, we invited 55 T3-ITDP participant teams to complete the impact survey. These teams were at least one year post-completion of the in-person portion of the program and thus had time to initiate their IPECP projects.
Results:
Forty-one (74.5%) teams responded to the survey. Of those teams, 31 (76%) used T3-ITDP content and/or approaches to develop their IPECP projects that targeted learners across the interprofessional learning continuum. Sustainability of IPECP projects was supported through several mechanisms, including institutional support or incorporating IPECP activities into existing courses. Almost half of the teams worked together on new projects, and 74% of teams planned to repeat a newly developed activity.
Discussion & Conclusions:
Results of the T3-ITDP impact survey demonstrated that team-based, project-focused professional development catalyzed the development, implementation, and sustainment of new IPECP projects at academic and community institutions throughout the U.S.
Keywords: Interprofessional Education, Interprofessional Collaborative Practice, Train-the-Trainer, Team Science, Faculty Development
1. Introduction
Improving patient outcomes and achieving the Institute for Healthcare Improvement’s (IHI) Quadruple Aim are explicitly linked using interprofessional collaborative practice (IPCP) approaches among healthcare teams.1–3 Healthcare teams that employ effective IPCP approaches share numerous characteristics such as good team communication, patient-centered care, and shared responsibility.1 Other benefits of effective IPCP teams include improved patient experiences and work satisfaction among healthcare professionals.4 For these reasons, healthcare professionals across the interprofessional learning continuum from students to continuing professional development5 should be trained to meet the Core Competencies for IPCP established by the Interprofessional Education (IPE) Collaborative.6
IPE is an effective approach to teach and develop IPCP core competencies.2 Faculty traditionally educate profession-specific learners rather than interprofessional groups of learners.7,8 Developing faculty in IPE prepares them to lead learners in the development of knowledge and skills to work collaboratively, provide optimal patient care, and improve healthcare outcomes upon entering practice.9,10 Faculty development is essential since faculty with diverse attitudes and values may inadvertently model behaviors that do not support IPE or IPCP.10
The field of faculty development in the health professions has grown substantially over the last decade.11 Successful models are often characterized by project-based interactive educational methods underpinned by adult learning theories.10–14 These include evidence-informed design principles, experiential and peer learning, feedback and reflection, coaching, opportunities for practice and application, projects, longitudinal program design, and institutional support.1,10,11,14 In addition, team- and project-based interprofessional faculty development creates the opportunity to build communities of practice among program participants and in the workplace.10 Thus, faculty benefit from being educated in interprofessional teams that provide an opportunity to practice interprofessional collaboration.7
A challenge in faculty development for IPE or IPCP (herein referred to as IPECP) has been expanding access to training and coaching to help interprofessional teams develop the knowledge and skills needed to create, implement, and sustain IPECP at their institutions. The Train-the-Trainer Interprofessional Team Development Program (T3-ITDP) was created to help close this gap. Starting with a pilot study, this program has been found to be successful at improving IPECP knowledge and skills among participants as well as in supporting them to become leaders and champions of IPECP at their institutions.10,15
To better understand the impact and scope of IPECP projects developed and implemented by participating T3-ITDP teams, a survey was developed and administered in 2018 to all teams who completed the program. The purpose of this paper is to describe survey results and to assess the impact of the T3-ITDP on the development and implementation of IPECP by participating teams at their home institutions.
2. Methods
2.1. The T3-ITDP Program
The T3-ITDP program is a national year-long training program for teams wanting to learn how to develop and implement effective IPE and IPCP projects at their respective institutions.15 The overall goal of the T3-ITDP is to develop expert IPE teams utilizing IPECP projects as a vector for learning. This is achieved through evidence-informed educational design principles, interactive sessions, experiential and peer learning, feedback and reflection, coaching, practice and application opportunities, and longitudinal program design.15 Teams of three or more members representing two or more professions complete pre-work, an in-person 3.5 day training, and then work together over the course of a year to develop and implement a collaborative IPECP project. Team size typically ranges from three to six members and, while individuals from over a two dozen professions have participated, the professions most commonly represented are nursing, medicine, pharmacy, or social work.15 Existing T3-ITDP program evaluation includes a longitudinal series of three surveys completed by each participant that focus on T3-ITDP program effectiveness, individual learning/development, and team dynamics over the course of the program year. Initial training sites were the University of Virginia (UVA), University of Washington (UW), and University of Missouri (MU). Currently, trainings are offered by UVA and the University of Texas at Austin.
2.2. Study participants and ethical considerations
Participants in this study were T3-ITDP teams that attended an in-person program at one of the original three training sites between November 2015 and August 2017. These cohorts were selected, so that study participants were at least one-year post-completion of the in-person training to allow time for project implementation. A total of 58 teams from 10 cohorts across the three training sites completed the in-person training, longitudinal coaching, and project implementation and thus were eligible to participate in this cross-sectional descriptive survey. Each team identified a “team lead” who could complete the survey on behalf of the group. T3-ITDP teams that agreed to participate in the study (n=55) were emailed instructions to complete the impact survey. The study was determined not to be human subjects research by the UW Institutional Review Board.
2.3. Data Collection
A cross-sectional study design was used, and data was collected via a web-based REDCap survey for four weeks in July and August 2018. During the data collection period, weekly reminders were sent to team leads that had not yet completed the survey.
The T3-ITDP impact survey was developed by the T3-ITDP evaluation team to collect data about team projects that were not captured by existing program evaluation activities. The survey consisted of twenty-seven questions. Twenty-three questions were multiple choice, and the remainder were short answer. The majority of multiple choice questions included the option to “select all that apply” as well as an “other” option. When “other” was selected as an answer option, respondents then had the opportunity to provide an alternate answer or additional detail. Because many survey items were “select all that apply”, cumulative percentages above 100% occur frequently. A copy of the survey is available in Appendix I.
2.4. Data Analysis
Descriptive statistics were utilized to summarize the survey results. Chi square and Fischer’s exact tests were performed to compare variables across the three training sites (UW, UVA, MU); this step did not yield any significant differences across sites. As a result, findings are reported at the aggregate program level as opposed to the T3-ITDP site.
3. Results
3.1. Survey Respondents:
Forty-one (74.5%) T3-ITDP teams completed the survey. Seventeen teams had participated in the T3-ITDP training at UVA, fifteen at UW, and nine at MU.
3.2. Characteristics of Activities Implemented by T3-ITDP Participant Teams
General themes of activities implemented by T3-ITDP participant teams included faculty development or training, interprofessional simulations, incorporation of learners into collaborative practice or community settings, implementation of new team-based care processes, and interprofessional curricular innovation. A list of exemplar project topics for different types of learners is provided in Table 1.
Table 1:
Examples of Project Foci Across the Interprofessional Learning Continuum
| Curricular Innovations in Foundational or Graduate IPE |
|
|
| • Creation and implementation of a foundations IPE seminar |
| • IPE disaster simulation |
| • IPE case conference course |
|
|
| Continuing Professional Development: IPE for Faculty |
|
|
| • Developing an interprofessional faculty development program |
| • Increasing health science faculty engagement in IPE |
| • Interprofessional faculty facilitator recruitment and training |
|
|
| Continuing Professional Development: IPE for IPCP |
|
|
| • Interprofessional collaborative practice model in primary care |
| • Creation of an interprofessional center of excellence on an inpatient cardiovascular unit |
| • Improving interprofessional collaboration on Family Centered Rounds |
In terms of characteristics of activities implemented by T3-ITDP teams (Table 2), the majority held either single stand-alone training sessions (12, 29.3%) or “other” training (13, 31.7%). For teams choosing “Other”, respondents developed and implemented faculty development programs, facilitated creation of multiple IPE activities through curriculum transformation, worked to develop modules for grant teams or meet grant objectives, utilized clinical time, and evaluated qualitative assessments in IPE. Two respondents stated their team projects were never implemented. The balance of teams implemented a single session within an existing course (2, 4.9%), multiple sessions within an existing course (8, 19.5%), or multiple stand-alone sessions (10, 24.4%). The majority of T3-ITDP teams plan to hold their activity again in the future: Yes (31, 75.6%), No (3, 7.3%), Unsure (6, 15.0%).
Table 2:
Characteristics of activities implemented by T3-ITDP participant teams
| Types of Activities | N (%) |
|---|---|
| Single session within an existing course | 2 (4.9) |
| Single session stand-alone trainings | 12 (29.3) |
| Multiple sessions within an existing course | 8 (19.5) |
| multiple sessions stand alone | 10 (24.4) |
| “other” trainings | 13 (31.7) |
|
| |
| Plan to hold activity again in the future | |
|
| |
| Yes | 31 (75.6) |
| No | 3 (7.3) |
| Unsure | 6 (15.0) |
|
| |
| Total contact hours of activities | |
|
| |
| 0–2 hours | 4 (9.8) |
| >2 hours to 4 hours | 11 (26.8) |
| >4 hours to 8 hours | 6 (14.6) |
| >8 hours to 20 hours | 5 (12.2) |
| >20 hours to 40 hours | 6 (14.6) |
| >40 hours to 80 hours | 3 (7.3) |
| >80 hours | 1 (2.4) |
| Other | 4 (9.8) |
|
| |
| Was activity required for learners? | CPD vs pre-professional |
|
| |
| Yes | 8 (40.0) vs 5 (27.8) |
| No | 12 (60.0) vs 8 (44.4) |
| Other | 0 (0.0) vs 5 (27.8) |
The total contact hours for activities varied among T3-ITDP teams and ranged from: 0–2 hours up to and exceeding 80 hours. The most common length of time was 2–4 hours (11, 26.8%), with activities that were more than 4 hours and up to 40 hours representing the next most common amount of time for an activity. For many of the projects, the activity was not required, with 60% of activities offered on an elective basis for those seeking continuing professional development (CPD), and 44% of activities offered on an elective basis to foundational/pre-professional learners.
T3-ITDP participant teams have been able to sustain their projects through several mechanisms, including grant funding (6, 15.0%), additional funding (non-grant) (4, 10.0%), institutional support (24, 60.0%), and integrating content into a course or other recurring activity (15, 37.5%). Examples of teams’ institutional support included funding for additional teams to attend the T3-ITDP training, administrative support, or funding incentives for participants such as food at the event. A high percentage of teams (16, 39%) received support from two or more places to sustain the project. The session type (single vs. multiple) or having the session within a course versus a stand-alone activity did not seem to impact sustainability of team projects.
3.3. Learners in T3-ITDP Activities Implemented by T3-ITDP Participant Teams
Learner characteristics are presented in Table 3. Pre-professional learners were the most common target audience for T3-ITDP activities (22, 53.7%). However, 19 teams (46.3%) had multiple levels of learners within the activity. The most common health professions involved in the activities were nursing (32, 78.0%) followed by medicine (24, 58.5%) and pharmacy (20, 48.8%), with close to three dozen professions represented overall.
Table 3.
Characteristics of learners who participated in activities offered by T3-ITDP teams.
| Level of Learners in Activity | N (%) |
|---|---|
| Foundational Education (i.e., pre-licensure health professions students) | 22 (53.7) |
| Graduate Education (i.e. graduate students or residents) | 16 (39.0) |
| Continuing Professional Development (e.g., practicing clinical teams and/or faculty) | 20 (48.8) |
| Other | 3 (7.3) |
|
| |
| Types of Learners Reported by T3-ITDP teams | |
|
| |
| Dentistry | 7 (17.1) |
| Dietetics | 5 (12.2) |
| Healthcare Administration | 5 (12.2) |
| Medical Assistant | 0 (0.0) |
| Medicine | 24 (58.5) |
| Nursing | 32 (78.1) |
| Occupational Therapy | 11 (26.8) |
| Pharmacy | 20 (48.8) |
| Physical Therapy | 14 (34.2) |
| Physician Assistant | 10 (24.4) |
| Respiratory Therapy | 5 (12.2) |
| Social Work | 18 (43.9) |
| Other | 24 (58.5) |
|
| |
| Numbers of Learners Reported by T3-ITDP teams | |
|
| |
| 0 to 25 | 17 (41.5) |
| 26 to 50 | 11 (26.8) |
| 51 to 100 | 5 (12.2) |
| 101 to 200 | 3 (7.3) |
| > 200 | 3 (7.3) |
Additionally, the number of learners was usually less than 50 per activity, with zero to 25 learners being most common (17, 41.5%), followed by 25 to 50 learners (11, 26.8%).
3.4. Training and Adaptation Strategies of T3-ITDP Participant Teams
An array of evidence-based and theory supported training strategies were utilized by T3-ITDP participant teams (Table 4). Small group discussions were most common (23, 56.1%), followed by reflective exercise and simulation (both at 18, 43.9%). Many of the teams chose multiple training strategies when developing and implementing their activities. Ten teams (24%) employed 4 training strategies, 7 teams (17%) chose 3 strategies, and 5 teams (12%) chose 2 training strategies. A high percentage of projects (≥72%) were sustained regardless of the training strategy selected by the teams.
Table 4.
Training strategies utilized by T3-ITDP participant teams.
| Training Strategies | N (%) |
|---|---|
| Small group discussion (e.g. patient case discussion) | 23 (56.1) |
| Project-based (e.g. group project) | 11 (26.8) |
| Facilitated Workshops (e.g., active learning, liberating structures) | 15 (36.6) |
| Large group lecture | 6 (14.6) |
| Clinical teaching/direct patient interaction | 9 (22.0) |
| Reflective exercises | 18 (43.9) |
| Simulation | 18 (43.9) |
| E-learning (e.g., online modules) | 12 (29.3) |
| Shadowing | 2 (4.9) |
| Community-based project | 6 (14.6) |
| Other | 4 (9.8) |
|
| |
| Adapted Content from T3-ITDP program into training or evaluation strategies? | |
|
| |
| Yes, training | 31 (75.6) |
| Yes, evaluation | 14 (34.2) |
| No | 4 (9.8) |
| Other | 4 (9.8) |
|
| |
| Willingness to Share with other T3-ITDP teams | |
|
| |
| Yes | 33 (80.5) |
| No | 4 (9.8) |
| Other | 4 (9.8) |
When implementing projects at their home institutions, most T3-ITDP teams adapted content from the in-person program for training or evaluation strategies (31, 75.6% training; 14, 34.1% evaluation). Teams were also very willing to share content developed with the community of T3-ITDP learners (33, 80.5%).
3.5. Assessment and Outcomes of Activities Implemented by T3-ITDP Participant Teams
When examining assessment strategies and outcomes targeted by T3-ITDP participant teams (Table 5), a layered or multi-method approach was taken by a number of teams. Multiple teams targeted two or more learning and health and system outcomes. The majority of T3-ITDP teams (78%) also utilized two or more assessment strategies to asses those outcomes. The most frequent types of assessment strategies were surveys (34, 82.9%), followed by session or program evaluation (19, 46.3%), reflection (18, 43.9%), and observations (15, 36.6%).
Table 5.
Assessments strategies and outcomes utilized by T3-ITDP participant teams
| Assessment Strategies | N (%) |
|---|---|
| Surveys (e.g., attitude, perceptions, self-report of change such as in knowledge or behavior) | 34 (82.9) |
| Knowledge tests | 6 (14.6) |
| Observations | 15 (36.6) |
| Interviews/focus groups/debriefs | 9 (22.0) |
| Reflection | 18 (43.9) |
| Skill performance ratings (e.g. TOSCE) | 5 (12.2) |
| Checklist | 5 (12.2) |
| Session or program evaluation/feedback | 19 (46.3) |
| Other | 5 (12.2) |
|
| |
| Types of Learning Outcomes Targeted | |
|
| |
| Reaction | 8 (19.5) |
| Attitudes/perceptions | 33 (80.5) |
| Knowledge/skills | 27 (65.9) |
| Collaborative behavior | 29 (70.7) |
| Performance in practice | 13 (31.7) |
|
| |
| Types of Health and System Outcomes Targeted | |
|
| |
| Individual Health | 19 (46.3) |
| Population/public health | 13 (31.7) |
| Organizational change | 25 (61.0) |
| System efficiencies | 14 (34.2) |
| Cost effectiveness | 6 (14.6) |
Respondents were asked to describe learning outcomes based on the Kirkpatrick model.16 Attitudes/perceptions was the most common outcome measured (33, 80.5%), followed by collaborative behavior (29, 70.7%) and knowledge/skills (27, 65.9%). Multiple learning outcomes were being targeted by 88% of teams, with 2 and 3 outcomes targeted most frequently at 32% and 34%, respectively.
Finally, when evaluating types of health or system outcomes targeted by T3-ITDP teams, the most frequent response was organizational change (25, 60.9%) followed by individual health (19, 46.3%). Just over half of teams (53.7%) chose to evaluate more than one health or system outcome. In comparison, 39% evaluated one outcome. Organizational change was more frequently the target system outcome for CPD learners (85%) compared to when graduates (56.3%) or foundational learners (40.9%) were the target audience. Activities designed for foundational learners or graduate education focused on individual health more of the time (68.2% and 62.5%, respectively).
3.6. Facilitation/Teaching Approaches Used in Activities Implemented by T3-ITDP Participant Teams
Characteristics of activity facilitators are presented in Table 6. The majority of T3-ITDP activity facilitators were faculty (38, 92.7%). Notably, few teams (5, 12.2%) utilized community members as activity facilitators. Facilitators were from health professions that mirrored the professions of learners involved in the activities. Similar to the number of learners involved, the number of facilitators was less than 50 for most teams (75.6%). Facilitation occurred as part of an interprofessional team almost every time (68.3%), and the majority of facilitators received faculty development training or resources (24, 58.5%).
Table 6.
Characteristics of facilitators used in activities implemented by T3-ITDP participant teams
| Types of Facilitators | N (%) |
|---|---|
| Faculty | 38 (92.7) |
| Staff | 13 (31.7) |
| Community Member | 5 (12.2) |
| Clinician | 14 (34.2) |
| Students | 8 (19.5) |
| Other | 1 (2.4) |
|
| |
| Numbers of Facilitators | |
|
| |
| 0 to 25 | 22 (53.7) |
| 26 to 50 | 9 (22.0) |
| 51 to 100 | 6 (14.6) |
| 101 to 200, | 1 (2.4) |
| > 200 | 1 (2.4) |
|
| |
| Did Facilitation Occur as Part of an Interprofessional Team? | |
|
| |
| Never | 1 (2.4) |
| Almost never | 1 (2.4) |
| Occasionally/Sometimes | 7 (17.1) |
| Almost every time | 10 (24.4) |
| Every time | 18 (43.9) |
|
| |
| Faculty Development Training/ Resources Provided to Facilitators | |
|
| |
| Yes | 24 (58.5) |
| No | 13 (31.7) |
|
| |
| Types of Faculty Development/Resources Provided | |
|
| |
| Just in time training(s) | 9 (22.0) |
| Participation in stand-alone faculty development | 7 (17.1) |
| Online module(s) | 4 (9.8) |
| Facilitator guide(s) | 9 (22.0) |
| Coaching & mentorship | 10 (24.4) |
| Other | 4 (9.8) |
Facilitator training and resources were provided in a variety of ways, with coaching and mentorship of facilitators occurring most frequently (10, 24.3%). Commonly, facilitators were provided a facilitator guide or just-in-time training for the activity (both at 9, 22% of teams). Some teams (29.3%) utilized more than one approach for facilitator training and development.
3.7. Results Dissemination and Future Directions
Many T3-ITDP participant teams have successfully disseminated the results of their activities (18, 43.9%) (Table 7). Scholarly success occurred regardless of the training site attended for the in-person program or year completing the program. The work was presented most commonly as a conference abstract or presentation (13, 56.5%). Other examples of venues where results were disseminated include sharing with IPE leadership and administration, IPE sharing or professional development days at the institution, and faculty lunch and learn. In addition, many of the teams (18, 43.9%) continued working together on projects beyond the primary project developed as part of the T3-ITDP program.
Table 7.
Results dissemination and future directions for T3-ITDP participant teams and projects
| Result Dissemination Venues | N (%) |
|---|---|
| Conferences abstracts/presentations | 13 (56.5) |
| Publications | 2 (8.7) |
| Local Meetings | 5 (21.7) |
| Blog posts | 0 (0.0) |
| Webinars | 1 (4.4) |
| Social Media | 4 (17.4) |
| Other (e.g., institutional IPE sharing day) | 9 (39.1) |
|
| |
| T3-ITDP Team Has Worked Together on Additional Projects (beyond primary T3 project) | |
|
| |
| Yes | 18 (43.9) |
| No | 19 (46.3) |
| Unsure | 4 (9.8) |
4. Discussion
The purpose of this paper is to describe the results of the 2018 T3-ITDP impact survey to assess the impact the T3-ITDP program had on developing effective and successful IPECP. Findings demonstrate that T3-ITDP teams were successful at developing, implementing, and sustaining IPECP at their home institutions.
Over 75% of teams reported adapting T3-ITDP program content into teaching strategies for their projects. This included interactive facilitation strategies such as Liberating Structures, use of a “Madness-to-Methods” game to learn and think creatively about IPE curriculum and evaluation, introduction of project management (i.e. Lean) content and tools, and a “watch one, do one, teach one” train-the-trainer approach.17–20 Nearly half of participating teams focused their efforts partially or completely on faculty development, so many of these strategies were directly relevant to their projects. Copies of slides, handouts, and other course resources were freely available to all participants from each cohort during and after the in-person T3-ITDP training on a Canvas course website so that materials were easy to retrieve and adapt. The high proportion of teams adapting training content or evaluation strategies from the T3-ITDP program indicates that the deliverables and tools provided during the program are transferable to other institutions.
These findings of high rates of strategy and content utilization by T3-ITDP teams are similar to what we found during the pilot study. During that study, an interprofessional error disclosure module that teams participated in with health professional students during the in-person program was adapted and implemented by the majority of program participants at their institutions.21 The scaled up version of the T3-ITDP program did not introduce this or other IPE modules in such a participatory way, but the continued high rates of adaptation and adoption of content and teaching strategies suggest that offering these types of resources to be useful to teams and impactful on their projects. It also points to an opportunity to carry out multi-institutional studies to test the adaptation and implementation of promising “best practice” IPE modules and IPCP models on a larger scale, which could strengthen the evidence base for IPECP.21,22
Another encouraging finding from this study was the majority (76%) of T3-ITDP teams plan to continue or hold their activity again. Continuation of team projects might reflect the strong institutional support for the teams, as evidenced by the fact that 60% of teams have received institutional support, which appears to be essential for sustainability. In addition, nearly half of T3-ITDP teams have already started working together on additional projects beyond their primary T3-ITDP project; this speaks to team effectiveness and that teams will continue contributing to the IPECP field’s future. Further, the program has created a community of learners represented by the many teams willing to share new content developed with others.
It is also notable that the majority of teams have disseminated results of their T3-ITDP activities either formally or informally. These dissemination efforts are especially promising for newly implemented projects. Conferences represented the largest dissemination venue and provided T3-ITDP teams opportunities to re-engage with each other, network with other colleagues, and incorporate the latest advances in IPECP into their programming.
There were fewer large-scale events than previously reported in terms of the activities implemented by T3-ITDP participant teams.23 This shift to smaller groups likely reflects the changing needs of programs implementing IPE and the changing learner audience. The majority of activities were either single or multiple stand-alone sessions (53.7%), and many learners were not required to participate. The high percentage of elective experiences may represent that many T3-ITDP teams were implementing projects for the first time and had not yet navigated the logistics of embedding the activity within each discipline’s curricula. If integration proves challenging, it may need to be a focus of future faculty development programs.
The T3-ITDP also appears to be fostering an interprofessional culture across the Interprofessional Learning Continuum by supporting the shift of IPE into continuing professional development (CPD) settings with faculty or practicing clinicians as the target audience for many projects.5 Teams incorporating the CPD learner utilized the Train-the-Trainer model in their home institutions to help build the capacity of faculty and clinicians trained in IPECP. Many teams also indicated that multiple levels of learners are involved in the activities. A layered learning approach is encouraging and further responds to calls following the IOM Interprofessional Learning Continuum Model’s publication to diversify learners’ levels.5
Most teams assessed learning outcomes at the attitudes/perceptions level, but large numbers were also beginning to assess higher-level outcomes like knowledge/skills, collaborative behavior, and performance in practice. Assessing higher levels of learning outcomes is imperative as IPECP research becomes more rigorous, and as institutions expect evidence of return on their investment for the support given to IPECP. In addition to higher-level learning outcomes, 61% of teams assessed organizational change as a system outcome, especially when CPD learners were the target audience. At the foundational level, individual health was the outcome of focus. This is another example of how T3-ITDP teams’ work ties to the IOM Interprofessional Learning Continuum and the Quadruple Aim.5,24 In a few instances, the assessment strategy utilized by T3-ITDP teams may not have matched the level of learning or health system outcome targeted. For example, 61.8% of teams evaluating knowledge/skills and 67.7% evaluating collaborative behavior also chose to utilize surveys as one of the team’s assessment strategies. Examination of the extent to which project objectives and approaches lead to Quadruple Aim outcomes is an area ripe for future studies in the field of IPECP.24
Facilitators for the activities implemented represented a multitude of professions, and T3-ITDP teams utilized a variety of training strategies and development for facilitators. Nearly one-quarter of teams included coaching or mentoring of IP facilitators at their home institution. The frequent focus of projects on facilitator development (i.e. coaching or mentoring) may be attributable to the T3-ITDP emphasis on using a Train-the-Trainer model to expand the body of IPECP work by fostering expertise in others. Facilitation of the activities also occurred as an interprofessional team the majority of the time, a best practice thatwas explicitly modeled for T3-ITDP teams during the in-person program. Low numbers of community members were involved as facilitators across the projects, which may indicate a continued area of focus to increase engagement with patients and family members involved in IPECP training.
Limitations
There are some limitations to consider from the data gathered by the T3-ITDP impact survey. It is unclear from the CPD categorization of learner type if faculty or clinical teams were the focus of the activities implemented. Additionally, when comparing assessment strategies utilized with learning outcomes targeted, there appeared to be a mismatch in some instances. For example, teams were choosing assessment strategies at the bottom of Kirkpatrick’s pyramid, even when assessing higher-level outcomes such as collaborative behavior. Survey items related to assessment strategies may not have been specific enough to compare the variables in a meaningful way, or the options available are too numerous. In addition, there is a lack of information around specific health and system outcomes impacted by team projects. Looking at the impact on patients and clinical process change is a potential area for future study. Finally, the impact survey was project-focused and did not reexamine participant confidence in skills learned at the in-person training once work began at their home institution. However, the confidence gained at the in-person training16 likely assisted with project implementation.
Conclusions
The T3-ITDP impact survey results indicate that the program has effectively trained interprofessional teams to successfully develop, implement, and sustain IPECP activities at their home institutions. This team-based, project-focused program provides an example of an interprofessional team development model that has catalyzed IPECP projects across the learning continuum.
Supplementary Material
Acknowledgements
The authors would like to acknowledge the invaluable partnerships and collaborations out of which this work emerges: the Josiah Macy Jr. Foundation, the National Center for Interprofessional Education (NCIPE), the T3 Program National Advisory Committee, the T3-ITDP invited speakers and content contributors from each three sites, and the T3-ITDP teams. We would also like to acknowledge the valuable assistance of Dr. Ken Pike with data analysis and Erich von Abele with copyediting and proofreading.
Funding for the overall project was received from the Josiah Macy Jr. Foundation. In addition to project funding from the Macy Foundation, the first author received funding from the NIH National Heart, Lung, and Blood Institute K12 (5K12HL137940) as part of the UW Implementation Science Training Program, which supported her time during data analysis, manuscript writing, and revision. Finally, study data were collected and managed using REDCap electronic data capture tools hosted at the Institute of Translational Health Sciences. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages, and 4) procedures for importing data from external sources. REDCap at ITHS is supported by the National Center For Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Declarations of Interest: None.
References
- 1.Willgerodt MA, Abu-Rish Blakeney E, Summerside N, Vogel MT, Liner DA, & Zierler B. Authors. Impact of Leadership Development Workshops in Facilitating Team-Based Practice Transformation. J Interprof Care. 2020;34(1):76–86. https://doi.org/dz8v. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Brashers V, Haizlip J, & Owen JA The ASPIRE Model: Grounding the IPEC core competencies for interprofessional collaborative practice within a foundational framework. J Interprof Care. 2020;34(1):128–132. 10.1080/13561820.2019.1624513 [DOI] [PubMed] [Google Scholar]
- 3.Brandt BF. Interprofessional Education and Collaborative Practice: Welcome to the “New”; Forty-Year Old Field. NexusIPE; 2015. http://tiny.cc/73i7qz. Accessed 26 June 2020. [Google Scholar]
- 4.Reeves S, Lewin S, Espin S, Zwarenstein M. Interprofessional Teamwork for Health and Social Care. Oxford: Wiley-Blackwell;2010: [1–191] [Google Scholar]
- 5.Committee on Measuring the Impact on Interprofessional Education on Collaborative Practice and Patient Outcomes, Board on Global Health, & Institute of Medicine (IOM). Measuring the impact of interprofessional education on collaborative practice and patient outcomes. Washington, D.C.: National Academics Press; 2015. https://www.ncbi.nlm.nih.gov/books/NBK338360/. Accessed 26 June 2020. [PubMed] [Google Scholar]
- 6.Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice: 2016 update. Washington, DC: Interprofessional Education Collaborative; 2016. https://www.ipecollaborative.org/core-competencies.html. Accessed 26 June 2020. [Google Scholar]
- 7.Steinert Y. Learning together to teach together: Interprofessional education and faculty development. J Interprof Care. 2005;19:60–75. https://doi.org/c8jvzx. [DOI] [PubMed] [Google Scholar]
- 8.Sargeant J, Hill T, Breau L. Development and testing of a scale to assess interprofessional (IPE) facilitation skills. J Contin Educ Health Prof. 2010;30:126–131. https://doi.org/b975vf. [DOI] [PubMed] [Google Scholar]
- 9.Ho K, Jarvis-Selinger S, Borduas F, et al. Making interprofessional education work: The strategic roles of the academy. Acad Med. 2008;83:934–940. https://doi.org/dhxfnj. [DOI] [PubMed] [Google Scholar]
- 10.Abu-Rish Blakeney E, Pfeifle A, Jones M, Hall LW, & K Zierler B. Findings from a mixed methods study of an interprofessional faculty development program. J Interprof Care. 2016;30(1):83–89. https://doi.org/dz8w. [DOI] [PubMed] [Google Scholar]
- 11.Steinert Y, Mann K, Anderson B, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Med Teach. 2016;38:769–786. 10.1080/0142159X.2016.1181851. [DOI] [PubMed] [Google Scholar]
- 12.Owen JA, Brashers VL, Littlewood KE, Wright E, Childress RM, & Thomas S. Designing and evaluating an effective theory-based continuing interprofessional education (CIPE) program to improve sepsis care by enhancing healthcare team collaboration. J Interprof Care. 2014;28(3):1–6. https://doi.org/dz9d. [DOI] [PubMed] [Google Scholar]
- 13.Shrader S, Mauldin M, Hammad S, Mitcham M, Blue A. Developing a comprehensive faculty development program to promote interprofessional education, practice and research at a freestanding academic health science center. J Interprof Care. 2015;29:165–167. 10.3109/13561820.2014.940417. [DOI] [PubMed] [Google Scholar]
- 14.Steinert Y, Naismith L, Mann K. Faculty development initiatives designed to promote leadership in medical education: BEME Guide No. 19. Med Teach, 2012;34:483–503. 10.3109/0142159X.2012.680937. [DOI] [PubMed] [Google Scholar]
- 15.Summerside N, Abu-Rish Blakeney E, Brashers V, Dyer C, Hall LW, Owen JA, Ottis E, Odegard P, Haizlip J, Liner D, Moore A, & Zierler BK Early outcomes from a national Train-the-Trainer Interprofessional Team Development Program. J Interprof Care. 2018;3:1–4. https://doi.org/dz9f. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kirkpatrick DL. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler;1994:[1–373] [Google Scholar]
- 17.Lipmanowicz H, & McCandless K (2014). The Surprising Power of Liberating Structures: Simple Rules to Unleash A Culture of Innovation (1 edition). Liberating Structures Press. [Google Scholar]
- 18.From Madness To Methods Learning Activity | Collaborate. (n.d.). Retrieved October 17, 2020, from https://collaborate.uw.edu/online-training-and-resources/faculty-development-ipe-training-toolkit/from-madness-to-methods-learning-activity/
- 19.Simpson D, Fenzel J, Rehm J, Marcdante K. Enriching Educators’ Repertoire of Appropriate Instructional Methods. MedEdPORTAL; 2010. Available from: www.mededportal.org/publication/7968 [Google Scholar]
- 20.Pearce J, Mann MK, Jones C, van Buschbach S, Olff M, Bisson JI. The most effective way of delivering a Train-the-Trainers program: A systematic review. J Contin Educ Health Prof. 2012;32:215–226. 10.1002/chp.21148. [DOI] [PubMed] [Google Scholar]
- 21.Dyer C, Abu-Rish Blakeney E, Johnson E, Shrader S, Gregory G, Knight A, Shannon S, McDonough K, Hall L, & Zierler B (2017). Implementation of an interprofessional error disclosure experience: A multi-institutional collaboration. Journal of Interprofessional Education & Practice, 9, 5–11. 10.1016/j.xjep.2017.05.004 [DOI] [Google Scholar]
- 22.Bauer MS, Damschroder L, Hagedorn H, Smith J, & Kilbourne AM (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1). 10.1186/s40359-015-0089-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Abu-Rish E, Kim S, Choe L, Varpio L, Malik E, White AA, Craddick K, Blondon K, Robins L, Nagasawa P, Thigpen A, Chen L-L, Rich J, & Zierler B (2012). Current trends in interprofessional education of health sciences students: A literature review. Journal of Interprofessional Care, 26(6), 444–451. 10.3109/13561820.2012.715604 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Bodenheimer T, & Sinsky C (2014). From Triple to Quadruple Aim: Care of the Patient Requires Care of the Provider. The Annals of Family Medicine, 12(6), 573–576. 10.1370/afm.1713 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.