Evaluation of an Interdisciplinary Curriculum Teaching Team-based Palliative Care Integration in Oncology

Abstract

For students of the health care professions to succeed in today's healthcare environment, they must be prepared to collaborate with other professionals and practice on interdisciplinary teams. As most will care for patients with cancer, they must also understand the principles of palliative care and its integration into oncology. This article reports the success of one university's effort to design and implement an interdisciplinary curriculum teaching team-based palliative care in oncology which was mandatory for medical, nursing, social work and chaplaincy students. Quantitative evaluation indicated that students made significant improvements related to palliative care knowledge and skills and readiness for interprofessional education. Qualitative feedback revealed that students appreciated the experiential aspects of the curriculum most, especially the opportunity to observe palliative teams at work and practice team-based skills with other learners. While there exist many obstacles to interprofessional education and hands-on learning, the value of such experiences to the learners justifies efforts to initiate and continue similar programs in the health sciences.

Introduction

Evidence supports early integration of interdisciplinary, team-based palliative care into cancer care [1]. If today's student is to be prepared for effective practice with seriously ill cancer patients, they must receive related education and training. Yet few institutions provide interdisciplinary education including collaborative palliative care practice in oncology. This paper presents the results of one effort to integrate a mandatory curriculum teaching palliative care in oncology and interdisciplinary team-centered care into the education of health science students.

Background

Cancer diagnosis and treatment, especially in advanced disease, entails extensive physical, psychosocial and spiritual distress impacting both patient and caregiver. Palliative care provides the expertise necessary to adequately manage symptoms, facilitate effective coping, assist with prognostic awareness, support shared decision-making and align the appropriate utilization of resources with patient-centered goals [1]. Randomized clinical trials have justified the benefits of early palliative care integration [2]. Leading cancer organizations including the American Society of Clinical Oncology [3] and the National Comprehensive Cancer Network [4] have recommended early integration of palliative care, and the World Health Assembly recently passed a resolution calling for the integration of hospice and palliative care into national health services, health policies, budgets and educational curricula [5]. Such integration cannot occur without adequate preparation of an interdisciplinary oncology workforce.

The need to educate health professional students collaboratively was recognized in the early 1970's and has been cited as a way to improve patient outcomes [6, 7]. Models of care foundational to recent healthcare reform mandate team-based, patient-centered, collaborative care. Numerous studies of healthcare education conducted by the Institute of Medicine concluded that interprofessional education is essential if future practitioners are to be prepared to work effectively in present healthcare environments [8-12]. Additionally, the World Health Organization has proclaimed interprofessional education and practice as necessary worldwide [13]. Healthcare education accreditation bodies, such as the Liaison Committee on Medical Education, are now mandating interprofessional education opportunities [14].

Unfortunately, the vast majority of health professional education still occurs in discipline-specific silos with little interaction among disciplines. Competencies, strategies and accreditation requirements for IPE are clearly outlined; however, developing and implementing IPE activities at the grass-roots level challenges institutions for a wide variety of reasons.

Several reports of successful interprofessional cancer education have been published in recent years. One of the earliest was the Network Project, an initiative of Memorial-Sloane Kettering Cancer Center offering two-week observerships in multidisciplinary cancer care [15]. In Ukraine, medical and nursing students were required to work together in cancer care teams following patients throughout the course of their illness [16]. Cancer Care Nova Scotia developed an interprofessional core cancer curriculum which realized post-course changes in clinical practice and interprofessional interactions for a high percentage of the participants.

Advances in technology have enabled the development of interprofessional cancer distance education. The Canadian Association of Psychosocial Oncology created the Interprofessional Psychosocial Oncology Distance Education (IPODE) Project offered as a graduate-level elective and continuing education opportunity for professionals across five disciplines – medicine, nursing, social work, spiritual care, and psychology. International experts worked together to create an online interprofessional course about the care of teenagers and young adults with cancer [17]. International participants, including medical, nursing and counseling students, partake in online, real-time discussions and other means of socialization.

A search of the literature revealed two efforts in interprofessional education teaching oncology-focused palliative care. In Australia, an online program for multidisciplinary practitioners trained over 500 individuals residing in remote areas [18]. The Cross Cancer Institute in Alberta Canada initiated a six week multidisciplinary team-based clinical placement offered to students from a variety of disciplines [19].

These efforts, while successful, have been scattered and largely voluntary or elective courses involving only a small percentage of health sciences students. The goal of this paper is to report the outcomes of one university's effort to educate interdisciplinary students via a mandatory palliative-focused oncology curriculum.

The Curriculum

Educational modalities used in the Interdisciplinary Curriculum for Oncology Palliative Education (iCOPE) curriculum are fourfold: on-line case-based didactics, the Interprofessional Case Management Experience (ICME), clinical rotations, and critical reflection writing exercises. Fourth-year medical students, nursing students in the final semester of a BSN program, masters-level social workers specializing in healthcare, and chaplain residents completing Clinical Pastoral Education (CPE) are required to complete all components as part of their curricula. A complete description of the curriculum has been published elsewhere [20].

Methods

Design

A pre-post-mixed-methods design was used to evaluate the impact of the iCOPE curriculum on a total of 527 students over five semesters beginning in Fall2012. Of these students, 373 completed both pre and post-measures and were included in that analysis. Over 500 completed evaluations of the individual curricular components.

Procedure

Students were invited to participate in the research; only data provided by those consenting to the research are included here. The study was approved by the University of Louisville Human Subjects Protection Program.

Students completed a variety of evaluation measures. Basic demographic information was collected at baseline as were responses to pre-test questionnaires. Upon completion of each learning modality, students completed evaluations including both structured and open-ended questions. Lastly, the students completed post-test instruments and an overall iCOPE evaluation. Students were also invited to participate in optional focus groups to provide feedback on overall content, process, outcomes, and logistical considerations.

Pre/post measures included two validated instruments. The End-of-Life Professional Caregiver Survey (EPCS) is a 28 item survey with three subscales evaluating palliative care-specific educational needs (Cronbach's α= 0.96). [21]. Students rate their comfort with palliative care skills on a scale of 0 (not at all) to 4 (very much). The Self-Efficacy for Interprofessional Experiential Learning Scale (SEIEL) is a 16 item scale with three subscales measuring student self-efficacy perceptions related to learning collaboratively in interprofessional teams (Cronbach's α = 0.96)[22]. Students rate their confidence related to 16 aspects of IPE on a scale of 1(low confidence) to 10 (high confidence). Additionally, an interdisciplinary palliative care knowledge survey (IPCKS) was developed by the research team and administered with the other pre/post instruments during the last two semesters of the project. Higher scores on all these instruments indicate increased ability.

Statistical Analysis

Demographic information was summarized by frequencies and percentages. The EPCS and SEIEL scales and subscales were calculated by averaging students’ item scores within each measure. A knowledge-based measure total score was created by summing students’ correct responses. A change score for each measure was calculated by subtracting the pre from the post score. For each discipline, paired sample t-tests were used to assess pre and post-score differences. Cohen's d, an effect size used to indicate the standardized difference between two means, was used to appraise the pre-post effects size for each group. One-way analysis of variance (ANOVA) was used to detect significant differences among the four disciplines on change scores. If the Omnibus F statistic was significant, the Least Significance Difference (LSD) post hoc test was performed to identify which groups differed significantly in change scores. All assumptions for ANOVA were tested and deemed acceptable except for one scale where the homogeneity of variance assumption was violated. For this scales, Welch's F test was performed in place of the Omnibus F statistic and if statistical significance was found the Games-Howell post hoc test was performed. These two tests do not require that groups have equal variances to obtain valid inferences. For the student evaluation data, descriptive statistics were used to summarize the four evaluation assessments. Statistical significance was set by convention at p < 0.05. All analyses were performed using IBM SPSS V. 22. Responses to open-ended questions and focus group content were summarized using thematic coding and analysis [23].

Results

Sixty-four percent of participants completing pre- and post-measures were nursing students, 25% medical students, 8% social work students, and 2% from chaplaincy. Females constituted 75% of the sample. The majority (83%) had some experience working in healthcare and over one-third had at least some exposure to palliative care. Forty-one per cent had at least some learning experiences with students in other health care programs (see Table 1).

Table 1.

Demographic and Experience Related Information (N=373)

		Freq (%)
Please choose your discipline:	Chaplaincy	8 (2%)
	Medicine	95 (25%)
	Nursing	240 (64%)
	Social Work	30 (8%)
What is your gender?	Female	280 (75%)
	Male	93 (25%)
How would you describe your previous experience working in healthcare environments?	No experience	63 (17%)
	Volunteer experience only	94 (25%)
	Employed in healthcare	102 (27%)
	Both volunteer experience and employed in healthcare	114 (31%)
How would you describe your previous exposure to palliative care?	0 = None	98 (26%)
	1 = A little bit	139 (37%)
	2 = Some	105 (28%)
	3 = Quite a bit	22 (6%)
	4 = Very much	9 (2%)
How would you describe your previous experiences learning with students in other health care programs?	0 = None	98 (26%)
	1 = A little bit	122 (33%)
	2 = Some	83 (22%)
	3 = Quite a bit	49 (13%)
	4 = Very much	21 (6%)

Appraisal of the pre and post-measures indicate that almost all disciplines demonstrated a significant improvement with the effect size for most being either large (Cohen's d > 0.80) or moderate (Cohen's d > 0.50). Chaplaincy students failed to achieve a significant improvement on some scales. This finding is possibly due to the small sample size as the effect size on all scales are notable (Cohen's d >0.60). The Patient and Family Communication EPCS subscale post mean scores for all disciplines were ≥ 3.60 on a scale ranging from 0 to 4, indicating that students reported high levels of comfort with this palliative care skill after the educational experience. Similar positive results occurred for the SEIEL scale and subscales, with most disciplines’ post mean confidence scores improving ≥8.5 on a scale ranging from 1 to 10 (see Table 2).

Table 2.

Pre and Post Comparisons and Group Change Score Differences for the End-of-Life Professional Caregiver Survey (EPCS), Self-Efficacy for Interprofessional Experiential Learning Scale (SEIEL) Scales and Subscales, and the Knowledge Item Summation Scores (IPCKS)

			Pre	Post			Change scores
		n	Mean (SD)	Mean (SD)	P-value€	Cohen's D§	Mean (SD)	P-valueΔ
Total EPCS Scale	Nursing	238	1.68 (0.65)	2.67 (0.65)	<0.001	1.46	0.99 (0.68)	0.011
	Medicine	94	1.77 (0.56)	2.92 (0.53)	<0.001	1.92	1.15 (0.60)
	Social Work	30	1.64 (0.61)	2.91 (0.64)	<0.001	1.86	1.27a (0.69)
	Chaplaincy	8	2.06 (0.53)	2.65 (0.42)	0.027	0.98	0.59a (0.60)
EPCS Subscales
Patient and Family Communication	Nursing	238	1.83 (0.71)	3.76 (0.66)	<0.001	2.68	1.93a,b(0.72)	0.026
	Medicine	94	1.85 (0.62)	3.97 (0.52)	<0.001	3.26	2.12a (0.65)
	Social Work	30	1.75 (0.63)	3.93 (0.60)	<0.001	3.35	2.18b(0.65)
	Chaplaincy	8	1.89 (0.53)	3.60 (0.35)	<0.001	3.49	1.71 (0.49)
Cultural and Ethical Values	Nursing	238	1 .80 (0.74)	2.60 (0.71)	<0.001	1.03	0.80 (0.78)	0.067
	Medicine	94	1.87 (0.65)	2.82 (0.64)	<0.001	1.36	0.95 (0.70)
	Social Work	30	1.93 (0.77)	2.92 (0.74)	<0.001	1.17	0.99 (0.85)
	Chaplaincy	8	2.88(0.62)	3.23 (0.36)	0.100	0.65	0.35 (0.54)
Effective Care	Nursing	238	1 .27 (0.67)	2.60 (0.72)	<0.001	1.68	1.33a (0.79)	0.031
	Medicine	94	1.51 (0.57)	2.94 (0.60)	<0.001	2.07	1.43 (0.69)
	Social Work	30	1.22 (0.70)	3.02 (0.52)	<0.001	2.17	1.80a (0.83)
	Chaplaincy	8	1.39 (0.72)	2.05 (0.75)	0.118	0.63	0.66 (1.05)
Total SEIEL Scale	Nursing	240	7.24 (1.70)	8.66 (1.18)	<0.001	0.80	1.42 (1.78)	0.669
	Medicine	95	7.52 (1.24)	8.72 (1.02)	<0.001	0.96	1.20 (1.25)
	Social Work	30	7.13 (1.58)	8.68 (1.09)	<0.001	0.83	1.55 (1.87)
	Chaplaincy	8	6.91 (1.93)	8.28 (1.00)	0.058	0.80	1.37 (1.71)
SEIEL Subscales
Interprofessional Interaction	Nursing	240	7.38 (1.74)	8.81 (1.17)	<0.001	0.79	1.43 (1.80)	0.296
	Medicine	95	7.82 (1.30)	8.87 (1.02)	<0.001	0.80	1.05 (1.32)
	Social Work	30	7.38 (1.56)	8.79 (1.12)	<0.001	0.75	1.41 (1.87)
	Chaplaincy	8	7.14 (2.01)	8.48 (0.99)	0.052	0.82	1.34 (1.63)
Interprofessional Team Evaluation and Feedback	Nursing	240	7.10 (1.73)	8.50 (1.25)	<0.001	0.75	1.40 (1.86)	0.833
	Medicine	95	7.21 (1.29)	8.56 (1.08)	<0.001	1.06	1.35 (1.27)
	Social Work	30	6.89 (1.66)	8.58 (1.10)	<0.001	0.88	1.69 (1.92)
	Chaplaincy	8	6.69 (1.90)	8.08 (1.06	0.069	0.76	1.39 (1.83)
Total IPCKS£	Nursing	116	13.21 (2.77)	15.96 (3.02)	<0.001	0.86	2.75 (3.21)	0.846
	Medicine	81	16.89 (2.28)	19.30 (1.78)	<0.001	0.99	2.41 (2.43)
	Social Work	7	12.29 (3.99)	15.14 (2.34)	0.033	1.05	2.86 (2.73)
	Chaplaincy	5	10.20 (2.86)	12.40 (4.16	0.151	0.79	2.20 (2.77)

^€P-value reflects significance level of paired sample t-test assessing pre-and post-measures for each discipline

^§Effect size of the difference between the pre and post- measures for each discipline

^ΔP-value reflects significance level of traditional F test or Welch's F test for omnibus one-way analysis of variance assessing change score differences among disciplines

^£Correct number of responses for 22 knowledge base items with sample size based on Fall 2013 And Spring 2014 data only

Note: The change scores sharing subscripted lowercase letters indicate the means are significantly different

Comparisons among disciplines were performed on the change scores for each measure. Significant differences were found on the Total EPCS Scale, p = 0.011; the Patient and Family Communication EPCS subscale, p = 0.026; and the Effective Care Delivery System p = 0.001. On the Patient and Family Communication EPCS subscale, medicine and social work students had significantly higher change scores than nursing students, although all three groups’ post-change scores peak at a mean >3.75 (on a 0 to 4 scale). Finally, for the Effective Care (social workers, mean = 1.80; nurses, mean = 1.33, see Table 2). The overall iCOPE evaluation showed all students rated the program highly on the 14 items. Eleven items had ratings > 4.0 on a 5 point scale, while the remaining 3 items’ scores were just slightly below 4.0 (see Table 3). Analogous results were found for the Interdisciplinary Case Management Experience evaluation, the Clinical Rotation/ Reflective Writing Evaluation, and the Computer-Based Modules Evaluation (see Table 3).

Table 3.

Student Evaluation of Total iCOPE Curriculum (N = 352)

	Strongly Disagree or Disagree (1-2)	Neither Disagree or Agree (3)	Agree or Strongly Agree (4-5)	Mean (SD)
The iCOPE faculty were available to students.	3%	9%	89%	4.24 (0.74)
The iCOPE learning activities (clinical rotation, modules, etc.) contributed to my learning.	3%	7%	90%	4.19 (0.79)
The iCOPE syllabus and instructional materials provided me with the information I needed to complete the iCOPE requirements.	4%	6%	90%	4.18 (0.75)
The iCOPE curriculum taught me how to provide patient/family-centered care that addresses their unique psychological, spiritual, social and cultural resources and needs.	3%	6%	91%	4.15 (0.76)
The iCOPE objectives were clearly explained to students.	4%	10%	86%	4.14 (0.77)
The iCOPE curriculum taught me to communicate effectively with patients, families, and colleagues.	4%	8%	88%	4.13 (0.77)
The iCOPE learning activities were well	5%	7%	87%	4.10 (0.79)
The iCOPE curriculum taught me to work effectively with colleagues of multiple professions, across multiple settings.	5%	7%	88%	4.09 (0.86)
The feedback I received about my performance during iCOPE learning activities was helpful.	5%	12%	83%	4.05 (0.80)
The iCOPE curriculum taught me to provide effective physical care to address palliative care needs.	5%	13%	83%	4.03 (0.82)
The iCOPE curriculum taught me to identify and address ethical and legal issues related to palliative care.	5%	12%	83%	4.00 (0.80)
The iCOPE learning activities provided adequate time to work with students preparing to work in other healthcare professions.	9%	11%	80%	3.96 (0.91)
The iCOPE assignments (critical reflective writing paper, iCOPE progress note) contributed to my learning.	9%	16%	76%	3.91 (0.94)
The iCOPE workload was appropriate.	9%	14%	77%	3.90 (0.93)
Mean Ratings of Post Evaluation Highest and Lowest Rated Items		Mean	(SD)
ICME Evaluation (14 items, n = 445)¥	Highest	4.36	(0.79)
	Lowest	3.98	(0.85)
Clinical Rotation/Critical Reflective Writing Evaluation (14 items, n = 409)	Highest	4.30	(0.74)
	Lowest	3.99	(0.81)
Computer-Based Modules Evaluation (10 items, n = 504)	Highest	4.14	(0.79)
	Lowest	3.87	(0.85)

^¥Item Responses Are Anchored with 1 Strongly Disagree to 5 Strongly Agree or 1 Very Ineffective to 5 Very Ineffective

Notable themes identified in the open-ended responses on the student evaluations were: ICME was the students’ favorite learning experience, followed closely by their clinical rotation; students appreciated getting to work with and as interdisciplinary teams; many students valued their experience as often it was their first exposure to interdisciplinary and palliative care and their only opportunity to “practice” teamwork; the clinical experience positively affected them both personally and professionally; modules and simulated patient scenarios were realistic and useful; and faculty facilitators were excellent. One student offered the following comment which captures the essence of much of the qualitative feedback:

“All aspects of the program together make it an incredible educational experience. Each step has its own unique purpose and value. I thoroughly enjoyed the Inter-Disciplinary Case Management Experience since it involved having to put various concepts learned throughout the curriculum into actual practice. It allowed students to encounter a true interdisciplinary team and take away tools for thriving amidst different disciplines for the sake of the patient.”

In addition to these common evaluation comments, focus group attendees frequently stated that their experience resulted in: a better understanding and more favorable view of palliative care; improved understanding of and respect for other disciplines; enhanced communication skills when talking with palliative patients and their families and about death and dying; recognition of the importance of patient and family-centered interdisciplinary care; and improved ability to function on a team. Students especially liked interacting with students from all four disciplines. For many of the students, their clinical experience included seeing a patient die for the first time. Constructive feedback included frequent statements that the online didactic modules required too much time to complete and added too much work to their already heavy academic load. At one focus group, a nursing student summed up her feelings regarding the interdisciplinary experience in this statement:

“ Interdisciplinary practice in healthcare is extremely important. For the first time I saw how important it is because when it comes down to certain procedures, we are not spiritual experts as nurses. Finances are an issue so that's when a social worker would come in. When you see the resources that they use, you yourself become a very valuable resource in those areas. You may not have the depth of knowledge that a chaplain or social worker has, but you know to refer. We are all teaching each other.”

Discussion

As with the other published interdisciplinary palliative oncology education efforts [18, 19], this effort was successful in realizing student learning goals. Overall evaluation revealed the curriculum had significant impact on palliative care knowledge and skills and self-efficacy related to learning with members of other disciplines with strong effects sizes. Overall student evaluation of the curriculum was above average as reflected in their ratings of the various aspects of the curriculum.

Significant differences in change scores between the disciplines on the EPCS Scale and subscales could be contributed to a number of factors. Medical and nursing students had higher means on the pre-test than social work on the total EPCS scale, the Patient and Family Communication Subscale, and the Effective Care Subscale; this may be a reflection of redundancy in the existing curricula of the disciplines. However, social workers had a higher pre-test mean on the Cultural and Ethical Values Subscale. Because cultural diversity and ethics are a strong focus in social work education, such curricular content may contribute to the students’ higher scores on this subscale. Chaplains had the highest pre-test mean on the overall EPCS scale and all subscales except the Effective Care subscale. These differences might be due to the increased maturity and higher educational level of the CPE students, all of whom had completed graduate work and had experience working with patients. Changes score differences between disciplines on the SEIEL Scale and subscales were not significant, which may be indicative of similar experience across disciplines with interprofessional experiential learning prior to the curriculum and similar increases in self-efficacy.

Student feedback revealed they found actual hands-on experiences (clinical rotations and interactive practice of teamwork) to be preferred teaching approaches. Historically, students have learned about palliative care and oncology via lectures and course content, but studies have shown that content related to seriously ill and dying patients is best taught via direct experience [24, 25]. While it is challenging to incorporate more clinical exposure and face-to-face learning with other disciplines into crowded curricula, our evaluation confirms the benefits of such inclusion. Utilizing existing curriculum structure to house IPE addressed this issue.

The online didactic modules, used to provide baseline content related to palliative care in oncology and to introduce students to team-based care, were less well received. The fact that students felt these required too much time to complete led to significant editing and revisions to reduce redundancy. While it is necessary to present new content and factual information related to symptom management and holistic care, it is also important to target only necessary information to be presented in this way. Most students of the health sciences in the later months of their training have already high academic demands which distract them from additional independent study; therefore, face-to-face, interactive learning activities which require them to be focused and engaged may be most effective.

Based on student feedback, the reflective writing assignment and the small group sharing of the reflections also proved to be effective. Reflection fosters self-discovery, self-regulation, and the therapeutic use of self in clinical situations and is important for professional development [26]. However, reflection in and of itself is not necessarily intuitive and warrants additional educational interventions [27]. Often students are asked to write reflections without being allowed to share with others and gain perspective via the structured feedback that adds another dimension to the learning experience. Faculty members were impressed with the depth of the sharing that occurred during the small group sessions; students learned not only from their own experiences but those of the other students. Students valued both the small group sharing and the faculty's written feedback on their reflections.

As is often the case with interprofessional educational efforts, it was a challenge to have adequate student representation from inherently unequal sized disciplines. In the didactic modules and the cases scenarios, all four disciplines were presented as equal contributors to patient/family care in order to model the essential role and contribution of the four disciplines. At each of the interdisciplinary case management sessions, each discipline was usually represented in each of the small groups, but at times chaplains and social workers (the two groups with fewer participants) were recruited from faculty or practitioners. Since medical social work is a specialty within the master's program, the number of social work participants was much less than nursing or medicine. Clinical Pastoral Education programs have limited resident positions; the residents in the programs involved in iCOPE attended ICME sessions 2-3 times (different cases were presented at the three sessions) in order to have greater exposure to team work and palliative care. Other disciplines benefitted from the exposure to teamwork with chaplains. Unfortunately, the number of participating chaplains impacted our efforts to evaluate outcomes for this group as the numbers often did not allow for significance in the data analysis.

Securing equal representation was further complicated because the curriculum was planned when medical students completed their palliative care rotation during the third year. The year the curriculum was initiated, the rotation was moved to the fourth year and resulted in the relative absence of medical students during the first two semesters of iCOPE. Representation from fourth year students who had completed the rotation the previous year was recruited , but full representation of medical students was delayed to later semesters. Therefore, nursing students were the predominant group in our study and, because nursing remains a predominantly female profession, only a fourth of our total participants were male.

Limitations

The results of this effort reflect successful student learning and anticipated practice changes upon completion of the iCOPE curriculum. As with most educational interventions, it is unknown what ongoing impact the curriculum will have on actual practice subsequent to graduation. Such evaluation presents a challenge as students often scatter after graduation and become difficult to locate. Future research should explore innovative ways to measure long-term impact of interprofessional, oncology and palliative care education.

Conclusion

This mandatory, multimodal, experiential, interdisciplinary oncology palliative care curriculum was successful in teaching palliative care skills and knowledge, increasing student's self-efficacy related to interprofessional learning and impacting their attitudes and abilities related to the practice of team-based palliative care in oncology. For palliative care and interdisciplinary practice to be successfully incorporated into cancer care, it is essential that students of the health professions participate in such programs as part of their preparation for clinical practice.