Using a Pocket Card to Improve End-of-life Care on Internal Medicine Clinical Teaching Units: A Cluster-randomized Controlled Trial

Abstract

Background

End-of-life care is suboptimally taught in undergraduate and postgraduate education in Canada. Previous interventions to improve residents’ knowledge and comfort have involved lengthy comprehensive educational modules or dedicated palliative care rotations.

Objective

To determine the effectiveness of a cheap, portable, and easily implemented pocket reference for improving residents’ knowledge and comfort level in dealing with pain and symptom management on the medical ward.

Design

Cluster-randomized controlled trial conducted from August 2005 to June 2006.

Setting

Medical clinical teaching units (CTUs) in 3 academic hospitals in Toronto, Canada.

Participants

All residents rotating through the medical CTUs who consented to participate in the study.

Intervention

Residents at 1 hospital received a pocket reference including information about pain and symptom control, as well as 1–2 didactic end-of-life teaching sessions per month normally given as part of the rotation. Residents at the other 2 hospitals received only the didactic sessions.

Main Outcome Measures

A 10-question survey assessing knowledge and comfort level providing end-of-life care to medical inpatients, as well as focus group interviews.

Results

One hundred thirty-six residents participated on 3 CTUs for a participation rate of approximately 75%. Comfort levels improved in both control (p < .01) and intervention groups (p < .01), but the increase in comfort level was significantly higher in the intervention group (z = 2.57, p < .01). Knowledge was not significantly improved in the control group (p = .06), but was significantly improved in the intervention group (p = .01). Greater than 90% of residents in the intervention group used the card at least once per week, and feedback from the focus groups was very positive.

Conclusions

Our pocket card is a feasible, economical, and educational intervention that improves resident comfort level and knowledge in delivering end-of-life care on CTUs.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-008-0582-4) contains supplementary material, which is available to authorized users.

BACKGROUND

End-of-life care is suboptimally taught at both the undergraduate and postgraduate level in Canada, although it constitutes a significant proportion of the care provided on general medicine clinical teaching units (CTUs). Indeed, 90% of CTU inpatients have chronic life-threatening disease and 40% of all deaths occur in hospital.¹ A recent survey of faculties of medicine in Canada noted the inconsistency and lack of support for palliative care education throughout clerkship and residency,² and many medical house staff report feeling unprepared to care for dying patients.^3–5 Not surprisingly, many hospitalized patients spend their final days in significant pain.^6,7

Previous studies have shown that comprehensive educational interventions or dedicated palliative care rotations can influence trainees’ comfort and proficiency in providing end-of-life care,^4,8–12 but these may be difficult for many residency programs to implement. By contrast, small, portable teaching aids such as pocket cards are inexpensive and popular among medical students and residents on CTUs. Two previous studies have suggested that such an intervention may be useful for improving end-of-life care. Okon et al. provided trainees with a lengthy pocket manual as part of a comprehensive curriculum on a dedicated palliative care rotation.¹³ Critchley et al. used a more concise card, but it was studied primarily using clinicians already in practice, and only using self-reported measures of efficacy.¹⁴

In the present study, we sought to investigate the utility of a portable, concise pocket card for improving knowledge and comfort with end-of-life care among residents on a medical CTU. We focused on 3 key areas within end-of-life care: pain management, nausea, and sedation.

METHODS

All residents rotating through the medical CTUs at Toronto General Hospital (TGH), Toronto Western Hospital (TWH), and Mount Sinai Hospital (MSH) were offered voluntary participation in the trial. The only exclusion criteria was withholding of consent to participate. To minimize contamination by sharing of the cards, 1 site was designated as the intervention site (MSH), and the other 2 sites (TGH and TWH) were designated as the control sites. The resident experience is similar on each CTU: all three are located in urban, academic tertiary referral hospitals in Toronto, Canada; all sites use the same “team-based” model for patient care; and all sites use a similar rota for scheduling didactic sessions for the residents with a common pool of specialist lecturers who frequently give the same lecture at multiple sites. The rotations were scheduled in advance by an administrator who had no involvement in or knowledge of the study. Neither the participants nor the evaluators were blinded to the assignment of the participants. Ethical approval was obtained from the Research Ethics Board of the University Health Network/Mount Sinai Hospital.

Residents at each site were given a 5-minute questionnaire at the start (first week) and at the end (final week) of their 1- to 2-month rotation through the CTU. The questionnaires included a section measuring comfort levels using 5-point Likert scales and 5 multiple-choice questions to assess knowledge in key areas of end-of-life care (Appendix A). The questionnaire had not been previously validated, but was designed to be a brief assessment of knowledge and comfort based on the authors’ prior experience on the CTU. We felt that it was important to use a brief assessment to minimize the inconvenience to the residents and maximize the participation rate. After the initial questionnaire was completed, residents at the intervention site were given a pocket card with information regarding symptom management in end-of-life care (see Online Appendix). They were allowed to use this card throughout their rotation but not during the second assessment at the end of the rotation. The information on the card was obtained from a number of reference sources.^15–17 Residents at the control sites were not given the pocket reference. Residents at all 3 sites received a 1-hour didactic end-of-life teaching session per month normally given as part of the rotation.

A concurrent qualitative study was performed to obtain direct feedback on the pocket card. At the end of each 2-month rotation, one of the Internal Medicine teams at the intervention site was randomly selected to participate in a focus group interview. The teams selected included residents who had completed either a 1-month or a 2-month rotation. These interviews were facilitated group discussions led by a research assistant. They consisted of a series of semistructured questions posed to the group. The objective was to identify positive and negative features of the pocket card encountered in day-to-day use on the wards.

Statistical Analysis

SPSS version 13.0.1 was used for all analyses, and p values <0.05 were considered statistically significant. To determine how comfortable residents were in end-of-life care before and after their rotation, frequencies and means were explored for each comfort item on the pretest and posttest. Similarly, to determine how knowledgeable residents were in end-of-life care before and after their rotation, proportions of correct responses were explored for each multiple-choice knowledge question on the pretest and posttest. A 2-way contingency analysis was performed to compare knowledge scores between groups and between pretest and posttest scores within a group for each knowledge question. In addition, t tests were used to identify items exhibiting a statistically significant change from pretest to posttest. During these tests, difference scores (post − pre) were compared to a baseline value of zero. Analyses were conducted for the individual comfort items, total comfort scores, and total content scores found by summing the number values associated with the residents’ responses to items.

For comparison of improvements in comfort levels between groups, Mann–Whitney U tests were used. The Mann–Whitney is analogous to an independent samples t test but is a nonparametric test appropriate for ordinal data.

Focus group interviews were transcribed in full and emergent themes were recorded. Representative quotes were recorded for all themes.

RESULTS

Between 29 August 2005 and 30 June 2006, 136 of approximately 185 (75%) eligible residents consented to participate in the trial from the 3 CTUs (Table 1). We cannot be sure of the precise number of eligible residents because of undocumented last-minute changes in resident schedules. Of those enrolled, 122 (90%) completed the study. The demographics of the participants were representative of the usual resident population on our CTUs (Table 2). Some eligible residents may not have been approached owing to our recruitment strategy of meeting residents at their “morning reports” in the first and last weeks of their rotation. Some eligible residents did not attend morning report at these times because of postcall days, vacations, educational sessions, or other personal commitments. All results were analyzed by intention to treat.

Table 1.

Participants by Hospital

	Intervention site	Control site #1	Control site #2
Consented	56	35	45
Lost to follow-up	5	5	4
Completed	51	30	41

Table 2.

Participant Demographics

	Intervention (n = 56)	Control (n = 80)	Total (n = 136)
Year of training
First	39	47	86
Second	11	21	32
Third	6	12	18
Program
Medicine	26	45	71
Surgery	5	5	10
Family medicine	8	6	14
Other (e.g., pathology, radiation oncology, neurology, psychiatry)	17	24	41

Comfort

The questionnaire evaluated the comfort level of the resident before (pre) and after (post) the rotation. Baseline comfort scores were significantly lower in the intervention group for 1 comfort item (#1). Each posttest response for both control and intervention groups had higher means than the pretest responses (Table 3). Frequency plots showed the responses to be normally distributed around their means. There was significant improvement in all of the individual comfort responses and the total comfort responses for both control and intervention groups (Table 4).

Table 3.

Comfort Scores

	Control group scores			Intervention group scores
Item	Pretest	Posttest	p value*	Pretest	Posttest	p value*
“How comfortable do you feel assessing a patient’s pain level?,” mean (SD)	3.59 (0.73)†	3.84 (0.58)	<.01	3.28 (0.69)†	3.80 (0.66)	<.01
“How comfortable do you feel dosing morphine or severe pain?,” mean (SD)	3.23 (0.95)	3.71 (0.80)	<.01	3.06 (0.86)	3.86 (0.69)	<.01
“How comfortable do you feel treating nausea at the end-of-life?,” mean (SD)	3.15 (0.85)	3.36 (0.76)	.02	3.04 (0.93)	3.61 (0.75)	<.01
“How comfortable do you feel asking a patient about code status?,” mean (SD)	3.86 (0.93)	4.26 (0.67)	<.01	3.69 (1.06)	4.33 (0.74)	<.01
“How comfortable do you feel treating agitation at the end-of-life?,” mean (SD)	2.82 (0.87)	3.33 (0.74)	<.01	2.83 (0.86)	3.45 (0.80)	<.01
Overall improvement in total comfort scores	t = 5.90	p < .01‡		t = 7.52	p < .01‡

Scores range from 1 (“very uncomfortable”) to 5 (“very comfortable”).

*p value for the comparison between pre and post intervention comfort scores.

†p < 0.05 for the difference between control and intervention groups.

‡p value for the comparison between total pre and total post comfort scores.

Table 4.

Comparison of Improvement in Individual and Total Comfort Scores between Control and Intervention Groups

Item	z score	p value
“How comfortable do you feel assessing a patient’s pain level?”	−2.31	.02*
“How comfortable do you feel dosing morphine for severe pain?”	−1.87	.06
“How comfortable do you feel treating nausea at the end-of-life?”	−1.92	.06
“How comfortable do you feel asking a patient about code status?”	−1.90	.06
“How comfortable do you feel treating agitation at the end-of-life?”	−0.86	.39
Overall	−2.57	.01†

*p < .05 indicating a significantly greater improvement in individual comfort scores in the intervention group compared with the control group.

†p < .05 indicating significantly greater improvement in total comfort scores in the intervention group compared with the control group.

In addition, overall comfort levels improved significantly more for residents in the intervention group than residents in the control group (z = −2.57, p = .01). Item 1 was also significant “How comfortable do you feel assessing a patient’s pain level?” (z = −2.31, p = .02).

Knowledge

The questionnaire evaluated the knowledge level of the resident before (pre) and after (post) the rotation (Table 5). There was 1 pretest item with significantly different scores between control and intervention groups, item 5: “When treating pain, breakthrough (PRN) doses of oral opiates should be given with what frequency?”; the control group had higher scores than the intervention group (χ² = 6.25, df = 1, p = .01). The only posttest item with significantly different scores between groups was item 1: “In treating pain, the approximate equivalence dose between oral and parenteral (IV) morphine is.” The intervention group had significantly higher scores than the control group (χ² = 6.05, df = 1, p = .01). The intervention group showed significant improvement in scores on 2 items (items #1 and #5), whereas the control group did not show significant improvement on any question. Overall, the intervention group showed significant improvement in total knowledge scores (t = 3.32, p < .01), whereas the control group did not (t = 1.956, p = .07).

Table 5.

Comparison of Improvement in Knowledge Scores between Control and Intervention Groups

	Control group scores			Intervention group scores
Item	Pretest	Posttest	p value	Pretest	Posttest	p value
“In treating pain, the approximate equivalence dose between oral and parenteral (IV) morphine is:,” % correct	62.5	69.0*	.49	57.1	86.3*	<.01
“Which of the following is an appropriate adjuvant medication for treating neuropathic pain?,” % correct	90.0	91.5	.79	83.9	86.3	.79
“All of the following are appropriate first-line drugs for opioid-induced nausea except:,” % correct	25.0	25.4	1.00	19.6	35.3	0.08
“The composition of Tylenol #2 is:,” % correct	60.0	64.8	.61	60.7	78.4	.06
“When treating pain, breakthrough (PRN) doses of oral opiates should be given with what frequency?,” % correct	63.8*	74.6	.16	42.9*	68.6	.01
Overall improvement in total knowledge scores	t = 1.956	p = .07†		t = 3.32	p < .01†

*p < .01 for difference between control and intervention groups.

†p value for the comparison between total pre and total post knowledge scores.

Focus Groups

Residents communicated overall enthusiasm for the pocket card. They found that having the card immediately at the point of care was convenient: “[I liked] not having to go run and look something up but having it there (focus group [FG] 4).” The residents also expressed general appreciation for the extensiveness of the card: “Everything you need to know is on it (FG2)”; “It has all the sections you need to think about (FG4).” Almost every resident interviewed indicated that the dosages section was the most helpful: “I thought the most useful thing was probably dose equivalency (FG2)”; “Equivalencies are the things that I would usually use the most (FG4).” One resident even reported increased confidence in practice from using the pocket card: “It helps to kind of verify what you do in your practice. You can refer to [the pocket card] and [it] makes you more confident in what you’re prescribing (FG4).”

When asked for examples of specific scenarios where they used the card, the residents provided a variety of responses: “It was helpful in writing end-of-life pain orders on palliative patients (FG2)”; “I used it because I needed some assistance with breakthrough pain medication and I also used it for medication to help with nausea and vomiting after first and second line therapy failed (FG2)”; “Cancer patients, palliative patients, trauma or traumatic injuries, it’s useful across the board (FG4).”

Several residents provided constructive feedback about the pocket card. They suggested the card could be improved if it was smaller, included color, and had a larger font size: “I think it’s a little bit difficult to find information quickly that you’re looking for because there’s too much text for the size of the card and there’s only one colour and only one type of font (FG2)”; “I thought maybe the printing was a bit small (FG3)”; “It’s a bit long. [You could] fold it in half (FG1).”

Overall, the residents interviewed found the pocket card helpful and most indicated they would continue using it in the future: “I actually make a point of putting it in my pocket (FG4)”; “I take it every day (FG4).”

Frequency of Use

Finally, within the intervention group, the frequency of card usage was explored. More than 90% of residents reported using the card, whereas >10% used it more than 5 times per week. The majority of residents used the card 1–2 times a week (Fig. 1).

Figure 1.

Frequency of use plotted against the percent of respondents.

DISCUSSION

In this cluster-randomized controlled trial, we demonstrated that residents who were given an end-of-life pocket card while rotating through an Internal Medicine CTU showed an improvement in both knowledge and comfort levels in providing end-of-life care compared with residents who did not receive the card. To our knowledge, this is the only large, controlled study to demonstrate that an easily implemented educational tool can improve measures of end-of-life care for residents.

Unlike previous studies that featured a lengthy, comprehensive education intervention or a dedicated palliative care rotation,^4,9–12,18 our study evaluated a cheap, portable, and easily implemented educational tool. It could be used alone or added to an existing curriculum and may be easily updated with new information. It can also be converted to an electronic version and uploaded to electronic devices. It is an ongoing learning tool that allows the trainee to consolidate knowledge with repeated use.

We studied the intervention in a large population of residents from multiple specialties at multiple sites. Over three-quarters of eligible residents participated, and nearly all who were approached consented to participate. The total number of residents participating was more than double that of similar studies of pocket references in the literature.^13,14

More than 90% of residents in the intervention group reported using the card at least once a week. Although participants may have been affected by reporting bias, we feel that this number confirms our hypothesis that a brief, portable, and practical card would be attractive to residents. From the focus group interviews, we learned that residents particularly liked the precise dosing schema for pain medications and the portability of the card.

We found that both the intervention and control groups experienced an improvement in comfort with end-of-life care, although the intervention group had a significantly larger improvement in comfort at the end of their rotation. Although comfort is not always measured in medical education, a recent large survey identified a lack of physician comfort as a major barrier to optimal end-of-life care.¹⁹ The improvement seen in the control group suggests that the rotation itself improves resident comfort with end-of-life care. This is consistent with previous studies which suggested that many practicing physicians gained their palliative care knowledge simply from their experience on the medical ward,^20,21 and not from any formal palliative care instruction.^21,22

Although Critchley et al. also demonstrated an improvement in comfort among health care providers given a pocket reference,¹⁴ our study showed that this was matched by an improvement in objective knowledge. Notably, the control group showed an improvement in comfort that was not accompanied by an improvement in objective knowledge. This underscores the need for objective measures of improvement when evaluating this type of educational intervention.

Resident knowledge scores were approximately 60% overall on the pretest, which is comparable to previous studies in the literature.^10,13,21 The improvement in overall knowledge scores in the intervention group was statistically significant and consistent with the effect size seen in other studies.^10,18 Among individual test items, resident scores improved most notably in the dosage-based questions (i.e., dose frequency and dose equivalence). Residents may have accessed this information more frequently, as suggested by the focus group interviews, or they may have been helped by the fact that this information appears in the first section of the pocket card.

There are several limitations to this study. We did not use a validated comfort measurement such as the Collett–Lester or Templer Death Anxiety Scales.^23,24 We felt that these were too lengthy to be completed twice by busy house staff on the medical service. Indeed, comments from some residents suggested that a longer questionnaire would have led to a lower participation rate. Instead, we developed a short list of comfort questions based on the specific anxieties that we felt would be most prevalent among residents treating patients at the end of life. We feel that the comfort questions have good face validity. In addition, the baseline knowledge scores and the degree of improvement were similar to that seen in other studies,^10,13,18,21 and the greatest improvements were seen on the questions concerning opioid dosing, which was the section most used by residents according to the focus groups.

We did not measure 2 important aspects of end-of-life care: communication and empathy. Although these are critical areas of end-of-life care, they are difficult to address on a pocket card and even more difficult to assess objectively. The SUPPORT trial showed the difficulty of designing interventions that improve these qualities.⁶

We could not control for the duration of the residents’ rotation on the CTU, and so some residents would have had 2 months’ experience and 2 didactic palliative care sessions, whereas others would have had only one of each. We could also not control or measure any informal palliative care teaching that was given in addition to the regularly scheduled didactic sessions. We also could not control the number and type of patients that each individual resident would care for on the CTU, and so individual experience may have varied considerably. However, end-of-life care is so common on the CTU that each resident should have had to care for at least 1 patient with pain or symptom control issues, and given the large size of the study, such variation in individual experience is unlikely to have accounted for the difference between the 2 groups. Because the residents were cluster-randomized, variations in patient population between CTUs may have affected the experience for each group of residents. Although we tried to limit this effect by designating 2 separate control sites, we cannot exclude the possibility that this may have confounded the results. The study was also conducted in only 1 academic program, which reduces the variability in experience but potentially limits the generalizability of the results. Finally, we cannot exclude the possibility of a Hawthorne or Pygmalion effect, whereby the unblinded participants altered their behavior based on the knowledge that they were in the control or intervention groups.²⁵

In conclusion, our pocket card is a feasible, economical, and educational intervention that improves resident comfort level and knowledge in delivering end-of-life care on CTUs.

Appendix A

Questionnaire: Pre and Post Intervention Survey (All Sites)

Year of Training CC3 CC4 PGY1 PGY2 PGY3

Program Medicine Surgery Family Other:

Date (months of rotation):

Were you given a copy of the Study Pocket Card? YES NO

Have you ever used the Pocket Card? YES NO

If yes, how many times per week did you use the card? __________

	Very uncomfortable			Very comfortable
How comfortable do you feel assessing a patient’s pain level?	1	2	3	4	5
How comfortable do you feel dosing morphine for severe pain?	1	2	3	4	5
How comfortable do you feel treating nausea at the end-of-life?	1	2	3	4	5
How comfortable do you feel asking a patient about code status?	1	2	3	4	5
How comfortable do you feel treating agitation at the end-of-life?	1	2	3	4	5

In treating acute pain, the approximate equivalence dose between oral and parenteral (IV) morphine is:

1. 1:1

2. 2:1

3. 4:1

4. 5:1

5. 7:1

Which of the following is an appropriate adjuvant medication for treating neuropathic pain?

1. Pamidronate (Aredia)

2. Carbamazepine (Tegretol)

3. Hydromorphone (Dilaudid)

4. Haloperidol (Haldol)

All of the following are appropriate first-line drugs for opioid-induced nausea except:

1. Metoclopramide (Maxeran)

2. Prochlorperazine (Stemetil)

3. Ondansetron (Zofran)

4. Haloperidol (Haldol)

The composition of a “Tylenol #2” is:

1. Acetaminophen 300 mg + Codeine 10 mg + Caffeine 15 mg

2. Acetaminophen 300 mg + Codeine 15 mg + Caffeine 15 mg

3. Acetaminophen 300 mg + Codeine 20 mg + Caffeine 15 mg

4. Acetaminophen 300 mg + Codeine 30 mg + Caffeine 15 mg

When treating pain, breakthrough (PRN) doses of oral opiates should be given with what frequency?

1. 1 hour

2. 2 hours

3. 4 hours

4. 8 hours