Abstract
BACKGROUND:
The lack of validated quality indicators is a major barrier to improving end-of-life communication and decision-making. We sought to show the feasibility of and provide initial validation for a set of quality indicators related to end-of-life communication and decision-making.
METHODS:
We administered a questionnaire to patients and their family members in 12 hospitals and asked them about advance care planning and goals-of-care discussions. Responses were used to calculate a quality indicator score. To validate this score, we determined its correlation with the concordance between the patients’ expressed wishes and the medical order for life-sustaining treatments recorded in the hospital chart. We compared the correlation with concordance for the advance care planning component score with that for the goal-of-care discussion scores.
RESULTS:
We enrolled 297 patients and 209 family members. At all sites, both overall quality indicators and individual domain scores were low and there was wide variability around the point estimates. The highest-ranking institution had an overall quality indicator score (95% confidence interval) of 40% (36%–44%) and the lowest had a score of 18% (11%–25%). There was a strong correlation between the overall quality indicator score and the concordance measure (r = 0.72, p = 0.008); the estimated correlation between the advance care planning score and the concordance measure (r = 0.35) was weaker than that between the goal-of-care discussion scores and the concordance measure (r = 0.53).
INTERPRETATION:
Quality of end-of-life communication and decision-making appears low overall, with considerable variability across hospitals. The proposed quality indicator measure shows feasibility and partial validity. Study registration: ClinicalTrials.gov, no. NCT01362855
Despite numerous calls to improve end-of-life care, and communication and decision-making in particular,1–4 over 2 decades there has been little progress in this arena.5,6 One of the major reasons for this limited progress may be the lack of quality indicators that are meaningful at a health system level to guide decision-makers as they introduce policies and practices to improve end-of-life care.7,8 We recently published a study that used a modified Delphi consensus process to develop a set of quality indicators related to advance care planning, goals-of-care discussions and documentation of these plans and goals (Table 1).9 Quality indicators should be evaluated based on their reliability, validity, responsiveness to change and ease of implementation.10 Because of the robust methodology used in developing these quality indicators,9 we believe they are clinically sensible and have face and content validity. Here we describe implementation of these quality indicators in a multicentre study to evaluate their feasibility and construct validity. A secondary objective was to establish a method for ranking institutions with respect to their performance in end-of-life–related communication and decision-making.
Table 1:
List of quality indicators
| Indicator | Mean of importance* |
|---|---|
| Advance care planning | |
| Before hospital admission, the patient discussed his or her preferences for using or not using life-sustaining treatments with his or her substitute decision-maker. | 6.58 |
| Before hospital admission, the doctor talked to the patient and/or a family member about a poor prognosis or indicated in some way that the patient has a limited time left to live. | 6.54 |
| Before hospital admission, the patient and/or a family member discussed his or her preferences for using or not using medically appropriate life-sustaining treatments with his or her family doctor or other doctor. | 6.25 |
| Before hospital admission, the patient discussed his or her preferences for using or not using medically appropriate life-sustaining treatments with other family members. | 6.17 |
| The patient has formally designated, in writing, someone he or she trusts to be his or her substitute decision-maker concerning medical treatment decisions in the event he or she is not able to do so (using appropriate legal documentation depending on jurisdiction). In case of power of attorney, it should be related to health care. | 6.04 |
| Before the patient’s admission to hospital, a member of the health care team offered to arrange a time when the patient and his or her family could meet with the doctor to discuss the use of medically appropriate life-sustaining treatments he or she would want, or not want, in the event the patient’s physical health deteriorates. | 6.00 |
| The patient has an advance directive or living will or has indicated in some other way (verbal, by video and so on) the medical treatments he or she would want (or not want) in the event he or she is unable to communicate for him- or herself as a result of a life-threatening health problem. | 5.88 |
| Before hospital admission, the patient and/or a family member discussed preferences for using or not using medically appropriate life-sustaining treatments with other health care professionals (i.e., nurse, social worker and spiritual carer). | 4.83 |
| Goals-of-care discussion | |
| Since admission, a member of the health care team has talked to the patient and/or substitute decision-maker about a poor prognosis or indicated in some way that the patient has a limited time left to live. | 6.75 |
| Since admission, a member of the health care team has talked to the patient and/or substitute decision-maker about the outcomes, benefits and burdens (or risks) of life-sustaining medical treatments. | 6.63 |
| Since admission, a member of the health care team has talked to the patient and/or substitute decision-maker about outcomes, benefits and burdens of focusing on comfort care as the goal of the patient’s treatment (e.g., palliative care or treating symptoms like pain without trying to cure or control their underlying illness). | 6.63 |
| Since the patient’s admission, a member of the health care team has offered to arrange a time when the patient or substitute decision-maker or the patient’s family can meet with the doctor to discuss the treatment options and plans. | 6.58 |
| Since the patient’s admission, a member of the health care team has asked if the patient (or substitute decision-maker, if patient is incapable) had prior discussions or has written documents about the use of life-sustaining treatments. | 6.50 |
| Since the patient’s admission, a member of the health care team has asked the patient or substitute decision-maker or the patient’s family what is important to them as they consider health care decisions at this stage of the patient’s life (i.e., values, spiritual beliefs and other practices). | 6.29 |
| Since admission, a member of the health care team has given the patient the opportunity to express his or her fears or discuss what concerns him or her. | 6.29 |
| Since admission, a member of the health care team has asked the patient or his or her family if they had any questions or needed things clarified regarding the patient’s overall goals of care. | 6.25 |
| Since admission, a member of the health care team has asked the patient what treatments he or she prefers to have or not have if he or she develops a life-threatening illness. | 6.2 |
| Since admission, the patient has been informed that he or she may change his or her mind about decisions around goals of care. | 5.92 |
| Since admission, the patient and family have been offered an opportunity to discuss with members of the health care team issues regarding capacity and consent with regard to advance care planning; specifically, what actions would take place in the possible event of losing capacity to consent to care. | 5.71 |
| Since admission, the patient and family have been offered support from the allied health care team (e.g., spiritual care, social work and clinical nurse specialist) as needed. | 5.63 |
| Since admission, a member of the health care team has provided the patient and his or her family with information about goal-of-care discussion to look at before conversations with the doctor. | 5.42 |
| Documentation | |
| Documentation of goals of care is present in the medical record. | 6.71 |
| The goals of care present in the medical record are consistent with the patient’s stated preferences. | 6.71 |
| If the hospital uses a standardized folder or other strategy to locate advance care planning/goals of care documents in the medical record, these are present in the medical record.† | 6.54 |
| Documentation of the outcomes of advance care planning conversations (including any prior expressed wishes, diaries and power of attorney documents) is present in the patient’s medical record. | 6.17 |
| Since admission, a member of the health care team has helped the patient and his or her family to access legal documents to communicate the patient’s advance care planning.† | 5.17 |
Weights were derived from prior consensus panel.9
Items flagged for removal because of poor internal consistency of this domain.
Methods
This work was guided by a conceptual model of communication and decision-making in seriously ill and older patients who had been admitted to hospital, and the impact of these processes on patient-centred outcomes (Figure 1).9 To validate the aforementioned quality indicators, we conducted a multicentre, prospective audit of communication and decision-making related to the use or nonuse of life-sustaining treatments during serious illness, from the perspectives of patients and their family members; this was called the Audit of Communication, Care Planning, and Documentation (ACCEPT) Study.12 The association between our conceptual model and the ACCEPT audit is illustrated by the box in Figure 1 (downstream medical care actually received and satisfaction with hospital care were not evaluated).
Figure 1:
Conceptual model of improving end-of-life communication and decision-making, showing Audit of Communication, Care Planning, and Documentation (ACCEPT) Study quality indicators.9 In this model, we propose that advance care planning should ideally occur before hospital admission and inform goals-of-care discussions that occur within hospital. Ultimately, these plans and goals are translated into written documents or medical orders for the use or nonuse of life-sustaining treatments. For medical care at end of life to be consistent with patient values and preferences (the primary outcome), medical orders documented within the hospital records (surrogate outcome) must be concordant with these expressed preferences. Ultimately, the processes of communication and decision-making affect patient and family-centred outcomes, such as knowledge, stress, anxiety, other measures of psychological well-being and overall satisfaction. Given that advance care planning should occur before serious illness or admission to hospital, admission is an opportunity to assess satisfaction with previous advance care planning conversations. Similarly, discharge from hospital is an opportunity to measure patient satisfaction with goal-of-care discussion and decision-making.
The method for the ACCEPT audit has been described previously.1 In brief, inpatients from 12 teaching hospitals in Ontario, Manitoba, Alberta and British Columbia, Canada, were screened for eligibility (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.160515/-/DC1). Patients were enrolled if they had advanced medical diseases (Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.160515/-/DC1) or if they were 80 years of age or older. If none of the above criteria were met, we also included any patient whose death within the next 6 months would not surprise any member of their care team.13
We approached consecutive, eligible patients and their family members for their written consent. Patients who were unable to communicate owing to language barriers (English or French only) or cognitive reasons were excluded, but if their family member was available and able to communicate, they were approached independently. Enrolled patients were asked to identify an adult family member who knew them best (inclusive of partners, significant others or close friends) and who had visited the patient in hospital at least once. We approached the patient 2 to 5 days after admission to allow for abatement of symptoms present at the time of admission and to allow for participation of both patient and family members.
Upon enrolment, the research assistant conducted separate face-to-face interviews with patients and family members using the previously validated ACCEPT questionnaire.14 In brief, we assessed respondents’ engagement in advance care planning before they were admitted to hospital and then asked questions about whether key steps related to goal-of-care discussion had occurred during the current hospital admission. These questions related to advance care planning and goal-of-care discussion were based on our previously developed quality indicators.9 To elicit preferences about use of life-sustaining treatments, we used the following taxonomy: respondents were asked to select the one that best represented their preferences: 1) Use machines and all possible measures including cardiopulmonary resuscitation (CPR) with a focus on keeping me (or my family member) alive at all costs; 2) Use machines and all possible measures with a focus on keeping me (or my family member) alive, but if my heart stops, no CPR; 3) Use machines only in the short term to see if I (or my family member) will get better, but if my illness is prolonged, change focus to comfort measures only. If the heart stops, no CPR; 4) Use full medical care to prolong my life (or life of my family member), but if my heart or breathing stops, no CPR or breathing machines; 5) Use comfort measures only, with a focus on improving quality of life and comfort. Allow natural death and no artificial prolongation of life and no CPR; or 6) Unsure.
We also assessed satisfaction with end-of-life care, including communication and decision-making, in the 4 weeks before hospital admission through the use of the validated Canadian Health Care Evaluation Project (CANHELP) Questionnaire.15 The CANHELP instrument is a questionnaire that quantifies patients’ and family caregivers’ satisfaction on a 5-point Likert scale.
After completion of the ACCEPT questionnaire, the research nurse reviewed the hospital record to document the medical orders for the use of life-sustaining treatments and items from the documentation domain of the quality indicator framework.9
Statistical analysis
Demographic characteristics of patients and their family members are described as means, standard deviations and ranges for continuous variables, and as counts and percentages for categorical variables. Each quality indicator belonged to one of the advance care planning, goal-of-care discussion or documentation domains. Item scores are the proportion of patients (or family members) who affirmed that each quality indicator listed in Table 1 had occurred, multiplied by a weighting. Weights for each item were derived from the importance ratings from our previous work on the development of these quality indicators (Table 1).9 The overall quality indicator score, known as the ACCEPT quality indicator score, is the raw sum of all quality indicator items divided by the raw sum of highest possible values of these items (on a scale of 0% to 100%). We combined both patients’ and family members’ responses to develop average site overall and domain scores. We evaluated the internal consistency of the overall and domain scores using the Cronbach’s α.
To provide a measure of validity for this quality indicator scoring approach, a priori, we hypothesized that the overall ACCEPT quality indicator score would correlate with concordance between stated preferences and documented goals of care; concordance was calculated only for participants who had both of these items available. Patients or family members who had missing data or who were “unsure” of their preferences were excluded from these analyses. If the patient or family member preferred CPR and if there were no documented goals of care on the chart, the case was considered concordant because the default in all participating institutions was resuscitation. If patients or family members preferred not to be resuscitated and there was no documentation, such cases were considered to be discordant and consistent with a medical error. This concordance measure is one of the quality indicators in the documentation domain but was excluded from this domain, to stand alone as an outcome (Figure 1) for this validation exercise.
Because the goal-of-care discussion items were more likely to influence the concordance statistic in hospital, we postulated that the advance care planning domain score (a measure of prehospital activity) would correlate less strongly with concordance than the goal-of-care discussion domain scores. Moreover, because the CANHELP satisfaction measure was administered early in the hospital course, we postulated that the pertinent domains and items from the CANHELP questionnaire relevant to advance care planning would correlate with the advance care planning domain of the quality indicators. Associations between the whole or individual domains of the quality indicator score, the concordance measure, and the items and domains of the CANHELP scores were assessed by Pearson correlation coefficients.
Ethics approval
The research ethics boards of all participating institutions approved the study.
Results
We approached 580 eligible patients and 318 eligible family members; 297 patients and 209 family members consented, yielding an enrolment rate of 51% and 66%, respectively (Figure 2). The average age of patients was 80 years and 43% lived alone (Table 2). The average age of family members was 61 years; most were children of the patient (55%; Table 3). Most patients and family members were white and spoke either English or French. Hospital site characteristics are provided in Table 4.
Figure 2:
Patient flow diagram. Patients or family members who are excluded may fall into multiple categories of exclusion. ACCEPT = Audit of Communication, Care Planning, and Documentation study.
Table 2:
Patient characteristics
| Characteristic | No. (%) of patients,* n = 297 |
|---|---|
| Age, mean ± SD | 79.8 ± 9.9 (55.0–107.0) |
| Sex | |
| Male | 123 (41.4) |
| Charlson Comorbidity Index, mean ± SD | 2.9 ± 2.4 (0.0–10.0) |
| Marital status | |
| Married or living as married | 100 (33.7) |
| Widowed | 113 (38.0) |
| Never married | 23 (7.7) |
| Divorced or separated; not remarried | 41 (13.8) |
| Missing | 20 (6.7) |
| Live-alone status (determined by question, “Do you live alone?”) | |
| Yes | 128 (43.1) |
| No | 149 (50.2) |
| Missing | 20 (6.7) |
| Last location of living in last month (if in hospital, then month before hospital admission) | |
| Home | 212 (71.4) |
| Retirement residence | 45 (15.2) |
| Long-term care or nursing home | 9 (3.0) |
| Ward in another hospital | 0 (0.0) |
| Other (specify) | 11 (3.7) |
| Missing | 20 (6.7) |
| Location of last residence noted above | |
| Rural | 24 (8.1) |
| Urban | 249 (83.8) |
| Declined | 3 (1.0) |
| Missing | 21 (7.1) |
| Health literacy (REALM-R†) score, mean ± SD | 7.3 ± 1.4 (1.0–8.0) |
| Education — highest level achieved | |
| Elementary school or less | 19 (6.4) |
| Some high school | 65 (21.9) |
| High school graduate | 71 (23.9) |
| Some college (including Collège d’enseignement général et professionnel)/trade school | 30 (10.1) |
| College diploma (including Diplôme d’études collégiales)/trade school | 32 (10.8) |
| Some university | 11 (3.7) |
| University degree | 40 (13.5) |
| Postgraduate | 7 (2.4) |
| Declined | 1 (0.3) |
| Missing | 21 (7.1) |
| Religion (determined by question, “Do you identify with a formal religious group or practice?”) | |
| Protestant | 96 (32.3) |
| Catholic | 61 (20.5) |
| Jewish | 2 (0.7) |
| Muslim | 5 (1.7) |
| Sikh | 0 (0.0) |
| None | 83 (27.9) |
| Other (specify) | 27 (9.1) |
| Declined | 2 (0.7) |
| Missing | 21 (7.1) |
| Race and language | |
| White | 280 (94.3) |
| Nonwhite | 17 (5.7) |
| White; speaking a language other than English or French on a daily basis | 54 (18.2) |
| Nonwhite; speaking a language other than English or French on a daily basis | 13 (4.4) |
| Frailty of patient (determined by question, “How fit or frail was the patient at this point?”) | |
| Very fit | 10 (3.4) |
| Well | 36 (12.1) |
| Managing well | 80 (26.9) |
| Vulnerable | 73 (24.6) |
| Mildly frail | 61 (20.5) |
| Moderately frail | 34 (11.4) |
| Severely frail | 3 (1.0) |
| Very severely frail | 0 (0.0) |
| Inclusion criteria | |
| Age ≥ 55 yr with COPD, congestive heart failure, cirrhosis, cancer or end-stage dementia | 136 (45.8) |
| Diagnosis | |
| COPD | 54 (18.2) |
| Congestive heart failure | 34 (11.4) |
| Cirrhosis | 5 (1.7) |
| Cancer | 51 (17.2) |
| End-stage dementia | 0 (0.0) |
| Age ≥ 80 yr and admitted to hospital from community for acute medical care team assessment | 156 (52.5) |
| Expected death within 6 mo | 5 (1.7) |
Note: COPD = chronic obstructive pulmonary disease, REALM-R = Rapid Estimate of Adult Literacy in Medicine, Revised, SD = standard deviation.
Unless stated otherwise.
The REALM-R Health Literacy score ranges from 0 to 8, with a higher score representing a greater degree of health literacy.
Table 3:
Family member characteristics
| Characteristics | No. (%)* of family members who reported “yes” n = 209 |
|---|---|
| Age, mean ± SD | 60.8 ± 12.4 (21.0–90.0) |
| Sex | |
| Male | 61 (29.2) |
| Relationship to patient | |
| Spouse/partner | 50 (23.9) |
| Parent | 1 (0.5) |
| Daughter/son | 114 (54.5) |
| Sister/brother | 3 (1.4) |
| Other (specify) | 19 (9.1) |
| Missing | 22 (10.5) |
| Education (highest level achieved) | |
| Elementary school or less | 9 (4.3) |
| Some high school | 13 (6.2) |
| High school graduate | 32 (15.3) |
| Some college (including CEGEP) or trade school | 24 (11.5) |
| College diploma (including DEC) or trade school | 40 (19.1) |
| Some university | 13 (6.2) |
| University degree | 36 (17.2) |
| Postgraduate | 16 (7.7) |
| Declined | 1 (0.5) |
| Missing | 25 (12.0) |
| Identification with formal religious group or practice | |
| Protestant | 54 (25.8) |
| Catholic | 49 (23.4) |
| Jewish | 1 (0.5) |
| Muslim | 7 (3.3) |
| Sikh | 1 (0.5) |
| Other (specify) | 15 (7.2) |
| None | 58 (27.8) |
| Missing | 24 (11.5) |
| Respondent is the substitute decision-maker | 127 (60.8) |
| Race and language | |
| White | 169 (80.9) |
| Nonwhite | 40 (19.1) |
| White; speaking a language other than English or French on a daily basis | 30 (14.4) |
| Nonwhite; speaking a language other than English or French on a daily basis | 34 (16.3) |
Note: CEGEP = Collège d’enseignement général et professionnel, DEC = Diplôme d’études collégiales, SD = standard deviation.
Unless stated otherwise.
Table 4:
Site characteristics
| Site characteristics | No. (%) of participating sites n = 12 |
|---|---|
| Academic or teaching facility | 12 (100.0) |
| A mechanism in place to enable access to the most current advance care planning/goal-of-care discussion documents | 11 (91.7) |
| A standardized folder or other strategy to locate advance care planning/goal-of-care discussion documents | 12 (100.0) |
| Ensures that clinical staff have access to the necessary professional development resources | 11 (91.7) |
| Documented advance care planning policies and procedures | 10 (83.3) |
| Policies and procedures in place so that “high-risk” patients participate in advance care planning/goal-of-care discussion processes | 7 (58.3) |
| A continuous quality improvement initiative that audits and provides feedback to teams | 6 (50.0) |
| Management evaluates advance care planning knowledge and skills among relevant staff | 8 (66.7) |
| A process in place whereby patients with a specific disease are offered disease-specific advance directives | 4 (33.3) |
Overall, all quality indicator scores were low, with wide confidence limits around these scores, and variability in performance among institutions (Table 5). Nevertheless, quality indicator scores for the highest-ranked hospital did not overlap with those from the lowest-ranked hospital; the highest-ranked hospital had an overall score (95% confidence interval [CI]) of 40% (36%–44%) and the lowest-ranked hospital had a score of 18% (11%–25%).
Table 5:
Scores and ranking of quality indicators by hospital*
| Site # | No. of participants n = 506 |
Overall ACCEPT quality indicator score | Advance care planning domain score | Goals-of-care discussion domain score | Documentation domain score† | Concordance | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|||||||
| Patients | Family members | Rank | % (95% CI) | Rank | % | Rank | % | Rank | % | Rank | % | |
| 4 | 25 | 1 | 1 | 40 (36–44) | 2 | 40 | 2 | 29 | 1 | 78 | 1 | 65 |
|
| ||||||||||||
| 20 | 40 | 9 | 2 | 33 (28–38) | 5 | 34 | 5 | 23 | 5 | 64 | 7 | 25 |
|
| ||||||||||||
| 79 | 20 | 16 | 3 | 33 (27–38) | 11 | 15 | 1 | 31 | 2 | 72 | 4 | 28 |
|
| ||||||||||||
| 24 | 25 | 4 | 4 | 32 (27–37) | 3 | 38 | 8 | 17 | 3 | 72 | 8 | 24 |
|
| ||||||||||||
| 2 | 23 | 1 | 5 | 29 (22–36) | 10 | 23 | 3 | 27 | 11 | 48 | 2 | 33 |
|
| ||||||||||||
| 19 | 45 | 9 | 6 | 29 (24–33) | 4 | 38 | 9 | 14 | 7 | 58 | 10 | 20 |
|
| ||||||||||||
| 13 | 20 | 2 | 7 | 28 (21–35) | 9 | 23 | 4 | 26 | 12 | 42 | 5 | 27 |
|
| ||||||||||||
| 23 | 14 | 7 | 8 | 28 (21–34) | 6 | 33 | 10 | 12 | 4 | 69 | 11 | 14 |
|
| ||||||||||||
| 21 | 27 | 30 | 9 | 27 (23–32) | 7 | 31 | 6 | 17 | 8 | 54 | 3 | 28 |
|
| ||||||||||||
| 25 | 21 | 17 | 10 | 26 (19–33) | 8 | 23 | 7 | 17 | 6 | 61 | 6 | 26 |
|
| ||||||||||||
| 50 | 25 | 0 | 11 | 25 (20–30) | 1 | 46 | 12 | 5 | 10 | 50 | 9 | 24 |
|
| ||||||||||||
| 22 | 12 | 4 | 12 | 18 (11–25) | 12 | 15 | 11 | 11 | 9 | 50 | 12 | 6 |
Note: ACCEPT = Audit of Communication, Care Planning, and Documentation, CI = confidence interval.
Participating hospitals are ranked by the overall score on ACCEPT quality indicators. Scores ranged from 0% to 100%. Domain scores for advance care planning, goals of care and documentation are also shown. Absolute scores for the concordance measures and ranking by those scores shown on far right (range 0–100).
Without concordance measure.
Concordance at a hospital level ranged from 6% to 65%. Overall, there was a strong and statistically significant correlation between the overall ACCEPT quality indicator score and concordance (Table 6; r = 0.72, p = 0.008). As expected, the correlations between the advance care planning domain, and concordance and overall quality indicator score were weaker than the correlations between the goal-of-care discussion domain and concordance (Table 6). There were no statistically significant correlations between the advance care planning quality indicator domain score and different aspects of the CANHELP satisfaction questionnaire (overall score, domain scores or selected relevant item scores; Table 7). The Cronbach’s α for the overall score was 0.82, and for the advance care planning, goal-of-care discussion and documentation domains, scores were 0.64, 0.82 and 0.37, respectively. The low scores in the documentation domain were driven largely by the third and fifth items; when these items were excluded, the Cronbach’s α was 0.77 and the correlation with the overall quality indicator score remained relatively unchanged (r = 0.71, p = 0.01).
Table 6:
Associations between site ACCEPT quality indicator score and concordance measures (n = 12)*
| Item or domain score | Concordance measure r (p value) | Advance care planning domain score r (p value) | Goals-of-care discussion domain score r (p value) | Documentation domain score r (p value) | Overall ACCEPT quality indicator score r (p value) |
|---|---|---|---|---|---|
| Concordance | 1.0 | 0.35 (0.3) | 0.53 (0.08) | 0. 26 (0.4) | 0.72 (0.008) |
| Advance care planning domain score | 1.0 | −0.37 (0.2) | 0.24 (0.4) | 0.38 (0.2) | |
| Goal-of-care discussion domain score | 1.0 | 0.27 (0.4) | 0.67 (0.02) | ||
| Documentation domain score | 1.0 | 0.70 (0.01) | |||
| Overall ACCEPT quality indicator score | 1.0 |
Note: ACCEPT = Audit of Communication, Care Planning, and Documentation.
This table describes the associations between the various domains and overall site ACCEPT quality indicator score and the concordance measure (the extent to which the patient’s preference matches the documentation in the medical record). The first number in each cell represents the correlation coefficient; the second number in brackets is the p value.
Table 7:
Associations between site advance care planning domain score and site CANHELP scores (n = 12 sites)*
| Correlation of advance care planning score CANHELP component scores | Pearson correlation coefficient (p value) |
|---|---|
| Rating of general satisfaction | 0.27 (0.4) |
| Combined “Relationship with doctors,” “Communication” and “Decision-making” domain scores | 0.10 (0.8) |
| Decision-making domain score | 0.17 (0.6) |
| q16 — Where you would be cared for | −0.24 (0.4) |
| q17 — Discussion about use of life-sustaining treatments | −0.01 (1.0) |
| q19 — Patient’s wishes for future care | 0.06 (0.9) |
Note: CANHELP = Canadian Health Care Evaluation Project.
This table explores the associations between the advance care planning domain score and various aspects of the CANHELP questionnaire (overall score, domain scores, and several individual items).
Interpretation
In this multicentre audit of end-of-life communication and decision-making practices, we implemented a quality indicator measurement framework to show the feasibility of multisite measurements, evaluate its internal consistency, add further construct validation to the quality indicator scoring system, and discriminate among sites based on the quality of their end-of-life communication and decision-making. The purpose of these quality indicators and performance rankings is to improve quality of care by emphasizing best practices, ensuring accountability of providers and stimulating healthy competition.16
Importantly, our analysis was based on an a priori conceptual framework. In this framework, we positioned concordance between a patient’s expressed wishes and documented orders for life-sustaining treatments as the reference measure. We observed that our overall quality indicator score correlated strongly with this measure and, consistent with our hypothesis, the correlations between the goal-of-care discussion domain and the concordance measure were greater than the correlation between the advance care planning domain and the concordance measure. Although we observed significant variability in hospital performance, overall and domain quality indicator scores were low. Notwithstanding this fact, we were able to discriminate the highest-ranked hospitals from the lowest-ranked hospitals.
We and others have previously documented substantial quality concerns related to end-of-life communication and decision-making in patients in hospital.1–6 Measuring quality indicators related to these important processes may be a stimulus for improvement. However, few health care organizations routinely measure the quality of end-of-life care in general, and even fewer routinely evaluate the quality of communication and decision-making.8 Recently, some quality indicators were developed in the broad field of palliative/end-of-life care.17,19,20 Some of these include aspects of communication and decision-making, but these domains are not the focus of these measures. Accordingly, we developed9 and have now validated a novel set of quality indicators that are specifically related to end-of-life communication and decision-making.
Validation of quality indicators is done only infrequently.21 In our initial attempt to validate our quality indicators, we postulated that prehospital communication and decision-making (advance care planning) would have less impact on overall quality of communication and decision-making than in-hospital activities (goal-of-care discussion and documentation), as judged by the correlation of these domains with concordance between stated preferences and documented goals of care. Indeed, this is exactly what we found. However, we could not show any association between prehospital advance care planning activities and a measure of satisfaction (CANHELP questionnaire) used at admission to hospital (and meant to reflect the satisfaction with prehospital activities over the last 4 weeks). This weakens our attempt to provide some validation to the quality indicator measures. However, it may be that advance care planning activities conducted in the community do not influence satisfaction (as this is not the primary objective of advance care planning), or that satisfaction with advance care planning communication that had occurred before the 4 weeks prior to hospital admission was not captured by the time-limited framing of the CANHELP questionnaire.
Our findings do not mean that community-based advance care planning activities are not relevant for in-hospital goal-of-care discussion or documentation; they contribute by preparing patients for in-hospital communication and decision-making.12,18 But these complementary prehospital advance care planning activities are clearly not a substitute for goal-of-care discussion and documentation that should occur at the time of an admission to hospital. Furthermore, the lack of a strong association between advance care planning and our overall quality indicator score and concordance measure may, in part, be due to the disconnect between what happens in the community and what happens in the hospital. In health care systems where primary care is more directly related and integrated into acute care21 (unlike the current situation in Canada), there may be a stronger correlation between advance care planning and the overall quality indicator score and the concordance measure. To simplify the measurement process, some hospitals could omit the advance care planning domain from the measurement framework and focus their time and resources on the domains that they are better positioned to address (goal-of-care discussion). We also acknowledge that although our weights were empirically derived, they may not be reflective of the constituents served by other hospitals. In considering the adoption of this measurement framework to local hospitals, these weights could be either re-evaluated or omitted all together. Finally, we suggest that users omit the third and fifth items from the documentation domain to achieve results with higher internal consistency.
Limitations
One limitation of this study is the small sample size. The limited number of participants from each site contributes to imprecision with respect to the estimates of site scores. In addition, all of the participating hospitals were teaching hospitals; therefore, our findings may not be generalizable to nonteaching hospitals. We also acknowledge that our initial validation work is limited to hospital-based outcomes. We did not evaluate the association between our quality indicators and long-term outcomes. Finally, we did not assess the reliability of these quality indicators; further studies need to include this type of psychometric evaluation.
Conclusions
In conclusion, we have developed and validated quality indicators that can be used to measure the quality of end-of-life communication and decision-making. The results of this analysis suggest that it is feasible to implement these quality indicators in a variety of hospitals, that they are internally consistent and have construct validity, and that they can discriminate between highest-and lowest-ranked hospitals. Widespread adoption of this quality indicator framework could assist in process improvement for individual hospitals and improve the experience of patients who have serious illnesses, and that of their family members.
Footnotes
Competing interests: Jessica Simon reports grants from Alberta Innovates Health Solutions and personal fees from Alberta Health Services, during the conduct of the study. James Downar reports receiving fees for statistical analysis from Toronto General/Toronto Western Hospital Foundation and personal fees from Associated Medical Services Inc., during the conduct of the study. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Xuran Jiang conducted the analysis. All authors participated in the acquisition of data, the interpretation of results and the drafting of the manuscript, and approved the final version to be published.
Funding: This study was supported by funding from the Canadian Institutes of Health Research, The Michael Smith Health Services Research Foundation in British Columbia, Alberta Innovates, and the Alternative Funding Plan Innovation Fund in Ontario. The funders had no role in the design, interpretation of results or decision to publish.
References
- 1.Heyland DK, Barwich D, Pichora D, et al. ACCEPT (Advance Care Planning Evaluation in Elderly Patients) study team; Canadian Researchers at the End of Life Network (CARENET). Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med 2013;173:778–87. [DOI] [PubMed] [Google Scholar]
- 2.Field MJ, Cassel CK, editors. Approaching death: improving care at the end of life. Washington: National Academy Press; 1997. [PubMed] [Google Scholar]
- 3.Carstairs S, Beaudoin GA. Quality end of life care: the right of every Canadian. Parliament of Canada; 2000. [Google Scholar]
- 4.Carstairs S. Still not there. Quality end-of-life care: a progress report. Ottawa: Canadian Hospice Palliative Care Association; 2005. [Google Scholar]
- 5.The 2015 Quality of Death Index: ranking palliative care across the world. London/NewYork/Hong Kong: The Economist Intelligence Unit; 2015. Available: www.eiuperspectives.economist.com/healthcare/2015-quality-death-index (accessed 2016 Apr. 19). [Google Scholar]
- 6.Teno JM, Freedman VA, Kasper JD, et al. Is care for the dying improving in the United States? J Palliat Med 2015;18:662–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Curtis JR, Engelberg RA. Measuring success of interventions to improve the quality of end-of-life care in the intensive care unit. Crit Care Med 2006;34(Suppl):S341–7. [DOI] [PubMed] [Google Scholar]
- 8.Weiss A, Downar J. Ontario hospitals are not using palliative care performance indicators in their balanced scorecards. J Pain Symptom Manage 2013;46:e1–5. [DOI] [PubMed] [Google Scholar]
- 9.Sinuff T, Dodek P, You JJ, et al. Improving end-of-life communication and decision making: the development of a conceptual framework and quality indicators. J Pain Symptom Manage 2015;49:1070–80. [DOI] [PubMed] [Google Scholar]
- 10.Stelfox HT, Straus SE. Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J Clin Epidemiol 2013;66:1328–37. [DOI] [PubMed] [Google Scholar]
- 11.Howard M, Bernard C, Tan A, et al. Advance care planning: let’s start sooner. Can Fam Physician 2015;61:663–5. [PMC free article] [PubMed] [Google Scholar]
- 12.Canadian Researchers at the End of Life Network [homepage]. Kingston (ON) Available: www.thecarenet.ca/784-researchers (accessed 2017 July 11). [Google Scholar]
- 13.Pattison M, Romer AL. Improving care through the end of life: launching a primary care clinic-based program. J Palliat Med 2001;4:249–54. [DOI] [PubMed] [Google Scholar]
- 14.Heyland DK, Pichora D, Dodek P, et al. The development and validation of a questionnaire to audit advance care planning. J Palliative Care Med 2012;2:119. [Google Scholar]
- 15.Heyland DK, Cook DJ, Rocker GM, et al. Canadian Researchers at the End of Life Network. The development and validation of a novel questionnaire to measure patient and family satisfaction with end-of-life care: the Canadian Health Care Evaluation Project (CANHELP) questionnaire. Palliat Med 2010;24:682–95. [DOI] [PubMed] [Google Scholar]
- 16.Adab P, Rouse AM, Mohammed MA, et al. Performance league tables: the NHS deserves better. BMJ 2002;324:95–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.De Roo ML, Leemans K, Claessen SJ, et al. EURO IMPACT. Quality indicators for palliative care: update of a systematic review. J Pain Symptom Manage 2013; 46:556–72. [DOI] [PubMed] [Google Scholar]
- 18.Sudore RL, Fried TR. Redefining the “planning” in advance care planning: preparing for end-of-life decision making. Ann Intern Med 2010;153:256–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Pasman HR, Brandt HE, Deliens L, et al. Quality indicators for palliative care: a systematic review. J Pain Symptom Manage 2009;38:145–56. [DOI] [PubMed] [Google Scholar]
- 20.Dy SM, Kiley KB, Ast K, et al. Measuring what matters: top-ranked quality indicators for hospice and palliative care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association. J Pain Symptom Manage 2015;49:773–81. [DOI] [PubMed] [Google Scholar]
- 21.Hammes BJ, Rooney BL, Gundrum JD. A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc 2010;58:1249–55. [DOI] [PubMed] [Google Scholar]