Skip to main content
PMC home page
PMC Canada Author Manuscripts logoLink to PMC Canada Author Manuscripts
. Author manuscript; available in PMC: 2013 Aug 21.
Published in final edited form as: Cancer. 2008 May 15;112(10):2301–2308. doi: 10.1002/cncr.23428

Toward Population-based Indicators of Quality End-of-Life Care

Testing Stakeholder Agreement

Eva Grunfeld 1,2,3,4, Robin Urquhart 1, Eric Mykhalovskiy 5, Amy Folkes 1, Grace Johnston 3,6, Frederick I Burge 4, Craig C Earle 7, Susan Dent 8
PMCID: PMC3749155  CAMSID: CAMS3258  PMID: 18361447

Abstract

BACKGROUND

Quality indicators (QIs) are tools designed to measure and improve quality of care. The objective of this study was to assess stakeholder acceptability of QIs of end-of-life (EOL) care that potentially were measurable from population-based administrative health databases.

METHODS

After a literature review, the authors identified 19 candidate QIs that potentially were measurable through administrative databases. A modified Delphi methodology, consisting of multidisciplinary panels of cancer care health professionals in Nova Scotia and Ontario, was used to assess agreement on acceptable QIs of EOL care (n = 21 professionals; 2 panels per province). Focus group methodology was used to assess acceptability among patients with metastatic breast cancer (n = 16 patients; 2 groups per province) and bereaved family caregivers of women who had died of metastatic breast cancer (n = 8 caregivers; 1 group per province). All sessions were audiotaped, transcribed verbatim, and audited, and thematic analyses were conducted.

RESULTS

Through the Delphi panels, 10 QIs and 2 QI subsections were identified as acceptable indicators of quality EOL care, including those related to pain and symptom management, access to care, palliative care, and emergency room visits. When Delphi panelists did not agree, the principal reasons were patient preferences, variation in local resources, and benchmarking. In the focus groups, patients and family caregivers also highlighted the need to consider preferences and local resources when examining quality EOL care.

CONCLUSIONS

The findings of this study should be considered when developing quality monitoring systems. QIs will be most useful when stakeholders perceive them as measuring quality care.

Keywords: end-of-life care, quality indicators, health services, quality care

Quality of care has been defined as the extent to which health-care services for individuals and populations increase the likelihood of desired outcomes and are consistent with current professional knowledge.1 Developing population-based measures of end-of-life (EOL) care has been identified as essential to improve the quality of EOL care.1–3 However, research in measuring EOL care has been slower than in other stages of the cancer care continuum.4,5

For most areas of health services, quality care assessment stems from evidence-based guidelines, yet limited evidence exists for quality measurement in EOL care.5,6 Commonly used endpoints, such as prevention of mortality, morbidity, or progression of disease,4,7 do not assess the quality of EOL care. Therefore, measurement of quality EOL care must be grounded in measures of structure (ie, access, availability, and organization of care) and process (ie, technical and interpersonal aspects of care).8–11 Quality care concepts, however, must be translated into measurable indicators before the quality of EOL care can be monitored at the population level.4

Quality indicators (QIs) are tools designed to measure and improve quality of care.2,7,12 A system of QIs should assess care in a population to obtain stable benchmarks (ie, standards) of quality performance.6 Tools to measure quality of care for individuals at EOL must be clinically meaningful, administratively manageable, and psychometrically sound.5 Moreover, these QIs must be acceptable to relevant stakeholders as truly measuring facets of quality healthcare.6

The overall objective of our research was to identify QIs of EOL care that are population-based and measurable by using administrative health databases. The specific objective of this study was to assess the acceptability of these QIs among stakeholder groups: cancer care health professionals, patients, and family caregivers. The EOL period was defined as a patient’s final 6 months of life. A companion study, which examined the validity, reliability, and measurability of these QIs, is reported elsewhere.13

MATERIALS AND METHODS

QI Selection

We identified 71 candidate QIs for EOL care based on extensive work that was undertaken by other organizations,4,5,8,10,14–18 previous work by members of our research team,19,20 and a review of relevant organizations’ websites.3,9,16,21 The 71 QIs included indicators related to access/availability; care processes; satisfaction and communication; and physical, psychological, social, and spiritual well being. A multidisciplinary team of experienced EOL researchers (ie, members of this research team) reviewed these QIs for their potential to be measured from currently existing administrative health databases in Nova Scotia and Ontario. Specifically, the review involved developing operational definitions for each QI based on the data available from administrative health databases.

If an operational definition could be formulated, then the QI was included in a final list of potentially measurable QIs (see Table 1). For each QI, the explicit metric by which it would be measured, including the numerator and denominator, was defined. Measurability of these QIs is reported elsewhere13: Seven QIs were identified as fully measurable, 3 to 7 QIs were identified as partially measurable (depending on the province), and 5 to 9 QIs were not measurable (depending on the province) (see Table 2). A QI can be partially measurable for a variety of reasons: For example, whereas enrollment in palliative care programs can be measured, data are available for only a subset of the population.

TABLE 1.

Quality Indicators of End-of-Life Care Considered Potentially Measurable From Population-based Administrative Health Databases

Quality indicator Rationale
1. RTx for uncontrolled bone pain for patients with painful bony metastases (Asch 20009) Providing patients with RTx for bony metastases improves pain management and improves quality of life
2. Potent antiemetic treatment for highly emetogenic CTx (Asch 20009) Potent antiemetic therapy for highly emetogenic CTx treatments greatly controls nausea and vomiting
3. Interval between last CTx and death (Earle 200319) Patients receiving CTx in the last 2 wk before death is an indicator of poor judgment and tendency toward aggressive management
4. Interval between new CTx and death (Earle 200319) Short interval between new CTx regimen and death indicates poor quality
5. Site of death (Earle 200319) High proportion of deaths in hospital indicates poor-quality care
6. Frequency of ER visits (Earle 200319) Although complications often cannot be prevented, high rates of ER visits indicate that attention has not been paid to symptomatic issues or that resources are not available to manage them in a timely manner
7. a) Hospital d near the end of life (Earle 200319) Hospital stays in the terminal period of cancer may indicate poor-quality care
 b) ICU d near the end of life (Earle 200319) ICU d in the terminal period of cancer may indicate poor-quality care
8. a) Enrollment in palliative care within 6 mo of death19 (Earle 200319) Low rate of use of palliative care close to death indicates poor-quality care
 b) Enrollment in palliative care within 3 d of death19 (Earle 200319) Contact with palliative care program or palliative care physician only three days before death is considered poor-quality care
9. Multidisciplinary care (Earle 200319; Johnston 200220) Multidisciplinary care is considered an element of quality cancer care, because it provides patients with access to practitioners with different perspectives and skill sets to achieve holistic care
10. Adverse events (Nerenz 20014) High proportion of adverse events may indicate poor control of symptoms and poor-quality care
11. Continuity of care (Teno 200118; Johnston 200220) Coordination of care by 1 provider and coordination of records indicates good-quality care
12. a) Time and location of care in last 2 wks of life (Johnston 200220) It is assumed that the ideal is clinician visits that are more frequent as death approaches
 b) Time and location of care at monthly visits (Johnston 200220) Medical attention is likely to increase and may require more visits to the patient at home as symptoms worsen and death approaches
13. Community-centered services (Johnston 200220) Well planned end-of-life care means that many care needs can be anticipated and arranged for provision in home
14. Access to care (Teno 200118) Fair and equitable access to care for patient and family, regardless of financial considerations, indicates good-quality care
15. Access to palliative care (Canadian Strategy for Cancer Control3) Palliative care will be accessible to all patients and families with a cancer diagnosis, in a timely manner, throughout the entire duration of their disease
16. Regular palliative care assessments (Canadian Strategy for Cancer Control3) Regular patient assessment allows for opportunities to control symptoms and to assess individual and family needs and treatment preferences
17. Physician reimbursement for home visits (Canadian Strategy for Cancer Control3) Reimbursement schedules will be provided to primary care providers for providing palliative care closer to home
18. Advance care directives (Teno 200118) Evidence shows that advance care preferences are a dominant concern for patients at end-of-life
19. Assessment of financial and caregiving resources (Teno 200118) Literature indicates that patients and their families often experience fear of financial and physical burdens
Open in a new tab

RTx indicates radiation therapy; CTx, chemotherapy; ER, emergency room; ICU, intensive care unit.

TABLE 2.

Cancer Care Health Professionals’ Delphi Ratings on Quality Indicator Acceptability: Meaningfulness and Importance

Quality indicator Meaningfulness
Importance
Median % Agreeing Median % Agreeing
QIs that demonstrated agreement and acceptability
 1. RTx for uncontrolled bone pain for painful bone mets* 9 95 9 90
 2. Potent antiemetics for emetogenic chemotherapy† 9 95 9 85
 6. Frequency of ER visits‡ 7 86 8 81
 7. b) ICU d near the end of life‡,§ 8 86 6.5 50
 8. a) Enrollment in palliative care within 6 mo of death* 7 70 7 70
  b) Enrollment in palliative care within 3 d of death*,§ 7 79 7 61
 9. Multidisciplinary care† 8 83 8 72
 12. b) Time and location of care (monthly visits)‡ 8 76 8 75
 14. Access to care† 8.5 95 8.5 94
 15. Access to palliative care* 9 95 8 89
 16. Regular palliative care assessments 8 89 8 88
 18. Advance care directives||, 8 58 8 72
 19. Assessment of financial and caregiving resources 8 75 8 84
QIs that demonstrated disagreement**
 3. Interval between last chemotherapy and death‡ 5.5 40 5 25
 4. Interval between new chemotherapy and death* 5.5 35 5 35
 5. Site of death‡ 3 60 4 35
 7. a) Hospital d near end of life‡ 6 40 6 37
 10. Adverse events‡ 7 58 7 56
 11. Continuity of care‡ 6.5 50 7 58
 12. a) Time and location of care in last 2 wk of total visits‡ 6.5 50 6.5 50
 13. Community-centered services 7 65 8 68
 17. Physician reimbursement for home visits 4 24 8 68
Open in a new tab

QI indicates quality indicator; mets, metastasis; ER, emergency room; ICU, intensive care unit.

*

On subsequent research, QI was partially measurable in Nova Scotia but was unmeasurable in Ontario. For full details on measurability findings, see Grunfeld 2006.13

†

On subsequent research, QI was partially measurable in both provinces (see Grunfeld 200613).

‡

On subsequent research, QI was fully measurable in both provinces (see Grunfeld 200613).

§

QIs demonstrated agreement on meaningfulness only.

||

QI demonstrated agreement on importance only.

On subsequent research, QI was unmeasurable in both provinces (Grunfeld 200613).

**

According to RAND agreement criteria, disagreement indicates uncertainty on the acceptability of the QI.

Study Participants

Stakeholder acceptability was assessed for each of the potentially measurable 19 QIs. Participants were cancer care health professionals involved in different aspects of EOL care (n = 21: 3 medical oncologists, 5 radiation oncologists, 4 palliative care physicians, 2 family physicians, 4 registered nurses, 2 social workers, and 1 nurse therapist), women with metastatic breast cancer (n = 16), and bereaved family caregivers (spouses, siblings, and children) of women who had died of metastatic breast cancer (n = 8).

Purposive sampling was used to select cancer care health professionals to represent a multidisciplinary group with extensive experience in EOL care. Patients were recruited though local cancer centers; family caregivers of women who died of metastatic breast cancer were recruited through caregiver support groups and palliative care teams. The ethical review boards at both institutions approved the study, and all participants gave written informed consent.

Delphi and Focus Group Methods

The modified Delphi methodology was used to reach agreement or disagreement on the acceptability of the 19 QIs among cancer care health professionals.9,19,22 This approach is a validated method for obtaining expert opinion.9 Four multidisciplinary Delphi panels were held at institutions in Nova Scotia and Ontario (2 panels per province). Each panel included 4 to 7 cancer care health professionals.

The modified Delphi methodology involved an anonymous first round in which panelists individually rated the meaningfulness (does the indicator truly measure the quality of care?) and importance (does the indicator reflect an important problem in quality of care?) of each indicator on a scale from 0 to 10 (0 = not meaningful/important; 10 = meaningful/important). Each QI description included specific metrics to ensure that panelists understood how the indicator could be measured (for detailed metric descriptions, see Grunfeld et al.13). After the first rating, panelists attended a group session, which was moderated by one of the authors (E.G.), to discuss the 19 QIs. At this session, panelists were given their initial ratings and group norms (ie, medians, ranges) for each QI. A discussion ensued, followed by a second anonymous rating. The panel rating scale and script were based on work conducted by Earle et al.19 to build on that previous research.

Focus group methodology was used to assess acceptability among patients (2 groups per province) and family caregivers (1 group per province). Focus group scripts were developed to discuss quality-of-care issues involving cancer patients at EOL, with specific focus on the 19 QIs.13 Participants were probed frequently with key phrases related to these QIs (ie, chemotherapy near death, site of death).

Analysis

For the Delphi panel ratings, descriptive statistics were calculated for each QI using the statistical software package SAS (version 9.1). Median scores and ranges were used to determine whether the panelists agreed on the meaningfulness and importance of each QI. Agreement was based on the number of panelists rating outside the region containing the median score (ie, agreement was established if ≥70% of panelists rated within the region).22 Either meaningfulness or importance scores were used to assess acceptability.

Both the Delphi panel and focus group discussions were audiotaped, transcribed verbatim, and audited. Thematic analyses were conducted according to standard qualitative methods,23,24 which involved coding, categorizing, theme identification, and constant comparison. Coding was performed manually before using the qualitative software package QSR NUD.IST to further support the coding process. Rigor of analysis was maximized by the coding of each transcript independently and then comparison of codes and categories by 3 members of the research team (E.G., A.F., R.U.). Areas of divergence were discussed and reconciled. The transcripts were used to identify representative quotations. More detailed information regarding the focus group methodology is reported elsewhere.25

RESULTS

Table 2 summarizes the results of the Delphi ratings. Cancer care health professionals identified 10 QIs and 2 QI subsections as acceptable indicators of quality EOL care. The meaningfulness and importance ratings were aligned closely, with the exception of 3 indicators: advance care directives, enrollment in palliative care within 3 days of death, and ICU days near EOL. Whereas advance care directives were deemed an important problem in EOL care, the Del-phi panelists did not believe the QI measured the quality of care provided; conversely, panelists indicated that enrollment in palliative care within 3 days of death and ICU days near EOL measured quality of care but did not reflect important problems in care provision, because they are not frequent occurrences. Table 3 presents the measurability of those indicators that were identified as acceptable (either meaningful or important) through the Delphi process.

TABLE 3.

Measurability of Quality Indicators Deemed Acceptable by Cancer Care Health Professionals

Quality Indicator Measurability*
Nova scotia Ontario
1. RTx for uncontrolled bone pain for painful bone mets P N
2. Potent antiemetics for emetogenic chemotherapy P P
6. Frequency of ER visits F F
7. b) ICU d near the end of life F F
8. a) Enrollment in palliative care within 6 mo of death P N
 b) Enrollment in palliative care within 3 d of death P N
9. Multidisciplinary care P P
12. b) Time and location of care (monthly visits) F F
14. Access to care P P
15. Access to palliative care P N
16. Regular palliative care assessments N N
18. Advance care directives N N
19. Assessment of financial and caregiving resources N N
Open in a new tab

RTx indicates radiation therapy; mets, metastasis; P, partially measurable; N, not measurable; ER, emergency room; F, fully measurable; ICU, intensive care unit.

*

For full details on measurability findings, see Grunfeld 2006.13

Analysis of the focus group data revealed support for those indicators related to the provision of palliative care and multidisciplinary care. In particular, early and continuing access to palliative care services and EOL-specific information were highly valued. In addition, patients and family caregivers emphasized continuity of care concepts as integral to high-quality EOL care (Table 4).

TABLE 4.

Patients’ and Caregivers’ Quotations Supporting Quality Indicators

Access to and early enrollment in palliative care programs represents quality care
 [Patient] “They are sort of preparing me for what would happen. . .Now, I know how the network works. I didn’t understand before.”
 [Caregiver] “They said to call them whenever I needed them, and they gave me a number to contact. I said ’I am taking up your time,’ and they said ’no, that is what we are here for.”’
Continuity of care represents quality care
 [Patient] “I had to explain my situation over again. It wasn’t written down anywhere.”
 [Caregiver] “My wife didn’t know the reason why he moved her to palliative care and he didn’t see her any more.”
Multidisciplinary care represents quality care
 [Patient] “Social work identified that I needed to see [a psychiatrist] because I was a basket case. . .it was a good experience.”
 [Caregiver] “I found the nurses were excellent. . .they had lots of tricks for Mom to get more comfortable and treated us really well.”
Open in a new tab

Qualitative analysis of the Delphi panel discussions revealed 3 themes that helped explain the disagreement among cancer care health professionals. Specifically, the themes involved 1) patient preferences at EOL, 2) knowledge of local resources, and 3) benchmarking (Table 5). Patients and family caregivers also highlighted the need to consider preferences and local resources when examining EOL care.

TABLE 5.

Patients’, Caregivers’, and Health Professionals’ Quotations Supporting Themes

Patient preferences at end-of-life should inform care
 [Cancer care health professional] “There is something very important to certain patients where maintaining ongoing delivery of care, whatever that is, is in their best interest. . .we spend an inordinate amount of time discussing options, trying to respect their wishes. It may be that their wish is for more palliative care. It may be that it is further therapy at all cost.”
 [Patient] “I never had chemotherapy or radiation, and the decisions around that are very important to me.”
 [Caregiver] “Her radiologist, her oncologist, her counselors, and everybody else, they made it clear to her that [the treatment] probably won’t do any good, but we [could] proceed and at least maybe try it. So basically they left it up to us. We would discuss it and that’s where I consider ourselves a little lucky.”
Knowledge of local resources should inform quality indicator use
 [Cancer care health professional] “Patients may be dying in their local acute care hospital and quite appropriately because they can get access to the palliative care services that they need and they’re close to home. I think it is a very suitable place to die.”
 [Caregiver] “She could flare with pain at any given moment, at any given time, in the middle of the night. . .You just can’t get a prescription filled in the middle of the night. It’s not that easy.”
The need to understand the concept of benchmarking (cancer care health professionals only)
 “[T]he temptation to interpret a high rate of admission to emergency room as inadequate care may be very incorrect. . .I see benchmarking but I fear use of that sort of data to draw conclusions about quality of care when there are things that happen for no foreseeable reason.”
 “I can’t see, for example, saying Dr. [Surname] has very poor quality of care because 16 of her patients have had a fall in hospital recently.”
Open in a new tab

Patient Preferences at EOL Should Inform Care

Patients’ preference, such as treatment decisions and care settings at EOL, has been identified by seriously ill patients as one of the most important quality care indicators during advanced illness.26 Respect for patient preferences was the primary reason Delphi panelists disagreed on the acceptability of several QIs.

Despite research suggesting that a short interval between the last chemotherapy regimen and death19 and a high rate of hospital deaths19,27 may indicate poor-quality care, Delphi panelists disagreed with both QIs, because they believed that individual patient preferences should guide these care decisions. Similarly, patients and family caregivers emphasized the need to offer patients a choice in their care, particularly surrounding treatment and site of death decisions.

Knowledge of Local Resources Should Inform QI Use

Delphi panelists indicated that the meaningfulness of many of the QIs, as they are measured currently, depends on the availability of local resources. This was true for both site of death and hospital days near death, again contributing to the disagreement. In instances in which patients do not have adequate home and supportive care resources, panelists viewed dying in hospital as signaling good-quality care. In the focus groups, family caregivers also emphasized the need to consider community and family resources when considering quality-of-care issues and hospital use at EOL.

The Need to Understand the Concept of Benchmarking

Delphi panelists had difficulty embracing the notion that QIs are meant to explore quality-care issues at the population level, not at the individual patient level. Accordingly, the concept of benchmarking must be understood better for QIs to be acceptable to healthcare professionals. An example is the QI related to adverse events. A certain proportion of patients will experience adverse events. However, the QI is meant to assess the proportion of adverse events in a particular population and compare that proportion with some agreed upon ‘endemic’ benchmark for the population. It is not meant to judge the performance of individual healthcare professionals.28

DISCUSSION

Interpretation

To effectively improve the quality of EOL care at the population level, first, we must identify and develop feasible, valid, reliable, and acceptable indicators that can be measured and compared across jurisdictions. We assessed the acceptability of 19 QIs of EOL care that potentially are measurable from population-based administrative health databases. Cancer care health professionals agreed on the acceptability of many of the candidate QIs, including those related to pain and symptom management, access to care, palliative care, and emergency room visits. These are congruent with several indicators that were included in recent quality-monitoring initiatives.29–31 Patients and family caregivers considered palliative care, multidisciplinary care, and continuity of care as elements of high-quality EOL care. The current study, together with our companion study on the measurability of these QIs (feasibility, validity, and reliability),13 make the results highly relevant to current quality monitoring initiatives.

It has been suggested that monitoring the quality of cancer care should assess the overuse of therapies with limited expectation of benefit.7 Earle et al.19 proposed that continuation of chemotherapy near death may indicate poor judgment and aggressive use of therapy.32 In an Ontario cohort of patients who died of cancer, continuation of chemotherapy near death was associated inversely with home care, palliative care assessment, and physician house calls.33 Nevertheless, cancer care health professionals, patients, and family caregivers in our study did not agree that chemotherapy use near death was an acceptable indicator of quality EOL care. The overriding opinion was that patient preferences for treatment should direct chemotherapy use. Patients at EOL have reported that they want to retain control of their care decisions,34 including those related to treatment options.26 However, QIs must be understood in context. Chemotherapy use near death may indicate patient preferences, yet it also may indicate poor communication, including avoiding discussion of EOL issues and advance directives.19

Our cancer care health professionals did not rate site of death as an acceptable indicator of quality EOL care, in part because of the highly personal nature of preferences for site of death and uncertainty regarding the community-based resources that are available to provide care in the home. This is surprising given evidence that most patients would choose to die at home or outside of an acute care setting.27,35 However, a multitude of factors affect where patients actually die: A systematic review demonstrated that the most important factors influencing home death included functional status, the expression of preference, home care use, and social supports/living arrangements.36

Furthermore, there are strong associations between resource availability and hospital use at EOL, including death in hospital.27,37 In the Study to Understand Prognoses and Preferences for Risks and Outcomes of Treatment (SUPPORT) study,27 variations in site of death were not explained by patient preferences or characteristics; rather, local health system characteristics (ie, hospital bed, nursing home, and hospice availability) strongly influenced where patients died. In Nova Scotia, out-of-hospital deaths have been associated with admission to palliative care programs.38 Therefore, although well planned EOL care should mean that needs can be anticipated and met in the home,20 hospital visits and deaths in acute care settings may not be the result of poor planning but, rather, a lack of appropriate palliative and community-based resources.19 The value of hospital use when community-based and home-based resources are lacking arose in both the cancer care health professionals’ and family care-givers’ discussions, in which participants viewed hospital admissions as high-quality care, because patients were able to access the hospital bed and, consequently, the services within the facility. Thus, measures like site of death may point to problems in the availability of local resources.

Coordination of care by 1 provider18,20,30 and continuity of physician contacts19 have been proposed as measures of good-quality care for patients at EOL.39 Our cancer care health professionals did not agree on the acceptability of continuity of care as an indicator of quality EOL care. However, through qualitative analysis, it was evident that the basis of the disagreement was the continuity of care metric proposed, in which continuity of care essentially declined linearly as the number of providers increased, not the concept of continuity of care (for detailed measurement criteria, see Grunfeld et al.13). This points to the need for further work on how to measure continuity of care, particularly because our patients and family caregivers viewed continuity of care as integral to quality EOL care.

QIs are not meant to direct care at the patient level7; rather, they aid in benchmarking and allow us to compare care across institutions or regions.10 Each of the processes involved in these measures may be entirely appropriate in individual episodes of care. However, if a provider organization is an ‘outlier’ with a very high proportion of patients receiving chemotherapy close to death and/or dying in acute care settings, then it may be a ‘red flag’ for potential problems, such as a culture of aggressive practice style or a lack of supportive and/or palliative care services in the region.33,37 To ensure the efficacy of QI development, it is imperative to educate health-care professionals on quality-improvement methodologies and to get them to understand and trust the benchmarking and quality-improvement processes more fully. This will involve effective knowledge translation strategies throughout all phases of quality monitoring and improvement.

Participants’ views should be considered when developing quality-monitoring systems. QIs will be most useful when stakeholders perceive them as measuring quality care.6 Because our disease model was breast cancer, further research will be required to determine whether the QIs are both acceptable and applicable to other disease sites and to different subpopulations (ie, men, cultural/ethnic or socioeconomic groups). In addition, jurisdictions will have to assess their local context (ie, structures, resources) to determine whether the QIs we have identified are relevant to their healthcare systems. In the current study, we assessed acceptability in 2 Canadian provinces, thereby exploring whether potentially measurable QIs are relevant within the local context.

The qualitative evidence underscored the finding that many aspects of quality EOL care will involve the degree to which the provision of care reflects the individual preferences of patients and families.4 The capacity of current population-based health databases to capture data on preferences is limited. Thus, many important QIs are not amenable to measurement from such databases (ie, satisfaction, communication). An inclusive set of QIs should include tools to assess these concepts25,30 and eventually measure them systematically at a population level. A final set of indicators also will require rigorous testing to establish reliability, feasibility, and validity data,8 a process we have commenced.13

In conclusion, cancer care health professionals agreed on several QIs of EOL care; they disagreed with some QIs principally because of concern for patient preferences and the need to assess local resource availability when defining the indicators. Although the views of patients and family caregivers were concordant with the views of cancer care health professionals on several indicators, including the merit of palliative and multidisciplinary care, they also highlighted the need to consider preferences and local resources when examining quality EOL care. The findings point to the need to educate stakeholders about QI improvement methodologies.

Acknowledgments

Supported by the Canadian Breast Cancer Research Alliance and the Canadian Institutes of Health Research (Grant 900310).

We thank all of the healthcare professionals, patients, and caregivers in Nova Scotia and Ontario who participated in this study. We also thank and recognize Lisa Cicchelli for her commitment to the implementation and completion of the study protocol.

References

  • 1.Donaldson M, Field M. Measuring quality of care at the end of life. Arch Intern Med. 1998;158:121–128. doi: 10.1001/archinte.158.2.121. [DOI] [PubMed] [Google Scholar]
  • 2.Carstairs S Subcommittee of the Standing Senate Committee on Social Affairs Science and Technology. [Accessed March 11, 2008];Quality end-of-life care: the right of every Canadian. Available at: http://www.parl.gc.ca/36/2/parlbus/commbus/senate/com-e/upda-e/rep-e/repfinjun00-e.htm.
  • 3.Canadian Strategy for Cancer Control. [Accessed on March 11, 2008];Report of the Palliative Care Working Group. Available at: http://www.virtual-hospice.ca/Resources/ViewDownloadableResource.asp?lngDownloadableResourceID=185&lngDownloadableResourceCategoryID=3&lngpagerequested=1.
  • 4.Nerenz D. Capacities and limitations of information systems as data sources on quality of care at the end of life. J Pain Symptom Manage. 2001;22:773–783. doi: 10.1016/s0885-3924(01)00329-3. [DOI] [PubMed] [Google Scholar]
  • 5.Teno J, Byock I, Field M. Research agenda for developing measures to examine quality of care and quality of life of patients diagnosed with life-limiting illness. J Pain Symptom Manage. 1999;17:75–82. doi: 10.1016/s0885-3924(98)00134-1. [DOI] [PubMed] [Google Scholar]
  • 6.Earle CC, Neville BA, Landrum ME, et al. Evaluating claims based indicators of the intensity of end-of-life cancer care. Int J Qual Health Care. 2005;17:505–509. doi: 10.1093/intqhc/mzi061. [DOI] [PubMed] [Google Scholar]
  • 7.Schneider E, Epstein A, Malin J, Kahn K, Emanuel EJ, Magnuson W. Developing a system to assess the quality of cancer care: ASCO’s National Initiative on Cancer Care Quality. J Clin Oncol. 2004;22:2985–2991. doi: 10.1200/JCO.2004.09.087. [DOI] [PubMed] [Google Scholar]
  • 8.Mandelblatt JS, Ganz PA, Kahn KL. Proposed agenda for the measurement of quality-of-care outcomes in oncology practice. J Clin Oncol. 1999;17:2614–2622. doi: 10.1200/JCO.1999.17.8.2614. [DOI] [PubMed] [Google Scholar]
  • 9.Asch SM, Kerr EA, Hamilton EG, Reifel JL, McGlynn EA. Quality of care for oncologic conditions and HIV: a review of the literature and quality indicators. Santa Monica, Calif: RAND Health; 2000. [Accessed March 11, 2008]. Available at: www.rand.org/publications/MR/MR1281/index.html. [Google Scholar]
  • 10.Fowler F, Coppola K, Teno J. Methodological challenges for measuring quality of care at the end of life. J Pain Symptom Manage. 1999;17:114–119. doi: 10.1016/s0885-3924(98)00133-x. [DOI] [PubMed] [Google Scholar]
  • 11.Stewart AL, Teno J, Patrick DL, Lynn J. The concept of quality of life of dying persons in the context of health care. J Pain Symptom Manage. 1999;17:93–108. doi: 10.1016/s0885-3924(98)00131-6. [DOI] [PubMed] [Google Scholar]
  • 12.Campbell S, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. BMJ. 2003;326:816–819. doi: 10.1136/bmj.326.7393.816. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Grunfeld E, Lethbridge L, Dewar R, et al. Towards using administrative databases to measure population-based indicators of quality end-of-life care: testing the methodology. Palliat Med. 2006;20:769–777. doi: 10.1177/0269216306072553. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Ganz PA. Measuring Health Care Quality: Breast Cancer. Portland, Ore: Foundation for Accountability; 1996. [Google Scholar]
  • 15.Patrick D, Engelberg R, Curtis J. Evaluating the quality of dying and death. J Pain Symptom Manage. 2001;22:717–726. doi: 10.1016/s0885-3924(01)00333-5. [DOI] [PubMed] [Google Scholar]
  • 16.Teno JM, McNiff K. Report Submitted to the Foundation for Accountability (FACCT) Portland, Ore: Foundation for Accountability; 1999. Measuring Quality of Medical Care for Dying Patients and Their Families Served by Managed Care Organizations. [Google Scholar]
  • 17.Cancer care during the last phase of life. J Clin Oncol. 1998;16:1986–1996. doi: 10.1200/JCO.1998.16.5.1986. No authors listed. [DOI] [PubMed] [Google Scholar]
  • 18.Teno J, Casey V, Welch L, Edgman-Levitan S. Patient focused, family-centred end-of-life medical care: views of the guidelines and bereaved family members. J Pain Symptom Manage. 2001;22:738–751. doi: 10.1016/s0885-3924(01)00335-9. [DOI] [PubMed] [Google Scholar]
  • 19.Earle C, Park ER, Lai B, Weeks JC, Ayanian JZ, Block S. Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol. 2003;21:1133–1138. doi: 10.1200/JCO.2003.03.059. [DOI] [PubMed] [Google Scholar]
  • 20.Johnston G, Burge F. Analytic framework for clinician provision of end-of-life care. J Palliat Care. 2002;18:141–149. [PubMed] [Google Scholar]
  • 21.National Cancer Policy Board. Indicators of cancer care quality available to consumers. Bethesda, Md: Institute of Medicine; 1997. [Google Scholar]
  • 22.Fitch K, Bernstein SJ, Aguilar MS, et al. [Accessed March 11, 2008];The RAND/UCLA Appropriateness Method user’s manual. Available at: http://www.rand.org/pubs/monograph_reports/MR1269.
  • 23.Lofland J, Lofland L. Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Belmont, Calif: Wadsworth; 1995. [Google Scholar]
  • 24.Rubin H, Rubin I. Qualitative Interviewing: The Art of Hearing Data. Thousand Oaks, Calif: Sage Publications; 1995. [Google Scholar]
  • 25.Grunfeld E, Folkes A, Mykhalovskiy E, et al. Do available questionnaires measure the communication factors that patients and families consider important at end-of-life? J Clin Oncol. doi: 10.1200/JCO.2007.15.8006. In press. [DOI] [PubMed] [Google Scholar]
  • 26.Yurk R, Morgan D, Franey S, Burk Stebner J, Lansky D. Understanding the continuum of palliative care for patients and their caregivers. J Pain Symptom Manage. 2002;24:459–470. doi: 10.1016/s0885-3924(02)00503-1. [DOI] [PubMed] [Google Scholar]
  • 27.Pritchard RS, Fisher ES, Teno JM, et al. Influence of patient preferences and local health system characteristics on the place of death. SUPPORT investigators: study to understand prognoses and preferences for risks and outcomes of treatment. J Am Geriatr Soc. 1998;46:1242–1250. doi: 10.1111/j.1532-5415.1998.tb04540.x. [DOI] [PubMed] [Google Scholar]
  • 28.Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med. 1989;320:53–56. doi: 10.1056/NEJM198901053200110. [DOI] [PubMed] [Google Scholar]
  • 29.Cancer Care Ontario. [Accessed March 11, 2008];Cancer system quality index 2007: patterns of end-of-life care. Available at: http://www.cancercare.on.ca/qualityindex2007/access/eol/index.html.
  • 30.Canadian Council on Health Services Accreditation. [Accessed March 11, 2008];Hospice palliative and end-of-life care [article online] Available at: http://www.cchsa.ca/default.aspx?page=58&cat=27.
  • 31.National Cancer Policy Board. Assessing the quality of cancer care: an approach to measurement in Georgia [article online] Washington, DC: National Academies Press; 2005. [Accessed March 11, 2008]. Available at: http://books.nap.edu/openbook.php?isbn=0309095697. [Google Scholar]
  • 32.Earle CC, Neville BA, Landrum MB, Ayanian JZ, Block SD, Weeks JC. Trends in the aggressiveness of cancer care near end of life. J Clin Oncol. 2004;22:315–321. doi: 10.1200/JCO.2004.08.136. [DOI] [PubMed] [Google Scholar]
  • 33.Barbera L, Paszat L, Chartier C. Indicators of poor quality end-of-life cancer care in Ontario. J Palliat Care. 2006;22:12–17. [PubMed] [Google Scholar]
  • 34.Singer PA, Martin DK, Kelner M. Quality end-of-life care: patients’ perspectives. JAMA. 1999;281:163–168. doi: 10.1001/jama.281.2.163. [DOI] [PubMed] [Google Scholar]
  • 35.Higginson I, Sen-Gupta G. Place of care in advanced cancer: a qualitative systematic literature review of patient preferences. J Palliat Med. 2000;3:287–300. doi: 10.1089/jpm.2000.3.287. [DOI] [PubMed] [Google Scholar]
  • 36.Gomes B, Higginson IJ. Factors influencing death at home in terminally ill patients with cancer: a systematic review. BMJ. 2006;332:515–521. doi: 10.1136/bmj.38740.614954.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK. Use of hospitals, physician visits, and hospice care during last 6 months of life among cohorts loyal to highly respected hospitals in the United States. BMJ. 2004;328:607–607. doi: 10.1136/bmj.328.7440.607. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Burge F, Lawson B, Johnston G. Trends in the place of death of cancer patients, 1992–1997. CMAJ. 2003;168:265–270. [PMC free article] [PubMed] [Google Scholar]
  • 39.Heyland DK, Dodek P, Rocker G, et al. What matters most in end-of-life care: perceptions of seriously ill patients and their family members. CMAJ. 2006;174:627–633. doi: 10.1503/cmaj.050626. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES