The Quality of Dying and Death

If you can’t measure it, you can’t improve it.

Lord Kelvin

Not everything that counts can be measured. Not everything that can be measured counts.

Albert Einstein

The juxtaposition of these two quotes is not novel, but it is particularly appropriate for a discussion of measuring the quality of dying and death. The quote by Lord Kelvin captures the challenge that gave rise to efforts to develop a measure of the quality of dying and death. In the past 3 decades, landmark studies highlighted the poor quality of end-of-life care, with many patients dying with significant pain and other symptoms while receiving a high intensity of life-sustaining treatments they did not want, and families often were left with tremendous emotional and financial burdens.^1-4 As a result of these findings, and in line with Lord Kelvin’s adage, efforts arose to develop accurate measures of the quality of the dying experience. The Quality of Dying and Death (QODD) questionnaire was one such measure developed from a conceptual model that separated the following three related concepts: the quality of end-of-life care, the quality of life at the end of life, and the quality of dying and death.⁵

In the 1990s, our research group set out to develop a measure of the quality of dying and death. We used focus groups; one-on-one interviews with patients, family members, and clinicians; and a literature review to create a 31-item measure that asked participants to rate each item on a scale from 0 to 10, where 0 was a terrible experience and 10 was an almost perfect experience.⁶ We chose to conduct these ratings after death to capture the full experience of dying, although this limited us to respondents other than the patient. We then conducted a series of studies that assessed the reliability and validity of this measure across a range of settings, including the community, hospice, hospital, and ICU.^7-11 We created a total score by simply summing the scores on the items that were completed and linearly transforming to a score from 0 to 100. Psychometric analyses of the QODD suggested moderate adherence to the following classic measurement standards: total scores with no floor or ceiling effects, an approximately normal distribution and good reliability based on Cronbach α (0.89); construct validity demonstrated by significant associations between the total score and other markers of quality of care (eg, symptom burden, patient-clinician communication, overall quality of care, medical record indicators of quality of care); and moderate interrater reliability among multiple family members.^7-12 The QODD has been studied in >4,000 deaths in multiple settings, regions, and countries and gained some prominence as a leading outcome measure in end-of-life care.^13-15

Unfortunately, the QODD also demonstrated important shortcomings. First, individual items often are skewed or missing, with many items receiving >25% endorsement at the floor or ceiling values. Second, the original qualitative work on which the QODD was based was done primarily with patients and clinicians rather than with family members, raising questions about the appropriateness of the instrument for the audience for whom it was primarily designed. Third, confirmatory factor analyses indicated unsatisfactory fit for a measurement model in which the QODD items served as indicators of a single latent variable and failed to support the six domains proposed in the original conceptual model.¹² Fourth, although the QODD includes implicit reference to respondent values by asking respondents to rate on a bad-to-good scale the dying person’s actual end-of-life experiences, it neither captures the details of those values nor allows respondents to weight the impact of each rating on the overall quality of dying. The absence of specific information about values makes individual ratings difficult to interpret. Finally, we have been unable to demonstrate the responsiveness of the QODD to interventions. Although we were able to identify a small, but statistically nonsignificant trend toward improved scores on the family QODD score in a before-and-after study of a palliative care intervention,¹⁶ a subsequent randomized trial of the same intervention found no intervention effect.¹⁷

In this context, this issue of CHEST (see page 357) contains an interesting article by Gerritsen and colleagues¹⁸ that examines the quality of dying and death for 100 consecutive patients in the ICU in The Netherlands. The QODD was translated into Dutch and administered to family members, physicians, and nurses. Family members completed the QODD on the phone with a research nurse, and clinicians completed a self-administered QODD. The study had a high response rate across all types of respondents. There are several key findings. First, the total QODD scores are quite high at 9 out of 10 for all types of raters and seem higher than those of similar studies conducted in various regions of the United States.^10,11,17 Whether this represents better quality of care in The Netherlands or regional differences in evaluators is not clear. Second, family members rated the quality of dying lower than physicians or nurses, especially in the domains of symptom control and communication. Third, a number of participants described difficulty in responding to some of the QODD items, and some items had high proportions of missing data, supporting the assertion that the QODD has important limitations. Fourth, the locus of decision-making shows a distribution that ranges from families reporting that physicians made decisions alone to family members reporting that they made decisions with information from the physicians. This distribution is not dissimilar to distributions seen in the United States and Canada, although there may be some regional differences in the distribution.^19,20

Given this research, is the QODD ready for use as an outcome measure? We would argue that it depends on the purpose of the study. We believe that the limitations of the QODD that Gerritsen and colleagues¹⁸ describe, as well as summarized herein, suggest that the QODD is not ready for use as a primary outcome in intervention studies. We do not yet know whether the quality of dying is a cohesive construct and whether the QODD works as a single score. Even if it is cohesive and works as a single score, we do not know whether it is responsive to interventions that improve the quality of dying. Given the 10 years that the QODD has been in development and the fact that >4,000 patient deaths have been studied with the QODD, this seems concerning. However, measurement of quality of life with the Short Form-36 (SF-36) (Medical Outcomes Trust), now a well-accepted primary outcome measure for intervention studies, took >20 years and 30,000 study patients to achieve its current status.²¹ The concept of quality of dying and the QODD require additional research if they are to achieve this status. We would also argue that the quality of dying is an important concept and the QODD a potentially promising measure that both warrant additional research.

Gerritsen and colleagues¹⁸ argue on the basis of the shortcomings they identified that the QODD may need local adaptation to achieve better reliability, validity, and responsiveness. It may be that dying and the experiences that surround it are so heavily influenced by the culture in which they occur that any measure must be adapted to specific circumstances. However, it is also possible that the experiences associated with dying may have universal and shared characteristics that may be evaluated with a generic measure, as has been done successfully for quality of life.²¹ In addition, reliance on measures that are specific to local circumstances would likely make it impossible to compare the quality of dying across regions and cultures.

As suggested by the statement from Albert Einstein, some components of the quality of end-of-life care and the quality of dying may be difficult or even impossible to measure. However, the development of reliable, valid, and responsive measures of the quality of dying as well as of the quality of life at the end of life and the quality of end-of-life care remain important goals that will better enable us to identify interventions to improve these critical aspects of health care and the human experience.