Conceptualizing and Counting Discretionary Utilization in the Final 100 Days of Life: A Scoping Review

Abstract

Context:

There has been surprisingly little attention to conceptual and methodological issues that influence the measurement of discretionary utilization at the end-of-life (DIALs), an indicator of quality care.

Objective:

To examine how DIALs have been operationally defined, and identify areas where evidence is biased or inadequate to inform practice.

Methods:

We conducted a scoping review of the English language literature published from 1/1/04-6/30/17. Articles were eligible if they reported data on ≥2 DIALs within 100 days of the deaths of adults ≥18 years old. We explored the influence of research design on how researchers measure DIALs and whether they examine demographic correlates of DIALs. Other potential biases were explored.

Results:

We extracted data from 254 articles published in 79 journals covering research conducted in 29 countries, mostly on cancer (69.1%). More than 100 DIALs have been examined. Relatively crude, simple variables (e.g., ICU admissions [56.9% of studies], chemotherapy [50.8%], palliative care [40.0%]) have been studied more frequently than complex variables (e.g., burdensome transitions; 7.3%). We found considerable variation in the assessment of DIALs, illustrating the role of norms and disciplinary habit. Variables are typically chosen with little input from the public (including patients or caregivers) and clinicians. Fewer than half the studies examined age (44.6%), gender (37.3%), race (26.5%), or socioeconomic (18.5%) correlates of DIALs.

Conclusions.

Unwarranted variation in DIALs assessments raises difficult questions concerning how DIALs are defined, by whom, and why. We recommend several strategies for improving DIALs assessments. Improved metrics could be used by the public, patients, caregivers, clinicians, researchers, hospitals, health systems, payers, governments, and others to evaluate and improve end-of-life care.

I. Introduction

End-of-life (EoL) care is an increasingly contested arena, as the industrialization of medicine,^1–5 rules of rescue,^6,7 medical assistance in dying^8,9 and healthcare costs¹⁰ are debated in professional journals, the popular press, and legislative bodies. Nowhere is this conflict more fraught than in the delivery of interventions shortly before patients die. Even the language used to describe EoL care is contested. Most individuals say they prefer to die peacefully at home, but many die in hospitals shortly after receiving treatments that have been described using value-laden terms, such as potentially avoidable,¹¹ burdensome,^12,13 inappropriate,^14–18 intensive,^19,20 unduly intensive,²¹ futile,^22–24 wasteful,²⁵ and aggressive.^26–31

In this paper, we use the neutral term discretionary to refer to interventions that typically confer benefits to patients but have limited or no evidence for their medical efficacy in particular clinical contexts. Offered at the discretion of clinicians, patients, and family members after weighing the benefits and burdens of their use, DIscretionary utilization At the end of Life (DIALs) can cause more harm than good if underused or overused.³²For example, intubation and mechanical ventilation are well-established medically beneficial interventions for respiratory failure in the context of severe pneumonia that allow clinicians time to treat and resolve the underlying pneumonia. However, those same interventions are rarely medically beneficial among patients with advanced cancer and comorbid pneumonia, even when they confer nonmedical (i.e., social, emotional) benefits that are important to some patients and family members. Thus, the same intervention can be either standard or discretionary depending on the clinical circumstances.

Whereas the debates about EoL care reflect remarkable sensitivity to clinical subtleties and language use, there has been surprisingly little attention to the nuanced measurement issues that influence the interpretation of DIALs data. Beginning with Earle’s seminal papers on DIAL measurement in cancer care,^33,34 hundreds of articles reporting data on DIALs have appeared, and efforts are underway to develop quality metrics.^35,36 Several impactful papers outlining methodological issues in EoL research have been published, but no paper has focused exclusively on DIALs measurement.^15,37–40

In an effort to improve the transparency of DIAL measurement and mitigate demographic disparities in EoL care, this scoping review will answer the following broad questions: how have DIALs been operationally defined in peer-reviewed research? Does research design influence how researchers measure DIALs, or whether they examine demographic correlates of DIALs? Is there other evidence of bias in the literature? Whereas meta-analyses and systematic reviews answer focused questions of a narrowly defined, mature literature,^41–46 scoping reviews aim to answer broad questions of an emerging literature that is dispersed across multiple disciplines and subspecialties. Unlike meta-analyses or systematic reviews, scoping reviews have a broad purview and can explore the scientific process itself, by, for example, critically analyzing the manner in which research questions have been generated, key variables have been operationalized, and knowledge has been created. Meta-analyses and systematic reviews can have immediate impact for practice but scoping reviews motivate and advance scholarship by identifying areas where concepts are under-developed and evidence is biased, absent or inadequate to inform practice.^47,48

This scoping review will provide a map to those interested in foraying into the poorly charted terrain of DIAL measurement. Its timeliness is underscored by the clinical and policy challenges posed by two demographic trends, the aging of the large baby-boom cohort^49,50 and the impending growth in populations worldwide of frail older patients with multiple comorbidities at risk for receiving DIALs.^51–53 DIALs have been shown to be associated with worse patient quality of life before death,⁵⁴ worse bereavement outcomes in personal caregivers,⁵⁵ elevated heathcare costs,^56,57 missed opportunities to divert precious resources elsewhere,²³ clinician burnout, and staff turnover.⁵⁸ Without identifying the conceptual and methodological traps that undermine reliable, valid, credible measurement of DIALs, it will be difficult to improve patients’ experiences at the EoL or validly evaluate systems of care designed to improve utilization and other important outcomes.⁵⁹ Likewise, it will be difficult to ascertain whether EoL care is improving over time or getting worse,^13,60,61 better in some regions than others.^35,36 or worse in some demographic groups than others.^62–66

II. Methods

A. Information Sources.

The literature search⁶⁷ was implemented primarily in PubMed as well as PsycINFO, Web of Science, and Google Scholar. After experimenting with combinations of keywords (aggressive, end-of-life, futile, health care utilization, health services, intensity, intensive care, older adult, quality, quality indicators, quality of care, terminal, treatment) we settled on the following: (end-of-life OR end of life) AND quality AND (hospice OR palliative OR invasive OR life-prolonging OR intensive OR intensity OR aggressive).

The reference sections of several review articles^41–46 were examined (so-called backward searches). Forward searches were conducted, the websites of governmental organizations (e.g., AHRQ, NCI, Medicare) were examined, and nontraditional search methods were used (e.g., Google recommended article function). No journals were hand-searched.

B. Inclusion Criteria

We included English language articles published in 2004 or later (until June 30, 2017) that reported quantitative data on EoL interventions, treatments, and procedures. The last search date was September 30, 2017. We chose to begin in 2004 because a seminal paper was published in 2003.³³ (That paper was itself motivated by a 2000 Institute of Medicine report.⁶⁸) We excluded review articles, commentaries, and qualitative or mixed-methods reports without quantitative data on specific DIALs. Economic analyses were included if data were provided on the costs of named components of EoL care (e.g., hospitalization). Three other eligibility criteria were required:

1. Reported on Two or More DIALs. Our starting assumption is that DIALs is a latent construct,^69,70 meaning that DIALs cannot be precisely measured in the same sense that one can measure physical attributes such as height, weight, or body temperature. Like other latent constructs, such as intelligence, depression, or consumer prices, DIALs can only be estimated or approximated using multiple (two or more) indicators that tend to be correlated.^31,71–75 Therefore, we excluded articles that reported on only one indicator of DIALs, such as chemotherapy,^76–78 ICU stays,⁷⁹ place of death,^55,80 or hospice.⁸¹

2. Reported Data on Adult Patients Presumed to Have Capacity to Make Medical Decisions. We excluded studies of children (< 18 years of age) and persons with dementia.⁸² Children rarely have legal capacity to make treatment decisions. Dementia is known to influence how EoL decisions are made. Patients typically take a less active role and receive less aggressive treatment than persons without dementia.⁸³ We considered confining the review to studies of participants who were cognitively intact, but few studies reported relevant data.

3. Reported Data on DIALs Within 100 Days of Death. Determining when the EoL begins -- either biologically (e.g., a “biomarker”) or temporally (e.g., days before death) – remains elusive. Given the unavailability of a reliable biomarker demarcating the EoL that is equally valid in persons with different diagnoses, we used a temporal marker. Recognizing that transitions from seriously ill to actively dying are rarely knowable,⁸⁴ and that any temporal cut-point is arbitrary, the review was confined to studies that reported data on the final 100 days. It is easier to predict death within 100 days than 6 months or longer.^85,86 The 100-day timeframe permitted us to examine DIALs prior to deaths that are more predictable.^87–89 Studies reporting data more than 100 days before death were included if they also reported data within the final 100 days.

C. Data Abstraction

The first author worked with a librarian and another author (M.C) to screen the search results (titles, abstracts) against the eligibility criteria. Potentially-relevant full-text articles were pre-screened. Discrepancies across screeners were resolved through discussion.

The data abstraction form (Appendix A) was iteratively modified beginning in July 2016, and finalized in April 2017. The first author trained all coders (undergraduate students, graduate students, research assistants) to use the form. Coders were allowed to independently abstract data after they demonstrated they could code with a high degree of accuracy (fewer than 5 errors). To minimize error, data were extracted by at least two coders, with small teams focusing on particular variables. The first author resolved all coding discrepancies in consultation with a dually-trained nephrologist/palliative care physician (F.S.) or a nurse with expertise in palliative care and intensive care (S.N). The first-author personally checked the accuracy of all coded variables on all forms.

1. DIALs (presence/timing). The abstraction form contained 13 pre-specified DIALs (closed-ended, dichotomous items) as well as open-ended items. Closed-ended items forced coders to determine whether articles reported data on the pre-specified DIALs. Largely reflecting Earle’s research in medical oncology^33,34,73 and Barnato’s studies of critical care,³¹ the pre-specified DIALS were: inpatient hospitalization, emergency department (ED) visits, admission to an intensive care unit (ICU), chemotherapy, cardiopulmonary resuscitation (CPR), mechanical ventilation, dialysis/hemodialysis, intubation, tracheostomy, artificial nutrition (e.g., tube feeding, parenteral nutrition), prescription medication (e.g., antibiotics, vasopressors), site of death, hospice and palliative care. As the terms palliative care and hospice do not have universally shared meanings, the words “palliative care” and “hospice” had to appear in the article to be coded. Palliative chemotherapy was coded as chemotherapy.

   The presence of open-ended items enabled coders to identify DIALs that were not pre-specified. Data on the number of days before death covered by the DIAL assessment (1,2,3,7,14, 30, 60, >60 days before death) were coded (e.g., inpatient hospitalization 7 days before death).

2. Sample Characteristics. We coded the number of decedents, diagnostic population (individuals with cancer, with another specific diagnosis [e.g. renal disease], or other), age group (≥65, unknown, other), gender composition (% men), and country of data collection.

3. Study Design. Latent constructs (e.g, DIALs, consumer prices) can only be estimated using multiple indicators, not measured with near-pinpoint precision as in the measurement of height. Consequently, this leaves room for unwarranted, systematic variation in estimates, reflecting the vagaries of scientific decision-making. Methods for deriving DIALs estimates are subject to a biases related to investigator attributes (motives, disciplinary norms, etc.), estimation methods (e.g., instrumentation), or study design. To examine the effects of study design on seemingly unwarranted variation, or bias, in DIALs assessments, studies were coded as prospective, retrospective, or terminal hospitalization. The latter focuses on a sample of hospitalized patients, all of whom died while hospitalized. Because these studies report data on a sample of decedents, they could be classified as retrospective, but they could also be classified as prospective since they use data collected prospectively and documented in a local medical record. Documentation could, in theory, affect EoL care, analogous to the Hawthorne effect.⁹⁰ We thus created a separate category to identify studies of terminal hospitalization.

4. Sociodemographic Correlates. We determined whether articles reported demographic (age, gender, race) or socioeconomic (annual household income, occupation, education) correlates of DIALs. Despite justifiable, ethical⁹¹ concerns about demographic disparities in EoL care in the United States,^64,65,92 and elsewhere,^93–96 this is the first attempt to explore the effect of study design on the examination of demographic differences in EoL care.

5. Publication Characteristics. We recorded journal names in an effort to gauge interest in DIALs outside of palliative care and oncology as well as to identify patterns of DIALs assessments as a function of medical specialty. Given the need for timely reporting of findings, we created a timeliness index by computing the temporal lag between the end of data collection (month/year) and date of initial (print or online) publication. To determine if authors justify their decisions to study particular DIALS (e.g., ED visits) over a particular timeframe (e.g., final 7 days of life), we coded whether they offered a citation to justify their methodology. Given our interest in identifying biasing influences on DIALs assessments, we attempted to reliably code author’s motives for conducting the research (e.g., demonstrate positive impact of palliative care, test hypotheses about EoL interventions, document harms of overtreatment).

D. Synthesis.

Synthesis occurred in three overlapping stages. The first stage occurred as data were being abstracted, coded, and entered. When coders were uncertain whether an article reported a particular DIAL, they were instructed to discuss the article with a supervisor (P.D. or M.H.) Final decisions were made by the first author in collaboration with a nurse and/or a physician.

In the second stage, data were statistically analyzed and then presented and discussed in a series of seven 60–90 minute meetings (6/8/17–8/31/17) attended by coders and a multidisciplinary team, representing a range of disciplines and specialties (palliative care, nursing, oncology, nephrology, internal medicine, family medicine, psychology). In these meetings, quantitative summary data were presented along with inferential statistics comparing prospective, retrospective and terminal hospitalization designs. Approaches to categorizing and tabulating the data were developed, emerging themes were identified, and preliminary drafts of this manuscript were reviewed. In the third stage, the manuscript was iteratively edited by all authors, and was critiqued by two palliative care scholars who were not involved in the project’s earlier stages, and did not co-author this paper. It was modified again after receiving input from five anonymous reviewers.

III. Results

A. General Overview

After screening more than 5,300 abstracts, the investigators read more than 400 articles, nearly half of which were excluded because the article reported data on only one intervention (e.g., chemotherapy use; see Methods: II.B.1); provided data on non-decedents and decedents, without distinguishing the two (because our focus is on decedents); provided no information about the timing of the assessment in relation to the date of death (because we focused on DIALs in the final 100 days of life); or solely reported economic (cost) analyses. This left us with 254 articles (Figure 1) over the course of the study period (1/1/04–6/30/17), with most (n=164, 64.6%) published since 2013 (Figure 2, see Appendix B for included articles). Six articles^{31,74,97–100} reported data on two types of cohorts (e.g., prospective, terminally hospitalized). The total number of unique cohorts examined was thus 260.

Figure 1. PRISMA Flow Diagram for Scoping Articles.

^aThis number and subsequent numbers are approximations, as traditional (backward search, forward search) and nontraditional (e.g. the Google “recommended articles” function) methods were used to identify additional records, and the nontraditional methods were not tallied.

^bSix articles reported data using more than one sample and research design; the total number of samples was 260. See text for details.

Figure 2. Growth of the Literature on DIALs, 2004–2016.

Note. There was a notable surge in interest in this topic in 2013, and was not confined to palliative care (PC) and oncology (Onc) journals. The review included articles published before July 1 2017; for consistency, 2017 data are not shown.

Studies have appeared in 79 different journals, ranging from general medicine (New England Journal of Medicine, Lancet), surgery (Journal of Surgical Research), and critical care (American Journal of Respiratory and Critical Care Medicine) journals to health disparities (Journal of Healthcare for the Poor and Underserved), health services (Medical Care), health policy (Health Affairs), and gerontology (Gerontologist) journals. Most articles appeared in cancer (96 articles) or palliative care (73 articles) journals, particularly the Journal of Palliative Medicine (27), Journal of Pain and Symptom Management (17), Cancer (17), Supportive Care in Cancer (17), and Journal of Clinical Oncology (16). Outside of oncology and palliative care, the most common outlets were the Journal of the American Geriatrics Society (8), JAMA Internal Medicine (8), and JAMA (7).

Studies have been conducted in 29 different countries, mainly in the United States (137 studies), Canada (26), the Netherlands (16), Italy (13) and Taiwan (13). With the exceptions of articles from Turkey, Lebanon, Brazil, and the People’s Republic of China, studies have originated from countries with advanced economies, as defined by the International Monetary Fund (www.imf.org).

Table 1 shows that most articles (180; 69.3%) focused on cancer; the remainder examined persons with another disease, nursing home residents, patients in primary care practices, or community samples. The median (IQR) number of deaths reported was 639.5 (265, 8148). Men constituted 55.1% (±19.0%) of the samples.

Table 1.

Descriptive Characteristics of Studies Included in the Review

Variable [median (IQR) or N (%)]	167 Retrospective studies	46 Prospective Studies*	47 TH studies	Total (k=260)
Number of deaths	2074 (319, 32800) a	321 (214, 387)	410 (117, 3065)	639.5 (265,8148)
Focused on cancer patients only	122 (73.1)	38 (82.6)	20 (42.6) b	180 (69.3)
Focused on older patients only	51 (30.5) c	5 (10.9)	5 (10.6)	61 (23.5)
Used administrative dataset**	104 (62.3) d	9 (19.6)	11 (23.4)	124 (47.7)

Note. TH=Terminal Hospitalization; IQR=Interquartile range. Bolded values in a column are significantly different from the values in the other two columns.

^az= −5.33; p<0.0001, non-parametric Wilcoxon ranksum,

^bPearson chi2=17.94, df=1, p<0.001

^cPearson chi2=12.92, df=1, p<0.001,

^dPearson chi2=39.80, df=1, p<0.001,

^*Most (38/46, 82.6%) articles in this category originated from the United States; eighteen articles used one dataset, Coping with Cancer (H.G. Prigerson, PI).

^**Medicare data were analyzed in 49 articles.

B. Comparing Prospective, Retrospective, and Terminal Hospitalization Designs

The literature is dominated by retrospective studies (Table 1). These studies had larger sample sizes, were more likely to focus on older adults, and more likely to use administrative datasets. Prospective articles reported more timely data than other articles. The [median (IQR)] number of months from last death observed to date of publication was 27.5 (17, 55) in prospective studies, compared to 45 (33,61) in retrospective studies and 37 (25, 58) in terminal hospitalization studies(z=3.04, p=0.0024, non-parametric Wilcoxon ranksum).

The most commonly studied variable was site of death, which was examined in 73% of articles that used a retrospective or prospective design. Table 2 shows that the most frequently examined DIALs were ICU stays, inpatient admissions, and chemotherapy.

Table 2.

Number (%) of Studies Reporting Data on 12 DIAL Indicators^*

Indicator	167 Retrospective studies	46 Prospective studies	47 TH studies	Total (k=260)
ICU stay	88 (52.7)	31 (67.4)	29 (61.7)	148 (56.9)
Inpatient admission	112 (67.1)	20 (43.5)	10 (21.3)	142 (54.6)
Chemotherapy	91 (54.5)	25 (54.3)	16 (34.0)	132 (50.7)
Hospice	83 (49.7)	32 (69.6)	7 (14.9) a	122 (46.9)
Palliative Care*	66 (39.5)	10 (21.7)	28 (59.6)	104 (40.0)
Emergency Department use	86 (51.5) b	11 (23.9)	5 (10.6)	102 (39.2)
Ventilation/Intubation	33 (19.8) c	27 (58.7)	25 (53.2)	85 (32.7)
Cardiopulmonary resuscitation	24 (14.4) d	24 (52.2)	24 (51.1)	72 (27.7)
Feeding Tube	23 (13.8)	14 (30.4)	20 (42.6)	57 (21.9)
Medications **	20 (12.0)	5 (10.9)	20 (42.6) e	45 (17.3)
Dialysis	15 (9.0)	5 (10.9)	17 (36.2) f	37 (14.2)
Tracheostomy	4 (2.4)	3 (6.5)	8 (17.0	15 (5.8)

Note. From top to bottom, indicators are listed from most frequently studied (ICU stay) to least frequently studied (tracheostomy). DIAL=discretionary utilization at the end of life; TH=terminal hospitalization; ICU=intensive care unit; ED=emergency department. Bolded values in a column are significantly different from the values in the other two columns;

^aPearson chi2=21.64, df=1, p<0.001,

^bPearson chi2=29.47, df=1, p<0.001,

^cPearson chi2=35.48, df=1, p<0.001,

^dPearson chi2=41.38, df=1, p<0.001,

^ePearson chi2=24.41, df=1, p<0.001,

^fPearson chi2=21.65, df=1, p<0.001.

^*We did not report data on site of death in the Table because, by definition, all decedents in TH studies died in hospital See text for data on site of death

^**Although palliative chemotherapy, palliative radiation, and palliative surgery are part of palliative care in a broad sense, they are also arguably invasive and thus were not considered palliative care for the purposes of this review. Our assumption is that the underuse of palliative interventions is inappropriate, but it is plausible that some palliative interventions are overused – just as hospice can be used inappropriately.

^***The most commonly studied medications were pain medications, antibiotics, vasopressors and psychotropics.

We examined the effects of study design on seemingly unwarranted variation, or bias in the ways DIALs have been operationally defined. Retrospective studies were more likely to report data on ED visits but less likely to report data on CPR and ventilation/intubation. Terminal hospitalization studies were more likely to report data on dialysis and medication use. Retrospective studies were more likely to report data on the final month of life, prospective studies were more likely to examine the final week, and terminal hospitalization studies were more likely to focus on the final day (Appendix C).

We also examined the effects of study design on seemingly unwarranted variation, or bias, in the examination of demographic correlates of DIALs. Table 3 shows that retrospective studies were more likely to report analyses of the effects of age, gender, and socioeconomic status. Prospective studies were more likely to report data on the effects of race/ethnicity.

Table 3.

Number (%) of Articles Reporting Demographic Correlates of DIALs

Examined effects of:	167 Retrospective studies	46 Prospective studies	47 TH studies	Total (k=260)
age	85 (50.9) a	12 (26.1)	19 (40.4)	116 (44.6)
gender	73 (43.7) b	11 (23.9)	13 (27.7)	97 (37.3)
race	40 (24.0)	18 (39.1) c	11 (23.4)	69 (26.5)
socioeconomic status	40 (24.0) d	7 (15.2)	1 (2.1)	48 (18.5)
≥1 demographic variable	96 (57.7)	19 (41.3)	26 (55.3)	141 (54.2)

Note. DIAL=discretionary utilization at the end of life; bolded values in a column are significantly different from the values in the other two columns.

^aPearson chi2=7.46, df=1, p=0.006,

^bPearson chi2=8.19, df=1, p=0.004,

^cPearson chi2=4.55, df=1, p=0.033 [11 of these articles used data from Prigerson’s Coping with Cancer study],

^dPearson chi2=9.35, df=1, p=0.002.

C. Emergent Themes: Conceptualizing and Measuring DIALs

As the research group discussed the findings in a series of meetings, and the manuscript was revised, five themes emerged.

1. Variables differ considerably in complexity, ranging from simple dichotomies with uncertain clinical relevance (e.g., received chemotherapy in the final 30 days) to more complex, clinically relevant variables (e.g., received goal-concordant care). As shown in Table 4, it is easier and cheaper for researchers to acquire simple variables, but their simplicity compromises clinical relevance. In the simple approach, what counts, and is counted is whether patients were hospitalized, on chemotherapy, visited an ED, died in the hospital, etc. Contextual nuance is rarely considered. Simple variables examine patient exposure to treatments, procedures, or interventions, not clinical appropriateness (or avoidability) for a particular patient at a particular moment.

   Reflecting dissatisfaction with simple variables, researchers have created complex variables (Table 5) in retrospective, prospective, and terminal hospitalization studies. ^101–106 These variables have been created using interview data, medical records, and administrative data to capture the avoidability of hospitalizations;¹¹ ED visits for chemotherapy toxicity;⁷⁴ the number of transfers and transitions across care settings;^{12,13,100,107–110} hospital deaths of nursing home residents within three¹⁰¹ or eight¹¹¹ days of hospital transfer; the deaths of terminally ill patients who had been transferred to the ICU and offered hemodialysis, vasopressors, or intubation;²⁴ ICU deaths of patients with cancer following resuscitation and/or mechanical ventilation; and receipt of goal-concordant care.^102,112

2. Motives for reporting data are not transparent, raising questions about bias. Our attempt to discern researchers’ motives for reporting data on particular DIALs (e.g., demonstrate positive impact of palliative care, document harms of overtreatment) was unsuccessful, Researchers’ motives were rarely transparent. In 60 articles, the authors provide no citation to justify the decision to study a particular variable. In some cases, researchers merely reported descriptive data on EoL utilization without explaining why they reported the data.

3. Some research appears to have been motivated by the need of specialists to identify specialty-specific quality indicators,²⁷ raising questions about bias. Many variables appear to have been chosen on the basis of data availability and input from medical specialists who have a shared understanding of that specialty’s norms. Professional societies, through their publications and committees, reinforce this shared understanding. Studies conducted by medical oncologists that appear in oncology journals typically report data on chemotherapy use prior to death;²⁷ studies conducted by nephrologists that are published in nephrology journals report on dialysis initiation;¹¹³studies conducted by surgeons that appear in surgery journals report data on surgical procedures;¹¹⁴ studies conducted by intensivists that appear in critical care journals report data on ICU length of stay.¹¹⁵ This publication pattern is understandable, given the varied clinical interests of specialists and subspecialists and the need to identify quality indicators that reflect care processes over which specialists can exert control.²⁷ Unfortunately, the focus of different constituencies (e.g., the membership of specialty professional organizations, payers, individual patients) is rarely well-aligned. As shown in Table 4, some constituencies favor simple variables, and others favor complex variables. This misalignment represents a methodological challenge that could lead to metrics with uncertain relevance to the evaluation of overall quality care and the patient experience.

   In an effort to illustrate a potential misalignment and critically analyze the manner in which key variables have been operationalized in the DIALs literature, we reviewed the logic underlying Earle et al.’s decision to omit ED visits from their metric. We chose this metric because of its perceived importance. It was endorsed by the National Quality Forum, Center for Medicaid and Medicare Services, and the Agency for Health Research and Quality.

   Earle et al.³³ originally included ED visits in their metric. Five years later, Earle et al.²⁷ abandoned that indicator, and others, arguing that the omitted indicators are “strongly influenced by comorbidity and, therefore, appear less useful as measures of aggressive cancer care” (pg. 3862). Even though risk for ED use would seem to be relevant to oncologists’ decision-making, and statistical adjustments for comorbidity are routinely performed, the authors reasoned that the newer metric would provide a more “methodologically sound” (pg. 3860) assessment of the work of medical oncologists. Earle et al. were appropriately circumspect about their methodology. They sought to identify variables reflective of an oncologist’s job performance, as viewed through a particular evaluative frame. They solicited patient and caregiver input for their 2003 paper, recognized that administrative data provides little information about treatment intent, and could under-estimate treatment aggressiveness.⁷³ They identified variables that were unavailable in administrative data (advance directives) but could eventually be included in metrics when available.³³

   Stakeholders were involved in developing the measure described in the 2003 paper,³³ but we uncovered no evidence that caregivers or patients were consulted prior to the decision to drop ED visits. Had Earle et al. had adopted a broader view of the oncologist’s role or solicited patient and caregiver input when contemplating their revised metric, they may have retained ED use. That said, the decision to remove ED visits from the metric simplified a methodology that could be used to evaluate oncologists, given a narrower view of their role.As we discuss later, this raises difficult questions about the nature of quality metrics and the ethical obligations of those charged with developing them, A key question for further research is how researchers’ biases have shaped the provision of EoL care.

4. Beyond the 13 pre-specified DIALs, researchers have studied more than 90 other activities. We classified these variables into nine categories (Appendix D). The most common were other disease treatments (e.g., radiotherapy), invasive diagnostics (e.g., colonoscopy) and general support/monitoring (e.g., blood draw).

   Three qualifiers provide a more refined appreciation of Appendix D. First, at this early stage of methodological development, we erred on the side of including variables that are not definitively discretionary. Many symptom-focused activities are presumed to be vital, not discretionary, for patients who are dying. For instance, the activities listed under “Fluid Removal” include common procedures which confer benefit and arguably little harm in patients with very short prognoses. There is little comparative effectiveness research on many of the identified activities, however, and some treatments (e.g., antibiotics) confer risks to patients who had not been directly exposed to the medication.¹¹⁶ The same may be true of some of the seemingly benign activities listed in Appendix D. Second, some interventions are offered both to relieve symptoms as well as to demonstrate caring, compassion, and tenderness, or other “off label” reasons. This scoping review uncovered no systematic research on off-label reasons for prescribing discretionary interventions. Several articles alluded to this phenomenon, however. One research group summed up this dilemma in a discussion of tube feeding dying nursing home residents: “Although tube feeding can be construed as more aggressive than careful handfeeding, “it may also be perceived as an indicator of caring and responsiveness toward their residents…. a (demonstration of) caring and not ‘abandoning’ residents.”⁸¹ (pp. 825).

5. Operational definitions of DIALs have changed over time. Even though our review covered a relatively brief timeframe, time trends in the assessment of DIALs were evident. In 2004, Earle et al.³⁴. noted that “palliative care measures of admission to hospice or length of hospice stay were not included in the composite measure because they do not themselves indicate aggressive treatment” (pg. 316). However, Earle et al. soon began reporting data on hospice utilization,^73,74,117 and others followed suit.^118–122 Other trends in the operationalization of DIALs were observed, perhaps reflecting changes in the perceived importance of certain outcome measures, developments in health care delivery, and the increased interest in DIALs in medical specialties and academic disciplines beyond palliative care and medical oncology. Some of these trends are shown in Figure 3.

Table 4.

Distinguishing Simple and Complex Variables

Parameter	Simple Variablesa	Complex Variablesb
Exemplar	Chemotherapy ICU stay Site of death ED visit	Burdensome transitions Receipt of preference-concordant care Potentially avoidable admissions
Easily Available	More	Less
Clinically Relevant	Less	More
Clinical Trial Outcome	Used extensively	Rarely used
Main constituencies	Professional societies, health systems, governments	Patients, caregivers, patient advocacy groups, clinicians

Note. Variables differ considerably in complexity, ranging from simple dichotomies with uncertain clinical relevance (e.g., received chemotherapy) to more complex, clinically relevant variables (e.g., received goal-concordant care). Simple and complex variables also differ in availability and the extent to which they have served as outcomes in clinical trials. Some constituencies have prioritized simple variables; others are more invested in complex variables.

^aSee Table 2 for a list of other simple variables

^bSee Table 5 for a list of complex variables

Table 5.

Articles Reporting Data on Complex Variables, 2004–2017^a

1st Author, Year, Design	Country	Variable	Data Source	Population	Methodology
Hui, 2009,17 Retrospective	Canada	Appropriateness of interventions	Medical records	Cancer inpatients	Researchers evaluated the appropriateness of blood work (e.g., electrolytes, liver function tests), microbiology, imaging, hormonal therapy, radiation therapy, enrollment in Phase I trial, chemoradiation, and surgery
Muni, 2011,103 Terminal hospitalization	United States	Dying in the setting of full support	Medical records	ICU deaths in 15 hospitals	Not withdrawing life-sustaining treatments
Wright, 2014,105 Retrospective	United States	Health care transitions	Claims	Individuals ≥66 years old diagnosed with ovarian cancer	Changes over the final 3 and 30 days of life in the institutional health care provider identification number based on Medicare billing data and hospice files
Hockley, 2010; Finucane, 2013,101,106 Prospective	United Kingdom	Inappropriate hospital days/death	Medical records	Nursing home residents	Researchers reviewed medical records to determine if death of frail, “deteriorating” nursing home residentsoccurred within 3 days of hospitalization for pneumonia or dehydration
Loggers, 2009;2013, and Wright, 2010,19 20,157 Prospective	United States	Intensive or aggressive EoL care	Medical records, caregiver report	Cancer patients enrolled in cohort study	EoL care was considered intensive or aggressive if patients died in an ICU after receiving resuscitation and/or mechanical ventilation in the last week of life
Moscovici da Cruz, 2015,24 Terminal hospitalization	Brazil	Medical futility	Medical records	Cancer patients admitted to the ICU	Researchers reviewed medical charts to determine if terminally ill patients had been transferred to the ICU and started on hemodialysis, vasopressors, or mechanical ventilation
Daly, 2016,11 Terminal hospitalization	United States	Potentially avoidable admissions	Medical records	Cancer patients admitted to the ICU	Using a standardized assessment tool, a multidisciplinary panel of 10 physicians reviewed the death note as well as provider and procedure notes and laboratory, imaging, pathology, and chemotherapy records. Records were reviewed beginning 3 months prior to the terminal hospitalization until death
Gozalo, 2011,12 Retrospective	United States	Potentially Burdensome transitions	Claims	Nursing home residents	Any transfer in the last 3 days of life, a lack of continuity of nursing home facilities before and after a hospitalization in the last 90 days of life, > 2 hospitalizations for any reason or >1 hospitalization for pneumonia, urinary tract infection, dehydration, or sepsisb in the final 90 days
Teno, 2013,13 Retrospective	United States	Potentially Burdensome transitions	Claims	Fee-for-service Medicare beneficiaries	Changes over the final 3 days of life in institutional health care provider identification number based on the Medicare billing data
Biola, 2010,112 Retrospective	United States	Receipt of preference-concordant care	Caregiver report, nursing staff report	Nursing home residents	Hospitalization or receipt of antibiotics, CPR, or tube feeding against the patient’s wishes (per proxy report)
Mack, 2010,102 Prospective	United States	Receipt of preference-concordant care	Medical records, caregiver report	Cancer patients enrolled in cohort study	Receipt of life extending care or symptom-directed care at the EoL that was inconsistent with preferences assessed 125 days (median) before death.
Pasman, 2013,104 Retrospective	Netherlands	Receipt of preference-concordant care	Caregiver report	older adults & those with advanced directive	Using a mortality follow-back design, researchers asked caregivers about the deceased person’s treatment preferences and whether those preferences were honored
TemkinGreener, 2013,111 Retrospective	United States	Site of death following transfer	Claims	Nursing home residents	Hospital deaths within 8 days of nursing home transfer
Setoguchi, 2008,74 Retrospective	United States	Toxicity-related ED visit or hospital admission	Claims	Cancer patients	Chemotherapy toxicity was coded as present if the diagnosis at the time of the ED visit or hospitalization included one or more of the following: fever or infection; neutropenia or thrombocytopenia; dehydration or electrolyte disorders; nausea, vomiting, or diarrhea; anemia; constitutional symptoms; deep venous thrombosis or pulmonary embolus; malnutritionc
Abarshi, 2010,108 Retrospective	Netherlands	Trajectories across care settings	Physician Report	Expected, nonsudden deaths	Researchers asked general practitioners to identify the settings where patients had received care and the duration in each. Settings were classified as homes, care homes, nursing homes, palliative care units, and hospitals
Gielen,2010,107 Retrospective	Belgium	Transitions across care settings	Claims	Persons > 40 years of age	Researchers classified care settings as hospital, inpatient palliative care unit, care home, home with home care, and home without home care

Note. CPR=cardiopulmonary resuscitation; ED=emergency department; GP=general practitioner; ICU=intensive care unit.

^aThis review covers articles published from 1/1/04 to 6/30/17. We are aware of at least one recent article that used a complex variable as an endpoint in a clinical trial. ¹⁵¹

^bThe authors¹² reasoned that more than one hospitalization for these conditions is equivalent to more than two hospitalizations for other conditions because these conditions are potentially manageable with “appropriate advance care planning, without the need for hospitalization” (pg. 1213)

^cIn 2004, Earle et al.³⁴ conducted secondary analyses using a similar approach, but they did not report the findings. They counted inpatient admissions or ED visits as measures of aggressiveness if the “diagnosis-related group or first-position International Classification of Diseases (9th revision) code description mentioned the patient’s cancer or its treatment” (pg. 316).

Figure 3. Longitudinal variation in the percentage of publications reporting each DIAL.

Note. Publications were ordered chronologically, and the z-test for proportions was used to compare the frequency with which each DIAL was reported in the early studies (n=84) versus the most recent studies (n=91). Relative to the early studies, reporting on inpatient admissions (47.6% vs. 65.9%, P=.016), palliative care (29.8% vs. 48.4%, P=.015), and radiotherapy (4.8% vs. 18.7%, P=.005) increased in recent studies, whereas reporting on ICU stays (66.7% vs. 47.3%, P=.008) and ventilation/intubation decreased (40.5% vs. 16.5%, P<.001)

IV. Discussion

This review documents sufficient interest in the measurement of DIALs to warrant the investment of further time, effort, and resources to refine this area of inquiry and develop more robust (reliable, valid, credible) EoL metrics. Research on DIALs has been conducted in more than 25 countries and published in outlets cutting across medical specialties (e.g., cardiology, critical care, dermatology nephrology, oncology, palliative care, surgery) and disciplines beyond clinical medicine (epidemiology, health services research, health policy). We found considerable variation in the assessment of DIALs as a function of study design. This variation is seemingly unwarranted, and illustrates the role of norms and disciplinary habit on the assessment of important outcomes. We also found that retrospective studies were more likely to report effects of patient age, gender, and socioeconomic status on DIALs. This is likely due the larger samples in retrospective studies. Prospective studies were more likely to examine race differences in DIALs, perhaps because data on race is more likely to be missing from clinical or administrative datasets.¹²³ Based on the data synthesized here, we offer five prioritized recommendations to improve research guiding the development of EoL metrics (e.g., DIALs) that could be used by the public (including patients and caregivers), clinicians, researchers, hospitals, health systems, payers, governments, and nongovernmental organizations.

A. The Future of DIALs Assessments

1. Motivate Public Involvement in the Development and Ongoing Revision of EoL Quality Metrics.

No fewer than 100 DIALs have been examined over the final 100 days of life. Many appear to have been chosen on the basis of input from medical specialists who have a shared understanding of that specialty’s norms and an interest in self-policing by developing quality standards. Patients,family members/caregivers, and other stakeholders have rarely been consulted, though there are exceptions.^31,33,124

Developing, refining, and revisingusable EoL care metrics for clinical trials, scorecards, and benchmarking requires the ongoing involvement of motivated citizens who understand the relevance of such metrics to their lives, local communities, and, some would argue, public health.¹²⁵ We are not advocating for a zero-tolerance approach to the involvement of experts and medical societies in metric development, but we are calling for a more inclusive, democratic approach that is publicly accountable. By ensuring that specialists do not exert disproportionate influence in the development of quality indicators,^125,126 the public interest can be prioritized. Parties with self-interest in a simple endpoint (e.g., chemotherapy use over the past 30 days) risk ignoring or discounting outcomes of greater relevance to other parties. This can lead to ethically difficult decisions.¹²⁷

Forging sustained^128,129 partnerships¹³⁰ with citizen-stakeholders, and encouraging public deliberation about care of the dying and EoL metrics^{125,126,131,132} should improve metric robustness. If motivating public involvement in discussions about health care and policy is difficult, motivating public involvement in discussions about EoL metrics and discretionary interventions might seem impossible. Nonetheless, there have been promising developments and some cautionary tales. The success of the Conversation Project illustrates public interest in improving EoL care,¹³³ but research on public attitudes toward low value treatments¹³⁴ shows that about one-third of the public cannot readily envision deriving any benefit from avoiding discretionary treatments.

To motivate productive public discussions, researchers and policymakers might consider deliberative polling.^135,136 This technique combines traditional public opinion polling with expert-led small group public deliberations. For example, small groups of citizens (including patients and family members) could be engaged in deliberative discussions to consider the items listed in Table 2, Table 5, and Appendix D. How would they rank-order (or prioritize) these indicators? Do they think health care systems or clinicians should be held accountable for failing to meet certain standards? How often do they think metrics should be revised? Why might there be a need for revision? What outcome variables should be used in clinical trials and tracked by governments, patient advocacy groups, and health care systems?

No single, definitive approach to EoL metric development (or EoL care) will be universally endorsed by all constituents. Without public input, experts and specialists with vested interests in the use of particular metrics, including corporations,¹³⁷ will continue to exert disproportionate influence on matters of public policy and resource allocation. Of course, public involvement is not a panacea, and conflicts between an ethic that prioritizes individual life-extension and an ethic of population well-being have historically been irreconcilable.¹²⁶ Nonetheless, by engaging the public in discussions of DIALs, the effects of special interests could be minimized. Metrics proposed to guide policy and clinical care will have more public credibility.

2. Develop Quality Metrics that Reflect the Complexity of How Clinicians Conceive of EoL Interventions.

Research has not properly reflected the realities of clinical practice, where interventions are offered for multiple reasons beyond disease treatment or symptom management^138,139 including, unfortunately, gaming the system.^140,141 We are aware of no efforts to accommodate these realities in metric development.

To what extent do clinicians think they are offering interventions for curative reasons, palliative reasons, psychological reasons, simply because technologically-intensive treatment is the norm in their institution,^3,97,142 or some combination thereof?^138,139,143 A cross-national survey of this topic could be illuminating, as would qualitative research. In addition, it will be important to understand the presence of variability in expert raters’ judgments of the appropriateness of healthcare interventions.¹⁴⁴ For example, are intensivists more likely to view a hospitalization as potentially avoidable than oncologists or hospitalists¹¹ because they are more likely to be exposed to medical futility and thus more likely to experience moral distress? By reflecting the multi-determined nature of EoL interventions, future quality metrics will provide a more genuine depiction of the care provided in clinical settings.

3. Develop Patient-Centered DIALs Assessments For Use in Clinical Trials

Beginning with SUPPORT,¹⁴⁵ many interventions studied in clinical trials have attempted to enhance the quality of EoL care. Nonetheless, utilization outcomes have been examined using simple variables,^145–149 with debatable relevance to clinical care. A dying young woman might wish to be transferred to the ICU and placed on a ventilator to have an opportunity to hold her son on his first birthday. A dying middle-aged man might opt for another round of chemotherapy and to begin dialysis, just to be able to attend his daughter’s wedding. Viewed from one decision frame, a clinician who starts an invasive treatment while patients are dying might be seen as acting unethically. Viewed differently, the clinician’s behavior might be construed as empathic and moral. Without data on patient goals or the perspectives of family members or caregivers, the clinician’s actions are uninterpretable.

Only recently have researchers begun to attempt to assess the receipt of goal-concordant care, and many challenges remain.⁴¹ The first clinical trial to report data on the concordance of patient-preference¹⁵⁰ and actual care received at the EoL was published in 2019, and was thus excluded from this review.¹⁵¹ A recently published observational study was similarly omitted.¹⁵² These early studies point to the need for further methodological and conceptual development. In particular, there is a need for mixed method research that links quantitative utilization data with data that provides contextual nuance about patients’ and family members’ shifting goals and values.

4. Continue Developing “Basket of Goods” Approaches to DIALs Assessments

Beyond evaluating the quality of care provided to individual patients enrolled in trials or seen in particular care settings, there is a need to develop indices that could be used to evaluate structural problems across entire health systems or in larger units of analysis (e.g., hospital referral regions; cities, counties, etc.).This is best accomplished using a “basket-of-goods” approach, analogous to those used to evaluate consumer prices.¹⁴⁰ Even though more than 100 variables have been used to assess DIALs, most articles report data on just two or three, such as site of death or chemotherapy. This is like estimating consumer prices or inflation without including housing, transportation, or medical costs. A more diverse basket-of-goods index approach to DIAL measurement will provide a truer reflection of EoL care while also making it more difficult for payees to manipulate metrics to generate revenue.^140,141 Efforts to create a basket-of-goods DIAL index are currently underway. Building on research conducted for the Dartmouth Atlas, the Coalition to Transform Advanced Care recently unveiled the Advanced Care Transformation Index^SM (ACT Index^SM)), a 25-item index developed with some public input.^35,36

5. Enhance the Manuscript Peer-Review Process to Improve the Timeliness and Transparency of Data

The following recommendations pertain to changes in the peer review process that are aimed to improve the credibility and utility of DIALs data. Surprisingly few articles provided a citation to justify the examination of specific DIALs. We recommend that authors provide credible scientific or clinical arguments and supporting citations to justify the ethically significant decision to report data on some indicators of EoL care while omitting other indicators. We also recommend that journal editors consider the age of the data when making editorial decisions. The median timeliness index was about 3.5 years, and some studies reported data that were more than a decade old. Finally, we tried to identify researchers’ motives (e.g., show that palliative care is beneficial) but found that it was not possible to do so reliably. Future reports in this area would be more actionable if authors’ motives for conducting research were transparent.

B. Recommendations for Improving Research Implementation

In addition to offering recommendations for improving DIALs assessments, we offer some suggestions related to study design. Specifically, we call for more geographically dispersed prospective/TH research, more rigorous retrospective research, and greater use of blended designs.

1. Prospective and Terminal Hospitalization Designs

Prospective and terminal hospitalization designs are flexible, permitting more timely reporting of findings. The median timeliness index was greater than 3 years in retrospective studies (45 months) and below 3 years in prospective studies (27 months). The interval between the last death observed in a cohort and the date of publication was shorter in prospective studies. This is important because changes in the quality of EoL care have been detected over intervals as brief as 3 years.^153,154 Prospective and TH designs also permit the acquisition of data on variables that are rarely available in electronic health records or administrative datasets. Little is known about disparities in DIALs as a function of LGBTQ status, immigrant status, occupation, or language status. Data on these variables are rarely available in administrative dataset or medical records, but could be collected prospectively.

Prospective studies have typically been mounted in well-resourced settings by researchers affiliated with academic health centers (AHC) who study patients receiving treatment in AHCs. These patients differ from those who do not seek treatment and those treated outside AHCs.¹⁵⁵

Going forward, prospective studies, including clinical trials, should be mounted in geographically dispersed regions, and they need to be much larger than they have been historically. Many of the apparently “null” effects in clinical trials are uninterpretable, as these studies have lacked power to detect effects on utilization outcomes.¹⁵⁶ Prospective and terminal hospitalization studies that cover patient and clinician populations that are large, geographically-dispersed, and diverse will enable researchers to answer pressing questions that cannot be answered retrospectively. For example, use of those designs will enable scholars to discern whether differences in treatments within Dartmouth Atlas hospital referral regions⁹² are due to patient preferences¹⁵⁷. local norms of care,^142,158 provider attitudes^159–161 or availability of resources (e.g., hospital beds).¹⁶² Given changes over time in demographic composition, and changes in societal, social, and family norms as well as norms of care, multilevel observational studies could be mounted at regular intervals by governments to eliminate disparities and improve EoL care.

Unlike Bach et al.,⁴⁰ we do not believe retrospective studies should be abandoned; rather, prospective and terminal hospitalization designs have methodological advantages that could inform the development of multilevel interventions.^163,164 However, these designs require considerable resources and foresight, neither of which is essential to the conduct of credible research on DIALs. Retrospective studies are suitable alternatives when prospective research is not feasible economically or politically (e.g., low-resource settings, low- or middle-income countries).

2. Retrospective Designs

Beyond feasibility, retrospective studies have another considerable advantage. They were more likely to use government--sponsored administrative datasets that capture service use by people seen across disparate care settings, including community hospitals. Just as retrospective reconstructions of airplane crashes and other rare adverse events (e.g., suicide) have led to improvements in safety,^165,166 retrospective research on DIALs can lead to improvements in EoL care.

If sampling is the main problem with the prospective design, a key problem with the retrospective design (e.g., using administrative data or medical charts) is measurement bias, especially:1) hindsight bias– the tendency to see something after-the-fact that is invisible prospectively; 2) rater bias –when one rater sees something that is invisible to other raters, due to their personal or professional history, perceptual acuity, or other characteristics,¹⁶⁷ 3) information bias –when information that is available retrospectively or to one rater is unavailable prospectively or to another rater, and 4) artifacts attributable to differential item functioning – this can arise when independent raters use response categories in predictably different ways.^168,169 For example, on a 5-point scale assessing the avoidability of EoL treatments, ranging from “highly likely” to “highly unlikely” with “somewhat likely,” “50/50” and “somewhat unlikely” in the middle, one raters’ “somewhat likely” will be mathematically equivalent to another rater’s “50/50” because they interpret the response categories differently.¹⁶⁹

In one study, researchers minimized biases by having multiple raters from different medical specialties review medical records and answer open- and closed-ended questions¹⁷⁰ about the avoidability of hospitalization. Critical care physicians were more likely than oncologists or hospitalists to view hospitalizations as avoidable, perhaps due to their work experiences. Alternatively, the apparent difference between specialists might be due to rater bias, information bias or differential use of response scales. Biases can be accounted for statistically, by adjusting for rater characteristics.¹¹ Artifacts can be minimized by design, using structured abstraction forms with evocative items or response options (e.g., anchoring vignettes).¹⁶⁹

3. Blend the Designs.

The advantages and disadvantages of prospective and retrospective designs in EoL research are well-known.^15,38–40 To increase the societal value of research,¹⁷ both designs ought to be evaluated by funders on how well they can attain multiple, often conflicting, deliverables. A major tradeoff is the prospective design’s potential to gain insight into the subjective experience of patients and family members who are at the EoL³⁹ (often at great expense) vs. the retrospective design’s capacity to yield larger, representative samples. Each design can include elements of the other, however. Prospective studies can include mortality follow-back components.¹⁴⁹ Retrospective studies using electronic health records or administrative datasets can be linked to datasets collected prospectively using common data models such as PCORnet and/or assessments such as SEER-CAHPS,¹⁷¹ the Minimum Data Set⁷⁵, or PROMIS.¹⁷²

C. Qualifiers

We note several qualifiers. First, the review covered previously unexplored terrain^{37,38,41–46} but was confined to English language articles, the unpublished, gray literature ¹⁷³(e.g., technical reports, conference proceedings, dissertations) was not systematically reviewed, the protocol for this review was not published, and we have no quantitative data on relatively new treatments (e.g., targeted therapies) Future reviews can overcome all of these problems. We make no claims about the review’s exhaustiveness; only studies reporting quantitative data on DIALs in the final 100 days of life were included, the exclusion of studies on children and persons with dementia is sub-optimal, and we became aware of several eligible papers just as this manuscript was being completed. Nonetheless, we believe that the inclusion of 254 articles covering research conducted in 29 countries and published in 79 journals over nearly 14 years is sufficiently comprehensive to justify our conclusions. Second, the statistical analyses did not account for the fact that some datasets, such as Medicare, yielded multiple publications. Third, we considered conducting a narrower review focused on one population (individuals with cancer) and a single methodology (analysis of administrative datasets). We opted for an expansive scoping review because there is a need for a less siloed discussion of methodological issues that cuts across multiple clinical specialties and research areas. Many dying patients suffer from multimorbidity, and EoL care needs to be improved across diagnostic groups at the population level, not just for patients with a particular disease.

Fourth, the task of compiling a list of potential indicators of DIALs is complicated by the fact that lists tend to ignore clinical context. For example, antibiotics may be discouraged in some dying patients, but are also used for urinary tract infections and symptom management. Venting gastrostomies may be used for disease treatment or palliative purposes.¹⁷⁴ In identifying potential DIALs in Appendix D, we opted for a more inclusive approach, one that is commonly used in early stages of improving the measurement of latent constructs. Fifth, metrics that captures quality at the population level may not capture quality of care delivery for a particular patient at a specific timepoint. That tension should not prevent the development of population-based metrics, but it does underscore the limitations of the clinical utility (and credibility) of metrics. Sixth, it is likely that cognitively impaired patients and those who lacked decisional capacity at some point in their treatment were included in eligible studies. Finally, this review excluded patient-reported and caregiver-reported judgments of the quality of EoL care. Although these subjective judgments are important,¹⁷⁵ we focused on widely-available, universally applicable utilization outcomes because societies are accountable to all its citizens, including those who cannot speak for themselves and those who die without surviving caregivers or relatives. ¹⁷⁶

C. Conclusion

This scoping review identified more than 100 variables that have been studied over the final 100 days of life, prioritized important research areas, and analyzed critical conceptual and methodological issues. Operational definitions of DIALs will continue to evolve with changes in available treatments, health care delivery, and health policy. Our findings underscore the needs to engage the public and reflect clinical realities when measuring DIALs for clinical, administrative, health policy, or research purposes.

APPENDIX A. CODING FORM

First Author’s Last Name: ___________________

PMID #: __ __ __ __ __ __ __ __

Coder’s initials: __ __

Date Completed: __ __/__ __ (mm/yy)

	Enter the code in this column
Var 0 Study Number	__ __ __	Whole number
Var 1 Journal Name If other, write out:	__	See List of Journal Names
Var 2 Professional Psych Society	__	1=YES, 2=NO
Var 3 # authors	__ __	Whole number
Var 4 Psychologist Author	__	1=YES, 2=NO
Var 5 Citation	__	1=cited Earle; 2=cited someone else; 3=no citation
Var 6 Date of Last Death	__ __/__ __	MM/YY
Var 7 Date of Publication	__ __/__ __	MM/YY
Var 8 Number of deaths	__ __ __ __ __ __ __	Whole number
Var 9 Older adults only	__	1=YES, 2=NO
Var 10 Percent men (deaths only)	__ __ __	Whole number (round)
Var 11 Diagnosis	__	1 = Cancer Only, 2 = Another Specific Diagnosis (e.g., heart disease), 3 = Other
Var 12 Country	Separate Coding Sheet
Var 13 State
Var 14 Province	__ __	See List of Provinces
Var 15 Design	__	1 = Retrospective 2 = Prospective Cohort 3 =Intervention (RCT, quasi-experimental, etc)
Var 16 Administrative Dataset	__	1=YES, 2=NO
Var 17 Medicare (US only)	__	1=YES, 2=NO, 9=Not Applicable (outside USA)
Var 18 Site of death	__	1=YES, 2=NO
Var 19 Inpatient admission	__	1=YES, 2=NO
Var20 ED visits	__	1=YES, 2=NO
Var 21 ICU stay	__	1=YES, 2=NO
Var22 Chemotherapy	__	1=YES, 2=NO
Var 23 CPR/resuscitation	__	1=YES, 2=NO
Var 24 Mechanical ventilation	__	1=YES, 2=NO
Var 25 Dialysis/hemodialysis	__	1=YES, 2=NO
Var 26 Intubation	__	1=YES, 2=NO
Var 27 Tracheotomy	__	1=YES, 2=NO
Var 28 Feeding tube	__	1=YES, 2=NO
Var 29 Hospice	__	1=YES, 2=NO
Var 30 Palliative care	__	1=YES, 2=NO
Var 31 Medication underuse/overuse List medications studied (not cancer chemo):	__	1=YES, 2=NO
Var 32 Other -- unknown	Describe:
Var 33 Other comfort care –includes, but not limited to, care received from nurses, social workers, counselors, psychiatrists, psychologists and clergy. If yes, describe:	___	1=YES, 2=NO
Var 34 Other invasive/life prolonging measures If yes, describe	__	1=YES, 2=NO
Var 35 Chemotherapy ≤1month prior to death	__	1=YES, 2=NO
Var 36 Chemotherapy ≤ 14 days prior to death	__	1=YES, 2=NO
Var 37 Number of days before death is ≤1	__	1=YES, 2=NO
Var 38 2 days before death	__	1=YES, 2=NO
Var 39 3 days before death	__	1=YES, 2=NO
Var 40 7 days before death	__	1=YES, 2=NO
Var 41 2 weeks before death	__	1=YES, 2=NO
Var 42 1 month (30–31days) before death	__	1=YES, 2=NO
Var 43 2 months before death	__	1=YES, 2=NO
Var 44 More than 2 months before death	__	1=YES, 2=NO
Var 45 Terminal hospitalization	__	1=YES, 2=NO
Var 46 and Var 47	BLANK
Var 48 Effects of psychological variable Describe:	__	1=YES, 2=NO
Var49 Effects of sex	__	1=YES, 2=NO
Var 50 Effects of age	__	1=YES, 2=NO
Var 51 Effects of socioeconomic status	__	1=YES, 2=NO
Var 52 Effects of race/ethnicity	__	1=YES, 2=NO
Var 53 Effects of religion/spirituality	__	1=YES, 2=NO
Var 54 Effects of medical comorbidity	__	1=YES, 2=NO
Var 55 Composite score	__	1 = each indicator analyzed separately, 2= created a composite or factor score 3=combination of 1 and 2.

Notes.

Countries

a) Australia	1=yes, 2=no
b) Belgium	1=yes, 2=no
c) Canada	1=yes, 2=no
d) China	1=yes, 2=no
e) Denmark	1=yes, 2=no
f) England/UK	1=yes, 2=no
g) Germany	1=yes, 2=no
h) India	1=yes, 2=no
i) Israel	1=yes, 2=no
j) Italy	1=yes, 2=no
k) Japan	1=yes, 2=no
l) Korea	1=yes, 2=no
m) Netherlands	1=yes, 2=no
n) New Zealand	1=yes, 2=no
o) Norway	1=yes, 2=no
p) Portugal	1=yes, 2=no
q) Singapore	1=yes, 2=no
r) Spain	1=yes, 2=no
s) Sweden	1=yes, 2=no
t) Taiwan	1=yes, 2=no
u) United States	1=yes, 2=no
v) Other	1=yes, 2=no

Journals

a. BMJ/British Medical Journal

b. Cancer

c. Journal of the American Geriatrics Society

d. JAMA/Journal of the American Medical Association

e. JAMA Internal Medicine/Archives of Internal Medicine

f. Journal of Clinical Oncology

g. Journal of General Internal Medicine

h. Journal of Pain and Symptom Management

i. Journal of Palliative Care

j. Journal of Palliative Medicine

k. New England Journal of Medicine

l. Palliative Medicine

m. Other

n. BMC Palliative Care

o. BMJ Supportive and Palliative Care

p. Critical Care Medicine

q. Gynecologic Oncology

r. Journal of Oncology Practice

s. Medical Care

t. The Oncologist

u. Supportive Care in Cancer

APPENDIX C.

Number (%) of Articles Reporting Data on DIALs over Different EoL Timeframes

Timeframe	Retrospective	Prospective	Terminal Hospitalizationa
Final day of life	6 (3.6)	1 (2.2)	8 (17.0) b
Final week of life	26 (15.6)	29 (63.0) c	10 (21.3)
Final 14 days of life	53 (31.7)	13 (28.3)	8 (17.0)
Final month of life	113 (67.7) d	15 (32.6)	9 (19.1)

Note. There were 167 retrospective cohorts, 46 prospective cohorts, and 47 terminal hospitalization cohorts. EoL=End-of-Life. Bolded values in a column are significantly different from the values in the other two columns.

^aMost (n=38, 80.9%) of these studies did not report a specific time interval prior to death.

^bPearson chi2=23.86, df=1, p<0.001,

^cPearson chi2=43.14, df=1, p<0.001,

^dPearson chi2=41.99, df=1, p<0.00

APPENDIX D.

DISCRETIONARY INTERVENTIONS THAT WERE NOT PRE-SPECIFIED

Category	Examples	Category	Examples
General Support and Monitoring	Arterial blood gas Blood draw/Blood test/Blood sampling/Blood work/Lab tests Catheter placement Central line/vascular access placement Imaging/radiologic testing/radiologic exam IV fluids/IV hydration/IV infusion/Microbiology Minor respiratory therapeutic procedure Secretion management Urinary catheter/urinary catheterization Vascular access placement/central line/peripherally Vital signs	Invasive Diagnostics	Biopsy Bronchoscopy/bronchus biopsy Cardiac angiogram Colonoscopy Cystoscopy Electromyelogram Endoscopic retrograde cholangiopancreatography Endoscopy/Upper Gastrointestinal endoscopy Esophagastroduodenoscopy Laryngoscopy Lumbar puncture/diagnostic spinal tap Nephrostogram Procedures requiring pathology consult Transesophageal echocardiogram Urinary tract— therapeutic procedures Urogram
Higher Intensity-Organ Support and Monitoring	Central venous catheter/venous access devices/central catheter Central venous pressure measurement Defibrillation/cardioversion Hemodialysis catheter Insertion of emergency airway Intra-aortic balloon pump Pulmonary artery catheter Therapy/boluses Ventricular assist device	Disease Treatments	Chemoradiation Hormonal therapy Injection of granulocyte colony-stimulating factor Radiation/Radiotherapy Surgery/surgical procedures Transfusion of red blood cells/ platelets/Blood products
Fluid Removal	Abdominal drainage/abdominal paracentesis/paracentesis Chest drain/Pleural drain/thoracentesis/pleural-peritoneal drain Incision of pleura Intraperitoneal drain Pleurodesis Pericardiocentesis Incision of pleura	Managing Obstructions	Cholecystectomy tube placement Nephrostomy/nephrostomy tube placement Obstructive uropathy Percutaneous nephrostomy Percutaneous transhepatic cholangiography and drainage Removal of urinary obstruction Small bowel resection Transurethral excision Urethral stent placement
Wound Management	Debridement of wound,infection, burn, artery Excision of skin lesion Incision and drainage	Bleeding and Clots	Ligation of vessel Ablation therapy Inferior vena cava filter placement Injection of epinephrine
Gastrointestinal, Gastrostomy and jejunostomy tubes *	Percutaneous enterogastric tube Jejunostomy Colorectal procedures/gastroenterology procedures Gastrostomy/percutaneous gastrostomy/G-tube

Note.

^*can be from obstructive causes or other but not artificial nutrition

The words and phrases that appear as examples in this Table were extracted from articles that were included in the review. In some cases, researchers merely reported descriptive data on EoL utilization without framing these activities as discretionary (or inappropriate). In the interest of completeness, we included all of these interventions, recognizing that many of them are symptom-focused activities that are considered vital to caring for patients who are dying.