Identifying the Population with Serious Illness: The “Denominator” Challenge

Abstract

Background: To ensure seriously ill people and their families receive high-quality primary and specialty palliative care services, rigorous methods are needed to prospectively identify this population.

Objective: To define and operationalize a definition of serious illness for the purpose of identifying patients and caregivers who need primary or specialty palliative care services.

Design/Setting: Two stages of work included (1) building expert consensus around a conceptual definition of serious illness and (2) using the National Health and Aging Trends Study linked to Medicare claims data to test a range of operational definitions composed of diagnoses, utilization, and markers of care needs.

Measurements: One-year outcomes included mean total Medicare costs, mortality, and percent hospitalized, as well as those reporting ≥2 measures of need and functional impairment. Sensitivity, specificity, and c-statistics (unadjusted and adjusted for age, sex, race, and Medicaid status) were calculated for each definition across the outcomes.

Results: Conceptually, “Serious illness” is a health condition that carries a high risk of mortality AND either negatively impacts a person's daily function or quality of life, OR excessively strains their caregivers. The range of operational definitions simulated all had low sensitivity and high specificity across all outcomes. None of the definitions reached an unadjusted c-statistic >0.6 (or adjusted >0.7) for identifying a population with ≥2 indicators of care needs.

Conclusions: Standard administrative data are inadequate to identify this population. Defining the seriously ill denominator with high specificity, as described here, will focus efforts toward the highest-need segment of the population, who may indeed benefit most.

Introduction

Health care spending is extremely concentrated and high spending is often a marker of low quality care. The top 5% of spenders account for nearly 60% of healthcare costs.^1–4 This spending is neither easily predictable nor consistent over time. Only 11% of the highest-cost individuals are in the last year of life, while even fewer have predictable prognoses.³ Despite highly concentrated spending, care of this patient population is frequently of low quality: poorly coordinated with multiple transitions across sites of care; marked by inadequate symptom control and low patient and family perceptions of the quality of care; and potentially discordant with personal goals and preferences.^5–12

Healthcare reform aims to improve the value (i.e., raise quality while holding stable or lowering costs) of care for these highest-cost seriously ill patients.¹³ In select groups, palliative care interventions (including an interdisciplinary team of physicians, nurses, social workers, and chaplains) have been shown to improve quality of life, manage symptoms, and support patients and families, and lower costs.^14–20 Yet, not all patients need all aspects of palliative care services and many patient who could benefit receive no palliative care services at all. Efficiently targeting resources, particularly in community settings, to those who need and will benefit from them most before a personal healthcare crisis occurs is a critical step in providing appropriate, value-driven, and patient-centered care to the seriously ill population.

Thus, central to the success of programs that serve high need, high cost patients, such as community-based palliative or serious illness care programs, is accurate identification of those with the greatest need, otherwise the programmatic costs will exceed the potential savings.²¹ Yet, to date, efforts both to efficiently target specialized clinical interventions and appropriately apply quality metrics for the care of the seriously ill have been hindered by our inability to prospectively identify the high-cost, high-need population.

One commonly used approach to identifying the high-cost population is the Centers for Medicare and Medicaid Services hierarchical condition categories (HCC), which rely on diagnostic codes from claims and a complex statistical algorithm to adjust Medicare Advantage capitation payments.^22,23 Yet diagnoses alone fail to adequately predict costs, hospital use, mortality, or, most importantly, unmet care needs.^1,3,24,25 The addition of functional status measures and prior healthcare utilization to identification algorithms may add predictive strength, but clearly defined criteria and consistent data are lacking.^1,25

This article aims to build upon prior work by conceptually defining serious illness and describing the steps needed to operationalize this definition within a healthcare population.^26–30 We align these methods with current alternative payment model (APM) proposals^31,32 and highlight critical challenges and tensions in specifying the approach. To illustrate these issues, we also describe simulated examples of population identification methods using a nationally representative cohort of Medicare beneficiaries. Finally, we outline recommendations for next steps.

Defining the Population

We first reviewed a wide range of literature and reports on conceptual and operational definitions of “serious illness,” “advanced illness,” and the “palliative care population.” We then asked a group of expert advisors and palliative care specialists to define serious illness, specifically for the purpose of identifying a population of patients and caregivers who need primary or specialty palliative care services. The group's iterative work ultimately reached consensus on the following definition (Box 1):

Box 1. Conceptual Definition.

“Serious illness” is a health condition that carries a high risk of mortality AND either negatively impacts a person's daily function or quality of life, OR excessively strains their caregivers.

Our purpose in conceptually defining serious illness is to identify a population of patients and caregivers who need primary or specialty palliative care services.

Next, we discussed the steps needed to operationalize this conceptual definition to create a template or algorithm for patient identification. Through this process, we identified several critical areas of tension to be wary of in one's choice of approach.

First, our primary intent is to identify those people and families with unmet palliative care needs, yet this is currently not possible on a population level due to lack of data and measurement challenges. Moreover, to date, there is no “gold standard” prospective measure of care needs amenable to palliative care services. Instead, many existing methods seek to identify those people at risk for high healthcare costs, a marker of potentially burdensome healthcare utilization and proxy indicator of care needs. While palliative care has been associated with reduced costs, this is an indirect result of providing appropriate care before a crisis and thereby reducing use of emergency services or nonbeneficial interventions. Cost savings is not a primary intent of palliative care services. Reduction of costs, however, is often one of the key components of risk-sharing payment models in which palliative care programs may participate and cost avoidance may support the business plan for a palliative care program's sustainability. In our simulations, we attempt to capture both measures of costs, utilization and care needs.

Data availability is central to the second tension identified. Lacking a gold standard, one might seek a range of measures signaling potential care needs or risk of poor outcomes (e.g., symptoms, functional and cognitive impairment, and caregiver strain or distress). These measures are rarely available in administrative data or electronic health records (EHRs), however, and when they are available, they usually have not been rigorously collected and may vary substantially across sites, providers, or populations. One possible solution is to use the Minimum Data Set (MDS) or Home Health (HH) OASIS data, which contain standardized and mandated assessments of cognition and function.³³ These assessments are only conducted on the subsets of Medicare beneficiaries accessing skilled nursing facility (SNF) or HH services, thereby excluding many who might otherwise meet the proposed definition. So we aim to highlight the population identified with SNF and HH claims data in comparison to those identified through survey-based functional assessments. By doing so, we may develop an approach based upon currently available data, while recognizing its limitations, and highlighting the need to mandate collection of these critical data elements in the future.

Third, the identification process must consider both sensitivity and specificity. In light of imprecise or inadequate data, accurate identification of the target population would require some component of point-of-care or patient-portal survey screening. Identifying too broad a population (highly sensitive definition) could misdirect finite resources to those with minimal care needs, while a method that identifies too narrow a population (highly specific definition) may result in missing a substantial proportion of those who could greatly benefit from services. Thus, a highly specific population definition may be best for a resource-intensive clinical program, but a more sensitive population definition would be preferred for a standard measure of care quality.

Finally, we considered what role, if any, prognosis should have in identifying the target population. While the last months of life are the highest-cost and highest-need period for many people, only 11% of all high-cost high-need patients are in the last year of life.^3,4 Many high-cost patients experience high levels of spending over multiple years.^27,34 By purposefully targeting the end of life, programs may miss or curtail the opportunity to meaningfully improve care and reduce burdensome or inappropriate utilization. Furthermore, hospice services are specifically targeted to patients in the last months of life, whereas palliative care services may be appropriate at any stage of illness. For these reasons, our approach does not aim to identify those people in the last year of life, but we report mortality as a proxy measure of care needs and severity of illness.

In sum, we recognize the inevitable need for a variety of denominator populations. The tensions and challenges described above offer opportunities to simulate and test a range of choices. In the following sections, we describe these preliminary simulations and offer a few specific methods that may fit the purpose of various entities, including community-based palliative care programs, institutions, and payers.

Approach to Identifying the Seriously Ill: Nationally Representative Simulations

Sample and data

Data are drawn from the 2011 National Health and Aging Trends Study (NHATS), linked individual Medicare claims data, and linked caregiver data from the National Study of Caregiving (NSOC).^35,36 NHATS is a population-based longitudinal survey of individuals aged 65 years and older living in the contiguous United States, drawn from the Medicare enrollment file, which represents 96% of all older adults in the United States. In-person interviews were completed between May and November 2011 and yielded a sample of 8245 persons, with a 71% response rate. Among these, 7609 were community dwelling. Nursing Home residents were enrolled, but did not complete full surveys. Proxy respondents were interviewed when the participant could not respond (6%). Study participants who received assistance with daily needs and activities (e.g., personal care, mobility, household activities, transportation, or medically oriented tasks) provided a roster of helpers who provided assistance. NSOC participants were identified from this NHATS helper roster, thus providing linked NHATS participants and NSOC caregivers. NHATS participants are also linked to individual Medicare claims data. To have complete claims data for these analyses, we limited the sample to those with six months of continuous fee-for-service (FFS) Medicare coverage before the NHATS survey date. All measures are adjusted for survey weights to provide estimates relative to the Medicare FFS population.

Components of the approach

Drawing upon the conceptual definition of serious illness, past literature, and expert consensus about elements to include in operationalizing this definition, we selected the following three components for simulations:

• 1. Diagnoses: We used two methods to approach serious medical diagnoses (Table 1). The first approach is drawn from the Dartmouth Atlas of Healthcare and identifies nine serious chronic conditions, associated with about 90% of all deaths, by using only diagnosis codes.^37,38 The second more complex approach is modified from prior work to identify the seriously ill,²⁷ and uses diagnosis codes and additional claims-based elements to provide evidence of advanced disease (e.g., COPD only if using home oxygen or as primary diagnosis on hospital admission). In practice, these diagnoses could be identified through claims data, EHR or billing data, or patient report.

• 2. Utilization: We used measures of healthcare utilization in the past six months as a proxy indicator of higher level of illness severity and care need. We considered acute care hospitalizations, HH and SNF claims, separately and in combination. In addition, we explored durable medical equipment (DME) claims (home oxygen, hospital bed, and wheel chair) as indicators of illness severity and functional impairment.

• 3. Measures of need: Five domains of potential need were identified through evaluation of current APM proposals and expert panel concensus.^31,32 No gold standard exists and various measures could be used to assess these domains. In addition, for this study, we are limited to those measures available from the NHATS and NSOC surveys. These include the following: functional dependence (NHATS: activities of daily living or ADL), nutritional decline (NHATS: unintentional weight loss), cognitive impairment (NHATS: probable dementia algorithm), symptoms (NHATS: pain limits activities), and caregiver strain (NSOC: caregiving-related financial, emotional, or physical difficulty). We considered a person to have “high need” if two or more of these domains were positive.

Table 1.

Two Approaches to Diagnoses for Serious Illness Denominator

Severe chronic conditions, Dartmouth Atlasa	Severe medical conditionsb
Malignant cancer or leukemia	Cancer (poor prognosis, metastatic or hematologic)
Chronic pulmonary disease	Chronic obstructive pulmonary disease or interstitial lung disease, only if using home oxygen or hospitalized for the condition
Renal failure	Renal failure, end stage
Dementia	Dementia with evidence of length of illness or advanced disease
Congestive heart failure	Congestive heart failure, only if hospitalized for the condition
Severe chronic liver disease	Advanced liver disease or cirrhosis
Diabetes with end-organ damage	Diabetes with severe complications (ischemic heart disease, peripheral vascular disease, and renal disease)
Peripheral vascular disease	Amyotrophic lateral sclerosis
Coronary artery disease	Hip fracture, age >70 years
	Multimorbidity, ≥3 chronic conditions (Dartmouth Atlas list)

^aList of diagnoses from Dartmouth Atlas of Health Care report on Variation in End-of-Life Care.³⁷

^bList of diagnoses from Kelley et al.²⁷

Outcomes

Given the tensions described above and the variety of purposes for which a denominator population may be needed, we present a range of descriptive information for each approach. We examined each identified group's one-year outcomes: mean total Medicare costs, mortality, and percent hospitalized. We report the proportion of the identified group who are within the top 10% of total costs; report two or more measures of need; and are dependent for at least one ADL. Finally, we provide measures of sensitivity, specificity, and c-statistic estimates for each approach across the outcomes. In addition to the unadjusted c-statistic, we also calculated c-statistics adjusted for age, sex, race, and Medicaid status. As a comparator, we provide these performance measures for a group identified by the top 10% of HCC scores. At each step, we note the size of the population identified relative to the FFS Medicare population.

Results

The mean total Medicare spending for the full population was $9,459, with 20% experiencing a hospitalization, and 5% dying within one year of follow-up. The top 10% of patients averaged $53,612 and together accounted for 57% of the observed FFS Medicare costs. Nursing home residents represented 4% of the total population and their average Medicare spending was $21,462, with 39% hospitalized and 30% dying within the year. The community-dwelling population had the following characteristics: 28% reported two or more measures of need, 15% had ADL impairment, and 8% had HH or SNF claims.

The first identification simulation took a straight-forward approach using only claims data: a combination of chronic condition diagnosis codes,³⁷ hospital use, and HH/SNF/DME claims over the past six months. This method captured 6% of the total FFS population and selected people with $32,384 average costs, 55% hospitalized and 25% died at one year. Thirty-eight percent were in the top 10% of spenders, 89% reported 2 or more measures of need, and 73% reported ADL impairment (Fig. 1).

FIG. 1.

Simulation of denominator definition using chronic conditions and claims. ^aChronic condition = 9 serious chronic conditions as per the Dartmouth Atlas of Healthcare method. ADL, activities of daily living; DME, durable medical equipment, specifically hospital bed, wheelchair or home oxygen; FFS, fee-for-service; HH, home health; SNF, skilled nursing facility.

In comparison, a more complex approach used claims data to identify serious medical conditions, that is, to limit diagnoses to those with evidence of advanced illness. As above, we then added the claims-based criteria of hospital use and HH/SNF/DME claims over the past six months. This method captured 4% of the total FFS population and selected people with $33,994 average costs, 56% hospitalized and 30% died at one year. Forty percent were in the top 10% of spenders, 91% reported two or more measures of need, and 78% reported ADL impairment (Fig. 2).

FIG. 2.

Simulation of denominator definition using serious medical conditions and claims. ^aSerious medical condition = diagnoses and additional claims-based elements to provide evidence of advanced disease.

As a third approach, we used the same serious medical conditions and hospitalization criteria, but substituted self-reported functional impairment, specifically receiving help with one or more ADL. This method captured 4% of the total FFS population and selected people with $34,425 average costs, 58% hospitalized and 30% died at one year. Forty-two percent were in the top 10% of spenders and 95% reported two or more measures of need (Fig. 3). All six simulated serious illness denominator definitions are displayed together in Figure 4.

FIG. 3.

Simulation of denominator definition using serious medical conditions, claims, and function. ^aSerious medical condition = diagnoses and additional claims-based elements to provide evidence of advanced disease.

FIG. 4.

Serious illness denominator definitions and the identified populations' characteristics over one year. HH, home health.

Next, we considered sensitivity [true positive/(true positive + false negative)], specificity [true negative/(true negative + false positive)], and c-statistics (a more global measure of concordance) for each of the approaches simulated (Table 2). Overall, the range of approaches offered similarly low sensitivity and high specificity across outcomes. None of the methods achieved an unadjusted c-statistic over 0.7, a threshold commonly considered good. Also, none of the methods reached an unadjusted c-statistic >0.6 for identifying a population with two or more indicators of care needs. C-statistics adjusted for age, sex, race, and Medicaid status, particularly those for one-year mortality and spending, resembled those of other published predictive models.^39–42

Table 2.

Denominator Definitions and Corresponding Sensitivity, Specificity, and c-Statistics across One-Year Outcomes

	Conditionaand hospitalization	Condition,ahospitalization, and HH/SNF/DME	Serious Medical Conditionband hospitalization	Serious Medical Condition,bhospitalization, and HH/SNF/DME	Serious Medical Condition,band ADL impairment	Serious Medical Condition,bhospitalization, and ADL impairment	HCC-predicted to be top 10% of costs
Top 10% spending
Sensitivity	0.32	0.23	0.25	0.18	0.30	0.17	0.41
Specificity	0.91	0.95	0.94	0.96	0.91	0.97	0.90
C-statistic	0.61	0.59	0.59	0.57	0.61	0.57	0.66
C-statistic, adjustedc	0.70	0.68	0.69	0.67	0.68	0.67	0.72
Hospitalization within one year
Sensitivity	0.25	0.18	0.19	0.14	0.23	0.13	0.29
Specificity	0.92	0.96	0.95	0.97	0.91	0.97	0.91
C-statistic	0.59	0.57	0.57	0.55	0.57	0.55	0.60
C-statistic, adjustedc	0.67	0.66	0.66	0.65	0.65	0.65	0.68
Death within one year
Sensitivity	0.33	0.26	0.29	0.23	0.42	0.22	0.43
Specificity	0.90	0.94	0.93	0.96	0.90	0.96	0.89
C-statistic	0.62	0.60	0.61	0.59	0.66	0.59	0.66
C-statistic, adjustedc	0.78	0.77	0.78	0.77	0.78	0.77	0.79
Two or more indicators of care need
Sensitivity	0.20	0.14	0.15	0.11	0.22	0.11	0.21
Specificity	0.95	0.98	0.97	0.99	0.96	1.00	0.93
C-statistic	0.57	0.56	0.56	0.55	0.59	0.55	0.57
C-statistic, adjustedc	0.67	0.67	0.67	0.66	0.69	0.66	0.68
n	615	378	415	287	606	276	691
National Estimate	2,334,035	1,339,666	1,552,984	1,005,124	2,050,528	966,236	2,375,360
Medicare FFS population, %	9.9%	5.7%	7.2%	4.2%	9.5%	4.1%	10.0%

Sensitivity = true positive/(true positive + false negative). Specificity = true negative/(true negative + false positive) for identifying those at risk for being among the costliest 10% of beneficiaries, death and hospitalization in the following year, and reporting two or more care need indicators. Bold = c-statistic ≥0.6.

^aChronic condition = 9 serious chronic conditions as per the Dartmouth Atlas of Healthcare method.

^bSerious medical condition = diagnoses and additional claims-based elements to provide evidence of advanced disease.

^cAdjusted C-statistic = adjusted for age, sex, race, and Medicaid status.

ADL, activities of daily living; DME, durable medical equipment, specifically hospital bed, wheelchair, or home oxygen; FFS, fee-for-service; HCC, hierarchical condition categories; HH, home health; SNF, skilled nursing facility.

Special subpopulations

We also evaluated those with dementia as a unique subpopulation (6% of the total population). We identified dementia with diagnosis codes and limited the sample to those with a hospital admission and living within the community (1% of total FFS population and 22% of total dementia population). This method captured people with $30,755 average costs, and 54% were hospitalized and 23% died at one year. Forty percent were in the top 10% of spenders, 83% report two or more measures of need, and 77% reported ADL impairment (data not shown).

Finally, those with multimorbidity (three or more severe chronic conditions) were evaluated as another unique subpopulation (11% of the total population). We identified multimorbidity with diagnosis codes and limited the sample to those with a hospital admission and living within the community (5% of total FFS population and 47% of total community-dwelling multimorbidity population). This method captured people with $30,291 mean costs, and 57% were hospitalized and 21% died at one year. Thirty-nine percent were in the top 10% of spenders, 73% report two or more measures of need, and 59% reported ADL impairment (data not shown).

Discussion

Ensuring the delivery of high-quality care to those with serious illness across all sites of care is inextricably linked to our ability to identify that population. We have refined a conceptual definition of serious illness, that is, the population of patients and caregivers who need primary or specialty palliative care services, and highlighted the current challenges and tensions inherent in operationalizing a definition of this patient population. The greatest barrier is the lack of a “gold standard”—valid and reliable measure of care needs amenable to palliative care services—against which the operational definitions can be tested on a generalizable population. In lieu of a gold standard, we have tested a range of definitions against outcomes that may proxy unmet care needs: hospitalization, costs, mortality, and survey-based measures of potential care need (i.e., functional and cognitive impairment, symptoms, nutritional decline, and caregiver strain). All the tested definitions had high levels of specificity and low sensitivity.

Recommendations to apply now

Despite limitations in available data for population identification, there are steps that can be applied now. Furthermore, we recognize that one single denominator definition is not appropriate for all purposes and have laid out the components that should be considered or included in each approach. Based upon the simulations evaluated, we offer the following recommendations for a few targeted purposes.

Community-based palliative care programs functioning with limited resources for screening and aiming to target the highest-cost and highest-need group should apply the most straightforward approach (Fig. 1) using only chronic condition diagnosis codes³⁷ and past hospital use with or without HH/SNF/DME claims. This errs on the side of specificity over sensitivity and recognizes that programs may have administrative data on diagnoses and hospitalizations, but may not have full claims records, including HH, SNF, and DME.

Risk-bearing entities, institutions with a large at-risk portfolio, such as an Accountable Care Organization (ACO), or payers should consider an approach with greater sensitivity. Such entities are able to access the data needed for the more complex approach to indentifying serious medical conditions,²⁷ and this combined system-wide collection of function data or telephonic screening would replicate the approach with the greatest sensitivity (Fig. 3).

Quality measures will require a range of denominators. Measures intended for use in the accountability sphere require high specificity, to promote high-quality care for those with greatest need, while avoiding unintended consequences for a broader group. Quality improvement metrics, on the other hand, must be sensitive. For example, measures of access to palliative care services should be based upon a denominator with the greatest sensitivity, so that it does not unintentionally exclude patients with unmet care needs. Many other measures should only be applied to highly specific groups or may be applicable only to specific disease or condition subgroups. And yet, others may require further specifying the denominator population (e.g., those with a stated preference to die at home).

Research priorities and recommendation for future

This work reveals several important research priorities. Further research is needed to refine these simulation methods and improve the measurement of need, so that it is better aligned with patient and family concerns that may be modifiable or are specifically addressed by palliative care services. This will encompass several focused areas of research and implementation, as listed below:

• 1. Establish a “gold standard” measure of care needs amenable to palliative care services and, within a broad population, test the sensitivity and specificity of utilization-based identification criteria against this “gold standard.”

• 2. Recruit several large healthcare systems to use one of the proposed denominator criteria to conduct recurring assessments of patients' specialty or primary palliative care needs and observe patients' outcomes over time.

• 3. Translate these findings to a comprehensive EHR strategy, perhaps in partnership with major EHR vendors, to identify the “denominator,” expand access to services, and ease quality measurement for accountability.

• 4. Assess the degree to which a utilization-based denominator definition can be enhanced by function and cognitive status measures, that is, those assessments mandated by the IMPACT Act³³ in the post-acute setting; measure what population is excluded from the denominator because of the lack of these assessments; and estimate the impact of expanding the IMPACT Act assessment mandate to the inpatient acute care setting.

• 5. Conduct additional identification simulations to identify special subpopulations: advanced dementia, multimorbidity, frailty, and others.

• 6. Develop a brief screening tool to identify patients' need for enhanced services, trial its use in patient-facing EHR portals or other modes of data collection, and evaluate the effectiveness and efficiency of additional screening to identify those who would benefit from enhanced services.

Conclusions

Seriously ill people and their families are at risk of low quality and overly burdensome healthcare, and timely primary or specialist palliative care services may ameliorate this risk. An upstream and proactive approach to meeting the care needs of this vulnerable population requires a method of prospective identification. The primary impediments currently are the lack of a “gold standard” for identification of need and inadequate assessment measures in administrative data. The simulated approaches to defining the seriously ill denominator population described in this work offer high levels of specificity. Clinical programs may choose to focus on this highest-need segment of the seriously ill population and may indeed have the highest positive impact on care for this group now. Further research is needed, however, to improve the sensitivity of the denominator definition, so that providers may be held accountable for providing high-quality care to all people with serious illness in the future.

Acknowledgments

This work was completed with support of the Gordon & Betty Moore Foundation and the National Palliative Care Research Center. Expert panel for conceptual definition development (Randal Curtis, Laura Hanson, Steve Pantilat, Christine Ritchie, Joan Teno, and James Tulsky). A.S.K. receives support from the National Institute on Aging (NIA) (K23-AG040774), the American Federation for Aging Research, and National Palliative Care Research Center.

Author Disclosure Statement

No competing financial interests exist.