The Advanced Dementia Prognostic Tool (ADEPT): A Risk Score to Estimate Survival in Nursing Home Residents with Advanced Dementia

Abstract

Context

Estimating life expectancy is challenging in advanced dementia.

Objectives

To create a risk score to estimate survival in nursing home (NH) residents with advanced dementia.

Methods

This was a retrospective cohort study performed in the setting of all licensed US NHs. Residents with advanced dementia living in US NHs in 2002 were identified using Minimum Data Set (MDS) assessments. Mortality data from Medicare files were used to determine 12-month survival. Independent variables were selected from the MDS. Cox proportional hazards regression was used to model survival. The accuracy of the final model was assessed using the area under the receiver operating characteristic curve (AUROC). To develop a risk score, points were assigned to variables in the final model based on parameter estimates. Residents meeting hospice eligibility guidelines for dementia, based on MDS data, were identified. The AUROC assessed the accuracy of hospice guidelines to predict six-month survival.

Results

Over 12 months, 40.6% of residents with advanced dementia (n=22,405) died. Twelve variables best predicted survival: length of stay, age, male, dyspnea, pressure ulcers, total functional dependence, bedfast, insufficient intake, bowel incontinence, body mass index, weight loss, and congestive heart failure. The AUROC for the final model was 0.68. The risk score ranged from 0–32 points (higher scores indicate worse survival). Only 15.9% of residents met hospice eligibility guidelines for which the AUROC predicting six-month survival was 0.53.

Conclusion

A mortality risk score derived from MDS data predicted six-month survival in advanced dementia with moderate accuracy. The predictive ability of hospice guidelines, simulated with MDS data, was poor.

Introduction

Dementia is a leading cause of death in the United States (1). Accurately estimating life expectancy in advanced dementia has been a major barrier to providing palliative care to persons dying with this condition (2–7).

Prognostication helps guide end-of-life decision-making (8–10). Nursing home (NH) residents with advanced dementia whose family members understand that their prognosis is poor have reduced likelihood of receiving burdensome interventions in the last 90 days of life (10). Prognosis is also a determinant for hospice eligibility, which requires an estimated life expectancy of less than six months (3–7, 11). While hospice is beneficial for patients dying with dementia (12–14), many do not receive hospice services (15). Physicians cite difficulty assessing prognosis as the greatest barrier to referring dementia patients to hospice (16). Hospice eligibility guidelines for dementia have been criticized because they were not derived from empirical data and do not accurately predict six-month survival (5, 7).

Prior efforts to develop and validate statistical models to predict survival in advanced dementia are limited (2, 5, 7, 17–19). In 2004, our group demonstrated the feasibility of deriving a mortality risk score based on the Minimum DataSet (MDS) data (20, 21), that estimated 6-month life expectancy in advanced dementia with moderate accuracy (5). However, this earlier work had several limitations including: data from only two states, inclusion of only newly admitted NH residents, survival truncated at six months, use of hazard ratios rather than the logistic regression coefficients (log hazard ratios) to derive point scores, and validation of the risk score using only retrospective, secondary data.

To build on this earlier effort and address its limitations, the National Institutes of Health funded a four-year study with the following two objectives: 1) to derive and validate a mortality risk score (Advanced Dementia Prognostic Tool (ADEPT) in a nationwide population of NH residents with advanced dementia (not limited only to new admissions) using 2002 MDS assessments and 12-month follow-up survival data from Medicare files, and 2) to conduct a prospective validation of the ADEPT risk score at the bedside of 600 NH residents with advanced dementia. Additional aims for both the secondary analyses and prospective validation included a comparison of the accuracy of the ADEPT risk score with current hospice eligibility guidelines to predict six-month mortality. With the prospective validation in progress, this report presents the results from the first of these two objectives; the derivation and validation of the ADEPT score using secondary data.

Methods

Study Sample

The cohort was identified from the 2002 National Repository of the MDS; a comprehensive, standardized resident assessment instrument federally mandated for use in all licensed US NHs (20, 21). Full MDS assessments are completed on all residents within 14 days of admission, annually, and whenever there is a significant change in status. Abbreviated MDS assessments are completed on a quarterly basis.

NH residents with an admission or annual full MDS assessment completed between January 1 and December 31, 2002 were selected. If residents only had a single assessment during that period, it was considered the baseline assessment, if they had multiple full assessments, then the baseline assessment was arbitrarily chosen as the one completed closest to April 1, 2002. Eligible subjects were age ≥ 65 years and had advanced dementia as determined from their baseline assessment. Advanced dementia was defined as having a diagnosis of dementia (Alzheimer’s disease or other causes) and a Cognitive Performance Score (CPS) of 5 or 6 (20, 22). The Alzheimer’s disease and dementia diagnoses have been used for epidemiological research (5, 12, 13) and have intraclass coefficients of 0.89 and 0.79, respectively (21). The CPS uses five MDS variables to group residents into seven cognitive performance categories: 0=intact, 1=borderline intact, 2=mild impairment 3=moderate impairment, 4=moderately severe impairment, 5=severe impairment, and 6=very severe impairment with eating problems. A CPS score of 5 corresponds to mean Mini-Mental State Examination score of 5.1 ± (SD) 5.3.

Mortality Data

To determine survival after baseline, the MDS assessments of eligible subjects were matched with 2002 and 2003 Medicare denominator files, which include dates of death obtained from the Social Security Administration records. Residents were considered to have died within the 12-month follow-up period when the date of death recorded in the Medicare denominator file was within 12 months of the date on the baseline MDS assessment. Matching was achieved using the residents’ Heath Insurance Claim (HIC) and Social Security numbers. Exact matches based on HIC and/or Social Security numbers were further validated using date of birth and gender. A match rate of 91% of eligible subjects was achieved with this approach.

Predictors of Survival

Based on the literature and our prior work (2, 5, 7, 11, 17–19, 23–30), variables were selected a priori from the baseline MDS assessment that were potential predictors of survival in advanced dementia. Factors exogenous to the health status of the resident that could be ordered at the discretion of providers or family members (i.e., advance directives, treatments) were not considered for analysis. The categories of predictor variables included: demographic data, functional status, cognitive status, comorbid diagnoses, nutrition and hydration markers, and other health conditions.

Demographic information included age, gender, and race (non-Hispanic white, black, Asian, American Indian, and Hispanic). Subjects whose baseline MDS assessment was completed for purposes of NH admission (vs. annual assessment) were identified and are hereafter referred to as “recent admissions.” Measures of cognition included a CPS score of 5 vs. 6, and the following specific MDS items: cognitive deterioration in prior 90 days (vs. improved or no change), not awake most of day, hallucinations or delusions, and rarely able to make oneself understood.

Functional ability was assessed using Activities of Daily Living (ADL) scale (range 0–28, higher scores indicate greater disability) (31). Other functional status variables included bedfast most of the day, and functional deterioration over the past 90 days (vs. no change or improved).

Active medical diagnoses included: diabetes mellitus, arteriosclerotic heart disease, cardiac dysrhythmias, congestive heart failure, hypertension, peripheral vascular disease, stroke, Alzheimer’s disease (vs. other causes of dementia), stroke, seizure disorder, Parkinson’s disease, chronic obstructive pulmonary disease (COPD), anemia, cancer (type of cancer not specified in the MDS), renal failure, pneumonia or other respiratory tract infection, urinary tract infection in the previous 30 days, other infections (septicemia, wound infection, methicillin resistant Staphylococcus areus, or Clostridium difficile), hip fracture in the prior 180 days, and other fracture in the prior 180 days.

Markers of nutrition and hydration included: weight loss (> 5% in the previous 30 days or > 10% in the previous 180 days), insufficient oral intake (did not consume almost all liquids in previous 3 days or 25% or more of food uneaten at most meals), chewing or swallowing problems, and body mass index (BMI) (kg/m²) (categorized as < 18.5, 18.5 < 25.0, 25.0 < 30.0, 30.0 < 35.0, and = ≥ 35.0).

Other health conditions present in the seven days prior to the MDS assessment included: peripheral edema, bowel incontinence (rarely or never vs. occasionally, frequently or always), fever, at least one pressure ulcer ≥ stage 2, and shortness of breath. The presence of recurrent lung aspirations in the prior 90 days was also ascertained.

Determining Hospice Eligibility with MDS Data

Based on National Hospice and Palliative Care Organization guidelines (11), patients with a primary diagnosis of dementia must meet two requirements to be eligible for hospice. First, patients must be at stage 7c on the Functional Assessment Staging (FAST) scale (32). Second, patients must have experienced at least one of the following medical conditions over the prior 12 months: aspiration pneumonia, pyelonephritis or other upper urinary tract infection, septicemia, multiple decubitus ulcers ≥ stage 3, recurrent fever after antibiotics, and severe eating problems or tube-feeding accompanied by a > 10% weight loss over the past six months (or serum albumin < 2.5 gm/dl).

In order to determine whether or not the subjects in our cohort were eligible for hospice, hospice eligibility guidelines were simulated with MDS variables (Table 1). The FAST scale assesses functional status in dementia and consists of seven major stages with a total of 16 sub-stages (possible range, stage 1–7f). ³² Stage 7 is the most advanced phase and consists of sub-stages 7a–7f, defined as follows: 7a=speech is limited to five words or less, 7b=all intelligible vocabulary is lost, 7c=non-ambulatory, 7d=unable to sit independently, 7e=unable to smile, and 7f=unable to hold head up. To be considered at stage 7c, patients must have progressed through all the previous FAST stages in a sequential fashion. Therefore, subjects who had all the MDS variables that most closely matched the description of the FAST stages 6 through 7c on their baseline assessments were identified (Table 1). These variables included: limited or more extensive assistance needed for dressing and toileting, supervision or more assistance needed bathing, urinary incontinence at least twice a week (and no indwelling catheter), fecal incontinence at least twice a week, rarely or never able to make themselves understood, and inability to ambulate without extensive assistance.

Table 1.

Description of Hospice Eligibility Guidelines for Dementia Simulated with Minimum Data Set Variables

Functional Assessment Stage	Minimum Data Set Variablea
6a = Improperly putting on clothes without assistance/cueing occasionally or more frequently over the past weeks.	Limited or more extensive assistance required to dress on least several occasions during the last 7 days.
6b = Unable to bathe properly occasionally or more frequently over the past weeks.	Supervision or more assistance required to bathe during the last 7 days.
6c = Inability to handle the mechanics of toileting occasionally or more frequently over the past weeks.	Limited or more extensive assistance to use the toilet at least several times during the last 7 days.
6d = Urinary incontinence occasionally or more frequently over the past weeks.	Urinary incontinence at least twice a week.
6e = Bowel incontinence occasionally or more frequently over the past weeks.	Bowel incontinence at least twice a week.
7a = Ability to speak limited to < 6 different intelligible words in an average day.	Rarely/never makes self understood
7b = Ability to speak limited to < 1 intelligible word in an average day.
7c = Unable to ambulate without assistance	Extensive assistance (or total dependence) required for locomotion (i.e., move between locations) during the last 7 days.
Medical Conditions in Prior Year	Minimum Data Set Variables
Aspiration pneumonia	Recurrent lung aspiration in past 90 days and pneumoniab
Pyelonephritis or another upper urinary tract infection	Urinary tract infection in prior 30 days
Septicemia	Septicemia
Multiple stage 3 or 4 decubitus ulcers	Multiple stage 3 or 4 decubitus ulcers in past 7 days
Recurrent fevers after antibiotic treatment	Fever
Insufficient oral intake or tube-feeding with impaired nutritional status (10% weight loss within the prior 6 months or serum albumin < 2.5 gm/dl).	Insufficient oral intake (did not consume almost all liquids in previous 3 days or ≥25% of food uneaten at most meals) or tube- feeding with 10% or more weight loss in the previous 6 months.b

^aMinimum Data Set Variables used to determine Functional Assessment Stage were ascertained from the baseline assessment. A medical condition was considered present if it occurred on any available MDS assessments completed during the year prior to baseline or on the baseline assessment.

^bConditions had to be present on the same assessment.

Variables were also identified from the MDS that most closely matched the description of the other medical conditions required for hospice eligibility (Table 1). These conditions were determined from the subjects’ baseline MDS assessment as well as all available full and quarterly assessments from the 12 months prior to baseline (i.e., from 2001 and 2002). Subjects having the following conditions on these assessments were identified: recurrent aspiration in prior 90 days and pneumonia (both conditions had to be present on the same assessment), urinary tract infection in the prior 30 days, septicemia, multiple decubitus ulcers ≥ stage 3, fever, insufficient oral intake (did not consume almost all liquids in previous three days or 25% or more of food uneaten at most meals) or tube-feeding and ≥ 10% weight loss in the previous six months (albumin levels are not available in the MDS).

Hospice eligibility was only assessed among subjects in the NH at least 12 months, as this was necessary to determine presence of co-existing medical conditions. Subjects were considered eligible for hospice if they met MDS criteria for FAST stage 7c at baseline and had at least one of the aforementioned medical conditions present within the prior 12 months.

Statistical Analyses

Descriptive analyses were conducted to examine the frequencies (categorical variables) and means with standard deviations (SD) (continuous variables) of all potential predictors (independent variables) of survival. Survival (dependent variable) was analyzed for all subjects. For subjects who died within 12 months of their baseline MDS assessment, survival was defined as the duration between the baseline assessment and death dates. Subjects who did not die within 12 months treated as censored observations.

The first step in deriving the survival model involved bivariable analyses to examine the associations between each individual independent variable and survival using unadjusted Cox proportional hazards regression with ties handled using Efron’s approximation method (33). The results of the bivariable analyses guided the selection of independent variables to be considered in a multivariable model. Continuous variables were transformed into categorical variables based on clinically and statistically meaningful subdivisions in order to facilitate their application in a practical risk score. For example, age was categorized into five-year intervals, ADL score was dichotomized to either a value equal to 28 (total functional dependence) vs. < 28, and BMI was grouped into 5 kg/m² intervals, with the lower group cut-off at 18.5 kg/m² which is the threshold for being underweight (34). The associations between survival and transformed variables were examined in both their original and newly defined forms. We assessed the proportional hazards assumption by examining log-log survival plots (35). No substantive violations of the assumption were identified.

Given the number of comparisons and the large sample size, associations that would ordinarily be classified as statistically significant (e.g., P<0.05) may be clinically insignificant. Rather than choosing an arbitrarily small P-value to decide which variables to retain for the multivariable analysis, the selection of independent variables for entry into the multivariable model building procedure was guided by the relative strength of statistical significance, collinearity, the feasibility of operationalizing the variable in a risk score, and clinical experience. Our goal was to create a parsimonious and practical risk score with no more than 12 variables.

Multivariable analysis using Cox proportional hazards regression with ties handled using Efron’s approximation method was used to derive a final survival model. The final model selection was based on all subsets regression optimizing the score chi-square statistic (36). Collinearity was assessed by examining variance inflation factors (37).

To create the ADEPT risk score, points were assigned based on the precise weighting of the regression coefficients (log hazard ratios) of each risk factor in the final multivariable survival model. In the unadjusted analysis, the regression coefficients for the five-year age categories were nearly perfectly linear. Therefore, the five-year age categories were incorporated as a linear term in the model, and the risk score points were based on the regression coefficient for the age variable coded in this manner in the multivariable model. With this approach, the lowest age category (65 < 70 years) was standardized to 1 point. Each five-year age increment was assigned an additional point. For all other variables, point values were assigned by multiplying their regression coefficients by the inverse of the regression coefficient for the age categories. Point scores were rounded to one decimal place for ease of calculation. To obtain a risk score for a subject, point values were summed for all mortality-related factors present for that individual on the baseline MDS assessment.

The final multivariable model was internally validated using cross-validation, and its accuracy was assessed with measures of discrimination and calibration (reliability). Cross-validation corrects these measures for overfitting (optimism). Discrimination was measured using a generalized area under the receiver operating characteristic curve (AUROC) which quantifies the ability of the prediction model to separate those who died from those who remained alive. ³⁸ Calibration was assessed by dividing subjects into deciles of risk according to their model predictions, and the observed mortality rate among the subjects in each decile was plotted against the average predicted probability of death and compared to the 45° line (perfect calibration). ³⁹ Separate calibration curves were constructed for six-month and 12-month survival.

In addition to assessing the accuracy of the final multivariable Cox proportional hazards model to predict survival, optimism-corrected estimates of the AUROCs were also used to measure the discriminatory power of the final ADEPT risk score for predicting the dichotomized outcomes of six-month and 12-month survival.

Finally, the discriminatory power of hospice eligibility guidelines to predict 6-month survival was compared to the newly derived ADEPT risk score. Whether or not a subject meets hospice eligibility was a binary outcome, whereas the new risk score was an ordinal measure. Therefore, to make a fair comparison of the two measures, the specificity of the hospice guidelines in estimating six-month survival was computed, and the cut-off necessary to give the same specificity for the ADEPT score was determined. After the two diagnostic criteria were set to give the same specificity, the AUROCs of the current hospice eligibility guidelines and the ADEPT risk score were compared.

All analyses were conducted using SAS (SAS Institute Inc., Cary, NC) and S-PLUS (Tibco Software Inc., Palo Alto, CA).

Results

Study Sample

Among the 2,333,662 NH residents with full MDS assessments between January 1, 2002 and December 31, 2002, 245,132 (10.5%) residents had met eligibility criteria for advanced dementia. Adequate matching between the MDS assessments and the Medicare denominator file was achieved for 222,532 (91%) of those residents. An additional 127 residents were excluded because their reported death date preceded the baseline MDS assessment date, leaving 222,405 subjects in the final cohort. Eligible residents who were excluded (9%) had similar distributions as the matched cohort with respect to age, gender, race, NH length of stay, and CPS score (i.e., 5 vs. 6).

A total of 49% of subjects were between 80–90 years (mean age 84.5 ± 7.5 (SD) years), 23.0% were male, 84.0% were white, and 36.2% were recent admissions to the NH (Table 2). Slightly over half (52.1%) of subjects had a CPS score of 6 (vs. 5), 35.5% were totally functionally dependent (ADL score = 28), and 9.1% were bedfast. The most common active medical diagnoses included: hypertension (48.8%), anemia (22.5%), stroke (21.0%), diabetes (19.1%), and congestive heart failure (17.0).

Table 2.

Baseline Characteristics of Nursing Home Residents with Advanced Dementia and Their Unadjusted Associations with Survival (n=222,405)

Characteristic	No. (%)	Unadjusted Association with Survival Hazard Ratio (95% Confidence Interval)
Demographics
Age (years)
65 < 70	6,766 (3.0)	Referent
70 < 75	15,685 (7.1)	1.18 (1.12–1.24)
75 < 80	33,060 (14.9)	1.30 (1.24–1.37)
80 < 85	51,970 (23.4)	1.46 (1.39–1.53)
85 < 90	56,979 (25.6)	1.67 (1.53–1.75)
90 < 95	39,407 (17.7)	1.91 (1.82–2.00)
95 < 100	15,379 (6.9)	2.16 (2.05–2.27)
≥ 100	3,159 (1.4)	2.51 (2.36–2.68)
Male	51,223 (23.0)	1.54 (1.52–1.57)
Racea
Non-Hispanic white	186,554 (84.0)	Referent
Black	25,655 (11.6)	0.82 (0.80–0.84)
Hispanic	7,228 (3.3)	0.85 (0.81–0.88)
Asian	1,935 (0.9)	0.93 (0.87–1.00)
American Indian	678 (0.3)	0.98 (0.87–1.11)
Recent NH admission	80,394 (36.2)	1.90 (1.87–1.92)
Cognitive status
Cognitive Performance Score of 6 (vs. 5)	115,902 (52.1)	1.44 (1.42–1.46)
Cognitive deterioration in past 90 days	26,773 (12.0)	1.72 (1.69–1.75)
Lethargic or not awake most of the day	60,808 (27.3)	1.50 (1.48–1.52)
Hallucinations or delusions	12,642 (5.7)	0.90 (0.87–0.92)
Rarely makes oneself understood	93,884 (42.2)	1.07 (1.06–1.09)
Functional status
Activity of Daily Living Score =28b	79,020 (35.5)	1.44 (1.42–1.45)
Bedfast most of day	20,116 (9.0)	2.09 (2.05–2.13)
Functional deterioration in past 90 days	48,686 (21.9)	1.66 (1.63–1.68)
Active diagnoses
Diabetes mellitus	42,410 (19.1)	1.16 (1.14–1.18)
Arteriosclerotic heart disease	29,915 (13.5)	1.13 (1.11–1.15)
Cardiac dysrhythmias	26,450 (11.9)	1.42 (1.39–1.44)
Congestive heart failure	37,756 (17.0)	1.44 (1.42–1.47)
Hypertension	108,602 (48.8)	1.09 (1.07–1.10)
Peripheral vascular disease	23,285 (10.5)	1.10 (1.08–1.13)
Stroke	46,782 (21.0)	1.20 (1.19–1.22)
Alzheimer’s disease (vs. other dementia)	103,838 (46.7)	0.80 (0.79–0.81)
Seizure disorder	20,914 (9.4)	0.91 (0.89–0.93)
Parkinson’s disease	20,750 (9.3)	1.07 (1.05–1.10)
Chronic obstructive pulmonary disease	23,007 (10.3)	1.36 (1.33–1.39)
Anemia	50,106 (22.5)	1.17 (1.15–1.18)
Cancer	14,140 (6.4)	1.51 (1.47–1.55)
Renal Failure	8,567 (3.9)	1.76 (1.71–1.81)
Pneumonia or respiratory tract infection	17,795 (8.0)	2.03 (1.99–2.07)
Urinary tract infection in prior 30 days	28,754 (12.9)	1.51 (1.49–1.54)
Other infectionsc	10,159 (4.6)	2.09 (2.03–2.14)
Hip fracture prior 180 days	7,196 (3.2)	1.37 (1.33–1.42)
Other (non-hip) fracture prior 180 days	5,308 (2.4)	1.22 (1.17–1.27)
Markers of nutrition and hydration
Weight lossd	20,302 (9.2)	1.80 (1.77–1.84)
Insufficient oral intakee	93,121 (41.9)	1.47 (1.45–1.49)
Chewing or swallowing problem	118,230 (53.2)	1.39 (1.37–1.40)
Body mass index (kg/m2)a
< 18.5	30,664 (14.0)	1.46 (1.43–1.48)
18.5 < 25.0	116,209 (53.0)	referent
25.0 < 30.0	54,549 (24.9)	0.76 (0.74–0.77)
30.0 < 35.0	14,903 (6.8)	0.66 (0.64–0.68)
≥ 35.0	3,030 (1.4)	0.66 (0.62–0.71)
Other health conditions
Peripheral edema	33,241 (15.0)	1.37 (1.35–1.39)
Bowel incontinencef	193,418 (87.0)	1.44 (1.41–1.47)
Fever in prior 7 days	7,081 (3.2)	2.15 (2.09–2.22)
Pressure ulcers, at least one ≥ stage 2	32,661 (14.7)	2.14 (2.10–2.17)
Shortness of breath	11,705 (5.3)	2.29 (2.24–2.35)
Recurrent lung aspirations in prior 90 days	2,317 (1.0)	2.45 (2.33–2.58)

^aMissing data: race, n=355; body mass index, n=3,050.

^bActivities of daily living score (0–28) is the sum of scores in seven domains of function including: bed mobility, dressing, toileting, transfer, eating, grooming, and locomotion. Each is scored on a 5-point scale (0, independent; 1, supervision; 2, limited assistance; 3, extensive assistance; and 4, total dependence). A score of 28 represents complete functional dependence.

^cOther infections include septicemia, wound infection, methicillin resistant Staphylococcus areus, Clostridium difficile.

^dWeight loss is defined as >5% body weight in prior 30 days, or > 10% in prior 180 days.

^eInsufficient oral intake: not consuming almost all liquids in previous three days or ≥ 25% of food uneaten at most meals

^fBowel incontinence occasionally, frequently or always (vs. rarely or never).

With respect to nutritional markers, 9.2% of subjects had had a recent weight loss, 41.9% had insufficient oral intake, 53.2% had a chewing and swallowing problem, and 14.0% had a BMI < 18.5 kg/m². Other health conditions experienced by the cohort included: bowel incontinence (87.0%), peripheral edema (15.0%), and at least one pressure ulcer > stage 2 (14.7%).

Modeling Survival

At total of 90,324 (40.6%) subjects died within 12 months of their baseline assessments. Table 2 presents the bivariable associations of each independent variable and survival. Owing to the large sample size and our a priori selection of variables expected to be associated with mortality, the unadjusted associations between almost all independent variables and survival were statistically significant. After examining the relative strength of these associations, collinearity, feasibility of operationalizing a given variable in a risk score, and applying clinical experience, the following variables were selected as candidates for the multivariable model: age, gender, recent admission, not awake most of the day, ADL score, bedfast, dysrhythmias, congestive heart failure, COPD, cancer, renal failure, pneumonia or respiratory tract infection, urinary tract infection, other infections, weight loss, insufficient oral intake, chewing or swallowing problems, BMI < 18.5 kg/m², bowel incontinence, fever, pressure ulcers, shortness of breath, and lung aspiration.

Using Cox proportional hazards models guided by the score chi-square criterion, the 12 variables that best predicted survival were selected (Table 3). The variables in the final model included: recent admission, age, male, shortness of breath, pressure ulcers, ADL score, bedfast, insufficient oral intake, bowel incontinence, BMI < 18.5 kg/m², weight loss, and congestive heart failure. The optimism-corrected AUROC for this final model was 0.68.

Table 3.

Final Multivariable Model of Characteristics Associated with Survival Among Nursing Home Residents with Advanced Dementia (n=218,088)^a

Characteristic	Adjusted Hazard Ratio (95% Confidence Interval)	Regression Coefficient (Log Hazard Ratio)b	Points in Risk Scorec
Recent nursing home admission	1.72 (1.69–1.75)	0.54207	3.3
Age (years)(per 5-year increment)	1.18 (1.17–1.18)	0.16431
65 < 70	-	-	1.0
70 < 75	-	-	2.0
75 < 80	-	-	3.0
80 < 85	-	-	4.0
85 < 90	-	-	5.0
90 < 95	-	-	6.0
95 < 100	-	-	7.0
≥ 100	-	-	8.0
Male	1.71 (1.68–1.74)	0.53623	3.3
Shortness of breath	1.57 (1.53–1.61)	0.44903	2.7
At least one pressure ulcers ≥ stage 2	1.44 (1.41–1.46)	0.36216	2.2
Activity of Daily Living Score =28d	1.42 (1.40–1.44)	0.34929	2.1
Bedfast most of day	1.41 (1.38–1.44)	0.34024	2.1
Bowel incontinencef	1.37 (1.34–1.40)	0.31275	1.9
Body mass index < 18.5 kg/m	1.35 (1.32–1.37)	0.29841	1.8
Weight lossg	1.30 (1.27–1.33)	0.26149	1.6
Congestive heart failure	1.28 (1.26–1.30)	0.24739	1.5

^an = 218,088 (vs. n=222,405 for total eligible cohort) because some subjects (n=4,317) have missing values for at least one characteristic in the final multivariable model.

^bPrecision of regression coefficients to 5 decimal places is needed to determine the point score accurately.

^cBased on the regression coefficients (log hazard ratios) in the final model, each variable was assigned points in the ADEPT risk score. The lowest age category (65 < 70 years) was standardized to 1 point. Each 5-year age increment was assigned an additional point. The regression coefficients of all other variables were multiplied by the inverse of the regression coefficient for the age categories (1/0.16431=6.08606).

^dActivities of daily living score (0–28) is the sum of scores in seven domains of function including: bed mobility, dressing, toileting, transfer, eating, grooming, and locomotion. Each is scored on a 5-point scale (0, independent; 1, supervision; 2, limited assistance; 3, extensive assistance; and 4, total dependence). A score of 28 represents complete functional dependence.

^eInsufficient oral intake: not consuming almost all liquids in previous three days or ≥ 25% of food uneaten at most meals.

^fBowel incontinence occasionally, frequently or always (vs. rarely or never).

^gRecent weight loss is defined as >5% body weight in prior 30 days, or > 10% in prior 180 days.

Risk Score Derivation and Accuracy

Based on the regression coefficients (log hazard ratios) in the final model, each variable was assigned points in the ADEPT risk score. The lowest age category (65 < 70 years) was standardized to 1 point. Each five-year age increment was assigned an additional point. The regression coefficients of all other variables were multiplied by the inverse of the regression coefficient for the age categories (1/0.16431=6.08606) (n.b., precision to 5 decimal places is necessary to calculate the correct point scores). For example, the regression coefficient for male sex was 0.53623; therefore, after rounding to the first decimal place, it was assigned a point score of 3.3 (0.53623 × 6.08606). With this approach, the points assigned for each variable were as follows: length of stay < 90 days, 3.3 points; age, 1 65< 70 years, 1 point, with 1 additional point for every 5-year increment; male, 3.3 points; shortness of breath, 2.7 points; at least one pressure ulcer > stage 2, 2.2 points; ADL score = 28, 2.1 points; bedfast, 2.1 points; insufficient oral intake, 2.0 points; bowel incontinence, 1.9 points; BMI < 18.5 kg/m², 1.8 points; recent weight loss, 1.6 points; and congestive heart failure, 1.5 points. The range of possible total point scores was 1–32.5.

To obtain a risk score for a subject, point values were summed for all mortality-related factors that individual had on his/her baseline MDS assessment. A value of ‘0’ was assigned for factors a subject did not have on that assessment. For example, a 76 year-old (3.0 points) male (3.3 points) resident who was a recent admission to the NH (3.3 points), totally functionally dependent (2.1), and experienced recent weight loss (1.6 points), would have a total of 13.3 points (3.0 + 3.3 + 3.3 + 2.1 + 1.6). Table 4 shows the number and proportion of subjects who had all possible total point scores and the observed six- and 12-month mortality rates for each score. Using our example, the patient with a total score of 13.3 had a 34% chance of dying within six months and a 52% chance of dying within 12 months.

Table 4.

No. (%) of Subjects With Each Possible Total Risk Score and the 6- and 12-Month Probabilities of Death With Each Total Score (n=218,088)

Total Risk Score	No. (%) of Subjects With Each Score	Observed Probability of Death Within
		6 Months	12 Months
1 (minimum score)	84 (0.04)	0.01	0.06
> 1–2	236 (0.11)	0.04	0.08
> 2–3	1,232 (0.56)	0.05	0.11
> 3–4	2,609 (1.20)	0.06	0.13
> 4–5	5,859 (2.69)	0.06	0.15
> 6–7	14,700 (6.74)	0.10	0.23
> 7–8	18,439 (8.45)	0.12	0.26
> 8–9	21,634 (9.92)	0.15	0.30
> 9–10	23,036 (10.56)	0.17	0.33
> 10–11	22,509 (10.32)	0.21	0.37
> 11–12	20,938 (9.60)	0.25	0.42
> 12–13	18,632 (8.54)	0.29	0.47
> 13–14	15,038 (6.90)	0.34	0.52
> 14–15	111,691 (5.36)	0.40	0.57
> 15–16	9,512 (4.36)	0.46	0.62
> 16–17	6,721 (3.08)	0.52	0.67
> 17–18	4,955 (2.27)	0.57	0.71
> 18–19	3,585 (1.64)	0.64	0.76
> 19–20	2,547 (1.17)	0.67	0.79
> 20–21	1,777 (0.81)	0.73	0.84
> 21–22	1,154 (0.53)	0.77	0.87
> 22–23	648 (0.30)	0.83	0.90
> 23–24	385 (0.18)	0.83	0.91
> 24–25	188 (0.09)	0.88	0.94
> 25–26	99 (0.05)	0.88	0.96
> 26–27	58 (0.03)	0.83	0.90
> 27–28	21 (0.01)	0.95	1.00
> 28–32	17 (< 0.01)	1.00	1.00

Figure 1 presents the receiver operating characteristic curves for the ADEPT risk score’s prediction of six-month and 12-month survival. The AUROCs were 0.73 and 0.70, for six-month and 12-month survival, respectively. Figure 2 presents the calibration curves for the risk score, where the 45° line represents perfect calibration. These curves show that the amount of error, the difference between the predicted values and the corresponding optimism-corrected calibrated values, was small.

Figure 1.

Receiver operating characteristic (ROC) curves for the risk score’s prediction of 6-month (Panel A) and 12-month (Panel B) survival among nursing home residents with advanced dementia (n=218,088). The 45° line represents chance. The area under the ROC curve is 0.73 for 6-month survival and 0.70 for 12-month survival.

Figure 2.

Calibration curves for the risk score’s prediction of 6-month (Panel A) and 12-month (Panel B) survival among nursing home residents with advanced dementia (n=218,088). Circles represent apparent calibration accuracy; X’s are estimates corrected for optimism. The 45° line represents perfect calibration.

Hospice Eligibility Guidelines and Accuracy

Among eligible subjects who resided in the NH at least 12 months (i.e., reason for baseline MDS assessment was an annual assessment vs. NH admission) (N=142011), 41.7% met the MDS-simulated definition of FAST stage 7c. The proportions of subjects experiencing each of the pre-existing conditions specified in hospice guidelines during the prior 12 months were: aspiration pneumonia, 0.7%; urinary tract infection, 24.8%; septicemia, 1.1%; fever, 6.3%, multiple decubitus ulcers ≥ stage 3, 2.9%; and insufficient oral intake (or tube-feeding with weight loss), 15.8%. A total of 38.7% (n=54,933) subjects had at least one of these conditions. Taken together, 15.9% (n=22,542) of subjects were both at FAST stage 7c and had at least one pre-existing condition and, therefore, met the full MDS-simulated guidelines for hospice eligibility.

The AUROC for hospice eligibility was 0.53 for six-month survival. A cut-off score of 12.1 on the ADEPT risk score achieved the same specificity as that of the simulated hospice guidelines (0.85). The AUROC for the ADEPT to predict six-month survival using a cut-off of 12.1 was 0.65 for the entire cohort and 0.59 for residents who resided in the NH at least 12 months (n=14,2011).

Discussion

In this study, clinical data from the MDS were used to derive and retrospectively validate a mortality risk score in a nationwide sample of NH residents with advanced dementia. The ADEPT score predicted survival with moderate accuracy and had greater discrimination in estimating six-month mortality compared to hospice eligibility guidelines simulated with MDS data.

Our study furthers prior efforts (2, 5, 7, 19, 40) examining factors associated with survival in advanced dementia by utilizing a nationwide sample of both long and short stay NH residents, extending follow-up to 12 months, and applying rigorous statistical methods to develop a mortality risk score. Although methodological differences limit comparisons with prior studies, characteristics most consistently reported to be associated with increased mortality in advanced dementia include: worse functional status (2, 5, 7), reduced oral intake (5,19, 40), and older age (2, 5, 7). Only four variables differentiated the 12-item risk scores derived in this study versus our earlier effort that examined only newly admitted NH residents with advanced dementia (5). The prior model included oxygen use, unstable medical conditions, cancer, and not being awake most of the day (5), whereas the ADEPT instrument did not. Oxygen use was not considered as a predictor in the current study as we purposefully excluded treatments. The MDS variable “unstable medical condition” was also excluded as a predictor because it was vaguely defined and hard to operationalize in practice. The lack of cancer in the ADEPT score may reflect the possibility that it is more strongly associated with survival in only short-stay (vs. long-stay) residents, as suggested by MDS-based mortality scores derived for the general NH population (30, 41). In place of these four variables, the ADEPT score included: recent admission to the NH, body mass index, recent weight loss, and pressure ulcers. The inclusion of admission status, which was strongly associated with survival, allows the flexibility of using this tool in both recently admitted and long-stay residents. Prior studies have also shown that recent weight loss (19, 28, 42) and pressure ulcers (40) are associated with survival in end-stage dementia. Despite these differences, the accuracy of the ADEPT and our prior risk score (AUROC was 0.74 and 0.70 in derivation and validation cohort, respectively) were comparable (5). However, the methodological advancements used to develop the ADEPT tool (national sample, short and long stay, 12-month follow-up) have resulted in an instrument with greater generalizability and applicability.

The accuracy of hospice guidelines for dementia, as simulated with MDS-data, to predict six-month survival was only slightly better than chance (AUROC was 0.53), corroborating prior research demonstrating the poor predictive ability of these guidelines (4, 5, 7, 19). The discriminatory power of the full hospice guidelines examined in this study was only marginally higher than the FAST stage 7c alone as reported in our prior work (AUROC 0.51) (5). We found the prevalence of several of the medical conditions included in hospice guidelines to be very low, which may be attributed, in part, to limitations of ascertainment of these conditions using MDS data. Among the medical conditions also considered as predictor variables in our model, only poor oral intake and pressure ulcers were included in the final risk score.

This study has several limitations that deserve comment. First, the ADEPT risk score was developed using MDS data. Thus, it is possible that important factors associated with survival in advanced dementia are not adequately captured in MDS assessments. Similarly, the poor predictive ability we observed for hospice guidelines may be attributed, in part, to inaccuracies in using MDS data to simulate these guidelines. Finally, the assessment of the accuracy of the ADEPT tool in this report was limited to retrospective analyses of secondary data, thus its validity and feasibility as a bedside tool remains unknown. The prospective validation of the ADEPT tool and hospice eligibility guidelines using primary data (i.e., not using MDS variables) that is currently in progress will address several of these limitations.

Despite our rigorous efforts, the ADEPT score, while better than hospice guidelines, has only moderate accuracy in predicting survival in advanced dementia. While methodological issues may partly explain our limited success, it is important to consider the possibility that more accurate estimation of life expectancy in advanced dementia, like other chronic conditions (43), is simply not possible. If true, this possibility seriously calls into question the validity of using prognosis to determine whether or not persons with advance dementia should receive hospice or palliative care services. In the meanwhile, while hospice guidelines continue to require an estimated life expectancy of less than 6 months, the ongoing prospective validation of the ADEPT risk score will further inform how well this can be practically achieved at the bedside of NH residents with advanced dementia.