Abstract
Background/Objectives: Identifying patients with limited life expectancy (LLE) in nursing homes (NH) is important to aid in appropriate care planning. The Minimum Data Set (MDS) Mortality Risk Index – Revised (MMRI-R) is a validated scale using MDS v2.0 assessment items to generate scores predicting 6-month mortality in NH residents. In transitioning from MDS v2.0 to v3.0, several items were changed or removed, putting into question translatability of this measure across MDS versions. Objectives were to: 1) evaluate the predictive validity of an adapted version of the MMRI-R based on MDS v3.0 assessment items (MMRI-v3) and 2) compare the predictive validity of the MMRI-v3 to a single MDS item indicating LLE.
Design:
Retrospective, cross-sectional study of MDS assessments. Other data sources included the VA Residential History File and VA Vital Status File.
Setting:
VA NHs
Participants:
Older adult Veterans ≥ 65 years old newly admitted to VA NHs between 07/01/2012-09/30/2015
Measurements:
The dependent variable was death within six months of admission date. Independent variables included MDS items used to calculate MMRI-v3 scores (renal failure, chronic heart failure, sex, age, dehydration, cancer, unintentional weight loss, shortness of breath, activities of daily living scale, poor appetite, acute change in mental status) and the MDS item indicating LLE.
Results:
The MMRI-v3 has good predictive ability for 6-month mortality compared to the original MMRI-R (c-statistic=0.81 vs 0.76). Scores generated by the MMRI-v3 showed greater predictive ability than the single MDS indicator for LLE (c-statistic=0.76), but using the two together resulted in further increases in predictive ability (c-statistic=0.86).
Conclusions:
The MMRI-v3 is a useful tool in research and clinical practice that accurately predicts 6-month mortality in Veterans residing in VA NHs. Identification of residents with LLE has great utility for studying palliative care interventions and may be helpful in guiding allocation of these services in clinical practice.
Keywords: Nursing Home, Mortality, Prognostic Index, MDS
INTRODUCTION
Accurate characterization of prognosis is an important component of patient assessment in nursing home (NH) residents.1,2 Tools that objectively identify residents with limited life expectancy (LLE) can be used to facilitate decisions that optimize end of life care by evaluating interventions to determine which residents may benefit most from palliative and hospice care.3,4
Several prognostic indices for mortality prediction have been developed using clinical characteristics, diagnoses, and demographics,5 but few are validated for use in NH populations.6–11 The Minimum Data Set (MDS) Mortality Risk Index - Revised (MMRI-R)9,10 uses items from MDS 2.0 assessments to predict likelihood of 6-month mortality in NH residents. It consists of a weighted sum, ranging from 0–85, using 11 MDS items and the 4-item Activities of Daily Living (ADL) short-form, with higher scores indicating greater probability of 6-month mortality. While other measures have demonstrated comparable or superior prediction accuracy to the MMRI-R12,13, they are either derived from the earliest version of the MDS6–8 or rely on claims data11. One of the main advantages of the MMRI-R is that it is based solely on assessment items from the MDS and thus is simple to compute from routinely collected data. Recently published similar measures have established acceptable validity for 30- and 60-day mortality13,14, which are clinically useful, but may have less utility for palliative care and hospice planning since patients qualify as early as 6-months before death. Accurate identification of qualifying residents and initiation of these services in a more timely manner may improve the overall quality of end of life care.
The transition from MDS version 2.0 to 3.0 created a challenge for continued use of the MMRI-R, as several items were revised or removed. (Supplementary Table S1.) Items assessing recent weight loss, shortness of breath, and ADLs have slightly revised response schemes in MDS 3.0, although these are easily reconcilable. Items evaluating poor appetite (K4C) and deteriorated cognition (B6) are more difficult to reconcile. Candidate replacement items exist in MDS 3.0 that evaluate similar constructs, but differences in phrasing and response categories may alter their meaning and association with mortality.
Both versions of the MDS also include a single question asking whether the physician believes the resident has a life expectancy of <6 months. Prior investigations have identified that this indicator accurately identifies residents who die within 6 months, but is rarely marked positive in MDS records and fails to identify ~20% who die within 6 months.15,16 However, it is unknown how the accuracy of this item compares to the MMRI-R and whether using both together would improve prediction.
The objectives of this study were to: 1) evaluate the predictive validity of an adapted version of the MMRI-R using revised assessment items in the MDS v3.0 (subsequently referred to as MMRI-v3) with regard to 6-month mortality and 2) compare the predictive validity of the MMRI-v3 to the single MDS item for LLE.
METHODS
Design and Data Sources
This was a retrospective, cross-sectional study of MDS admission assessments for a national sample of 63,024 older adult Veterans residing in VA NHs, known as Community Living Centers (CLCs). The VA Pittsburgh Healthcare System’s Institutional Review Board approved this study.
Data sources included the VA Residential History File (RHF) 17,18, MDS 3.0 assessment records for all Veterans ≥ 65 years old admitted to VA NHs, and the VA Vital Status File (VSF). Data sources were linked using scrambled social security numbers. The VA RHF tracks transitions across healthcare settings by linking VA utilization data, Medicare claims for Veterans dually enrolled in VA and Medicare, and MDS data,18 and was used to identify admission and discharge dates. The MDS is a comprehensive evaluation containing hundreds of clinical variables that evaluate the health of NH residents. It is required by the Centers for Medicare and Medicaid Services (CMS) for all residents in CMS-certified NHs and is mandated in VA CLCs. The MDS is administered within 14 days of admission and quarterly thereafter or in the event of an acute status-change. The study start date (July 1, 2012) aligns with the VA transition to from the MDS 2.0 to 3.0. The VSF provides date of death for all Veterans and has good agreement with state databases.19
Cohort Construction
The cohort used in this analysis included new VA NH admissions from October 1, 2008-September 30, 2015 (n=200,497), identified using the VA RHF (Supplementary Figure S1). We first limited the sample to admissions with a full MDS v3.0 admission assessment, after VA implementation of version 3.0 on July 1, 2012 (n=92,713, 46.2%). New admissions were required to have a corresponding admission assessment within the first 30 days of each episode.20 We then used date of birth to limit the cohort to episodes in which the Veteran was ≥65 years old, resulting in a final cohort of 63,024 admissions. Veterans may appear in the cohort more than once, if admitted multiple times during this period.
Dependent Variable
The dependent variable was whether death occurred within six months of the admission date, using date of death from the VSF.
Independent Variables
MMRI-v3 elements.
Independent variables included MDS items from the MMRI-R as specified by Porock et al.10 if available, or candidate replacement items. Members of the study team reviewed MDS assessments and codebooks to identify MDS 3.0 items that were equivalent or could be coded to be equivalent to the original MDS 2.0 items to create the MMRI-v3 (Supplementary Table S1). This review suggested 10 of 12 variables could be created using MDS 3.0 items with minor modification. Minor modifications included: 1) coding the 3.0 weight loss item as “yes” for non-physician-prescribed weight loss (K0300=2) and “no” for no weight loss (K0300=0) or physician-prescribed regimen (K0300=1), whereas the 2.0 item did not differentiate between prescribed and non-prescribed weight loss; 2) coding residents as “yes” for shortness of breath if any of three items in 3.0 assessing shortness of breath in different situations were endorsed, to match the single 2.0 item; and 3) coding ADL item responses of “7” (occurred once or twice) or “8” (never occurred) to “4” (total dependence).21
Two MDS 2.0 items used in the MMRI-R were not available in MDS 3.0: poor appetite (K4C) and deteriorated cognitive skills or status in past 3 months (B6). However, two items were identified as possible replacements. For poor appetite, item D0200E1, part of the PHQ-9 depressive symptoms measure22, assesses presence of “poor appetite or overeating” in the past 2 weeks. For deteriorated cognitive skills, item C1600 in the delirium assessment asks, “Is there evidence of an acute change in mental status from the resident’s baseline?” This item is similar to v2.0 item B6 (“Resident’s cognitive status, skills, or abilities have changed as compared to status of 90 days ago”), but the assessor cannot differentiate between deterioration or improvement in v3.0.
MDS LLE Indicator.
We also evaluated the predictive ability of the single item indicating LLE (J1400), which asks “Does the resident have a condition or chronic disease that may result in a life expectancy of less than 6 months?”.
Socio-demographics.
In addition to age, we captured sex and race/ethnicity from the MDS.
Statistical Analysis
Analyses were conducted using Stata v15.0. Some items in the MMRI-v3 had a small amount of missing data (≤5% for any given variable). We used chained equations to impute a single complete dataset using the “mi impute chained” procedure, including all individual MMRI-v3 items and race/ethnicity in the imputation.23 Descriptive statistics (n, %) were computed for all variables before and after imputation.
After generating a dataset with no missing values, total MMRI-v3 scores were computed by applying weights specified by Porock et al. (Supplementary Table S2.) to each equivalent item or combination.10 We used logistic regression with the total MMRI-v3 score as the independent variable and 6-month mortality as the dependent variable to generate a Receiver Operating Characteristic (ROC) curve and corresponding c-statistic for the Area under the Curve. The RoC curve is a depiction of the balance between sensitivity and specificity for a prediction instrument (i.e. MMRI-v3) versus a gold-standard (i.e. actual deaths). Given an individual who died within 6 months and one who did not, the c-statistic represents the proportion of times the model will assign a higher probability to the individual who died. We also computed frequency of MMRI-v3 scores and the proportion of Veterans dying within 6 months across 5-point intervals.
To examine the utility of different MMRI-v3 scores for identifying LLE, we calculated sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV) for several cut-points. An optimal decision-point was identified by comparing 10-fold cross-validated c-statistics for a range of individual MMRI-v3 scores.
We also calculated sensitivity, specificity, PPV and NPV of the single MDS indicator of LLE for 6-month mortality. To compare its predictive validity with the MMRI-v3, we generated C-statistics for a model containing the LLE indicator as the sole predictor, and then a model containing both the LLE indicator and the MMRI-v3 score. We also compared the predictive validity of the LLE indicator against the MMRI-v3 optimal decision-point and evaluated agreement with a kappa statistic.
Finally, we conducted several sensitivity analyses. First, to evaluate whether the original weights published by Porock et al. were valid for this cohort, we compared model fit using regression coefficients derived from our cohort versus the originally published weights. The sample was randomly divided into developmental (50%) and validation (50%) samples. Logistic regression was used in the developmental sample with individual MDS items as the independent variables and 6-month mortality as the dependent variable. Regression coefficients for items were extracted, scaled, and applied to the validation sample to evaluate prediction using logistic regression and a c-statistic. Optimism-corrected c-statistics were also calculated using bootstrapping to account for the potential for bias due to overfitting.24 We also evaluated the predictive validity of the MMRI-v3 at alternative mortality time points (30-day, 60-day, and 1-year mortality).
RESULTS
Descriptive Statistics
Summary statistics for demographics and MMRI-v3 items before and after imputation are presented in Table 1. Of the 63,024 new admissions, most were male (97.7%) and white, non-Hispanic (79.3%) Veterans, the majority over 75 years old. Median length of stay of was 28 days, and 6-month mortality rate was 32.9%. The MMRI-v3 scores ranged from 10–78 (mean=33.4, median=32, IQR=25–41, SD=11.1). Table 2 shows the distribution of MMRI-v3 scores and the proportion of observed deaths, compared to those reported by Porock et al.
Table 1.
Sample Characteristics (N=63,024)
Demographics (no values imputed) | n(%) | |
---|---|---|
Age | ||
65–74 | 27,972 (44.4) | |
75–84 | 19,191 (30.4) | |
85+ | 15,861 (25.2) | |
Sex | ||
Male | 61,587 (97.7) | |
Female | 1,437 (2.3) | |
Race | ||
White | 49,944 (79.3) | |
Black | 9,344 (14.8) | |
Hispanic | 2,535 (4.0) | |
Other | 1,201 (1.9) | |
LoS (med [IQR]) | 28 [14–62] | |
Time until Death (med [IQR]) | 160 [35–511] | |
Died within 6 months | 20,751 (32.9) | |
Life expectancy < 6 months (Item J1400) | 13,590 (21.6) | |
MMRI-v3 Responses (missing values imputed) | n(%) Prior to Imputation | n(%) After Imputation |
Lost weight unintentionally in last 3 months | ||
Yes | 15,685 (24.9) | 16,468 (26.1) |
No | 44,585 (70.7) | 46,556 (73.9) |
Missing | 2,754 (4.4) | - |
Renal failure | ||
Yes | 7,378 (11.7) | 7,380 (11.7) |
No | 55,627 (88.2) | 55,644 (88.3) |
Missing | 19 (<0.1) | - |
Chronic heart failure | ||
Yes | 9,986 (15.8) | 9,996 (15.9) |
No | 53,006 (84.1) | 53,028 (84.1) |
Missing | 32 (<0.1) | - |
Poor Appetite or Overeating | ||
Yes | 14,731 (23.4) | 15,563 (24.7) |
No | 46,012 (73.0) | 47,461 (75.3) |
Missing | 2281 (3.6) | - |
Dehydrated | ||
Yes | 696 (1.1) | 698 (1.1) |
No | 62,328 (98.8) | 62,326 (98.9) |
Missing | 82 (0.1) | - |
Short of breath | ||
Yes | 16,340 (25.9) | 16,345 (25.9) |
No | 46,615 (74.0) | 46,679 (74.1) |
Missing | 69 (0.1) | - |
Cancer | ||
Yes | 14,419 (22.9) | 14,432 (22.9) |
No | 48,561 (77.0) | 48,592 (77.1) |
Missing | 44 (<0.1) | - |
Acute Change in Mental Status | ||
Yes | 2,751 (4.4) | 3,077 (4.9) |
No | 57,021 (90.4) | 59,947 (95.1) |
Missing | 3252 (5.2) | - |
ADL Score (med [IQR]) | 7 [4–11] | 7 [4–11] |
Missing | 126 (0.2) | - |
Table 2.
Distribution of MMRI-v3 Scores and Deaths
MMRI Points | % (Porock et al.) | % (VA NH Cohort) | % died (Porock et al.) | % died* (VA NH Cohort) |
---|---|---|---|---|
Overall | N=43,311 | N=63,024 | 23 | 33 |
0–5 | 1.4 | 0.0 | 4 | 0.0 |
6–10 | 6.9 | 0.0 | 4 | 3.9 |
11–15 | 12.2 | 2.2 | 7 | 4.1 |
16–20 | 17.8 | 9.3 | 11 | 5.5 |
21–25 | 20.3 | 15.6 | 17 | 9.6 |
26–30 | 16.6 | 17.6 | 27 | 16.8 |
31–35 | 10.9 | 16.8 | 36 | 28.3 |
36–40 | 6.7 | 13.3 | 47 | 43.0 |
41–45 | 3.8 | 10.1 | 58 | 57.2 |
46–50 | 1.9 | 6.9 | 69 | 69.4 |
51–55 | 0.9 | 4.5 | 79 | 79.0 |
56–60 | 0.4 | 2.3 | 89 | 85.9 |
61–65 | 0.2 | 1.0 | 90 | 93.7 |
66–70 | 0.1 | 0.4 | 93 | 94.2 |
71–75 | 0 | 0.1 | 100 | 91.1* |
Interpretation: Among all patients with a score between 71–75, 91.1% died.
Predictive Ability of the MMRI-v3 for 6-month Mortality
The logistic regression model with the continuous MMRI-v3 score as the sole predictor for 6-month mortality showed good prediction with a c-statistic of 0.81. Sensitivity, specificity, PPV, NPV, and c-statistic for various cut-points of the MMRI-v3 score are found in Table 3. A score of ≥35 was the cut-point producing the highest c-statistic (0.74). Figure 1 shows the proportion of observed deaths in the sample versus the predicted mortality probability for each score. The MMRI-v3 shows excellent agreement, except for a few values at the upper extreme.
Table 3.
MMRI-v3 Predictive Statistics
MMRI-v3 Cut Point | Sensitivity | Specificity | PPV | NPV | C-statistic+ |
---|---|---|---|---|---|
11 | 100.0 | 0.1 | 32.9 | 100.0 | 0.495 |
16 | 99.7 | 3.3 | 33.6 | 95.8 | 0.510 |
21 | 98.2 | 16.5 | 36.6 | 94.8 | 0.567 |
26 | 93.6 | 37.4 | 42.3 | 92.3 | 0.650 |
31 | 84.7 | 59.1 | 50.4 | 88.7 | 0.715 |
32 | 82.1 | 63.4 | 52.4 | 87.8 | 0.724 |
33 | 79.5 | 67.2 | 54.3 | 86.9 | 0.730 |
34 | 76.8 | 70.9 | 56.5 | 86.2 | 0.735 |
35* | 73.7 | 74.2 | 58.4 | 85.2 | 0.737 |
36 | 70.5 | 77.1 | 60.1 | 84.2 | 0.735 |
37 | 67.0 | 79.7 | 61.9 | 83.1 | 0.731 |
38 | 63.6 | 82.3 | 63.8 | 82.2 | 0.726 |
39 | 60.0 | 84.5 | 65.6 | 81.2 | 0.719 |
40 | 56.5 | 86.6 | 67.5 | 80.2 | 0.712 |
41 | 53.2 | 88.4 | 69.3 | 79.4 | 0.703 |
46 | 35.7 | 94.9 | 77.3 | 75.0 | 0.648 |
51 | 21.0 | 98.0 | 83.6 | 71.6 | 0.591 |
56 | 10.3 | 99.4 | 89.2 | 69.3 | 0.544 |
61 | 4.1 | 99.9 | 93.5 | 68.0 | 0.515 |
66 | 1.3 | 100.0 | 94.2 | 67.3 | 0.501 |
71 | 0.2 | 100.0 | 93.9 | 67.1 | 0.496 |
C-statistics reported represent the average from 10-fold cross-validation
MMRI-v3 = 35 is score at which c-statistic reached maximum value
Figure 1.
Observed and Model Predicted Mortality at 6 months
MDS LLE Indicator
The single-item LLE indicator was endorsed for <6 months of life expectancy in 21.4% of admissions. There was moderate agreement with MMRI-v3 score ≥35 (kappa=0.41). Using the LLE indicator as the sole predictor of 6-month mortality resulted in a c-statistic of 0.75, with sensitivity=53.2%, specificity=88.4%, PPV=79.4%, and NPV=79.4%. Using the LLE indicator and MMRI-v3 scores together yielded a c-statistic=0.85.
Sensitivity Analyses
In the first sensitivity analysis, regression coefficients generated from our cohort were similar in magnitude and direction to those published by Porock et al. (Table S3). Application of cohort-specific regression weights to the sample resulted in similar accuracy to the primary analysis, (c-statistic = 0.82) with little evidence of bias due to over fitting (mean optimism <0.001). This suggests that the original MMRI-R weights are also valid in this population. In our second analysis, the MMRI-v3 maintained good prediction for 30-day (c-statistic=0.85), 60 day (c-statistic = 0.80), and 1-year (c=statistic = 0.79) mortality, with slight variation in optimal cut-points for each time-frame (Supplementary Table S4).
DISCUSSION
In a national sample of Veterans residing in VA NHs, we found that the MMRI-v3, which adapts the MMRI-R to MDS 3.0 assessments, had good prediction for 6-month mortality. Use of two substitute items for poor appetite and recent deterioration in cognition that are no longer available in MDS 3.0 did not compromise prediction for six-month mortality; in fact, our c-statistic (0.81) compares favorably to the MMRI-R (0.76).10 Our results also showed that MMRI-v3 scores had greater predictive ability for 6-month mortality compared to the single MDS LLE indicator (c-statistic=0.76), but that using both together resulted in increased predictive ability (c-statistic=0.86).
There are a few potential explanations for the improved prediction in the revised model. First, it is possible that MDS 3.0 items with minor changes and/or the substantively revised items had reduced measurement error, and thus stronger associations with mortality than previously. Second, improvement in accuracy may also be attributable differences in admission type and mortality rate across cohorts. This sample included only new admissions for primarily male Veterans, while Porock et al. included non-Veteran NH residents of both sexes who represented a mix of new admissions and those residing in the NH for one or more years.10 The more homogeneous cohort in our study may be inflating model prediction. It is also worth noting that in recent years, there has been a shift in the types of patients admitted to VA CLCs, with a shift in focus from long-term to short-term care and an expansion of hospice care by 12% between 2004 to 2011.25 Therefore, it is possible that changes in the proportion of hospice stays in CLCs may have also improved model prediction as compared to older studies. Future research testing the predictive validity of the MMRI-v3 in a nationally representative sample of both newly admitted and long-stay non-VA NH residents would provide insights into whether improved prediction is due to sample characteristics or revised items.
The results of this study, specifically Table 3 regarding prediction at various MMRI-v3 cut points, can help guide clinicians and researchers in identifying residents with a high risk of death within 6 months. For example, in research a cut-point may be useful to establish cohort inclusion if the goal is to identify and study residents at end of life. In practice, clinicians may choose to use a cut-point as a threshold for appropriateness of palliative care consultations or hospice enrollment if resources are limited. We found that a cut-point of ≥35 may maximize model fit, with balanced sensitivity and specificity of 73.7% and 74.2%, respectively. However, alternate cut points may be more appropriate depending on the intended use and the amount of acceptable error.
We also found that the single MDS indicator of LLE had lower, but relatively good, accuracy as a sole predictor for 6-month mortality. Therefore, this may be a more efficient method for characterizing mortality risk at the population level, recognizing that its accuracy is influenced by the proportion of hospice residents, since it is always marked positive for these residents. The LLE item improved 6-month mortality prediction when used with the MMRI-v3, suggesting that they may be identifying different residents. This suggests that physicians’ subjective judgment of prognosis offers additional information beyond objective indicators of health status. Researchers using the MDS wishing to control for variation in NH residents’ mortality risk or to identify residents with the highest likelihood of death may want to utilize both the MMRI-v3 and the single MDS indicator for LLE. Future work should establish validity of the LLE item in non-VA populations.
There are several limitations to address. Generalizability to other NH populations, particularly females, non-Veterans, and long-stay residents, is unknown. Compared to non-VA NH residents, a larger proportion, approximately two-thirds, of new VA NH admissions are short-stays for post-acute care or rehabilitation for an acute illness following hospitalization.26,27 These residents are more likely to be re-hospitalized.27 Long-stay residents, by contrast, are often admitted in the setting of disability for assistance with daily activities.26,27 Veterans also tend to have higher comorbidity burden.28,29 Thus, it is possible that the accuracy of the MMRI-v3 does not hold true for non-Veteran NH residents. However, sensitivity analyses revealed that the original scoring by Porock et al. is valid across populations, as the use of cohort-specific scores did not substantially affect prediction. Nevertheless, a future goal is to replicate this analysis in Medicare NH residents to further evaluate prediction ability. Missing data requiring imputation is another limitation, but given the small frequency of missing observations (<=5%) it is unlikely that this substantially influenced results.30 Finally, the relationship between objective characteristics from administrative data and mortality is complex. It is possible that these MDS items do not actually represent the optimal combination of variables to predict mortality, especially considering new and updated versions of MDS items. Two recent studies created new models to predict short-term13,14 (30-day and 60-day) and/or long-term14 (1-year) mortality in NH residents using MDS v3.0 items, but ultimately resulted in slightly inferior long-term prediction ability14 or used a larger number of items13 compared to our model. Further optimization of mortality-risk models may require approaches more advanced than standard regression techniques. The MMRI-v3 still achieves good accuracy that can be replicated across different populations without using advanced methods. Nevertheless, future investigations should determine whether advanced methods can better describe these relationships.
CONCLUSIONS/RELEVANCE
The MMRI-v3, an adaptation of the MMRI-R to MDS v3.0 assessment items, is a useful tool in research and clinical practice that accurately predicts 6month mortality in Veterans residing in VA NHs. In addition, we found that a physician-endorsed indicator for LLE has good accuracy for 6-month mortality and improves prediction when added to the MMRI-v3. Identifying LLE has great utility for studying palliative care interventions and the general quality of care provided for NH residents at end of life. Utilization of the MMRI-v3 in practice may also be helpful in guiding allocation of hospice services and palliative interventions to optimize care. Future research should evaluate whether advanced model building techniques can be used to develop more parsimonious models with comparable or improved accuracy.
Supplementary Material
Figure S1. Cohort Flow Diagram
Table S1. MMRI-R Items in MDS v2.0 and v3.0
Table S2. MMRI-v3 Scoring Algorithm
Table S3. Sensitivity Analysis with Cohort-specific Regression Weights
Table S4. MMRI-v3 Prediction Statistics for Alternate End-points.
IMPACT STATEMENT:
We certify that the research presented in this manuscript is novel and has high potential to impact clinical care for older adults from both a clinical perspective and a research perspective. Our analysis demonstrates the validity of the Minimum Data Set Mortality Risk Index for predicting 6-month mortality in older adult Veteran nursing home residents when adapted to items from the Minimum Data Set (MDS) version 3.0 (MMRI-v3). The original version of this mortality risk index was developed using MDS version 2.0, but this is the first study to establish the validity of the scale for 6-month mortality when adapted to MDS version 3.0 items. We also investigated the predictive validity of the single item indicator of limited life expectancy within the MDS and compared this to our mortality risk index. Few studies have utilized this item to predict death, and to date, there has been no extensive validation of its predictive validity nor any comparison to other mortality measures. Our study demonstrates that the MMRI-v3 has good predictive validity, justifying its continued use for identifying nursing home residents with high mortality risk. We also demonstrate that the MDS item for limited life expectancy also has good predictive validity, although with a different level of sensitivity and specificity. Finally, no studies have attempted to validate mortality risk measures in Veteran nursing home populations, making this study the first to demonstrate the validity in a non-Medicare nursing home population.
To summarize, our research examines the validity of different methods for identifying residents with high-risk for mortality that until now have not been studied in MDS version 3.0. The MMRI-v3 has value for clinicians in that it provides an objective means to accurately determine which residents may benefit most from hospice or palliative care services. It also provides researchers with a systematic means for identifying these residents to study the risks and benefits of interventions at the end of life. By contrast, the MDS item for limited life expectancy may provide a more efficient potential alternative for identifying residents with high risk for mortality.
ACKNOWLEDGEMENTS
The authors would also like to acknowledge the VA Geriatrics and Extended Care Data Analysis Center (GECDAC) for providing critical feedback for the duration of the project and for providing the VA Residential History File through a data sharing mechanism from operations to research.
FUNDING
Funding was provided by: Department of Veterans Affairs (VA IIR 14-306 HSR&D; PI: C. Thorpe)
Dr. Niznik was funded by a T32 Award from the National Institutes on Aging (T32AG021885)
Dr. Springer was funded by a Postdoctoral Fellowship through the Veterans Administration Office of Academic Affairs
Sponsor’s Role: The U.S. Department of Veterans affairs had no role in the study design, data collection and analysis, manuscript preparation or the decision to submit the manuscript for publication. The views expressed in this paper are those of the authors, and no official endorsement by the Department of Veterans Affairs or the United States Government is intended or should be inferred.
This paper will be presented at: the 2018 American Geriatrics Society Annual Meeting as an oral presentation and the 2018 AcademyHealth Annual Research Meeting as a poster presentation.
Footnotes
Conflict of Interest: The authors have no conflicts of interest to disclose.
Separate files:
Supplementary Materials – MDS Item Changes, Cohort Flow, MMRI Scoring Algorithm, and Sensitivity Analyses
REFERENCES
- 1.Tazkarji B, Lam R, Lee S, et al. Approach to preventive care in the elderly. Can Fam Physician. 2016;62(9):717–721. [PMC free article] [PubMed] [Google Scholar]
- 2.American Geriatrics Society Expert Panel on the Care of Older Adults with Multimorbidity. Guiding principles for the care of older adults with multimorbidity: an approach for clinicians. J Am Geriatr Soc. 2012;60(10):E1–E25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Walter LC, Covinsky KE. Cancer screening in elderly patients: a framework for individualized decision making. JAMA. 2001;285(21):2750–2756. [DOI] [PubMed] [Google Scholar]
- 4.Lee SJ, Leipzig RM, Walter LC. Incorporating lag time to benefit into prevention decisions for older adults. JAMA. 2013;310(24):2609–2610. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Yourman LC, Lee SJ, Schonberg MA, et al. Prognostic indices for older adults: a systematic review. JAMA. 2012;307(2):182–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mitchell SL, Kiely DK, Hamel MB, et al. Estimating prognosis for nursing home residents with advanced dementia. JAMA. 2004;291(22):2734–2740. [DOI] [PubMed] [Google Scholar]
- 7.Flacker JM, Kiely DK. A practical approach to identifying mortality-related factors in established long-term care residents. J Am Geriatr Soc. 1998;46(8):1012–1015. [DOI] [PubMed] [Google Scholar]
- 8.Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc. 2003;51(2):213–221. [DOI] [PubMed] [Google Scholar]
- 9.Porock D, Oliver DP, Zweig S, et al. Predicting death in the nursing home: development and validation of the 6-month Minimum Data Set mortality risk index. J Gerontol A Biol Sci Med Sci. 2005;60(4):491–498. [DOI] [PubMed] [Google Scholar]
- 10.Porock D, Parker-Oliver D, Petroski GF, et al. The MDS Mortality Risk Index: The evolution of a method for predicting 6-month mortality in nursing home residents. BMC Res Notes. 2010;3:200. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Buntinx F, Niclaes L, Suetens C, et al. Evaluation of Charlson’s comorbidity index in elderly living in nursing homes. J Clin Epidemiol. 2002;55(11):1144–1147. [DOI] [PubMed] [Google Scholar]
- 12.Kruse RL, Parker Oliver D, Mehr DR, et al. Using mortality risk scores for long-term prognosis of nursing home residents: caution is recommended. J Gerontol A Biol Sci Med Sci. 2010;65(11):1235–1241. [DOI] [PubMed] [Google Scholar]
- 13.Thomas KS, Ogarek JA, Teno JM, et al. Development and Validation of the Nursing Home Minimum Data Set 3.0 Mortality Risk Score (MRS3). J Gerontol A Biol Sci Med Sci. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ogarek JA, McCreedy EM, Thomas KS, et al. Minimum Data Set Changes in Health, End-Stage Disease and Symptoms and Signs Scale: A Revised Measure to Predict Mortality in Nursing Home Residents. J Am Geriatr Soc. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Parker-Oliver D, Porock D, Zweig S, et al. Hospice and nonhospice nursing home residents. J Palliat Med. 2003;6(1):69–75. [DOI] [PubMed] [Google Scholar]
- 16.Porock D, Oliver DP, Zweig SC, et al. A profile of residents admitted to long-term care facilities for end-of-life care. J Am Med Dir Assoc. 2003;4(1):16–22. [DOI] [PubMed] [Google Scholar]
- 17.Intrator O, Hiris J, Berg K, et al. The residential history file: studying nursing home residents’ long-term care histories(*). Health Serv Res. 2011;46(1 Pt 1):120–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Intrator O, Makinei R, Cai S, et al. The Veterans’ Health Administration Residential history File: A resource for Research and Operations. VA HSR&D Research Meeting; July 2015; Philadelphia, PA. [Google Scholar]
- 19.Dominitz JA, Maynard C, Boyko EJ. Assessment of vital status in Department of Veterans Affairs national databases. comparison with state death certificates. Ann Epidemiol. 2001;11(5):286–291. [DOI] [PubMed] [Google Scholar]
- 20.Centers for Medicare & Medicaid Services. Long-Term Care Facility Resident Assessment Instrument 3.0 User’s Manual. Baltimore, Maryland: Centers for Medicare & Medicaid Services; 2017. Accessed February 15, 2018. [Google Scholar]
- 21.Morris JN, Fries BE, Morris SA. Scaling ADLs within the MDS. J Gerontol A Biol Sci Med Sci. 1999;54(11):M546–553. [DOI] [PubMed] [Google Scholar]
- 22.Saliba D, DiFilippo S, Edelen MO, et al. Testing the PHQ-9 interview and observational versions (PHQ-9 OV) for MDS 3.0. J Am Med Dir Assoc. 2012;13(7):618–625. [DOI] [PubMed] [Google Scholar]
- 23.StataCorp. Multiple-Imputation Reference Manual. Stata Press; 2017: https://www.stata.com/bookstore/multiple-imputation-reference-manual/. Accessed April 18, 2017.
- 24.Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1998. February 28; 15(4): 361–87. [DOI] [PubMed] [Google Scholar]
- 25.Thomas KS, Cote D, Makineni R, et al. Change in VA Community Living Centers 2004–2011: Shifting Long-Term Care to the Community. J Aging Soc Policy. 2018;30(2):93–108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wei YJ, Simoni-Wastila L, Zuckerman IH, et al. Algorithm for Identifying Nursing Home Days Using Medicare Claims and Minimum Data Set Assessment Data. Med Care. 2016;54(11):e73–e77. [DOI] [PubMed] [Google Scholar]
- 27.United States Government Accountability Office House Committee on Veterans’ Affairs. VA long-term care : oversight of nursing home program impeded by data gaps : report to the Chairman, Committee on Veterans’ Affairs, House of Representatives. Washington, D.C; 2004. [Google Scholar]
- 28.Agha Z, Lofgren RP, VanRuiswyk JV, et al. Are patients at Veterans Affairs medical centers sicker? A comparative analysis of health status and medical resource use. Arch Intern Med. 2000;160(21):3252–3257. [DOI] [PubMed] [Google Scholar]
- 29.Campbell DG, Felker BL, Liu CF, et al. Prevalence of depression-PTSD comorbidity: implications for clinical practice guidelines and primary care-based interventions. J Gen Intern Med. 2007;22(6):711–718. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Graham JW. Missing data analysis: making it work in the real world. Annu Rev Psychol. 2009;60:549–576. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Figure S1. Cohort Flow Diagram
Table S1. MMRI-R Items in MDS v2.0 and v3.0
Table S2. MMRI-v3 Scoring Algorithm
Table S3. Sensitivity Analysis with Cohort-specific Regression Weights
Table S4. MMRI-v3 Prediction Statistics for Alternate End-points.