Improving prediction of long-term care utilization through patient-reported measures: Cross-sectional analysis of high-need U.S. Veterans Affairs patients

Abstract

This paper examines the relative merit of augmenting an electronic health record (EHR)-derived predictive model of institutional long-term care (LTC) use with patient-reported measures not commonly found in EHRs. We used survey and administrative data from 3,478 high risk Veterans aged ≥65 in the U.S. Department of Veterans Affairs, comparing a model based on a VA geriatrics dashboard, a model with additional EHR-derived variables, and a model that added survey-based measures (i.e., activities of daily living (ADL) limitations, social support, and finances). Model performance was assessed via Akaike Information Criteria, C-statistics, sensitivity, and specificity. Age, a dementia diagnosis, Nosos risk score, social support, and ADL limitations were consistent predictors of institutional LTC use. Survey-based variables significantly improved model performance. While demographic and clinical characteristics found in many EHRs are predictive of institutional LTC, patient-reported function and partnership status improve identification of patients who may benefit from home- and community-based services.

INTRODUCTION

Over the past two decades, the U.S. Veterans Health Administration (VA) has attempted to “rebalance” long-term care (LTC) expenditures away from institutional care and towards home- and community-based services (HCBS) (Thomas et al., 2018). Similar to other health care systems, VA’s emphasis on growing HCBS is motivated by patients’ preferences to remain in the home for as long as possible, as well as the potential for cost savings through the substitution of more affordable HCBS for more costly institutional LTC services (Iglehart, 2016; Konetzka, 2014; Wolff et al., 2008). The most recent HCBS expansion efforts include the 2016 implementation of the Veteran-Directed Care and Medical Foster Home programs and the 2017 implementation of the Choose Home Initiative, which increased community support for Veterans with substantial impairments (Duan-Porter et al., 2020; Thomas et al., 2018; Yuan et al., 2019). When expanding HCBS in this way, a health system needs to ensure that it is optimizing the use of these limited resources. One way to optimize HCBS is to use existing data to identify those likely to thrive with these types of services.

Optimization depends on two related effects. The first is a substitution effect among people who would have otherwise used nursing home care. Evidence from Medicaid suggests that in states where new HCBS precisely targeted patients who required nursing-home level care, HCBS substituted for institutional care, and there were declines in overall LTC expenditures (Grabowski, 2006; Kaye, 2012; Kaye et al., 2009). The second effect relates to increased demand for HCBS among patients who are primarily reliant on informal care, but who would use HCBS if they were made available (Grabowski, 2006). In the present paper, we focus on optimizing a health system’s ability to facilitate the first effect through patient-reported measures. If well-targeted and early access to HCBS can delay or lower patients’ probability of entry into a nursing home, patients will be able to remain in their preferred setting for longer, and the health system can maximize the reach of its limited LTC budget. Optimizing a health system’s ability to identify patients at higher risk of eventually using institutional LTC services is, therefore, important from both a patient and health system perspective.

In recent years, the VA has developed dashboards for providers to identify patients who are likely to benefit from long-term care services so that HCBS can be targeted towards these patients. One such dashboard was created for the Geriatric Patient Aligned Care Team (GeriPACT) program, which provides frail, elderly Veterans and their caregivers access to multidisciplinary geriatric care through a single point-of-contact (Powers et al., 2018). This dashboard shows providers which patients might benefit from HCBS based on information that is currently available in administrative datasets, such as age and computed risk scores. It also provides descriptive information about factors that are predictive of LTC utilization, such as gender, rurality, service-connected disability information, and diagnoses. Given that these dashboards may inform how HCBS are targeted towards Veterans, it is important to understand the extent to which the characteristics highlighted in these dashboards actually predict institutional LTC utilization and whether there are ways to enhance the information in these dashboards. This is especially important in light of the substantial body of literature that suggests factors excluded from this dashboard are key predictors of LTC utilization. In particular, functional status, access to informal care, and socioeconomic characteristics have all been found to be strong patient-level predictors of entry into institutional LTC settings (Gaugler et al., 2007).

New Contributions

In the present study, we examined the extent to which the addition of factors not commonly captured in VA structured data from the electronic health record (EHR) improve our ability to predict Veterans’ use of institutional LTC. We compare models predicting the probability of using institutional care based on risk factors currently used in the VA’s GeriPACT Dashboard with models that include additional EHR variables as well as variables that are not consistently captured in the VA EHR. We compare fit statistics across models and identify which predictors are consistently associated with institutional LTC use. Results from this study can guide integrated health system investment decisions in data capture systems if the inclusion of patient-reported variables not routinely captured substantially improves the ability to identify Veterans most likely to benefit most from LTC services.

METHODS

Study Design

We compare the performance of three models that predict the probability of VA institutional LTC utilization over a one-year period for Veterans aged ≥65. The first model is based on the EHR-derived characteristics currently used to identify patients who would benefit from geriatric services through the VA’s GeriPACT Dashboard (US Department of Veterans Affairs, 2021). The second model includes additional variables that are not currently included in the GeriPACT Dashboard but that have been found to predict LTC institutionalization in the existing literature and could be added from the EHR without collecting new information. The third model includes all EHR variables from the first two models as well as patient-reported measures that are not consistently captured in VA’s EHR. Patient-reported measures were collected via survey methods, as described below. Because the third model includes additional well-established predictors of LTC utilization, we hypothesize that this model will perform better than the first two models.

Survey Methods

Patient-reported measures included in the survey-based model (Model 3 outlined below) were collected in 2018 using a mail survey sent to a nationally representative stratified random sample of 10,000 Veterans. The sampling frame included “high risk” Veterans, as defined by a one-year risk of hospitalization or death on March 16, 2018 that was in the 75^th or higher percentile based on the VA’s Care Assessment Need (CAN) score (DeSalvo et al., 2006). A sample of 10,000 high-risk Veterans across all US VA medical centers was selected using PROC SURVEYSELECT (SAS, version 9.4; SAS Institute Inc), via stratified random sampling proportional to the percentage of Veterans with a CAN score of 75 or higher in each VA medical center. This enabled us to obtain a nationally representative sample of Veterans who had at least one VA outpatient visit from March 20, 2017 to March 18, 2018, were alive, and had a valid home address at the time of cohort generation. (Zulman et al., 2020). Using the Dillman method (Dillman et al., 2014; Fillenbaum & Smyer, 1981), the survey was sent with a cover letter, a $2 bill incentive, a prepaid return envelope, and information to opt out from further mailings. The cover letter described the purpose of the survey, how Veterans were identified as potential survey respondents, and a statement that their return of the mailed survey constituted informed consent. Veterans who did not opt out and who did not respond within a 6-week period were mailed a second survey with a prepaid envelope and cover letter. Survey measures were informed by the Cycle of Complexity Model (Berkowitz et al., 2014), which posits that patient complexity is a multi-factorial construct comprised of workload, acute shocks and medical events, capacity/resilience, and access/utilization.

Study Measures

Outcome.

Our outcome measure was institutional LTC utilization at 12 months. Institutional LTC provided by or funded by the VA included care provided in Community Living Centers, Community Nursing Homes, and State Veterans Homes. This care was identified through the VA’s Corporate Data Warehouse (CDW). We also used Centers for Medicare and Medicaid data to identify non-VA institutional LTC. We applied a method based on Yun et al. (2010) and Goodwin et al. (2017) which identifies all LTC stays where Medicare funded services are provided and excludes any stays that overlap with a Skilled Nursing Facility (SNF) claim for post-acute care. This approach has been found to have sensitivity of 88–90% for identifying nursing home residents relative to Medicaid claims data (Goodwin et al., 2017; Yun et al., 2010). With this approach, Medicare Part A SNF claims are used to identify dates billed for SNF services. Medicare Part B claims are used to identify care provided in an institutional LTC setting using two types of indicators, both of which are used regardless of whether the patient is receiving SNF or LTC services. The first are Evaluation and Management (E&M) charge codes in either the carrier or outpatient file for professional services provided in a nursing home: 99304–99310, 99315–99316, or 99318. The second are carrier file place-of-service codes for nursing homes (i.e., 32 or 33). If these codes were used for services provided on dates outside of a SNF stay identified in the Part A data, then the stay was classified as an institutional LTC stay (Goodwin et al., 2017).

Selection of institutional LTC predictors.

Our selection of institutional LTC predictors was informed by the Andersen Model of Health Behavior, as adapted by Bass and Noelker (1987). This framework outlines predisposing, enabling, and health care need factors that predict LTC service utilization (Bass & Noelker, 1987). Predisposing characteristics are intrapersonal factors that affect one’s propensity for using LTC services. Studies have consistently found that advanced age and race predict institutional LTC use (Borrayo et al., 2002; Gaugler et al., 2007). Enabling factors are the resources that are available to the individual to facilitate the use of formal LTC services. These include living arrangements, marital status, availability of informal care, income, and formal care markets, which consistently predict LTC utilization (Arling et al., 2011; Borrayo et al., 2002; Charles & Sevak, 2005; Jette et al., 1995; Kitchener et al., 2003; Segelman et al., 2017; Temkin-Greener & Meiners, 1995). Health care needs refer to the perceived or evaluated illnesses or impairments that necessitate LTC service use and are the strongest predictors of LTC utilization. Activities of daily living (ADL) limitations (particularly ≥3 limitations) are strong predictors of LTC institutionalization (Borrayo et al., 2002; Gaugler et al., 2007). A diagnosis of Alzheimer's disease or dementia has been found to be positively associated with institutionalization (Severson et al., 1994; Welch et al., 1992), as has the presence of psychiatric and emotional problems (Borrayo et al., 2002), the presence of chronic health conditions (Mitchell et al., 2000), and prior hospitalizations (Gaugler et al., 2007).

Table 1 summarizes the above outlined predictors of LTC utilization and notes the extent to which these factors are captured in the VA’s GeriPACT Dashboard as well as the VA’s EHR. Overall, many well established predictors of LTC utilization are overlooked completely (i.e., race, marital status, living arrangements, informal care availability, income, and cognitive impairment), while other key predictors are captured only indirectly (i.e., ADLs, formal care markets, and mental health diagnoses). Below, we discuss how each of these factors are incorporated into the three predictive models that we compare.

Table 1:

Predictors of Long-term care utilization captured by the GeriPACT dashboard

Common Predictors of LTC utilization	Degree to which the concept is captured in the GeriPACT dashboard	Degree to which the concept is captured in the VA EHR
Predisposing a
Age	Directly	Directly
Race	Not at all	Directly
Gender	Directly	Directly
Enabling a
Living arrangement	Not at all	Not at all
Marital status	Not at all	Directly
Informal care availability	Not at all	Not at all
Income	Not at all	Indirectly (Priority statusb, means testing, copay status)
Formal LTC care market/access	Indirectly (rurality; Priority 1A status)	Indirectly (geographic region; rurality; Priority status)
Health care need a
Activity of Daily Living limitations (3+)	Indirectly (JFIc)	Indirectly (JFI)
Cognitive impairment	Not at all	Indirectly (diagnoses)
Alzheimer's disease/dementia diagnosis	Directly	Directly
Mental health diagnoses	Indirectly (suicide risk)	Directly
Chronic conditions	Directly (CAN scored, Nosose)	Directly
Prior hospitalizations	Directly	Directly

^aPredisposing, enabling and health care need factors derive from Bass and Noelker’s (1987) adaptation of the Andersen model of health service use (Bass & Noelker, 1987).

^bPriority status reflects a veteran’s service-connected disability rating.

^cThe JEN Fraily index, a risk scoring system that uses 13 impairment categories pulled from approximately 1800 diagnoses found to be related to risk of LTC admissions.

^dThe CAN score is the Care Assessment Need score, a risk adjustment score used in the VA to predict risk of death or hospitalization.

^eThe Nosos Score is a risk adjustment system for developed for VA cost data based on CMS’ Hierarchical Condition Categories, psychiatric comorbidities, and medication use (Wagner et al., 2016).

Model 1 predictors.

The variables included in our GeriPACT Dashboard-derived model were age (75–85, and ≥85); gender; rural or highly rural residence; number of hospitalizations in the prior 12 months; and an indicator for whether the Veteran was priority 1a status (reflecting that the Veteran has a service connected disability rating of ≥70% and access to all VA’s Geriatric and Extended Care services without co-pay). Additional EHR measures included JEN Frailty Index (JFI), a risk scoring system that uses 13 impairment categories pulled from approximately 1800 diagnoses found to be related to risk of LTC admissions (Kinosian et al., 2018) and the Nosos Score, a risk adjustment system developed for VA cost data using Hierarchical Condition Categories (HCC) based primarily on patient diagnoses, age and gender (Wagner et al., 2016). In sensitivity analyses, we also included additional risk adjustment scores in all three models (i.e., the CAN score and the Prevention Quality Indicator measure). Presence of a mental health diagnosis was a binary indicator for whether the Veteran had an ICD-9 or ICD-10 diagnosis of depression, anxiety, substance use disorder, post-traumatic stress disorder, bipolar disorder, or schizophrenia in the prior 12 months. These variables were selected based on information currently available to clinicians in the GeriPACT Dashboard. The dashboard identifies a cohort of patients aged 65 and older who are not currently enrolled in the GeriPACT program, who have a JFI Score between 3 and 5, and who have a CAN Score ≥75. The dashboard provides descriptive information for all of the above outlined Model 1 predictors for this cohort (US Department of Veterans Affairs, 2021).

Model 2 predictors.

The extended EHR model included all variables from Model 1, as well as race/ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, and other or unknown) and a count of 28 chronic conditions identified through ICD-9 and ICD-10 codes (see Appendix Table 1 for a list of these conditions). Because there were no Veterans utilizing institutional LTC in the “other” category for race/ethnicity, we combined this category (n=68) with the “unknown” category in regression analyses to enable the calculation of marginal effects. In sensitivity analyses, we also included EHR-derived marital status in this model. Marital status was collected from CDW’s Outpatient Visits file and is based on the most recent values available prior to the survey date.

Model 3 predictors.

Model 3 included all the variables from Model 2, as well indicators for the presence of ADL limitations, whether the Veteran is married or living with a partner, informal care availability, and financial status). Limitations with activities of daily living were ascertained using the Older Americans Resources and Services (OARS) ADL questionnaire (Fillenbaum & Smyer, 1981). Based on prior findings that the presence of ≥3 ADL limitations is associated with receipt of institutional care (Gaugler et al., 2007), we created a binary variable indicating the presence of ≥3 ADL limitations. Living arrangement was ascertained by asking: “Are you currently married or living with a partner?” Access to informal care was based on the tangible support components of the Medical Outcomes Survey-8 (Moser et al., 2012). Respondents were asked, “If you needed it, how often is someone available to…” 1) help you if you were confined to bed, 2) take you to a doctor if you need it, 3) prepare your meals if you are unable to do it yourself, and 4) help with daily chores if you were sick. Respondents indicated whether they had access to someone for these tasks all of the time, most of the time, some of the time, a little of the time, or none of the time. We considered someone to have access to informal care if they had access to all four of these tangible supports most or all of the time. Financial status was included in the model as a binary indicator for whether the respondent was “barely getting by,” “falling behind,” or “in serious financial trouble” versus “in good shape” or “okay” in response to the following question: “Which of the following statements best describes your own personal economic situation?”

Analysis

To compare baseline characteristics of respondents and non-respondents, we calculated standardized differences, and considered a standardized difference of 0.2 to indicate a small but meaningful difference between the two groups (Sawilowsky, 2009). Missing data from the Nosos score (n=3), finances (n=102), and tangible supports (n=131) variables implied that we would lose 6.27% of our sample without accounting for missing data. We used multiple imputation by chained equations (MICE) to impute missing data. The MICE approach creates imputed datasets based on a set of imputation models, with one model for each variable with missing values. It involves producing multiple complete datasets, after which the results of analyses of each dataset are combined. White et al. (2011) suggest that the number of imputed datasets should be at least equal to the percentage of incomplete cases in the dataset, which in this case would be larger than six (White et al., 2011). As such, we multiply imputed data 10 times using Stata’s mi command (Royston & White, 2011).

We used logistic regression models which were selected over linear probability models given the possibility of misclassification for our outcome variable. In instances where misclassification is likely, linear probability models are more likely than maximum likelihood models to lead to an inability to identify parameters and calculate marginal effects (Hausman et al., 1998). Because our survey sampling design was clustered at the VA medical center level, standard errors for all regressions were clustered at the medical center level. Model performance was compared via Akaike Information Criteria (AIC) and C-statistics. We calculated the mean AIC, considering a decrease in AIC of >2 to indicate considerable improvement and a decrease of >10 to indicate substantial improvement (Heinze et al., 2018; Sanchez-Pinto et al., 2018). We also report mean C-statistics for each model and considered a C-statistic above 0.8 to indicate a well discriminating model (Steyerberg et al., 2010). Predictive performance was also assessed by comparing each model’s sensitivity and specificity. We established a cutoff value for the predicted probability using Youden’s J, which maximizes the sum of the sensitivity and specificity, thereby not favoring one over the other (Youden, 1950). We compared observed institutional LTC utilization to the predicted outcomes from our models. The cohort in our base case analysis was limited to Veterans aged ≥ 65 due to our focus on LTC utilization and the use of Medicare claims data when constructing the outcome variable. Sensitivity analyses that included all survey respondents, regardless of age, were also conducted.

Role of the funder

This work was funded by the VHA Office of Primary Care. The evaluation was reviewed and designated as non-research quality improvement by the supporting Veterans Health Administration program office and the Durham VA Medical Center and Stanford University institutional review boards and therefore did not require approval or informed consent.

RESULTS

Sample overview

Of the 10,000 Veterans who received a paper survey, 4,685 returned a survey. Eight Veterans were excluded from analysis because they were deceased prior to the survey field period (n=5) or because they were missing essential EHR data (n=3). Based on variables with standardized differences greater than 0.2 (Appendix Table 2), survey respondents tended to be older, were less likely to be non-Hispanic Black, and were less likely to have 0–1 chronic conditions relative to non-respondents. (Figure 1). Amongst our analytic sample of Veterans over the age of 65 (n=3,478), there were no significant differences between respondents and non-respondents.

Figure 1:

Sample Flow Diagram

Table 2 provides a descriptive overview of the sample. 42.3% of the analytic sample of 3,478 Veterans was over the age of 75 and 38.6% lived in rural or highly rural locations. 35.4% of the sample had a Priority 1a status that entitled them to full coverage of LTC services. 25.2% had a JFI score of 6 or higher, 5.6% a diagnosis of dementia, 26.2% a mental health diagnosis, and 51.7% had 4 or more chronic conditions. Non-Hispanic Black respondents comprised 12.0% of the sample and Hispanic respondents 3.8%. 12.0% of the sample had 3 or more ADL limitations, 61.6% were married or living with a partner, and 54.3% indicated that they would have access to tangible support if they needed it. Most respondents (65.3%) indicated that their finances were good or okay, while the remaining 34.7% indicated that they were barely getting by, falling behind, or in serious financial trouble. Overall, the rate of institutional LTC utilization in the 12 months following the survey date was 4.97% (n=173).

Table 2.

Mean and proportions of respondent characteristics at baseline

Variables	N(%)/ Mean (95% CIa)
Age
65 to 74	2007 (57.7%)
75 to 84	984 (28.3%)
≥85	487 (14.0%)
Rural residence	1343 (38.6%)
Female	90 (2.6%)
Priority 1a status	1230 (35.4%)
JEN frailty index
0 to 2	457 (13.1%)
3 to 5	2123 (61.0%)
6 to 13	875 (25.2%)
Missing	23 (0.7%)
Dementia diagnosis	195 (5.6%)
Mental health diagnosis	910 (26.2%)
Nosos Concurrent Score	1.4 (0.8, 2.7)
Nosos missing	3 (0.1%)
Inpatient admissions
None	3102 (89.2%)
One	250 (7.2%)
≥2	126 (3.6%)
Race/ethnicity
Non-Hispanic White	2657 (76.4%)
Non-Hispanic Black	418 (12.0%)
Hispanic	131 (3.8%)
Other	68 (2.0%)
Missing	204 (5.9%)
Chronic conditions
0–1	368 (10.6%)
2 to 3	1313 (37.8%)
≥4	1797 (51.7%)
≥ 3 ADL limitationsb	416 (12.0%)
Married or living with partner	2074 (61.6%)
Married missing	113 (3.2%)
Presence of tangible support	1817 (54.3%)
Finances - Barely getting by/in serious trouble	1172 (34.7%)
Missing	102 (2.9%)
N	3,478

^aCI: Confidence interval;

^bADL: Activity of daily living limitation

Model performance comparison

Model performance statistics are presented at the bottom of Table 3. The mean AIC value was significantly lower for the survey-based model (Model 3) (AIC: 1188) compared to the GeriPACT model (Model 1) (AIC: 1229) and the extended EHR model (Model 2) (AIC: 1229). The decrease in AIC for Model 3 relative to both Models 1 and 2 met our criteria for a substantial improvement in AIC, which was an observed decrease of 10 or more. The C-statistic for Model 3 was also the highest at 0.829 versus 0.780 and 0.786 for Models 1 and 2 respectively. The C-statistics for Model 3 was the only one that met our criteria for a well-discriminating model (i.e., a C-statistics ≥0.80). Model 3 had the highest sensitivity (0.728) and specificity (0.796) relative to Model 1 (sensitivity: 0.619, specificity: 0.79) and Model 2 (sensitivity: 0.624, specificity: 0.782).

Table 3.

Marginal effects and model fit statistics for logistic regression models predicting one-year institutional long-term care utilization for Veterans aged ≥65

	GeriPACT Model (Model 1)				EHR Model (Model 2)				Survey-based Model (Model 3)
	Marginal Effect	Standard Error	95% confidence interval		Marginal Effect	Standard Error	95% CI		Marginal Effect	Standard Error	95% confidence interval
Age
65 to 74 (Ref)
75 to 84	0.023	0.009	0.006	0.040	0.024	0.009	0.006	0.041	0.017	0.008	0.001	0.032
≥85	0.068	0.019	0.031	0.106	0.067	0.019	0.031	0.104	0.040	0.015	0.011	0.068
Rural residence	0.008	0.006	−0.005	0.021	0.007	0.006	−0.005	0.020	0.008	0.006	−0.004	0.020
Gender
Male (Ref)
Female	0.011	0.023	−0.034	0.056	0.010	0.022	−0.034	0.054	0.002	0.018	−0.033	0.037
Priority 1a status	0.003	0.005	−0.008	0.014	0.003	0.005	−0.007	0.014	0.000	0.005	−0.009	0.010
JEN frailty index
0 to 2 (Ref)
3 to 5	0.012	0.010	−0.007	0.031	0.014	0.010	−0.004	0.033	0.014	0.009	−0.003	0.031
6 to 13	0.003	0.012	−0.019	0.026	0.008	0.013	−0.017	0.034	0.011	0.012	−0.013	0.034
Dementia diagnosis	0.059	0.020	0.019	0.098	0.061	0.020	0.022	0.101	0.035	0.015	0.005	0.065
Mental health diagnosis	−0.002	0.006	−0.014	0.010	−0.002	0.006	−0.014	0.010	−0.002	0.006	−0.013	0.009
Nosos score	0.009	0.001	0.006	0.011	0.009	0.001	0.007	0.012	0.007	0.001	0.005	0.010
Inpatient admissions
None (Ref)
One	0.012	0.012	−0.012	0.035	0.011	0.011	−0.011	0.033	0.009	0.010	−0.011	0.029
≥2	0.038	0.025	−0.011	0.087	0.036	0.024	−0.011	0.083	0.024	0.020	−0.015	0.063
Race/ethnicity
Non-Hispanic White (Ref)
Non-Hispanic Black					−0.002	0.009	−0.019	0.015	−0.005	0.007	−0.019	0.009
Hispanic					−0.019	0.011	−0.040	0.002	−0.020	0.008	−0.035	−0.004
Other/unknown					−0.010	0.010	−0.029	0.008	−0.009	0.009	−0.027	0.009
Chronic conditions
0–1 (Ref)
2 to 3					0.002	0.011	−0.020	0.023	0.001	0.010	−0.019	0.021
≥4					−0.011	0.011	−0.033	0.010	−0.010	0.010	−0.029	0.010
≥ 3 ADLa limitations									0.066	0.017	0.033	0.099
Married or living with partner									−0.015	0.007	−0.028	−0.002
Presence of tangible support									0.004	0.006	−0.007	0.016
Finances
Good or ok (Ref)
Barely getting by/in financial trouble									−0.007	0.006	−0.019	0.005
n	3478				3478				3478
Mean AICb	1229.1				1228.8				1187.7
C-statistic	0.780				0.786				0.829
Sensitivity %	61.85				62.43				72.83
Specificity %	78.79				78.15				79.55
Pseudo R2	0.148				0.154				0.171

^aADL: Activity of daily living limitation;

^bAIC: Akaike Information Criteria

Model results

Table 3 summarizes the marginal effects and 95% confidence intervals (CI) for all the covariates in each model, with predicted probabilities for all covariates summarized in Appendix Table 3. Across all 3 models, we see similar patterns in terms of which factors consistently predict LTC institutionalization. Unless otherwise stated, all marginal effects reported below are from the survey-based model (Model 3, Table 3). Age is a consistent and strong predictor of LTC utilization. Respondents aged 75 to 84 and 85-plus were respectively 1.7% points (95% CI: 0.001, 0.032) and 4.0% points (95% CI: 0.011, 0.068) more likely to use institutional LTC relative to those under 75. The association was attenuated slightly in the survey-based model relative to the models based on EHR variables alone (Model 2). Veterans with a diagnosis of dementia were 3.5% points (95% CI: 0.005, 0.065) more likely to use institutional LTC, while a unit increase in the Nosos value was associated with a 0.7% point (95% CI: 0.005, 0.010) increase in institutional LTC use. In the survey-based model, Hispanic Veterans were 2.0% points (95% CI −0.035, −0.004) less likely to use institutional LTC.

From the survey-based variables, the presence of functional limitations was the strongest predictor of institutional LTC use. Veterans with ≥3 ADL limitations were 6.6% points (95% CI: 0.033, 0.099) more likely to use institutional LTC. Having a partner or being married was negatively associated with institutional LTC use, with partnered Veterans being 1.5% points (95% CI: −0.028, −0.002) less likely to use institutional LTC.

Sensitivity analyses

Alternate models that used the full survey sample without age restrictions (n=4,677) yielded similar results with respect to model performance. In the full sample model, however, survey-reported partnership status was non-significant (Appendix 4). In sensitivity analyses that included additional risk adjustment scores (i.e., the CAN score and the Prevention Quality Indicator measure), we did not see significant improvement in model performance. In analyses that included EHR-derived marital status in Model 2 as opposed to survey-reported partnership status in Model 3, there was no improvement in model performance, and being married was not significantly associated with institutional LTC use.

DISCUSSION

We compared the performance of three models predicting use of institutional LTC services among a cohort of high-risk Veterans aged 65 and older. We found that age, rurality, a diagnosis of dementia, Nosos risk score, partnership status, and having 3 or more ADL limitations were significant predictors of institutional care use. These findings are in line with previous literature exploring predictors of institutional LTC use (Gaugler et al., 2007). We found that only the survey-based model met our criterion for a well discriminating model, with a C-statistic of 0.82. The best performing model with respect to AIC minimization was also the survey-based model, which resulted in a decrease in AIC of 41.4. The additional explanatory power from the survey-based model was predominantly due to the inclusion of information on functional limitations (≥3 ADL limitations) and partnership status. We also observed an increase in sensitivity from 61.85% to 72.83%, with no decrease in specificity. This implies that of the 42,158 Veterans who used VA provided or funded institutional long-term care in 2018 (United States Government Accountability Office, 2020), an additional 4,627 Veterans would be correctly identified using predictions from the survey-based model (Model 3) relative to the GeriPACT model (Model 1) without additional false positives.

These results indicate that although LTC institutionalization models using variables included in the GeriPACT dashboard perform well, their predictive power could be improved with additional information that is not currently included in the EHR. ADL information, in particular, was highly predictive of institutional LTC use, and its inclusion in these models had the largest impact on the models’ performance. This is in line with prior research that has identified ADL information as the most important predictor of institutional LTC utilization (Gaugler et al., 2007). Our findings indicate that even with the inclusion of measures that are highly correlated with functional status information, such as JFI (Kinosian et al., 2018), ADL information still adds significant predictive power to these models. This improvement in predictive power from ADL information could facilitate earlier intervention with HCBS.

These results highlight the importance of finding ways to collect ADL information, such as through regular functional assessments and the addition of structured EHR fields. Health care systems aiming to implement additional screening in clinical settings can learn from ongoing efforts to integrate social determinants of health (SDH) into EHRs. While there is often widespread agreement among providers about the utility of SDH information (Davidson & McGinn, 2019), there is less agreement about how the collection of this information should be integrated into existing workflows (Gold et al., 2017), who should be conducting SDH screening (Schickedanz et al., 2019), and whether the collected information should be limited to factors that are actionable by clinicians (Davidson & McGinn, 2019; Schickedanz et al., 2019). As health systems work through these challenges, they may also want to consider capitalizing on functional status information that can be found in unstructured EHR sources, such as free-text notes, by using methods such as natural language processing.

We also found that survey-collected partnership status information was predictive of institutional LTC while EHR-derived marital status was not. The differences in how each variable performed in our models is likely due to the discordance we observed between the two variables. 10.2% (n=344) of survey respondents who indicated they were married or had a live-in partner in the survey were coded as divorced, never married, separated, or widowed in the EHR. A smaller proportion (5.53%) (n=186) self-identified as not having a partner in the survey but were coded as married in the EHR. This discordance may be due to the broader phrasing of the survey question, which asked about both married and live-in partners, as well as the likelihood that the survey-based information was more recent than the EHR-derived marital status variable. These findings suggests that there may be added value in more frequent collection of information about the presence of a co-residential partner.

Our results should be interpreted while keeping in mind certain limitations. First, in this paper we focus on practical considerations for targeting HCBS. There are also important ethical considerations that should be considered, but which were beyond the scope of the present paper. In particular, the assessment of the predictive power of models such as these is based on historical data that reflect prior utilization patterns. Predicted probabilities are compared to a “gold standard” of observed utilization patterns in order to help identify Veterans with LTC needs. These prior utilization patterns, however, do not necessarily reflect the optimal or most equitable practice patterns. If LTC utilization is not observed for certain groups due to access barriers or discrimination, as opposed to differences in need or preferences for LTC, then policies informed solely by predictive models will not highlight and help mitigate these issues. Second, contrary to prior studies, we did not find that common predictors of LTC use, such as gender, race/ethnicity, chronic condition and mental health diagnoses, presence of a caregiver, and financial status, consistently predicted LTC utilization. In some instances, these findings may be due to the use of survey measures that do not accurately capture the concept in question. The MOS-8 tangible support questions, for instance, capture the respondents’ perceived access to informal care in a hypothetical situation as opposed to the respondents’ actual receipt of informal care. Most studies exploring the relationship between informal care and LTC institutionalization use the latter (Lo Sasso & Johnson, 2002; Van Houtven & Norton, 2004). In other instances, the relatively small number of observations for certain sub-groups (e.g., women and respondents who were non-Hispanic Black or Hispanic) in combination with a rare outcome, may explain why some common predictors of LTC use were not significant at p<0.05. Finally, we also note that in the years prior to our survey, VA introduced several HCBS programs such as the Veteran-Directed Care and Medical Foster Home Programs. While the introduction of new institutional LTC substitutes such as these may affect the relationship between the predictors and outcome, we note that over the course of our follow up period, these programs still had very limited reach.

Conclusion

Our findings indicate that the information currently provided to VA clinicians to help identify patients who would benefit from home and community-based services (HCBS) is predictive of who ultimately uses VA institutional LTC. This is in line with what we would expect from a health system that is aiming to substitute HCBS for institutional LTC. Information about functional status as well as co-residential partnership, however, would significantly improve clinicians’ ability to target HCBS towards Veterans who are likely to be eventual users of institutional LTC. This finding has important implications for the majority of older adults, both inside and outside the VA, who are cared for in non-geriatric primary care settings where this information is not routinely collected (Nicosia et al., 2019).