Innovation in a learning health care system: Veteran Directed Home and Community Based Services

Abstract

A path breaking example of the interplay between geriatrics and learning health care systems is the Veterans Health Administration’s (VHA’s) planned roll-out of a program for providing participant-directed home and community based services to veterans with cognitive and functional limitations. Here, we describe the design of a large-scale stepped wedge cluster randomized trial of the Veteran Directed Home and Community Based Services (VD-HCBS) program. From 3/2017 through 12/2019, up to 77 Veterans Affairs Medical Centers will be randomized to times to begin offering VD-HCBS to veterans at risk of nursing home placement. Services will be provided to community-dwelling participants with support from Aging and Disability Network Agencies. The evaluation is coordinated by the VHA Partnered Evidence-based Policy Resource Center (PEPReC) and includes collaboration from operational stakeholders from the VHA and Administration for Community Living, and interdisciplinary researchers from the Center of Innovation in Long Term Services and Supports and the Center for Health Services Research in Primary Care. Among older veterans with functional limitations who are eligible for VD-HCBS, we will evaluate health outcomes (hospitalizations, emergency department visits, nursing home admissions, days at home) and health care costs associated with VD-HCBS availability. Learning health care systems facilitate diffusion of innovation while enabling rigorous evaluation of effects on patient outcomes. The VHA’s randomized roll-out of VD-HCBS to veterans at risk of nursing home placement is an example of how to simultaneously achieve these goals. PEPReC’s experience designing an evaluation with researchers and operations stakeholders may serve as a framework for others seeking to develop rapid, rigorous large-scale evaluations of delivery system innovations targeted to older adults.

Introduction

High-quality geriatrics care should be person-centered, community-based, and supported by rigorous evidence.^1–2 The Veterans Health Administration (VHA) is committed to these goals in the provision of care for four million veterans over the age of 65.³ As the nation’s largest integrated health care system and a learning health care system, the VHA prioritizes rigorous evaluation of health care delivery innovations and rapid translation of evidence into practice.

A path breaking example of the interplay between geriatrics and the learning health care system is the VHA’s planned roll-out of a program that provides financial resources and options counseling for home and community-based care to veterans at risk of nursing home placement (Veteran-Directed Home and Community Based Services [VD-HCBS]). Accompanying the roll-out, VHA and Administration for Community Living operational stakeholders are collaborating with an interdisciplinary group of researchers and advisors to facilitate its evaluation. The evaluation is being coordinated by the VHA Partnered Evidence-based Policy Resource Center (PEPReC) and includes experts in participant-directed care from Applied Self Direction and The Lewin Group (the technical assistance lead), and interdisciplinary VHA research partners from the Center of Innovation in Long Term Services and Supports (LTSS COIN) and the Center for Health Services Research in Primary Care. The unifying goal is to evaluate the health outcomes and costs associated with this person-centered, community-based geriatrics intervention.

In the following sections, we describe VD-HCBS and the design of a roll-out that allows diffusion to be accompanied by rigorous analysis to inform decisions about whether and how best to sustain or improve VD-HCBS and avoid unnecessary institutionalization of older veterans. Our experience designing a systematic evaluation with researchers and operations stakeholders from within and outside of VHA may serve as a framework for others seeking to develop rapid, rigorous large-scale evaluations of delivery system innovations targeted to older adults.

Veteran-Directed Home and Community Based Services

In order to align budgetary priorities with patient preferences, the VHA began offering VD-HCBS in 2009 through several Veterans Affairs Medical Centers (VAMCs) to veterans at risk of placement in a long-term care facility. The VD-HCBS program is modeled after Medicaid’s Cash and Counseling demonstration, which was associated with fewer unmet needs for assistance with daily activities and greater satisfaction with care.^4–6 The VHA’s early experience with VD-HCBS suggests that it may allow veterans to remain safely at home for longer time periods and improve patient and caregiver quality of life.⁷ However, early VD-HCBS evaluation did not include a control group nor collect systematic data on health care use and costs.

The VD-HCBS program enables veterans with functional or cognitive limitations to purchase care services in their homes that maximize their independence. In general, patients are referred to VD-HCBS when they need more than 20 hours of home care per week, are eligible for nursing home-level care, or have three or more dependencies in activities of daily living (ADLs) or significant cognitive impairment. However, sites have flexibility in choosing whom to refer to the program. VHA clinicians refer veterans to VD-HCBS, wherein the VHA pays for services coordinated by Aging and Disability Network Agencies (ADNAs). Veterans receive monthly budgets based on disability, need, and geographic location ($464-$26,585/month in 2015).⁸ ADNA options counselors work with veterans to identify care needs, facilitate purchasing of appropriate services or equipment to address these needs, and monitor spending. Allowable services and equipment include home modifications, medical equipment, and personal care workers (including family members) chosen by veterans.

One VAMC may work with one or more ADNAs. Each ADNA works with the technical assistance lead to complete a readiness review before providing VD-HCBS services.⁹ Readiness reviews include education on self-direction and veterans’ rights and responsibilities, development of manuals and policies outlining allowable purchases, and identification of a financial management services provider. Following readiness review completion, the ADNA and VAMC enter into a provider agreement, and the VAMC can begin referring patients to VD-HCBS.

The VHA’s Office of Geriatrics and Extended Care (GEC) wishes to expand this program nationwide, but budget considerations¹⁰ and personnel constraints preclude simultaneous diffusion across the VAMCs yet to implement the program. This fact, combined with promising pilot data and the current lack of rigorous evaluation, provide a unique opportunity to conduct a large-scale stepped wedge randomized evaluation of VD-HCBS. As part of the VHA’s learning health care system initiative, PEPReC identifies promising health care delivery innovations that are suitable for randomized roll-out and evaluation.^11–12 The VD-HCBS program is the first of several such programs. PEPReC staff and research and operations partners have worked together to design a roll-out and evaluation that balances clinical and administrative needs with the development of a rigorous evidence base for VD-HCBS.

Staggered Roll-out of VD-HCBS

GEC has begun randomizing medical centers to times when they may begin referring patients to VD-HCBS. VD-HCBS will be compared to usual care and by the end of the three-year study period, all eligible VAMCs will have operational VD-HCBS programs. This stepped wedge design will provide needed information on the program’s value while allowing every site to receive the intervention.^7,13-14 The randomized design and inclusion of a representative sample of veterans eligible for VD-HCBS (rather than volunteers) overcomes deficiencies noted in past evaluations of participant-directed care.¹⁵ Evaluations of outcomes associated with the roll-out have been approved by the VA Boston Healthcare System and VA Central institutional review boards (IRBs). This evaluation was prospectively registered at ISRCTN and clinicaltrials.gov (ISRCTN12228144, NCT03145818). Note that the randomization and roll-out of VD-HCBS is a GEC operational activity and does not require IRB approval.

All VAMCs without operational VD-HCBS programs as of 3/2017 (n=77) are eligible for the roll-out. Medical centers that are unable to delay VD-HCBS referrals will be excluded from randomization, as will VAMCs in which facility leadership is not interested in implementing VD-HCBS. At the beginning of each six to ten-month period (wave), GEC will identify about 14 VAMCs (sites) that could begin offering VD-HCBS to veterans within the first half of the wave if randomized to do so. These sites will be in the process of creating contract agreements with local ADNAs. The subset of sites will be randomized to begin referring patients to VD-HCBS immediately (early) or halfway through the wave (late). This process will be repeated every six to ten months until the remaining VAMCs have implemented VD-HCBS (Figure 1).

Figure 1.

Overview of randomization.

Simple randomization in a stepped wedge approach like this is likely to lead to imbalance across sites that refer patients earlier versus later in important characteristics, such as facility case-mix, that may be associated with our outcomes of interest. Imbalance could occur because GEC is constrained in each wave to randomize start times for only about 14 VAMCs. By chance, those that are randomized to start early could differ in important ways from those randomized to start later.

To address this concern, covariate constrained randomization is being used to assign start times. This allows for better baseline balance on potential confounders than simple randomization, matching, or stratification.¹⁶ Patient case-mix, geriatrics practice patterns, and state or county HCBS access may be associated with VD-HCBS referral and health care utilization (Table 1).^17–21 From the VAMCs ready for participation in each wave, all potential variations in start times among them are evaluated, and combinations are ranked according to confounder balance. For each combination, differences between earlier and later facilities’ mean values of standardized variables are calculated, squared, and summed to create a balancing score.²² From the combinations in the top 25% of covariate balance, one option is randomly selected.²³ Sites are told that start times will be staggered and when they may begin referring patients (Figure 2).

Table 1.

Variables included in covariate constrained randomization

Variable	Description (Source)
Patient case-mix at VAMC
Number of patients age 75 years and older per site	(GEC-DAC)
Percentage of patients at site who are age 75 years and older and at least 70% service connected	(GEC-DAC)
Mean CAN score among patients age 75 years and older	Predicts 1-year mortality24 (GEC-DAC; based on FY15 VHA CDW data)
Mean Jen Frailty Index score among patients age 75 years and older	Associated with functional limitations18-19(GEC-DAC; based on FY15 VHA CDW data on 1800 diagnoses in 13 categories associated with risk of nursing home placement [e.g., self-care impairment, dementia])
Prospective NOSOS score among patients age 75 years and older	Measure of chronic disease burden27-28 (GEC-DAC ; based on FY15 VHA CDW data)
Site-specific patterns of caring for older patients
VAMC spending on HCBS	Percentage of FY16 LTSS expenditures going to HCBS (VSSC)
VAMC has nursing home on campus	FY14 data (VSSC)
Market penetration of HCBS among veterans 75 and older	Number of veterans age 75 years and older receiving HCBS for every 1000 veterans 75 and older (VSSC)
State or county access to HCBS
Urban/rural location	VAMC in urban (RUCA code 1.01 or 1.1) vs rural or highly rural location (VSSC)
State participation in early participant-directed care initiatives	Participation in Cash & Counseling Demonstration and Evaluation 2004-2008 and/or Community Living Program 2007-2008 (Administration on Aging29, published report6)
Percentage of state Medicaid LTSS expenditures going to HCBS	Medicaid HCBS expenditures as a % of total Medicaid LTSS expenditures by state, FY14 (Published report30)

ACL = Administration for Community Living, CAN = Care Assessment Need, CDW = corporate data warehouse, FY = fiscal year, GEC-DAC = Geriatrics & Extended Care Data Analysis Center, HCBS = home and community-based services, LTSS = long-term services and supports, RUCA = rural-urban commuting area, VHA = Veterans Health Administration, VSSC = Veterans Health Administration Support Service Center

Figure 2. Stepped wedge design.

Shaded blocks represent medical centers assigned to begin referring patients to VD-HCBS. Exact number of medical centers in each step is subject to change.

Planned Evaluation Activities

Here, we detail planned analyses of changes in health outcomes and health care costs associated with VD-HCBS availability that require the comparison group data afforded by our randomized stepped wedge design. Investigators at the LTSS COIN and the Center for Health Services Research in Primary Care have planned evaluations of veterans’ quality of life and unmet needs for services, caregiver well-being, and facilitators and barriers of uptake of VD-HCBS that will provide important contextual information not available from administrative data. Details of the rich data collection planned for these evaluations are beyond the scope of this paper.

Data sources

Data sources will include VHA Corporate Data Warehouse inpatient, outpatient, purchased care, and cost data, and Medicare (inpatient, outpatient, MedPAR, hospice, home health agency, durable medical equipment) and Medicaid Analytic eXtract (MAX) data on cost and utilization.

Analytic cohort

Because sites are allowed flexibility in whom they refer to the program, our ability to detect a treatment effect will depend on how well we can identify patient factors associated with VD-HCBS enrollment. Based on Fiscal Year 2016–2017 enrollment data (all available data before randomization began), we will create a logistic regression model that predicts patient likelihood of enrollment in VD-HCBS. From this model, we will identify a cut-point of enrollment likelihood that patients must meet to be included in the analytic cohort. Patients receiving care at multiple VAMCs will be assigned to the site responsible for the majority of their past-year care. Potential predictors of VD-HCBS enrollment include other GEC service use, receipt of Veterans Benefit Administration Aid & Attendance benefits, chronic illness burden, frailty (Jen Frailty Index^18–19), mortality risk,²⁴ traumatic brain injury, spinal cord injury, and dementia diagnoses, VHA inpatient and outpatient health care use in the year before VD-HCBS enrollment, and sociodemographic characteristics. We hypothesize that our analytic cohort will primarily be veterans with the following characteristics: age 75 years or older, Jen Frailty Index of 5 or higher, and at least one VHA inpatient or outpatient visit in the past year. ^17–19 Patients will be included in the cohort during the first month they meet inclusion criteria and followed for the study’s duration. In addition, we will analyze the subset of patients with at least 70% service connected disability rating, for whom the VHA is obligated to pay for nursing home level care.

Outcomes

The main outcomes we will analyze are: any hospitalization, any emergency department visit, any residential or post-acute care nursing home admission, and total health care costs per patient per month alive. Cost data will capture costs of the VD-HCBS program, other VHA purchased care, and VHA inpatient and outpatient medical care and medications. All cost analyses will use the Gross Domestic Product Deflator to adjust costs to 2019 dollars. We will assess traditional Medicare and Medicaid health care use and expenditures among veterans enrolled in Medicare and/or Medicaid.

Secondary outcomes include hospitalization frequency, hospital and nursing home length of stay, ambulatory care sensitive hospitalization incidence and frequency, and costs attributed to HCBS, nursing home stays, hospitalization, and outpatient care. Utilization outcomes also will be combined into one measure looking at number of days at home (days not in an emergency department, inpatient care, or long-term care facility).²⁵

Explanatory variables

All outcome models will include the site-level covariates used for covariate constrained randomization²⁶ and the variables used to create our analytic cohort.

Analyses

Our primary analysis will be an intent-to-treat analysis: once a facility has been randomized to begin offering VD-HCBS, patients in our analytic cohort and associated with that facility are included in the treatment group. Covariate-constrained randomization will minimize the influence of site-level characteristics on our estimates of VD-HCBS’ effect on utilization and costs. However, there may still be site-specific variation in VD-HCBS program execution, and observations within sites may be correlated. To account for these factors, we will use generalized linear mixed models (GLMMs) with random effects for VAMCs.¹⁴Binary outcomes will be modeled with a binomial distribution and logit link, costs with a gamma distribution and log link, and count data with a Poisson or negative binomial distribution and log link.

Our data will be structured in terms of person-months. That is, for each month a patient is included in the analytic cohort, we will measure his outcomes and whether VD-HCBS was available at his facility. The GLMMs will model outcomes as a function of VD-HCBS availability in the patient’s facility at a given month, patient baseline characteristics, fixed time effects, and random VAMC effects. Using data from all patients in our cohort during the entire study period, we will use the coefficient corresponding to VD-HCBS availability to estimate the average change in outcomes that occurs when VD-HCBS is available at a patient’s facility. We will calculate bootstrapped standard errors.

We do not expect VD-HCBS availability to immediately impact patient outcomes. To test whether this is the case, our GLMMs will include indicators for lagged treatment effects. This will allow us to estimate the effect of VD-HCBS availability in a given month, as well as the cumulative effect of VD-HCBS availability over several months.

Power

Even if only 60 sites are included in the evaluation, we will still have 80% power to detect clinically meaningful differences in hospital and emergency department use and costs by the end of the study period (see Supplementary Text).

Challenges and Potential Solutions

We are unable to allocate start times for every VAMC at the beginning of the study. We will control for time period in our analyses, but there still may be unobserved differences between year 1 and year 3 adopters that are associated with VD-HCBS referrals and our outcomes of interest. However, we can still take advantage of the study’s randomized structure by treating within-wave randomization to early or late referrals as an instrumental variable. Within wave, randomization to early or late referrals should be associated with VD-HCBS receipt but not outcomes. Other potential instrumental variables include ADNA “readiness” to provide services at the time VAMCs are randomized to begin referring patients (e.g., completed financial management services contract); readiness should only influence patient outcomes through its impact on service provision timing. This secondary analysis will allow us to estimate the effect of VD-HCBS receipt on health outcomes and costs among patients whose chance of receiving VD-HCBS varies with random assignment and ADNA readiness.

In addition, procedures for forming new VAMC-ADNA partnerships and providing VD-HCBS likely will improve over time as we learn from our experiences. We will explore differences in treatment effects by wave to ensure that results from the entire study period can be aggregated.

Evaluation Goals

By randomly staggering VD-HCBS start times at VAMCs, each VAMC acts as its own control. This allows us to rigorously evaluate the relationships among VD-HCBS, health care use, and costs among older veterans. We hypothesize that VD-HCBS availability will be associated with an overall reduction in hospital admissions, emergency department visits, nursing home admissions, and reduced costs associated with avoidable health care utilization among older veterans with functional limitations. Our evaluation is directly responsive to calls for evidence to sustain and optimize the provision of VD-HCBS.⁷

Beyond building the VD-HCBS evidence base, a secondary product of this partnered evaluation will be increased knowledge about best practices for interdisciplinary researchers and stakeholders who wish to design and conduct rapid, systematic roll-outs and evaluations of delivery system innovations targeted to older adults.

Summary

The VHA’s systematic roll-out of VD-HCBS to veterans at risk of nursing home placement is an example of how to simultaneously roll out an intervention while enabling evaluations of effects on patient outcomes. Accompanying the roll-out, operational stakeholders within and outside of the nation’s largest integrated delivery system are collaborating with an interdisciplinary group of researchers to facilitate its evaluation. Through this partnered evaluation, we will rigorously evaluate the costs and health outcomes associated with VD-HCBS, a program based on the tenets of high-quality geriatrics care.¹

Supplementary Material

Appendix

Supplementary Text S1: Power analysis for VD-HCBS evaluation