Medicaid long‐term care workforce training intervention and value‐based payment metrics

Abstract

Objective

To examine the impact of a scaled implementation of workforce training intervention on value‐based payment measures in a large home‐based Medicaid managed long‐term care plan population in New York.

Data Sources

Managed long‐term care clients' health assessments from the Uniform Assessment System of New York merged with paid claims, home health aide operational visit data, and workforce training rosters between 2018 and early‐2020.

Study Design

A quasi‐experimental design was used. Exposure and control groups were constructed using the proportion of service hours delivered by trained aides between clients' baseline and follow‐up/outcome assessments. Multivariate logistic generalized linear and additive models were estimated to examine associations between exposure to trained aides and value‐based payment measures.

Data Collection/Extraction Methods

The analytic sample consisted of 19,212 pairs of assessments from 13,320 long‐term care clients continuously enrolled in the plan between baseline and follow‐up/outcome assessments. Matched assessment pairs were 6–10 months apart.

Principal Findings

Over 27% of the study population (n = 3656 clients) received services from one or more of 8683 trained aides. Statistically significant associations were observed for four of seven value‐based payment measures; however, the presence and magnitudes of positive training effects differed by client service needs. With covariate adjustment, workforce training had the largest estimated positive impacts on rates of flu vaccination among average‐need clients (1.60%, standard error [SE] = 0.01), not experiencing uncontrolled pain among above‐average‐need clients (0.69%, SE = 0.001), stable/improved pain intensity among heavy‐need clients (1.25%, SE = 0.01), and stable/improved shortness of breath among light‐need clients (0.88%, SE = 0.003).

Conclusion

Although we found mixed associations between scaled workforce training implementation and value‐based payment metrics, we noted workforce training could benefit high‐need long‐term care recipients. Health indicators more sensitive to the daily support provided by direct care workers should be integrated into value‐based health care models.

What is known on this topic

• Training and retaining a competent direct care workforce is essential to meet the growing demand for Medicaid long‐term care services nationwide.

• Capacity‐building interventions to strengthen the long‐term care workforce have received increased policy attention and investments.

• The health impact of workforce training intervention on the value‐based payment metrics of Medicaid home‐based long‐term care recipients has not been adequately examined in current literature.

What this study adds

• Our study found mixed associations between workforce training intervention and value‐based payment metrics, implying differential opportunities for trained home health aides to intervene in client health.

• Our findings suggest workforce training can benefit the frail, high‐need long‐term care recipients whose daily living depends mostly on their home health aides.

• Quality improvement training for the long‐term care workforce should prioritize aides serving high‐need care recipients.

1. INTRODUCTION

Home‐ and community‐based long‐term care have received much policy attention lately. ¹ In 2018, New York State embarked on a three‐year Workforce Investment Organization (WIO) initiative to strengthen the managed long‐term care workforce through large‐scale implementation of training programs among direct care workers. ² , ³ The WIO is a “capacity‐building” intervention for long‐term care organizations to develop a workforce to deliver quality services to care recipients and reduce avoidable inpatient care expenditures. ³ , ⁴ This study examined the impact of WIO training programs developed to improve the performance of value‐based payment measures of a home‐dwelling Medicaid Managed long‐term care (MLTC) plan population. Study findings will contribute to a better understanding of the health impact of workforce development interventions within value‐based payment models.

New York's Medicaid program helps cover health care costs for the State's eligible enrollees who otherwise could not afford these expenses. ⁵ To coordinate and deliver care services to enrollees with long‐term health and functional needs due to chronic illness or disabilities, the State utilizes an MLTC system that manages the planning and provision of care through privately operated long‐term care health plans. ⁶ Care services delivered by the plans' contracted providers are critical supports for the health and daily functioning of Medicaid MLTC members and help them stay in their homes and communities. ⁶ , ⁷ , ⁸ However, rising health care costs, enrollment expansion, higher spending and poorer enrollee health outcomes than the national average, and health disparities among racial, ethnic, and socioeconomic groups are major challenges facing New York's Medicaid program. ⁹

To improve care quality and outcomes and secure financial sustainability of the Medicaid program, the State's Medicaid Redesign Team implemented several systemwide multiyear reforms, including a value‐based payment (VBP) reform in 2015. ² , ⁹ Embedded within the State's longer standing Quality Improvement Program, this reform was designed to transform the health delivery system and payment structure by rewarding quality and cost‐effectiveness over volume. ² , ⁹ Along these lines, VBP measures are financially consequential quality metrics the State utilizes to judge the performance of MLTC plans. Performance on these metrics affects whether a plan and its VBP‐contracted providers jointly receive monetary incentives or penalties. ² , ⁹ , ¹⁰ , ¹¹ While the State imposed a financial risk‐bearing agreement on MLTC plans and their contracted providers through VBP arrangements, ⁷ , ⁸ limited evidence currently exists to guide and substantiate the incorporation of direct care workforce into value‐based payment models for the long‐term care sector. ¹²

Focusing on WIO‐funded training programs designed for home health aides to improve VBP measures, this study sought to examine the health impact of these programs on a Medicaid MLTC plan population. These programs represent an intraorganizational strategy employed by risk‐sharing health care organizations in their adaptation to a value‐based payment environment. The programs were distinct from other training initiatives in that they were developed based on shared quality goals and risk‐sharing terms in VBP contracts and involved substantial coordination efforts to implement.

2. METHODS

2.1. Study design and setting

We used a quasi‐experimental design to examine the association between WIO VBP training and client VBP metrics. The Study Protocol was approved for Human Subject Protection by the Visiting Nurse Service of New York Institutional Review Board. Study agreements and data sharing contracts were developed and signed off by participating organizations prior to data collection.

The participating health plan is one of the largest MLTC plans in New York. Four of the plan's contracted licensed home care services agencies (“Agencies”) that implemented WIO‐funded VBP training programs to home health aides (“aides”) agreed to participate in the study. Five WIO VBP programs implemented by these Agencies as mandatory home health aide training were included. The programs varied by start dates, lengths, curriculum structure, and contents; yet, they all shared a focus on improving the VBP metrics, were highly valued by stakeholders, and reached more than 20% of the Agency's workforce serving the plan's membership by February 2020.

2.2. Data

The Uniform Assessment System of New York (UAS‐NY) was our primary data source for MLTC client demographic characteristics, health status, conditions, and VBP measures. The UAS‐NY is a biannual clinical assessment required from Medicaid MLTC beneficiaries. MLTC plans utilize clients' assessment results to develop or modify their “plan of care,” which contains plan‐authorized health care services and home health aide hours. Clients' first assessments are conducted upon their enrollment with the MLTC plan, and regular follow‐up assessments are conducted semiannually after a prior assessment, with a range between 6 and 10 months.

We started data preparation by pairing clients' sequential assessments and preparing items to calculate VBP metrics. Each client in our sample had at least one paired baseline and follow‐up/outcome assessment. Some clients (40.3%) continuously enrolled with the plan for a long time had multiple pairs wherein the follow‐up/outcome assessment in a prior pair became the baseline of the subsequent pair. Next, we matched plan members' assessment pairs with paid claims, home health aide operational visit data, and the training rosters provided by participating Agencies to compute the total amount of home health aide hours clients received between assessment pairs and classify whether the hours were delivered by aides who completed WIO VBP training (Appendix 1 of Supporting information).

We then analyzed the VBP metrics alongside clients' proportion of home health aide hours received from WIO‐trained aides between their paired assessments. The analytic period spanned between October 25, 2018, and March 10, 2020, a period where we could accurately identify service hours provided by WIO‐trained aides and avoid impacts associated with the COVID‐19 pandemic. Our final analytic sample consisted of 19,212 pairs of qualifying assessments (main analytic units; average months between assessment pairs = 5.7, SD = 0.86, Median = 6) from 13,320 unique MLTC clients continuously enrolled with the plan in‐between their assessment pairs. Table 1 displays the characteristics of the study population and their estimated propensity scores of receiving services from WIO‐trained aides (see Section 2.3).

TABLE 1.

Characteristics of managed long‐term care clients within analytic sample

	All clients with claim‐paid home health aide services (total unique N = 13,320; 100%)	Clients served by study‐participating home care agencies (total unique N = 4611 a ; 34.6%)	Clients served by non‐study‐participating home care agencies (total unique N = 9001 a ; 67.5%)
Female (%)	73.9%	75.6%	73.0%
Race groups (%)
Non‐hispanic White	25.7%	17.6%	29.6%
Non‐hispanic Black	22.5%	25.6%	21.0%
Hispanic	31.0%	33.3%	29.7%
Non‐hispanic other	10.3%	12.5%	9.3%
Unknown	10.6%	11.0%	10.4%
Age groups (%) at baseline assessment
54 years or younger	9.6%	5.8%	11.6%
55–64 years old	10.7%	9.0%	11.6%
65–74 years old	18.4%	17.4%	18.9%
75–84 years old	28.9%	32.5%	27.0%
85 years and older	32.4%	35.3%	30.9%
Average months of enrollment with plan by outcome assessment [range] (standard deviation/SD)	44.6 [5.2–252] (38.9)	61.6 [6–252] (39.3)	34.3 [5.2–240.1] (34.5)
Service needs (%)
Light‐need (clients receiving <3 h of claim‐paid home health aide hours/day)	5.4%	6.6%	5.2%
Average‐need (clients receiving 3–7 claim‐paid home health aide hours/day)	51.8%	58.7%	47.3%
Above‐average‐need (clients receiving more than 7 but less than 12 claim‐paid home health aide hours/day)	26.5%	24.2%	27.7%
Heavy‐need (Clients receiving 12+ claim‐paid home health aide hours/day)	16.3%	10.4%	19.8%
Proportion of clients served by aides completing targeted training programs	27.4%	88.9%	—
Region
Residing in counties with heavy reliance on public transportation	83.1%	90.6%	78.9%
CDPAS status‐ Ever hired family/friend as personal paid caregiver (%)	22.2%	1.5%	32.4%
Average estimated propensity scores of receiving services from aides completing targeted training programs (SD)	0.283 (0.20)	0.423 (0.16)	0.223 (0.19)

Note: CDPAS, consumer‐directed personal assistance services: A program available to New York State Medicaid long‐term care beneficiaries, which allows care recipients to hire their family or friends as paid caregivers. ¹³

^a A small number of clients changed their home care services agencies or were served by multiple agencies during our study period, resulting in the difference between the total N of unique clients and the sum of clients served by participating and nonparticipating agencies.

2.3. Measures

Seven VBP measures were selected for analysis (Table 2 and Appendix 2 of Supporting information). These measures were prioritized quality improvement metrics by the participating plan, drove the design of WIO VBP programs, and aligned with the interests of a cross‐organizational evaluation committee representing the leadership of WIO partners overseeing this project. Each metric measures the presence or change of a health‐related status/condition. Whenever possible, we coded study measures following New York State's 2018 MLTC VBP methodology, ¹⁴ , ¹⁵ the latest version available at our study time.

TABLE 2.

Operational definitions of key measures

	Measure name	Description	Type
Outcome measures: value‐based payment metrics a	Flu vaccination	Odds of client receiving an influenza vaccination in the last year	Binary
	Pain controlled	Risk‐adjusted odds of client not experiencing uncontrolled pain
	Dyspnea (shortness of breath) stable/improved	Risk‐adjusted odds for client to remain stable or demonstrate improvement in shortness of breath
	No emergency room visit	Risk‐adjusted odds of client not having an emergency room visit in the last 90 days
	No falls with injury	Risk‐adjusted odds of client not experiencing falls resulting in injury in the last 90 days
	Pain intensity stable/improved	Risk‐adjusted odds of client whose pain intensity remains stable or improved
	Urinary continence stable/improved	Risk‐adjusted odds for client to remain stable or demonstrate improvement in urinary continence
Focal explanatory measures: WIO‐exposure and client service needs	Amount of WIO‐exposure (dosage of intervention received)	Between baseline and outcome assessments, proportion (%) of claim‐paid service hours received from home health aides who completed targeted training programs implemented by participating home care agencies	Continuous
	No exposure	Between baseline and outcome assessments, 0% of service hours received from trained aides	Categorical
	Light exposure	Between baseline and outcome assessments, receiving 1%–<25% of service hours from trained aides
	Moderate exposure	Between baseline and outcome assessments, receiving 25%–<75% of service hours from trained aides
	High exposure	Between baseline and outcome assessments, receiving 75%–100% service hours from trained home health aides
	Service needs: Weekly home health aide service hours	Between baseline and outcome assessments, typical weekly claim‐paid home health aide service hours received (total service hours between assessments standardized to 7‐day cycle)	Continuous
	Light‐need clients	Clients receiving less than 3 h of claim‐paid home health aide service hours/day	Categorical
	Average‐need clients	Clients receiving 3–7 claim‐paid home health aide service hours/day
	Above‐average‐need clients	Clients receiving more than 7 but less than 12 claim‐paid home health aide service hours/day
	Heavy‐need clients	Clients receiving 12 or more claim‐paid home health service hours/day

Note: WIO, Workforce Investment Organization: A 3‐year initiative to provide training programs for direct care workforce in New York State to strengthen the State's Medicaid long‐term care system. ² , ³ Targeted training programs in this study were funded through this initiative.

^a For the purpose of this study, to be able to examine client‐level outcomes by the number of service hours provided by aides who completed targeted training, we modified the measure descriptions slightly from the initial New York State managed long‐term care value‐based payment measures definition. ¹⁴ See Appendix 2 of Supporting information for detail.

For confounding adjustment, ¹⁶ we controlled for client characteristics using a propensity score of receiving services from WIO‐trained aides given their gender, race/ethnicity, age group, region, length of plan enrollment, and status of participation in the Consumer‐Directed Personal Assistance Services program (i.e., State program which allows Medicaid MLTC beneficiaries to hire family or friends as paid caregivers). This propensity score was included in our multivariate models as a covariate to balance differences in observable characteristics between the intervention and control groups. We also controlled for the State's VBP risk adjustors (outcome measure specific), ¹³ known comorbidities (outcome measure specific), and time effect with a constructed study month indicator.

Table 2 describes our operationalization of the outcome metrics and two focal explanatory measures, exposure to WIO‐trained aides and service needs, for which we examined both continuous and categorical versions. The exposure measure assessed clients' amount or dosage of intervention received given their proportion of claim‐paid home health service hours received from WIO‐trained aides, ¹⁷ ranging from 0% to 100%. Measuring treatment status with a continuous scale of proportional exposure to the intervention allowed valid comparison with a concurrent control group within our sample.

We used standardized claim‐paid home health aide hours to approximate the average duration of service sessions clients received to measure their service needs. ¹⁷ We examined the interplay of intervention dosage and duration that clients would receive with the interaction of the exposure and service needs measures. We inspected measure distributions on their continuous scales, checked sample sizes in cross‐tabulation cells, and consulted the MLTC plan about home health aide service delivery patterns. We then developed categories of “no exposure” (0%), “light exposure” (1%–<25%), “moderate exposure” (25%–<75%), and “high exposure” (75%–100%) for the exposure measure, and categories of “light‐need” (<3 service hours/day), “average‐need” (3–7 h/day), “above‐average‐need” (over 7 but <12 h/day) and “heavy‐need” (≥12 h/day) for service needs measure. Our final models used the continuous WIO‐exposure measure and categorical service needs measure, which allowed us to retain the full distribution of the exposure measure and meaningfully interpret the interaction of exposure by service needs without compromising model explanatory power (see Appendix 3 of Supporting information, full regression models).

2.4. Statistical analysis

Multivariate logistic generalized linear models (GLM) and generalized additive models (GAM), an extension of GLM, were used to investigate associations between WIO‐exposure and VBP outcomes. ¹⁸ GLM models characterize an overarching association between WIO‐exposure and VBP outcomes within the study population and provide the direction of this association. GAM models assess whether the associations between WIO‐exposure and VBP metrics should be specified as linear or nonlinear, as well as simultaneously fitting a regression line for each service need group.

For VBP metrics where positive training effects were detected, we compared predicted outcome metric performance at the level of WIO‐exposure where the largest training effect was observed against predicted metric performance at the level of nonexposure (i.e., 0% of WIO‐exposure) controlling for all covariates included in the model. We report the differences in predicted metric performance as “the largest possible estimated net impacts” of the WIO VBP programs by respective client service need groups using the State's VBP methodology (Appendix 4 of Supporting information). ¹⁴ , ¹⁵ , ¹⁹ The net impact estimates are a form of adjusted marginal effects of WIO‐exposure on the outcome metrics we could expect to observe from an average, “typical” client within their respective service need group in our study sample. These estimates provide more robust and interpretable magnitudes of effect size than odds ratios and can help better project the potential for scaled workforce training implementation to improve VBP metrics. ¹⁹ , ²⁰ , ²¹

We used likelihood ratio tests, the GAM model's proportion of deviance explained, and GLM's McFadden's pseudo R ² to assess model goodness‐of‐fit and explanatory power. All our reported associations between WIO‐exposure and VBP metrics were adjusted for covariates listed in Section 2.3. Since flu vaccination was not a risk‐adjusted metric per the State's VBP methodology, ¹⁴ , ¹⁵ risk adjustors were omitted for this metric.

3. RESULTS

Monthly enrollment for the participating MLTC plan varied from 13,000 to 20,000 clients across our analytic period. Approximately 60% of these clients were 75 years or older, three‐quarters (73.9%) were female, and one‐third (31.0%) were Hispanic. Roughly one‐third (34.6%) of clients were served by the four Agencies that implemented WIO VBP programs.

3.1. Descriptive findings

A total of 11,163 aides were trained across the VBP programs during the study period. Three‐quarters (77.8%; n = 8683) of the trained aides provided services to the MLTC clients with study‐eligible assessment pairs. Of these, most trained aides (64.4%; n = 5589) served multiple clients. More than 1 in 4 (27.4%; n = 3656) of the clients examined were in the intervention group (i.e., receiving claim‐paid service hours from WIO‐trained aides). Among clients served by participating Agencies (n = 4611), the vast majority (92%) were cared for by multiple aides, with an increase in service hours that client received positively correlated with the number of aides they received services from (r = 0.3; p < 0.000).

Nearly 8 in 10 (79.3%) clients cared for by participating Agencies received services from trained aides between their baseline and follow‐up/outcome assessments (average exposure = 50.8% of service hours delivered by trained aides; range = 1%–100%). Across service needs groups, the WIO‐exposure measure had a comparable distribution: within each group, roughly three‐quarters of the clients (ranging between 72.1% and 78.3%) received services from WIO‐trained aides, with the average amount of exposure ranging between 48.9% (among “heavy‐need” clients) and 51.8% (among “average‐need” clients). This suggested WIO VBP programs had a similar reach to the clients served by participating Agencies regardless of their service needs.

3.2. VBP performance modeling results

We found mixed results between WIO‐exposure and VBP outcomes. Regression results suggested systematic associations between WIO‐exposure and four of the seven VBP metrics we analyzed: receiving an influenza vaccination in the previous year (flu vaccination), not experiencing uncontrolled pain (pain controlled), remaining stable or demonstrating improvement in pain intensity (pain intensity), and remaining stable or demonstrating improvement in shortness of breath (shortness of breath). However, the patterns, strengths, and directions of associations varied across VBP metrics, client service needs, and clients' amount of WIO‐exposure (Table 3). Positive effects of exposure to trained aides (“positive training effects”) were most frequently and consistently observed among above‐average‐need clients, followed by heavy‐need clients. Meanwhile, we did not observe systematic associations between WIO‐exposure and metrics of not having an emergency room visit in the last 90 days (no emergency room visit), not experiencing falls resulting in major or minor injury in the last 90 days (no falls with injury), and remaining stable or demonstrating improvement in urinary continence (urinary continence). Below, we summarize analytic results where systematic associations between WIO‐exposure and VBP metric were observed.

TABLE 3.

Summary of analytic results

	Flu vaccination: odds of receiving a flu vaccination	Pain controlled: Risk‐adjusted odds of not experiencing uncontrolled pain	Pain intensity: Risk‐adjusted odds of remaining stable or demonstrating improvement in pain Intensity	Shortness of breath: Risk‐adjusted odds of remaining stable or demonstrating improvement in shortness of breath
Generalized additive model estimated degree of freedom (edf) of coefficient of WIO‐exposure by service needs interaction	2.37 a	3.33**	2.08*	3.76*
	p = 0.06	p = 0.01	p = 0.01	p = 0.02
Proportion of deviance explained by model	3.00%	15.80%	3.99%	6.21%
Odds ratio of observing positive training effect at peak measure performance given WIO‐exposure versus no exposure (Main training effect independent of influence from controlled covariates)
Light‐need clients	NA		NA
Odds ratio		1.40		1.23
[95% confidence interval]		[0.42–4.74]		[0.74–2.04]
Average‐need clients			NA
Odds ratios	1.06	1.39		1.23
[95% confidence interval]	[1.04–1.09]	[0.89–2.17]		[0.97–1.57]
Above‐average‐need clients
Odds ratio	1.04	7.26	1.27	1.79
[95% confidence interval]	[1.03–1.04]	[2.61–20.15]	[1.02–1.58]	[1.19–2.69]
Heavy‐need clients	NA
Odds ratio		3.90	1.10	1.20
[95% confidence interval]		[1.12–13.46]	[0.84–1.45]	[0.72–1.97]
Estimated net impacts (amount of metric performance improvement attributable to WIO‐exposure after adjusting for controlled covariates)
Light‐need clients	NA	0.54%	NA	0.88%
Standard error (SE)		(SE = 0.002)		(SE = 0.003)
[Estimated metric performance change]		[98.90% ➔ 99.44%]		[97.61% ➔ 98.49%]
Average‐need clients	1.60%	0.54%	NA	0.86%
Standard error (SE)	(SE = 0.01)	(SE = 0.001)		(SE = 0.002)
[Estimated metric performance change]	[83.09% ➔ 84.69%]	[99.09% ➔ 99.62%]		[97.74% ➔ 98.60%]
Above‐average‐need clients	1.37%	0.69%	0.62%	0.84%
Standard error (SE)	(SE = 0.01)	(SE = 0.001)	(SE = 0.01)	(SE = 0.003)
[Estimated metric performance change]	[81.70 ➔ 83.08%]	[98.99% ➔ 99.68%]	[93.66% ➔ 94.28%]	[97.93% ➔ 98.77%]
Heavy‐need clients	NA	0.56%	1.25%	0.61%
Standard error (SE)		(SE = 0.002)	(SE = 0.01)	(SE = 0.002)
[Estimated metric performance change]		[99.16% ➔ 99.71%]	[93.87% ➔ 95.11%]	[98.46% ➔ 99.07%]

Note: WIO‐exposure: Between baseline and outcome assessments, % of claim‐paid service hours received from home health aides who completed targeted training programs. Light‐need clients: Managed long‐term care clients receiving less than 3 h of claim‐paid home health aide service hours/day. Average‐need clients: Managed long‐term care clients receiving 3–7 claim‐paid home health aide service hours/day. Above‐average‐need clients: Managed long‐term care clients receiving more than 7 but less than 12 claim‐paid home health aide service hours/day. Heavy‐need clients: Managed long‐term care clients receiving 12 or more claim‐paid home health service hours/day.

^a Marginally significant at p < 0.07.

^* p ≤ 0.05; **p ≤ 0.01.

3.3. Flu vaccination

A marginally significant effect of WIO‐training on MLTC clients' odds of receiving a flu vaccination was observed at the population level (Table 3, the interaction of exposure by service needs “estimated degrees of freedom”/edf = 2.37; p = 0.06). The GAM model plot indicated different patterns of association between WIO‐exposure and flu vaccination by service needs groups (Figure 1, Flu vaccination plot). The plot suggested, relative to clients in the other two groups, light‐need and heavy‐need clients exhibited the highest odds of receiving a flu vaccination independent of WIO‐exposure. For average‐need and above‐average‐need clients, increases in WIO‐exposure were mostly associated with greater odds of receiving a flu vaccination, with peaks of outcome metric performance reached at different levels of exposure for each group (80% vs. 55%, respectively). Beyond performance peaks, additional increases in WIO‐exposure were not associated with greater odds of receiving a flu vaccination. Net impact estimation suggested for average‐need clients when at least 50% of service hours were delivered by trained aides (level of WIO‐exposure approaching peak outcome metric performance), WIO VBP programs could improve their flu vaccination rate by 1.60 percentage points, moving the metric's performance for a typical client in this group from 83.09% to 84.69% (Table 3). For above‐average‐need clients, when at least 75% of service hours were delivered by trained aides, the programs could improve their flu vaccination rate by 1.37 percentage points, moving metric performance from 81.70% to 83.08%.

FIGURE 1.

Associations between exposure to trained home health aides and performance of value‐based payment metrics. Percent of WIO exposure: Between baseline and outcome assessments, proportion (%) of claim‐paid service hours clients received from home health aides who completed targeted training programs. Metric peak performance: The point where best performance of the outcome metric was observed. Beyond the point, additional increases in exposure (X‐axis values) are not associated with additional improvement in metric performance (Y‐axis values). These plots were generated from results of generalized additive models. Exhibited associations between WIO‐exposure and outcome metrics were independent of influence from controlled covariates mentioned in Section 2.3. See Appendix 3 of Supporting information for full regression models

3.4. Pain controlled

Exposure to WIO‐trained aides was significantly associated with the odds that MLTC clients would not experience uncontrolled pain (Table 3, the interaction of exposure by service needs edf = 3.33; p = 0.01). Positive training effects were seen across service need groups (Figure 1, pain controlled plot), with the strongest association between WIO‐exposure and better performance on pain controlled observed among above‐average‐need clients (steepest regression slope, greatest increase in the odds of observing better metric performance alongside increases in the exposure measure). The linear regression lines suggested a constant amount of improvement in metric performance on the log‐odds scale for every unit increase in WIO‐exposure within each group, indicating increases in WIO‐exposure were constantly associated with better metric performance. For all service need groups, peak performance on this metric was found when 100% of service hours were delivered by WIO‐trained aides. Estimated net impacts suggested that with 100% of service hours delivered by trained aides, the rate of not experiencing uncontrolled pain could improve from 98.90% to 99.44% for typical light‐need clients, from 99.09% to 99.62% for average‐need clients, from 98.99% to 99.68% for above‐average‐need clients, and from 99.16% to 99.71% for heavy‐need clients, respectively.

3.5. Pain intensity

Mixed results about WIO‐exposure and performance of pain intensity were observed. Whereas a significant association between WIO‐exposure and pain intensity was observed at the population level (Table 3, the interaction of exposure by service needs edf = 2.08; p = 0.01), the directions of association differed by service need groups (Figure 1, pain intensity plot). Specifically, minimal WIO‐training effect on pain intensity was observed among light‐need clients (nearly flat regression line), a negative relationship between WIO‐exposure and pain intensity was observed among average‐need clients, and positive relationships were seen among above‐average‐need and heavy‐need clients. Within the two groups where positive training effects were observed, peak performance of the outcome metric was reached at 100% of exposure. Estimated net impacts suggested when 100% of service hours were delivered by WIO‐trained aides, rate of remaining stable or demonstrating improvement in pain intensity could improve from 93.66% to 94.28% for typical above‐average‐need clients, and from 93.87% to 95.11% for heavy‐need clients.

3.6. Shortness of breath

We again observed positive and linear associations between WIO‐exposure and the odds of remaining stable or demonstrating improvement in shortness of breath (Table 3, the interaction of exposure by service needs edf = 3.76; p = 0.02; Figure 1, shortness of breath plot). WIO‐exposure was associated with the greatest odds of observing better performance on shortness of breath among above‐average‐need clients (steepest slope, strongest association between WIO‐exposure and better metric performance). For the other three groups, the odds of observing a positive training effect appeared similar (comparable slopes). Estimated net impacts suggested when 100% of service hours were delivered by WIO‐trained aides, rate of remaining stable or demonstrating improvement in shortness of breath could improve from 97.61% to 98.49% for light‐need clients, from 97.74% to 98.60% for average‐need clients, from 97.93% to 98.77% for above‐average‐need clients, and from 98.46% to 99.07% for heavy‐need clients, respectively.

4. DISCUSSION

Capacity‐building training interventions for direct care workers serving long‐term care recipients have received increased policy attention and support. ²² Using GAM models to unpack variability in observed training effect on VBP metrics, our findings provide some insights into potential health impacts of scaled implementation of workforce training in a value‐based payment environment. Among the study population, we found that whether and how clients' exposure to trained aides was associated with the performance of their VBP measures varied across outcome metrics, the proportion of service hours delivered by trained aides (amount of exposure), and service needs (daily home health aide hours received). This result indicates that selection of outcome metrics, amount of intervention delivered, and average duration of each service session can all affect observed impacts of workforce training on client health. Careful consideration of these aspects could help develop more tailored approaches to scaled intervention investments to achieve optimal outcomes, including defining a “threshold” where training programs could be expected to influence population‐level health metrics. ¹⁷

The fact that we did not find statistically significant associations between WIO‐exposure and metrics of no emergency room visits, no falls with injury, and urinary continence suggests that workforce training, or workforce training alone, may not be sufficient intervention for these metrics. A thorough needs assessment of the social‐cultural and environmental contexts of targeted service recipients could help identify risk factors and potential social determinants of health to inform future initiative designs. ²³ , ²⁴ For instance, to support trained aides to effectively improve these VBP metrics, intervention developers may examine care recipients' and their aides' accessibility to proper vehicles when urgent care centers could be used in place of emergency rooms, and the presence of hazards in care recipients' home environment (e.g., poor lighting, unstable furniture, and narrow bathroom layout). Understanding of care recipients' social and environmental contexts can help incorporate mediators needed to successfully enable positive workforce training effects on these metrics.

Of the four metrics where systematic associations between WIO‐exposure and metric performance were found, positive training effects were most frequently and consistently observed among above‐average‐need clients (i.e., for flu vaccination, pain controlled, pain intensity, and shortness of breath), followed by heavy‐need clients among whom positive training effects were observed for three metrics (i.e., pain controlled, pain intensity, and shortness of breath). In comparison, positive training effects were less frequently observed among light‐need clients, and mixed results were observed among average‐need clients. These findings suggest that clients receiving greater daily home health aide service hours (i.e., over 7 hours per day) are most likely to benefit from a scaled workforce training implementation. Since home health aide hours are authorized based upon clients' clinical assessment results, this implies a trained workforce could benefit the vulnerable high‐need long‐term care recipients whose daily living depends much on their aides.

One plausible explanation for the varying patterns of association between the exposure measure and VBP metrics is differential opportunities for trained aides to impact client health within their scope of day‐to‐day work. For instance, with knowledge acquired from training, aides could encourage clients to receive a flu vaccination, but aides do not have the license to vaccinate clients nor mandate vaccination. By contrast, aides could practice acquired pain management strategies to assist clients in coping with pain and reducing pain intensity. Likewise, trained aides may become vigilant about signs and triggers of shortness of breath, and they could implement strategies to assist clients to breathe more smoothly and notify care managers for an assessment of client health.

Opportunities for aides to intervene in client health are also influenced by client clinical conditions, functional status, prescribed care plans with specified tasks for aides to complete during service hours, and the continuity of patient–caregiver relationship between clients and their aides. ²⁵ This interpretation also helps understand the absence of impacts of WIO‐exposure on metrics of no emergency room visits, no falls with injury, and urinary continence, as a trained workforce may not be sufficient to mitigate risk factors affecting these metrics from clients' environmental settings. Opportunities to intervene impy a twofold message: impacts of workforce training on VBP metrics vary, and existing VBP metrics in New York's MLTC system may not represent the ideal set of measures to comprehensively assess the health impacts of home health aides on their clients.

Meanwhile, the opposite effects of WIO‐exposure on the performance of pain controlled and pain intensity observed among average‐need clients might be due to their subjective and/or perceived heightened sensitivity of pain. To illustrate, as WIO‐trained aides taught clients about pain management strategies, average‐need clients might become more sensitive and expressive of their pain, which could be interpreted as worsening pain intensity by some UAS‐NY assessors and led to the observed negative effect of WIO‐exposure. With the large proportion of average‐need clients in our sample (51.8%), the negative association between the exposure measure and pain intensity exhibited among these clients may explain the negative WIO‐exposure coefficient we observed at the population level in the GLM model (Appendix 3.6 of Supporting information).

Our net impact estimations suggest that with approximately one‐quarter (27.4%) of clients receiving services from trained aides, accounting for observable confounders, workforce training intervention could improve MLTC VBP metrics between 0.54 percentage points (pain controlled, among light‐ and average‐need clients) and 1.60 percentage points (flu vaccination, among average‐need clients). Analysis of statewide performance trends of MLTC VBP metrics reveals that when put into context, these seemingly small improvements could still be financially impactful to health care organizations under risk‐sharing contracts. ²⁶ The current percentile range by which a plan is judged as underperforming, meeting, or exceeding performance expectations to receive financial penalty or bonus is sometimes smaller than a full percentage point, ²⁶ magnitudes similar to most net training impacts we estimated. Under the present value‐based payment model, the financial consequences and the plan's relative performance to state average are important contexts to properly evaluate the significance of percentage differences in VBP metrics attributable to WIO programs.

Based on study findings, we developed four recommendations for policy and practice. First, to properly reflect the value of direct care workers and their services, health policy makers should convene health care organizations and workforce representatives to institute quality metrics which better align with aspects of client health that this workforce could influence directly, such as quality of life, emotional health, or social well‐being measures. ²⁷ Second, quality improvement workforce training should prioritize aides serving high‐need long‐term care recipients since a trained workforce could mean more to these clients. Third, to address potential environmental hazards and social determinants of health experienced by long‐term care recipients, the State and local governments should consider incentivizing health care and community organizations in the development and implementation of community improvement and home modification interventions. ²⁴ Fourth, the GAM model can be a useful analytic tool to study complex public health interventions. ²⁸

5. LIMITATIONS

Our study has several limitations. First, our measurement of the intervention and VBP metrics was constrained by data items consistently collected across home care agencies and training programs. With the exposure and service needs measures, we studied variations in training effect by the interplay of dosage and duration. Yet, we did not measure program components, leaving the pathways connecting the programs to specific outcomes and the intensity aspect of the intervention unstudied. ¹⁷ Second, a ceiling effect of high‐performing metrics among our study population likely limited measurable impacts of the training programs, resultant in small estimated net impacts for pain controlled, pain intensity, and shortness of breath. Third, unobserved confounders, including concurrent quality initiatives implemented by participating organizations, the time lag between an aide's completion of VBP training and their scheduled service assignments to clients in our sample, and other client‐care training programs aides serving our study population may have attended could also limit detectable WIO training impacts. Furthermore, although we used propensity scores to improve comparability between the WIO‐exposure and control groups, selection bias from unobservable characteristics may still exist.

6. CONCLUSION

Training and retaining a competent workforce is essential to meet the growing demand for long‐term care services nationwide. ²² Current policy and public support for home‐ and community‐based long‐term care offers opportunities to strengthen this workforce. ¹ Our study found varying impacts of large‐scale workforce training implementation on value‐based payment metrics among a Medicaid long‐term care plan population in New York. Nonetheless, we noted a trained workforce is beneficial to high‐need long‐term care recipients whose daily living depends mostly on their aides. Given the essential services direct care workers provide, workforce‐level quality metrics and client health indicators more sensitive to the day‐to‐day support offered by this workforce should be incentivized in value‐based health care models. ¹² , ²⁷ Future research could incorporate a cost‐effectiveness analysis to assess the financial impacts of workforce capacity‐building interventions.

Supporting information

Appendix S1. Supporting information.

Fong M‐C, Russell D, Brickner C, Gao O, Vito S, McDonald M. Medicaid long‐term care workforce training intervention and value‐based payment metrics. Health Serv Res. 2022;57(2):340-350. doi: 10.1111/1475-6773.13930