Measuring Quality in Home Health Care

Abstract

Background/Objectives:

Improving quality performance in home health is an increasingly high priority. The objective of this study was to examine trends in industry performance over time using three quality measures: a composite quality metric (Q index), an infection prevention measure (vaccination verification), and an outcome measure (hospital avoidance).

Design/Setting/Participants/Measures:

We linked Home Health Compare and Provider of Services data from 2012 to 2016, which included 39,211 observations during the 5-year study period and 7,670 agencies in 2016. The Q index was developed to allow comparability over time equally weighting the contributions of each element. After examining summary statistics, we developed three regression models stratified by ownership (for-profit/nonprofit) and included two constructs of nurse staffing, in addition to controlling for known confounders.

Results:

Most agencies (80.4%) were for-profit. The Q index and vaccination verification improved substantially over time, but there was no change in hospital avoidance. Ownership status was associated with all 3 measures (p<0.001). Registered nurse staffing (relative to licensed practical nurses and home health aides) was associated with higher Q index and vaccination verification (p<0.001).

Conclusion:

The Q index allows for assessment of trends over time in home health care. Ownership and nurse staffing are important factors in the quality of care. The overall home care market is driven by for-profit agencies, but their characteristics and outcomes differ from nonprofit agencies.

INTRODUCTION

Home health care is a growing and viable alternative to many institutional post-acute care settings,¹ particularly for patients aged 65 years and older.² In 2016, over 12,000 Centers for Medicare & Medicaid Services (CMS)-certified home health agencies (HHAs) made more than 110 million home visits to over 3.5 million Medicare beneficiaries.³ Although for-profit agencies were banned from Medicare prior to 1980, they now account for over 75% of HHAs in the United States.⁴

Improving quality of care in the home health sector is a focus of CMS. In 2003, CMS mandated that Medicare-certified HHAs provide performance data for a variety of quality indicators, which became the basis of the publicly-available Home Health Compare database.⁵ To help consumers understand these data, CMS developed the Quality of Patient Care Star Rating, a composite measure that summarizes process and outcome measures related to quality, and calculates an overall score comparing each agency to all others during a calendar year.⁶ Until recently, the Star Rating used nine measures: timely initiation of care, drug education, influenza vaccination verification, acute care hospitalization, and improvement in ambulation, transferring, bathing, pain, and shortness of breath.⁷ While some of the individual measures have been reported since 2003, the Star Rating became publicly available in 2015,⁷ and emerging evidence suggests home health quality improved with public reporting.^8–10

A frequently-used indicator of the quality of care provided by HHAs is unplanned hospitalizations, which many home health patients are at risk for.^1,11,12 Using national Outcome and Assessment Information Set (OASIS)¹³ data from 2010, Shang et al. found the second most common reason for unplanned acute care hospitalizations was respiratory infections.¹⁴ A systematic review also found estimates of infections in home health patients varied dramatically among studies.¹⁵ Many respiratory infections, which are often associated with influenza, could be prevented in home health patients with flu vaccination.¹⁶ For elderly home health patients, infections pose significant health risks, and infection prevention and control has been recognized as a national patient safety goal by organizations, such as the Joint Commission and CMS.^17,18

In other healthcare settings, there is evidence that ownership and nurse staffing can impact the quality of care provided to patients. For example, for-profit status is associated with lower quality and higher costs than nonprofit status in hospitals and other health care sectors.^19–21 However, evidence examining these relationships in HHAs is limited and mixed.^9,10,22 One study found that, in Michigan, for-profit HHAs performed better than nonprofits on 5 of 12 measures studied.²² Another study found that, compared to nonprofits, for-profit HHAs performed worse on all five of the aggregated quality measures examined.¹⁰ Similarly, using national data from 2011 to 2015, Wang et al. observed small but significant overall improvements in HHA performance during the period but worse performance among for-profits relative to nonprofits.⁹ The mixed results may be associated with variation in the quality metrics used in these studies, leaving important questions about quality and agency ownership unanswered. Additionally, although the importance of nurse staffing in the provision of high quality care has been demonstrated in hospitals and nursing homes,^23,24 its role in home health care settings remains unresolved. To our knowledge, there are no contemporary studies addressing these issues.^25–27

In this study, because the Star Ratings are based on relative performance within year, we developed an alternative composite measure can be used to evaluate performance over time. Our objectives were to examine trends in industry performance over time using three quality measures: our composite quality metric (Q index), an infection prevention measure (vaccination verification) and an outcome measure (hospital avoidance); and to identify predictors of quality including ownership and staffing, while controlling for known confounders.

METHODS

Data:

We linked Provider of Services (POS) and Home Health Compare data. The POS includes a wide range of HHA characteristics each year.²⁸ Home Health Compare data are drawn from OASIS and Medicare claims data (the latter for acute care hospitalizations only). We chose 2012 to 2016 data because 2016 was the most recent year for POS data at the time of analysis, and 2012 was the year Home Health Compare began using Medicare claims for hospitalizations.

Dependent variables:

We developed the composite Q index by aggregating the nine Quality of Patient Care Star Rating components (see Supplementary Figure S1), each increasing with quality and normalized to have variance of one, such that the contribution of each to overall quality was equivalent. We then normalized the Q index to be on a scale of 0 (bad) to 100 (good) while preserving the overall variance in the raw data. Thus, differences in the scale from one year to the next, or between two types of facilities, can be interpreted as percentage point differences in quality. Our metric ensures that 1) each of the nine Star Rating components equally influenced the Q index and 2) overall absolute trends in quality were maintained such that changes in quality can be tracked over time. Patient survey data are not included in the Quality of Patient Care Star Rating, and therefore not included in our Q index.

We examined vaccination verification and hospital avoidance as stand-alone quality measures because they are important post-acute care processes related to overall quality, infection prevention, and other national priorities.^17,29 Vaccination verification is defined as the percent of patients for which influenza vaccination status was determined among all patients served by that agency during flu season.³⁰ To generate an outcome measure with higher scores indicating better outcomes, we calculated hospital avoidance from Home Health Compare acute care hospitalization data, and defined the measure as the percent of patients with no hospitalizations during the home health care episode.

Independent and confounding variables:

Key independent variables are ownership status (nonprofit or for-profit) and nurse staffing. We used two sets of measures for nurse staffing: 1) skill mix (defined as % Registered Nurses [RN], % Licensed Practical/Vocational Nurses [LPN], and % Aides); and, 2) in-house staffing (a binary measure for both aides and nursing services defined as staffed fully in-house or at least partially by contract).

To control for confounding, we quantified other binary measures, including: number of health service types provided (<5 or ≥5 of 13 service types tracked [e.g., nursing services, physical therapy]), hospital-based, was a sub-unit of another agency, operated subunits, part of a system of branch agencies, compliant with CMS program requirements at time of accreditation, accepted only Medicare patients, participated in Medicare hospice, and rural location.

We excluded all government-run agencies (n = 344 in 2016) and all observations (agency-years) that had incomplete Star Rating data (n = 17,912) or data on any of the HHA characteristics enumerated above (n = 18). Star Rating data were incomplete if an agency: 1) had fewer than 20 complete patient episodes, or 2) failed to report any of the nine Star Rating components in a year. The final dataset included 39,211 observations during the 5-year study period, including a maximum of 8,009 agencies in 2013 and a minimum of 7,670 in 2016. Agencies were located in the 50 states, Washington D.C., and 4 US territories (Guam, Puerto Rico, US Virgin Islands, and the Northern Mariana Islands).

Summary statistics were computed overall and by ownership status. Bivariate associations between the outcomes and agency characteristics were examined (data not shown). To test for trends over time, annual unadjusted means of the three quality measures were examined by ownership status, and differences were tested using student-t statistics. We specified ordinary least squared (OLS) regressions stratified by ownership status for each quality outcome. We normalized the staffing skill mix variables to have a standard deviation (SD) of one, so the regression coefficients can be interpreted as a percentage point change in the outcome resulting from a 1 SD change in the variable. We specified each model as a function of the agency-level characteristics enumerated above, as well as calendar year and state indicator. Huber-White standard errors were used to account for clustering at the agency level.

RESULTS

Table 1 presents summary statistics for 2016. Of the 7,670 agencies, 80.4% were for-profit and 19.6% nonprofit. Approximately 62.1% of the total nursing staff were RNs, with almost equal use of LPNs and aides (19.6% and 18.2% respectively). This distribution differed substantively by ownership, with for-profit agencies employing fewer RNs and more LPNs than nonprofits. Most agencies hired only in-house aides (88.2%) and nursing staff (91.6%), with relatively minor differences across ownership types. Only 12.4% of agencies provided fewer than 5 service types. Most agencies were in compliance with CMS program requirements during the most recent accreditation survey. Less than a quarter (23.2%) of HHAs served Medicare-only patients (as opposed to Medicare and Medicaid), and the vast majority were for-profit. There were large differences by ownership in the operation of Medicare hospices (i.e., 3.4% of for-profit compared to 25.3% of nonprofit). Nearly 16% of agencies were in rural counties and 8.1% were hospital-based; both were over-represented by nonprofits. Most agencies had no sub-units, fewer than 18% had branches, and these branches were slightly more likely to be operated by nonprofits than for-profits.

Table 1.

Summary Statistics, Overall and by Ownership (2016)

	All Agencies	For-profit Agencies	Nonprofit Agencies
N of Agencies	7,670	6,170 (80.4%)	1,500 (19.6%)
	Mean (SD)
Agency Characteristics
Staffing Skill Mix  % RN  % LPN  % Aide	62.1 (21.52) 19.6 (18.61) 18.2 (16.78)	59.9 (21.59) 22.1 (18.97) 18.0 (16.96)	71.3 (18.62) 9.3 (12.49) 19.5 (15.98)
	% of column total
In-house Staffinga  Aides  Nursing services	88.2 91.6	88.4 92.0	87.4 90.1
Provided <5 health service types	12.4	12.3	13.0
CMS compliance	94.1	94.1	94.3
Medicare-only	23.2	26.3	10.1
Participated in Medicare hospice	7.7	3.4	25.3
Rural location	15.7	11.6	32.4
Hospital-based	8.1	1.1	37.1
Agency Systems  Operated subunits  Sub-unit of other  Part of a system of branches	0.8 0.6 17.8	0.7 0.5 16.7	1.0 1.2 22.1

^aNote: Non-contracted services.

RN = registered nurses, LPN = licensed practical & vocational nurses.

Figure 1 presents trends in unadjusted means of the three quality measures over time, by ownership status. Differences between for-profits and nonprofits are significant in each year for all quality measures (p<0.001). The Q index monotonically increased approximately 7 percentage points for both for-profits and nonprofits (p<0.001), but nonprofits had higher quality each year. Vaccination verification grew for both for-profits and nonprofits (p<0.001); however, not only did nonprofits have higher vaccination verification rates in each year, but they improved more (i.e., about a 5 percent relative increase in nonprofits compared to for-profits, p<0.001). Hospital avoidance showed no change over time for either ownership type; however, for-profits slightly outperformed nonprofits each year (p<0.001).

Figure 1.

Annual trends in mean quality measures, by for-profit vs nonprofit status (2012 to 2016). For all graphs, blue lines represent for-profit agencies, and orange lines represent nonprofit agencies. A, Trends in mean composite Q index scores for for-profit and nonprofit home health agencies from 2012 to 2016a,b. B, Trends in mean influenza vaccination verification rates for for-profit and nonprofit home health agencies from 2012 to 2016a,b,c. C, Trends in mean hospital avoidance rates for for-profit and nonprofit home health agencies from 2012 to 2016a,b. aFor all graphs (panels A-C), differences by ownership are significant in each year (P < .001). bTrends over time are significant (P < .001) for the Q index and influenza vaccination (panels A and B), but not for hospital avoidance (panel C). cTime trends by ownership type are significantly different (P < .001) for vaccination verification (panel B).

Stratified OLS estimates are provided in Table 2. All coefficient estimates (β) cited below have p-values <0.001 unless otherwise indicated. In the Q index model, when compared to RN staffing, higher proportions of LPN and aide staffing were associated with lower quality for both ownership types (β ranging from −0.84 to −1.19). The use of in-house staffing for nursing services and aides was associated with higher overall quality in for-profit agencies (β = 1.06 and 1.67, respectively). Among nonprofit agencies, however, in-house staffing of nursing services and aides were not associated with the Q index. The trends in the Q index over time were large and similar in for-profit and nonprofit agencies (β = 6.97 and 7.08 in 2016, respectively).

Table 2:

Multivariable Regression Results, by Ownership

	Q Index		Vaccination Verification		Hospital Avoidance
	For-profit	Nonprofit	For-profit	Nonprofit	For-profit	Nonprofit
N of Agencies	31,369	7,842	31,369	7,842	31,369	7,842
	β (SE)
Agency Characteristics
Staffing Skill Mix
% LPN	−0.92 (0.10)	−0.84 (0.23)	−2.19 (0.21)	−1.46 (0.52)	0.003 (0.03)	−0.21 (0.10)
% Aide	−1.19 (0.10)	−0.96 (0.17)	−1.48 (0.23)	−0.36 (0.31)	0.10 (0.04)	−0.16 (0.08)
% RN	Referent
In-house Staffinga
Aides	1.67 (0.27)	0.19 (0.43)	2.21 (0.61)	0.74 (0.84)	−0.28 (0.09)	−0.21 (0.19)
Nursing services	1.06 (0.29)	−0.61 (0.38)	3.64 (0.69)	1.56 (0.70)	−0.44 (0.10)	0.14 (0.17)
Calendar Year
2012	Referent
2013	1.23 (0.10)	1.54 (0.12)	2.33 (0.24)	3.13 (0.25)	0.82 (0.06)	0.56 (0.11)
2014	1.73 (0.11)	2.10 (0.15)	0.08 (0.27)	1.71 (0.30)	0.78 (0.06)	0.61 (0.11)
2015	3.98 (0.13)	4.28 (0.17)	−1.12 (0.31)	1.69 (0.35)	0.45 (0.07)	0.24 (0.11)
2016	6.97 (0.14)	7.08 (0.18)	5.28 (0.33)	9.90 (0.37)	−0.03 (0.07)	−0.05 (0.12)
R-squared	0.1872	0.2348	0.1013	0.4279	0.0641	0.0850
Adjusted R-squared	0.1854	0.2281	0.0993	0.4228	0.0620	0.0770
F-statistics	69.593	31.792	36.945	81.791	29.182	8.132

^aNon-contracted services.

Bolded values indicate significance at α<0.05.

Note: All models control for <5 health service types provided (versus 5+), was a sub-unit, operated subunits, part of a system of branch agencies, compliant with CMS program requirements, hospital-based, Medicare-only, participated in Medicare hospice, and rural location.

RN = registered nurses, LPN = licensed practical & vocational nurses.

The vaccination verification model estimates were similar to the Q index model, with some exceptions. When compared to RN staffing, higher proportions of aide and LPN staffing were associated with reduced vaccination verification (β = −1.48 and −2.19, respectively) in for-profits; in nonprofit agencies, the association was strong for LPN relative to RN staffing (β = −1.46), but there was no association found with higher proportions of aides. Like the Q index model, improved vaccination verification was positively associated with in-house staffing for both aides and nursing services in for-profit agencies (β = 2.21 and 3.64, respectively). In nonprofit agencies, better vaccination verification was associated with in-house nursing services (β = 1.56), but not with in-house staffing of aides. Both for-profits and nonprofits obtained substantial improvements in vaccination verification from 2012 to 2016, but the improvements were largely realized in 2016, and more evident in nonprofit (β = 9.90) than in for-profit agencies (β = 5.28).

The hospital avoidance models showed substantively different patterns. Compared to RN staffing, higher proportions of LPN and aide staffing were associated with lower rates of hospital avoidance (higher rates of hospitalization) for nonprofit agencies (β = −0.21 and −0.16, respectively). However, in for-profit agencies, having more aides relative to RNs was associated with a higher rate of hospital avoidance (lower rates of hospitalization), though the effect was small (β = 0.10), and there was no association found for LPN relative to RN staffing. Among for-profit agencies, in-house staffing of both aides and nursing services was associated with more hospitalization; neither was significant in the nonprofit model. With regard to trends, there were no statistically significant changes from the beginning to end of the study period in rates of hospital avoidance for nonprofit and for-profit agencies; however, from 2013 to 2015, rates were slightly higher relative to 2012 (p<0.001) for both for-profit and nonprofit agencies.

DISCUSSION

We investigated trends in home health care quality over time and factors associated with these trends, and found important results. First, the Q index and vaccination verification rates improved substantially from 2012 to 2016, with the largest increases occurring in 2016 after public reporting of the Quality of Patient Care Star Rating began. Second, the quality of home health services varied by ownership, both in terms of trends (larger improvements for nonprofit agencies), and as a function of nurse staffing.

External factors may be responsible for the trends we found. CMS has developed a number of quality and value-based purchasing (VBP) programs that are impacting the home health care industry.^31,32 Regardless of whether consumers used the Star Ratings to choose HHAs, quality improvements may have been influenced by agencies’ response to VBP and public reporting.³³ In hospitals, mandatory public reporting of healthcare-associated infections was associated with improvement in both patient outcomes and hospital processes.³⁴ Increased attention to quality and infection prevention in the home health sector may have a similar effect. Others have found that for-profit agencies are more likely than nonprofits to respond to financial incentives.³⁵ Our results do show stronger associations between quality metrics and staffing measures for the for-profit agencies, but we also found that nonprofit agencies outperformed for-profits on overall quality and vaccination verification, though not on hospital avoidance.

Despite improving trends, CMS decided to drop influenza vaccination verification from the Star Ratings beginning in April 2018. The importance of influenza vaccination leaves us concerned about its exclusion, especially with new changes to the HHA Conditions of Participation, which call for a quality assessment and performance improvement committee that includes infection prevention issues, and an infection prevention and control program with standards focused on best practices.¹⁸ Vaccination verification is the only current home health care process measure that directly reflects infection prevention and control. Considering that 9.7% unplanned hospitalizations among home health care patients were related to respiratory infections, inclusion of infection prevention and control quality measures is important.¹⁴

Quality metrics are important but limited by the underlying data and the extent to which they adequately cover relevant dimensions of quality.³³ The completeness of Home Health Compare data is a limitation of our measures. In particular, Home Health Compare data include no information on agencies with fewer than 20 complete patient episodes in a year, and required complete data on six of the nine component measures for the Star Rating.³⁶ The Q index requires complete data on all nine components, and thus has more missing data. In addition, eight of the nine measures are self-reported by agencies, potentially raising concern regarding their accuracy; however, HHAs have strong incentives to provide accurate data, particularly those participating in the VBP pilot program. A recent report suggests that agencies are educating staff based on Star Rating performance with the goal of increasing accuracy, specifically around functionality measures.³⁷ Last, CMS annually risk-adjusts the five outcome measures, and we must trust that these are appropriately specified models.

Another limitation is our skill mix measure. In general, health care staffing is measured as skill mix and/or hours per patient-day.³⁸ Our data do not include the number of patients served or number of visits made; therefore, we could not calculate nursing hours per patient-day. The term “skill mix” is the mix of posts, grades or occupations needed for each job within an organization.³⁹ Our skill mix measure should reflect agency-level variation in the distribution of nursing profession classifications in home health care; it may also be a proxy for the HHA case-mix (i.e., increased use of aides may reflect patients that need more assistance). Furthermore, because we are limited to relative staffing measures, we cannot control for staffing in other disciplines (e.g., physical therapy). However, approximately 70% of the home healthcare workforce is comprised of nurses (including RNs, LPNs and aides), making the distribution of nurse staffing important to examine.⁴⁰ Nevertheless, more research investigating the impact of various HHA personnel with more precise measures would be beneficial.

Although hospital admission (or readmission) is a common quality metric, we found that hospital avoidance did not change much from 2012 to 2016. Furthermore, emergency department visits and observational stays, which may act as substitutes for hospital admissions, have been increasing and are not included in hospital avoidance.⁴¹ Their increases may be related to incentives to decrease hospital readmissions and may not reflect real changes in patient care quality. Further research regarding the roles of emergency department visits and observational stays is needed.

The Quality of Patient Care Star Rating is designed to identify high and low performing agencies in a given year. To do so, the Star Rating aggregates an agency’s performance on each of the components relative to all other agencies in that year. As a result, it has no absolute standard and is not a consistent metric over time. Thus, the ratings cannot be used to evaluate trends in the industry or the role of agency characteristics in quality improvements. Conversely, the Q index is a cardinal metric that is anchored by absolute values of the nine underlying components, thus allowing for both longitudinal and cross-sectional comparisons. It is not a function of an agency’s relative placement on metrics each year and, therefore, it is not optimally designed to evaluate a single agency’s quality relative to others. Instead, it conveys the actual performance in each metric (e.g. the fraction of patients for whom vaccination status was verified or the fraction of patients who avoided hospitalization) consistently across years.

In summary, the Q index allows for assessment of trends and factors related to those trends over time in home health care. Nurse staffing is an important factor in the quality of care provided by HHAs. The overall market is driven by for-profit agencies, but their characteristics and quality outcomes differ from nonprofit agencies. More research is needed with increased focus on quality in the home health care industry.