Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Aug 7;22(10):2021–2025.e1. doi: 10.1016/j.jamda.2021.07.034

U.S. Nursing Home Quality Ratings Associated with COVID-19 Cases and Deaths

Saif Khairat a,b,c,, Lauren C Zalla d, Julia Adler-Milstein e, Christine E Kistler c,f
PMCID: PMC8346327  PMID: 34454922

Abstract

Objectives

To inform future policies and disaster preparedness plans in the vulnerable nursing home setting, we need greater insight into the relationship between nursing homes’ (NHs’) quality and the spread and severity of COVID-19 in NH facilities. We therefore extend current evidence on the relationships between NH quality and resident COVID-19 infection rates and deaths, taking into account NH structural characteristics and community characteristics.

Design

Cross-sectional study.

Setting and Participants

15,390 Medicaid- and Medicare-certified NHs.

Methods

We obtained and merged the following data sets: (1) COVID-19 weekly data reported by each nursing home to the Centers for Disease Control and Prevention’s National Healthcare Safety Network, (2) Centers for Medicare & Medicaid Services Five Star Quality Rating System, (3) county-level COVID-19 case counts, (4) county-level population data, and (5) county-level sociodemographic data.

Results

Among 1-star NHs, there were an average of 13.19 cases and 2.42 deaths per 1000 residents per week between May 25 and December 20, 2020. Among 5-star NHs, there were an average of 9.99 cases and 1.83 deaths per 1000 residents per week. The rate of confirmed cases of COVID-19 was 31% higher among 1-star NHs compared with 5-star NHs [model 1: incidence rate ratio (IRR) 1.31, 95% confidence interval (CI) 1.23-1.39], and the rate of COVID-19 deaths was 30% higher (IRR 1.30, 95% CI 1.20, 1.41). These associations were only partially explained by differences in community spread of COVID-19, case mix, and the for-profit status and size of NHs.

Conclusions and Implications

We found that COVID-19 case and death rates were substantially higher among NHs with lower star ratings, suggesting that NHs with quality much below average are more susceptible to the spread of COVID-19. This relationship, particularly with regard to case rates, can be partially attributed to external factors: lower-rated NHs are often located in areas with greater COVID-19 community spread and serve more socioeconomically vulnerable residents than higher-rated NHs.

Keywords: Nursing homes, infectious diseases, quality improvement, COVID-19, deaths


Between May and December 2020, more than 92,000 deaths from the novel coronavirus disease 2019 (COVID-19) occurred in nursing homes (NHs), accounting for 30% of all COVID-19 deaths in the United States.1 , 2 However, within these high levels, there was substantial variation in both deaths and case rates across NHs, suggesting that some were either better positioned or better able to respond to the pandemic. We have a limited understanding of which factors may drive such variation. In 8 states, NHs with nurse staffing shortages were found to be more susceptible to the spread of COVID-19 during the initial phase of the pandemic.3 NHs with high proportions of nonwhite residents experienced COVID-19 death counts that were 3 times higher than those of facilities with the highest proportions of white residents.4 Significant inequities are present in COVID-19 infection rates in NHs with larger proportions of racial-minority residents.5 Prior work has found that NHs with higher quality ratings had lower COVID-19 cases, particularly when adjusted for NH size and resident characteristics, but these studies were conducted early in the pandemic and only within specific regions of the United States.3 , 6, 7, 8 It is therefore not clear whether these relationships persist at a national level and at a later period after NHs were able to address some of the immediate challenges at the start of the pandemic.9, 10, 11, 12 To the extent that we find that these relationships persist, it suggests that COVID-19 case and death rates may be associated with NH quality ratings. Therefore, we hypothesized that COVID-19 case and death rates would differ based on NH star rating even after adjusting for differences in NH and community characteristics.

To inform future policies and disaster preparedness plans in the vulnerable nursing home setting, we need greater insight into the relationship between NH quality and the spread and severity of infectious diseases like COVID-19 in NHs. We therefore extend current evidence on the relationships between NH quality [based on the Centers for Medicare & Medicaid Services (CMS) Five Star Quality Rating System] and resident COVID-19 infection rates and deaths, taking into account NH structural characteristics and community characteristics.

Methods

We conducted a cross-sectional study of NHs in the United States. We merged data from the following sources: (1) COVID-19 weekly data reported by each nursing home to the Centers for Disease Control and Prevention’s National Healthcare Safety Network between May 25 and December 20, 20201; (2) NH star ratings published by the CMS Five Star Quality Rating System on October 8, 202013; (3) county-level COVID-19 case counts from the New York Times 14; (4) county-level population data from the US Census Bureau15; and (5) county-level sociodemographic data from the US Census Bureau16; and the Health Resources and Services Administration.17 Because of the COVID-19 pandemic, the CMS held all NH star ratings constant as of April 29, 2020. We used the CMS Five Star Quality Rating data provided before the hold—as of October 1, 2020.13 We included data from NHs that submitted data to the National Healthcare Safety Network and passed its quality assurance check at least once in 2020. Data quality checks are performed regularly by the Centers for Disease Control and Prevention and CMS to identify data entry errors, such as facilities entering cumulative case counts over time instead of incident cases, or other outliers.1 Facilities that submitted erroneous data were given a value “N” in the “Passed Quality Assurance Check” column. For this analysis, we excluded all data that did not pass the quality assurance check.

The primary outcomes of the study were confirmed cases of COVID-19 and deaths from COVID-19 per 1000 NH residents per week between May 25 and December 20, 2020. The primary exposure was NH star rating (1-5). We calculated “excess” cases and deaths among 1-, 2-, 3-, and 4-star NHs as (total confirmed cases or deaths) – [(confirmed case or death rate at 5-star NHs) × total resident-weeks].

In estimating the association between NH star rating and NH burden of COVID-19 cases and deaths during the prevaccination period in the United States, we sought to examine potential confounders likely to be associated with both star rating and burden of COVID-19 by expanding on prior literature that used 3 community characteristics to examine COVID-19 case rates in high- and low-performing NHs during the initial periods of the pandemic and within specific regions of the United States.3 First, to address confounding due to greater community spread in the neighborhoods surrounding lower-rated NHs, we considered total county-level COVID-19 cases, minus county-level cases at NHs, per 100,000 population in 2020. Second, to address confounding by case mix, defined as the characteristics of NH residents that may be associated with higher infection and mortality rates at lower-rated NHs, we considered sociodemographic characteristics measured at the county level, including median household income, percentage less than high school education, percentage non-Hispanic white, and designation as a Medically Underserved Area. Finally, we considered 2 factors hypothesized to be highly correlated with both NH rating and NH burden of COVID-19: for-profit status and size of the NH.

We fit a series of negative binomial regression models to estimate the associations between NH star rating and rates of confirmed cases and deaths from COVID-19. We estimated unadjusted associations in model 1. In model 2, we adjusted for community spread of COVID-19. In model 3, we further adjusted for county-level sociodemographic characteristics (as proxies for case mix). Finally, in model 4, we adjusted for NH for-profit status and number of certified beds.

For each model, we present incidence rate ratios (IRRs) comparing the average rate of COVID-19 cases or deaths per 1000 residents per week among 1-, 2-, 3-, and 4-star NHs to that of 5-star NHs. In a supplemental analysis, we calculated IRRs for all possible comparisons of NH star rating (eg, 1-star vs 4-star, 1-star vs 2-star, etc). These comparisons are adjusted for all covariates included in model 4 of the main analysis; results are reported in Supplementary Table 1. To quantify the uncertainty around our estimates, we estimated robust standard errors and constructed 95% confidence intervals (CIs). CIs that exclude the null value of 1 indicate that a given estimate is statistically significant at an alpha level of 0.05. CIs that include 1 indicate a lack of statistical significance (ie, P ≥ .05).

This study was deemed exempt from review by the institutional review board at the University of North Carolina at Chapel Hill. All analyses were conducted using SAS, version 9.4 (SAS Institute, Cary, NC).

Results

Of 15,390 NHs, 12,808 NHs (83%) submitted data and passed the quality assurance in all 30 weeks of the reporting period; 14,281 (93%) submitted data and passed the quality assurance check at least 29 weeks out of 30. Of the 14,944 NHs that passed the quality assurance check at least once, 14,690 (98.3%) had a star rating (Table 1 ). A greater proportion of 1-star NHs were for-profit (87%) compared with 2-star (79%), 3-star (74%), 4-star (68%), and 5-star NHs (54%). On average, 1-star NHs had 36 more certified beds and 1.2 fewer nursing staff hours per resident than 5-star NHs. NHs with lower star ratings tended to be located in counties with lower median household income, a lower proportion of the population with a high school diploma, and a lower proportion of the population identifying as non-Hispanic white. Lower-rated NHs were also more likely than higher-rated NHs to report staff shortages in 2020. We did not observe substantial differences in reported shortages of personal protective equipment.

Table 1.

Characteristics of Nursing Homes and Occurrence of COVID-19 Cases and Deaths by Star Rating, May 25–December 20, 2020

Star Rating
1 2 3 4 5 All
Number of nursing homes 2086 2767 2662 3237 3938 14,690
Community spread of COVID-19: County-level cases per 100,000 population in 2020 6429 (2186) 6337 (2334) 6390 (2242) 6341 (2376) 6149 (2370) 6310 (2318)
County-level sociodemographic characteristics, mean (SD)
 Median household income $55,387 ($23,623) $59,657 ($26,557) $58,965 ($26,044) $61,156 ($27,776) $66,412 ($31,121) $61,067 ($27,890)
 Percentage less than high school education 13.4 (8.6) 12.5 (8.8) 12.7 (8.6) 11.8 (8.1) 10.8 (8.4) 12.0 (8.5)
 Percentage non-Hispanic white 72.5 (25.1) 75.3 (24.2) 77.2 (23.2) 78.3 (23.0) 79.5 (21.7) 77.0 (23.4)
 Percentage aged ≥65 y 30.7 (12.6) 31.6 (14.2) 32.2 (13.8) 33.3 (15.1) 34.4 (16.9) 32.7 (15.0)
 Medically Underserved Area, % 42 39 44 42 36 40
NH characteristics, %
 Percentage for-profit 87 79 74 68 54 70
 Number of certified beds 125 (57) 116 (59) 112 (61) 104 (58) 89 (60) 107 (61)
Staffing and PPE shortages
 Percentage of weeks with staff shortage
 Nursing staff 22 (32) 18 (30) 16 (28) 15 (28) 12 (25) 16 (28)
 Clinical staff 3 (8) 3 (9) 2 (8) 2 (7) 2 (6) 2 (8)
 Aides 24 (33) 21 (32) 19 (31) 18 (30) 15 (28) 19 (30)
 Other staff 12 (23) 10 (22) 9 (21) 9 (21) 8 (19) 9 (21)
 Percentage of weeks with PPE shortage
 N95 masks 13 (24) 14 (25) 13 (25) 13 (25) 13 (25) 13 (25)
 Surgical masks 7 (18) 8 (20) 8 (19) 7 (19) 7 (19) 7 (19)
 Eye protection 7 (17) 7 (18) 7 (19) 7 (18) 7 (18) 7 (18)
 Gowns 8 (18) 9 (20) 9 (20) 10 (20) 10 (21) 9 (20)
 Gloves 4 (13) 4 (13) 4 (12) 4 (12) 4 (12) 4 (13)
 Hand sanitizer 4 (13) 4 (13) 4 (13) 3 (12) 4 (13) 4 (13)
COVID-19 cases and deaths among residents, n
 Total confirmed cases 65,152 78,528 75,008 81,811 75,559 376,058
 Confirmed cases per 1000 residents per week 13.19 12.32 12.71 11.76 9.99 11.77
 Excess confirmed cases 13,380 12,097 12,606 10,053 0 48,136
 Total deaths 11,663 13,783 13,336 14,102 13,159 66,043
 Deaths per 1000 residents per week 2.42 2.22 2.33 2.06 1.83 2.13
 Excess deaths 2647 2214 2498 1605 0 8964

COVID-19, coronavirus disease 2019.

Unless otherwise noted, values are mean (standard deviation).

On a scale of 1 to 5, with 1 being the lowest and 5 being the highest. Overall quality rating was used, and ratings were based on pre–COVID-19 CMS star ratings. NH facilities without an available overall star rating were excluded.

Based on the Nursing Home COVID-19 Public File, which includes data reported by NHs to the Centers for Disease Control and Prevention’s National Healthcare Safety Network system from May 25 to December 20, 2020. Based on data submitted by NHs that passed the quality assurance check at least once during the reporting period and had a CMS star rating.

As of December 20, 2020, there were 376,058 COVID-19 cases and 66,043 COVID-19 deaths among NH residents in the United States. Among 1-star NHs, there were an average of 13.19 cases and 2.42 deaths per 1000 residents per week between May 25 and December 20, 2020 (Figure 1 ). Among 5-star NHs, there were an average of 9.99 cases and 1.83 deaths per 1000 residents per week. If the case and death rates among all NHs had been equivalent to the case and death rates of 5-star NHs, we would have observed 48,136 (12.8%) fewer cases and 8964 (13.6%) fewer deaths between May 25 and December 20, 2020.

Fig. 1.

Fig. 1

COVID-19 confirmed case and death rates among nursing homes by star rating, May 25–December 20, 2020.

The rate of confirmed cases of COVID-19 was 31% higher among 1-star NHs compared with 5-star NHs (model 1: IRR 1.31, 95% CI 1.23-1.39). This statistically significant association was partially explained by differences in community spread of COVID-19, case mix, and the for-profit status and size of NHs. After adjusting for these factors, the case rate remained 17% higher among 1-star NHs compared with 5-star NHs (model 2: IRR 1.17, 95% CI 1.10, 1.25). A similar pattern was observed for 2-star, 3-star, and 4-star NHs, all of which had significantly higher case rates than 5-star NHs even after adjustment for community spread, case mix, for-profit status, and size (Table 2 ). Supplementary Table 1 presents all pairwise comparisons by star rating after adjustment for the same set of covariates. Unsurprisingly, IRRs comparing NHs with more similar star ratings tend to be smaller. Although there remain statistically significant differences for 1- and 3-star compared with 4-star NHs, comparisons of 2- vs 4-star, 1- vs 3-star, 2- vs 3-star, and 1- vs 2-star NHs are not statistically significant.

Table 2.

Occurrence of COVID-19 Cases and Deaths per 1000 Resident-Weeks Among 1-, 2-, 3-, and 4-Star NHs Compared With 5-Star NHs, May 25–December 20, 2020

CMS Overall Star Rating
1
2
3
4
5
IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI
COVID-19 cases per 1000 resident-weeks
 Model 1 1.31 1.23, 1.39 1.22 1.16, 1.30 1.26 1.19, 1.34 1.17 1.11, 1.24 1
 Model 2 1.28 1.20, 1.36 1.20 1.13, 1.27 1.23 1.16, 1.30 1.14 1.08, 1.21 1
 Model 3 1.25 1.18, 1.33 1.18 1.11, 1.24 1.19 1.13, 1.26 1.12 1.06, 1.18 1
 Model 4 1.17 1.10, 1.25 1.13 1.06, 1.19 1.15 1.09, 1.22 1.09 1.03, 1.15 1
COVID-19 deaths per 1000 resident-weeks
 Model 1 1.30 1.20, 1.41 1.20 1.12, 1.30 1.26 1.17, 1.35 1.12 1.05, 1.20 1
 Model 2 1.27 1.17, 1.37 1.18 1.10, 1.27 1.22 1.13, 1.31 1.10 1.02, 1.18 1
 Model 3 1.28 1.18, 1.39 1.18 1.10, 1.27 1.21 1.12, 1.30 1.09 1.02, 1.17 1
 Model 4 1.26 1.16, 1.37 1.17 1.09, 1.26 1.20 1.11, 1.29 1.08 1.01, 1.16 1

Model 1 is unadjusted; model 2 adjusts for total county-level COVID-19 cases minus county-level cases at NHs per 100,000 population in 2020; model 3 further adjusts for county-level median household income, percentage less than high school education, percentage non-Hispanic white, percentage aged ≥65 years, and designation as a Medically Underserved Area; model 4 additionally adjusts for NH for-profit status and number of certified beds.

The rate of deaths from COVID-19 was 30% higher among 1-star NHs compared with 5-star NHs (model 3: IRR 1.30, 95% CI 1.20-1.41). This association was only slightly attenuated after accounting for community spread, case mix, and the for-profit status and size of NHs (model 4: IRR 1.26, 95% CI 1.16, 1.37). A similar pattern was observed for 2-star, 3-star, and 4-star NHs. Pairwise comparisons of death rates by star rating reveal statistically significant differences between 1-, 2-, and 3-star NHs compared with 4-star NHs, but no significant differences among 1-, 2-, and 3-star NHs. This suggests that our results are not being driven only by 5-star NHs.

Discussion

Our cross-sectional study of NHs in the United States examined differences in COVID-19 case and death rates by CMS Star Rating. We found that COVID-19 case and death rates were substantially higher among NHs with lower star ratings, suggesting that NHs with quality much below average are more susceptible to the spread of COVID-19. If all NHs had achieved the lower case and death rates of 5-star NHs, nearly 13% of all cases and 14% of all deaths among NH residents in the United States would have been averted.

We found that the relationship between star rating and COVID-19 case rates was partially attributable to external factors: lower-rated NHs are often located in areas with greater COVID-19 community spread and serve more socioeconomically vulnerable residents than higher-rated NHs. Additionally, lower-rated NHs have more beds, making them more susceptible to outbreaks, and are more likely to be for-profit than higher-rated NHs, whose affiliations with nonprofit organizations and academic medical centers may have improved their access to personal protective equipment and other resources to prevent the spread of COVID-19. However, significant associations between NH star rating and COVID-19 case and death rates remained even after accounting for these factors, suggesting that 1- or 5-star rating is a revealing indicator of NH vulnerability to COVID-19. Moreover, differences in death rates by star rating were only slightly attenuated after accounting for these factors, which suggests either uncontrolled confounding, such as differences in individual patient characteristics and health conditions, or a stronger relationship with NH quality for deaths than for cases.

We observed differences in the percentage of weeks in which NHs experienced staff shortages by star rating, suggesting that staff shortages may have contributed to higher case and death rates among lower-rated NHs. Lower-rating NHs may experience substantially higher rates of staff shortages because they include a larger number of occupied beds and because they are typically for-profit in nature. We did not observe differences in shortages of personal protective equipment between NHs with different star ratings. Future disaster preparedness plans may consider prioritizing the distribution of human and other resources to lower-rated NHs, which experience higher infection and death rates than higher-rated NHs, even among NHs located in areas with similar levels of community spread, with similar resident case mix, and of similar size and for-profit status. Although the allocation of more resources may be looked on as reward to NHs with below average performance, the striking difference in infection and mortality rates shows the importance of sufficient staffing and resource for a better disaster response among lower-rated NHs.

Our findings corroborate previous evidence of an association between CMS quality rating and COVID-19 cases during the initial periods of the pandemic within specific regions of the United States.3 Previous studies have also reported that COVID-19 infection rates are higher in NHs with a larger proportion of Medicaid patients, in NHs with high shortages of staff, and in NHs with a lower proportion of non-Hispanic white residents.3 , 5 Additionally, our findings confirm prior reports that because NHs are tied to geographic locations, county characteristics play a role in health disparities within NHs related to residential segregation.18 Future studies should further investigate the relationship between the individual components of the 5-star system and COVID-19 reported outcomes. For example, it will be important to understand whether staffing rating, measured based on the combination of nursing hours per resident-day and total staffing hours per resident-day, is associated with COVID-19 case rates among NH residents as this suggests a potential mechanism (increasing staffing) through which NHs could achieve better outcomes in a pandemic.

Limitations

Nursing home populations are typically aging adults with chronic conditions, which may be a confounding factor to COVID-19 deaths. CMS quality assurance checks were not validated for accuracy. Additionally, we used county-level rather than resident-level data to account for confounding by resident case mix, which allowed us to account for socioeconomic status and important contextual factors that determine vulnerability to COVID-19 but may result in residual confounding by resident age and comorbidities, particularly given that 1-star NHs typically serve sicker patients. To the extent this is true, the reported COVID-19 death rates in 1-star NH may be confounded by the poor health status of residents. To the extent this is true, the reported COVID-19 death rates in 1-star NH may be confounded by the poor health status of residents. Another limitation was the absence of the proportion of Medicaid patients within NHs, which was not included in the data provided by CMS. Because private insurers pay more for NH residents compared with Medicaid, NHs with a larger proportion of Medicaid patients may have different characteristics that may influence COVID-19 case and death rates.19

Conclusions and Implications

This cross-sectional study of NHs in the United States provides real-time information about the association between the quality of NHs and their vulnerability to COVID-19. The spread of COVID-19 within the nursing home’s neighborhood, case mix, and the for-profit status and size of beds were contributing factors to the higher rates of COVID-19 cases in lower star rating NHs. These factors alone do not account for the strong association between NH star rating and rates of COVID-19 deaths among NH residents. Our findings suggest that lower-rated NHs are more vulnerable to disease outbreaks and should be prioritized in staffing and vaccination efforts as well as other public health measures to prevent the spread of COVID-19, and to inform future disaster preparedness efforts.

Footnotes

The authors declare no conflicts of interest.

Supplementary Data

Supplementary Table 1.

Adjusted Pairwise Comparisons of the Occurrence of COVID-19 Cases and Deaths per 1000 Resident-Weeks by NH Star Rating, May 25–December 20, 2020

CMS Overall Star Rating
1
2
3
4
5
IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI
COVID-19 cases per 1000 resident-weeks 1.17 1.10, 1.25 1.13 1.06, 1.19 1.15 1.09, 1.22 1.09 1.03, 1.15 1
1.07 1.01, 1.14 1.03 0.97, 1.09 1.06 1.00, 1.12 1
1.02 0.95, 1.09 0.98 0.92, 1.04 1
1.04 0.98, 1.11 1
COVID-19 deaths per 1000 resident-weeks
1.26 1.16, 1.37 1.17 1.09, 1.26 1.20 1.11, 1.29 1.08 1.01, 1.16 1
1.16 1.07, 1.26 1.08 1.00, 1.16 1.10 1.02, 1.19 1
1.05 0.97, 1.15 0.98 0.91, 1.06 1
1.08 0.99, 1.17 1

CMS, Centers for Medicare & Medicaid Services; CI, confidence interval; IRR, incidence rate ratio.

Adjusted for total county-level COVID-19 cases minus county-level cases at NHs per 100,000 population in 2020, county-level median household income, percentage less than high school education, percentage non-Hispanic white, percentage aged ≥65 years, designation as a Medically Underserved Area, NH for-profit status, and number of certified beds.

References


Articles from Journal of the American Medical Directors Association are provided here courtesy of Elsevier

RESOURCES