Associations Between Hospital Characteristics, Measure Reporting, and the Centers for Medicare & Medicaid Services Overall Hospital Quality Star Ratings

Abstract

This study uses CMS Overall Hospital Quality Star Rating data to estimate associations between hospital characteristics, number and types of measures reported, and the CMS Hospital Quality Star Ratings.

The Centers for Medicare & Medicaid Services (CMS) introduced the Overall Hospital Quality Star Ratings in July 2016 to provide consumers a summary rating of hospital performance using measures already reported to CMS. Hospitals achieve 1 to 5 stars based on 62 measures across 7 weighted measure groups relating to health care quality: mortality, readmissions, safety, patient experience, efficient use of imaging, effectiveness of care, and timeliness of care. To receive a star rating, hospitals must report at least 3 of the 7 measure groups and at least 1 Outcome group (mortality, readmissions, or safety). Within each measure group, at least 3 individual measures must be reported for inclusion of that group. If a hospital does not submit enough measures for a group to be included, the weight from that group is redistributed to the other groups for which the hospital has sufficient data.

Concerns have been raised regarding the measures, methodology, and potential bias toward or against certain hospital types. To assess these concerns, we evaluated associations between hospital characteristics, number and types of measures reported, and the star ratings.

Methods

This study was deemed nonhuman subjects research by the Northwestern University institutional review board. Data from the CMS Overall Hospital Quality Star Ratings database (obtained July 27, 2016), American Hospital Association 2014 Annual Survey of Hospitals, and CMS Fiscal Year 2015 Payment Update Impact File were merged. We examined associations between hospital characteristics (Table), number of individual measures reported, number of group categories (Outcome or other) reported, and the star ratings (4-5 stars [high] vs 1-3 stars [low]) using χ² analysis and Kruskal-Wallis tests. Two-sided P values less than .05 were used to indicate statistical significance. Analyses were performed using Statistical Analysis Software (SAS Institute), version 9.4.

Table. Hospital Characteristics by Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings.

	Low CMS Star Rating (1-3 Stars)		High CMS Star Rating (4-5 Stars)		P Valuea
	No. of Hospitals	% (95% CI)	No. of Hospitals	% (95% CI)
Total hospitals	2593	72.3 (70.7-73.7)	998	27.7 (26.3-29.3)
Hospital type
Major teaching	192	84.2 (79.5-88.9)	36	15.8 (11.1-20.5)	<.001
Other teaching	829	81.2 (78.8-83.6)	192	18.8 (16.4-21.2)
Community	1199	69.8 (67.6-71.9)	520	30.2 (28.1-32.4)
Critical access	363	66.7 (62.8-70.7)	181	33.3 (29.3-37.2)
Specialtyb	10	12.7 (5.3-20.0)	69	87.3 (80.0-94.7)
No. of beds
<100	949	66.0 (63.6-68.5)	488	34.0 (31.5-36.4)	<.001
100-299	1062	76.7 (73.4-77.9)	341	24.3 (22.1-26.6)
300-499	367	75.8 (72.0-79.6)	117	24.2 (20.4-28.0)
≥500	215	80.5 (75.7-85.3)	52	19.4 (14.7-24.2)
Percentage of DSH paymentsc
Q1 (≤18.1)	433	57.1 (53.6-60.6)	325	42.8 (39.4-46.4)	<.001
Q2 (18.2-26.0)	532	70.3 (67.0-73.5)	225	29.7 (26.5-33.0)
Q3 (26.1-35.5)	583	76.9 (73.9-79.9)	175	23.1 (20.1-26.1)
Q4 (>35.5)	669	88.4 (86.1-90.7)	88	11.6 (9.3-13.9)
Joint Commission accredited
Yes	1943	72.6 (68.2-74.1)	734	27.4 (25.9-31.8)	.39
No	650	71.1 (70.9-74.3)	264	28.9 (25.7-29.1)
No. of Outcome measure groups reportedd
1 of 3	117	54.4 (47.8-61.1)	98	45.6 (38.9-52.2)	<.001
2 of 3	474	68.8 (65.3-72.3)	215	31.2 (27.7-34.7)
3 of 3	2002	74.5 (72.9-76.2)	685	25.5 (23.8-27.1)
No. of other groups reportede
0 of 4	1	50.0 (0.0-100.0)	1	50.0 (0.0-100.0)	.30
1 of 4	124	79.0 (72.6-85.4)	33	21.0 (14.6-27.4)
2 of 4	241	70.3 (65.4-75.1)	102	29.7 (24.9-34.6)
3 of 4	380	72.7 (68.8-76.5)	143	27.3 (23.5-31.2)
4 of 4	1847	72.0 (70.2-73.7)	719	28.0 (26.3-29.8)
Total measures reported, No.
Q1 (<39)	599	66.9 (63.8-70.0)	296	33.1 (30.0-36.2)	<.001
Q2 (39-52)	719	74.0 (71.3-76.8)	252	26.0 (23.2-28.7)
Q3 (53-57)	733	75.7 (73.0-78.4)	235	24.3 (21.6-27.0)
Q4 (>57)	542	71.6 (68.4-74.8)	215	28.4 (25.2-31.6)

Abbreviation: DSH, disproportionate share hospital.

^aχ² test.

^bSpecialty hospitals self-identified as cardiac, orthopedic, surgical, cancer, or other specialty primary service type.

^cCritical access hospitals are excluded from the CMS Impact File due to payment through alternative mechanism.

^dOutcome groups defined by CMS consisted of mortality, readmissions, and safety. To be eligible to receive a CMS star rating, hospitals had to report at least 3 measures in at least 1 Outcome group.

^eOther groups defined by CMS consisted of patient experience, efficient use of medical imaging, effectiveness of care, and timeliness of care.

Results

Of 3591 hospitals receiving a star rating, 4 or 5 stars were awarded to 15.8% of major teaching hospitals, 18.8% of other teaching hospitals, 30.2% of community hospitals, 33.3% of critical access hospitals, and 87.3% of specialty hospitals (P < .001 for all compared with specialty hospitals) (Table). Smaller hospitals more frequently achieved a high star rating compared with larger hospitals (No. of beds: <100, 34.0%; 100-299, 24.3%; 300-499, 24.2%; ≥500, 19.4%; P < .001). Hospitals with lower disproportionate share payments achieved a high star rating more frequently (Q1, 42.8%; Q2, 29.7%; Q3, 23.1%; Q4, 11.6%; P < .001). Specialty hospitals comprised 2.2% of hospitals but accounted for 41.1% of 5-star hospitals.

Hospitals reported a mean of 46.6 (75.2%) of 62 possible measures. Major teaching (mean, 56.9), other teaching (mean, 52.1), and community hospitals (mean, 49.5) reported approximately twice as many measures as critical access (mean, 25.9) and specialty hospitals (mean, 26.5; P < .001) (Figure). Hospitals reporting more Outcome measures less frequently received a high star rating: high star ratings were given to 46.9% of hospitals reporting the minimum threshold of 1 Outcome measure group, 31.2% of hospitals reporting 2 Outcome measure groups, and 25.5% of hospitals reporting all 3 Outcome measure groups (P < .001) (Table). Star ratings were based on the minimum eligibility threshold of only 3 of 7 measure groups for 49.4% of specialty and 33.3% of critical access hospitals; whereas, 94.3% of major teaching, 80.8% of other teaching, and 96% of community hospitals reported all 7 measure groups.

Figure. Mean No. of Hospital Quality Measures Reported (of 62 Total Possible Measures) by Hospital Type^a.

^aError bars represent 95% CIs. Includes hospitals that met the reporting threshold of at least 3 measures within at least 1 Outcome group (mortality, readmissions, safety). Using the Kruskal-Wallis test, P<.001 for all pairwise comparisons except between critical access and specialty hospitals (P = .96).

Discussion

Hospitals less frequently received a high star rating if they were larger, academic hospitals or cared for a higher proportion of disproportionate share patients. Specialty and critical access hospitals more frequently earned high star ratings compared with acute care hospitals. Critical access hospitals and some specialty hospitals (ie, certain cancer centers) are exempt from reporting-based payment incentives through the CMS Inpatient Quality Reporting system and may not collect many measures used in the star ratings. Consequently, specialty and critical access hospitals reported systematically fewer measures. Although hospital type influenced the number and type of measures reported, the study was limited by the inability to determine whether differences in individual measure reporting by hospital type explained the differences in star ratings.

Because the measures used as the basis for calculating the star ratings differed by hospital type, failure to account for these differences may limit the utility of the star ratings, particularly when comparing different hospital types.

Section Editor: Jody W. Zylke, MD, Deputy Editor.