Table 5.

Agreement in Identifying Nursing Home Quality Outliers—CMS-Unadjusted Measure Compared with Risk-Adjusted Measures

	Outlier Status Based on Risk Adjustment
Outlier Status Based on CMS-Unadjusted Method	Method 1*			Method 2†			Method 3‡			Method 4§
	Low	Medium	High	Low	Medium	High	Low	Medium	High	Low	Medium	High
Low	928	85	0	773	240	0	946	67	0	945	68	0
Medium	598	5,728	17	258	5,978	107	867	5,458	18	445	5,861	37
High	0	704	1,276	0	453	1,527	0	729	1,251	0	1,171	809
False-positive rate**		0.12			0.10			0.13			0.17
False-negative rate**	0.39		0.01	0.25		0.07	0.48		0.01	0.32		0.04
Overall κ		0.69			0.76			0.64			0.59

Low-quality outliers are nursing homes whose quality measures of decline in ADL functioning are significantly higher than the national average rate 17.96% according to the 95% confidence intervals of the measures. Nursing homes of medium quality are those whose quality measures of decline in ADL functioning do not significantly differ from the national average rate 17.96% according to the 95% confidence intervals of the measures. High-quality outliers are nursing homes whose quality measures of decline in ADL functioning are significantly lower than the national average rate 17.96% according to the 95% confidence intervals of the measures.

^*Based on classical logistic regression model.

^†Based on fixed-effects model.

^‡Based on random-effects model.

^§Based on the shrinkage estimators of the random-effects model.

^**Each risk-adjusted measure was treated as “gold standard” when calculating the false-positive (for nursing homes of medium quality) and false-negative (for low-or high-quality outliers) rates.