Association between Traditional Nursing Home Quality Measures and Two Sources of Nursing Home Complaints

Abstract

Objective

To test for an association between traditional nursing home quality measures and two sources of resident- and caregiver-derived nursing home complaints.

Data Sources

Nursing home complaints to the North Carolina Long-Term Care Ombudsman Program and state certification agency from October 2002 through September 2006 were matched with Online Survey Certification and Reporting data and Minimum Data Set Quality Indicators (MDS-QIs).

Study Design

We examine the association between the number of complaints filed against a facility and measures of inspection violations, staffing levels, and MDS-QIs.

Data Extraction

One observation per facility per quarter is constructed by matching quarterly data on complaints to OSCAR data from the same or most recent prior quarter and to MDS-QIs from the same quarter. One observation per inspection is obtained by matching OSCAR data to complaint totals from both the same and the immediate prior quarter.

Principal Findings

There is little relationship between MDS-QIs and complaints. Ombudsman complaints and inspection violations are generally unrelated, but there is a positive relationship between state certification agency complaints and inspection violations.

Conclusions

Ombudsman and state certification agency complaint data are resident- and caregiver-derived quality measures that are distinctive from and complement traditional quality measures.

The most commonly used nursing home quality measures come from data maintained by the Centers for Medicare and Medicaid Services (CMS), including information from the survey and certification process and resident-level measures from the Minimum Data Set (MDS). Survey data and MDS data are both generated from state certification agencies and the nursing homes themselves. Included among these traditional measures of quality are inspection deficiencies, staffing levels, and resident-level MDS quality indicators (MDS-QIs).

Data on complaints against nursing homes are another potential source of quality information. Two key state-level organizations investigate nursing home complaints: the certification agency (responsible for the annual inspection) and the Long-Term Care Ombudsman Program. Both sources of resident and caregiver (family and facility staff) complaints may be particularly relevant to consumers as they represent a resident or caregiver perspective and can occur anytime. Among the limited research on complaints, Stevenson (2005, 2006) studied complaints to state certification agencies and Allen, Klein, and Gruman (2003) considered complaints to the Connecticut Ombudsman. In recent work, Troyer and Sause (2011) consider the association between these two sources of complaints for North Carolina (NC) nursing homes and the roles of the Ombudsman and state certification agency in receiving and responding to complaints. When looking at specific categories of complaints, they find few statistically significant relationships between the two complaint sources, suggesting that the sources may be measuring different dimensions of quality in nursing homes. They suggest that future work should consider the relationship between traditional quality measures and both types of complaints, which is the objective of this study. Using quarterly complaint data for 2002–2006 from the NC Long-Term Care Ombudsman Program and from the NC Division of Health Service Regulation (DHSR, which is the state certification agency), we consider associations between facility-level and resident-level measures of quality and both types of complaints.

Literature Review

Traditional Nursing Home Quality Measures

Beginning with Nyman (1985) and Gertler (1992), measures of nursing home quality have included facility-level measures of staffing and inspection deficiencies cited during the approximately annual survey process. These data are transmitted to CMS and become part of the OSCAR (Online Survey Certification and Reporting) system. In addition to these traditional measures of quality, recent work has focused on resident health outcomes such as the prevalence of decubitus ulcers; incontinence or resident hydration; and the use of restraints, catheters, and psychotropic medication (Wiener 2003; Zhang and Grabowski 2004; Castle 2006). Castle and Ferguson (2010) provide a comprehensive review of the literature on nursing home quality measurement.

Complaints as Quality Measures

Recently, there has been a call to find quality measures that reflect the perspective of the nursing home consumer (Kane and Kane 2001; Kane 2003), such as measures of quality of life, care experience, and satisfaction. However, gathering consumer satisfaction data directly from nursing home residents is difficult due to expense, cognitive limitations of some residents, and accessibility of researchers to this population.

One mechanism nursing home residents have for expressing opinions regarding concerns about quality is the complaint investigation process by each state-level certification agency. Unlike approximately annual certification surveys, complaint investigation surveys may be triggered anytime a complaint that is within the scope of CMS regulations is received. Complaint data are incorporated into a facility's certification file. However, little research has examined complaints. Stevenson (2006) argues that complaints provide a more dynamic, “real-time” perspective of quality than annual survey measures. Unlike other measures of nursing home quality, complaints require a resident or caregiver to take action. Complaints may provide regulators with a signal of resident and caregiver sentiments regarding shortfalls in facilities' care provision, where preferences as expressed by complaints may not match poor care identified in the facility survey process. Examining complaints made to the nursing home licensure agency in Massachusetts, Stevenson (2005) found that complaints were correlated with survey deficiencies and were predictive of the number of deficiencies on an upcoming survey visit. However, the Massachusetts complaint data did not correlate well with MDS-QIs and staffing levels. Stevenson (2006) also used national data to consider complaints for nursing homes within the CMS OSCAR database from 1998 to 2002. He found that complaints are positively associated with serious deficiencies during an annual survey and negatively associated with staffing. Both Stevenson (2005, 2006) and Grabowski (2005) have suggested that the use of nursing home resident and caregiver complaints should be considered as an additional signal of facility quality.

Another source of information that is grounded in the resident's and caregiver's perspective is complaints to the Long-Term Care Ombudsman Program. Nelson, Huber, and Walter (1995) suggest that Ombudsman complaints may be a more accurate reflection of nursing home problems than CMS survey results given that the Ombudsman is in contact with the residents in nursing facilities on an ongoing basis. The program has been viewed as being effective at mediating and resolving complaints and advocating for residents (Harris-Wheeling, Feasley, and Estes 1995; Estes et al. 2004). The Ombudsman also shares aggregate facility-level complaint data to those searching for appropriate long-term care placement.

Despite the presence of the Ombudsman within each state and the numerous complaints the program receives directly from nursing home residents and caregivers, Ombudsman complaints have seldom been studied for two reasons. The categorization of complaints and databases used to track them vary widely across states (Huber et al. 2000). Also, Ombudsman programs are often concerned about confidentiality issues.

Besides the work by Troyer and Sause (2011), one of the few studies to consider Ombudsman complaints was done by Allen, Klein, and Gruman (2003) using Connecticut data. They found a relationship between the rate of complaints to the Ombudsman program and the number of regulatory deficiencies, but no relationship between Ombudsman complaints and staffing.

Background

While both the state certification agency and Ombudsman strive to be resident centered, there are differences between the two due to federal and state regulations. For a detailed discussion of differences between NC's Long-Term Care Ombudsman Program and the Division of Health Service Regulation, see Troyer and Sause (2011); a summary of the differences is contained in Table 1. Briefly, in terms of accessibility, Ombudsman staff have more regular interaction with residents, staff, and family members of residents than survey staff from the DHSR. Complaints may be submitted to the DHSR by phone, fax, or e-mail. In the area of access to information, the DHSR has more authority to access resident records and all areas of the facility. However, the DHSR is only able to investigate complaints that fall under the purview of federal regulations set forth by CMS. Examples of complaints not considered by the DHSR include complaints regarding the operations of a facility's residents' council and nighttime visitation policies. The two entities also differ with respect to complaint-related actions, where the DHSR has the ability to cite a deficient practice and follow-up, whereas the Ombudsman is more likely to use mediation to resolve complaints. While maintaining distinct roles, the programs do work together on several levels.

Table 1.

Comparison of the North Carolina (NC) Division of Health Service Regulation (DHSR) and the Long-Term Care Ombudsman Program

	NC DHSR	NC Long-Term Care Ombudsman Program
Primary objective	Maintenance of a minimum standard of care based upon federal regulations and guidelines	Seeks to be a resident advocate, assisting and empowering residents in resolving complaints
Program accessibility	Residents, family, staff may call a toll-free number, fax, or e-mail complaints into the DHSR	A representative of the Ombudsman Program is in each facility at least quarterly, regardless of receipt of complaint
	DHSR staff only enters the facility at annual surveys and in response to receipt of a complaint	Ombudsman often participates in training sessions for residents, staff, and family members
		Ombudsman often participates in resident or family councils
		Representatives of the Program are local individuals
Access to information	Has authority to access medical and financial records without explicit permission from the resident or legal representative	Has authority to access medical and financial records only with permission from resident, legal representative, or through a court order
	Has authority to access all physical areas within the facility, including kitchen, medication carts, etc	Access to certain physical areas of the facility is limited
Complaint parameters	Able to accept complaints which fall within the purview of federal regulations set forth by CMS	Able to accept any complaint that the Ombudsman feels is a valid concern for the resident
Complaint actions	Cite deficient practice, and follow-up to ensure plan of correction is completed	May provide mediation, staff/resident education and training, consultation and appropriate referral to assist in complaint resolution

Data and Analytical Approach

Data

We consider complaints against NC nursing homes to the Ombudsman and the DHSR. We use complaints to the Ombudsman in NC from fourth quarter 2002 through third quarter 2006. While the Ombudsman handles complaints on all skilled nursing facilities in the state, we limit our analysis to complaints against CMS-certified, non-hospital-based facilities.

The Ombudsman complaint data indicate the quarter and year of each complaint filed; whether the complaint was substantiated; the complaint category; and by whom the complaint was investigated. A list of complaint codes from the NC Ombudsman database is available from the authors upon request. The descriptive statistics on Ombudsman complaints include complaints investigated by the DHSR, which are likely to show up in the OSCAR complaint database. Therefore, we exclude these complaints from the subsequent analyses of Ombudsman complaints to avoid double counting.

We also consider all complaints against NC nursing homes filed with DHSR that appear in the OSCAR database over the same 4-year period. The OSCAR database includes information regarding the timing of the complaint and whether the complaint was substantiated by DHSR investigators. When a complaint is substantiated, it often results in a deficiency citation indicating that the facility has failed to meet a regulatory standard. In this analysis, we group DHSR complaints from OSCAR with deficiencies into the nine categories used for publicly reporting quality indicators for nursing homes on the Nursing Home Compare website (http://www.medicare.gov/nhcompare). Specific deficiencies contained in each category are found in the State Operations Manual: Appendix PP (CMS 2007).

The OSCAR database also contains detailed information regarding each facility's periodic inspection by the DHSR, including the following facility characteristics: proportion of Medicaid, Medicare, and otherwise-funded residents; whether the facility is for-profit, not-for-profit, or government owned; whether the facility is part of a chain; the number of residents; the proportion of residents with activities of daily living limitations in four areas (toileting, transferring, incontinence, and eating); and the number of beds. We also consider measures of inspection deficiencies, total deficiencies, health deficiencies, life/safety deficiencies, and we consider registered nurse (RN), licensed practical nurse (LPN), licensed vocational nurse (LVN), and nurse aide hours per resident per day. 1

Finally, we examine four quality indicators based on facility-reported clinical assessments of residents contained in the MDS that are available in the Nursing Home Compare data: incontinence without a toileting plan, indwelling catheters, decline in late-loss activities of daily living (ADLs), and pressure sores among high-risk residents. Late-loss ADLs include bed mobility, transfers, toilet-use, and eating. These four MDS measures chosen are publicly available and match four of the six MDS-QIs considered by Stevenson (2006).

Analytical Approach

We begin with descriptive statistics for (1) both types of complaints, (2) facility characteristics, and (3) traditional quality measures, including staffing, inspection deficiencies, and MDS-QIs.

Next, we match the complaint data to the traditional quality measures. Nursing home inspection OSCAR measures are only available every 9–15 months. So we match each set of complaints to the facility characteristics from OSCAR available for the current (same quarter) or nearest prior inspection. The quarterly MDS Quality Indicators are matched to complaints from the same quarter. The staffing and inspection deficiencies are not measured quarterly. To consider the relationship between these survey-related quality measures and complaints, we match the two sets of quality measures in two ways. First, quarterly data on complaints are matched to the inspection data from the current quarter or most recent quarter, resulting in one observation per nursing home per quarter. This results in the same inspection-related value for more than one quarter for a given facility; however, given the quarterly nature of the Quality Indicators and complaints and for ease of exposition, we maintained a data structure based on the quarterly measures of complaints. Second, inspection data are matched to both current complaint totals and to complaint totals from the immediate previous quarter; in this case, there is one observation per inspection.

Ombudsman complaints, DHSR complaints, and subsets of both types of complaints are modeled using a conditional fixed-effects Poisson model, controlling for facility characteristics and including time and facility fixed effects. Wooldridge (1999) shows this estimator produces consistent estimates under very general conditions for a nonnegative dependent variable. Each observation represents a facility in a particular quarter and year, where the dependent variable is a nonnegative count measure of all complaints. The fixed effects specification allows us to control for characteristics that are constant at the facility level over the 4 years considered. Although some of facility characteristics that we include as control variables (chain affiliation, ownership type) are constant for some facilities over time, we do have some variation in these measures. 2

Each model of Ombudsman complaints, DHSR complaints, and subsets of both types of complaints includes as an explanatory variable one of the traditional quality measures. Following Hickson et al. (2002) and Stevenson (2005, 2006), we present z-scores corresponding to significance levels to indicate the correlation between one of the complaint measures and a quality measure. The z-scores are constructed using cluster-robust standard errors. For instance, we consider the association between the number of quality of care complaints to the Ombudsman and survey deficiencies. The z-score indicates whether there is a statistically significant positive or negative association between the number of Ombudsman complaints and the recent volume of survey deficiencies, controlling for facility characteristics and fixed effects.

In the estimates discussed above, we consider whether current or recent past traditional quality measures are related to the number of current complaints, controlling for factors that influence complaint volume. In the final set of estimates, we use the inspection as the unit of observation to examine the relationship between each inspection-related quality measure (as the dependent variable) and the number of current and past complaints (as an explanatory variable), conditional on other factors that may influence inspection-related quality measures.

Results

Shown in Table 2, during the period analyzed, the Ombudsman received 7,896 complaints against NC nursing homes. On average, facilities had 1.23 Ombudsman complaints per quarter, or approximately 5 complaints per year. In an average quarter, 78.9 percent of all facilities had either one or no complaints, 2.8 percent of facilities had 10 or more complaints, and the highest level of reported complaints for a single facility in a quarter was 58 complaints. An overwhelming proportion (90.79 percent) were substantiated. All Ombudsman complaints are assigned a complaint category, regardless of whether they are substantiated. The top category was quality of care (39.74 percent), followed by administration (23.76 percent), which includes all complaints that deal with the policies, procedures, and resources of a facility and/or the attitudes of the administrative staff. Third most common was residents' rights (12.84 percent). While its own category, complaints dealing with resident rights are also dispersed throughout the other Ombudsman complaint categories. As noted above, for the subsequent analyses, we removed the Ombudsman complaints that were referred to the DHSR from the Ombudsman complaint counts, reducing the number of Ombudsman complaints by 284 (3.6 percent).

Table 2.

Descriptive Statistics for North Carolina Facility Characteristics 2002–2006

Variable	Description	% or Mean
		Ombudsman	DHSR
Substantiated	Complaint was substantiated	90.79%	48.29%
Nutrition	Complaint related to issues with nutrition or food	6.36%	3.73%
Pharmacy	Complaint related to issues with pharmaceuticals	3.28%	5.17%
Quality of care	Complaint related to quality of care	39.74%	34.02%
Residents' rights	Complaint related to residents' rights	12.84%	19.40%
Administration	Complaint related to nursing home administration	23.76%	9.38%
Environment	Complaint related to the nursing home environment	5.26%	15.74%
Physician	Complaint related to physician care	2.94%	NA
Financial	Complaints related to financial issues	5.83%	NA
Mistreatment	Complaint related to mistreatment	NA	7.70%
G level + deficiency	Complaint had a G-level or higher deficiency, meaning that the complaint was associated with actual harm or immediate jeopardy	NA	19.27%
Number of complaints against NC facilities		7,896	6,345
Average number of complaints per nursing home per quarter		1.23	1.04
Number of DHSR complaints against NC facilities with deficiencies		NA	2,922
Traditional quality measures
Total deficiencies	Total deficiencies found during the inspection per 100 residents		11.47
Health deficiencies	Health deficiencies found during the inspection 100 per residents		6.51
Life deficiencies	Life and safety deficiencies found during the inspection per 100 residents		4.97
RN	Registered nurse hours per resident per day		0.54
LPN/LVN	Licensed practical nurse and licensed vocational nurse hours per resident per day		1.18
Aides	Nurse aide hours per resident per day		3.33
Incontinence	Proportion of residents with incontinence without toileting plan		52.00%
Indwelling catheter	Proportion of residents with an indwelling catheter		4.68%
Late-loss ADLs	Proportion of residents with a decline in late loss activities of daily living		21.27%
Pressure sores	Proportion of residents with pressure sores among high-risk residents		13.34%
Facility characteristics
% Medicaid	Proportion of Medicaid funded residents		67.43%
% Medicare	Proportion of Medicare funded residents		16.47%
For-profit	For-profit facility		80.99%
Not-for-profit	Not-for-profit facility		17.54%
Chain owned	Chain affiliated facility		71.40%
Limited ADL index	Proportion of residents with activities of daily living limitations in four areas (toileting, transferring, incontinence, and eating)		26.96%
Beds	Total number of beds		108.6
Total number of facilities			379

The DHSR received 6,245 complaints against NC nursing homes during the same period. On average, facilities had 1.04 DHSR complaints per quarter, or approximately four complaints per year. In an average quarter, 76.6 percent of all facilities had one or zero complaints, 0.4 percent of facilities had 10 or more complaints, and the highest level of reported complaints for a single facility in a quarter was 24 complaints. Less than half (48.29 percent) were substantiated. Complaint categories were recorded for complaints that were assigned a deficiency code, which includes primarily substantiated complaints. Similar to complaints to the Ombudsman, over one third of DHSR complaints are for quality of care concerns, and residents' rights complaints make the top-three list.

Table 2 also contains means or proportions for the NC facility-level noncomplaint quality measures and other characteristics. NC facilities are similar to the national average in terms of the number of deficiencies (9.2 per facility nationally and 10.96 per facility in NC) and NC homes have modestly more staff than the national average (Harrington, Carrillo, and Blank 2007). Using national statistics from Nursing Home Compare (CMS 2008), the average values for the MDS-QI measures for NC facilities are similar to the national average, with the exception of the proportion of residents with a decline in late-loss activities of daily living (21.3 percent in NC vs. 15 percent nationally).

Table 3 reports z-values from the Poisson fixed-effects models for the association between MDS-QIs and the number of Ombudsman complaints (top half of the table) or DHSR complaints (bottom half of the table). In general, the association between the number of complaints and health outcomes in a quarter is not statistically significant. Considering Ombudsman complaints, an increase in the number of late-loss ADLs is associated with more nutrition complaints and fewer financial complaints. For DHSR complaints, a higher prevalence of residents with pressure sores is associated with more administration complaints, whereas a higher prevalence of residents with indwelling catheters is associated with more pharmacy complaints.

Table 3.

Association between Complaints and MDS Quality Indicator Measures in North Carolina, 2002–2006

Health Outcomes	All	Substantiated	Nutrition	Pharmacy	Quality of Care	Residents' Rights	Administration	Environment	Physician	Financial
Dependent variable: LTC ombudsman complaints
Incontinence	0.17	−0.02	0.14	−1.15	0.52	−0.03	0.05	0.49	−1.27	0.47
Indwelling catheter	−0.35	−0.49	0.11	0.08	0.40	−2.19*	−0.46	0.70	−2.11*	0.97
Late-loss ADLs	1.01	1.03	2.02*	1.33	1.68	−0.72	0.99	0.45	−0.44	−1.99*
Pressure sores	0.70	0.29	0.04	−0.04	1.01	−0.26	0.59	1.38	−0.43	0.82

Health Outcomes	All	Substantiated	Nutrition	Pharmacy	Quality of Care	Residents' Rights	Administration	Environment	Mistreatment	G Level + Deficiency
Dependent variable: DHSR complaints
Incontinence	0.39	0.70	0.87	−0.60	−0.52	0.88	0.15	1.07	−0.11	−0.71
Indwelling catheter	0.16	−0.40	−0.70	2.16*	0.07	0.10	0.56	−1.14	0.16	−0.88
Late-loss ADLs	0.78	0.69	0.32	1.18	−0.22	−0.13	1.72	−0.16	−0.12	−0.38
Pressure sores	1.39	0.92	−0.01	1.86	1.46	0.10	1.93*	−0.55	0.20	0.99

Each cell represents the z-value from a Poisson model that includes facility fixed effects and time fixed effects for each quarter in each year, where the z-value is based on the robust standard errors. The columns represent the dependent variable, and the key explanatory variable is in the row. Other explanatory variables include all facility characteristics found in Table 2. n = 5,130 for the entire sample. However, the sample used to estimate each model only includes facilities with at least one nonzero value for the dependent variable over the 16 quarters examined. In addition, health outcomes data are not available for some facilities in some quarters due to the small number of patients in the facility. When this happens, the observation for a facility in a quarter is not included in the analysis.

^*Significance at the .05 level.

Table 4 considers the relationship between the number of current quarter complaints (dependent variable) and current or past values of inspection deficiencies and staffing levels. The unit of observation is a facility in a quarter, where each cell contains a z-value for the coefficient on the measure of deficiencies or staffing from the current or most recent prior inspection. In the top half of Table 4, we find no significant contemporaneous relationship between deficiency levels and the number of Ombudsman complaints. In contrast, there are positive and significant relationships between deficiencies at the most recent survey and the number of DHSR complaints in many categories for both total deficiencies and health deficiencies; complaint categories with significant relationships include all, substantiated, pharmacy, quality of care, residents' rights, mistreatment, and complaints with a G-level or higher deficiency.

Table 4.

Association between Current Complaints and Current or Past Survey Deficiencies or Staffing Levels in North Carolina, 2002–2006

Current or Past Survey Deficiencies and Staffing Levels	All	Substantiated	Nutrition	Pharmacy	Quality of Care	Residents' Rights	Administration	Environment	Physician	Financial
Dependent variable: LTC ombudsman complaints
Total deficiencies	0.33	0.21	0.39	0.04	0.47	−0.31	0.09	0.91	0.33	−0.58
Health deficiencies	0.43	0.30	0.21	0.24	0.54	0.25	0.24	0.14	0.75	−0.38
Life deficiencies	−0.06	−0.06	0.87	−0.37	0.01	−0.80	−0.27	1.63	−0.13	−0.58
RN per resident	2.25*	2.30*	1.21	1.86	4.89**	−0.48	0.62	2.21*	0.07	−1.11
LPN/LVN per resident	3.42**	4.04**	2.82**	2.52*	5.57**	−0.27	1.22	2.10*	−0.40	−1.73
Aides per resident	1.14	1.29	1.45	1.48	1.33	0.41	0.15	1.03	1.05	−2.15*

Current or Past Survey Deficiencies and Staffing Levels	All	Substantiated	Nutrition	Pharmacy	Quality of Care	Residents' Rights	Administration	Environment	Mistreatment	G Level + Deficiency
Dependent variable: DHSR complaints
Total deficiencies	2.07*	3.65**	1.46	1.30	2.42*	3.00**	0.95	1.67	2.49*	3.74**
Health deficiencies	3.46**	3.90**	1.39	1.99*	2.99**	2.46*	0.65	1.94	2.21*	3.00**
Life deficiencies	1.65	0.93	0.36	−0.80	−0.12	1.97*	0.99	0.03	1.21	1.59
RN per resident	1.28	1.81	−0.82	−1.62	8.08**	−1.92	3.47**	−2.59**	−1.74	2.64**
LPN/LVN per resident	2.55*	5.94**	−1.14	−0.55	9.70**	−1.65	4.53**	−2.31*	−0.46	5.72**
Aides per resident	1.37	0.85	−0.92	−0.64	2.52*	−2.04*	2.51*	−1.60	1.40	1.84

Each cell represents the z-value from a Poisson model that includes facility fixed effects and time fixed effects for each quarter in each year, where the z-value is based on the robust standard errors. The columns represent the dependent variable and the key explanatory variable is in the row. Other explanatory variables include all facility characteristics found in Table 2. n = 5,130 for the entire sample. However, the sample used to estimate each model only includes facilities with at least one nonzero value for the dependent variable over the 16 quarters examined.

^*Significance at the .05 level.

^**Significance at the .01 level.

The results regarding the relationship between staffing and the number of complaints are similar for the two complaint types. For most Ombudsman complaint categories, we find a positive relationship between the number of complaints and RN or LPN/LVN staffing levels measured during the survey conducted in the current or most recent past quarter; in other words, the higher the number of nurses, the more complaints. Aides staffing is only significantly related to the volume of financial complaints.

For DHSR complaints, we generally find positive relationships between the volume of complaints and RN or LPN/LVN staffing levels, where statistically significant. The exception is for environmental complaints, where we see a negative relationship. Aides staffing is positively related to the number of DHSR complaints in the quality of care and administration categories and negatively related to the number of DHSR complaints in the residents' rights category.

The final set of results is contained in Table 5, where the unit of observation is a facility inspection and the dependent variables are traditional quality measures derived from the inspection data. The key explanatory variables are the number of complaints from the quarter of the inspection and the number of complaints from the quarter prior to the quarter of the inspection. The objective of these analyses is to consider the extent to which (1) current complaints are related to current inspection values and (2) recent past complaints are related to current inspection values. The association with current complaints indicates whether complaints in a quarter are associated with inspection values from that quarter, which may suggest an overlap between the two quality measures. In contrast, association between inspection values and past complaints can be used to consider whether prior complaint activity can act as a signal of future inspection problems. For instance, in the first column of results, the dependent variable is total deficiencies. In this example, lines one and two of the results from the first column indicate z-value results from a model with the number of complaints in the current quarter (z = 0.87) and the number of complaints in the prior quarter (z = 0.21) as the key explanatory variables. The first three columns are for deficiency-related dependent variables: total deficiencies, health deficiencies, and life/safety deficiencies. Columns four through six contain results for staffing levels.

Table 5.

Association between Current Survey Deficiencies or Staffing Levels and Current and Past Complaints in North Carolina, 2002–2006

Current or Past LTC Ombudsman Complaints	Dependent Variable: Current Survey Deficiencies			Dependent Variable: Current Staffing Levels
	Total Deficiencies	Health Deficiencies	Life Deficiencies	RN per Resident	LPN/LVN per Resident	Aides per Resident
All: Survey quarter	0.87	1.87	0.56	−3.15**	−1.05	−1.24
All: Prior to survey quarter	0.21	−0.05	0.40	−0.82	−1.37	1.15
Substantiated: Survey quarter	0.91	1.85	0.65	−3.09**	−0.95	−1.23
Substantiated: Prior to survey quarter	0.30	−0.32	0.46	−0.79	−1.38	−1.17
Nutrition: Survey quarter	−0.98	0.19	−2.16*	0.09	0.06	−0.06
Nutrition: Prior to survey quarter	1.63	0.98	1.49	−0.58	−1.43	−1.23
Pharmacy: Survey quarter	−0.67	−0.44	−1.00	0.37	0.11	−0.44
Pharmacy: Prior to survey quarter	0.07	0.34	−0.48	−0.96	−1.36	−1.57
Quality of care: Survey quarter	0.91	2.02*	0.44	−2.94**	−0.66	−0.90
Quality of care: Prior to survey quarter	−0.29	−0.17	−0.80	−0.66	−1.45	−1.38
Residents' rights: Survey quarter	0.95	1.46	1.24	−2.88**	−0.74	−1.15
Residents' rights: Prior to survey quarter	0.83	0.36	1.54	−1.01	−1.23	−1.12
Administration: Survey quarter	0.85	1.09	0.81	−2.70**	−1.73	−1.74
Administration: Prior to survey quarter	−0.49	−1.73	0.42	−0.68	−0.76	0.16
Environment: Survey quarter	1.27	1.57	0.91	−1.61	0.90	1.00
Environment: Prior to survey quarter	−0.06	−1.43	0.51	−1.16	−1.35	−1.56
Physician: Survey quarter	0.10	0.25	0.57	−2.35*	−2.43*	−2.16*
Physician: Prior to survey quarter	1.27	0.71	1.17	1.04	−1.23	−0.82
Financial: Survey quarter	0.54	1.14	−0.22	−1.98*	−1.17	−1.20
Financial: Prior to survey quarter	0.44	−0.55	0.35	−0.83	−1.68	−1.16
Current or past DHSR complaints
All: Survey quarter	4.88**	5.14**	1.16	−1.49	−1.65	−1.29
All: Prior to survey quarter	2.13*	2.33*	0.91	−2.34*	−1.39	−1.69

Current or Past DHSR Complaints	Dependent Variable: Current Survey Deficiencies			Dependent Variable: Current Staffing Levels
	Total Deficiencies	Health Deficiencies	Life Deficiencies	RN per Resident	LPN/LVN per Resident	Aides per Resident
Substantiated: Survey quarter	5.17**	5.34**	0.31	−2.31*	−1.77	−1.39
Substantiated: Prior to survey quarter	2.13*	2.62**	1.78	−1.75	−1.94	−1.19
Nutrition: Survey quarter	2.48*	2.27*	−0.43	−1.16	−1.30	−0.86
Nutrition: Prior to survey quarter	2.33*	1.23	0.72	−0.72	0.53	0.35
Pharmacy: Survey quarter	3.98**	3.39**	−0.59	−1.23	−0.07	−0.14
Pharmacy: Prior to survey quarter	1.96*	3.09**	1.78	−2.28*	−1.31	−0.61
Quality of care: Survey quarter	6.23**	6.51**	0.39	−2.15*	−1.90	−1.61
Quality of care: Prior to survey quarter	1.87	3.53**	2.60	−2.42*	−1.65	−1.73
Residents' rights: Survey quarter	3.80**	3.62**	0.39	−1.98*	−1.55	−1.70
Residents' rights: Prior to survey quarter	2.30*	2.29*	1.20	−0.06	0.65	0.24
Administration: Survey quarter	3.07**	4.20**	−0.66	−1.65	−0.57	0.02
Administration: Prior to survey quarter	2.01*	2.59**	1.28	−1.46	−1.72	−1.22
Environment: Survey quarter	3.15**	3.37**	−1.43	−1.66	−1.66	−1.37
Environment: Prior to survey quarter	2.07*	2.03*	1.80	−0.08	1.56	−0.66
Mistreatment: Survey quarter	0.90	1.77	−0.19	−0.43	−0.27	−0.29
Mistreatment: Prior to survey quarter	2.08*	−0.32	0.25	0.87	−0.70	0.42
G level + deficiency: Survey quarter	4.69**	4.51**	0.53	−1.92	−1.61	−1.29
G level + deficiency: Prior to survey quarter	2.06*	3.55**	1.06	−1.78	−2.19*	−1.48

Each cell represents the z-value from a Poisson model that includes facility fixed effects and time fixed effects for each quarter in each year, where the z-value is based on the robust standard errors. The columns represent the dependent variable and the key explanatory variables (two for each model) are in the rows. Other explanatory variables include all facility characteristics found in Table 2. n = 1,497 for the entire sample, which includes facilities with at least two inspections.

^*Significance at the .05 level.

^**Significance at the .01 level.

Beginning with the deficiency results in the top half of Table 5, there is only one significant relationship between current or past quarter Ombudsman complaint volume and deficiencies. In contrast, the results for the DHSR complaint volume, found in the bottom half of Table 5, indicate positive relationships between nearly all categories of DHSR complaints (from both the current and prior quarter) and total or health deficiencies.

Turning to the staffing measures in Table 5, the number of complaints to the Ombudsman from the previous quarter appears to be unrelated to current staffing levels. However, more Ombudsman complaints in a quarter are associated with lower staffing levels in the same quarter, with a greater number of statistically significant relationships found for RNs than for the other staffing categories. In contrast, the DHSR complaints show three measures of complaint volume as being predictive of low RN staffing in the next quarter: total, quality of care, and pharmacy. The findings regarding current DHSR complaint volume and RN staffing mirror the results for the Ombudsman complaint volume but with fewer significant effects.

Discussion

Our research considers the relationship between traditional measures of quality (deficiencies, staffing, and health outcomes) and both state certification agency (DHSR) and Ombudsman complaints against nursing homes in NC.

The relationship between the two types of complaints and other quality measures is complex. First, MDS health outcome measures are unrelated to the volume of both types of complaints. Consistent with Stevenson's (2005) study of Massachusetts complaints, complaints to both the DHSR and the Ombudsman appear to be measuring different dimensions of quality than the MDS-QIs.

The relationship between the number of deficiencies and Ombudsman complaint levels is quite weak, supporting the notion that Ombudsman complaints cannot forecast deficiencies and do not overlap with quality concerns uncovered in the inspection process. Our results mirror those of Allen, Klein, and Gruman (2003), who use a binary indicator for the presence of any deficiencies in their cross-sectional study of Connecticut Ombudsman complaints.

In contrast, consistent with national results by Stevenson (2006), there are positive and significant relationships between deficiencies found during the state certification survey and current and subsequent DHSR complaint volume in many categories. We also find that the number of DHSR complaints is strongly associated with deficiencies found in the current or next quarter. Policy makers might consider monitoring complaints to the state certification agency between inspections, which may lead to reductions in deficiencies at the time of the survey. Future research might focus on identifying a level of complaint volume between inspections that might be used to trigger additional off-cycle inspections and/or more frequent visits by the Ombudsman.

The staffing results show similar patterns for the Ombudsman and DHSR complaints. In the models where the complaint volume is the dependent variable, we find that higher staffing levels of RNs and LPN/LVNs are associated with more complaints to the Ombudsman and DHSR in most cases. Our results are similar to Allen, Klein, and Gruman (2003), who find a positive result for Connecticut Ombudsman complaints, but our results contrast with work by Stevenson (2005, 2006) that finds a negative correlation between staffing and state certification agency complaints. Our research suggests that staff may play a key role in encouraging complaints by nursing home residents. It is also possible that residents in better quality, highly staffed facilities are more likely to be of higher socioeconomic status and have advocates who are better able to navigate the complaint reporting processes.

When staffing level at inspection is the dependent variable, we find that more Ombudsman and DHSR complaints in the same quarter as the inspection are associated with lower staffing levels, with more statistically significant results for RN staffing than for the other staffing categories. This suggests that complaints and staffing measured at inspection may be picking up related quality concerns. For the DHSR complaints, we find that complaint volume from the previous quarter is predictive of low RN staffing in the next quarter for several complaint categories. This suggests that staffing dynamics are related to the generation of complaints, where resources used to address complaints may result in lower staffing levels in the same or subsequent quarters. However, it may also be the case that both low staffing and complaint generation are being influenced by some unmeasured facility attribute that is simultaneously negatively influencing both quality measures.

While this study is the first to look at how two different types of complaints are related to traditional measures of nursing home quality, readers should keep several limitations in mind. First, while complaints may indicate poor care, the lack of such complaints is not necessarily a sign of good care. Second, we focus on one state. Between states, the state certification agencies and Ombudsman programs may vary widely in their complaint substantiation processes, and future researchers should examine other states. Third, future research should consider the way that complaints are categorized by the Ombudsman relative to the categorization by the state survey agency. Fourth, we do not have data on the source of the complaint, which limits our understanding of the complaint generation mechanism and the interpretation of our staffing results. Future researchers may wish to conduct qualitative work that focuses on determining the pathways to complaint filing and complaint resolution. Qualitative work could also be used to attempt to learn more about the severity of complaints. Finally, when considering the volume of complaints, we acknowledge that we do not control for the level of empowerment of residents and caregivers. If resident empowerment results in more complaints, all else equal, it may confound any relationships between traditional quality measures and complaints. In addition, a higher level of resident empowerment may be positively related to quality attributes of the nursing home at the time of admission.

Conclusion

In sum, nursing home complaints to the Ombudsman and to the state certification agency are two potentially valuable sources of consumer-oriented quality information regarding nursing homes. Data on complaints to the state certification agency are available in Nursing Home Compare. We find that a higher volume of complaints to the state agency in the quarter before an inspection is associated with higher levels of deficiencies at inspection. Given this result, residents and caregivers may use the complaint information between inspections to help guide them in decision making about the likely future state of a facility's quality. In contrast, our results suggest a weak or no relationship between Ombudsman complaints and both deficiencies and health outcomes. This suggests that the Ombudsman complaint information is not duplicating the information in Nursing Home Compare, and complaints are likely to be tapping different quality constructs. Given our results, we would encourage CMS to consider working with the Long-Term Care Ombudsman to standardize complaint categorization across states to allow for a fuller exploration of the uniqueness of the quality signal from Ombudsman complaints. If research from other states validates our findings, CMS should consider incorporating aggregate information on all Ombudsman complaints at the facility level into Nursing Home Compare to complement currently reported nursing home quality measures.

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.