Case Mix Adjustment in Nursing Systems Research: The Case of Resident Outcomes in Nursing Homes

Abstract

Case mix indicates, for a resident population, the degree of risk for developing favorable or unfavorable outcomes. In a study of 164 nursing homes, we explored two methods for combining resident assessment data into a case mix index (CMI). We compared a facility-level, composite CMI to a prevalence-based CMI comprised of 22 separate resident characteristics for their adequacy in explaining resident outcomes. The prevalence-based CMI consistently explained more variance in outcomes than the facility level, composite CMI. This study indicates a reasonable method for using administrative databases containing resident assessment data to adjust for the influence of case mix on nursing home resident outcomes.

Case Mix Adjustment in Nursing Systems Research: The Case of Resident Outcomes in Nursing Homes

Making judgements about patient outcomes is a fundamental concern of nursing systems research. Outcomes are end results of care and reflect either favorable or adverse changes in health status. In nursing homes, outcomes are due to the interventions given as well as to the predisposing risks and severity of illness. Judgements about the effectiveness or quality of resident care and subsequent health status are made, in part, by accounting for the predisposing risks a population has for good versus bad outcomes. Failure to adjust for residents’ risks can prove misleading when attempting to evaluate the relative merits of different nursing home practices. Therefore, risk adjustment is an essential component of an outcomes study.

“The goal of risk adjustment is to account for pertinent patient characteristics before making inferences about the effectiveness or quality of care based upon patient outcomes” (Iezzoni, 1997, p. 3). It is an accounting of those factors that are causally related to the outcomes under study. Severity, case mix, intensity, complexity, and comorbidity are some of the terms that have been used synonymously with the term “risk adjustment.” Each term refers to sources of risk associated with favorable or adverse health outcomes. The validity of severity of illness indices, commonly used to adjust outcomes of medical care in hospitalized patients, has been well studied (Shortell et al., 1994). Severity of illness has been linked to inpatient resource use in both acute care (Clauser & Fries, 1992; Iezzoni) and nursing home settings (Fries, 1990). However, little research has addressed the appropriateness of the above indicators for risk adjustment in studies of the outcomes of nursing care.

In this study, we use the term case mix to denote a collection of those indicators that reflect sources of risk associated with favorable or unfavorable outcomes for a population of nursing home residents. The purpose of this study was to examine selected case mix indicators for their usefulness in separating the variation in outcomes due to differences in resident characteristics versus variation due to differences in nursing care. The problem addressed in this study is that individual residents have certain characteristics, for example functional ability and clinical conditions, that predispose them to adverse outcomes (Zimmerman et al., 1995). When these certain characteristics are present, adverse outcomes are likely to occur despite the nursing care rendered, as in the resident with terminal cancer who develops malnutrition. Thus, comparisons of outcomes between nursing homes are only valid when variation due to the characteristics of the resident population are controlled, serving to “level the playing field” for the nursing homes under study (Zimmerman et al.).

Sophisticated and well-researched case mix indices (CMIs) exist for the nursing home setting. These indices were derived from a systematic assessment of resident conditions and characteristics using tools such as the Minimum Data Set (MDS) or the Client Assessment, Review, and Evaluation Form 3652-A (CARE). Initially, CMIs derived from resident assessment data were used to describe the resource needs of residents in order to estimate reimbursement rates for the facility (Fries, 1990; Zbylot, Job, McCormick, Boulter, & Moore, 1995). Recently, however, these same indices have been used to make inferences about the quality of care in nursing homes (Zbylot et al.; Zimmerman et al., 1995).

In either instance, residents are placed into homogeneous groups and assigned a CMI indicative of resident clinical status and the resources required to care for the average resident assigned to that group. Researcher have demonstrated that grouping residents according to various functional and clinical parameters is predictive of differences in service use across various groups of caregivers (Manton, Cornelius, & Woodbury, 1995). Because it is a composite score, however, the composite CMI leaves unanswered questions about whether differences in resident outcomes between nursing homes are the result of unmeasured resident characteristics, nursing staff not performing therapeutically indicated activities, or nursing staff not performing those activities well enough. It is reasonable to expect, therefore, that a multidimensional CMI, which can account for multiple complexities, comorbidities, and functional limitations, will explain differences in resident outcomes better than a composite index, and therefore is more useful for making judgements about quality of nursing care.

Hence, in this study we sought to compare the effectiveness of a facility level, composite index of case mix to a prevalence-based, multi-indicator CMI in explaining differences in resident outcomes across nursing homes. Specifically, the research question was: Using resident assessment data, does a facility-level composite CMI or a prevalence based, multi-indicator CMI explain the most variance in nursing sensitive resident outcomes?

In the first method, we created a facility-level, composite CMI (Daley & Shwartz, 1994; Holmes, 1996) for a nursing home by combining and weighting individual resident assessments. This is the same approach that has been used when estimating reimbursement rates for a nursing home. In this method all of the information in the resident assessment is summarized to develop a composite CMI for the facility. Resident characteristics have been linked to resource use in development of the case mix payment system (Fries et al., 1994). Residents with similar attributes are placed into mutually exclusive groups. Each group has been assigned a CMI that reflects a common pattern of resource use, including nursing time, resident needs for rehabilitation, complexity of health conditions, severity of behavioral problems, and level of physical functioning. The number of actual groups may vary. In all cases, however, the groups are ordered hierarchically from high to low resource use, indicating severity of condition or functional dependency, and then assigned a case mix weight. By counting the number of patient days a nursing home had in each case mix group and applying that group’s case mix weight, a nursing home level CMI is calculated (Provider Reimbursement Department, 1995).

When making comparisons across numerous nursing homes, the use of a facility-level, composite CMI has many advantages. First, a single, composite index is useful because it is simple, unlikely to compromise power in statistical analyses, and minimizes problems with collinearity. This is particularly beneficial when the research question calls for case mix to be part of the model. In addition, a facility-level, composite CMI is calculated on all aspects of the resident assessment. However, some indicators of case mix may contribute independently to an outcome, and this level of explanatory power is lost in a composite metric.

In the second method, we created a prevalence-based, multi-indicator CMI, comprised of 22 separate resident characteristics. A prevalence rate was calculated for each resident characteristic (Zimmerman et al., 1995). The 22 characteristics were selected from the resident assessment items that describe risk factors, such as problems in mobility-ambulation, dressing/grooming, transferring, eating, toileting, orientation/memory, level of consciousness, hemi or paraplegia, amputation, and terminal illness, and chronic illnesses. Such risk factors are resident characteristics that “either increase or decrease the resident’s probability of having a specific [poor outcome]” (Zimmerman et al., p. 110). Thus, a nursing home with greater proportions of residents with predisposing conditions may be unfairly rated as having lower quality of care than a nursing home in which residents don’t have the same prevalence of such predisposing conditions. While it is true that some of these risk factors may improve with appropriate nursing care, they must be considered in making comparisons between nursing homes (Zimmerman et al., 1995). Examples of resident risk factors include

The use of prevalence-based, multi-indicator CMI has the advantage of more easily partitioning the influence of resident risks from the influence of the care processes (interventions) on resident outcomes. Consequently, conclusions about the effectiveness of nursing interventions on resident outcomes can be made with greater confidence. Using the prevalence-based, multi-indicator CMI, therefore, it is less likely that inappropriate attributions will be made to adverse outcomes than when using a single, composite CMI. A major disadvantage of the prevalence-based, multi-indicator CMI is complexity of the model--more variables in the statistical analysis, affecting power. There will also be collinearity among the predictor variables. If the purpose of multiple variables in the model is strictly to control for case mix, however, interpretation of relationships between individual predictor variables and the outcome is not an issue (Pedhazur, 1982).

Method

Data

A proportional, stratified random sample was selected to represent the population distribution of profit and nonprofit nursing homes and to capture the geographic and racial diversity of Texas. The criterion for inclusion in the study sample was nursing homes that had one or more RN FTE(s). A total of 380 nursing homes were invited to participate. Of the 380 nursing homes contacted, 195 (51%) participated in the study, with 164 (43%) providing complete data. The sample for this study was part of a larger, ongoing study about the outcomes of nursing management practice in nursing homes. The research reported here used secondary data that were obtained from the Texas Department of Human Services (TDHS). Data on resident characteristics and resident outcomes were obtained from the 1995 Client Assessment, Review, and Evaluation (CARE) Form 3652-A (TDHS, 1990). In pilot studies, nurses from TDHS compared their assessments with those of nursing home facility nurses every 9 months and estimated interrater reliability was .94 or greater (Wilson et al., 1990). Data for calculating a single, facility-level case mix CMI were obtained from the Texas Facility Medicaid Cost Reports (TDHS, 1995). Data in both the CARE and Texas Facility Medicaid Cost Reports shared the characteristics of comprehensiveness and inclusiveness (Iezzoni, 1997), thus facilitating comparisons.

We conducted t-tests to assess the extent to which the proportional stratified sample of 164 nursing homes represented the population characteristics of nursing homes in Texas. Bonferroni’s test was used to adjust the alpha level (p < .05) for inflation of Type I errors, which occur due to multiple comparisons (Hays, 1988, p. 411). The t-tests results showed that differences existed for only one of the 10 resident outcomes and only one of the 22 resident risk factors. There was a higher prevalence of aggressive behavior (t[164] = p < .005) and a lower prevalence of hearing impairment (t[164] = p < .002) in the sample that in the population at large. These results suggest that the sample is representative of the population with respect to resident outcomes and risk factors.

Procedure

Case mix Indices.

Two types of case mix indictors were examined in this study. A facility-level, composite index of case mix was compared to a prevalence-based, multi-indicator of case mix, which measured the prevalence of selected resident characteristics.

The facility level, composite CMI

is a weighted combination of the percentage of resident days in each of 11 TILEs (Texas Index of Level of Effort) obtained from the Medicaid Cost Reports (TDHS, 1995). TILEs are described in Table 1. The CMI was calculated using a standardized formula originally designed to assign reimbursement to a facility based upon its overall CMI (TDHS, 1995). The formula is as follows:

Table 1.

TILE – Texas Index of Level of Effort^a

	Activity of Daily Living	CMI Value	Descriptor	Definition
TILE 1	Severely compromised	2.11	Heavy Care	Ability for self-care in activities of daily living is severely limited and resident requires sophisticated treatments such as frequent wound care or tracheostomy care.
TILE 3	Moderately compromised	1.65
TILE 2	Any level	1.81	Rehabilitation	Requires physical or occupational therapy 3 or more times per week.
TILE 4	Severely compromised	1.3	Clinically Complex	Has acute or chronic condition requiring increased monitoring and or therapies such as oxygen administration or respiratory therapy.
TILE 6	Moderately compromised	1.18
TILE 8	Minimally compromised	.95
Tile 5	Severely compromised	1.60	Clinically Stable	Requires assistance and or supervision at varying levels depending on self-care abilities in activities of daily living.
TILE 7	Moderately compromised	1.01
TILE 9	Minimally compromised	.85
TILE 10	Minimally compromised	.65		Mental or behavioral condition
TILE 11	Minimally compromised	.61		No mental or behavior condition

Note: CMI = case mix index. TILE = Texas Index of Level of Effort.

^aObtained from Texas Department of.

$$\begin{array}{l}\text{CMI}=(2.1128\times \text{percent of resident days in Tile }1)+\\ (1.8144\times \text{percent of resident days in Tile }2)+\\ (1.6815\times \text{percent of resident days in Tile }3)+\\ (1.2987\times \text{percent of resident days in Tile }4)+\\ (1.1594\times \text{percent of resident days in Tile }5)+\\ (1.1798\times \text{percent of resident days in Tile }6)+\\ (1.0118\times \text{percent of resident days in Tile }7)+\\ (.9552\times \text{percent of resident days in Tile }8)+\\ (.8472\times \text{percent of resident days in Tile }9)+\\ (.6539\times \text{percent of resident days in Tile }10)+\\ (.6066\times \text{percent of resident days in Tile }11)\end{array}$$

The facility level, prevalence-based CMI

was derived from the presence or absence of 22 resident characteristics reflecting risk (defined in Table 2). The 22 items were obtained from the CARE Form 3652-A (TDHS, 1990). A facility-level prevalence rate was calculated for each risk factor to indicate the percentage of residents in the facility possessing the risk factor. To reduce the influence of natural variation in resident characteristics over time, we averaged values for each risk factor from two time periods, which together reflected resident assessments done in 1995. Skewness, a natural attribute of resident characteristics in nonrandom samples, required that we subject 12 of the 22 resident risk factors to log linear transformation procedures. The log transformation was applied to the following resident risk factors: dressing-grooming, vision, level of consciousness, seizures, dyspnea, tremors, stasis ulcer, hemi or paraplegia, quadriplegia, amputation in past six months, internal bleeding, and terminal illness.

Table 2.

Definitions, Means^a, and Standard Deviation of Resident Risk Factors

Variable names	Definitions	M	SD
	Percentage of residents in the facility:
Mobility-ambulation	Who cannot independently move about or walk	78.43	11.45
Dressing-grooming	Who cannot independently obtain, put on, fasten, take off clothing or maintain personal hygiene	89.70	8.07
Transferring	Who cannot independently move between positions, such as from bed to chair	68.66	12.27
Eating	Who cannot independently get food or fluids by any means from a receptacle into the body	45.27	12.77
Toileting	Who cannot independently get to and from the toilet (including bedpan, beside commode, urinal), transfer on and off toilet, cleanse self after elimination, and adjust clothing	71.32	10.92
Bladder control	Without ability to exercise voluntary control over bladder elimination under normal circumstances	66.62	11.17
Bowel control	Without ability to exercise voluntary control over bowel elimination under normal circumstances or provide self care to ostomy	51.27	10.28
Vision	Who are visually impaired and require supervision or physical assistance to complete visually cued tasks	7.66	5.27
Hearing	Who cannot hear speech at regular conversational levels even in noise and in groups	59.74	19.09
Functional communication	Who cannot communicate desires and needs for physical, mental, and/or social comforts	34.80	10.59
Orientation-memory	Who have more than occasional episodes of disorientation or forgetfulness and require assistance from staff for reorientation more than once a week.	77.59	11.35
Level of consciousness	Who are semi-conscious (reacts to sensory stimuli but unaware of surroundings) or comatose (unresponsive to any stimuli)	1.86	3.03
Seizures	With on seizure precautions or with seizure activity within the last four weeks	11.06	7.68
Dyspnea	With difficult or labored breathing within the past four weeks	9.06	10.36
Edema	With retention of fluid in subcutaneous body tissues, resulting in swelling (not associated with soft tissue injury), within the past four weeks	16.52	9.90
Tremors	Who exhibited within the past four weeks, involuntary movement or shaking of the upper extremities that interferes with function such that supervision or assistance is required for activities of daily living	4.52	5.41
Stasis ulcer	Who exhibited within the past four weeks, open lesion, caused by chronic venous insufficiency	1.38	2.12
Hemi. Or paraplegia	Who exhibited within the past four weeks, paralysis/paresis of one side of the body including both the arm and the leg or of the lower part of the body including both legs	9.96	7.97
Quadriplegia	Who exhibited within the past four weeks, paralysis/paresis of the body including all four extremities and has a medical diagnosis of quadriplegia	.93	1.52
Amputation in past 6 months	Who have had amputation of a limb within the past six months	.86	1.26
Internal bleeding	Who exhibited within the past four weeks, blood loss caused by a subacute or chronic condition such as gastro-intestinal, respiratory, or genito-urinary disorders	.29	1.14
Terminal illness	Whose medical record contains a prognosis that the resident’s condition is likely to rapidly deteriorate and death may be within six months	.75	1.67

^aUnadjusted means – interpreted as percentage of residents with the condition.

Resident outcomes.

Resident outcomes were defined as the results of nursing care experienced by the residents within each home and were derived from the CARE Form 3652-A (TDHS, 1995). Ten indicators (defined in Table 3) were selected because they reflected quality of the nursing care. A facility level prevalence rate for each outcome was calculated to indicate the percentage of residents in the home for which the outcome occurred. To reduce the influence of natural variation over time, we averaged values for each indicator from two time periods, which together reflected the resident assessments done in 1995. Factor analysis using maximum-likelihood estimation and Varimax rotation was used to reduce the 10 resident outcome indicators into a meaningful, but smaller, number of indicators for inclusion in selected analyses. An outcome was retained if it had a factor loading of .35 or greater and the factor explained at least 5 percent of the variance (Munro & Page, 1993).

Table 3.

Definitions, Means^a, and Standard Deviations of the Resident Outcome Indicators (N = 164)

Variable names	Definitions	M	SD
	Percentage of residents in the facility:
Verbal aggression	Displaying verbal aggression within last 4 weeks	12.29	8.99
Physical aggression	Displaying physical aggression within last 4 weeks	8.16	7.09
Other disruptive behavior	Displaying other disruptive behavior within last 4 weeks	12.72	10.67
Geriatric-chair	For whom geriatric chair restraints were used within last 4 weeks	7.57	6.11
Vest-belt restraint	For whom vest-belt restraints were used within last 4 weeks	17.93	9.43
Wrist-mitten restraint	For whom wrist-mitten restraints were used within last 4 weeks	.69	1.11
Decubitus ulcer	Exhibiting stage I or higher decubitus ulcer in last 4 weeks	7.68	4.30
Contractures	Exhibiting contracture in one or more extremities in past 4 weeks	18.21	8.69
Urinary tract infection	Exhibiting urinary tract infection in past 4 weeks	3.63	4.30
Fracture	For whom fracture(s) occurred within past 3 months	2.86	2.83

^aUnadjusted means – interpreted as percentage of residents with the condition.

A four-factor solution accounted for 49% of total variance in resident outcomes. The four factors were theoretically meaningful and were labeled as behavior problems, fractures, complications of immobility, and use of physical restraints (Table 4). The goodness of fit test (χ²[11, N = 164] = 9.83, p = .55) indicated that the factor model was a good estimate of the observed relationships among the items. Two additional tests increased our confidence in the solution. Kaiser-Meyer-Olkin (KMO) values greater than .6 indicate appropriate sampling adequacy (Kaiser, 1974). Bartlett’s sphericity test statistic for the presence of an identity matrix indicates an appropriate factor model when the resulting statistic is large and significance level is small. In this study, the KMO was .69 and Bartlett’s sphericity test statistic was large (χ²[45, N = 164] = 428.67, p < .001), indicating that the factor analysis model far exceeded the values indicating adequacy (SPSS, 1997). Factor scores were retained and used later as the criterion variable in a series of regression analyses. Because an orthogonal rotation was used in the factor analysis procedure, the factor scores captured the unique variance for that set of items.

Table 4.

Rotated Factor Loadings for the Resident Outcome Indictors

	Factor 1	Factor 2	Factor 3	Factor 4
Variable name	Behavior Problems	Fracture	Complications of Immobility	Use of physical restraints
Verbal aggression behavior	.98
Physical aggression behavior	.79
Other disruptive behavior	.78
Fracture		.98
Contracture			.64
Urinary tract infection			.48
Decubitus			.38
Geriatric-chair restraints				.43
Wrist-mitten restraints				.41
Vest-belt restraints				.36
Eigenvalue	2.32	1.07	.83	.68
Percent variance explained	23.22	10.65	8.29	6.83
Cumulative variance explained	23.22	33.87	42.16	48.99

Analysis

Hierarchical multiple regression analysis was used to compare the ability of the facility-level, composite CMI to the prevalence-based, multi-indicator CMI, comprised of 22 separate risk indicators, in explaining the variance contained within the four resident outcomes. The explanatory power of each CMI approach was evaluated by reversing the hierarchical order of entry for each method and then comparing the proportions of explained variance. In the first regression model, each of the four outcomes were regressed on the facility-level, composite CMI followed by the block of 22 resident risk factors comprising the prevalence-based, multi-indicator CMI. Conversely, in a second regression model for each outcome, the 22 separate resident risk factors were entered in a block first, followed by the facility-level, composite CMI.

Results

Means and standard deviations of each resident risk factor and outcome are displayed in Tables 2 and 3, respectively. Bivariate Pearson correlation coefficients between the 22, prevalence-based, resident risk factors and the four outcomes and the facility level, composite CMI are summarized in Table 5. The correlations demonstrated that the facility level, composite CMI was not related to behavioral problems or fractures but was significantly correlated to complications of immobility and restraint use. In a similar fashion, the relationships between the 22 prevalence-based risk factors and resident outcomes were strongest between the complications of immobility and restraint use. For the 22 prevalence-based risk factors, statistically significant relationships with immobility and restraint use clustered among ADL-function and bowel and bladder control.

Table 5.

Correlations between Resident Outcomes, Facility-Level Composite CMI, and 22 Prevalence-Based Resident Risk Factors (N = 164)

Variable Name	Behavioral Problems	Fractures	Complications of Immobility	Use of Physical Restraints	Facility Level, Composite CMI
Facility Level, Composite CMI	.03	.00	.39***	.31***	1.00
Mobility-Ambulation	−.18*	−.02	.30***	.10	.31***
Dressing-Grooming	.01	−.02	.25**	.14	.31***
Transferring	−.12	.04	.28***	.17*	.41***
Eating	.32***	.04	.26***	.28***	.37***
Toileting	.02	.07	.26***	.35***	.42***
Bladder Control	.09	−.06	.34****	.36***	.34***
Bowel Control	−.04	−.01	.22**	.43***	.49***
Vision	−.01	.05	.00	.00	.15
Hearing	.09	−.07	.14	−.11	−.02
Functional Communication	.21**	.04	.12	.34***	.28***
Orientation-Memory	.19*	−.04	−.02	.03	.01
Level Of Consciousness	−.10	−.23**	−.06	−.10	.14
Seizures	.10	−.26***	.08	.04	.09
Dyspnea	.10	−.19*	.02	−.23**	−.01
Edema	.30***	−.04	.16*	.01	−.01
Tremors	.19*	−.04	.08	−.02	.15*
Stasis Ulcer	−.07	.03	.13	−.13	.02
Hemi. Or Paraplegia	−.19*	−.06	.30***	−.04	.25**
Quadriplegia	−.03	.03	.24**	.06	.33***
Amputation In Past 6 Months	−.24**	.09	.09	.07	.23**
Internal Bleeding	−.06	.10	−.03	.05	.15
Terminal Illness	−.15	−.07	.08	−.19*	.11

Note: CMI = case mix index.

^*p ≤ .05.

^**p ≤ .01.

^***p ≤ .001.

Conversely, the relationships between the same 22 prevalence-based risk factors and behavior problems and fractures did not show a recognizable pattern. Behavioral problems related positively to need for assistance with eating, functional communication, orientation, and edema and related negatively to problems in mobility/ambulation, hemi or paraplegia, and amputation in past 6 months. Fracture related negatively to level of consciousness, seizures, and dyspnea.

The relationships between the facility level, composite CMI and the 22 prevalence-based resident risk factors were similar to the relationships identified between the 22 risk factors and the immobility and restraint use outcomes. A moderately positive correlation pattern emerged between the CMI and all of the ADL-function and bladder and bowel control risk factors. In addition, the CMI was positively correlated with functional communication, stasis ulcer, para and quadriplegia, and amputation in past 6 months.

Table 6 displays the results of the hierarchical regression analysis comparing the ability of each case mix approach to explain the variance in the four resident outcomes under study. The prevalence-based, multi-indicator CMI, composed of 22 separate resident risk factors, consistently explained more variance in outcomes than the facility level, composite CMI. The prevalence-based, multi-indicator CMI explained 40% of the variance in behavior problems, 22% of the variance in fractures, 37% of the variance in complications of immobility, and 36% of the variance in physical restraint use. On the other hand, the facility level, composite CMI explained no variance in behavior problems or fractures. It performed somewhat better when accounting for the influence of case mix on complications of immobility and use of physical restraints, explaining 15% and 9% of the variance, respectively. Figure 1 depicts the proportions of variances explained by each CMI approach.

Table 6.

Results of Hierarchical Regression Analysis Comparing Two Case Mix Indicators

	Behavior Problems			Fractures			Complications of Immobility			Use of Physical Restraints
	R2 Cumulative	R2 Δ	F Δ	R2 Cumulative	R2 Δ	F Δ	R2 Cumulative	R2 Δ	F Δ	R2 Cumulative	R2 Δ	F Δ
Model 1
Facility-level, composite CMI	.00	.00	.14	.00	.00	.00	.15	.15	29.41***	.09	.09	16.78***
Prevalence-based, muti-indicator CMI -(22 resident risk factors)	.42	.42	4.63***	.22	.22	1.79*	.42	.27	2.86***	.37	.28	2.77***
Model 2
Prevalence-based, muti-indicator CMI -(22 resident risk factors)	.40	.40	4.37***	.22	.22	1.80*	.37	.37	3.74***	.36	.36	3.56***
Facility-level, composite CMI	.42	.02	3.88*	.22	.00	.13	.42	.05	11.31***	.37	.01	2.54

Note: CMI = case mix index.

^*p ≤ .05.

^**p ≤ .01.

^***p ≤ .001.

Discussion

In nursing systems outcomes research, failure to adjust for case mix can lead to “inaccurate interpretation of organizational factors related to … [an adverse event]” (Mitchell & Shortell, 1997, p. NS28). Because there is more than one way to control for case mix (Iezzoni, Ash, Shwartz, Landon, & Mackiernan, 1998), in this study, we explored the results of two approaches and discuss the implications of each. A facility-level composite CMI and a prevalence-based, multi-indicator CMI, comprised of 22 separate resident risk factors, were compared in attempts to examine potential and appropriate uses of case mix data that is currently available in administrative databases. By studying these approaches to case mix adjustment, we attempt to address weaknesses identified in previous work (Mitchell & Shortell) and extend this work to take advantage of the information available in nursing home facility administrative data.

The facility-level, composite CMI was developed for payment purposes. Its developers state that “the principal goal of case mix measurement is to identify patient characteristics associated with measured resource use” (Fries et al., 1994). Because the facility-level composite CMI is based on assessment data, however, it has the potential to provide “risk adjustments for evaluating outcomes” (Fries et al.).

Our findings suggest that the composite CMI does not explain substantial variance in resident outcomes, at least as defined in this study. It explains some variance in complications of immobility (15%) and use of physical restraints (9%) but none in behavior problems and fractures. One possible explanation for this result is that the facility-level composite CMI, designed to explain resource use, was sensitive to the complications of immobility including decubitus ulcers, contractures, and urinary tract infections because of the relatively greater amount of resources required to successfully treat these conditions. Resources typically used in the treatment of these conditions include registered nurses and license vocational/practical nurses’ time. Registered nurses time was heavily weighted in the development of the facility-level, composite CMI (Phillips, Hawes, & Fries, 1993). Certified nurse assistants’ time will be used heavily in the presence of these conditions for turning and toileting activities. Use of physical restraints is relatively resource intensive because of the time and attention needed to fulfill regulatory standards for periodic release and exercise and to provide care for incontinence and activities of daily living. This conclusion is supported by Phillips et al. who found that after controlling for differences in impairment and care needs, “residents who are physically restrained require more nursing care than other residents” (Phillips et al., p. 342).

Reasonable explanations for the finding of no relationship between the facility-level composite CMI and behavior problems and fractures are more difficult to offer than when attempting to explain the variance observed in complications of immobility and restraint use. With regard to behavior problems, Whall, Gillis, Yankou, Booth, & Beel-Bates (1992) found that, of methods for preventing or mitigating disruptive behavior, nurses reported using verbal discussion (counseling) most often, followed by chemical and physical restraints. We know from Phillips et al.’s (1993) study that restraint use is resource intensive. These prior studies would suggest that a resource-based CMI, such as the facility-level composite CMI, should relate to behavior problems. We would also expect a relationship between the facility-level composite CMI and fractures because residents with new fractures should need rehabilitation, greater than normal assistance with ADL, and special attention to range of motion exercises. The empirical evidence in this study, however, showed no relationships between behavior or fractures, and the facility-level, composite CMI. These findings could be an artifact of the facility-level composite CMI, resulting from the way in which weights were derived during its development. For example TILE 10, mental or behavioral condition, is weighted .65 while weights for heavy care and rehabilitation (TILEs 1 and 2) range from 1.65 to 2.11 (Table 1). This weighting might explain the lack of relationship between the facility-level composite CMI and behavior problems. However, it does not help to understand the findings for fractures, which would be likely to fall into TILE 2, rehabilitation, with a relatively large weight of 1.81. It is possible that fractures occurred so infrequently in the study sample that the facility-level composite CMI was not sensitive to it. Investigators who examine fractures and behavior problems as outcomes in future research should be cautious about using the facility-level, composite CMI to control for case mix without further study.

The prevalence-based, multi-indicator CMI, with 22 resident risk factors was superior to the facility-level, composite CMI for explaining resident outcomes. The proportion of variance in outcomes that the risk factors accounted for ranged from 22% to 40%. While the facility-level, composite CMI has been suggested as a potentially useful tool for risk adjustment, prior research in hospitals suggests that a summary measure, such as the composite CMI, will never be as useful as multiple indicators for explaining differences in outcomes (Elixhauser, Steiner, Harris, & Coffey, 1998). This is because some risk factors will be relevant for some outcomes and not relevant for others. Including several risk indicators allows for the varying impact of the risk factors on different outcomes, something that a composite index cannot do.

A drawback to using multiple risk factors is that statistical analysis of the model becomes complex when multiple control variables are included. This can be avoided easily, however, by using the strategy described by Anderson, Hsieh, and Su (1998) and Zinn, Aaronson, and Rosko (1993). The strategy is to regress the outcome indicator on the risk factors using multiple regression and saving the residual as a risk-adjusted outcome variable. Figure 1 depicts the portions of variance explained by the facility-level composite CMI (row 1) and the prevalence-based, multi-indicator CMI (row 2). The portions of variance that are unexplained are labeled “remaining variance.” The residual variance is risk-adjusted outcome. It is the difference in outcome between what is predicted (expected) due to the risk factors and the actual outcome achieved. The unexplained variance (i.e., the risk-adjusted outcome) is open for explanation by varying levels of nursing and health care quality. Comparing row 1 to row 2 highlights the danger of inappropriate case mix adjustment. The figure suggests that it is more likely that poor quality will unjustly be attributed to a nursing home when the facility-level, composite CMI is used (top row) to adjust for case mix than when the prevalence-based, multi-indicator CMI is used (bottom row).

Several limitations associated with the use of administrative data should be considered when interpreting the findings reported herein. First, there will always be a more clinically precise method for identifying risk factors than is possible when using a secondary database (Elixhauser et al., 1998). The present work, however, offers a reasonable method for using existing administrative data to adjust for case mix. In addition, it is a reasonable alternative to extracting precise clinical data through record review or other methods, which are not always feasible when doing large-scale systems studies requiring comparisons among numerous nursing homes. Hence, the appropriate use of secondary databases is a requirement for the advancement of nursing systems research ((Mitchell, Heinrich, Moritz, & Hinshaw, 1997); Ozbolt, 1992).

Second, when using the Client Assessment, Review, and Evaluation Form 3652-A (TDHS, 1990) it was not possible to know whether a resident was admitted with a preexisting adverse condition such as the decubitus ulcer. In such instances, poor quality of care could be unjustly attributed to a nursing home. In large-scale studies with random samples, however, this should not be a concern because most nursing homes will be affected similarly thus reducing systematic effects. Nonetheless, in the absence of temporal markers for the onset of adverse events, this is a potential consideration for interpretation of quality findings.

A third limitation of this study is that assumptions were made that the risk factors identified in this study were preexisting to the adverse outcome. This assumption, however, cannot be verified and is a limitation common to most administrative databases (Elixhauser et al., 1998; Iezzoni et al., 1998). It is quite possible that the factors identified as predisposing risks to adverse outcomes were in reality a result of the adverse outcome. The development of conditions after the onset of an adverse outcome could be a result of the resident’s underlying risk factors or the result of the quality of care problems associated with the management of the adverse outcome (Iezzoni et al.). The inability to discern whether a risk factor was preexisting to an adverse outcome or the result of an adverse outcome increases the potential for over estimating the influence of risk factors on outcomes. Thus, it should be noted that our estimates might overstate the contributions of the facility-level, composite CMI and the prevalence-based, multi-indicator CMI in explaining the variance associated with the outcomes under study. Overestimating the influence of case mix will weaken the ability to detect differences in resident outcomes that are due to quality of care.

Despite the limitations inherent to using administrative data, they are of value because they provide a means to adjust for case mix when doing large-scale studies that would not otherwise be possible. Using risk adjustment usually is necessary when comparing one facility to another (Shaughnessy, Kramer, David, & Steiner, 1995). It is also necessary when comparing resident outcomes from one time period to the next or between 2 nursing care units in a single nursing home (Zimmerman, et al., 1995). Because of this, researchers should chose risk adjustment systems that make sense to practitioners (Iezzoni et al., 1998). Researchers must anticipate claims by nursing facility owners, administrators, and nurses that their resident outcomes are worse because their residents are sicker than in comparison facilities. Practitioners and managers will be more open to research findings if they understand how “the playing field was leveled” for all facilities participating in the study. The case mix strategy employed will be more acceptable to practitioners and managers if it has face validity. In the present work, it is likely that clinicians will find the 22 risk factors comprising the prevalence-based multi-indicator CMI clinically relevant and more useful for case mix adjustment than the facility-level, composite CMI. Managers who do not accept the risk adjustments used in research will be reluctant to allow their facilities to be included in quality comparisons. In addition, they will be more likely to turn away residents with several existing risk factors because their heavier-care needs may contribute to higher facility costs (Iezzoni et al.).

In nursing systems research, the aim is for meaningful results from which to make strong recommendations for changes in practice. Conservative risk adjustment, such as that provided by the prevalence-based, multi-indicator CMI, might serve as the basis for those recommendations. Managers may be more willing to consider practice changes when they know the recommendations are based on studies that adjusted for variations in resident populations between facilities. Risk adjustment methods that are sensitive to critical patient differences can more readily inform practice changes that are responsive to those differences.

Recommendations for further study are as follows. Replicating this study in samples from states that use a different resident assessment form in case mix reimbursement would help in refining an approach to risk-adjustment for nursing outcomes research in nursing homes. To identify states that are using case mix reimbursement systems see Fries et al. (1994) and Weissert and Musliner (1992). Weissert and Musliner describe the nature of the data collected in those systems. Another area for research would be to explore comparable systems for adjusting case mix in hospital-based nursing outcomes research. The ORYX requirements of the Joint Commission on the Accreditation of Healthcare Organizations (1997) may provide avenues for such research. ORYX is the Joint Commission’s initiative to integrate performance measurement data into the survey process. The Outcome and Assessment Information Set (OASIS), being phased into use in home health agencies during 1999 (Health Care Financing Administration, 1998), also may provide avenues for such research. Finally, other approaches to combining multiple risk factors such as using epidemiological methods (Zimmerman et al., 1995) may provide useful directions for research.