Abstract
Purpose
The purpose of this research is twofold. The first purpose is to utilize a new methodology (Bayesian networks) for aggregating various quality indicators to measure the overall quality of care in nursing homes. The second is to provide new insight into the relationships that exist among various measures of quality and how such measures affect the overall quality of nursing home care as measured by the Observable Indicators of Nursing Home Care Quality Instrument. In contrast to many methods used for the same purpose, our method yields both qualitative and quantitative insight into nursing home care quality.
Design and Methods
We construct several Bayesian networks to study the influences among factors associated with the quality of nursing home care; we compare and measure their accuracy against other predictive models.
Results
We find the best Bayesian network to perform better than other commonly used methods. We also identify key factors, including number of certified nurse assistant hours, prevalence of bedfast residents, and prevalence of daily physical restraints, that significantly affect the quality of nursing home care. Furthermore, the results of our analysis identify their probabilistic relationships.
Implications
The findings of this research indicate that nursing home care quality is most accurately represented through a mix of structural, process, and outcome measures of quality. We also observe that the factors affecting the quality of nursing home care collectively determine the overall quality. Hence, focusing on only key factors without addressing other related factors may not substantially improve the quality of nursing home care.
Keywords: Bayesian networks, Nurse staffing, Nursing home quality, Occupancy rate, Quality indicators
Determining what factors significantly influence the quality of care in nursing homes has presented a difficult challenge and is a growing concern. Despite calls for reform (Institute of Medicine, 1986), recent reports conclude that the quality of care provided in some facilities still leaves much to be desired (Institute of Medicine, 1996, 2001). As the United States will encounter an unprecedented number of Americans who require skilled nursing care in the upcoming decades, the need for a more thorough understanding of nursing home care quality has never been more prominent.
In this study we examine what factors influence the quality of nursing home care through a multivariate framework known as Bayesian networks. The intent of our research is twofold. We seek to utilize a new methodology for aggregating various quality indicators to measure the overall quality of care in nursing homes. We also seek to provide new insight into the relationships that exist among various measures of quality and how such measures affect the overall quality of nursing home care as measured by the Observable Indicators of Nursing Home Care Quality Instrument (Rantz & Zwygart-Stauffacher, 2006; Rantz, Mehr, et al., 2006). Two aspects of the study provide such insight. First, the Bayesian network framework allows us to examine previous research to determine which combinations of variables most significantly influence the quality of care. Second, an inspection of the structure and parameters that characterize a Bayesian network both qualify and quantify aspects of nursing home care quality. The insights gained from this analysis not only provide a unique perspective on the makeup of nursing home care quality, but they also afford practical applications for quality improvement.
Background
In recent decades, researchers have conducted numerous studies to gain a better understanding of the quality of nursing home care. In this section we do not intend to provide an exhaustive review of such studies. Rather, we offer a survey of studies, many of which we think are representative of important advances in this field. The survey is divided into two sections. The first addresses studies focusing on specific factors that influence nursing home care quality; the second is devoted to studies that aggregate information to measure the overall quality of care in nursing homes.
Factors Influencing the Quality of Nursing Home Care
Factors influencing the quality of nursing home care delivery can be separated into three categories: structure, process, and outcome. Common structural measures of quality include facility size, occupancy, ownership type, staffing, and percentage of Medicaid and Medicare residents. Most studies concur that larger facilities exhibit lower levels of quality than do smaller facilities (Harrington, O’Meara, Kitchener, Simon, & Schnelle, 2003; O’Neill, Harrington, Kitchener, & Saliba, 2003). Research examining the relationship between the quality of nursing home care and occupancy rate has produced mixed results. Some studies show that higher occupancy rates are associated with a higher use of restraints, more pressure sores, a greater use of psychoactive drugs, and lower total nurse and registered nurse staffing hours (Castle, 2001; Harrington & Swan, 2003). In contrast, research conducted by Zinn, Aaronson, and Rosko (1993) indicates that higher mortality rates are associated with lower occupancy rates. Investigations into the relationship between the proprietary status of nursing homes and the quality of care overwhelmingly favor nonprofit institutions over for-profit institutions (Aaronson, Zinn, & Rosko, 1994; Harrington, Woolhandler, Mullan, Carrillo, & Himmelstein, 2001; O’Neill et al.). Although one study reports that staffing levels account for only a small portion of the total variation in quality of care (Harrington, Zimmerman, Karon, Robinson, & Beutel, 2000), the general consensus of research is that more staffing of all types improves nursing quality (Schnelle et al., 2004).
According to the authors of some studies, higher percentages of Medicaid residents tend to negatively influence nursing quality and staff levels (Grabowski, 2001; Harrington & Swan, 2003). However, Chesteen, Helgheim, Randall, and Wardell (2005) posited that, in their study, the percentage of Medicaid residents did not contribute adversely to outcome quality. Research conducted by Harrington and colleagues (2001, 2003) seems to indicate that the percentage of Medicare residents has a positive affect on the quality of care in nursing homes. Although research to date has yielded various results on the relationship of structural measures to overall nursing home quality, such measures are clearly an influential aspect of the quality of nursing home care.
Process measures of nursing home quality describe the process of care provided. Outcome quality measures, in contrast, measure the results of nursing care processes. An often utilized set of clinical quality indicators (which encompass both process and outcome measures) was developed by researchers at the Center for Health Systems Research and Analysis (CHSRA) at the University of Wisconsin–Madison. These quality indicators (QIs) measure the proportion of nursing home residents with the QI condition. The latter portion of Table 1 lists the 24 CHSRA QIs (QI variable names followed by the letter N were revised in 1997). Despite their widespread use, clinical QIs are not without criticism. Critics point to flaws with the Minimum Data Set as well as with the QIs themselves (General Accounting Office, 2002; Office of the Inspector General, 2001). Although a number of studies indicate that clinical QIs require further validation and testing in order for researchers to determine how accurately they predict the quality of care in nursing homes, critics admit that facilities with better quality can be differentiated among those with average or poorer quality on the basis of such indicators (Harrington et al., 2003).
Table 1.
Variable | Description | Model(s) |
---|---|---|
QOC | The QOC as determined by the total OIQ score | All |
BedSz | Number of beds in a facility | BN2 |
Occupancy | Number of beds occupied in a facility | BN3; Reg2 |
Medicare | Indicates whether or not a facility accepts Medicare residents | BN2 |
Chain | Indicates whether or not a facility is part of a nursing home chain | BN2; Reg2 |
RNhrs | The number of RN hours per resident day | BN2; Reg2 |
CNAhrs | The number of CNA hours per resident day | BN2,3 |
Totalhrs | The number of total staff hours per resident day | BN2; Reg2 |
Deficiencies | The number of deficiencies issued to a facility | BN3; Reg1, Reg2 |
FP | Indicates FP facilities | BN2 |
NP | Indicates NP facilities | BN2 |
GOV | Indicates GOV facilities | BN2 |
QI1N | Prevalence of any injury | BN1, Reg2 |
QI2 | Prevalence of falls | BN1 |
QI3 | Prevalence of behavioral symptoms affecting others | BN1 |
QI4N | Prevalence of diagnosis or symptoms of depression | BN1 |
QI5N | Prevalence of depression with no treatment | BN1 |
QI6 | Use of 9 or more different medications | BN1 |
QI7N | Onset of cognitive impairment | BN1 |
QI8 | Prevalence of bladder or bowel incontinence | BN1 |
QI9 | Prevalence of occasional bladder or bowel incontinence without a toileting plan | BN1 |
QI10 | Prevalence of indwelling catheters | BN1 |
QI11 | Prevalence of fecal impaction | BN1 |
QI12 | Prevalence of urinary tract infections | BN1 |
QI14 | Prevalence of weight loss | BN1 |
QI15 | Prevalence of tube feeding | BN1; Reg2 |
QI16 | Prevalence of dehydration | BN1 |
QI17 | Prevalence of bedfast residents | BN1,3 |
QI18 | Incidence of decline in late-loss ADLs | BN1 |
QI20N | Lack of training–skill practice or ROM for mobility-dependent residents | BN1 |
QI21N | Prevalence of antipsychotic use, in the absence of psychotic and related conditions | BN1; Reg2 |
QI23N | Prevalence of antianxiety or hypnotic use | BN1 |
QI24 | Prevalence of hypnotic use more than two times in past week | BN1 |
QI26 | Prevalence of daily physical restraints | BN1,3 |
QI27 | Prevalence of little or no activity | BN1; Reg2 |
QI29 | Prevalence of Stage 1–4 pressure ulcers | BN1 |
Note: BN1, BN2, BN3 = Bayesian networks; QOC = quality of care; OIQ = Observable Indicators of Nursing Home Care Quality Instrument; RN = registered nurse; CNA = certified nurse assistant; FP = for profit; NP = not for profit; GOV = government; ADLs = activities of daily living; ROM = range of motion. Reg1 = the first benchmark model (a simple regression); Reg2 = the second benchmark model (a multiple regression), in which variables were chosen by means of a stepwise selection procedure.
When a facility does not comply with federally imposed standards, the facility may receive a deficiency. As deficiencies may address standards related to processes or outcomes (or both) in a nursing home, they have often been utilized as process and outcome measures, depending on their type and severity. In addition, the number and type of deficiencies have often been utilized as a measure of overall quality in nursing homes. Consequently, relationships between deficiencies and many of the aforementioned structural measures of quality have been rigorously researched.
Models for Assessing the Quality of Nursing Home Care
A popular model that has been applied to the assessment of nursing home care quality was developed by Donabedian (1988). Donabedian proposed three aspects of quality assessment: structure, process, and outcome, known as the SPO framework. The SPO framework served as the foundation for two notable models. Atchley (1991) modified the framework to include a time dimension. Unruh and Wan (2004), noting the lack of causality among structure, process, and outcomes in models utilizing the SPO framework, constructed a “framework for nursing home quality that links contextual components of quality with structure, structure with process, and process with outcomes, focusing on nursing care quality.” The authors suggest further development of the model through structural equation modeling.
Two studies have utilized clinical QIs as predictors of overall nursing home quality. Karon, Sainfort, and Zimmerman (1999) established the stability of QIs over time by ordering facilities by their QI ranks. The QI rank (or percentile), therefore, provides a relative measure of overall quality within a group of nursing homes. Instead of employing the QI rank method, Rantz and colleagues (2004) classified nursing homes on the basis of their individual QI scores and compared them to established QI thresholds (Rantz, Petroski, et al., 2000). The conglomeration of the individual QI ratings yielded an overall classification for the facility—good, average, or poor. Table 2 (where G, A, and P denote the number of QIs in the good, average, and poor ranges, respectively) illustrates the number of QIs within a certain range required to arrive at an overall facility rating.
Table 2.
Classification | Criteria |
---|---|
Good (G) | G ≥ 5 and G–P ≥ 2 |
Poor (P) | P ≥ 5 and P–G ≥ 5 |
Average (A) | A ≥ 15 and neither G nor P |
The Observable Indicators of Nursing Home Care Quality Instrument (OIQ) measures the overall quality of a nursing facility during an on-site visit to a nursing home. Development of the OIQ began through a series of qualitative studies to identify and describe the many dimensions of the quality of care in nursing homes of importance to consumers, providers, and regulators (Rantz et al., 1999). As a result of these studies, Rantz and colleagues proposed two theoretical models of nursing home care quality: one from the point of view of direct-care providers and regulators, and one from the view of consumers. The researchers merged the dimensions from the models into one comprehensive model defining nursing home care quality as a multidimensional concept with seven dimensions: central focus of the agency is on residents, families, staff, and community; care; communication; staff; environment; home; and family involvement. On the basis of this qualitative work, Rantz and colleagues constructed an initial instrument from the descriptive statements of study participants. They constructed items to measure each of the dimensions of the theoretical model of nursing home care quality. As the instrument name implies, each item refers to some directly observable aspect of any nursing home. Four field tests in 123 Missouri nursing homes and a small international sample delivered promising reliability and validity results (Rantz, Mehr, et al., 2000). Subsequent field testing, refinement, and psychometric evaluation were conducted as a field test in 407 nursing homes in three states. Based on the results of exploratory factor analysis, confirmatory factor analysis, classical test theory, and generalizability theory, Rantz and colleagues determined the OIQ to be a valid and reliable instrument for measuring the quality of care in nursing homes (Rantz & Zwygart-Stauffacher, 2006; Rantz et al., 2006).
Because we employ the OIQ as a measure of overall quality in this study, we give a brief explanation of how the instrument gauges nursing home quality. During an on-site visit to a nursing home, an evaluator answers a series of questions related to the aforementioned dimensions of quality. To answer each question, the evaluator chooses from answers associated with a Likert scale. The sum total of the answers constitutes a score for the facility. The score is then transformed into a facility classification of Below Average, Average, or Above Average. The thresholds for these classifications correspond to the 20th and 80th percentiles of nursing homes examined in the studies mentioned earlier. The authors of the OIQ provide more details regarding the OIQ at their Web site (www.nursinghomehelp.org).
Bayesian Networks
Bayesian networks (BNs) are a multivariate method that is useful in modeling complex relationships among variables. Using Bayesian probability theory, BNs estimate the probability distributions of unknown variables from known probabilistic relationships. In this section we address three key components of BNs: structure, parameters, and inference. We illustrate each of these aspects with a simple hypothetical example, and we provide appropriate references for readers seeking additional information. In addition, we also discuss the advantages and weaknesses of BNs as a modeling tool.
Structure
The structure of a BN is a graph, composed of a set of nodes V={X1, …, Xn} and a set of edges E. The nodes of the graph represent random variables, each of which may take on certain values, whereas the edges represent influence among variables. Consider, for example, the hypothetical BN depicted in Figure 1 with variable definitions in Table 3. In this example V = {QI17, QI26, QOC} and E = {(QI17, QI26), (QI26, QOC)}. The structure suggests that the overall quality of care (QOC) in a nursing home is directly affected by the prevalence of daily physical restraints (QI26), which in turn is influenced by the prevalence of bedfast residents (QI17).
Table 3.
Variable | Value | When the Variable Takes on This Value |
---|---|---|
QI17 | QI17L | <50% of residents are bedfast |
QI17H | >50% of residents are bedfast | |
QI26 | QI26L | <50% of residents have daily physical restraints |
QI26H | >50% of residents have daily physical restraints | |
QOC | BelowAvg | The QOC delivered is below average |
Average | The QOC delivered is average | |
AboveAvg | The QOC delivered is above average |
Note: QOC = quality of care.
Given a set of variables, one can develop the structure of a BN in two ways. First, a domain expert may place edges among variables to represent his or her understanding of how the variables interact. Second, one may determine the structure of BNs by means of structure-learning algorithms. Such methods seek for a network structure that maximizes the probability of a particular network configuration, G, given a set of data, d, which one may write as P(G|d). One calculates this quantity by using Bayes’ theorem, which states that P(G|d) is proportional to P(d|G)P(G), where P(d|G) is the likelihood of G and P(G) is the prior probability. The details of structure-learning algorithms are found in the research of Geiger and Heckerman (1994) and Bøttcher (2001).
Parameters
The parameters of a BN provide additional details as to the relationships among variables. Each node in a BN contains a conditional probability table that provides the conditional probability distribution of the node given the various states of its parents (node A is a parent of node B if the BN contains a directed edge from A to B). For example, associated with node QOC in Figure 1 is a conditional probability table that gives the probability of QOC given the two possible values of QI26. The qualitative relationships imposed by a BN’s structure are thereby quantified by its parameters.
Given a network structure, the parameters of a BN may be developed in two ways. First, a domain expert may determine the conditional probability distributions on the basis of his or her knowledge. Parameters may also be developed through parameter-learning algorithms. Such methods determine conditional probability distributions based on a provided data set with known values for each variable. As each nursing home in the data set is presented to the BN, an estimate of the conditional probability distributions is gradually revised. As the details of parameter-learning algorithms are beyond the scope of this article, the interested reader is referred to Spiegelhalter, Dawid, Lauritzen, and Cowell (1993) and Geiger and Heckerman (1994).
Inference
With the structure and parameters of a BN in place, probabilistic inference can be performed within the network. More specifically, one may query the BN for the probability distribution of one or more variables of unknown value given the known values of other nodes in the network. Methods for performing inference in a BN use Bayes’ rule to predict the outcome of events that are dependent on other events, where dependence is given by the network structure.
As an example of inference, the probability distribution of QOC in Figure 1 may be of interest. For purposes of comparison, we begin by calculating this distribution when none of the values of the variables in Figure 1 are known. An outline of the basic calculations involved is given in the following set of equations, which yields P(QOC) = {0.443, 0.378, 0.178}.
If it is known that QI26 = QI26L, then similar calculations yield P(QOC) = {0.600, 0.300, 0.100}. Thus, as additional information becomes available, one adjusts the distribution of QOC accordingly. The various types of inference that can be performed in a BN and the associated algorithms are found in the books by Jensen (1996) and Pearl (1988).
Advantages and Weaknesses of BN Modeling
The primary advantage of a BN is an efficient representation of the joint probability distribution of a set of variables V, denoted P(V). Normally, P(V) would require an entry for each combination of the variables in V and their associated states. As the number of variables increases, the number of entries in P(V) increases exponentially, thereby making it intractable to determine and update P(V) even for moderate numbers of variables. BNs overcome this intractability by exploiting the conditional probability relationships among variables. More specifically, the joint probability distribution can be represented as a product of conditional probabilities,
where pa(Xi) denotes the parents of variable Xi (Jensen, 1996).
As we mentioned earlier, establishing the structure and parameters of a BN by means of learning algorithms may uncover relationships among variables missed by conventional statistical methods. Because these relationships may be highly nonlinear, a BN approach is likely to provide a more instructive model than traditional methods, such as regression. A particularly salient benefit of a BN approach is the ability to obtain rich insight into nursing home care quality. As illustrated in the examples given herein, rich qualitative and quantitative insight is available by inspecting a BN’s structure and parameters. Further insight is available upon querying the BN for the probability distributions of key variables, such as QOC. A simultaneous disadvantage is the unfamiliarity of such measures. For example, regression coefficients and correlation measures may be more familiar to researchers than are conditional probabilities.
Another disadvantage of a BN approach is the method of evaluation. Although some standard classification measures may be applied to a BN, much of a network’s evaluation is application dependent. As an example of this, in a subsequent section we use two measures to evaluate potential BNs. Although we believe these measures are well suited for this study, they may not be applicable for BNs developed for other purposes.
To summarize, BN methodology is in a period of development analogous to structural equation modeling in the 1970s. Additional improvements in search strategy and estimation will likely emerge. For the time being, however, the current state of the methodology provides a very practicable means for studying nursing home care quality. Further explanation of BNs, directed toward nurse researchers, is given by Lee and Abbott (2003).
Methods
Data and Software
As is the case in many modeling studies, the quality of the data was of paramount importance. In this work, the BN structure and parameter-learning algorithms required a preexisting overall measure of nursing home care quality with which to build models that would subsequently serve as a means to gain insight. Of particular interest to this study was the availability of the overall quality score (QOC in Table 1) as determined by the OIQ. As we discussed in the Background section, the OIQ has been determined to be a valid and reliable instrument for measuring the QOC in nursing homes. Therefore, the OIQ score served as our overall measure of nursing home care quality for each nursing home in our analysis.
The data that we analyzed in this study were provided by the University of Missouri–Columbia Sinclair School of Nursing and were gathered as part of a 3-year research project funded by the National Institute of Nursing Research of the National Institutes of Health. Data collected for each nursing home included the items listed in Table 1. The first variable, QOC, is derived from the OIQ. The next 11 variables constitute structural quality measures often utilized in previous research to evaluate the quality of nursing home care. The remaining portion of Table 1 depicts the 24 CHSRA QIs. The far right column of Table 1 specifies variable inclusion in the resulting models. QI data were collected within a 90-day period prior to the determination of the OIQ score. The average time between QI assessment and OIQ assessment was 43 days, with a standard deviation of 26 days. We decreased the initial data set of 256 nursing homes to 234 nursing homes by removing facilities with missing information and potential OIQ scoring errors.
For the purpose of constructing BN models, we transformed the data items in Table 1 into discrete ranges as follows. We labeled the QOC score for each facility as Below Average, Average, or Above Average on the basis of the classification scheme discussed in the Background section. We divided the variables of number of beds in a facility, number of beds occupied in a facility (occupancy), number of registered nurse hours per resident day, number of certified nurse assistant hours per resident day (CNAhrs), number of total staff hours per resident day, and number of deficiencies issued to a facility (deficiencies) into three levels (high, average, and low) based on the 20th and 80th percentiles of the facilities in the data set. We did not modify the variables of whether or not a facility accepts Medicare residents, whether or not a facility is part of a nursing home chain, and whether the facility is for profit, not for profit, or government, as they were binary variables to begin with. Finally, we divided the QIs into three levels on the basis of established QI thresholds. These thresholds, established by experts, classify a nursing home’s performance on a particular QI as good, average, or poor (Rantz, Petroski, et al., 2000).
We partitioned the data set into two sets—a learning set (n = 184) to be used in determining BN structures and parameters and a validation set (n = 50) to be used in validating the BNs. We repeated this process a total of 10 times, resulting in 10 randomly partitioned learning and validation data set pairs. All data sets were proportional to the original data set, with respect to the OIQ score (20% with a Below Average OIQ score, 60% with Average, and 20% with Above Average). Such a method, similar to cross-validation, is a standard procedure for the testing and validation of classifiers (Tan, Steinbach, & Kumar, 2005).
Using the learning data sets, we created BN models with two software packages. We used Deal, a BN software package developed in Denmark by Bøttcher and Dethlefsen (2004), to learn the structures of the BN models. We conducted subsequent parameter learning in Netica, developed by Norsys Corporation. Using the validation data sets, we then evaluated the models with the assistance of Netica as described in the following paragraphs.
Model Creation and Variable Selection
To utilize the BN framework to better understand the relationships among various measures of quality and the overall QOC in nursing homes, we created three BN models (denoted by BN1, BN2, and BN3). We designed the models to utilize research discussed in the Background section of this article and to uncover previously undiscovered relationships among quality measures and the overall QOC. The following paragraphs give a description of the variables included in each model. The far right column of Table 1 also indicates variable inclusion in specific models.
We included variable QOC in each model with the aim of examining relationships among QOC and other sets of variables. The variables in BN1 include the 24 CHSRA QIs; the variables in BN2 are composed of structural measures of quality; and we selected the variables in BN3 by means of the Pearson chi-square test of independence. This standard test, which is useful in detecting associations between categorical variables, assesses whether or not paired observations on two variables (expressed as a contingency table) are independent of each other (Hogg, Craig, & McKean, 2004). We performed the test for each pairwise combination of QOC and the other variables listed in Table 1. We selected variables with a chi-square probability of 0.10 or smaller for inclusion in BN3. Such a method for variable selection in BNs is not uncommon (Marshall, McClean, Shapcott, Hastie, & Millard, 2001).
Evaluation Measures
In this section we describe three models used as a benchmark for comparison with the BN models. Because this is the first application of BNs to nursing home care quality, a point of comparison is necessary to justify the use of BNs. We also describe the evaluation criteria employed to assess the performance of the BN models and the benchmark models. We use these criteria to select the most appropriate BN model with which to gain insight into nursing home care quality.
The first benchmark model (labeled Reg1 in Tables 1 and 4) is a simple regression and follows a more common approach to the examination of nursing home care quality by utilizing the number of deficiencies as a representation of the QOC (Grabowski, 2001; Harrington & Swan, 2003; Harrington et al., 2000, 2001; O’Neill et al., 2003). In this model, (number of) deficiencies is the independent variable employed to predict QOC (the numeric OIQ scores, in this case). The second benchmark model is a multiple regression (labeled Reg2 in Tables 1 and 4) in which we chose variables by means of a stepwise selection procedure. Again, we use these variables to predict the numeric OIQ scores. To calculate prediction accuracy (subsequently described), we transformed the OIQ score predicted by the regression models into Below Average, Average, or Above Average by using the classification criteria of Rantz and colleagues (2006). The third benchmark model is the QI classification method put forth by Rantz and associates (2004). An explanation of this model is provided in the Background section and Table 2.
Table 4.
Model | Prediction Accuracy | Correlation |
---|---|---|
BN1 | 0.494 | 0.211 |
BN2 | 0.568 | 0.186 |
BN3 | 0.560 | 0.364 |
Regression1 | 0.590 | 0.213 |
Regression2 | 0.568 | 0.293 |
QI Classification | 0.460 |
Note: BN = Bayesian network; QI = quality indicator.
After we created the BN and benchmark models with the learning data sets, we used two criteria to evaluate them by means of the validation data sets. The criteria—prediction accuracy and correlation—each provide a different perspective as to what associations do or do not exist among variables in the models and the overall QOC.
Prediction accuracy indicates how accurately each model predicted the likelihood of QOC. For each of the 50 facilities in a validation data set, each model was given knowledge of all variables in the model except for the actual QOC. By means of probabilistic inference schemes for BNs, we obtained the probability distribution of QOC. We considered the state of QOC corresponding to the highest value of the probability distribution to be what the model predicted as the overall quality of care. For example, if the distribution for QOC was determined to be {Pr(QOC = Above Average) = 0.2, Pr(QOC = Average) = 0.3, Pr(QOC = Below Average) = 0.5}, then the model predicted the QOC to be below average for this nursing home. A model’s prediction accuracy is then composed of a comparison of the model’s prediction to the actual QOC for each of the 50 nursing homes in a validation data set.
The correlation measure builds upon prediction accuracy by further utilizing the probability distribution of QOC. For the BN models, each nursing home in a validation set is assigned a score of −1 × Pr(QOC = Below Average) + 0 × Pr(QOC = Average) + 1 ×Pr(QOC = Above Average). The resulting scores are then correlated (using the Pearson product–moment correlation) with the actual QOC values (the numeric OIQ scores). For the regression models, the numeric output is simply correlated with the actual QOC values. A higher correlation is indicative of a higher degree of association between the selected variables and the overall QOC, whereas a lower correlation implies the opposite.
Results
Performance of the three BN models, the regression models, and the QI classification model based on the aforementioned criteria are displayed in Table 4. The prediction accuracy and correlation measures represent the average value for the 10 data sets (any measure referenced hereafter is an average value). The following first compares the performance of the BN models among themselves and then compares BN3 (deemed to be the most accurate BN model) with the regression and QI classification models.
In comparison with the other BNs, BN1 (which includes the 24 CHSRA QIs) incurred the lowest prediction accuracy and posted mediocre correlation. This led us to concur with other researchers that Minimum Data Set QIs, when used as the sole variables, are not very effective in a predictive model (Rantz et al., 2004; Schnelle et al., 2004).
BN2 provides insights into the relationships among structural measures of quality and the overall QOC in nursing homes. The inclusion of only structural variables resulted in an improvement of 7.4% in prediction accuracy when BN2 was compared with BN1. This led us to conclude that structural variables tend to have a greater influence on the overall QOC than do QIs. Despite this improvement, the sole consideration of structural variables as predictors of the overall QOC does not present the most effective network.
Variables selected by the chi-square test of independence for BN3 resulted in a marked improvement in correlation and performed as well as or better than the other models with respect to prediction accuracy. An examination of the structure and parameters of BN3, given in the next section, provides additional insight into the factors that affect nursing home care quality.
Additionally, BN3 constitutes a more accurate representation of the factors influencing the quality of care than do the regression and QI classification models. When comparing Reg1 to BN3, we find that Reg1 achieves a higher rate of prediction accuracy.
However, a more in-depth look reveals that Reg1 almost always predicts the QOC to be average for all of the nursing homes. This is confirmed by its low correlation value. Although it is not shown in Table 4, Reg1 also performs poorly with respect to other relative measures of accuracy. Compared with BN3, Reg2 makes a marginal improvement in prediction accuracy but falls short with respect to correlation. The QI classification method substantially underperforms BN3 on prediction accuracy. In addition, the QI classification method often predicts a facility with above average quality as below average quality and vice versa. The rate of these totally wrong predictions is 13.5%; the BNs rarely make such a prediction error.
Discussion
One strength of the BN approach is its ability to provide both qualitative and quantitative insight into the multidimensional makeup of nursing home care quality. Accordingly, we examine BN3 in more detail. The qualitative insight gained from BN3’s structure depicts how various factors interact with each other to produce certain levels of nursing home care quality. We obtain further understanding of these interactions from the quantitative analysis of BN3’s parameters. Considered together, the quantitative and qualitative aspects of the BN afford a more complete perspective of the QOC in nursing homes. We conclude this section by highlighting the differences between BN methodology and the methods employed in other studies.
Structure
The structure of BN3 (see Figure 2) indicates that the overall quality of nursing home care is tightly related to five quality measures: certified nurse assistant staffing levels (CNAhrs), occupancy rate (occupancy), prevalence of bedfast residents (QI17), prevalence of daily physical restraints (QI26), and number of deficiencies (deficiencies). The direction of the arc between QOC and deficiencies implies that the overall QOC is indicative of the number of deficiencies issued to a facility. This direction of causation makes sense from an evaluation perspective (i.e., the number of deficiencies results from the QOC provided). Conversely, the arc direction between QOC and the remaining variables implies that CNAhrs, occupancy, QI17, and QI26 each influence the overall quality.
A further analysis of BN3’s structure reveals that it is composed of structural, process, and outcome measures. As illustrated in Figure 2, CNAhrs and occupancy provide structural quality measures, QI26 provides a process measure, QI17 supplies an outcome measure, and the variable of deficiencies encompasses both process and outcome measures of nursing home care quality. The inclusion of quality measures from each group provides a more complete view of the QOC in nursing facilities.
Insights from the structure of BN3 also have significant practical implications because the identification of factors that directly influence the overall quality of nursing home care provides direction for nursing home administrators.
Parameters
We obtain further understanding of how the quality of nursing home care is influenced by the aforementioned factors through an examination of the parameters of BN3. To illustrate how the parameters provide insight into the QOC, we present excerpts from the conditional probability tables of BN3 in Table 5.
Table 5.
QI17 (Bedfast) | CNAhrs | Occupancy | QI26 (Restraints) | Pr(QOC = Above Average) |
---|---|---|---|---|
Good | Good | High | Good | 0.674 |
Average | Good | High | Good | 0.542 |
Poor | Good | High | Good | 0.276 |
Good | Good | High | Good | 0.674 |
Good | Average | High | Good | 0.250 |
Good | Poor | High | Good | 0.155 |
Poor | Poor | High | Poor | 0.150 |
Poor | Poor | Average | Poor | 0.150 |
Poor | Poor | Low | Poor | 0.150 |
Note: QI = quality indicator; QOC = quality of care; Pr = probability; QI17 (Bedfast) = prevalence of bedfast residents; CNAhrs = the number of CNA hours per resident day; Occupancy = number of beds occupied in a facility; QI26 (Restraints) = prevalence of daily physical restraints.
The first portion of Table 5 inspects the relationship between the overall quality of nursing home care (QOC) and the prevalence of bedfast residents (QI17). To facilitate this examination, we hold the levels of CNAhrs, occupancy, and QI26 constant while we vary the level of QI17. As shown, the probability that the overall quality of nursing home care is above average decreases with an increased likelihood of poor levels of QI17. In other words, the probability of above average QOC declines as the instances of bedfast residents increase. This is an important observation for nursing home administrators seeking to improve the overall QOC. The magnitude of decline is not severe when the level of QI17 is average, and hence, relatively good overall quality of nursing home care can be maintained as long as the level of QI17 is average or good.
We examine the relationship between certified nurse assistant staffing levels and overall QOC in the second portion of Table 5. As before, we hold the other variables at a constant level while we allow the level of CNAhrs to vary. Similar to the relationship between QI17 and QOC, decreased CNA staffing levels significantly shift the probability distribution for the overall QOC. Unlike the previous case, it is imperative to have good CNA staffing levels to keep above average QOC.
The last portion of Table 5 reveals an interesting observation: The occupancy rate has no affect on the overall QOC when QI17, CNAhrs, and QI26 are at their lowest levels. This is apparent from the repetitive probability distribution of QOC as occupancy moves from high to average to low. Therefore, despite a facility’s occupancy rate, one can achieve little or no QOC improvement without first addressing the prevalence of bedfast residents, the prevalence of daily physical restraints, and certified nurse assistant staffing levels. When other factors such as QI17, CNAhrs, and QI26 are held at their middle and upper levels (as opposed to their lower levels in the preceding scenario), the distribution for QOC changes when the occupancy rate is varied (not shown in Table 5).
The analysis just presented demonstrates the complicated nature of nursing home care quality. Although it is evident that some factors have a greater influence on the overall QOC, the effect of other factors is not readily apparent. Knowledge of such interdependencies has important implications for QOC improvement. Nursing home administrators should recognize the relative significance of each factor influencing the overall quality of nursing home care. Accordingly, attention should not only be given to key factors but also to closely related factors in an effort to further improve the overall QOC. In other words, focusing on only one key factor while ignoring closely related factors may not result in the desired outcome.
Methodological Differences
We conclude this section by highlighting the differences between our method and the methods utilized in other studies. As we discussed in the Background section, many studies have been conducted that focus on specific factors influencing nursing home care quality. The methods employed in these studies include ordinary least-squares regression, two-stage and three-stage least-squares regression, Tobit multivariate techniques, logistic regression, and various experimental designs. Studies devoted to aggregating information to measure the overall QOC in nursing homes have used additional methods. These methods include conceptual frameworks, analysis of deficiencies, hierarchical decomposition, factor analysis, path analysis, cluster analysis, structural equation modeling, QI rank and classification techniques, and methods utilizing on-site surveys.
In contrast to methods used in previous studies, this work relies on BN methodology. This methodology is fundamentally different from the other approaches. We use BN methodology in this study for two purposes. First, it serves as a method to aggregate various indicators of quality to predict an overall measure of nursing home care quality. Second, it is used to gain qualitative and quantitative insight into nursing home care quality through an analysis of BN structure and parameters. Thus, this research makes contributions to both types of studies addressed in the Background section.
Conclusion
The analysis and results presented in this article provide researchers, administrators, and policy makers alike with valuable insight into the multidimensional nature of the quality of nursing home care. Consideration of the new perspective afforded by the BN approach, in combination with previous research in this area, will yet provide heretofore unattained insight and understanding. Toward this end, the salient points of this research include the following.
First, a more accurate representation of nursing home care quality is obtained through a mix of structural, process, and outcome measures of quality. Moreover, sole consideration of variables belonging to any of these three categories is not as powerful in describing nursing home care quality as is their collective influence.
Second, a BN framework for nursing home care quality provides both qualitative and quantitative explanations of the factors that influence the overall QOC in nursing homes. Consequently, previously unattained insight into the quality of nursing home care delivery can be afforded through a BN approach.
Third, nursing home administrators seeking to improve the overall QOC should first consider ensuring good staffing levels of certified nurse assistants, decreasing the prevalence of bedfast residents, and reducing the prevalence of daily physical restraints. In addition to improving quality, such efforts are likely to result in a decreased number of deficiencies.
Fourth, the factors affecting nursing home care quality collectively determine the overall quality. Hence, focusing on only key factors without addressing other related factors may not substantially improve the quality of nursing home care.
Future research with BNs may provide additional utility when they are used in conjunction with a larger data set. This would provide more data for structure and parameter learning, which may in turn yield a model that provides more in-depth insights into nursing home care quality. Furthermore, a larger data set would allow for more extensive testing of the networks.
In addition to augmenting the data set, future efforts may benefit from considering further measures for the overall QOC. In this research, we used the OIQ for this purpose. Although the OIQ has provided valuable insights into the QOC in nursing homes, other measures may offer further insight and may yield more accurate results. Despite the need for some improvements in future research, the positive implications of assessing the QOC in nursing homes through a Bayesian network—as illustrated in this analysis—outweigh any limitations or difficulties encountered in this research.
Acknowledgments
We acknowledge the contributions of other University of Missouri–Columbia Minimum Data Set and Nursing Home Quality Research Team members. Research activities were partially supported by the National Institute of Nursing Research under Grant 1R01NR/AG05287-01A2. The opinions expressed herein are those of the authors and do not represent the institute.
References
- Aaronson WE, Zinn JS, Rosko MD. Do for-profit and not-for-profit nursing facilities behave differently? The Gerontologist. 1994;34:775–786. doi: 10.1093/geront/34.6.775. [DOI] [PubMed] [Google Scholar]
- Atchley S. A time-ordered, systems approach to quality assurance in longterm care. Journal of Applied Gerontology. 1991;10:19–34. doi: 10.1177/073346489101000103. [DOI] [PubMed] [Google Scholar]
- Bøttcher SG. Artificial Intelligence and Statistics. San Francisco: Morgan Kaufmann; 2001. Learning Bayesian networks with mixed variables; pp. 149–156. [Google Scholar]
- Bøttcher SG, Dethlefsen C. R package version 1.2-22 (Version 1.2-22) [Computer software] 2004. Deal: Learning Bayesian networks with mixed variables. [Google Scholar]
- Castle NG. Deficiency citations for physical restraint use in nursing homes. Journal of Gerontology: Social Sciences. 2001;55B:S33–S40. doi: 10.1093/geronb/55.1.s33. [DOI] [PubMed] [Google Scholar]
- Chesteen S, Helgheim B, Randall T, Wardell D. Comparing quality of care in non-profit and for-profit nursing homes: A process perspective. Journal of Operations Management. 2005;23:229–242. [Google Scholar]
- Donabedian A. The quality of care: How can it be assessed? Journal of the American Medical Association. 1988;260:1743–1748. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
- Geiger D, Heckerman D. Learning Gaussian networks. Uncertainty in artificial intelligence: Proceedings of the Tenth Conference; San Mateo, CA: Kaufmann; 1994. [Google Scholar]
- General Accounting Office. Nursing homes: Federal efforts to monitor resident assessment data should complement state activities (Publication No. GAO-02-279) Washington, DC: Author; 2002. [Google Scholar]
- Grabowski DC. Does an increase in the Medicaid reimbursement rate improve nursing home quality? Journal of Gerontology: Social Sciences. 2001;56B:S84–S93. doi: 10.1093/geronb/56.2.s84. [DOI] [PubMed] [Google Scholar]
- Harrington C, O’Meara J, Kitchener M, Simon LP, Schnelle JF. Designing a report card for nursing facilities: What information is needed and why. The Gerontologist. 2003;43(Spec Issue II):47–57. doi: 10.1093/geront/43.suppl_2.47. [DOI] [PubMed] [Google Scholar]
- Harrington C, Swan JH. Nursing home staffing, turnover, and case mix. Medical Care Research and Review. 2003;60:366–392. doi: 10.1177/1077558703254692. [DOI] [PubMed] [Google Scholar]
- Harrington C, Woolhandler S, Mullan J, Carrillo MS, Himmelstein DU. Does investor ownership of nursing homes compromise the quality of care? American Journal of Public Health. 2001;91:1452–1455. doi: 10.2105/ajph.91.9.1452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harrington C, Zimmerman D, Karon SL, Robinson J, Beutel P. Nursing home staffing and its relationship to deficiencies. Journal of Gerontology: Social Sciences. 2000;55B:S278–S287. doi: 10.1093/geronb/55.5.s278. [DOI] [PubMed] [Google Scholar]
- Hogg R, Craig A, McKean J. Introduction to mathematical statistics. Englewood Cliffs, NJ: Prentice-Hall; 2004. [Google Scholar]
- Institute of Medicine. Improving the quality of care in nursing homes (No. Report IOM-85-10) Washington, DC: Author; 1986. [Google Scholar]
- Institute of Medicine. Nursing staff in hospitals and nursing homes. Washington, DC: Author; 1996. [Google Scholar]
- Institute of Medicine. Improving the quality of long-term care. Washington, DC: Author; 2001. [Google Scholar]
- Jensen FV. An introduction to Bayesian networks. London: UCL Press; 1996. [Google Scholar]
- Karon SL, Sainfort F, Zimmerman DR. Stability of nursing home quality indicators over time. Medical Care Research and Review. 1999;37:570–579. doi: 10.1097/00005650-199906000-00006. [DOI] [PubMed] [Google Scholar]
- Lee S, Abbott P. Bayesian networks for knowledge discovery in large datasets: Basics for nurse researchers. Journal of Biomedical Informatics. 2003;36:389–399. doi: 10.1016/j.jbi.2003.09.022. [DOI] [PubMed] [Google Scholar]
- Marshall A, McClean S, Shapcott C, Hastie I, Millard P. Developing a Bayesian belief network for the management of geriatric hospital care. Health Care Management Science. 2001;4:25–30. doi: 10.1023/a:1009645531478. [DOI] [PubMed] [Google Scholar]
- O’Neill C, Harrington C, Kitchener M, Saliba D. Quality of care in nursing homes—An analysis of relationships among profit, quality, and ownership. Medical Care. 2003;41:1318–1330. doi: 10.1097/01.MLR.0000100586.33970.58. [DOI] [PubMed] [Google Scholar]
- Office of the Inspector General. Nursing home resident assessment: Quality of care. Washington, DC: Department of Health and Human Services; 2001. [Google Scholar]
- Pearl J. Probabilistic reasoning in intelligent systems. San Mateo, CA: Kaufmann; 1988. [Google Scholar]
- Rantz MJ, Hicks L, Grando V, Petroski GF, Madsen RW, Mehr DR, et al. Nursing home quality, cost, staffing, and staff mix. The Gerontologist. 2004;44:24–38. doi: 10.1093/geront/44.1.24. [DOI] [PubMed] [Google Scholar]
- Rantz M, Mehr D, Petroski G, Madsen R, Popejoy L, Hicks L, et al. Initial field testing of an instrument to measure “Observable indicators of nursing home care quality. Journal of Nursing Care Quality. 2000;14:1–12. doi: 10.1097/00001786-200004000-00002. [DOI] [PubMed] [Google Scholar]
- Rantz M, Petroski GF, Madsen RW, Mehr DR, Popejoy L, Hicks L, et al. Setting thresholds for quality indicators derived from MDS data for nursing home quality improvement reports: An update. The Joint Commission Journal on Quality Improvement. 2000;26:101–110. doi: 10.1016/s1070-3241(00)26008-2. [DOI] [PubMed] [Google Scholar]
- Rantz M, Zwygart-Stauffacher M. A new reliable tool for nurse administrators, nursing staff, regulators, consumers, and researchers for measuring quality of care in nursing homes. Nursing Administration Quarterly. 2006;30:178–181. [Google Scholar]
- Rantz M, Zwygart-Stauffacher M, Mehr D, Petroski G, Owen S, Madsen R, et al. Field testing, refinement, and psychometric evaluation of a new measure of nursing home care quality. Journal of Nursing Measurement. 2006;14:129–148. doi: 10.1891/jnm-v14i2a005. [DOI] [PubMed] [Google Scholar]
- Rantz M, Zwygart-Stauffacher M, Popejoy L, Grando V, Mehr D, Hicks L, et al. Nursing home care quality: A multidimensional theoretical model integrating the views of consumers and providers. Journal of Nursing Care Quality. 1999;14:16–37. doi: 10.1097/00001786-199910000-00004. [DOI] [PubMed] [Google Scholar]
- Schnelle JF, Simmons SF, Harrington C, Cadogan M, Garcia E, Bates-Jensen BM. Relationship of nursing home staffing to quality of care. Quality. 2004;39:225–250. doi: 10.1111/j.1475-6773.2004.00225.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG. Bayesian analysis in expert systems. Statistical Science. 1993;8:219–283. [Google Scholar]
- Tan P, Steinbach N, Kumar V. Introduction to Data mining. Reading, MA: Addison-Wesley; 2005. [Google Scholar]
- Unruh L, Wan TTH. A systems framework for evaluating nursing care quality in nursing homes. Journal of Medical Systems. 2004;28:197–213. doi: 10.1023/b:joms.0000023302.80118.74. [DOI] [PubMed] [Google Scholar]
- Zinn JS, Aaronson WE, Rosko MD. Variations in the outcomes of care provided in Pennsylvania nursing homes: Facility and environmental correlates. Medical Care. 1993;31:475–487. doi: 10.1097/00005650-199306000-00001. [DOI] [PubMed] [Google Scholar]