Abstract
Background
Lack of randomization of nursing intervention in outcome effectiveness studies may lead to imbalanced covariates. Consequently, estimation of nursing intervention effect can be biased as in other observational studies. Propensity score analysis is an effective statistical method to reduce such bias and further derive causal effects in observational studies.
Objectives
To illustrate the use of propensity score analysis in quantitative nursing research through an example of pain management effect on length of hospital stay.
Methods
Propensity scores are generated through a regression model treating the nursing intervention as the dependent variable and all confounding covariates as predictor variables. Then propensity scores are used to adjust for this nonrandomized assignment of nursing intervention through three approaches: regression covariance adjustment, stratification, and matching in the predictive outcome model for nursing intervention.
Results
Propensity score analysis reduces the confounding covariates into a single variable of propensity score. After stratification and matching on propensity scores, observed covariates between nursing intervention groups are more balanced within each stratum or in the matched samples. The likelihood of receiving pain management is accounted for in the outcome model through the propensity scores. Both regression covariance adjustment and matching methods report a significant pain management effect on length of hospital stay in this example. The pain management effect can be regarded as causal when the strongly ignorable treatment assignment assumption holds.
Discussion
Propensity score analysis provides an alternative statistical approach to the classical multivariate regression, stratification and matching techniques for examining the effects of nursing intervention with a large number of confounding covariates in the background. It can be used to derive causal effects of nursing intervention in observational studies under certain circumstances.
Keywords: matching, nursing effectiveness research, nursing interventions, propensity score
Nursing care is a crucial element of an integrated health care system. Although adequate nursing care has been shown to affect patient outcomes positively, the majority of studies done to date have linked quality of patient care to nurse staffing and not to interventions provided by nurses (Aiken, Clarke, Sloane, Sochalski, & Silbert, 2002; Blegen, Goode, & Reed, 1998; Needleman, Buerhaus, Matke, Stewart, & Zelevinsky, 2002; Sochalski, 2001). Nursing care usually consists of a series of nursing interventions. A nursing intervention is defined as “any intervention, based upon clinical judgment and knowledge that a nurse performs to enhance patient/client outcomes” (Dochterman & Bulechek, 2004, p.3). The Nursing Interventions Classification (NIC) is a standardized language that provides labels, definitions, and activities for 514 nursing interventions (Dochterman & Bulechek, 2004). Some clinical settings have already incorporated the NIC into their electronic health records to document nursing care.
Although documenting nursing interventions in electronic health records hopefully will facilitate quantitative nursing effectiveness research, the estimation of nursing intervention effect on health outcomes can be biased due to the nature of observational studies. Unlike a randomized clinical trial (RCT), whether a patient receives a nursing intervention in an observational study (nursing outcomes effectiveness research) is based on the patient’s needs. The investigator does not assign patients randomly to receive the nursing intervention of interest. Lack of randomization may lead to imbalanced observed covariates across patient groups of nursing intervention. The heterogeneity of patients in terms of demographics, clinical conditions, and other treatments makes any direct comparison of health outcomes misleading in the outcomes effectiveness studies. On the other hand, a true RCT to examine the efficacy of a specific intervention is costly and results of such efficacy studies (RCT) may not result in similar outcomes when examined in outcomes effectiveness research in which patients receive multiple interventions in a real world setting (D’Agostino & D’Agostino, 2007).
Common strategies to adjust for heterogeneity include regression covariance adjustment, stratification, and cohort matching. Regression covariance adjustment is a traditional technique in statistical analysis to estimate treatment effects controlling for confounding covariates. However, this estimation still can be biased if covariates are imbalanced heavily or treatment effects vary across different levels of those covariates (D’Agostino & D’Agostino, 2007). Stratification and cohort matching are statistical approaches that can be applied early in the statistical design stage to alleviate patient heterogeneity. However, their applications in nursing outcomes effectiveness studies are limited by the small number of covariates that can be accommodated. Furthermore, these techniques are deemed to show only an association type of relationship between a nursing intervention and the outcome of interest in an observational nursing outcomes effectiveness study. A causal inference for the nursing intervention on the health outcome of interest, even though preferred, seems to be beyond reach.
Rosenbaum and Rubin (1983) first proposed the propensity score analysis to establish the framework of making a causal inference for an observational study. The propensity scores were used not only to reduce estimation bias of the treatment effect (e.g., a specific nursing intervention), but also derive a causal conclusion in observational studies when appropriate (Rosenbaum & Rubin, 1983). The propensity score method has become a powerful technique to reduce bias and study causal effects in observational studies (Rubin & Waterman, 2006). Nonetheless, there have been few applications of propensity score analysis in observational nursing studies. One nursing study used propensity score for matching treatment and control groups in studying the effect of organizational change on clinical outcomes (Aiken, Sochalski, & Lake, 1997). Another study applied propensity score method to examine effects of hospital-based skilled nursing facility closures on health care utilization, spending, and outcomes using a stratification approach (White & Seagrave, 2005). Neither organizational change nor hospital-based skilled nursing facility closure is in the domain of the nursing interventions.
The purpose of this manuscript is to provide a deeper understanding of the propensity score method and promote its application in nursing effectiveness research. First reviewed is the concept of a propensity score and its principles, and then described is how to carry out a propensity score analysis for an observational study using examples from a large nursing effectiveness study. Finally, some practical issues of propensity score analysis encountered in nursing outcome effectiveness research will be discussed.
Methods
In a RCT, patients are assigned to either a treatment or control group through a randomization mechanism, which theoretically guarantees there is no systematic difference between the groups. However, this is generally not true for an observational study, where inherent imbalance of the observed covariates introduces bias and hinders the exploration of causal effect due to treatment. In cases where the confounding covariates can be measured, they can be adjusted for and therefore correct for the imbalance between groups. A function of the observed covariates is called a balancing score if it can be used to correct such imbalance across groups. Conditional on a balancing score, the observed covariates should be independent of the assignment of treatment and control: whether a patient receives a treatment or not in an observational study.
Rosenbaum and Rubin (1983) showed that a propensity score was the coarsest balancing score that can be used and thus proposed the propensity score method to balance the inequality of the confounding covariates in observational studies. Causal effects can be derived further in a propensity score analysis with an assumption of strongly ignorable treatment assignment. Treatment assignment is strongly ignorable if it is independent of the outcome after controlling for the observed covariates (Rosenbaum & Rubin, 1983). Given a strongly ignorable treatment assignment, the difference in outcomes between patients who received treatment and those who did not is an unbiased estimate of treatment effect after controlling for the observed covariates through propensity scores. A propensity score is the likelihood that a patient received a treatment (e.g., nursing intervention) given all the observed covariates. It is a conditional probability of receiving treatment and thus always has a value between 0 and 1. The larger a propensity score, the more likely a patient was to receive the specified treatment.
The treatment variable of interest must be dichotomous in a propensity score analysis. One example of how to dichotomize the treatment variable is whether a patient received intervention (1) or not (0). Another example is whether or not the patient received the intervention at least once a day (1) or less than once a day (0). How to dichotomize the treatment variable is the first decision the researcher makes in a propensity score analysis (see the bolded box at the top of Figure 1) and should be guided by clinical and empirical knowledge. The necessary steps for a propensity score analysis are shown in Figure 1 (Shever et al., 2008); this was influenced by a conceptual model developed for nursing effectiveness research (Titler, Dochterman, & Reed, 2004).
Figure 1.
Model for Nursing Effectiveness Research Using Propensity Scores
Propensity score analysis usually starts with an assessment of the imbalance of baseline demographics and other covariates between treatment and control groups. This assessment can be done by significance tests (i.e., Student’s t-test, Wilcoxon rank-sum test for continuous covariates, Chi-square test for categorical covariates). Some recent research recommends that standardized difference be used because they are a better diagnostic tool than significance tests to assess imbalance (Austin, 2007). A standardized difference is defined as the ratio of absolute mean difference and square root of average variance for both continuous and categorical covariates. A large standardized difference of greater than 10%, not necessarily reaching significance, usually exhibits enough imbalance to be adjusted for by a propensity score analysis (Austin, Grootendorst, & Anderson, 2007).
In the propensity score model, the dichotomous treatment is treated as a dependent variable, where the observed covariates are considered to be predictors. Based on the relationships with treatment and outcome, observed covariates can be categorized into three groups: covariates only related to treatment assignment; covariates related to both treatment assignment and outcome (i.e., confounders); and covariates only related to outcome. There are still debates on the variable selection in a propensity score model (Brookhart et al., 2006). Simulation studies seemed to suggest that a propensity score model with both confounders and covariates only related to the outcome resulted in smaller variance in the estimation of treatment effect, better stratifications, and more matchings (Austin et al., 2007; Brookhart et al., 2006; Rubin & Thomas, 1996). However, it is recommended that only confounders be included in the propensity score model (Perkins, Tu, Underhill, Zhou, & Murray, 2000) and to leave the covariates related only to outcome to the next step of the outcome model.
Some potential confounding covariates along the left-hand side are shown in Figure 1. These potential confounders may be patient characteristics (e.g., gender, age, socioeconomic status), clinical conditions (e.g., severity of illness, primary medical diagnosis, comorbid conditions), context of care variables (e.g., nurse-to-patient ratios, skill mix, number of units resided on during hospitalization), medical treatments (e.g., surgical versus medical, number, specific types), pharmaceutical treatments (e.g., number; specific medications like diltiazem, potassium, morphine), and nursing interventions (e.g., number; specific interventions like fluid management, surveillance, bathing). The question in the triangle in Figure 1 is to encourage the researcher to select confounding covariates based on their relationships with both the treatment and outcome variables.
The likelihood of the subject receiving treatment based on the selected confounders is reduced to a propensity score for each patient. This propensity score is generated for each subject (follow the yes arrow off the triangle in Figure 1) from the selected confounders. Since the treatment variable of interest is dichotomous, the common methods adopted to produce propensity scores are either logistic regression or discriminant analysis. The confounders used to calculate the propensity scores are not prohibited from entering the subsequent outcome model. However, caution is advised for potential multicollinearity between confounders and the generated propensity score in the regression covariance approach.
For this manuscript, also considered is a case where those covariates unrelated to the assignment of treatment but potentially predictive of the outcome are incorporated into the outcome analysis. In Figure 1, the potential relationship between possible covariates and the outcome variable is captured in the box below the triangle for the researcher to consider whether or not the variable is related to the outcome. If the answer is no, the variable should not enter the last step of the analysis. If the answer is yes, the variable enters the final regression step along with the treatment variable (Figure 1).
D’Agostino (1998) outlined three ways of making use of propensity scores in observational studies with the aim to reduce bias and make causal inference: regression covariance adjustment, stratification, and matching, which are chosen by the researcher, as indicated by the dotted arrows coming down from the propensity score box in Figure 1. A common way to use the propensity scores is in a regression model for the outcome. Regression covariance adjustment is essentially a multivariate regression analysis with the dichotomous treatment variable of interest (e.g., nursing intervention), the propensity score, and all other covariates (Figure 1). In this method, the propensity score is treated as a continuous instrumental variable controlling for all of the confounders that were used to generate the propensity score. Regression covariance adjustment is the only method where the propensity score actually enters into the model as a predictor variable. Having propensity scores in the model does not replace the treatment variable, but it does account for the fact that patients did not receive treatment randomly. The significance of propensity score in the outcome model suggests the importance of applying propensity score analysis.
In the stratification approach, the propensity scores are used to stratify patients, usually into five strata with quintiles as the cutting points, but this could vary depending on sample size. Within each stratum, the propensity scores of patients that received the treatment and those who did not are all in the same range. Separate multivariate regression analyses can be done to determine the treatment effect for each stratum or an overall treatment effect can be analyzed by treating strata as another factor in the outcome model. When separate analyses are run on each of the stratum, the results must be interpreted with respect to the patient stratum--from the group with lowest likelihood of receiving treatment (lowest propensity scores) to the group with the higher likelihood (higher propensity scores).
Another method to use propensity scores is matching patients who received treatment and those who did not. Compared with multivariate covariate matching, matching on a scalar of propensity score is much easier. This means that patients who received treatment are matched on their propensity scores to patients who did not receive treatment. The logit function of propensity score is often recommended for matching in practice. In essence, a large number of confounders have been controlled for by matching patients on the propensity scores. It would not be feasible to perform cohort matching on such a large number of confounders. Multivariate regression analysis can be applied to the matched sample along with other covariates believed to influence the outcome variable. Due to the matched nature of the samples, appropriate statistical methods must be selected to account for the matching structure (Austin, 2007). A causal inference for the treatment effect may be achievable because patients in the matched sample have similar likelihoods of receiving treatment (D'Agostino, 1998).
Example
In a large observational study of nursing interventions and outcomes among three elder acute care populations (NINR R01 NR05331, PI: Titler), one of the research aims was to determine the distinct contribution of selected nursing interventions on healthcare outcomes. Data for hip procedure patients, fall prevention patients, and patients with congestive heart failure at one large academic medical center over a period of 4 years from July 1998 to June 2002 were extracted from nine clinical and administrative data repositories. The core set of variables pulled out from different electronic repositories were patient characteristics, clinical conditions, context of care variables, medical treatments, pharmaceutical treatments, and nursing interventions. Previous studies have shown certain nursing interventions were associated highly with the occurrences of outcomes such as discharge disposition (Titler et al., 2006), and cost (Titler et al., 2005; Titler et al., 2007; Titler et al., 2008).
One nursing intervention, pain management, and an associated patient outcome, length of stay (LOS), from the hip procedure patient group were chosen to illustrate the complete process of propensity score analysis. Pain management is defined as “alleviation of pain or a reduction in pain to a level of comfort that is acceptable to the patient” in the NIC (Dochterman & Bulechek, 2004, p. 529). There were 523 hip procedure patients (41 patients have 2 hospitalizations and 2 patients have 3 hospitalizations) resulting in a total of 568 hospitalizations during the study period. Pain management was a common nursing intervention delivered to patients hospitalized with a hip procedure. Among these 568 hospitalizations, 214 received pain management at least once during their hospitalizations. Propensity score analysis was used to answer the question: What was the distinct contribution of pain management upon LOS in older, hospitalized, hip procedure patients?
Pain management was a dichotomous treatment variable that indicated whether this nursing intervention occurred during a hospitalization. The pain management also could have been dichotomized on its use rate; that is, high use vs. low use instead of ever use vs. never use. Thus, the research question becomes: How does the high use of pain management affect LOS? This is particularly useful when a nursing intervention is used at various dosages through the sample (Reed et al., 2007).
The covariates between hospitalizations that received pain management at least once and those which never received pain management were compared in Table 1 using standardized difference and significance test. Quite a few covariates were considered to be imbalanced according to the rule of >10% standardized difference. As a matter of fact, standardized difference disclosed more covariates to be unbalanced due to less stringent criteria. Therefore, propensity score analysis was recommended in this example for studying the effect of pain management on the outcome of LOS.
Table 1.
Comparison of Observed Covariates for Hospitalizations Received Pain Management vs. Those Did Not
| Pain Management | Standardized Difference in % | Significance Test | ||||
|---|---|---|---|---|---|---|
| No (n = 354) | Yes (n = 214) | |||||
| Mean | SD | Mean | SD | |||
| Patient Characteristics | ||||||
| Gender | 0.373 | 0.484 | 0.388 | 0.488 | 3.086 | .722 |
| Marital status | 1.720 | 0.893 | 1.935 | 0.932 | 23.556 | .025 |
| Site admitted from | 2.712 | 0.687 | 2.678 | 0.708 | 4.874 | .395 |
| Admission age | 74.034 | 8.000 | 75.327 | 9.031 | 15.156 | .096 |
| Race | 1.068 | 0.252 | 1.094 | 0.292 | 9.533 | .268 |
| Religion | 2.048 | 0.786 | 2.061 | 0.764 | 1.677 | .868 |
| Clinical Conditions | ||||||
| Nontraumatic joint disorders | 0.454 | 0.499 | 0.351 | 0.478 | 21.080 | .015 |
| Fractures | 0.294 | 0.456 | 0.383 | 0.487 | 18.866 | .028 |
| Complications | 0.189 | 0.392 | 0.173 | 0.379 | 4.150 | .625 |
| Other | 0.062 | 0.242 | 0.094 | 0.292 | 11.933 | .167 |
| Medical Treatments | ||||||
| Arthroplasty | 0.751 | 0.433 | 0.724 | 0.448 | 6.129 | .475 |
| Blood transfusion | 0.740 | 0.439 | 0.696 | 0.461 | 9.775 | .258 |
| Other therapeutic procedures | 0.441 | 0.497 | 0.692 | 0.463 | 52.259 | <.001 |
| Pharmacy Treatments | ||||||
| Morphine | 0.904 | 0.295 | 0.930 | 0.256 | 9.414 | .286 |
| Fentanyl | 0.887 | 0.317 | 0.869 | 0.338 | 5.493 | .526 |
| Acetaminophen with codeine | 0.692 | 0.462 | 0.622 | 0.486 | 14.763 | .084 |
| Meperidine/Demerol | 0.178 | 0.383 | 0.187 | 0.391 | 2.325 | .789 |
| Hydromorphone/Dilaudid | 0.161 | 0.368 | 0.145 | 0.353 | 4.437 | .606 |
| Other opiates | 0.071 | 0.257 | 0.084 | 0.278 | 4.856 | .556 |
| Opiate agonists | 0.011 | 0.106 | 0.028 | 0.166 | 12.207 | .142 |
| Local anesthetics | 0.003 | 0.053 | 0.014 | 0.118 | 12.026 | .122 |
| Antipruritics | 0.006 | 0.075 | 0.033 | 0.178 | 19.768 | .012 |
| Nursing Treatments | ||||||
| Analgesic administration | 1.757 | 0.983 | 1.687 | 1.070 | 6.813 | .119 |
| Postoperative care | 1.777 | 1.009 | 1.762 | 0.971 | 1.515 | .796 |
| Sleep enhancement | 1.045 | 1.126 | 1.056 | 1.213 | 0.940 | .052 |
| Exercise therapy | 0.715 | 1.094 | 0.612 | 0.990 | 9.873 | .053 |
| Wound care | 0.520 | 0.985 | 0.290 | 0.732 | 26.505 | .022 |
| Traction or Immobilization care | 0.263 | 0.719 | 0.290 | 0.793 | 3.567 | .597 |
| Positioning | 0.294 | 0.755 | 0.523 | 0.992 | 25.978 | .022 |
| Vital signs monitoring | 0.331 | 0.797 | 0.234 | 0.726 | 12.724 | .045 |
| Learning readiness enhancement | 0.201 | 0.658 | 0.336 | 0.810 | 18.295 | .078 |
| Surgical preparation | 0.226 | 0.687 | 0.238 | 0.695 | 1.737 | .684 |
| Cardiac care | 0.093 | 0.426 | 0.262 | 0.761 | 27.405 | .007 |
| Venous access device maintenance | 0.122 | 0.510 | 0.150 | 0.578 | 5.137 | .438 |
| Exercise promotion | 0.186 | 6.620 | 0.154 | 0.597 | 0.681 | .382 |
| Dying care | 0.099 | 0.469 | 0.140 | 0.547 | 8.047 | .261 |
| Restraint | 0.085 | 0.444 | 0.136 | 0.527 | 10.466 | .166 |
| Chest physiotherapy | 0.028 | 0.166 | 0.033 | 0.178 | 2.905 | .762 |
| Splinting | 0.031 | 0.174 | 0.019 | 0.136 | 7.684 | .373 |
| Anxiety reduction | 0.006 | 0.075 | 0.056 | 0.231 | 29.115 | <.001 |
| Propensity score | 0.279 | 0.198 | 0.539 | 0.227 | 122.069 | <.001 |
Notes. Independent sample t-test for continuous and Chi-square test for categorical
covariates.
Knowledge derived from clinical expertise and empirical evidence (Herr, Bjoro, Steffensmeier, & Rakel, 2006) was used to determine confounders to be included in the propensity score model for pain management. The researchers selected core confounding variables consisting of patient characteristics, clinical conditions, context of care, medical treatments, pharmaceutical treatments, and other nursing interventions. One problem encountered by the research team was the inclusion of too many confounders in the propensity score model given the sample size of 568 hospitalizations. The number of confounders had to be reduced after a first attempt failed due to a convergence problem in running generalized estimating equations (GEE) analysis. The convergence problem usually is caused by too many parameters specified in the model. The researchers chose to eliminate any confounders that were not used by at least 2% of the sample. The final list of confounders were displayed in Table 1; a survey of their relationships with treatment and outcome through Spearman’s correlation revealed that nine of the confounders were related strongly to treatment and outcome, 13 confounders were related weakly to treatment but related strongly to outcome, and 5 confounders were related strongly to treatment but related weakly to outcome.
The propensity score analysis consisted of both propensity score model and outcome model in sequence. Propensity scores were generated for each patient from the propensity score model. In order to adjust for the correlation between hospitalizations from the same patient, the GEE instead of logistic regression was used for the propensity score model. This GEE approach was done by PROC GENMOD in SAS version 9.1. Then the propensity scores were utilized in each of the three approaches outlined in the methods section. So the outcome model for LOS included pain management and those covariates only related to LOS in a regression setting. These covariates were controlled for in the outcome model so that the treatment effect of pain management could be estimated as accurately as possible. In the regression covariance adjustment method, the propensity score was a continuous covariate in the model, while in the latter two approaches, the propensity scores were used for stratification and matching only. A SAS macro gmatch was applied to implement greedy matching on the logit of propensity scores of hospitalizations with and without Pain Management (Bergstralh & Kosanke, 2003), with calipers of width 0.2 standard deviation of the logit of the propensity scores. Due to the matched-sample structure, the GEE model that treats each pair as a block was used to estimating the pain management effect.
Examples of the results of propensity score analyses are shown in Tables 2, 3, and 4. The treatment effects of pain management are shown at the top of each table, with not receiving pain management during hospitalization as the reference category. The rest of the covariates in the tables were those considered to be related to LOS but not related to pain management assignment.
Table 2.
Example of Results from Regression Covariance Adjustment Analysis (n = 568 hospitalizations)
| Parameter estimate | p-value | |
|---|---|---|
| Propensity score | −1.395 | .152 |
| Pain Management | 1.140 | .002 |
| Clinical conditions | ||
| Severity of illness | .596 | |
| Context of Care | ||
| Number of units resided on | .004 | |
| CGPR RN dip proportion | .004 | |
| RN skill mix | .516 | |
| Average CGPR_RN | .004 | |
| Percent of time in ICU | .031 | |
| Medical Treatment | ||
| Number of procedures | .001 | |
| Pharmacy Treatment | ||
| Number of unique medications | <.001 | |
| Nursing Treatments | ||
| Number of unique nursing interventions | .227 | |
Note: Specific nursing interventions are omitted but full-length table can be seen on the journal Web site at http://www.nursing-research-editor.com.
CGPR = Caregiver Patient ratio
Table 3.
Example of Results from Stratification Analysis (n = 568 hospitalizations)
| Stratum 1 (n=113 ) |
Stratum 2 (n=114) |
Stratum 3 (n=114) |
Stratum 4 (n=114) |
Stratum 5 (n=113) |
||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Estimate | p-value | Estimate | p-value | Estimate | p-value | Estimate | p-value | Estimate | p-value | |
| Pain Management | 4.600 | .001 | −.523 | .264 | −0.137 | .624 | 0.487 | .284 | 2.040 | .031 |
| Clinical conditions | ||||||||||
| Severity of illness | .100 | .002 | .662 | .019 | .010 | |||||
| Context of Care | ||||||||||
| Number of units resided on | .050 | .051 | .001 | .280 | .400 | |||||
| CGPR RN dip proportion | .001 | .008 | <.001 | .017 | .312 | |||||
| RN skill mix | .001 | .740 | .960 | .006 | .166 | |||||
| Average CGPR_RN | .192 | .458 | .188 | .688 | .014 | |||||
| Percent of time in ICU | .320 | <.001 | .002 | .153 | .002 | |||||
| Medical Treatment | ||||||||||
| Number of procedures | .324 | .066 | .162 | .236 | .264 | |||||
| Pharmacy Treatment | ||||||||||
| Number of unique medications | .227 | .646 | .002 | <.001 | .218 | |||||
| Nursing Treatments | ||||||||||
| Number of unique nursing interventions | .005 | .025 | .903 | .438 | .282 | |||||
Note: Specific nursing interventions are omitted but full-length table can be seen on the journal Web site at http://www.nursing-research-editor.com.
Table 4.
Example of Results from Matching Analysis (n=308 hospitalizations)
| Parameter estimate | p-value | |
|---|---|---|
| Pain Management | 0.947 | .006 |
| Clinical conditions | ||
| Severity of illness | .946 | |
| Context of Care | ||
| Number of units resided on | .001 | |
| CGPR RN dip proportion | .243 | |
| RN skill mix | .022 | |
| Average CGPR_RN | .280 | |
| Percent of time in ICU | .513 | |
| Medical Treatment | ||
| Number of procedures | .007 | |
| Pharmacy Treatment | ||
| Number of unique medications | <.001 | |
| Nursing Treatments | ||
| Number of unique nursing | .768 | |
| interventions |
Note: Specific nursing interventions are omitted but full-length table can be seen on the journal Web site at http://www.nursing-research-editor.com.
The results of the covariance adjusted propensity score analysis are shown in Table 2. As stated earlier, this method is the only one of the three that uses the propensity score as a predictor in the regression. The propensity score was not significant in the outcome model which suggested that the likelihood of receiving pain management was not associated with LOS. However, pain management was associated significantly (p = .002) with LOS; a positive parameter estimate indicated that hospitalizations that received pain management at least once had longer LOS than hospitalizations that did not receive pain management.
The results from the analysis that used propensity scores for stratification are shown in Table 3. The propensity scores were used to divide the hospitalizations into five groups. Stratum one represents the hospitalizations with the lowest propensity scores, whereas Stratum five contains those hospitalizations with the highest propensity scores. The results of the regression analysis for each stratum show that pain management was significant in the lowest (p =.001) and highest (p =.031) strata but not the three strata with moderate propensity scores.
The results in Table 4 are those associated with the matching analysis. The propensity scores were used to match hospitalizations for this analysis. As indicated in the title of the table, only 308 hospitalizations were used in this analysis due to the greedy matching with calipers of width 0.2 standard deviation of the logit of the propensity scores.
The outcome models associated with both the regression covariance and matching propensity score approaches indicated that receiving pain management during hospitalization increased LOS after controlling for the covariates only related to outcome. The stratification approach only reported some significant effects related to pain management in the lowest and highest strata. The balances of the confounders across pain management groups were reassessed critically in each stratum and the matched samples. As expected, stratification and matching on the propensity score effectively balanced the confounders, with most of them with a new standardized difference within 10% (not shown).
Discussion
Randomized clinical trials of nursing interventions are not always feasible due to the complexity of patient's disease and ethical considerations. Observational studies are used more practically in nursing outcomes-effective research to study the effect of nursing intervention on patient outcomes. In the example above, it was demonstrated that propensity score analysis can be very useful for studying nursing intervention in an observational study. A similar conclusion was arrived at in the propensity score analysis with three approaches regarding the pain management effect on LOS except for the three strata from stratification approach. Even though matching was recommended for propensity score analysis in recent research (Austin et al., 2007), it cannot be verified from an empirical example. Unlike simulation studies, even the degree of bias reduction cannot be shown by propensity score analysis of an empirical example study.
Theoretically, propensity score analysis can be used to study causal effects of nursing interventions in observational studies. Arriving at a stronger causal relationship rather than an association of the nursing intervention on the outcome is definitely preferred but not without a price. The assumption of strongly ignorable treatment assignment has to be verified before any causal inference can be made through a propensity score analysis. No previous publication was found on how to test this assumption in a propensity score analysis. Checking residuals from the propensity model against residuals from a new outcome regression model with the same set of confounders is recommended. A scatter plot showing no particular pattern or an insignificant test of statistical independence may provide some insights on this assumption.
Practical problems emerge when the researchers had to determine which observed covariates to be considered in the propensity score model. In addition to true confounders, some researchers recommended that all covariates related to the outcome be included. It is unclear whether this would become a definite rule or if a sensitivity analysis should always be conducted. Depending on the sample size, there are also limitations on the number of observed covariates that can be included in either propensity score model or the outcome model. In particular, the stratification method is most affected in this aspect due to the fact that analysis is conducted on a fraction (e.g., one-fifth) of the sample.
Conclusion
Use of propensity scores shows promise in outcomes effectiveness research in examining distinct contributions of specific nursing treatments (e.g., pain management) on patient outcomes (e.g., LOS). RCTs of nursing intervention are often not economically or practically feasible but propensity score analysis provides a statistical approach in examining nursing intervention effects in observational studies. Further application of propensity score analysis in nursing outcomes effectiveness studies is warranted to continue to build understanding of how care provided by nurses effects patient outcomes.
Acknowledgments
This research was supported by a grant from NIH (PI: Titler; NINR R01 NR05331).
Contributor Information
Rui Qin, Division of Biostatistics Mayo Clinic Rochester, Minnesota.
Marita G. Titler, The University of Iowa Hospitals and Clinics and Director of Nursing Research, Quality and Outcomes Management, Department of Nursing Services and Patient Care The University of Iowa Hospitals and Clinics, Iowa City, Iowa.
Leah L. Shever, Nursing Interventions and Outcomes Effectiveness in 3 Older Populations The University of Iowa College of Nursing Iowa City, Iowa.
Taikyoung Kim, Nursing Interventions and Outcomes Effectiveness in 3 Older Populations The University of Iowa College of Nursing Iowa City, Iowa.
References
- Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silbert JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987–1993. doi: 10.1001/jama.288.16.1987. [DOI] [PubMed] [Google Scholar]
- Aiken LH, Sochalski J, Lake ET. Studying outcomes of organizational change in health services. Medical Care. 1997;35(11 Suppl):NS6–NS18. doi: 10.1097/00005650-199711001-00002. [DOI] [PubMed] [Google Scholar]
- Austin PC. Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: A systematic review and suggestions for improvement. The Journal of Thoracic and Cardiovascular Surgery. 2007;134(5):1128–1135. doi: 10.1016/j.jtcvs.2007.07.021. [DOI] [PubMed] [Google Scholar]
- Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine. 2007;26(4):734–753. doi: 10.1002/sim.2580. [DOI] [PubMed] [Google Scholar]
- Bergstralh E, Kosanke J. Computerized matching of cases to controls using the greedy matching algorithm with a fixed number of controls per case (gmatch) [SAS macro] 2003 Available from the Mayo Clinic Division of Biostatistics: http://mayoresearch.mayo.edu/mayo/research/biostat/sasmacros.cfm.
- Blegen MA, Goode CJ, Reed L. Nurse staffing and patient outcomes. Nursing Research. 1998;47(1):43–50. doi: 10.1097/00006199-199801000-00008. [DOI] [PubMed] [Google Scholar]
- Brookhart MA, Schneeweiss S, Rothman KJ, Glynn RJ, Avorn J, Sturmer T. Variable selection for propensity score models. American Journal of Epidemiology. 2006;163(12):1149–1156. doi: 10.1093/aje/kwj149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- D'Agostino RB., Jr Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine. 1998;17(19):2265–2281. doi: 10.1002/(sici)1097-0258(19981015)17:19<2265::aid-sim918>3.0.co;2-b. [DOI] [PubMed] [Google Scholar]
- D'Agostino RB, Jr, D'Agostino RB., Sr Estimating treatment effects using observational data. JAMA. 2007;297(3):314–316. doi: 10.1001/jama.297.3.314. [DOI] [PubMed] [Google Scholar]
- Dochterman JM, Bulechek GM. Nursing interventions classification. 4. St. Louis, MO: Mosby; 2004. [Google Scholar]
- Herr K, Bjoro K, Steffensmeier J, Rakel B. Evidence-based guideline: Acute pain management in older adults. Iowa City, IA: Research Translation and Dissemination Core, Gerontological Nursing Interventions Research Center, University of Iowa College of Nursing; 2006. [Google Scholar]
- Needleman J, Buerhaus P, Matke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. The New England Journal of Medicine. 2002;346(22):1715–1722. doi: 10.1056/NEJMsa012247. [DOI] [PubMed] [Google Scholar]
- Perkins SM, Tu W, Underhill MG, Zhou XH, Murray MD. The use of propensity scores in pharmacoepidemiologic research. Pharmacoepidemiology and Drug Safety. 2000;9(2):93–101. doi: 10.1002/(SICI)1099-1557(200003/04)9:2<93::AID-PDS474>3.0.CO;2-I. [DOI] [PubMed] [Google Scholar]
- Reed D, Titler MG, Dochterman JM, Shever LL, Kanak M, Picone DM. Measuring the dose of nursing intervention. International Journal of Nursing Terminologies and Classifications. 2007;18(4):121–130. doi: 10.1111/j.1744-618X.2007.00067.x. [DOI] [PubMed] [Google Scholar]
- Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. [Google Scholar]
- Rubin DB, Thomas N. Matching using estimated propensity scores: Relating theory to practice. Biometrics. 1996;52(1):249–264. [PubMed] [Google Scholar]
- Rubin DB, Waterman RP. Estimating the causal effects of marketing interventions using propensity score methodology. Statistical Science. 2006;21(2):206–222. [Google Scholar]
- Shever LL, Titler MG, Kerr P, Qin R, Kim T, Picone DM. The effect of high surveillance use on hospital cost. Journal of Nursing Scholarship. 2008;40(2):161–169. doi: 10.1111/j.1547-5069.2008.00221.x. [DOI] [PubMed] [Google Scholar]
- Sochalski J. Quality of care, nurse staffing, and patient outcomes. Policy, Politics & Nursing Practice. 2001;2(1):9–18. [Google Scholar]
- Titler MG, Dochterman J, Reed D. Guideline for conducting effectiveness research in nursing and other health care services. Iowa City, IA: The University of Iowa College of Nursing, Center for Nursing Classification & Clinical Effectiveness; 2004. [Google Scholar]
- Titler M, Dochterman J, Picone DM, Everett LQ, Xie XJ, Kanak M, et al. Cost of hospital care for elderly at risk of falling. Nursing Economic$ 2005;23(6):290–306. [PubMed] [Google Scholar]
- Titler M, Dochterman J, Xie XJ, Kanak M, Fei Q, Picone DM, et al. Nursing interventions and other factors associated with disc harge disposition in older patients after hip fractures. Nursing Research. 2006;55(4):231–242. doi: 10.1097/00006199-200607000-00003. [DOI] [PubMed] [Google Scholar]
- Titler MG, Dochterman J, Kim T, Kanak M, Shever L, Picone DM, et al. Cost of care for seniors hospitalized for hip fractures and related procedures. Nursing Outlook. 2007;55(1):5–14. doi: 10.1016/j.outlook.2006.06.006. [DOI] [PubMed] [Google Scholar]
- Titler MG, Jensen GA, Dochterman JM, Xie XJ, Kanak M, Reed D, et al. Cost of hospital care for older adults with heart failure: Medical, pharmaceutical, and nursing costs. Health Services Research. 2008;43(2):635–655. doi: 10.1111/j.1475-6773.2007.00789.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- White C, Seagrave S. What happens when hospital-based skilled nursing facilities close? A propensity score analysis. Health Services Research. 2005;40(6 Pt 1):1883–1897. doi: 10.1111/j.1475-6773.2005.00434.x. [DOI] [PMC free article] [PubMed] [Google Scholar]